By default compression doesn't work in CloudFront for a website backed by an Amaxon S3 bucket.
The first step is pretty obvious - switch on compression in CloudFront:
To get to this setting open you distribution, go to the Behaviors tab and edit your behavior(s). Scroll down to the bottom and toggle Compress Objects Automatically to On. Save and drum your fingers while the distribution updates.
The less obvious piece is that CloudFront will only compress files between 1,000 and 10,000,000 bytes (as of writing this post) and it detects the filesize from the Content-Length header. What the documentation doesn't mention is that S3 does not send the Content-Length header by default and so no compression is applied.
Go to S3 and open the properties for your bucket (not for individual files). Expand Permissions and then click Edit CORS Configuration. You need to add Content-Length as an allowed header like this: