Improving the accuracy of the new Catfood Earth clouds layer
Last weekend I rushed out a new version of Catfood Earth because the clouds layer stopped working. I'd been using xplanet clouds which published a free 2048x1024 image infrequently and for some reason the site has vanished (probably because it was based on data from the Dundee Satellite Receiving Station which has apparently closed). The University of Wisconsin-Madison Space Science and Engineering Center makes global cloud data available for free for non-commercial use so I plugged this in to get Catfood Earth up and running.
Clouds started working again last Saturday but they were slightly out of position. With a pixel on a HD screen being up to 13 miles this isn't ideal. I hope nobody is using Catfood Earth for weather forecasting or anything too serious but this bothered me enough that I had to fix it. From today the clouds layer is as close to perfect as I can make it.
Mostly for my own sanity in case I ever need to fix this again here's the process.
I use the globalir product because it covers both day and night. This is available in a number of resolutions and you just need to download a set of tiles and stitch them together - I'm using 4096x4096 as the source for the new layer. Catfood Earth uses equirectangular images which have a 2:1 aspect ratio so to start with I resized this less at the equator and more at the poles and got an image that was almost right.
The source image is Spherical (or Web) Mercator which is useful for Google Maps but not right for Catfood Earth. I found a Stack Exchange post which pointed me in the right direction - use GDAL (Geospatial Data Abstraction Library) to transform the Mercator image to equirectangular by tagging the corners and then warping it. This got me closer but it still wasn't right. More googling led me to a post by Robert Simmon with a gdalwarp tip to use -te and -te_srs to force the output to match the dimensions of the input.
The last problem is that the Mercator image doesn't reach all the way to the poles. In this case there is no alternative but to invent some data and so I flip the top and bottom 50 pixels to cover the gap. You can see this if you look carefully but as this region is rendered over ice in Catfood Earth it won't make a difference in practice (and the old image used this trick as well for a greater proportion of the clouds).
One final visual tweak is to drop the lowest intensity pixels which end up blurring out the background in Catfood Earth. You can adjust the transparency and color used to render the layer to suit your tastes (the default is 50% transparency and a gray color for the clouds which I think strikes a good balance).
I also increased the frequency of updates so a new image is available every hour. I said last week that an advantage of this new system was that I wouldn't need to push a new version of Catfood Earth to tweak the cloud logic. That's partly true - you'll get the new improved image with the current version, but I stepped down the update frequency to once a day when I started using xplanet clouds. There will be another update soon to the Windows and Android versions to switch to hourly updates.
Updated 2022-11-05 12:28:
A couple of updates here. The first is that RealEarth started watermarking images over a certain resolution and/or usage volume. You now need an API key and potentially to pay based on usage. More details here. I managed to stay in the free tier by asking nicely and scaling back to HD. The clouds download is still 4k but this is upscaled from a HD version.
Grabbing the latest image (for globalir) worked until sometime in October and then the tiles stopped lining up correctly. I'm not sure what changed, I guess that 'latest' used to mean the latest composite and now means the latest data for any given tile regardless of it is the same time as the others. I just fixed this to use the latest complete image (this API provides available dates and times, and then you need to use the date and time in the globalir call to get a working composite image).
Updated 2023-09-03 01:03:
There is now a video made from the last 48 hours of cloud images here, updated hourly. The video skips the step of dropping low intensity pixels so you get 100% of the cloud cover.
Related Posts
- Catfood Earth
- Animation of a year of Global Cloud Cover
- Global Cloud Cover 48 Hour Video Updated Hourly
You Might Also Like
Catfood Earth 3.45
Catfood Earth 3.45 is now available to download. Catfood Earth for Android 1.60 is available on Google Play and will update automatically if you have it installed.
I only just released 3.44 with some timezone updates but in the past week the location I had been using for global cloud cover abruptly shut down. If you like up to date clouds you'll want to install the new versions as soon as possible. With this update I'm building a cloud image every three hours and serving through this blog (and thankfully CloudFlare) so any further changes should not require a code release.
Related Posts
You Might Also Like
Humpback Whales in Monterey Bay
Photos of Humpback Whales in Monterey Bay taken from a whale watching boat trip.
Related Posts
- Humpback Whale at Fort Funston
- Humpback Whale Feasting off Rockaway Beach in Pacifica
- Humpback Whales in Golden Gate
You Might Also Like
- Summer Solstice 2013
- Enable GZIP compression for Amazon S3 hosted website in CloudFront
- 2021 California Gubernatorial Recall
Game of Death
Conway's Game of Life, for 1,830 generations, starting from a random pattern. Instead of showing the live cells this animation focuses on death - each dead cell gets a little bit greener with each generation. You can just about make out a few static patterns in the darkness and the lines cruising through are left behind by gliders. Mostly though you're watching the horrible loss of life caused by cellular social isolation.
Related Posts
- Life, Non-locality and the Simulation Hypothesis
- Draw the rest of the Hummingbird
- Coronavirus Visualization Update
You Might Also Like
ITHCWY Newsletter for March 2019
Catfood Earth 3.44 is available to download. This version updates the timezone database to 2018i, moves to a new source for timezone mapping an fixes a bug in the volcanoes layer.
Golden Gate Bridge timelapse - shot from the Marin headlands, the Golden Gate Overlook and near Fort Point.
Colorado looks set to join the National Popular Vote Interstate Compact. What you can do to help.
Could we solve the youth turnout problem with age weighted voting?
Previously:
- 2018: Facebook shouldn't own your social graph
- 2017: Book reviews for February 2017
- 2016: Chromecast won't connect to wifi - finally found the fix
Related Posts
You Might Also Like
Book reviews for March 2019
Killing Commendatore by Haruki Murakami
5/5
Related Posts
You Might Also Like
Age and Life Expectancy Weighted Voting
Youth turnout for elections is famously dismal. In 2016 less than half of 18-29 year-olds voted, compared to over two thirds as you get to 45 and older (US Census). The impact is an incentive to cater to the old - trying to make America great again (like you remember from when you were young) vs doing something about climate change or house prices.
One fix is compulsory voting, like in Australia. I'm not sure I want to force people without an opinion to vote though.
What if we just weighted votes by the total size of the demographic group?
I took the demographic breakdown of 2016 voters from the US Census Bureau and multiplied these by the age breakdown from CNN exit polls. This gave Clinton a lead of just under a million votes - somewhat lower than the actual result. This is likely a polling error in the exit poll, but it's a reasonable baseline with Clinton beating Trump in the popular vote by 48% to 47%.
To age weight the result I just applied the exit poll percentages to the total population in each age bracket - i.e. what would have happened if everyone in each age group voted the same way as their peers. This obviously increases the size of the electorate so absolute numbers are less interesting. Clinton now beats Trump 48% to 46%, possibly enough to reverse the electoral college outcome (I haven't attempted this projection state by state).
Making up for poor turnout is an interesting adjustment, but what about life expectancy? All of those baby boomers have plenty of free time to vote but are not going to be around to die of obscure tropical diseases in the Minnesotan jungle. So I also weighted each population segment by life expectancy (18-29 year-olds are going to be around for another 55 years, 65+ more like 7). Clinton now has a majority instead of a plurality - she beats Trump 50% to 42%.
All three models are shown in terms of total votes counted in the chart above.
Related Posts
- 2020 Results
- What if the Senate Voted Proportionally to Population
- Change in Presidential Vote from 2000 to 2020 by US County
You Might Also Like
Vernal (Spring) Equinox 2019
Spring starts now in the Northern Hemisphere. Rendered in (the recently updated) Catfood Earth.
Related Posts
You Might Also Like
Golden Gate Bridge Timelapse
Shot from the Marin headlands, the Golden Gate Overlook and near Fort Point. I used an RX10 IV with an ND3.0 filter. Raw images were captured every four seconds with a one second exposure time. Edited with LRTimelapse and scored with Filmstro Pro.
I was expecting a pretty sunny day but ended up with regular showers and some pretty wild swings between sunshine and cold soggy overcast weather. I think the occasional raindrop and the mood swings work quite well, although the wind caused a bit of wobble on the long zoom onto the deck of the bridge.
Related Posts
You Might Also Like
Catfood Earth 3.44
Catfood Earth 3.44 is now available to download.
The timezone database has been updated to 2018i.
Eric Muller's shapefile map of timezones is no longer maintained and so Catfood Earth has switched to Evan Siroky's timezone boundary builder version.
A bug that could cause all volcanoes to be plotted at 0,0 depending on your system locale has been fixed.
