Improving the accuracy of the new Catfood Earth clouds layer

By Robert Ellison. Updated on Sunday, September 3, 2023.

Equirectangular clouds for Catfood Earth

Last weekend I rushed out a new version of Catfood Earth because the clouds layer stopped working. I'd been using xplanet clouds which published a free 2048x1024 image infrequently and for some reason the site has vanished (probably because it was based on data from the Dundee Satellite Receiving Station which has apparently closed). The University of Wisconsin-Madison Space Science and Engineering Center makes global cloud data available for free for non-commercial use so I plugged this in to get Catfood Earth up and running.

Clouds started working again last Saturday but they were slightly out of position. With a pixel on a HD screen being up to 13 miles this isn't ideal. I hope nobody is using Catfood Earth for weather forecasting or anything too serious but this bothered me enough that I had to fix it. From today the clouds layer is as close to perfect as I can make it.

Mostly for my own sanity in case I ever need to fix this again here's the process.

I use the globalir product because it covers both day and night. This is available in a number of resolutions and you just need to download a set of tiles and stitch them together - I'm using 4096x4096 as the source for the new layer. Catfood Earth uses equirectangular images which have a 2:1 aspect ratio so to start with I resized this less at the equator and more at the poles and got an image that was almost right.

The source image is Spherical (or Web) Mercator which is useful for Google Maps but not right for Catfood Earth. I found a Stack Exchange post which pointed me in the right direction - use GDAL (Geospatial Data Abstraction Library) to transform the Mercator image to equirectangular by tagging the corners and then warping it. This got me closer but it still wasn't right. More googling led me to a post by Robert Simmon with a gdalwarp tip to use -te and -te_srs to force the output to match the dimensions of the input.

The last problem is that the Mercator image doesn't reach all the way to the poles. In this case there is no alternative but to invent some data and so I flip the top and bottom 50 pixels to cover the gap. You can see this if you look carefully but as this region is rendered over ice in Catfood Earth it won't make a difference in practice (and the old image used this trick as well for a greater proportion of the clouds).

One final visual tweak is to drop the lowest intensity pixels which end up blurring out the background in Catfood Earth. You can adjust the transparency and color used to render the layer to suit your tastes (the default is 50% transparency and a gray color for the clouds which I think strikes a good balance).

I also increased the frequency of updates so a new image is available every hour. I said last week that an advantage of this new system was that I wouldn't need to push a new version of Catfood Earth to tweak the cloud logic. That's partly true - you'll get the new improved image with the current version, but I stepped down the update frequency to once a day when I started using xplanet clouds. There will be another update soon to the Windows and Android versions to switch to hourly updates.

Updated 2022-11-05 12:28:

A couple of updates here. The first is that RealEarth started watermarking images over a certain resolution and/or usage volume. You now need an API key and potentially to pay based on usage. More details here. I managed to stay in the free tier by asking nicely and scaling back to HD. The clouds download is still 4k but this is upscaled from a HD version.

Grabbing the latest image (for globalir) worked until sometime in October and then the tiles stopped lining up correctly. I'm not sure what changed, I guess that 'latest' used to mean the latest composite and now means the latest data for any given tile regardless of it is the same time as the others. I just fixed this to use the latest complete image (this API provides available dates and times, and then you need to use the date and time in the globalir call to get a working composite image).

Updated 2023-09-03 01:03:

There is now a video made from the last 48 hours of cloud images here, updated hourly. The video skips the step of dropping low intensity pixels so you get 100% of the cloud cover.

Add your comment...

Related Posts

(All Code Posts)

(Published to the Fediverse as: Improving the accuracy of the new Catfood Earth clouds layer #code #earth #clouds Using the Geospatial Data Abstraction Library to transform a Spherical Mercator cloud image to equirectangular for Catfood Earth. )

Catfood Earth 3.45

By Robert Ellison. Updated on Monday, May 31, 2021.

New cloud layer image in Catfood Earth

Catfood Earth 3.45 is now available to download. Catfood Earth for Android 1.60 is available on Google Play and will update automatically if you have it installed.

I only just released 3.44 with some timezone updates but in the past week the location I had been using for global cloud cover abruptly shut down. If you like up to date clouds you'll want to install the new versions as soon as possible. With this update I'm building a cloud image every three hours and serving through this blog (and thankfully CloudFlare) so any further changes should not require a code release.

Add your comment...

Related Posts

(All Code Posts)

Humpback Whales in Monterey Bay

Humpback Whales in Monterey Bay

Humpback Whales in Monterey Bay

Humpback Whales in Monterey Bay

Photos of Humpback Whales in Monterey Bay taken from a whale watching boat trip.

Add your comment...

Related Posts

(Recent Photos)

(Published to the Fediverse as: Humpback Whales in Monterey Bay #photo #whales Photos of Humpback Whales in Monterey Bay from a whale watching boat trip. )

Game of Death

By Robert Ellison. Updated on Saturday, February 19, 2022.

Generation 390

Conway's Game of Life, for 1,830 generations, starting from a random pattern. Instead of showing the live cells this animation focuses on death - each dead cell gets a little bit greener with each generation. You can just about make out a few static patterns in the darkness and the lines cruising through are left behind by gliders. Mostly though you're watching the horrible loss of life caused by cellular social isolation.

Add your comment...

Related Posts

(More Timelapses)

(Published to the Fediverse as: Game of Death #timelapse #animation #conway #life #death #video Animation of Conway's Game of Life flipped to focus on the fate of the dead cells. )

ITHCWY Newsletter for March 2019

Facebook shouldn't own your social graph

Catfood Earth 3.44 is available to download. This version updates the timezone database to 2018i, moves to a new source for timezone mapping an fixes a bug in the volcanoes layer.

Golden Gate Bridge timelapse - shot from the Marin headlands, the Golden Gate Overlook and near Fort Point.

Colorado looks set to join the National Popular Vote Interstate Compact. What you can do to help.

Could we solve the youth turnout problem with age weighted voting?

Previously:

Add your comment...

Related Posts

(All Etc Posts)

Book reviews for March 2019

Age and Life Expectancy Weighted Voting

Actual, age weighted and age and life expectancy weighted results of the 2016 Presidential Election

Youth turnout for elections is famously dismal. In 2016 less than half of 18-29 year-olds voted, compared to over two thirds as you get to 45 and older (US Census). The impact is an incentive to cater to the old - trying to make America great again (like you remember from when you were young) vs doing something about climate change or house prices.

One fix is compulsory voting, like in Australia. I'm not sure I want to force people without an opinion to vote though.

What if we just weighted votes by the total size of the demographic group?

I took the demographic breakdown of 2016 voters from the US Census Bureau and multiplied these by the age breakdown from CNN exit polls. This gave Clinton a lead of just under a million votes - somewhat lower than the actual result. This is likely a polling error in the exit poll, but it's a reasonable baseline with Clinton beating Trump in the popular vote by 48% to 47%.

To age weight the result I just applied the exit poll percentages to the total population in each age bracket - i.e. what would have happened if everyone in each age group voted the same way as their peers. This obviously increases the size of the electorate so absolute numbers are less interesting. Clinton now beats Trump 48% to 46%, possibly enough to reverse the electoral college outcome (I haven't attempted this projection state by state).

Making up for poor turnout is an interesting adjustment, but what about life expectancy? All of those baby boomers have plenty of free time to vote but are not going to be around to die of obscure tropical diseases in the Minnesotan jungle. So I also weighted each population segment by life expectancy (18-29 year-olds are going to be around for another 55 years, 65+ more like 7). Clinton now has a majority instead of a plurality - she beats Trump 50% to 42%.

All three models are shown in terms of total votes counted in the chart above.

Add your comment...

Related Posts

(All Politics Posts)

(Published to the Fediverse as: Age and Life Expectancy Weighted Voting #politics #politicalreform What if we solved the youth turnout problem by weighting election results by demographics, or to be completely fair by life expectancy as well. )

Vernal (Spring) Equinox 2019

Spring Equinox 2019 in Catfood Earth

Spring starts now in the Northern Hemisphere. Rendered in (the recently updatedCatfood Earth.

(Previously)

Add your comment...

Related Posts

(All Code Posts)

(Published to the Fediverse as: Vernal (Spring) Equinox 2019 #code #earth #equinox #spring #autumn #vernal Spring or Vernal Equinox 2019 at 21:58 UTC on Match 20, 2019. Rendered in Catfood Earth. )

Golden Gate Bridge Timelapse

By Robert Ellison. Updated on Saturday, February 19, 2022.

Golden Gate Bridge from near Fort Point

Shot from the Marin headlands, the Golden Gate Overlook and near Fort Point. I used an RX10 IV with an ND3.0 filter. Raw images were captured every four seconds with a one second exposure time. Edited with LRTimelapse and scored with Filmstro Pro.

I was expecting a pretty sunny day but ended up with regular showers and some pretty wild swings between sunshine and cold soggy overcast weather. I think the occasional raindrop and the mood swings work quite well, although the wind caused a bit of wobble on the long zoom onto the deck of the bridge.

(Previously)

Add your comment...

Related Posts

(More Timelapses)

(Published to the Fediverse as: Golden Gate Bridge Timelapse #timelapse #ggb #video Time lapse of the Golden Gate Bridge shot from the Marin headlands, the Golden Gate Overlook and Fort Point in San Francisco, CA. )

Catfood Earth 3.44

By Robert Ellison. Updated on Monday, May 31, 2021.

Updated timezones in Catfood Earth 3.44

Catfood Earth 3.44 is now available to download.

The timezone database has been updated to 2018i.

Eric Muller's shapefile map of timezones is no longer maintained and so Catfood Earth has switched to Evan Siroky's timezone boundary builder version.

A bug that could cause all volcanoes to be plotted at 0,0 depending on your system locale has been fixed.

Download Catfood Earth.

(Previously)

Add your comment...

Related Posts

(All Code Posts)