Catfood Earth 3.45 is now available to download. Catfood Earth for Android 1.60 is available on Google Play and will update automatically if you have it installed.
I only just released 3.44 with some timezone updates but in the past week the location I had been using for global cloud cover abruptly shut down. If you like up to date clouds you'll want to install the new versions as soon as possible. With this update I'm building a cloud image every three hours and serving through this blog (and thankfully CloudFlare) so any further changes should not require a code release.
Autumnal Equinox 2018 #code#earth#equinox#autumnal The moment of Autumnal Equinox 2018 (the start of Autumn in the Northern Hemisphere) rendered in Catfood Earth.
It's Autumn for the Northern Hemisphere and Spring south of the equator.
(Published to the Fediverse as:
Autumnal Equinox 2018 #code#earth#equinox#autumnal The moment of Autumnal Equinox 2018 (the start of Autumn in the Northern Hemisphere) rendered in Catfood Earth.)
Better related posts with word2vec (C#) #code#software#word2vec#ithcwy#ml How to use word2vec to create a vector representation of a blog post and then use the cosine distance between posts to select improved related posts.
Updated on Tuesday, February 13, 2024
I have been experimenting with word2vec recently. Word2vec trains a neural network to guess which word is likely to appear given the context of the surrounding words. The result is a vector representation of each word in the trained vocabulary with some amazing properties (the canonical example is king - man + woman = queen). You can also find similar words by looking at cosine distance - words that are close in meaning have vectors that are close in orientation.
This sounds like it should work well for finding related posts. Spoiler alert: it does!
My old system listed posts with similar tags. This worked reasonably well, but it depended on me remembering to add enough tags to each post and a lot of the time it really just listed a few recent posts that were loosely related. The new system (live now) does a much better job which should be helpful to visitors and is likely to help with SEO as well.
I don't have a full implementation to share as it's reasonably tightly coupled to my custom CMS but here is a code snippet which should be enough to get this up and running anywhere:
The first step is getting a vector representation of a post. Word2vec just gives you a vector for a word (or short phrase depending on how the model is trained). A related technology, doc2vec, adds the document to the vector. This could be useful but isn't really what I needed here (i.e. I could solve my forgetfulness around adding tags by training a model to suggest them for me - might be a good project for another day). I ended up using a pre-trained model and then averaging together the vectors for each word. This paper (PDF) suggests that this isn't too crazy.
For the model I used word2vec-slim which condenses the Google News model down from 3 million words to 300k. This is because my blog runs on a very modest EC2 instance and a multi-gigabyte model might kill it. I load the model into Word2vec.Tools (available via NuGet) and then just get the word vectors (GetRepresentationFor(...).NumericVector) and average them together.
I haven't included code to build the word list but I just took every word from the post, title, meta description and tag list, removed stop words (the, and, etc) and converted to lower case.
Now that each post has a vector representation it's easy to compute the most related posts. For a given post compute the cosine distance between the post vector and every other post. Sort the list in ascending order and pick however many you want from the top (the distance between the post and itself would be 1, a totally unrelated post would be 0). The last line in the code sample shows this comparison for one post pair using Accord.Math, also on Nuget.
I'm really happy with the results. This was a fast implementation and a huge improvement over tag based related posts.
Updated 2023-01-29 22:00:
I have recently moved this functionality to use an OpenAI embeddings API integration.
(Published to the Fediverse as:
Better related posts with word2vec (C#) #code#software#word2vec#ithcwy#ml How to use word2vec to create a vector representation of a blog post and then use the cosine distance between posts to select improved related posts.)
Update April 19, 2020: I made a longer, higher resolution version which you can find here.
Here's an animation showing a year of global cloud cover (from July 2017 to July 2018) :
The clouds are sourced from the free daily download at xplanet. I run a Google apps script that saves a copy of the image to Google Drive every day (basically the same as this script to save Nest cam images). The hard part was waiting a year to get enough frames. Xplanet combines GEOS, METEOSAT and GMS satellite imagery with some reflection near the poles. Although I didn't need to for this project note that you can subscribe to higher quality / more frequent downloads.
As well as the clouds you can also see the terminator between day and night change shape over the course of the year. This video starts and ends with the Summer equinox when days are longest in the Northern hemisphere.
Where it's nighttime the image is based on NASA's Black Marble. The daytime is based on Blue Marble, but blended with a different older image which has better ocean colors and interpolated daily between twelve monthly Blue Marble satellite images. The result of this is that you can see snow and ice coverage changing over the course of the year. If you look closely you'll also notice vegetation growing and dying back with the seasons.
Rendered in a slightly modified build of Catfood Earth (the main release doesn't know how to access my private cache of xplanet cloud images). As well as combining day, night and cloud images Catfood Earth can also show you earthquakes, volcanoes, US weather radar, political borders, places and time zones. It has been enlivening Windows desktop wallpaper for fifteen years now (as shareware back when that was a thing, these days it's a free download for Windows and Android).
Summer Solstice 2018 #code#solstice#summer#winter#earth#northern#estival Summer (or estival) Solstice 2018 rendered in Catfood Earth. This means the start of Summer in the Northern Hemisphere and Winter for the Southern.
Summer starts now in the Northern Hemisphere and the Sun is at its highest point in the sky. For those in the Southern Hemisphere I'm sorry to report that the opposite is true. Rendered in Catfood Earth.
(Published to the Fediverse as:
Summer Solstice 2018 #code#solstice#summer#winter#earth#northern#estival Summer (or estival) Solstice 2018 rendered in Catfood Earth. This means the start of Summer in the Northern Hemisphere and Winter for the Southern.)
Catfood Earth 3.43 updates the timezone database to 2018e. The big change is that North Korea is moving back to UTC +09 today (May 5, 2018). The time zones layer in Catfood Earth shows the current time in each zone at the top of your screen and color codes each country and region. You can also display a list of specific places (either an included list of major cities or your own custom locations).
Catfood Earth is dynamic desktop wallpaper for Windows that includes day and night time satellite imagery, the terminator between daytime and nighttime, global cloud cover, time zones, political borders, places, earthquakes, volcanoes and weather radar. You can choose which layers to display an how often the wallpaper updates.
Vernal (Spring) Equinox 2018 #code#earth#equinox#spring#autumn#vernal Catfood Earth render of the moment that Spring starts in 2018 (the Vernal Equinox), or Autumn if you live south of the Equator.
Springtime in the Northern Hemisphere, Autumn starts south of the Equator. Rendered in Catfood Earth.
(Published to the Fediverse as:
Vernal (Spring) Equinox 2018 #code#earth#equinox#spring#autumn#vernal Catfood Earth render of the moment that Spring starts in 2018 (the Vernal Equinox), or Autumn if you live south of the Equator.)
