Sunset #9

Sunset #9

Shot as the rather moist storm of March 27, 2022 rolled into San Francisco. Some decent clouds and then a good soaking, probably the last of the season. Looking north-west from West Portal.

Add your comment...

Related Posts

(More Timelapses)

(Published to the Fediverse as: Sunset #9 #timelapse #video #sunset Time lapse of the March 27, 2022 storm in San Francisco, California. Shot looking north-west from West Portal. )

Treasure Island Perimeter

By Robert Ellison. Updated on Sunday, April 24, 2022.

Treasure Island Perimeter

Treasure Island Perimeter

Treasure Island Perimeter

Treasure Island Perimeter

Today, for no great reason, I decided to see if it's possible to walk the full perimeter of Treasure Island. I parked at the Treasure Island Museum and headed north up the Avenue of the Palms. I've been here many times and it offers stunning views of San Francisco and both the Bay Bridge and the Golden Gate Bridge. The path continues around the north of the island and then you need to work around a blocked off area to get back to the coast. The east side is less appealing with a lot of garbage along the path and a view of many barges and the Port of Oakland peeking out under the new section of the Bay Bridge. Finally I was fully thwarted as the entire south west corner of the island is a mess of construction (and a rather unpleasant portable toilet staging area). I had to head inland all the way to 9th Street before cutting back over to the Avenue of the Palms. Overall I'm glad I tried but I can't recommend it.

Hike starts at: 37.817257, -122.371726. View in Google Earth.

Add your comment...

Related Posts

(Hike Map)

(Published to the Fediverse as: Treasure Island Perimeter #hike #sanfrancisco #treasureisland #map 4 mile hike around the perimeter of Treasure Island in the San Francisco Bay. It's not possible to fully circumnavigate the island. )

Vernal (Spring) Equinox 2022

Spring Equinox 2022 rendered in Catfood Earth

Spring starts right now in the northern hemisphere, Autumn if you happen to find yourself south of the equator. Rendered in Catfood Earth.

Add your comment...

Related Posts

(All Code Posts)

(Published to the Fediverse as: Vernal (Spring) Equinox 2022 #code #earth #equinox #spring #autumn #vernal Spring Equinox at 15:33 UTC on March 20, 2022 rendered in Catfood Earth. )

Both sides of the Golden Gate, a Time Lapse

Both Sides

A timelapse of the Golden Gate, shot from Kirby Cove in the Marin Headlands and then from Grand View Park in San Francisco.

Add your comment...

Related Posts

(More Timelapses)

(Published to the Fediverse as: Both sides of the Golden Gate, a Time Lapse #timelapse #video #sanfrancisco #kirbycove #grandview Shot from Kirby Cove in the Marin Headlands and Grand View Park in San Francisco, a timelapse of the Golden Gate. )

Book reviews for February 2022

Mickey7: A Novel by Edward Ashton

Mickey7: A Novel by Edward Ashton

3/5

 

The Nineties: A Book by Chuck Klosterman

The Nineties: A Book by Chuck Klosterman

4/5

 

Product-Led SEO: The Why Behind Building Your Organic Growth Strategy by Eli  Schwartz

Product-Led SEO: The Why Behind Building Your Organic Growth Strategy by Eli Schwartz

3/5

 

Add your comment...

Related Posts

(All Reviews)

TLOTW #5

TLOTW #5

South from Golden Gate Heights Park, Sutro Tower from Twin Peaks, south from Golden Gate Heights again with more clouds and west from Grand View Park. Photo is a panorama of Golden Gate Park from Grand View.

I'm switching from the more ambitious specific weeks to a timelapse of the week (TLOTW) when I have one.

(Previously: Timelapse, Week of Jan 31 2022)

Add your comment...

Related Posts

(More Timelapses)

(Published to the Fediverse as: TLOTW #5 #timelapse #video #tlotw #sanfrancisco #grandview #goldengateheights #sutrotower #twinpeaks Time lapse of the week #5 featuring views from Golden Gate Heights Park and Grand View Park in San Francisco. )

Catfood WebCamSaver 3.27

By Robert Ellison. Updated on Saturday, March 19, 2022.

Catfood WebCamSaver 3.27

Catfood WebCamSaver 3.27 is available to download.

This release contains the latest webcam list and will upgrade any current set of webcams. I'm currently releasing updates for WebCamSaver every three months with the latest cams.

Add your comment...

Related Posts

(All Code Posts)

Anemone Cave

Sea Anemones in a cave

Google Pixel 6 Pro 7mm f1.9 1/180s ISO44

A cave full of sea anemones at Marshall's Beach in San Francisco.

Add your comment...

Related Posts

(Recent Photos)

(Published to the Fediverse as: Anemone Cave #photo #anemone Photo of sea anemones in a cave at Marshall's Beach in San Francisco, California. )

Using the Todoist API to set a due date on the Alexa integration to-do list (with Apps Script)

By Robert Ellison. Updated on Tuesday, December 6, 2022.

Todoist Alexa Fix

I love almost everything about Todoist. It's rich enough to scratch all my productivity itches while also being basic enough that I don't spend time gardening my tasks. Also, their Android app is just gorgeous, the exact opposite of Google's ascetic descent into identical lists of black in Material Design.

The one problem is the Alexa integration. You can add tasks to your shopping list or your to do list. The shopping list works great for me. I check it once a week when I do my grocery shopping and everyone knows to just tell Alexa when they need something. The to-do list is a disaster. These go to a separate to-do list project without a due date and I will never ever find them there. Anything else I add will end up in the Inbox with a due date of today so I'm forced to classify and if necessary reschedule it. Which is exactly what I want.

When I say 'Alexa, add x to my to-do list' I want that task to be visible. This integration design flaw could lead to at least one child growing up in an unfamiliar part of town without parents, or worse.

I emailed Todoist and they politely declined to change the way the integration works. After a brief period of steaming I've rolled up my sleeves and fixed it with their API. Which doesn't use OAuth so now I love them even more.

The script is below. Create a new Apps Script project in Google Drive (New and then choose More to find this) and copy in the code. You can get the API token from the bottom of the Integrations section in Todoist settings. Then just click the clock in the Apps Script project and schedule checkForAlexaTasksWithNoDate() to run as often as you need. The script will check the Alexa To-do List project and if anything is in there without a due date it will set it to today to force you to deal with it.

Updated 2022-12-06 21:27:

Todoist have updated their API to v2 (migration guide). There are no breaking changes for this code and so I got it working again by changing v1 to v2 in the API calls. The code above has also been changed to update dates in multiple projects, this is because I started checking for tasks with no due date in the Inbox project as well as Alexa tasks. You can delete the Inbox call if you don't want this behavior (and add additional projects if needed as well).

Add your comment...

More Google Apps Script Projects

(All Code Posts)

(Published to the Fediverse as: Using the Todoist API to set a due date on the Alexa integration to-do list (with Apps Script) #code #appsscript #google #drive #todoist #productivity Script to fix an annoyance with the Todoist Alexa integration where tasks are added without a due date leading them to be overlooked. Requires Google Drive and Todoist. )

Monitor page index status with Google Sheets, Apps Script and the Google Search Console API

By Robert Ellison. Updated on Friday, October 4, 2024.

GSC Monitor

Overview

Google just released URL inspection as part of the Search Console API. I check for issues periodically in Search Console but it would be great to just get an email when an issue crops up. The Apps Script project below does just that by monitoring URLs from your sitemap for changes and sending an email whenever anything is detected. The Search Console API has a limit of 2,000 calls per day and Apps Script also imposes a time limit on scripts. The approach I take below assigns a random day of the week to each URL to limit the number checked on each run. Depending on the size of your site you may want to remove this check (or go the other way if you have a large number of URLs to monitor). Follow the steps below to get your own monitoring spreadsheet up and running.

Google Sheet

Create a new spreadsheet in Google Drive and call it anything you want. Rename an empty sheet to 'gsc', this sheet will store the index data. You don't need to make any other changes to this sheet.

Choose Apps Script from the Extensions menu. This will open up the script editor for your spreadsheet. I find that sometimes the editor opens with the wrong account if you're signed into more than one. If that applies to you, check quickly to make sure the right account is selected. With Code.gs selected copy and paste the script below replacing the default function:

There are a few configuration variables to enter at the top. AlertEmail is the email address to notify when index status changes are detected. SitemapUrl is the full URL of your sitemap. The current implementation does not support sitemap index files, this needs to be a regular sitemap containing URLs. SearchConsoleProperty should be the URL of the site to monitor, or sc-domain: followed by the domain for domain properties (for this site sc-domain:ithoughthecamewithyou.com).

Click Project Settings (the cog in the left hand menu of the script editor) and copy the Script ID. Make a note of this for later.

API Console

Next we need to configure the Search Console API in the Google Cloud Platform Console.

  1. Create  a new project (click the drop down to the right of 'Google Cloud Platform Console' and then New Project). Pick any name you like. 
  2. Once the project is created, find APIs and Services in the left hand menu and choose Library.
  3. Search for Search Console, click on the Search Console API and then click Enable. 
  4. A new screen will load, click Credentials in the left hand menu. 
  5. Click Configure Consent Screen and choose the internal type.
  6. Fill in the required fields - application name and contact emails. 
  7. Add the ./auth/webmasters.readonly scope for readonly access to Search Console data.
  8. Once the consent screen is complete click Credentials in the left hand menu again.
  9. Click Create Credentials at the top and choose OAuth Client ID. 
  10. Choose Web Application.
  11. Add https://script.google.com/macros/d/{SCRIPTID}/usercallback to authorized redirect URLs, replacing {SCRIPTID} including the brackets with the Apps Script ID you noted above. 
  12. Make a note of the Project ID.

Complete the Apps Script

Return to the Apps Script project and find the settings page. Set the GCP project to the project ID you noted above. Also on this page check the Show "appsscript.json" manifest file in editor option.

Go to the code editor and open appsscript.json. Add the following line:

"oauthScopes": ["https://www.googleapis.com/auth/script.external_request", "https://www.googleapis.com/auth/webmasters.readonly", "https://www.googleapis.com/auth/spreadsheets.currentonly", "https://www.googleapis.com/auth/script.send_mail"],

Make sure the script is saved and close the script editor window. Reload the spreadsheet. Once the reload completes you should have a Search Console menu at the top of the spreadsheet. Choose Update Data from the Search Console menu. This will run for a few minutes and then populate the 'gsc' sheet with your URL data. See UrlInspectionResult in the API documentation for more information about the meaning of each field. You should also get a lengthy email with a notification for each URL that was inspected. This will continue for the first week and then you'll only get updates for interesting status changes.

Scheduling

Now the project is working, open the script editor again (Apps Script from the Extensions menu) and open Triggers (the clock icon in the left hand menu). Click Add Trigger at the very bottom right of the window. Select runUpdate as the function to run. Change event source to time-driven, and then select day timer and an hour to run the script. Lastly click Save. Your Google Search Console monitor will now run every day, and if the index status of a page changes you'll get an email about it within a week. The spreadsheet will also come in handy for other analysis and reporting.

Troubleshooting

You might need to tweak the script if you start hitting limits. The URL inspection API currently has a 2,000 call / day quota. Apps Script will only run for around 7 mins on a free account and 20 mins if you have Google Workspace. If either of these limits apply you could modify the 'checkDay' logic to use day of the month (or year, or ...) to reduce the number of URLs inspected on each run. If you need to do this remember to update the Check Day column on the 'gsc' sheet as well.

The script assumes that the URLs you want to monitor are in your sitemap. If this is not the case you can add URLs to the sheet directly. As long as they are part of the configured property you will still get results. If you use this method you might want to comment out the updatePagesFromSitemap() call in runUpdate() to save time.

If anything else goes wrong please leave a comment below and I'll do my best to help you.

Updated 2022-02-08 17:40:

Monitor page index status with Google Sheets, Apps Script and the Google Search Console API

After a couple of days I have a full dump of my sitemap from the page index status API. I wrote this script for the alerting possibilities but couldn't resist some analytics once the dataset was complete.

The chart above shows sessions vs days since the last Google Crawl. Pretty stark - Google keeps a close eye on the pages it sends traffic to and not so much on the others.

I set lastmod honestly and there is good news here. I could only find two cases where Google had not crawled the page since the last modified date. So when the sitemap says a page has changed the odds are good that it will get another crack at the index. The two exceptions are unusual posts that are updated hourly and weekly respectively and both have been crawled recently.

The breakdown of index status matches Google Search Console pretty well but I have a handful of pages that are 'Indexed, not submitted in sitemap', even though they are in the sitemap and no such status is shown on Search Console. I don't know if this is a glitch in the index status API or something to do with how the pages were discovered. Some light searching suggests that this message is usually what you would expect it to be.

Lastly, updating the sheet for my site is more bound by script execution time than the API limits. I changed it to run every hour and instead of partitioning by day of week I used a random hour of the day which means I check every URL at least once every 24 hours.

Updated 2022-04-24 10:50:

I just updated the code and post above. I've had occasional issues where updating the sheet failed which caused the next run to go back to the beginning with no saved index status. To reduce the chance of this happening I've added some retry logic and also improved the speed of the sheet load and save functions. I also had a comment on the code that suggested an easier way to handle OAuth and have incorporated this in the new version.

Add your comment...

More Google Apps Script Projects

(All Code Posts)

(Published to the Fediverse as: Monitor page index status with Google Sheets, Apps Script and the Google Search Console API #code #searchconsole #appsscript #google #sheets #drive #seo How to use Apps Script in Google Sheets to automatically monitor index status in Google Search Console and get an email alert if anything changes. )