Android 11 Gripes

11

You have to make allowances for the fact that many people are working from home (where maybe it's harder to test code and you certainly can't do hallway usability testing). Things also improved somewhat with the October patch. But Android 11 was a Windows ME level disaster. Google says that they dropped the desserts to make Android 'more accessible to a global audience' but I think it's probably because they know that new updates are no longer sweet.

Multi-tasking is completely broken. In pandemic mode I'm on video calls all day, and dodging the 2-3 video calls that are usually going on elsewhere in my house. Being able to have Teams and notes running at the same time is pretty important right now and with 11 it's not possible.

Other than conference calls and Kindle the other main use I have for my phone is podcasts. Android 11 improved the media controls by moving them to the quick settings area of the notification shade, providing easy switching between playback devices and allowing you to swipe through recent playback apps. All good, except that none of it works. The controls are there but do nothing so I have to run the app to pause. Also, there is a weird ghost of a previous media playback that shows up and then disappears when pulling down the notification shade. And as for dismissing previous sessions that seems to require a reboot.

Multi-tasking and media were fixed in the most recent patch, but there is also some new notification system to separate out conversations. In practice this seems to mean I get multiple groups of Gmail and Teams notifications instead of a single cluster per app. This isn't what I want, gives me more work to do and so far I haven't found a way to turn it off. Notifications have steadily improved over the last few major Android releases so it's upsetting to see them becoming worse.

This is all on a Pixel 4XL which you would assume would get some level of testing love. It used to be that the main advantage of a Pixel was getting new versions of Android quickly. With 11 I'm for the first time wishing I was waiting a few months while the kinks were worked out.

Did anyone tell Material Design about Gesture Navigation?

Did anyone tell Material Design about Gesture Navigation?

The screen shot above is from Google Fit. Which icon is active? I can't tell any more. Is it the blue one or the underlined one which is a much stronger cue?

Of course none of the icons are underlined. This is a bottom navigation bar on top of the Android navigation bar on Android 10 with gesture navigation enabled. My brain knows this but my finger still tries to click on Home. Journal just looks so much more active I can't help it. This friction is also in Google Photos and Google Maps and presumably Google everything before too long.

Maybe the Google app developers don't have access to recent Pixels, or maybe the Material Design team all have iPhones?

(Previously: Material Design 3)

Google search-for-your-own-verified-sites Console

Google Search Console

I don't know about you, but when it comes to Google Search Console I spend about 0.01% of the time adding sites and 99.99% analyzing existing ones. And yet when signing into Search Console with many verified sites the interface is ALL about adding a new one. Maybe 10% of the UX would be reasonable but it looks for all the world like I have nothing added.

To get to my sites I need to click the hamburger. Come on Google, being mobile first doesn't have to mean being desktop hostile.

Clicking the hamburger isn't even enough. This just brings up a practically blank sidebar. I then need to expand the 'Search property' drop down. Finally I get a needlessly scrolling list of my sites.

Using the Azure Monitor REST API from Google Apps Script

Updated on Wednesday, May 13, 2020

Average Server Response Time in Azure Metrics

This post describes how to get metrics (in this case average response time) from an Azure App Service into a Google Sheet. I’m doing this so I can go from the sheet to a Data Studio dashboard. I already have a report in Data Studio that pulls from Ads, Analytics and other sources. I’d rather spend hours adding Azure there than be forced to have one more tab open. You might have different reasons. Read on. 

  1. Create a Google Sheet and give it a memorable name. Rename the first sheet to AvgResponseTime and put ‘Date’ in A1 and ‘Average Response Time’ in B1.
  2. Create a script (Script editor from the Tools menu) and give that a good name as well.
  3. In the script editor pick Libraries from the Resources menu. Enter 1B7FSrk5Zi6L1rSxxTDgDEUsPzlukDsi4KGuTMorsTQHhGBzBkMun4iDF which is the Google OAuth library, pick the latest version and click Save.
  4. Select Project properties from the File menu and make a note of the Script ID.
  5. Log into your Azure Console and then go to https://resources.azure.com/. You are looking for a metricdefinitions node for the resource that you want to monitor. In my case it’s subscriptions / MyName Subscription / resourceGroups / providers / Microsoft.Web / sites / MySite / metricdefintions. Browse through this list to find the id of the metric you’re interested in. For me it’s AverageResponseTime. Finding this was the hardest part. Microsoft’s documentation for resourceUri is literally ‘The identifier of the resource.’ Why even bother Microsoft? Make a note of the id and remove the ‘metricDefinitions/AverageResponseTime’ from the end, because of course the ID isn’t quite right for some reason. Mine looks something like this: /subscriptions/mylongid/resourceGroups/mysomethingResourceGroup/providers/Microsoft.Web/sites/mysiteid
  6. Go back to the Azure Console and open Azure Active Directory. Select App registrations under Manage and create a New registration. Time to come up with another name. You probably want ‘Accounts in this organizational directory only’. The redirect URL is https://script.google.com/macros/d/SCRIPTID/usercallback - replace SCRIPTID with the Script ID you made a note of in step 4.
  7. Click the View API permissions button, then Add a permission and then pick Azure Service Management. I’m using Delegated permissions and the user_impersonation permission. Then click Grant admin consent for Default Directory.
  8. Go to Certificates & secrets (under manage) and create a new client secret. Make a careful note of it.
  9. Go to Authentication (under Manage), check Access tokens under Implicit grant and then click Save at the top of the page.
  10. Go to Overview and make a note of your Application (client) ID and Directory (tennant) ID.
  11. You are getting close! Go to the script editor (from step 2) and paste in the code at the bottom of this post. There are four variables to enter at the top of the script. ClientID and TennantID are from step 10. ClientSecret is from step 8. ResourceID is from step 5. Save the script.
  12. Reload the spreadsheet (back from step 1). You should get an Azure Monitor menu item. Choose Authorize from this menu. Google will ask you to authorize the script, do this for the Google account you’re using. Choose Authorize again, this time a sidebar will appear with a link. Follow the link and authorize against Azure (if you’re logged in this might just go directly to success). If you get authorization errors in the future run this step again. If that does help use Reset Settings and then try again.
  13. You should be ready to get data. Choose Fetch Data from the Azure Monitor menu. If this doesn’t work check through steps 1-12 carefully again!
  14. Last step - automate. Go back to the script editor. Choose Current project’s triggers from the Edit menu. Add a trigger (the small blue button hidden at the bottom right of the screen - not everything needs a floating action button Google!) to run fetchData daily at some reasonable time.
You should now have a daily record of average response time flowing to a Google sheet. This can easily be extended to support other metrics, and other time periods (you could get data by minute and run the script hourly for instance. See the metrics documentation for more ideas. I got this working for an App Service but it should be roughly the same flow for anything that supports metrics, you’ll just need to work on finding the right resourceUri / ID.

Please fix phone spam Google!

95% of my incoming calls are now spam. Most of them are some strange pre-recorded Chinese voice with music playing in the background but I occasionally get a free hotel stay as well.

So far Google has rolled out Call Screen. This means I can waste my time watching Google Assistant talk to the spammer. It's way faster not to bother, hang up all calls and delete the voicemails later.

It seems like instead of Call Screen there could be a better way to deal with this.

Firstly, send any call not from someone in my contacts directly to voice mail. This would actually solve a lot of the problem.

Next, for extra credit, run spam detection on the voice mail before sending it to me. If it's two seconds long and blank then just bin it. If it's Chinese with music bin it. Only if it passes the smell test should it appear in my actual voice mail. Google is very good at this for Gmail.

Please!

How to backup Google Photos to Google Drive automatically after July 2019 with Apps Script

Updated on Monday, January 13, 2020

Google Photos backup to Google Drive shutting down in July 2019

Google has decided that backing up your photos via Google Drive is 'confusing' and so Drive based backup is going away this month. I love Google Photos but I don't trust it - I pull everything into Drive and then I stick a monthly backup from there onto an external drive in a fire safe. There is a way to get Drive backup working again using Google Apps Script and the Google Photos API. There are a few steps to follow but it's pretty straightforward - you should only need to change two lines in the script to get this working for your account.

First two four caveats to be aware of. Apps Script has a time limit and so it's possible that it could fail if moving a large number of photos. You should get an email if the script ever fails so watch out for that. Secondly and more seriously you could end up with two copies of your photos. If you use Backup and Sync to add photos from Google Drive then these photos will be downloaded from Google Photos by the script and added to Drive again. You need to either upload directly to Google Photos (i.e. from the mobile app or web site) or handle the duplicates in some way. If you run Windows then I have released a command line tool that sorts photos into year+month taken folders and handles de-duplication.

One more limitation. After a comment by 'Logan' below I realized that Apps Script has a 50MB limitation for adding files to Google Drive. The latest version of the script will detect this and send you an email listing any files that could not be copied automatically.

And a fourth limitation after investigating a comment by 'Tim' it turns out there is a bug in the Google Photos API that means it will not download original quality video files. You get some lower resolution version instead. Together with the file size limit this is a bit of a deal breaker for most videos. Photos will be fine, but videos will need a different fix.

On to the script. In Google Drive create a new spreadsheet. This is just a host for the script and makes it easy to authorize it to access Google Photos. Select 'Script editor' from the Tools menu to create a new Apps Script project.

In the script editor select 'Libraries...' from the Resources menu. Enter 1B7FSrk5Zi6L1rSxxTDgDEUsPzlukDsi4KGuTMorsTQHhGBzBkMun4iDF next to 'Add a library' and click add. This will find the Google OAuth2 library Pick the most recent version and click Save.

Select 'Project properties' from the File menu and copy the Script ID (a long sequence of letters and numbers). You'll need this when configuring the Google Photos API.

In a new window open the Google API Console, part of the Google Cloud Platform. Create a new project, click Enable APIs and Services and find and enable the Google Photos API. Then go to the Keys section and create an OAuth Client ID. You'll need to add a consent screen, the only field you need to fill out is the product name. Choose Web Application as the application type. When prompted for the authorized redirect URL enter https://script.google.com/macros/d/{SCRIPTID}/usercallback and replace {SCRIPTID} with the Script ID you copied above. Copy the Client ID and Client Secret which will be used in the next step.

Go back to the Apps Script project and paste the code below into the Code.gs window:

Enter the Client ID and Client Secret inside the empty quotes at the top of the file. You also need to add an email address to receive alerts for large files. There is a BackupFolder option at the top as well - the default is 'Google Photos' which will mimic the old behavior. You can change this if you like but make sure that the desired folder exists before running the script. Save the script.

Go back to the spreadsheet you created and reload. After a few seconds you will have a Google Photos Backup menu (to the right of the Help menu). Choose 'Authorize' from this menu. You will be prompted to give the script various permissions which you should grant. After this a sidebar should appear on the spreadsheet (if not choose 'Authorize' from the Google Photos Backup menu again). Click the authorize link from the sidebar to grant access to Google Photos. Once this is done you should be in business - choose Backup Now from the Google Photos Backup menu and any new items from yesterday should be copied to the Google Photos folder in Drive (or the folder you configured above if you changed this).

Finally you should set up a trigger to automate running the script every day. Choose 'Script editor' from the Tools menu to re-open the script, and then in the script window choose 'Current project's triggers' from the Edit menu. This will open yet another window. Click 'Add Trigger' which is cunningly hidden at the bottom right of the window. Under 'Choose which function to run' select 'runBackup'. Then under 'Select event source' select 'Time-driven'. Under 'Select type of time based trigger' select 'Day timer'. Under 'Select time of day' select the time window that works best for you. Click Save. The backup should now run every day.

The way the script is written you'll get a backup of anything added the previous day each time it runs. If there are any duplicate filenames in the backup folder the script will save a new copy of the file with (1) appended in front of the filename. Let me know in the comments if you use this script or have any suggestions to improve it.

Material Design 3

Updated on Friday, May 22, 2020

Material Design 3

Material Design brought bland consistency to the Android ecosystem. Every app had some sort of bold header and a floating action button. There is some value in consistency and at least some personality was retained. It's red, it's probably Gmail. Yellow, I must be in Keep. Boring but tolerable.

Material Design 2 solves mainly for the problem of knowing which app you're looking at. Colors have gone. Every Google app is now an oppressive black list with some oppressive black icons. To add to the misery the icons have a shade of stock-library amateurism and are just a little too heavy. Unless I look really closely or the what-icon-did-I-just-click region of my brain is on top form there is no longer any way to tell the difference between Google apps.

I'm pretty sure Material Design 3 is just going to be a command prompt. What Android customers really want is telnet or wget and some raw JSON.

How does the Nest Learning Thermostat work?

Nest learning thermostat, learning

Not only does it know when you're home but the Nest Learning Thermostat also knows when you're nearby. Here's how it works.

You crank up the heat to 70 and walk away. Nest then immediately returns to 62 degrees.

Thinking there must be something screwy with the algorithm you turn it back up to 70. Nest knows that it's in trouble so it displays a comforting message like 'Heat set until 10pm', waits for you to leave and then sets the temperature back to 62 degrees.

Giving up on the learning part you use the app to manually program it to keep the heat on. Nest now uses its WiFi connection to phone the gas company and disconnect your service.

Export Google Fit Daily Steps, Weight and Distance to a Google Sheet

Updated on Sunday, October 25, 2020

Google Fit Daily Step Export

Google Fit is a great way to keep track of your daily step count without needing to carry a Fitbit or other dedicated tracker. It's not easy to get that data out though, as far as I can tell the only way is Google Takeout which is not made for automation. Luckily there is an API and you can do almost anything with Google Sheets.

If you're looking to export your step count, weight and distance this post has everything you need, just follow the instructions below to get your spreadsheet up and running. This is also a good primer on using OAuth2 with Google Apps Script and should be a decent starting point for a more complex Google Fit integration. If you have any questions or feedback please leave a comment below.

To get started you need a Google Sheet, an apps script project attached to the sheet and a Google API Project that will provide access to the Fitness API. That might sound intimidating but it should only take a few minutes to get everything up and running.

In Google Drive create a new spreadsheet and call it whatever you like. Rename the first tab to 'Metrics'. Enter 'Date' in cell A1, 'Steps' in B1, 'Weight' in C1 and 'Distance' in D1. To grab history as well create another tab called 'History' with the same headers. Next select 'Script editor...' from the Tools menu which will open a new apps script project.

Give the apps script project a name and then select 'Libraries...' from the Resources menu. Next to 'Add a library' enter 1B7FSrk5Zi6L1rSxxTDgDEUsPzlukDsi4KGuTMorsTQHhGBzBkMun4iDF and click Add. This will find the Google OAuth2 library. Choose the most recent version (24 at the time of writing) and click Save. Then select 'Project properties' from the File menu and make a note of the Script ID (a long series of letters and numbers).

Open the Google API Console. Create a new project and name it something like 'Google Fit Sheet'. From the Dashboard click Enable APIs and Services and find and select the Fitness API. Then go to Keys and create an OAuth Client ID. You'll be asked to create a consent screen, the only field you need to enter is the product name (i.e. 'My Fit App'). Then choose Web Application as the application type. You need to set the name and the authorized redirect URL. The redirect URL is https://script.google.com/macros/d/{SCRIPTID}/usercallback replacing {SCRIPTID} with the actual Script ID you made a note of above. After adding this make a note of the Client ID and Client Secret.

Go back to the apps script project and paste the code below into the Code.gs window:

Right at the top of the code there are spaces to enter the Client ID and Client Secret from the API Console. Enter these and save the project.

Switch back to your Google Sheet and reload. After reloading there will be a Google Fit menu item. First select Authorize... You'll get a screen to authorize the script and then a sidebar with a link. Click the link to authorize the script to access your Google Fit data. You can then close the sidebar and select Get Metrics for Yesterday from the Google Fit menu. You should see a new row added to the spreadsheet with yesterday's date and fitness data.

The final step is to automate pulling in the data. Go back to the apps script project and select Current project's triggers from the Edit menu. Add a trigger to run getMetrics() as a time driven day timer - I recommend between 5 and 6am. You can also click notifications to add an email alert if anything goes wrong, like your Google Fit authorization expiring (in which case you just need to come back and authorize from the Google Fit menu again.

At this point you're all set. Every day the spreadsheet will automatically update with your step count from the day before. You can add charts, moving averages, export to other systems, pull in your weight or BMI, etc. I want to add a seven day moving average step count to this blog somewhere as a semi-public motivational tool... watch this space.

Note that weight will be blank in the spreadsheet for days with no weight data. Google Fit doesn't return the last known weight, only the known value for days where an update was recorded.

If you are looking to extend this sample to other data types then this API explorer page is very helpful for finding data types that the API documentation doesn't list.

A couple of times working on this script I got my authorization in a bad state and started getting a 400 error response from the API. If this happens run your Google Fit app, click the Profile icon at the bottom and then the Settings icon at the top right. Click Manage connected apps and then disconnect the script from Google Fit. Finally run the Reset Settings option from the menu in the sheet and then authorize again.

I updated this post on Jan 21, 2019 to extend the sample to handle weight and distance as well as steps. I also improved the history function to handle many days in one API call rather than a quick hack I added earlier that pulled a day at a time. I'd recommend using the code above rather than anything included in comments below (at least comments before this update).

I'm not afraid of Google

A Project Fi display ad on an article about Google's insane targeting prowess

At BGR Chris Smith writes about Google's prodigious data collection:

"But that doesn’t change the fact that Google collects an incredible amount of data about you, especially from that device you use most, your Android phone"

And so I was amused as a Google Fi subscriber on a Pixel XL running Google Chrome and signed into my Google Account that the ad in the middle of the article was for Project Fi. If Google can't help paying BGR under these set of circumstances then we're some way off from the adtech singularity.