How to Automate PageSpeed Insights for Multiple URLs on a Schedule using Logic Apps or Flow

For the website I’m responsible for, I was interested in capturing the data from the Google PageSpeed Insights tool, and having the data recorded somewhere on a schedule. There’s a blog post on Moz.com that talked about doing this with a Google Sheet, but it wasn’t quite what I was after; I wanted the data to be collected more regularly.

Instead of using Google Sheets (and a fair amount of code), I decided to use an Azure Logic App (you can use this or Microsoft Flow), which is part of Microsoft’s Cloud platform.

The Logic App is run on a Recurrence trigger which I set to every 6 hours. By collecting the results automatically over time, you’ll see how the changes you’re making to your site affect your PageSpeed scores.

recurrence-hr

The first step simply defines the URLs you want to check, then it’ll loop over each one & call the PageSpeed API. Go get an API key, and make sure PageSpeed API is enabled.

Results from the API call are parsed out and pushed into a new row in an Excel Online sheet.

If you’re interested in setting this up yourself, I recorded a short video which shows how it works in more detail.

There are a few foibles in Logic Apps which caught me out, first, getting the list of URLs into an Array didn’t work as expected. I had to switch to Code View to correct the escaping of the return character to read;

@split(variables('urlList'), '\n')

The JSON payload from the PageSpeed API is pretty large, so I’ve listed the path to the elements you’ll be interested in below. I’m using split (on space) purely to get at the numerical value, which is more useful in the spreadsheet;

First Contentful Paint

@{split(body('HTTP')?['lighthouseResult']?['audits']?['first-contentful-paint']?['displayValue'], ' ')[0]}

First Meaningful Paint

@{split(body('HTTP')?['lighthouseResult']?['audits']?['first-meaningful-paint']?['displayValue'], ' ')[0]}

Speed Index

@{split(body('HTTP')?['lighthouseResult']?['audits']?['speed-index']?['displayValue'], ' ')[0]}

Time To Interactive

@{split(body('HTTP')['lighthouseResult']['audits']['interactive']['displayValue'], ' ')[0]}

Time to First Byte

@{split(body('HTTP')?['lighthouseResult']?['audits']?['time-to-first-byte']?['displayValue'], ' ')[3]}

Overall, this was quite easy to put together and shows the power of Azure Logic Apps. Being able to do this without any code or (your own) servers, and getting things live in a couple of hours is a fantastic tool to have at your disposal.

Make your own £5 ambient TV backlight

After clearing out some junk, which included an old halogen desk lamp, I was thinking about putting in an LED light behind the PC monitor.

Then I remembered I’d bought a ring of 24 RGB LEDs from Aliexpress last year & hadn’t used it in a project.

I also had a spare Arduino Nano, and all the things I’d need to allow me to hook up a dial (potentiometer) for the light level, and button to cycle through different colour modes.

Here’s a quick video of it in action..

Parts

LED Ring – £2
Arduino Nano – £1.75
Breadboard & bits – £1.25

Wiring it up

It’s an easy one to wire up.. I took a few basic examples and mashed them together to get what I wanted from the design.

I’m not an electronics expert, and approached this like I approach software development; write it in manageable/testable chunks, which I can implement and test individually, then bolt it all together.

fritz-led-ring

The code for the project is pretty simple.. I think the most complicated bit is handling the button, which needed debounce functionality.

And here are a few pictures of it in place behind the PC monitor..

Samsung Q9FN Tips, Tricks, Secrets & Problems

The 2018 flagship TV from Samsung is the Q9FN (Amazon link: https://amzn.to/2IvwmkO). This is a FALD (Full Array Local Dimming) display with 480 LEDs lighting the display, rather than being edge-lit like a lot of the models. This generally means it’s much more capable of providing good contrast ratios. Overall I think it’s a good TV … most of the time….

Problems

However, the reality is different from the headlines and reviews in the major publications. What a lot of people have found is that the Contract Enhancement & Local Dimming features can cause light fluctuation problems, which are especially apparent in dark scenes with subtitles.

Light Fluctuations

You can see for yourself in this clip from Narcos Mexico S01E05 at about 45 minutes. This is being viewed via the built-in Netflix player in HDR, with Contrast Enhancement turned on, since without it enabled, dark scenes are waaay too dark to see anything!

It shows how subtitles affect the light levels in other areas of the screen.. like right at the top, nowhere near the subtitles.

Backlight Flicker

This shows how I’m seeing a flicker certain scenes. It’s like the TV can’t quite decide on the light level it’s supposed to display, and clicks into place. I’ve run this at standard speed, then slowed it right down to illustrate the flicker. You’ll need to look closely at the background & look for the light fluctuation.

FALD Confusion!

There are also instances when FALD gets in the way of drawing a background with a solid colour. This video shows a short excerpt from the film Searching (2018).. at about 52 min 30 sec. The FALD back light system has real trouble working out what to do with the dark blue satnav background which should be a solid/uniform colour.. but the bright white roads cause it a lot of problems. This could be a disadvantage of FALD over edge-lit or OLED. At least that’s the way it seems. I’m not entirely sure whether you could even solve this in software.

The blue lights/dots on the left are a reflection of the Xmas tree lights.. so nothing to do with the TV ;)

Tips & Tricks

Okay, enough with the problems, and onto the tips!

Secret Buttons!

At first glance the TV has no physical buttons to control it.. so if you’ve misplaced the remote, it looks like there’s nothing you can do. However, take a look under frame near the logo and there’s a neat directional control + OK button.

Steam Link for Free!

Instead of buying a Steam Link device, there’s actually a free app that lets you stream games from your Gaming PC to your TV. Install the app & plug your controller into the TV and you’re pretty much good to go!

Removing Adverts(!!) from the Menus

Yes, adverts.. in the menus.. on a brand new TV that you paid a lot of money for!

This video shows how I’ve been able to get rid of the movie trailers which annoyingly play so easily. They’re coming from apps like TV Plus & Rakuten. It’s not that easy to figure out how to get rid of them.. you certainly can’t uninstall the apps.. Samsung don’t let you do that :(

I found ads in the menu too.. if you go back into the policy agreements and make sure you’ve not ticked any of the options to agree to them, the ads should go away.

Here’s a link to the Q9FN on Amazon; https://amzn.to/2IvwmkO

Installing SABnzbd on a Raspberry Pi running OSMC

For quite some time I’ve been running SABnzbd on a PC, downloading files, and then transferring them over the local network to a USB drive attached to a Raspberry Pi which is running OSMC. There’s a Linux version of SABnzbd which means I can cut out the PC and have the Pi handle the downloads. It’ll mean I can queue up the downloads from a web interface running on whatever device I have to hand, like an iPad.

First Try

The initial installation of SABnzbd was quite easy;

sudo apt-get install python-openssl unrar par2

sudo apt-get install sabnzbdplus

Edit the settings so that the web client starts up on port 8085..

sudo nano /etc/default/sabnzbdplus

USER=osmc
HOST=0.0.0.0
PORT=8085

sudo service sabnzbdplus restart

This then allowed me to connect to SABnzbd and transfer over all my settings that I was using on my PC.

Delayed Start

What I found was that SABnzbd started before the USB drive was properly mounted by OSMC, so I disabled the main service from starting up, and added a script to wait for the USB drive to get mounted at a particular path.

There was a good forum post here that pointed me in the right direction.

Disable the default service…

sudo update-rc.d sabnzbdplus disable

Write a quick shell script to wait for the directory/USB drive to be mounted…

nano /home/osmc/startsabnzb.sh

#!/bin/sh

# Wait for this folder to be mounted...
DIR=/media/Elements

while [ ! -d "$DIR" ]; do
sleep 120
done

/etc/init.d/sabnzbdplus start

chmod a+x /home/osmc/startsabnzb

Add the script to system startup…

sudo nano /etc/rc.local

/home/osmc/startsabnzb.sh

Upgrading

The version of SABnzbd that installed above was very dated. That repo doesn’t get updated very often. Here’s how I updated it to the latest version.

sudo su root

echo "deb http://ppa.launchpad.net/jcfp/nobetas/ubuntu xenial main" | tee -a /etc/apt/sources.list
echo "deb http://ppa.launchpad.net/jcfp/sab-addons/ubuntu xenial main" | tee -a /etc/apt/sources.list

apt-key adv --keyserver hkp://pool.sks-keyservers.net:11371 --recv-keys 0x98703123E0F52B2BE16D586EF13930B14BB9F05F

sudo apt-get update

Upgrading sabyenc

This solved the issue where SABnzbd was complaining that sabyenc wasn’t the right version. It uses the 2nd repo (sab-addons) we added in the steps above.

sudo apt-get install python-sabyenc

Final thoughts

SABnzbd runs quite well on the Pi. It is a lot slower than it was on a PC.. it only manages about 3 MB/s on the download on a wired connection (compared to 6 MB/s on a Wifi connection on a laptop), and unpacking is slow.

However, the files are unpacked onto the device which I was manually copying the files to anyway, so that saves time.

Samsung Galaxy S9 Super Slow-Mo Videos

The Samsung Galaxy S9 is a bit of a bargain if you look at the International versions. When it retailed for about £700 in the UK, you could import it from Italy for about £500! It’s available now on Amazon for £480.

When I bought the S9 I didn’t realise it had a special video mode called Super Slow-Mo. I’ve had slow mo on a Canon Ixus camera before, but it was poor resolution and needed a lot of light to work.

The S9 shoots super slow-mo at 960 frames per second, and captures 0.2 seconds of motion, stretching it to 6 seconds when you play it back.

One big difference with the S9 is how you trigger super slow-mo. When its set to automatic, a yellow square appears over the video preview.. when you start recording, the S9 looks for motion within the square, and when it sees movement it’ll trigger super slow-mo.

For me, the automatic mode makes it more than a gimmick .. you can get some great videos out of it without much effort. Here are a few (non-professional) examples that I’ve taken over the past few months;

 

 

 

 

 

 

These were quite simple to capture.. just practice a bit to see how it triggers and you can get good results.

Sharing attached USB storage in OSMC using NFS

As well as being attached to the living room TV for use as a media centre, I also wanted to be able to use my Raspberry Pi 3 B+ as a simple NAS for other TVs in the house to stream from.

The Pi I’m using has a 1Tb desktop hard drive attached to it over USB, and I wanted a way to easily share the contents. It was actually relatively easy to set up… this is how to do it in OSMC;

  1. Install SSH to OSMC via the Store
  2. Now you can remote shell into the Pi to set up the network share
  3. Install NFS services using the following command;
    sudo apt-get install nfs-kernel-server
  4. Edit the file shares;
    sudo nano /etc/exports

    Add a share like this;

    /media 192.168.1.0/255.255.255.0(rw,fsid=0,insecure,no_subtree_check,async,crossmnt)

    (crossmnt fixed an issue where I could see the folders but no files)

  5. Restart the NFS service;
    sudo /etc/init.d/nfs-kernel-server restart

That’s it.. you should now be able connect to the Raspberry Pi and see the files on any of the USB drives you’ve got attached.

Automated mains socket power-off for OSMC on a Raspberry Pi

I’ve chosen to replace an ageing mini-PC which I’ve used since 2010 with a new Raspberry Pi 3 B+ running OSMC. It makes for a really capable media centre which can playback newer h.265 HEVC video files at 1080p without any problems, or it can serve 4K files over NFS to a box with a hardware h.265 chip like the Fire TV Box.

This form factor is easy to take on holiday and you can use an old infrared remote control (or Harmony learning remote) with it too.

However, the one thing I’ve struggled with is how to make it easy for my family to use in regard to switching it on and off. The Pi doesn’t have a power button. Some power supplies have an inline rocker switch, which almost fits the bill. I wanted something more automated.

Fortunately I had a spare Energenie power socket from a previous project where I use one to turn off our bass speaker when the TV isn’t on. These power sockets are controlled remotely (over RF) from a Pi which you attach an Energenie control board/shield to.

What I’ve done with the Pi 3 is have it powered through an Energenie socket, and set up a service that executes when it detects OSMC is shutting down. That service will make a quick HTTP call to the Pi with the Energenie controller shield, which will in turn send an RF signal to turn the mains socket off.

 

osmcshutdown

 

Here’s how you can set it up like I have…

Scripts for the Pi running OSMC

First, add a new service script.. create a new file in this folder;

/etc/systemd/system/callenergenie.service

Then enable it with;

sudo systemctl enable callenergenie.service

There are a couple of useful things happening in this service, the After parameter makes sure the code runs before the network code is shut down, and Conflicts parameter is looking for OSMC shutting down.

Now add a helper script… this will make the webserver call as a background task, so control will be given back to the service immediately, rather than it waiting for the wget to complete.

/home/osmc/callenergenie.sh

This calls the PHP script, telling it which socket to turn off, and how long to delay before sending the command, which we’re doing so that the Pi has time to shut down before the power is cut.

Scripts for the Pi with the Energenie shield

This is the PHP script I added to the other Pi which was already configured to be a PHP web server.

/var/www/html/callenergenie.php

To allow PHP to run the script as root, I needed to add the Apache user to the list of sudo-ers.. not that secure tho :( I’d be interested in anyone who knows how to run the Energenie scripts a regular user.. their Python doesn’t like it when it’s not root.

nano /etc/sudoers

www-data ALL=(ALL) NOPASSWD:ALL

The nice thing about the PHP script is that we can actually call it to turn the Pi on remotely too.. so you could configure that into a widget on your phone, or add it to Alexa.