Thatcham Trains

This is the final article in my brief series on the National Rail API. As usual, the code can be found on github:

The Idea

There are a million and one different websites and apps which will tell you the next direct train from London Paddington to Thatcham (or between any other two railway stations) but all those apps are very general. You have to struggle through the crowds on the Circle Line while selecting the stations from drop-downs and clicking “Submit”, for example. Wouldn’t it be good if there was a simple way to see the information you need without any user input? Even better, what if you could get notifications when the direct trains are delayed or cancelled?

Enter stage left, the Twitter API. This article is all about a simple mash-up of the National Rail and twitter APIs to show information on direct trains between London and Thatcham. You can use it for other stations too – it’s all in the command line parameters.

People who live in Thatcham can use my twitter feed @ThatchamTrains or you can set up your own feed and run the python script to populate it with the stations you’re interested in.

The script also sends direct messages if the trains are more than 15 minutes late or cancelled.

Using the script

I host my instance of the script on my raspberry pi, which is small, cheap, quiet and can be left on 24×7 without much hassle. These instructions are therefore specific to setup on the pi, but the script will work on Windows and other version of Linux too.

1. Install the python libraries you need. You may already have these installed.

$ sudo easy_install argparse
$ sudo easy_install requests
$ sudo easy_install xmltodict
$ sudo easy_install flask

2. Get a twitter account and a set of API keys by following the steps on the Twitter developers page. You’ll need four magic strings in total, which you pass to the script as command line parameters.

3. Get a national rail API key from their website. You just need one key for this API, which is nice!

4. Clone the source and run the script using the three commands below… simples!

$ git clone
$ cd national-rail
$ python --rail-key YOUR_NATIONAL_RAIL_KEY --consumer-key YOUR_CUST_KEY --consumer-secret YOUR_CUST_SECRET --access-token YOUR_ACCESS_TOKEN --access-token-secret YOUR_ACCESS_TOKEN_SECRET --users YourTwitterName --forever

When run with the –forever option, the script will query the NR API and post to twitter every 5 minutes. Note that there are some basic checks to prevent annoying behaviour and duplicate messages. You can specify one or more usernames who you’d like to receive direct messages when there are delays and cancellations; note that only users who follow you can receive DMs on twitter.

You can use other stations by specifying the three character station codes (CRS) for “home” and “work” on the command line. Here are the command line options:

$ python --help

usage: [-h] [--home HOME] [--work WORK] [--users USERS]
                      [--forever] --rail-key RAIL_KEY --consumer-key
                      CONSUMER_KEY --consumer-secret CONSUMER_SECRET
                      --access-token ACCESS_TOKEN --access-token-secret

Tweeting about railways

optional arguments:
  -h, --help            show this help message and exit
  --home HOME           Home station CRS (default "THA")
  --work WORK           Work station CRS (default "PAD")
  --users USERS         Users to DM (comma separated)
  --forever             Use this switch to run the script forever (once ever 5 mins)
  --rail-key RAIL_KEY   API Key for National Rail
  --consumer-key CONSUMER_KEY
                        Consumer Key for Twitter
  --consumer-secret CONSUMER_SECRET
                        Consumer Secret for Twitter
  --access-token ACCESS_TOKEN
                        Access Token for Twitter
  --access-token-secret ACCESS_TOKEN_SECRET
                        Access Token Secret for Twitter

The Code

There’s not much to say about the code, since I’ve covered the National Rail API in graphic detail in a previous article. The only real difference between this script and my previous adventures with the API is that this time I did unit testing.

There’s a fair bit of business logic in the twitter app: rules about when to post and when to be quiet, duplicate message detection and all sorts of time- and data-based rules which can’t be tested using real data. It’s also pretty bad form to test code like this against a live API, so I mocked out the NR query and the Twitter API and wrote a small suite of tests to check that the behaviour is right.

Like I said, all the code is on GitHub, so I won’t bang on about it here.

Live Train Route Animation

The code for this article is available on my github, here:

Building on the Live Departures Board project from the other day, I decided to try out mapping some departure data. The other article shows pretty much all the back-end code, which wasn’t changed much.


The AngularJS app takes the routes of imminent departures from various stations and displays them on a Leaflet map as polylines. I used this great Snake library to animate the lines as they appear. Map tiles come from CartoDB, which is free, unlike Mapbox.


Here’s the code-behind for the Angular app:

var mapApp = angular.module('mapApp', ['ngRoute']);

		    	controller: 'MapController',
			    templateUrl: 'map.html'
		    .otherwise({redirectTo: '/'});
	.controller('MapController', function($scope, $http, $timeout, $routeParams) {

        var mymap ='mapid').fitBounds([ [51.3933180851, -1.24174419711], [51.5154681995, -0.174688620494] ]);
        L.tileLayer('http://{s}{z}/{x}/{y}.png', {
            attribution: '© <a href="">OpenStreetMap</a> © <a href="">CartoDB</a>',
            subdomains: 'abcd',
            maxZoom: 19

        $scope.routeLayer = L.featureGroup().addTo(mymap);
        $scope.categoryScale = d3.scale.category10();

        $scope.doStation = function(data) {
                var color = $scope.categoryScale(route[0].crs)
                var path = []

                route.filter(function(x) {return x.latitude && x.longitude}).forEach(function(station) {
                    var location = [station.latitude, station.longitude];

                var line = L.polyline(path, {
                    weight: 4,
                    color: color,
                    opacity: 0.5


        $scope.refresh = function() {

            $scope.crsList.forEach(function(crs) {
                $http.get("/routes/" + crs).success($scope.doStation);

            }, 10000)

        $http.get("/loaded-crs").success(function(crsData) {
            $scope.crsList = crsData;


Train Departure Board

You can find the code for this article on my github:

Having found myself time-wealthy for a couple of weeks I’ve been playing around with some open data sets. One of which is the National Rail SOAP API. It’s not a new dataset, I think it’s been around for a decade or so, but it seemed like a sensible thing for me to play with as I’ll be on trains a lot more when I start selling my time to a new employer next month!

I live about a mile away from the local station (Thatcham) so it only takes a few minutes to get there. If a train is delayed or cancelled I’d like to know so I can have another coffee. So what I need is a live departures board, for my local station, in my house. Something like this:


The UI is web based – using AngularJS again. Sadly though, the cross origin nonsense means I can’t make the soap calls directly from the web client, I need a back-end to gather and store the data for use on the UI. I used Python for this because that gives me the option to (easily) run it on a Raspberry Pi, reducing power and space costs as well as noise. Python’s library support is stunning, and this is another great reason to use it for small “hacks” like this one.

SOAPing Up

SOAP is horrible. It’s old, it’s heavy, it’s complex and worst of all it’s XML based. This isn’t a huge handicap though, as we can hit a SOAP service using the requests HTTP library – simply sending a POST with some XML like so:

import requests
import xmltodict

xml_payload = """<?xml version="1.0"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="" xmlns:ns1="" xmlns:ns2="">

# url: The URL of the service
# key: Your National Rail API key
# crs: Station code (e.g. THA or PAD)
def fetch_trains(url, key, crs):
    headers = {'content-type': 'text/xml'}
    payload = xml_payload.replace("{KEY}", key).replace("{CRS}", crs)
    response =, data=payload, headers=headers)

    data = xmltodict.parse(response.content)
    services = data["soap:Envelope"]["soap:Body"]["GetDepBoardWithDetailsResponse"]["GetStationBoardResult"]["lt5:trainServices"]["lt5:service"]

    for service in services:
        raw_points = service["lt5:subsequentCallingPoints"]["lt4:callingPointList"]["lt4:callingPoint"]

        calling_points = map(lambda point: {
            "crs": point["lt4:crs"],
            "name": point["lt4:locationName"],
            "st": point.get("lt4:st", "-"),
            "et": point.get("lt4:et", "-")
        }, raw_points)

        cp_string = "|".join(
                map(lambda p: "{0},{1},{2},{3}".format(p["crs"], p["name"], p["st"], p["et"]), calling_points)

        yield {
            "crs": crs,
            "origin": service["lt5:origin"]["lt4:location"]["lt4:locationName"],
            "destination": service["lt5:destination"]["lt4:location"]["lt4:locationName"],
            "std": service.get("lt4:std"),
            "etd": service.get("lt4:etd"),
            "platform": service.get("lt4:platform", "-"),
            "calling_points": cp_string

So, I take a pre-formatted XML request, add the key and station code then POST it to the API URL. Easy. The result comes back in XML which can be parsed very easily using the xmltodict library. I used Postman to test the calls before translating to Python (and if you’re lazy, Postman will even write the code for you!).

The Python script takes the data its gathered and stores it in an SQLite database. I’m not going to show the code because it’s all in github anyway.

Having a REST

So the data is all in a DB, now it needs to be made available to the Javascript client somehow. To do this I created a simple REST service using the excellent Flask library for Python. It’s in the same genre as Sinatra, Nancy, Scalatra and all the other microservice frameworks I love to use. Here’s all the code you need to serve up data via REST:

import argparse
import sqlite3
from flask import Flask, jsonify

app = Flask(__name__)

def departures():
    return jsonify(fetch_departures("departures"))

def departures_for(crs):
    return jsonify(fetch_departures("departures", crs))

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='National Rail Data REST Server')
    parser.add_argument('--db', help='SQLite DB Name', default="../data/trains.db")
    args = parser.parse_args()
    db = args.db

Front End

The front end is a very simple Angular JS app. Not much point showing the code here (see github) it’s about as simple as it gets – using a dot matrix font I found via Google and a theme from Bootswatch.

The design is based on a real life station departures board like this:

All in all the project took me a little over a day. A leisurely day with many interruptions from my daughters! Feel free to pull the code down and play with it – let me know what you do.

2016-06-14 11.32.01


A few weeks ago I made a smoker out of an exhaust pipe and two old party-balloon helium cylinders. Since then I have smoked many a brisket! Check out the pics.

Quick TeamCity Build Status with AngularJS

So, this isn’t supposed to be the ultimate guide to AngularJS or anything like that – I’m not even using the latest version – this is just some notes on my return to The World of the View Model after a couple of years away from WPF. Yeah, that’s right, I just said WPF while talking about Javascript development. They may be different technologies from different eras: one may be the last hurrah of bloated fat-client development and the other may be the latest and greatest addition to the achingly-cool, tie dyed hemp tool belt of the Single Page App hipster, but under the hood they’re very very similar. Put that in your e-pipe and vape it, designer-bearded UX developers!


Anyway, when I started, I knew nothing about SPA development. I’d last done JavaScript several years ago and never really used it as a real language. I still contend that JavaScript isn’t a real language (give me Scala or C# any day of the week) but you can’t ignore the fact that this is how user interfaces are developed these days… so, yeah, I started with a tutorial on YouTube.

I decided to do an Information Radiator to show build status from TeamCity on the web. Information Radiators are my passion – at least they’re one of the few passions I’m allowed to pursue at work – and we use Team City for all our continuous integration, release builds, automated tests and so on. Our old radiators are coded in WPF, which looks awesome on the big TVs dotted around the office, but doesn’t translate well for remote workers.

There is no sunshine and there are no rainbows in this article. I found javascript to be a hateful language, filled with boilerplate and confusion. Likewise, though TeamCity is doubtless the best enterprise CI platform on planet earth, the REST APIs are pretty painful to consume. With that in mind, let’s get into the weeds and see how this thing works…

Enable cross-site scripting (CORS) on your Team City server

You can’t hit a server from a web page unless that server is the server that served the web page you’re hitting the server with… unless of course you tell the server you want to hit that the web page you want to hit it with, served from a different server, is allowed to hit it. Got that? Thought so. This is all because of a really logical thing called “Cross Origin Resource Sharing”, which you can enable pretty easily in TeamCity as long as you have admin permissions.

Check out Administration -> Server Administration -> Diagnostics -> Internal Properties. From there you should be able to edit, or at least get the location of the file. Weirdly, if the file doesn’t exist, there is no option to edit, so you have to go and create the file. Since my TeamCity server is running on a Windows box, I created the new file here:


and added the following:*

You might want to be a little more selective on who you allow to access the server this way – I guess it depends on how secure your network is, how many clients access the dashboard and so on.

Tool Chain

This article is about AngularJS and it’s about TeamCity. It’s not about NPM or Bower or any of that nonsense. I’m not going to minify my code or use to crazy new-fangled pseudo-cosmic CSS. So setting up the build environment for me was pretty easy: create a folder, add a file called “index.html”, fire up the fantastic Fenix Web Server and configure it to serve up the folder we just created. Awesome.

If you’re already confused, or if you just want to play with the code, you can download the lot from GitHib:

I promise to do my best

Hopefully you’ve watched the video I linked above, so you know the basics of an AngularJS app. If not, do so now. Then maybe Google around the subject of promises and http requests in AngularJS. Done that? OK, good.

Web requests take a while to run. In a normal app you might fetch them on another thread but not in JavaScript. JavaScript is all about callbacks. A Promise is basically a callback that promises to get called some time in the future. They are actually pretty cool, and they form the spinal column of the build status app. This is because the TeamCity API is so annoying. Let me explain why. In order to find out the status (OK or broken) and state (running, finished) of each build configuration you need to make roughly six trillion HTTP requests as follows:

  1. Fetch a list of the build configurations in the system. These are called “Build Types” in the API and have properties like “name”, “project” and “id”
  2. For each Build Type, make a REST request to get information on the latest running Build with a matching type ID. This will give you the “name”, “id” and “status” of the last finished build for the given Build Type.
  3. Fetch a list of the currently running builds.
  4. Use the list of finished builds and the list of running builds to create a set of status tiles (more on this later)
  5. Add the tiles to the angular $scope and let the UI render them

Here’s how that looks in code. Hopefully not too much more complicated than above!

	.then(function(responses) {
		$scope.buildResponses = responses
			.filter(function(r) { return (r.status == 200 && > 0)})
			.map(function(r){ return[0] })
	.then(function(data) {
		$scope.runningBuilds = { return row.buildTypeId })
	.then(function() {
		$scope.builds = $ { return buildFactory.decodeBuild(b, $scope.runningBuilds); });
	.then(function() {
		$scope.tiles = buildFactory.generateTiles($scope.builds)
	.then(function() {
		$scope.statusVisible = false;

Most of the REST access has been squirrelled away into a factory. And yes, our build server is called “tc” and guest access is allowed to the REST APIs and I have enabled CORS too… because sometimes productivity is more important than security!

angular.module('buildApp').factory('buildFactory', function($http) {
	var factory = {};
	var getBuildTypes = function() {
		return $http.get('http://tc/guestAuth/app/rest/buildTypes?locator=start:0,count:100');
	var getBuildStatus = function(id) {
		return $http.get('http://tc/guestAuth/app/rest/builds?locator=buildType:' + id + ',start:0,count:1&fields=build(id,status,state,buildType(name,id,projectName))');
	factory.getRunningBuilds = function() {
		return $http.get('http://tc/guestAuth/app/rest/builds?locator=running:true');

// etc

Grouping and Tiles

We have over 100 builds. Good teams have lots of builds. Not too many, just lots. Every product (basically every team) has CI builds, release/packaging builds, continuous deployment builds, continuous test builds, metrics builds… we have a lot of builds. Builds are good.

But a screen with 100+ builds on it means very little. This is an information radiator, not a formal report. So, I use a simple (but messy) algorithm to convert a big list of Builds into a smaller list of Tiles:

  1. Take the broken builds (hopefully not many) and turn each one into a Tile
  2. Take the successful builds and group them by “project” (basically the category, which is basically the team or product name)
  3. Turn each group of successful builds into a Tile, using the “project” as the tile name
  4. Mark any “running” build with a flag so we can give feedback in the UI


Displaying It

Not much very exciting here. I used Bootstrap, well, a derivative of Bootstrap to make the UI look nice. I bound some content to the View Model and that’s about it. Download the code and have a look if you like.

Here’s my index.html (which shows all the libraries I used):

<html ng-app="buildApp">
  <title>Build Status</title>
  <link href="" rel="stylesheet">
  <!--link href="" rel="stylesheet"-->

  <div ng-view>

  <script src=""></script>
  <script src=""></script>
  <script src=""></script>
  <script src=""></script>
  <script src="utils.js"></script>
  <script src="app.js"></script>
  <script src="build-factory.js"></script>

Here’s the “view” HTML for the list (in “templates/list.html”). I love the Angular way of specifying Views and Controllers by the way. Note the cool animated CSS for the “in progress” icon.

	.glyphicon-refresh-animate {
		-animation: spin 1s infinite linear;
		-webkit-animation: spin2 1s infinite linear;

	@-webkit-keyframes spin2 {
		from { -webkit-transform: rotate(0deg);}
		to { -webkit-transform: rotate(360deg);}

	@keyframes spin {
		from { transform: scale(1) rotate(0deg);}
		to { transform: scale(1) rotate(360deg);}
	<div class="page-header">
		<h1>Build Status <small>from TeamCity</small></h1>
    <div class="container-fluid">
		<div class="row">
    		<div class="col-md-3" ng-repeat="tile in tiles | orderBy:'status' | filter:nameFilter">
        		<div ng-class="getPanelClass(tile)">
               <h5><span ng-class="getGlyphClass(tile)" aria-hidden="true"></span>   {{ | limitTo:32 }}{{ > 32 ? '...' : ''}}   {{ tile.buildCount > 0 ? '(' + tile.buildCount + ')' : ''}} </h5>
               <p class="panel-body">{{ tile.project }}</p>
  <nav class="navbar navbar-default navbar-fixed-bottom">
  <div class="container-fluid">
    <p class="navbar-text navbar-left">
		<input type="text" ng-model="nameFilter"/>  <span class="glyphicon glyphicon-filter" aria-hidden="true"></span>  
		<span class="glyphicon glyphicon-refresh glyphicon-refresh-animate" ng-hide="!statusVisible"></span>

That’s about it!

I think I summarized how I feel about this project in the introduction. It looks cool and the MVC MVVM ViewModel vibe is a good one. The data binding is simple and works very well. All my gripes are with JavaScript as a language really. I want Linq-style methods and I want classes and objects with sensible scope. I want less syntactic nonsense, maybe the odd => every now and again. I think some or all of that is possible with libraries and new language specs… but I want it without any effort!

One thing I will say: that whole page is less than 300 lines of code. That’s pretty darned cool.

Feel free to download and use the app however you like – just bung in a link to this page!


Super Quick Mini Quad

Super quick to build and super quick buzzing round the garden! Inspired by a recent video series by RC Model Reviews on YouTube I bought some bits and bobs from Hobby King to build my very own mini-quad.

2015-04-19 15.35.03

It’s been ages since I did anything in the garage, other than stir tins of Deep Bronze Green and repair punctures, so I was totally unable to wait for the second of my two parcels to arrive from the People’s Republic. Since I’d received everything except the frame within a couple of days and faced a two week wait for the frame itself, I knocked up a little quad frame from spare bits of carbon fibre and two meagre scraps of 6mm aluminium.

2015-04-19 15.35.11

It’s much smaller than my old quads. It was supposed to be a 250-class – so 250mm from motor to motor – but I chucked it all together free from the shackles of forethought and it ended up a little wide at the rear end. I drilled out, parted off and tapped four aluminium pillars, chopped and drilled some chunks of carbon fibre sheet and reinforced the motor arms with some square carbon tube with a balsa dowel insert.

One of the best things I did is make a power distribution board from a bit of veroboard. This makes wiring things up so much easier than faffing about trying to make a wiring loom. It also keeps the inside of the quad much tidier. I mounted it under the flight controller on some plastic standoffs which works very well.

2015-04-19 15.35.20

To fly it’s blooming amazing. I’ve only flown it round the (tiny) garden so far, watched by a curious three year old. The Naze32 board keeps things incredibly stable and the light frame and beefy little motors means it easily hovers at mid-stick. Leaving (hopefully) spare capacity for some FPV kit and a GoPro in future.

Huge thanks to Bruce from RC Model Reviews for making such a useful video series – and especially for recommending a list of cheap but super-effective components available for about Β£100.

The Chuffer

I just completed my second ever air engine: The Chuffer. With this project I decided that I’d go for complexity, rather than the simplistic approach I took when I built The Wobbler earlier this year.

The Chuffer is a twin-cylinder double acting engine – which means each of its two cylinders both push and pull the piston. This means it operates much more smoothly and can self-start as there’s always a piston on a power stroke, regardless of where the flywheel is.

Unlike its predecessor, the Chuffer has fixed cylinders and airflow is controlled by two spool valves, operated by eccentrics next to the flywheel via some pushrods and a hopelessly over-engineered lever mechanism.

And of course everything was made by yours truly, in the garage, from chunks of brass, steel and aluminium bar and flat stock I bought on eBay. Needless to say, I have learned a lot since I started the project this February!

What’s next? Well, maybe a Watt Governor or a little generator