On Rail Franchises, Public Sector Pay and Modelling Failures

[danieldwilliam]

I’m going to talk a little bit about the rail franchise issue, public sector finance and public sector pay.

I don’t work for the Department for Transport and I’m not a hardcore modeller but I have spent a bit of time in the public sector finance teams and some time working with hardcore models. This is some educated guesswork.

I suspect that what’s happened with the rail franchises is a modelling failure.  When I use the word model I mean an excel spreadsheet which seeks to replicate the terms and conditions, prices and costs of a particular business or contract as a way on understanding how various decisions and events will affect the financial performance of that venture. The more complex the undertaking the larger and more complex the model. These are not simple spreadsheets.  They are about as far from back of the envelope as you can get.

The model I used to work with was for a power station.  The power station had four large contracts, each of which was over 100 pages long.  2 for power sales, 1 for gas purchases, 1 for maintenance of the gas turbines.  It also traded gas and power and carbon certificates in the open market and participated in a very technical power industry thing called the Balancing Mechanism. These all interacted with each other. As did the air and water temperature, rainfall, air pressure, water quality and a dozen or so other variables.

It even contained a model of what the contracts looked like from the other side, so we could guess what our counterparties would do.

Every business, contractual or operating decision was run through the model. It had to be able to answer any question asked of it.

The model ran to about 50 sheets on excel and weighed in at about 100 mb.

You put one number or one formula wrong in 1 cell out of several hundred thousand and your model will tell you lies.

I think the rail franchise models are going to be no less complex. The contracts involved are at least as big as a power station, perhaps an order of magnitude bigger.

In my experience working with models like this they are a full time job. You have one modeller working full time on one model.  If you are modelling one of these bad boys you live in it constantly.  It’s a very difficult job.  The skills and the attention to detail required are rare and hard to acquire. The introductory course in how to do it costs £2k. It takes months to build a model. It takes months to learn how to work with one.

Modellers at my old employer cost in the region of £70k a year, salary, pension, bonus and benefits.  A team of five of them were managed by a director level post who cost about £100,00. These are not cheap individuals to employ. However, compared to getting it wrong and costing your employer, say £40m, paying this money might seem cheap. In hindsight. (Which is somewhat defeats the point of a model.)

As I say, this sounds to me like a modelling failure. It sounds like either some duff information has been fed into the model, or the contracts involved haven’t been properly understood, or the  architecture of the model, the way it calculates different elements of the franchise agreements was built incorrectly. It could be that someone has just put a decimal point it the wrong place. These things happen. Everyone I know who has worked with models has made one howler and found someone else’s howler.

It may be the case that one error is to blame for the whole franchise problem. That the first model had an error buried deep in it and this was carried to each subsequent model and that the Department of Transport’s modelling team are entirely competent, just really, really unlucky.  Or it could be that the modelling team have been systematically under-staffed, with team members coming and going, staff not properly trained, or trying to handle too many models, with insufficient help, leadership and support. In which case, a howler was bound to happen, and probably had already and been ignored.

I don’t know for sure but I’m willing to bet that anyone who was good at modelling the rail franchise contracts for the Department of Transport was offered a modelling job somewhere else for more money. Probably, by one of the rail operators.

Here government are undertaking complex contractual arrangements which are difficult and therefore expensive to manage.  The public are rightly concerned that something has gone wrong.  I’d make this comment.

You can complain about the government’s ability to manage complex contracts and the costs of getting them wrong.  You can complain about public sector pay.  You can’t complain about both.

EDITED: to add

 

And it looks like the problem lies not in the modelling per se but in the assumptions going into the model.

 

http://www.bbc.co.uk/news/business-19816359

and a further Peston article on the capital requirements assumptions. via andrewducker
http://www.bbc.co.uk/news/business-19881240

Comments

[widgetfox.livejournal.com]

Yes, there could be several meanings of this, I think. I think it's possible to be "in the business of" something without actually being "in business" but it might be that in practice this distinction is too difficult to maintain.

[danieldwilliam.livejournal.com]

I think there is actually a useful distinction there.

Thinking back to my days as FD for an archaeology unit, we were certainly in the business of archaeology. We did archaeology. Money was an issue, both in that we needed money to do archaeology and that we often did archaeology in order to earn money. So we had to mindful of the financial implications and we also had to be mindful that we were dealing with people who were financially motivated in their transactions with us. There were deals to be done.

We arguably weren’t in business doing archaeology, the money wasn’t a significant motivating factor. We wouldn’t move into say, mine clearing, where our skill set might be useful to boost profits.

I would contrast this with, say, the bulk of the NHS, where they have a budget to be mindful of but most of the work doesn’t bump up against counter-parties and so on.