Reliable data

Recently we have seen BDUK announce the funding allocations to local authorities and the devolved assemblies, and the companies aiming hoping to get on the national framework have been short-listed. The sums awarded to councils were modelled by BDUK according to their understanding of need, and at the moment the framework companies are trying to develop a consistent understanding of what will be required of them and their shareholders should they be successful.

At stake is the investment of billions of pounds and public and private money, and the future competitiveness of the UK economy. Yet questions have been raise in several quarters for quite some time now about the accuracy of BDUKs data on which all this investment sits. So for the record I decided to correlate a source of data I have grown to trust – from Samknows who in turn get their DSL data from BT – against a set of BDUK data for the same area. The sample included a little over 19,000 postcodes.

BDUK Broadband Speed data

(click the graph to see a bigger version)

The plot shows BDUK speeds along the horizontal with BT speeds on the vertical, with each point representing a postcode average. If the two sets agreed the points should broadly align along the diagonal but its clear there is a limited correlation between the two sets.

This data is for Oxfordshire, so the first location I checked was my own postcode. BDUK suggests that I should get 13971.456kbps while BT suggests I get 6Mbps with ADSL2+. With an ordinary ISP I do in fact get 6 Mbps (Be There uniquely allow me to tune the connection so I get a shade more).

In fact on 76% of occasions the BDUK data offers faster speeds than BT’s reported data, and on average 52% faster.

When focussing in on just the 2 Mbps Universal Service Commitment, relying on BDUK data would result in about 900 postcodes having a problem addressed which doesn’t exist, yet almost 40% of the areas which do suffer broadband at less than 2 Mbps would have been missed altogether.

In 63 cases the discrepancy was more than 22 Mbps – or rather BDUK expected people to receive what they now consider “superfast broadband” when in fact no broadband was available at all.

From what I understand none of the usual sources relied on by the industry provided BDUK with this data and that the speeds are reported to thousandths of a kilobit suggests Excel may have been involved somewhere along the line rather than empirical data.

This information was provided to BDUK but they were largely unconcerned about the discrepancy at the time.

I’ll allow you to come to your own conclusions about the impact this might have had on the decisions BDUK is making and the fairness of funding allocations. For organisations seeking to be part of the framework, this data appears to be having a continuing impact.

NOTE: This is one of a number of blog articles which had gone unpublished for some time, occasionally dusted off and updated but left on the spike. For much of BDUK’s existence I have been supportive, and after it became clear that they were ignoring offers of help and advice from many of the people I know I had remained reluctant to be openly dismissive. But as the programme evolved it has become harder and harder to be supportive, there became fewer and fewer good news stories to write about, and my own postings became less frequent and rarely positive good news stories.

I’m publishing this now to draw a line under the whole process – time to get on with projects that make a difference in reality.

People, politics and technology

MIT recently published a fascinating piece of research, looking at how social interactions can help to define regions based on a massive sample of land-line telephone calls across Great Britain . They used the anonymised information on the 12 billions phone calls made in Britain in a typical month to see if there were any patterns which could describe natural regions based on human interactions. The results are quire extraordinary!

A quick glance at the resulting maps will tell you that Scots only really talk to other Scots (left-most map below), similarly Londoners but to a lesser extent. The rest of the country tells a rather different story, challenging some of our traditional assumptions about regional identity.

Three iterations of partitioning

The right-hand map shows an optimised partitioning of the call data, showing that, for example, Welsh people fall into three regions, only one of  which is solely in Wales. North Wales communicates most strongly with Manchester and the southern part of what we normally associate with the North West; while Mid-Wales links most naturally with the West Midlands. A fair conclusion from this is that if the UK were to be fully devolved, it would make little difference to the day to day communications of the Scottish population, but it would have a profound impact on the Welsh population.

The Yorkshire-Lancashire rivalry also takes a bit of a knock, with West Yorkshire more likely to communicate with the people of Lancashire than their White Rose brethren; and the more rural Cumbrians are perhaps a mini region of their own.

In checking the validity of their approach, the researches aggregated a number of alternative partitioning models, and this generated additional insight into regional identity.

NUTS regions overlaid on the aggregated partitioning models To the west of London the team identified what they consider to be a new region in the making – a Western Crescent formed of Oxfordshire, Berkshire and Buckinghamshire. This is the heart of England’s high-tech industries with the Silicon Corridor along the M4 and Oxfordshire’s Science Vale with Oxford University and Harwell. What it interesting is the ambiguity of the areas communication patterns – rather than having a very clear and arguably insular regional identity, this Western Crescent is something of a communications hub, reaching out to much of central England.

Why is this important?

The traditional regional boundaries being largely consigned to civic roles with political and economic control being passed to new Local Enterprise Partnerships. What these maps suggest is that the regional identities were already being challenged and that perhaps the more fluid LEP structure would be more able to mould itself to our day to day lives. While the South England region, which spanned Kent and Oxfordshire, meant very little to anyone except central Government, an Oxfordshire LEP able to partner with a Thames Valley LEP may be more successful.

And from my own personal perspective such an approach also means its possible to map telecommunications networks to human interactions. The formative signs of a new high-technology region around Oxfordshire sure deserves a commensurate broadband infrastructure? And its role as a natural communications hub surely makes it the place to start building the future? The research should also have a big impact on Cumbria’s Big Society “vanguard”, and ought to shape Herefordshire’s thinking as they develop their broadband pilot.

As I start to work with the new Oxfordshire LEP on their approach to broadband I’m sure this research will become something we refer back to.

Get Adobe Flash player