Thursday 24 February 2011

Poor Data Quality a Crime

Posted - Bob Chell:  As we decided to launch a blog for 1Spatial Consultants, there is no better current example that brings focus on data to the people’s minds than the Police Crime Maps.  My honest first thoughts, when I heard the site had been down due to demand, echo’s that from Adrian Short’s blog - ‘too much traffic to our website is a problem we’d all like to have’.

As I got the train from London to Cambridge, I then saw that what we could also get hold of was the raw data itself – fantastic!  Then, just as we were arriving into Cambridge, I actually started to read numerous blogs about issues people actually had with it all. All these have already been written about and I found informative summaries from Steven Feldman and the previous mentioned post from Craig Short to cover everything.

My attention is always drawn to comments about ‘understanding about the data’. One of the first articles I came across was on the Guardian website, which contained an eye-opening ranked list of a streets and crimes.  Interestingly, Cambridge had a hot spot, Peas Hill, in at number eight. Working and living in Cambridge means I already have context about this place - I know this area. And I also know that Jamie Oliver recently opened Jamie’s Italian which is right in the same place, right on the corner of this street - and it’s not a particularly long street! I’m pretty sure that Jamie Oliver was not advised to open a high profile restaurant in a crime hot spot. So I became a little more curious about this information. What rules were there behind its creation?

Having got a basic glimpse of the data, I needed to investigate further since the raw data themselves without context have limited value. So Step 1 - discover and learn more about this data’s provenance and governance. Without this type of understanding, you won’t really be able to make an informed decision on whether the data is suitable for you or your business. Is this suitable for you the mission that you and your company are on?

I share many people’s views that this is huge improvement on what we had before. I’m also slightly biased, having worked with the likes of British Transport Police, and empathise with the effort that they are putting into making continued improvements in managing the vast quantities of information that keeps passing by their desks.

The data are now open. However we are simply at the beginning. First we need a baseline to work out a where we are with the quality of the data, then we can start to determine how to improve it to make it fit for purpose, give it context. There appears to be valid thought-processes or rules that have gone on to justify why the information is as it is. So all this needs to be put into context, so that the issues and understanding around inaccuracies are not what these transparency initiatives are remembered for.
The data are available, so the government is hoping that others can easily look to take a different approach to using it and analysing and presenting it in novel ways. I like the ITO world updates to Open Street Map as Berners-Lee showed at TED2010.

There are other examples already appearing, I’m sure you can find them through the search engines. One thing is for sure though, the now monthly snapshots of crime figures for England and Wales should plug straight into all sorts of Business Intelligence tools and I’m looking forward to how this grows up. I might look at Oracle Business Intelligence Enterprise Edition (OBIEE) and others in the Consulting group will no doubt look at BI tools of their own preference. Keep an eye out for future posts.

No comments:

Post a Comment