Big Data

Earlier in April I had the pleasure of delivering the closing remarks following the Big Data Hui hosted by Treasury. The hui highlighted a number of trends that will see a significant change in the way we gather information, share information and use it for decision-making.

We know that information to support decisions in community based social services is expanding rapidly. This is necessitating greater investment in information infrastructures and their application in decision support.

New Zealand Government has a vision to improve or increase access to the data it holds about individuals.   This is in line with the principle that data held by government on individuals should be made available to those individuals or their agents when requested. This will enable what is known about particular processes and populations and their issues to be drawn on in an immediately accessible form by providers. It is a way of ensuring that those delivering services have access to a comprehensive history, and that commitments for future engagement are monitored properly.

There are many issues to face, mostly seen as legal, cultural and structural, rather than technological. In every aspect, collaboration and partnering is recognised as the critical success factor to building a better functioning sector-wide social services system. However, partnering to promote change is also a big challenge.

There is a need to formalise, in some thoughtful way, how information sharing would be accepted by citizens. Those in the custody of information seek to collaborate, connect information and engage with them. As a country which has no written constitution, New Zealanders have a degree of pragmatism about the ability to evolve. This however may be challenged if changes in information sharing are made in an arbitrary manner.

The Privacy Act was initiated to enable rather than introduce a prescriptive approach. And the focus should be on protecting privacy, issues of governance of information, standards, new legal protections, a responsible authority, and transparency, as well as a right to opt out were the mainstays in this.

Evaluation data needs to be viewed more broadly than simply information to inform social investment decisions. Evaluations can provide valuable information for continuous improvement and that needed real time decision support. The public sector reforms of the 1980s narrowed expectations of departmental and contracted organizations so much that opportunity for innovation and raising service overall had been lost. This did not mean that we did not need good research based models to judge what we can change and foresee what might arise. But rather that they would add much more value in the context of sound service provision.

At the Hui, it was clear there was some tension between the public sector agency view of customer centric and that of community organisations. For government, integrated services cannot just be what the big agencies think people should get if they work together, but recognised that a strong commitment to continuous improvement and evaluation was needed for citizens to benefit.

Finally, I wanted to mention one of the presentations I found particularly aspirational. In the presentation by New Zealand Police, we saw what could be achieved when one well organised agency makes innovative use of technology. The real-time application of technology that supports decision-making is used alongside geographic information for strategic decisions about priorities. By taking a wide view of its place in society, New Zealand Police put citizens first, focusing on areas of greatest public concern.

Like all important initiatives that answer problems we have now, it raises new questions.

  • Can we make use of “Big Data” without deeper thinking?
  • If not, where will that deeper thinking come from?

4 Responses

    1. dixie neil

      data collection has so many varibles, trust an repore is not great, consistentcy in courts leaves alot to be desired,, victims rights are deminished when the perpetrator is an informant. It would be wise to put the abuser, perpetration into a data base with a record of people s/he has offended against an reported an convicted, possibly this would create a clear picture, of a person’s form


Comments are closed.