Data Valley. Credit: MIT Libraries Although we currently live in the "Information Age" what we actually generate, and most of the time fail to make meaningful use of, is data. The Australian Government's Productivity Commission has released a report that highlights the economic, health and social costs of Australia's failure to make effective use of the increasing data produced within Australia. Currently, it estimates that companies, governments and researchers are only making use of just 5% of the useful data that is currently available.
Getting access to data in order to carry out research and analysis is often a long and tortuous process.
In the health sector for example, a request for de-identified data about CT scans and cancer notifications took 5 years to reach researchers at the University of Melbourne. Medical guidelines about CT use in young people were changed after the researchers showed a relationship between CT scans and an increased risk of cancer. This link could have been identified much earlier, possibly preventing many young patients from being put at risk.
The Commission rightly makes the point that the ongoing use of data for the assessment of medical procedures and drug treatments in particular should be the norm, not the subject of occasional research.
There are a wide range of reasons why data is not being used more effectively. In the case of health data, there are concerns by hospitals and health authorities relating to the privacy and confidentiality of patient data. There is perhaps a more practical block however which is that sharing data, or making use of it for analysis, is not seen as a priority by many organisations and consequently, they are unwilling to devote much money to those activities. Certainly, they are even less willing to spend money on preparing data to share with other organisations, especially those that they regard as competitors.
Another issue is that CEOs and managers are often not highly numerate, and consequently don't see the value in the information and knowledge to be gained from data analytics.
What the Commission is proposing however is legislative changes to force the increased availability of data from the public and private sector. It also wants to give individuals ultimate control over what data is held about them and control over what organisations and government can do with that data. It pushes for a more radical culture of data use than is currently the case and will face an uphill struggle to change that culture.
One of the recommendations for example suggests that linked data, including statistical linkage keys, that is used for research should not have to be destroyed at the completion of the project. This was one of the issues raised through public concern about the Australian Bureau of Statistics retaining names and addresses from the 2016 census for a longer period. In fact, the Commission is arguing for more widespread use of identified data.
What this means is that if the recommendations are accepted and become law, private and public organisations will have to do more to catalogue and publish what data is held, share it more widely and in particular allow individuals with access and more direct control.
For international companies, this may not be anything new. Legislation in other countries, especially the European Union has meant that companies like Google for example already have mechanisms by which individuals can access the data that is held about them.
For a strategy of justifying the increased use of data to work, the public will need to be convinced that the data is going to be used in the their interest and not as just another way to justify increased surveillance and recording of private information, especially by the Government. The benefits of data availability and analysis need to be clearly communicated to the public. It will be important to give clear examples of cases where there has been a direct association between using data and better health outcomes. Of course, this is harder to do if the benefits of analysis of data are in the future and depend on the changes to data use being implemented successfully.
Perhaps a bigger challenge however is the lack of people with the right skills to handle and analyse data. Universities are only recently starting to focus on data science and related degrees and even then, the focus is on the analysis part and not on the other skills required for making use of information produced from these activities in an organisation or in government.
Those companies that understand the benefits of data are already struggling to get trained staff to work on these problems. Many organisations however, just don't understand how the collection and analysis of data will help them. Consequently, they are unwilling to make it a priority in their budgets or activities. Ironically enough, this includes universities themselves.
Explore further: The future will be built on open data – here's why
