The 8 Biggest Risks of Big Data Projects

There is a reason people are calling it Big Data.

The phenomenon of “Big Data” exacerbates the tension between potential benefits and privacy risks by upping the ante on both sides of the equation.  Any project can fail for any number of reasons: bad management, poor budget  management, or just a lack of relevant skills. However, big data projects bring their own specific risks.

Disturbingly Currently Only 13% of Companies Achieve Full-scale Implementation of Their In-house Big Data Projects

Such a low success rate should be concerning for organisations  embarking on   big data projects, since many businesses are choosing to adopt big data without a clear understanding of what the return on investment (ROI)  will be.

Big Data Investment Projects Requires Experimentation to Discover Where They Can Best Be Used to Yield Significant ROI

Given the amount of risk involved, we need to be aware of the dangers that could potentially arise if we fail to cover all of the bases.  Here are the eight  biggest risks of big data projects, essentially  a basic  checklist that should be taken into account when developing a  strategy  for your “Big Data” project.

Big Data Presents New Challenges Impacting the Entire Risk Spectrum

1. Data Security

This risk is obvious and often uppermost in our minds when we are considering the logistics of data collection and analysis. Data theft is a rampant and growing area of crime and attacks are getting bigger and more damaging. The bigger your data, the bigger the target it presents to criminals.

2. Data Privacy

Closely related to the issue of security is privacy.  This ocean of Big Data is also intrusive. Risk mitigation strategies are essential for protecting privacy.  You need to be sure that the sensitive information you are storing and collecting isn’t going to be divulged through damaging misuse by yourself or by the people to whom you have delegated the responsibility for analyzing and reporting on it.

On the one hand, big data unleashes tremendous benefits not only to individuals but also to communities and society at large, including breakthroughs in health research, sustainable development, energy conservation and personalized marketing. On the other hand, big data introduces new privacy and civil liberties concerns including high-tech profiling, automated decision-making, discrimination, and algorithmic inaccuracies or opacities that strain traditional legal protections.

3. Costs

Data collection, aggregation, storage, analysis, mapping and reporting costs a lot of money. These costs can be mitigated by careful budgeting, but getting it wrong at that point can lead to spiralling costs, potentially negating any value added to your bottom line by your data-driven initiative. A well-developed strategy will clearly set out what you intend to achieve and the benefits that can be gained so they can be balanced against the resources allocated to the project.

4. Time to Deployment
The amount of time required to deploy a big data solution can vary significantly depending on the type of implementation. It’s worth considering that an in-house solution can take  over six months  to build depending on the , however, a cloud-based solution requires no internal infrastructure.

5. Scalability
The ability to scale a project up or down is crucial. Organisations underestimate how quickly their data  can and  will grow, or fail to take into account varying usage levels. Cloud-based systems will offer more scalability, allowing businesses to increase or decrease usage as  required.

6. Enhanced Transparency

Big data analysis is prone to errors, inaccuracies and bias. Consequently, organisations should provide more transparency into their automated processing operations and decision-making processes, including eligibility factors and marketing profiles.

7. Bad Data

Collecting irrelevant, out of date, or erroneous data. The big data revolution  has led to a “collect everything and analyse it later approach.  If you are not analysing the right up-to-date data, you won’t be drawing the right conclusions  to provide value.

8. Accessibility
Big data is only useful if it is accessible  by  the people who can actually learn something from the data and implement it into everyday business practices.

Currently 57% of Organisations Cite Skills Gap as a Major Inhibitor to New System Adoption

Once again, ease of accessibility will vary based on implementation, so selecting a vendor that matches your internal capabilities will be crucial. Consider the time and investment it will take to train teams when calculating time to value and overall ROI.

Here are some real-world examples of Big Data in action:

  • Manufacturers are monitoring minute vibration data from their equipment, which changes slightly as it wears down, to predict the optimal time to replace or maintain.
  • The government is making data public at both the national, state, and city level for users to develop new applications for public good.
  • Financial Services organisations are using data mined from customer interactions to slice and dice their users into finely tuned segments.
  • Hospitals are analysing medical data and patient records to predict those patients that are likely to seek readmission within a few months of discharge.
  • Web-based businesses are developing information products that combine data gathered from customers to offer more appealing recommendations.


What do you think?

627 Points

Written by George Brzozowski

I want to help you to see news events as starting points for constructive conversations. I seek to cut through the froth of the political spin cycle to underlying truths and values. I want to be so focused on progress that together we can provide a credible and constructive counter-narrative to the hopelessness-, anger-, and fear-inducing brand of discourse that is so pervasive in the news.


Leave a Reply



In a sign of things to come, a Swedish train operator is using new technology that employs big data to predict the entire commuter train system two hours into the future.

Playing God: Swedish Train operator uses Big Data to ‘avoid train delays that haven’t happened yet’

Are Women Better Project Managers Than Men?