Drowning in Data

Add bookmark

The IT world is all a-flutter about the potential of big data. But, beware of losing sight of the real value to your business, writes Daniel Senter.

The rate at which we are creating data is increasing at almost exponentially: mobile phone GPS, social media posts and sensors on objects collect information about almost everything while decreases in storage costs mean that the volume, variety and velocity of data we collect and store will remain on its current rapidly increasing trajectory.

A study published in 2011 by Discovery News, for instance, claimed that up to 94% of data worldwide is stored digitally, compared to just 0.8% 25 years ago!

However the question is ‘What are we doing with all this data?’

A 2011 Gartner survey highlighted corporate data is growing by around 60% each year, however 73% of those interviewed felt their competitors could use the data better. More interesting - or perhaps shocking - is that only 17% of the respondents claimed to be using more than 75% of the data they collect. So companies are capturing and storing data they are not even using!

The question to answer is WHY?

· Do they fully understand what data they are collecting?

· Do they know how to process the data - structured or unstructured?

· Is the data actually useful?

Many authors have taken the view that whilst the increase in data offers the potential to drive better decision making through business insights, there is a note of caution around this. As the volume and variety of data increases, so does the uncertainty, ambiguity and complexity, i.e. data noise. The removal or correction of data noise requires considerable thought and appropriate tools to ensure that true insights are extracted from the data.

To complicate this further another shift we are seeing within the data growth is the increase in unstructured data. By unstructured data I mean data that does not sit with in a database and exists as textual (text in documents, blogs, social media, etc) or non textual data (images, video and music). Analysing this type of data requires different tools to those traditionally used and asks us the same challenging questions above in finding the nuggets of information amongst the noise...

The effort required in filtering and processing the wealth of data we collect requires time and money, which should decrease over time.

However, the high start up investments may deter many businesses.

Business that are able to take on the challenge in digesting and processing the data will become more efficient in doing so and gain quick wins along the way. Although as seen with other specialisms, technologies will soon become available to automate and improve this activity thus closing the gap and making it more affordable to the masses.

Therefore one of the biggest challenges in the short term to stay ahead is to really understand and focus on the data that is meaningful to you. Define what you or your business value in the data being produced and then look at ways in which to capture, process and understand it.

[inlinead]


Latest Webinars

From Chaos to Clarity – A Playbook for Orchestrating Outcomes at Scale

2025-12-02

11:00 AM - 12:00 PM SGT

Join us for the final webinar of Making AI Work Webinar Series. Uncover the path to reducing complex...

End-to-end portfolio and asset management transformation with Powerco and TNB

2025-10-23

01:00 PM - 02:00 PM SGT

This session goes beyond theory, offering concrete examples of how two transmission and distribution...

G7 Tech - October webinar series

2025-10-23

10:00 AM - 11:00 AM EDT

Join us for a series of webinars from G7 exploring how to improve processes, business operations, ef...

Recommended