Using big data for product development

Big data is turning up everywhere. In this PEX Network interview, Kaare Buch Petersen, Information Architect in Global IT at Christen Hansen, a Denmark-based supplier of bioscience products to food and health industries, talks about how his company has been using big data in the development of their new products.

PEX Network: What’s the background to your big data program at Chr. Hansen?

Kaare Buch Petersen: We were seeing that our scientists were using more and more time collecting data instead of analyzing it and we wanted to speed up that process and make our scientists more effective and use their skills in better ways. That’s why we started looking at different technologies. We found that big data was very interesting and felt that it was mature enough to set up a project. That was how we got started.

PEX Network: My understanding is that one of the main aims of big data at your company is in product development. How does big data help with that?

Kaare Buch Petersen: There used to be islands of data in Chr. Hansen. Our scientists generated a lot of data but there were great walls between the different data. Big Data makes it possible to "glue" all these different experimental data together and have an opportunity to look at data in new ways that our scientists didn’t have before. Furthermore, we’ve been looking at automating all the dataflow in our product development. We had a lot of manual steps where people were extracting data and trying to merge them into one dataset; this is where big data helps us by automating all these processes.

PEX Network: And what kind of data are you looking at for this and how do you actually then analyze it?

Kaare Buch Petersen: The data we’re looking at is primarily from our electronic laboratory system and that is mainly experimental data, i.e. the results of the experiments going on in the labs. Furthermore, we also looked at more data outside the laboratories where we combine the data from other sources into that as well, such as project management systems or sales data.

The way that we analyze it is that usually we put it into different kind of reports or give the raw data to the scientists. It depends very much of the receiver of the data. If the people are in sales or just more in the commercial part we normally do a standardized report, whereas if it’s scientists they also want more datasets and then they put them into their preferred tool of analyzing.

PEX Network: I understand too that you started this on a relatively small scale and have been gradually scaling up. It sounds ironic doing small scale big data! Could you tell me more about what you did?

Kaare Buch Petersen: As in most companies we don’t have all the resources in the world so we wanted to show that big data was a technology that we could use and deliver. That’s why we had very limited focus on a small part of data. We showed that we could use this Big data technologies to collect data very easily and speed up the process. As soon as people saw that the project has just expanded and it’s gone far beyond our expectation how well it’s gone.

PEX Network: So you learned those lessons, proved the concept and then took it more globally?

Kaare Buch Petersen: Yes, exactly. We have a proof of concept, that was in five days, actually, and within those five days we could show the first results. And then we have gradually improved our model.

PEX Network: So what has been the biggest challenge from your perspective of using big data for product development?

Kaare Buch Petersen: The biggest challenge has been getting ahold of the unstructured data. In our case the data that we collect has often had a human hand involved and that means that people put data into Excel sheets but there’s no real structure in them. So we had a challenge that asking people to structure their data so we could collect them and put the data together in a valid and reliable way. Furthermore, we have had some technical issues because it’s a immature product - but nothing we couldn’t overcome.

PEX Network: Now, my final question, then, is outside of the product development arena where else do you think big data holds most promise?

Kaare Buch Petersen: I think that big data holds promise everywhere. In our case, we are a production company so there’s a lot of potential of expanding this big data into production, meaning that tracking our production side and optimizing production and all those things.

I actually think that big data could be used in almost every division and area in Chr. Hansen, all from our legal department to our marketing. We had a request from our legal area where they wanted us to build a system that could read the different laws and when they changed the law in different companies, because we are a global company, and that is also possible within the big data technology to make a lot of text analysis and that stuff to see what has changed.