The Internet of Things Meets Big Data, with Chris Curran | Big Think

The Internet of Things is an idea that's been around for maybe 10 or 15 years or so. And I think the first time I heard about it there was some discussion of a European appliance manufacturer who had a refrigerator that was connected to the network, the Internet.

And I think the scenarios that were playing out there were: "Okay, what if the refrigerator was smart and knew enough to adjust temperatures?" Maybe it knew enough to send a request or a message to the manufacturer saying, "Hey, the compressor is going bad." That was the first time I had heard about this idea of connecting a product to the network. A product that wasn't a traditional computer; wasn't something that we thought was supposed to be connected to the network or interconnected.

And that started to open up a lot of people's eyes, I think, to this idea. It may not have been the first theoretical time that it was discussed, but it was the first time that I think popular media and culture started talking about this idea. But, for whatever reason, that connected refrigerator didn't really take off. The idea really didn't kind of explode.

Then we started hearing about the connected washing machine in the same kind of context. And I think that that idea of the Internet of Things then started to refine a bit and we heard about machine-to-machine communication. The idea that instead of human to machine, so sort of through a screen or through a webpage or through our mobile phones or whatever, that machines would talk to one another.

So the idea of a stock quote generating a message that would be sent to another machine that might think about, "Okay now I may need to make a trade," so the automated trading world. Or a weather forecast that's reported online triggering a message to another system that's going to adjust stock levels. So this idea that machines might talk to one another without a human in between was sort of a refinement, I think, of the Internet of Things idea.

And now, over the last handful of years, we're seeing more consumer-facing ideas and concepts and products coming out that I think has regenerated interest in this machine-to-machine communication. So now we're hearing things about the connected car and the connected thermostat and the smart home.

So now that the consumer side of things is heating up, we're also asking ourselves from a business perspective: "What is the Internet of Things for a business?" Maybe where you don't have a consumer product or a widget that you sell. Maybe you're a services company or a bank; what does the Internet of Things mean for a business like that, particularly a services company?

And so for companies that aren't as consumer-facing or product-oriented, I think there's more opportunity but more sort of clouds around what the Internet of Things means for business. And I think that anything that we think about, that we estimate, that we operate in, places we work in, people we work with, we think about the effectiveness of those things, the effectiveness of our workplace, the effectiveness of a warehouse, the effectiveness of routes that a truck takes or that a forklift makes.

And we make guesses about the right ways to design a workspace or design a logistics path, design a warehouse. And the question is: can we use the idea of sensors to collect data about things that we were just guessing about before, that we were estimating, that we were sort of using our gut to design?

Can we collect real data about the performance of people and places and processes that can give us more insight into optimizing and evolving our designs for the way our business works? And that's what I call the Internet of Business Things. It's not the consumer-facing stuff; it's the business stuff.

Most of the data we think about from an enterprise perspective is sort of modular, it's transactional, it's a call center call or it's a web transaction or it's a sale or it's a quote or it's a piece of data about a product. But the Internet of Things will be creating streams of data.

So one of the analogies is how social media creates streams of data. So we've got tweets and we've got flows on our wall and such that are more streams of information. And so think about a sensor that might be on a door in a warehouse. Every time the door opens, the sensor records a door opening, a time of day, a day of the week.

Maybe it creates some other ambient information - it records the temperature in the facility or it records how long the door was open or whatever. So these are little bits of data that are happening every time the door is opened. And that door might be open a lot of times during an hour or a day or a week or a month, but it's creating these streams of data.

And it's not like that one piece of data might be that interesting, but it's the trends and the patterns and things that we're looking for in the data that may be what we're looking for. So I want to know how often that room is used during the morning, the afternoon, certain days of the week, weeks of the month, months of the year, to help me be more efficient about the way I staff people in that office or the temperature or the cooling or the electricity that I feed to that part of my facility.

So that data that's created, these streams of data are sets of data that I may not already have a technology platform to process. I don't know how to process streams maybe. So now I have to think about a new architecture, a new way to process a stream of data, not just a stream but maybe I have a hundred facilities that I have these sensors in and they're creating these streams of data.

So now I've got a new challenge around data. One of the challenges starts to feed and link into the big data topic. So, as I'm collecting these individual sensor data points over hundreds or thousands or tens of thousands of sensors every minute or every second, I've now got these parallel streams of data I have to process.

And so it's unclear exactly how much data I would want to keep for how long, but it's certainly a new kind of data architecture, and for some companies will be a big data problem because there's going to be a lot of this data for large companies for a lot of sensors and a lot of facilities or a lot of products that they want to track or whatever it is that they want to get more insight into by putting tags or sensors on them.

So, to me, this is one of the sort of subsets, subdomains of big data around sensor data, streams of data that we're going to have to think about new architectures and new designs of and new technologies for capturing them, storing them, processing them, abrogating them, analyzing them, and so on.

So I think it's going to be a really interesting evolution to see how the big data technologies and technologies in general are emerging to deal with streams of sensor data.