Remember: The IoT Is Primarily About Small Data, Not Big

Posted on 16th March 2015 in data, Internet of Things, M2M, management, manufacturing, open data

In one of my fav examples of how the IoT can actually save lives, sensors on only eight preemies’ incubators at Toronto’s Hospital for Sick Children yield an eye-popping 90 million data points a day!  If all 90 million data points get relayed on to the “data pool,” the docs would be drowning in data, not saving sick preemies.

Enter “small data.”

Writing in Forbes, Mike Kavis has a worthwhile reminder that the essence of much of the Internet of Things isn’t big data, but small. By that, he means:

a dataset that contains very specific attributes. Small data is used to determine current states and conditions  or may be generated by analyzing larger data sets.

“When we talk about smart devices being deployed on wind turbines, small packages, on valves and pipes, or attached to drones, we are talking about collecting small datasets. Small data tell us about location, temperature, wetness, pressure, vibration, or even whether an item has been opened or not. Sensors give us small datasets in real time that we ingest into big data sets which provide a historical view.”

Usually, instead of aggregating  ALL of the data from all of the sensors (think about what that would mean for GE’s Durathon battery plant, where 10,000 sensors dot the assembly line!), the data is originally analyzed at “the edge,” i.e., at or near the point where the data is collected. Then only the data that deviates from the norm (i.e., is significant)  is passed on to to the centralized data bases and processing.  That’s why I’m so excited about Egburt, and its “fog computing” sensors.

As with sooo many aspects of the IoT, it’s the real-time aspect of small data that makes it so valuable, and so different from past practices, where much of the potential was never collected at all, or, if it was, was only collected, analyzed and acted upon historically. Hence, the “Collective Blindness” that I’ve written about before, which limited our decision-making abilities in the past. Again, Kavis:

“Small data can trigger events based on what is happening now. Those events can be merged with behavioral or trending information derived from machine learning algorithms run against big data datasets.”

As examples of the interplay of small and large data, he cites:

  • real-time data from wind turbines that is used immediately to adjust the blades for maximum efficiency. The relevant data is then passed along to the data lake, “..where machine-learning algorithms begin to understand patterns. These patterns can reveal performance of certain mechanisms based on their historical maintenance record, like how wind and weather conditions effect wear and tear on various components, and what the life expectancy is of a particular part.”
  • medicine containers with smart labels. “Small data can be used to determine where the medicine is located, its remaining shelf life, if the seal of the bottle has been broken, and the current temperature conditions in an effort to prevent spoilage. Big data can be used to look at this information over time to examine root cause analysis of why drugs are expiring or spoiling. Is it due to a certain shipping company or a certain retailer? Are there re-occurring patterns that can point to problems in the supply chain that can help determine how to minimize these events?”

Big data is often irrelevant in IoT systems’ functioning: all that’s needed is the real-time small data to trigger an action:

“In many instances, knowing the current state of a handful of attributes is all that is required to trigger a desired event. Are the patient’s blood sugar levels too high? Are the containers in the refrigerated truck at the optimal temperature? Does the soil have the right mixture of nutrients? Is the valve leaking?”

In a future post, I’ll address the growing role of data scientists in the IoT — and the need to educate workers on all levels on how to deal effectively with data. For now, just remember that E.F. Schumacher was right: “small is beautiful.”


Egburt: key tool to make IoT pay off NOW

Posted on 31st October 2014 in data, energy, Internet of Things, maintenance, management, retail

As I’ve remarked before, writing the Managing the Internet of Things Revolution e-guide to IoT strategy for SAP was an eye-opener for me, shifting my attention from the eye-popping opportunities for radical reinvention through the IoT (products as services, user-customizable products, seamless smart phone-car integration, etc.) to very practical ways the IoT could begin optimizing companies’ current operations TODAY (BTW: much-deserved shout-out to SAP’s Mahira Kalim: it was dialogue with her that led to this insight!).


In that vein, I was blown away at this week’s IoT Global Summit by the roll-out of Egburt by Camgian.

Egburt stresses two crucial, inter-related obstacles to widespread IoT solution deployment by mainstream businesses:

  • low cost-of-ownership sensing (by using very little energy, thereby extending battery life)
  • reducing potentially huge cloud-computing costs (because of the sheer volume of 24/7 sensor data) by allowing “fog computing,” where the processing would be done right at the collection process, with only the small amount of really relevant data being passed on to a central location.

The highlight of the product launch was a live demo of Egburt in real-time use at a chain of dollar stores in the south, monitoring a wide range of factors, from floor traffic to freezer operation (Camgian pointed out the system paid for itself in the first month of operation when it recorded failure of a freezer when the store was unoccupied, in time for immediate repairs to avoid loss of frozen foods).

Think about it: the very volume of Big Data possible with constant monitoring by a whole range of sensors can also be the IoT’s undoing. Since all that’s of interest in many cases is data that deviates from the norm, doesn’t it make sense to process that data at the collection point, then only pass on the deviations?

The company has targeted three IoT segments:

  • retail to reduce heating and lighting, and maximize sales through tracking foot traffic patterns to optimize product placement.
  • infrastructure: with sensors at key points such as bridges that will detect flooding and stress.
  • smart cities: optimizing emergency response.

In a sponsored white paper by ABI Research, “Evolution of the Internet of Things: from connected to intelligent devices,” they documented the benefits of going beyond first-generation, “connected,” IoT devices that were just sensors collecting and passing on data, to a second generation of “intelligent ones” such as Egburt the combine sensors and processing and offer not only lower operating costs but also — critically — more data security:

  • “Communication Latency: Handling more processing at the network’s edge reduces latency from the device’s actions. Use cases that are highly time-sensitive and require immediate analysis of, or response to, the collected sensor data are, in general, unfeasible under cloud- centric IoT architectures, especially if the data are sent over long distances.
  • “Data Security: By and large, sensitive and business-critical operational data are safer when encrypted adequately on the endpoint level. Unintelligent devices transmitting frequent and badly secured payloads to the cloud are generally more vulnerable to hacking and interception by unauthorized parties. Additionally, many enterprises may need to secure and control their machine data on the edge level for compliance reasons.
  • “Total Cost of Ownership: Perhaps most significantly, the paradigm shift can reduce the IoT systems’ total cost of ownership, or TCO. Intelligent devices are usually more expensive than less sophisticated alternatives, but their TCO over a long service life can be substantially lower.”

IMHO, for the IoT to be widely deployed, especially in SMEs, devices such as Egburt that reduce the cost of collecting and processing data are a critical component.

(PROMINENT DISCLAIMER: I actually won a FitBit in Camgian’s drawing at the conference. That has no impact on this review. Had I won the iPhone 6 that they also gave away, I would have totally been in the bag, LOL…)