OtoSense: the next level in sound-based IoT

It sounds (pardon the pun) as if the IoT may really be taking off as an important diagnostic repair tool.

I wrote a while ago about the Auguscope, which represents a great way to begin an incremental approach to the IoT because it’s a hand-held device to monitor equipment’s sounds and diagnose possible problems based on abnormalities.

Now NPR reports on a local (Cambridge) firm, OtoSense, that is expanding on this concept on the software end. Its tagline is “First software platform turning real-time machine sounds and vibrations into actionable meaning at the edge.”

Love the platform’s origins: it grows out of founder Sebastien Christian’s research on deafness (as I wrote in my earlier post, I view suddenly being able to interpret things’ sounds as a variation on how the IoT eliminates the “Collective Blindness”  that I’ve used to describe our past inability to monitor things before the IoT’s advent):

“[Christian} … is a quantum physicist and neuroscientist who spent much of his career studying deaf children. He modeled how human hearing works. And then he realized, hey, I could use this model to help other deaf things, like, say, almost all machines.”

(aside: I see this as another important application of my favorite IoT question: learning to automatically ask “who else can use this data?” How does that apply to YOUR work? But I digress).

According to Technology Review, the company is concentrating primarily on analyzing car sounds from IoT detectors on the vehicle at this point (working with a number of car manufacturers) although they believe the concept can be applied to a wide range of sound-emitting machinery:

“… OtoSense is working with major automakers on software that could give cars their own sense of hearing to diagnose themselves before any problem gets too expensive. The technology could also help human-driven and automated vehicles stay safe, for example by listening for emergency sirens or sounds indicating road surface quality.

OtoSense has developed machine-learning software that can be trained to identify specific noises, including subtle changes in an engine or a vehicle’s brakes. French automaker PSA Group, owner of brands including Citroen and Peugeot, is testing a version of the software trained using thousands of sounds from its different vehicle models.

Under a project dubbed AudioHound, OtoSense has developed a prototype tablet app that a technician or even car owner could use to record audio for automated diagnosis, says Guillaume Catusseau, who works on vehicle noise in PSA’s R&D department.”

According to NPR, the company is working to apply the same approach to a wide range of other types of machines, from assembly lines to DIY drills. As always with IoT data, handling massive amounts of data will be a challenge, so they will emphasize edge processing.

OtoSense has a “design factory” on the site, where potential customers answer a variety of questions about the sounds they must monitor (such as whether the software will be used indoors or out, whether it is to detect anomalies, etc. that will allow the company to choose the appropriate version of the program.

TechCrunch did a great article on the concept, which underscores really making sound detection precise will take a lot of time and refinement, in part because of the fact that — guess what — sounds from a variety of sources are often mingled, so the relevant ones must be determined and isolated:

“We have loads of audio data, but lack critical labels. In the case of deep learning models, ‘black box’ problems make it hard to determine why an acoustical anomaly was flagged in the first place. We are still working the kinks out of real-time machine learning at the edge. And sounds often come packaged with more noise than signal, limiting the features that can be extracted from audio data.”

In part, as with other forms of pattern recognition such as voice, this is because it will require accumulating huge data files:

“Behind many of the greatest breakthroughs in machine learning lies a painstakingly assembled dataset.ImageNet for object recognition and things like the Linguistic Data Consortium and GOOG-411 in the case of speech recognition. But finding an adequate dataset to juxtapose the sound of a car-door shutting and a bedroom-door shutting is quite challenging.

“’Deep learning can do a lot if you build the model correctly, you just need a lot of machine data,’ says Scott Stephenson, CEO of Deepgram, a startup helping companies search through their audio data. ‘Speech recognition 15 years ago wasn’t that great without datasets.’

“Crowdsourced labeling of dogs and cats on Amazon Mechanical Turk is one thing. Collecting 100,000 sounds of ball bearings and labeling the loose ones is something entirely different.

“And while these problems plague even single-purpose acoustical classifiers, the holy grail of the space is a generalizable tool for identifying all sounds, not simply building a model to differentiate the sounds of those doors.

…”A lack of source separation can further complicate matters. This is one that even humans struggle with. If you’ve ever tried to pick out a single table conversation at a loud restaurant, you have an appreciation for how difficult it can be to make sense of overlapping sounds.

Bottom line: there’s still a lot of theoretical and product-specific testing that must be done before IoT-based sound detection will be an infallible diagnostic tool for predictive maintenance, but clearly there’s precedent for the concept, and the potential payoff are great!

 


LOL: as the NPR story pointed out, this science may owe its origins to two MIT grads of an earlier era, “Click” and “Clack” of Car Talk, who frequently got listeners to contribute their own hilarious descriptions of the sounds they heard from their malfunctioning cars.   BRTTTTphssssBRTTTT…..

#IoT Sensor Breakthroughs When Lives Are On the Line!

One of my unchanging principles is always to look to situations where there’s a lot at stake — especially human lives — for breakthroughs in difficult issues.

Exhibit A of this principle for the IoT is sensor design, where needing to frequently service or recharge critical sensors that detect battlefield conditions can put soldiers’ lives at stake (yes, as long-time readers know, this is particularly of interest to me because my Army officer son was wounded in Iraq).

FedTech reports encouraging research at DARPA on how to create sensors that have ultra-low power requirements, can lie dormant for long periods of time and yet are exquisitely sensitive to critical changes in conditions (such as vehicle or troop movements) that might put soldiers at risk in battlefield conditions.

The  N-ZERO (Near Zero RF and Power Operations)  program is a three-year initiative to create new, low-energy battlefield sensors, particularly for use at forward operating bases where conditions can change quickly and soldiers are constantly at risk — especially if they have to service the sensors:

“State-of-the-art military sensors rely on “active electronics” to detect vibration, light, sound or other signals for situational awareness and to inform tactical planning and action. That means the sensors constantly consume power, with much of that power spent processing what often turns out to be irrelevant data. This power consumption limits sensors’ useful lifetimes to a few weeks or months with even the best batteries and has slowed the development of new sensor technologies and capabilities. The chronic need to service or redeploy power-depleted sensors is not only costly and time-consuming but also increases warfighter exposure to danger.”

…. (the project has) the goal of developing the technological foundation for persistent, event-driven sensing capabilities in which the sensor can remain dormant, with near-zero power consumption, until awakened by an external trigger or stimulus. Examples of relevant stimuli are acoustic signatures of particular vehicle types or radio signatures of specific communications protocols. If successful, the program could extend the lifetime of remotely deployed communications and environmental sensors—also known as unattended ground sensors (UGS)—from weeks or months to years.”

A key goal is a 20-fold battery size reduction while still having the sensor last longer.

What cost-conscious pipeline operators, large ag business or “smart city” transportation director wouldn’t be interested in that kind of product as well?

According to Signal, the three-phase project is ahead of its targets. In the first part, which ended in December, the DARPA team created “zero-power receivers that can detect very weak signals — less than 70 decibel-milliwatt radio-frequency (RF) transmissions, a measure that is better than originally expected.” This is critical to the military (and would have huge benefits to business as well, since monitoring frequently must be 24/7 but reporting of background data  (vs. significant changes) would both deplete batteries while requiring processing of huge volumes of meaningless data). Accordingly, a key goal would be to create “… radio receivers that are continuously alert for friendly radio transmissions, but with near zero power consumption when transmissions are not present.” A target is  “exploitation of the energy in the signal signature itself to detect and discriminate the events of interest while rejecting noise and interference. This requires the development of passive or event-powered sensors and signal-processing circuitry. The successful development of these techniques and components could enable deployments of sensors that can remain “off” (that is, in a state that does not consume battery power), yet alert for detecting signatures of interest, resulting in greatly extended durations of operation.”

The “exploitation of .. energy in the signal signature itself sounds reminiscent of the University of Washington research I’ve reported in the past that would harness ambient back-scatter to allow battery-less wireless transmission, another key potential advance in IoT sensor networks.

The following phrases of N-ZERO will each take a year.

Let’s hope that the project is an overall success, and that the end products will also be commercialized. I’ve always felt sensor cost and power needs were potential IoT Achilles’ heels, so that would be a major boost!

Updating my “SmartAging” device design criteria

Could seniors be the ideal test group for user-friendly consumer IoT devices?

Two years ago I created a series of criteria by which to evaluate IoT devices that seniors might use (N.B., I didn’t really focus on ones specifically designed for seniors, because I have an admitted bias against devices with huge buttons or that look like mid-century period tube radios — it’s been my experience that seniors aren’t crying out to be labeled as “different.”) to improve their quality of life.

The particular emphasis was on what I called “SmartAging,” which synthesizes two aspects of the IoT:

  • Quantified Self health devices to keep seniors healthier longer and to become partners with their doctors rather than passive recipients of care, and
  • smart home devices to make it easier to run their homes, so that seniors could remain on their own as long as possible rather than entering some drab, sterile assisted-living facility (again, my bias showing…).

A lot has happened since I compiled the list. The changes have solidified my conviction that seniors, especially the less technologically minded, might be the acid test of consumer IoT user friendliness because they can’t be expected to work as hard at mastering devices, they don’t have the automatic openness of digital natives, and encounter differing degrees of reduced agility, etc. 

Also, given the current political climate, it makes sense to try to improve seniors’ lives as much as possible without requiring costly public services that are in jeopardy (I am trying to be civil here, OK?).

The most dramatic of these developments is the amazing success of Amazon’s voice-activated Echo.  I’ve praised it before as an ideal device for seniors, partially because voice is such a natural input for anyone, and particularly because it means that the tech-averse don’t have to learn about interfaces or programs, just speak! Even better, as the variety of “skills” increases, the Echo really is becoming a unified SmartAging hub: I can now control my Sensi smart thermostats and the “Ask My Buddy” skill can even call for assistance, so it works for both halves of SmartAging.  Although I haven’t tested it, I assume much of this also holds true for the Google Home.

There’s an increasing variety of other new Quantified Self devices, some of which are specifically focused on seniors, such as the GreatCall Jitterbug Smart phone, which comes with a simplified, over-size home page featuring “brain games” a la Lumosity, and an Urgent Response system (all of these features are available on an iPhone and, I assume, on Android, but must be set in Settings rather than being the default settings).

In addition, on the personal level, I convinced my Apple Store (disclaimer: I’m at the bottom of the food chain with Apple, not privy to any policies or devices under consideration, so this is just my opinion) to let me start bi-weekly classes at the local senior center on how to use Apple devices, especially the iPad. I continue to work with a lot of seniors who come into the store who are often leery of tech products.

Silver Medal!

Most directly, last month’s companywide Apple Wellness Challenge was life-changing for me.  This year the friendly competition focused on the Apple Watch (important, since a watch is a familiar form-factor to geezers). After wasting three days trying to find the app, I really got into the event because we could share results with friends to encourage (or shame, LOL) them — that really motivated me.  Bottom line: I managed to win a Silver Medal, Apple featured my experience on the event website, and, most important, I made lasting changes to my fitness regimen that I’ve sustained since then, now exercising almost an entire hour a day. I couldn’t help think afterward that the program really did show that user-friendly technology can improve seniors’ lives.

Sooo, with a few more years to think about them and more progress in devices themselves, (as well as increased sensitivity to issues such as privacy and security) here are my amended criteria for evaluating products and services for seniors. As I mentioned the first time, Erich Jacobs of OnKöl assisted with the specs):

Ease of Use

  1. Does it give you a choice of ways to interact, such as voice, text or email? Voice in particular is good for seniors who don’t want to learn about technology, just use it.
  2. Is it easy for you to program, or — if you them give your permission — does it allow someone else to do it remotely?
  3. Does it have either a large display and controls or the option to configure them through settings?
  4. Is it intuitive?
  5. Does it require hard-wired, professional installation?
  6. Is it flexible: can it be adjusted? Is it single purpose, or does it allow other devices to plug in and create synergies? Can it be a true hub for all your IoT devices?
  7. Does it complicate your life, or simplify it?
  8. Do any components require regular charging, or battery replacement?

Privacy, Security, and Control

  1. Is storage local vs. cloud or company’s servers? Is data encrypted? Anonymized?
  2. Do you feel creepy using it?
  3. Is it password-protected?
  4. Is security “baked in” or an afterthought?
  5. Can you control how, when, and where information is shared?
  6. If it is designed to allow remote monitoring by family or caregivers, can you control access by them?
  7. Will it work when the power goes out?

Affordability

  1. Are there monthly fees? If so, low or high? Long term contract required?
  2. Is there major upfront cost? If so, is that offset by its versatility and/or the contrast to getting the same services from a company?
  3. Does full functioning require accessories?

Design/UX

  1. Is it stylish, or does the design” shout” that it’s for seniors? Is it “Medical” looking?
  2. Is the operation or design babyish?
  3. Would younger people use it?
  4. Is it sturdy?
  5. Does it have “loveability” (i.e., connect with the user emotionally)? (This term was coined by David Rose in Enchanted Objects, and refers to products that are adorable or otherwise bond with the user.)

Architecture

  1. Inbound
    1. Does it support multiple protocols (eg. Bluetooth, BluetoothLE, WiFi, etc)
    2. Is the architecture open or closed?
  2. Outbound
    1. Does it support multiple protocols (eg. WiFi, Ethernet, CDMA, GSM, etc)
    2. Data path (cloud, direct, etc)
  3. Remote configuration capability (i.e., by adult child)? If so, can the user control amount of outside access?

Features and Functions

  1. Reminders
    1. Passive, acknowledge only
    2. Active dispensing (of meds)
  2. Home Monitoring
    1. Motion/Passive Activity Monitoring
    2. Environmental Alarms (Smoke, CO, Water, Temp)
    3. Intrusion Alarms (Window etc)
    4. Facilities/Infrastructure (Thermostat)
  3. Health Monitoring
    1. Vitals Collection
    2. Wearables Activity Monitoring
    3. Behavioral/Status Polling (How are you feeling today?)
    4. Behavioral Self-improvement
  4. Communications Monitoring
    1. Landline/Caller ID
      1. Identify scammers
    2. eMail and computer use
      1. Identify scammers
    3. Mobile phone use
  5. Fixed Personal Emergency Response System (PERS)
  6. Mobile Personal Emergency Response System (PERS)
  7. Fixed Fall Detection/Prediction
  8. Mobile Fall Detection/Prediction
  9. Telehealth (Video)
  10. New and Innovative Features

If you’re thinking about developing an IoT product and/or service for seniors I hope you’ll consider the SmartAging concept, and that these criteria will be helpful. If you’re looking for consulting services on design and/or implementation, get in touch!

Libelium: flexibility a key strategy for IoT startups

I’ve been fixated recently on venerable manufacturing firms such as 169-yr. old Siemens making the IoT switch.  Time to switch focus, and look at one of my fav pure-play IoT firms, Libelium.  I think Libelium proves that smart IoT firms must, above all, remain nimble and flexible,  by three interdependent strategies:

  • avoiding picking winners among communications protocols and other standards.
  • avoiding over-specialization.
  • partnering instead of going it alone.
Libelium CEO Alicia Asin

Libelium CEO Alicia Asin

If you aren’t familiar with Libelium, it’s a Spanish company that recently turned 10 (my, how time flies!) in a category littered with failures that had interesting concepts but didn’t survive. Bright, young, CEO Alicia Asin, one of my favorite IoT thought leaders (and do-ers!) was recently named best manager of the year in the Aragón region in Spain.  I sat down with her for a wide-ranging discussion when she recently visited the Hub of the Universe.

I’ve loved the company since its inception, particularly because it is active in so many sectors of the IoT, including logistics, industrial control, smart meters, home automation and a couple of my most favorite, agriculture (I have a weak spot for anything that combines “IoT” AND “precision”!) and smart cities.  I asked Asin why the company hadn’t picked one of those verticals as its sole focus: “it was too risky to choose one market. That’s still the same: the IoT is still so fragmented in various verticals.”

The best illustration of the company’s strategy in action is its Waspmote sensor platform, which it calls the “most complete Internet of Things platform in the market with worldwide certifications.” It can monitor up to 120 sensors to cover hundreds of IoT applications in the wide range of markets Libelium serves with this diversified strategy, ranging from the environment to “smart” parking.  The new versions of their sensors include actuators, to not simply report data, but also allow M2M control of devices such as irrigation valves, thermostats, illumination systems, motors and PLC’s. Equally important, because of the potentially high cost of having to replace the sensors, the new ones use extremely little power, so they can last        .

Equally important as the company’s refusal to limit itself to a single vertical market is its commitment to open systems and multiple communications protocols, including LoRaWAN, SIGFOX, ZigBee and 4G — a total of 16 radio technologies. It also provides both open source SDK and APIs.

Why?  As Asin told me:

 

“There is not going to be a standard. This (competiting standards and technology) is the new normal.

“I talk to some cities that want to become involved in smart cities, and they say we want to start working on this but we want to use the protocol that will be the winner.

“No one knows what will be the winner.

“We use things that are resilient. We install all the agents — if you aren’t happy with one, you just open the interface and change it. You don’t have to uninstall anything. What if one of these companies increases their prices to heaven, or you are not happy with the coverage, or the company disappears? We allow you to have all your options open.

“The problem is that this (not picking a standard) is a new message, and people don’t like to listen.  This is how we interpret the future.”

Libelium makes 110 different plug and play sensors (or as they call them, “Plug and Sense,” to detect a wide range of data from sources including gases, events, parking, energy use, agriculture, and water.  They claim the lowest power consumption in the industry, leading to longer life and lower maintenance and operating costs.

Finally, the company doesn’t try to do everything itself: Libelium has a large and growing partner network (or ecosystem, as it calls it — music to the ears of someone who believes in looking to nature for profitable business inspiration). Carrying the collaboration theme even farther, they’ve created an “IoT Marketplace,” where pre-assembled device combinations from Libelium and partners can be purchased to meet the specific needs of niches such as e-health,  vineyards, water quality, smart factories, and smart parking.  As the company says, “the lack of integrated solutions from hardware to application level is a barrier for fast adoption,” and the kits take away that barrier.

I can’t stress it enough: for IoT startups that aren’t totally focused on a single niche (a high-stakes strategy), Libelium offers a great model because of its flexibility, agnostic view of standards, diversification among a variety of niches, and eagerness to collaborate with other vendors.


BTW: Asin is particularly proud of the company’s newest offering, My Signals,which debuted in October and has already won several awards.  She told me that they hope the device will allow delivering Tier 1 medical care to billions of underserved people worldwide who live in rural areas with little access to hospitals.  It combines 15 different sensors measuring the most important body parameters that would ordinarily be measured in a hospital, including ECG, glucose, airflow, pulse, oxygen in

It combines 15 different sensors measuring the most important body parameters that would ordinarily be measured in a hospital, including ECG, glucose, airflow, pulse, blood oxygen, and blood pressure. The data is encrypted and sent to the Libelium Cloud in real-time to be visualized on the user’s private account.

It fits in a small suitcase and costs less than 1/100th the amount of a traditional Emergency Observation Unit.

The kit was created to make it possible for m-health developers to create prototypes cheaply and quickly.

Siemens’s Mobility Services: Trains Become IoT Labs on Wheels

George Stephenson's Killingworth locomotive Source: Project Gutenberg

George Stephenson’s Killingworth locomotive
Source: Project Gutenberg

As those of you who know rail history understand, with Stephenson as your last name, you’re bound to have a strong interest in railroads! Add in the fact that I was associate producer of an award-winning documentary on the subject back in the early 70’s, and it’s no wonder I was hooked when I got a chance to meet with some of Siemens’s top rail executives on my trip to Barcelona last week (Disclaimer: Siemens paid my expenses, but didn’t dictate what I covered, nor did they have editorial review of this piece).

What really excites me about railroads and the IoT is that they neatly encapsulate the dramatic transformation from the traditional industrial economy to the IoT: on one hand, the railroad was perhaps THE most critical invention making possible 19th century industry, and yet it still exists, in recognizable but radically-evolved form, in 2016. As you’ll see below, trains have essentially become laboratories on wheels!

I dwelt on the example of the Union Pacific in my e-book introduction to the IoT, SmartStuff, because to CIO Lynden Tennison was an early adopter, with his efforts focused largely on reducing the number of costly and dangerous derailments, through measures such as putting infrared sensors every twenty miles along the rail bed to spot “hotboxes,” overheating bearings. That allowed an early version of what we now know as predictive maintenance, pulling cars off at the next convenient yard so the bearings could be replaced before a serious problem. Even though the technology even five years ago was primitive compared to today, the UP cut bearing-related derailments by 75%.

Fast-forward to 2016, and Siemens’s application of the IoT to trains through its Mobility Services is yielding amazing benefits: increasing reliability, cutting costs, and even leading to possible new business models. They’ve taken over maintenance for more than 50 rail and transit programs.

While I love IoT startups with a radical new vision and no history to encumber them, Siemens is a beacon to those companies firmly rooted in manufacturing which may wonder whether to incorporate the IoT in their services and strategy. I suspect that its software products are inherently more valuable than competitors from pure-play software firms at commercial launch because the company eats its own dogfood and applies the new technology first to the products it manufactures and maintains — closing the loop.

Several of its executives emphasized that one of the advantages Siemens feels they enjoy is that their software engineers in Munich work in a corner of an old locomotive factory that Siemens still operates, so they can interact with those actually building and maintaining the engines on a daily basis. When it comes to security issues, their experience as a manufacturer means they understand the role of each component of the signaling system. Dr. Sebastian Schoning, ceo of Siemens client Gehring Technologies, which manufactures precision honing tools, told me that it was easier to sell these digital services to its own client base because so much of their current products include Siemens devices, giving them confidence in the new offerings. GE enjoys the same advantages of combining manufacturing and digital services with its Evolution Series locomotives.

The key to Siemens’s Mobility Services is Sinalytics, its platform architecture for data analysis not just for rail, but also for industries ranging from medical equipment to wind farms. More than 300,000 devices currently feed real-time data to the platform,   Consistent with my IoT-centric “Circular Company” vision, Sinalytics capitalizes on the data for multiple uses, including connectivity, data integration, analytics, and the all-important cyber security — they call the result not Big Data, but Smart Data. As with data services from jet turbine manufacturers such as Rolls Royce and GE, the platform also allows merging the data with data from sources such as weather forecasts which, in combination, can let clients optimize operating efficiency on a real-time M2M basis.  

With the new approach, trains become IoT laboratories on wheels, combining all of the key elements of an IoT system:

  • Sensing: there are sensors on the engines and gearboxes, plus vibration sensors on  microphones measure noises from bearings in commuter trains. They can even measure how engine oil is aging, so it can be changed when really needed, rather than on an arbitrary schedule.
  • Algorithms to make sense of the data and act on it. They read out patterns, record deviations & compare them with train control systems or vehicles of the same type.
  • Predictive maintenance replaces scheduled maintenance, dramatically reducing down-time and catastrophic failure.For example: “There’s a warning in one of the windows (of the control center display): engine temperature unusual. ‘We need to analyze the situation in greater depth to know what to do next  — we call it  ‘root cause analysis,” (say) Vice-President for Customer Support Herbert Padinger. ‘We look at its history and draw on comparative data from the fleet as a whole.’ Clicking on the message opens a chart showing changes in temperature during the past three months. The increased heat is gradually traced to a signal assembly. The Siemens experts talk with the customer to establish how urgent the need for action is, and then takes the most appropriate steps.”  He says that temperature and vibration analyses from the critical gearboxes gives Siemens at least three days advance notice of a breakdown — plenty of time for maintenance or replacement.  Predictive maintenance is now the norm for 70-80% of Siemens’s repairs.
  • Security (especially important given all of the miles of track and large crowds on station platforms): it includes video-based train-dispatch and platform surveillance using its SITRAIL D system, as well as cameras in the trains. The protections have to run the gamut from physical attacks to cyber attacks.  For security, the data is shared by digital radio, not networks also shared by consumers.

When operations are digitized, it allows seamlessly integrating emerging digital technologies into the services. Siemens Digital Services also included augmented reality (so repair personnel can see manuals on heads-up displays), social collaboration platforms, and — perhaps most important — 3-D printing-based additive manufacturing, so that replacement parts can be delivered with unprecedented speed. 3-D printing also allows dramatic reduction in parts inventories and allows replacement of obsolete parts that may no longer be available through conventional parts depots or even — get this — to improve on the original part’s function and/or durability, based on practical experience gained from observing the parts in use.  Siemens has used 3-D printing for the past last 3 years, and it lets them assure that they will have replacements for the locomotive’s entire lifespan, which can exceed 30 years.

The results of the new approach are dramatic.

  • None of the Velaro trains that Siemens maintains for several operators have broken down since Sinalytics was implemented. Among those in Spain only 1 has left more than 15 min. behind time in 2,300 trips: .0004%!
  • Reliability for London’s West Coast Mainline is 99.7%

  • Perhaps most impressive, because of the extreme cold conditions it must endure, the reliability rate for the Velaro service in Russia is 99.9%!

Their ultimate goal is a little higher: what Siemens calls (pardon the pun) 100% Railability (TM).

And, consistent with what other companies find when they fully implement not only IoT technology, but also what I like to call “IoT Thinking,” when it does reach those previously inconceivable quality benchmarks, the company predicts that, as the software and sensors evolve, the next stage will be new business models in which billing will be determined by guaranteeing customers availability and performance.

PS: I’ll be posting more about my interviews with Siemens officials and the Gartner event in coming days.

Smart Disposables: Could This Be Birth of Internet of Everything?

Could EVERYTHING be “smart?” It may be happening sooner we thought, and with implications that are hard to fathom today.

That’s the potential with new technology pioneered by Shyam Gollakota, an assistant professor at the University of Washington.  For the first time, it would let battery- and cordless-less devices harvest signals from Wi-Fi, radio, or TV to communicate and power themselves.

Astounding!

For a long time, the most “out there” idea about IoT sensors has been Prof. Kris Pister’s “smart dust” concept, which aimed at a complete sensor/communication system in a package only one cubic millimeter in size. Pister argued that such devices would be so small and cheap that they could be installed — or perhaps even scattered — almost everywhere. The benefits could be varied and inconceivable in the past. According to Pister, possible applications could include:

  • “Defense-related sensor networks
    • battlefield surveillance, treaty monitoring, transportation monitoring, scud hunting, …
  • Virtual keyboard
    • Glue a dust mote on each of your fingernails.  Accelerometers will sense the orientation and motion of each of your fingertips, and talk to the computer in your watch.  QWERTY is the first step to proving the concept, but you can imagine much more useful and creative ways to interface to your computer if it knows where your fingers are: sculpt 3D shapes in virtual clay, play  the piano, gesture in sign language and have to computer translate, …
    • Combined with a MEMS augmented-reality heads-up display, your entire computer I/O would be invisible to the people around you.  Couple that with wireless access and you need never be bored in a meeting again!  Surf the web while the boss rambles on and on.
  • Inventory Control
    • The carton talks to the box, the box talks to the palette, the palette talks to the truck, and the truck talks to the warehouse, and the truck and the warehouse talk to the internet.  Know where your products are and what shape they’re in any time, anywhere.  Sort of like FedEx tracking on steroids for all products in your production stream from raw materials to delivered goods.
  • Product quality monitoring
    • temperature, humidity monitoring of meat, produce, dairy products
      • Mom, don’t buy those Frosted Sugar Bombs, they sat in 80% humidity for two days, they won’t be crunchy!
    • impact, vibration, temp monitoring of consumer electronics
      • failure analysis and diagnostic information, e.g. monitoring vibration of bearings for frequency signatures indicating imminent failure (back up that hard drive now!)
  • Smart office spaces
    • The Center for the Built Environment has fabulous plans for the office of the future in which environmental conditions are tailored to the desires of every individual.  Maybe soon we’ll all be wearing temperature, humidity, and environmental comfort sensors sewn into our clothes, continuously talking to our workspaces which will deliver conditions tailored to our needs.  No more fighting with your office mates over the thermostat.
  • Interfaces for the Disabled (courtesy of Bryndis Tobin)
    • Bryndis sent me email with the following idea: put motes “on a quadriplegic’s face, to monitor blinking & facial twitches – and send them as commands to a wheelchair/computer/other device.”  This could be generalized to a whole family of interfaces for the disabled.  Thanks Bryndis!”

Now imagine that a critical component of such a tiny, ubiquitous device was removed. Because it didn’t need a battery it could be even smaller and cheaper (because of cheaper and simpler radio hardware circuitry).

The goal is having billions of disposable devices start communicating,” Gollakota said (my emphasis).

You may remember that I’ve written before about my metaphor of a pre-IoT era of “Collective Blindness,” the universal inability to peer (literally or figuratively) inside things in the past, which forced us to create all sorts of work-arounds to cope with that lack of real-time data. Imagine how precise our knowledge about just about everything will be if Gollakota’s technology becomes commonplace.

.As Technology Review reported, the critical challenge is making it possible for a device lacking a traditional power source to communicate: “Transferring power wirelessly is not a new trick. But getting a device without a conventional power source to communicate is harder, because generating radio signals is very power-intensive and the airwaves harvested from radio, TV, and other telecommunication technologies hold little energy.”

The principle making the innovation possible is “backscattering,” reflecting waves, particles or signals back in the direction they came from, which creates a new signal.

The early results are encouraging. Gollakata has made a contact lens that can connect with a smartphone. Think I’ll pass on that one, but other devices he and his team have created include brain implants and “a flexible skin patch that can sense temperature and respiration, a design that could be used to monitor hospital patients.”  Marketers will love this one: a concert poster broadcasting a bit of the featured band’s music over FM radio!

Jeeva Wireless, Gollakata’s commercial spinoff, is using a variety of the technology, “passive Wi-Fi.” Devices using it can data up to 100 feet and connect through walls.

Tiny passive devices using backscatter could be manufactured for as little as a dollar. “In tomorrow’s smart home, security cameras, temperature sensors, and smoke alarms should never need to have their batteries changed.”

Gollakata sums up the potential impact: “We can get communication for free” (my emphasis).

That’s incredible, but in light of the continuing series of major DDoS attacks made possible by weak or non-existent IoT security measures, I must remind everyone that speed, power, and ubiquity aren’t everything: we also need IoT security, so I hope the low cost and ability to function without a dedicated energy source won’t obscure that need as well.


 

BTW: a MIT profile on Gollakata mentions one of his other, related, inventions, which I think would mesh beautifully with my SmartAging vision to help seniors age in place in better health.

It’s called  WiSee, which uses wireless signals such as Wi-Fi to “enable whole-home sensing and recognition of human gestures. Since wireless signals do not require line-of-sight and can traverse through walls, WiSee can enable whole-home gesture recognition using few wireless sources (e.g., a Wi-Fi router and a few mobile devices in the living room).”

I love the concept for seniors, because (like Echo, which I’m finally getting!!) it doesn’t require technical expertise, which many seniors lack and/or find intimidating, to launch and direct automated devices. In this case, the activation is through sensing and recognition of human gestures. According to Gollakata,“’Gestures enable a whole new set of interaction techniques for always-available computing embedded in the environment. As an example, he suggests that a hand swiping motion in the air could enable a user to control the radio volume while showering – or change the song playing on the stereo in the living room while you are cooking in the kitchen.”

He goes on to explain:

“…. that the approaches offered today to enable gesture recognition – by either installing cameras throughout a home/office or outfitting the human body with sensing devices – are in most cases either too expensive or unfeasible. So he and his group members are skirting these issues by taking advantage of the slight changes in ambient wireless signals that are created by motion. Since wireless signals do not require line-of-sight and can traverse through walls, he and his group have achieved the first gesture recognition system that works in those situations. ‘We showed that this approach can extract accurate information about a rich set of gestures from multiple concurrent users.”

Combine that with speaking to Alexa, and even the most frail seniors could probably control most of the functions in a smart home. Gollakota says that the approaches offered today to enable gesture recognition – by either installing cameras throughout a home/office or outfitting the human body with sensing devices – are in most cases either too expensive or unfeasible. So he and his group members are skirting these issues by taking advantage of the slight changes in ambient wireless signals that are created by motion. Since wireless signals do not require line-of-sight and can traverse through walls, he and his group have achieved the first gesture recognition system that works in those situations. “We showed that this approach can extract accurate information about a rich set of gestures from multiple concurrent users, “he says.

Incredible work, professor!

I’ll be on live Thursday morning talking the IoT and Smart Cities

Cities are the future of global civilization and the economy, and smart cities are the only way they’ll survive and prosper!

Join me and two SAP experts on the subject, Dina Dayal (global vice president for Digital Enterprise Platform Group) and Saj Kumar (vice president of Digital Transformation and Internet of Things) as we guest on Bonnie D. Graham’s always-enjoyable Coffee Break With Game Changers, 11 AM EDT, 8 AM PDT (it will be archived at the site if you can’t listen live.

Bonnie likes us to start with a provocative (and relevant) quote, and mine will be from Jane Jacobs’ great Death and Life of American Cities:

Cities have the capability of providing something for everybody, only because,
and only when, they are created by everybody.”

… with the emphasis on everybody: I’ll explain that there really is an important role in smart cities for city government, the private sector, and — often ignored — grassroots innovators.

A critical key is the global Things Network, created by Wienke Gieseman and his Gang of Ten in Amsterdam,  who created a free LoRaWAN city-wide data network for $12,000 and in less than a month, and then went on to create a global network and a crowdsourced campaign to bring the cost of LoRaWAN hubs down to $200.

I like to think I was there at the beginning, working with Vivek Kundra, then the DC’s CTO (before his accomplishments there led Obama to name him the first US CIO). Vivek and Mayor Fenty took the bold move of releasing more than 40 major city data bases on a real-time basis, then held a contest to get smart developers to create new-fangled “apps” (remember, this was 2008!) to capitalize on them. Because the apps were open-source, they’ve been constantly copied and improved in the years since then.

And that’s only the beginning:

  • creative startups such as Alicia Asin’s Libelium, working with an enlightened city government, have made Barcelona a massive testlab for the Iot, and arguably THE smart city of the day
  • Columbus OH won the Obama Administration’s Smart City competition for its all-inclusive transportation scheme (and I do mean all-inclusive: who ever thought a better transportation network could be used to cut infant mortality???)
  • Smart Cities organizations have been formed in cities worldwide to share ideas — we’re all in this together!

And, of course, I’m going to bring the discussion down to earth by really getting down and dirty — yessiree, we’re gonna talk trash cans.

Be there or be square!

 

The Internet of Things Enables Precision Logistics (& Could Save Planet!)

A degree of precision in every aspect of the economy impossible before the IoT is one of my fav memes, in part because it should encourage companies that have held back from IoT strategies to get involved now (because they can realize immediate benefits in lower operating costs, greater efficiency, etc.), and because it brings with it so many ancillary benefits, such as reduced environmental impacts (remember: waste creation = inefficiency!).

       Zero Marginal Cost Society

Zero Marginal Cost       Society

I’m reminded of that while reading Jeremy Rifkin’s fascinating Zero Marginal Cost Economy which I got months ago for research in writing my own book proposal and didn’t get around to until recently.  I’d always heard he was something of an eccentric, but, IMHO, this one’s brilliant.  Rifkin’s thesis is that:

“The coming together of the Communications Internet with the fledgling Energy Internet and Logistics Internet in a seamless twenty-first-century intelligent infrastructure, “the Internet of Things (IoT),” is giving rise to a Third Industrial Revolution. The Internet of Things is already boosting productivity to the point where the marginal cost of producing many goods and services is nearly zero, making them practically free.”

Tip: when the marginal cost of producing things is nearly zero, you’re gonna need a new business model, so get this book!

At any rate, one of the three revolutions he mentioned was the “Logistics Internet.”

I’m a nut about logistics, especially as it relates to supply chain and distribution networks, which I see as crucial to the radically new “circular enterprise” rotating around a real-time IoT data hub. Just think how efficient your company could be if your suppliers — miles away rather than on the other side of the world, knew instantly via M2M data sharing, what you needed and when, and delivered it at precisely the right time, or if the SAP prototype vending machine notified the dispatcher, again on a M2M basis, so that delivery trucks were automatically re-routed to machine that was most likely  to run out first!

I wasn’t quite sure what Rifkin meant about a Logistics Internet until I read his reference to the work of Benoit Montreuil, “Coca-Cola Material Handling & Distribution Chair and Professor” at Georgia Tech, who, as Rifkin puts it, closes the loop nicely in terms of imagery:

“.. just as the digital world took up the superhighway metaphor, now the logistics industry ought to take up the open-architecture metaphor of distributed Internet communication to remodel global logistics.”

Montreuil elaborates on the analogy (and, incidentally, places this in the context of global sustainability, saying that the current logistics paradigm is unsustainable), and paraphrases my fav Einstein saying:

“The global logistics sustainability grand challenge cannot be addressed through the same lenses that created the situation. The current logististics paradigm must be replaced by a new paradigm enabling outside-the-box paradigm enabling meta-systemic creative thinking.”

wooo: meta-systemic creative thinking! Count me in!

Montreuil’s answer is a “physical Internet” for logistics, which he says is a necessity not only because of the environmental impacts of the current, inefficient system (such as 14% of all greenhouse gas emissions in France), but also its ridiculous costs, accounting for 10% of the US GDP according to a 2009 Department of Transportation report!  That kind of waste brings out my inner Scotsman!

Rifkin cites a variety of examples of the current system’s inefficiency based on Montreuil’s research:

  • trucks in the US are, on average, only 60% full, and globally the efficiency is only 10%!
  • in the US, they were empty 20% of miles driven
  • US business inventories were $1.6 trillion as of March, 2013 — so much for “just-in-time.”
  • time-sensitive products such as food, clothes and medical supplies are unsold because they can’t be delivered on time.

Montreuil’s “physical Internet” has striking parallels to the electronic one:

  • cargo (like packets) must be packaged in standardized module containers
  • like the internet, the cargo must be structured independently of the equipment, so it can be processed seamlessly through a wide range of networks, with smart tags and sensors for identification and sorting (one of the first examples of the IoT I wrote about was FedEx’s great SenseAware containers for high-value cargo!)

With the Logistics Internet, we’d move from the old point-to-point and hub-and-spoke systems to ones that are “distributed, multi-segment, intermodal.” A single, exhausted, over-worked (and more accident-prone) driver would be replaced by several. It’s a  little counter-intuitive, but Montreuil says that while it would take a driver 240 hours to get from Quebec to LA under the current system, instead 17 drivers in a distributed one would each drive about 3 hours, and the cargo would get there in only 60 hours.

Under the new system, the current fractionated, isolated warehouse and distribution mess would be replaced by a fully-integrated one involving all of the 535,000 facilities nationwide, cutting time and dramatically reducing environmental impacts and fuel consumption.

Most important for companies, and looping back to my precision meme, “Montreuil points out that an open supply network allows firms to reduce their lead time to near zero if their stock is distributed among some of the hundreds of distribution centers that are located near their final buyer market.” And, was we have more 3-D printing, the product might actually be printed out near the destination. How cool is that?

Trucking is such an emblematic aspect of the 20th-century economy, yet, as with the neat things that Union Pacific and other lines are doing with the 19th-century’s emblematic railroads, they can be transformed into a key part of the 21-st century “precision economy” (but only if we couple IoT technology with “IoT thinking.”

Now let’s pick up our iPads & head to the loading dock!


 

PS: I’ll be addressing this subject in one of my two speeches at the SCM2016 Conference later this month. Hope to see you there! 

 

FedEx package…

I’ll Speak Twice at Internet of Things Global Summit Next Week

I always love the Internet of Things Global Summit in DC because it’s the only IoT conference I know of that places equal emphasis on both IoT technology and public policy, especially on issues such as security and privacy.

At this year’s conference, on the  26th and 27th, I’ll speak twice, on “Smart Aging” and on the IoT in retailing.

2015_IoT_SummitIn the past, the event was used to launch major IoT regulatory initiatives by the FTC, the only branch of the federal government that seems to really take the IoT seriously, and understand the need to protect personal privacy and security. My other fav component of last year’s summit was Camgian’s introduction of its Egburt, which combines “fog computing,” to analyze IoT data at “the edge,” and low power consumption. Camgian’s Gary Butler will be on the retail panel with me and with Rob van Kranenburg, one of the IoT’s real thought leaders.

This year’s program again combines a heady mix of IoT innovations and regulatory concerns. Some of the topics are:

  • The Internet of Things in Financial Services and the Insurance sector (panel includes my buddy Chris Rezendes of INEX).
  • Monetizing the Internet of Things and a look at what the new business models will be
  • The Connected Car
  • Connected living – at home and in the city
  • IoT as an enabler for industrial growth and competition
  • Privacy in a Connected World – a continuing balancing act

The speakers are a great cross-section of technology and policy leaders.

There’s still time to register.  Hope to see you there!

 

 

AliveCor Mobile ECG: the IoT Can Save Your Life!

Got your attention? I find there’s nothing like the fear of death to focus one’s attention.

AiiveCor

AliveCor

Somehow I managed to forget blogging about one of the real highlights of last Spring’s RE-WORK Connect Summit here in Boston: the AliveCor Mobile ECG.*

Perhaps the most important thing about the Mobile ECG is that it is not just a helpful Quantified Self fitness device, but has past the rigors of the FDA licensing process, building both users’ and docs’ confidence in its reliability as a diagnostic tool, and also underscoring that  IoT devices can be significant parts of mobile health strategies. As Dr. Albert said to Forbes, ““No one cares whether their Fitbit is accurate or not …. A point of here or there. With ECGs, that’s different.”  In 2015 the FDA also approved an algorithm instantly letting you know if your reading was normal.

Because of the FDA approval, I put the Alive ECG in that special category of IoT devices and services that are important both in their own right and because of their symbolic role, especially when they meet my test of the IoT allowing “what can you do that you couldn’t do before,” in this case, a self-administered device that isn’t just generally informative about your fitness level, but also gives reliable medical documentation (especially since this allows that documentation to come as part of your activities of daily living, not requiring you to be in the artificial setting of a doctor’s office or hospital). 

I see it as a critical tool in my “Smart Aging” paradigm.

Atrial fibrillation (a common abnormal heart rhythm), the condition the ECG documents, is a huge, and growing, problem. The latest figures I could find, from four years ago, show that people who suffer from it are hospitalized twice as frequently as those who don’t have it, and the annual costs in the US alone are $26 billion.

I found the price on Froogle as low as $86 for one to fit a 5s. Sweeeet!

Here’s how it works.  The AliveCor is always available when you suspect you may have a heart problem, because it’s your smart-phone’s case! How brilliant is that?  You just rest the two metal pads on your fingers or chest to record an ECG in 30 seconds.

AliveCor ap reading

AliveCor ap reading

AliveCor has recently beefed up its app by adding the “Heart Journal.” After each reading, you just tap on a Symptom, Activity or Diet tag to add it to your recording, or, like a lot of Quantified Self apps, you can also add in notes between readings about possible indicators such as what you’re eating or your activities. The Beat Fluctuation feature lets you see how your heartbeat changes from beat to beat.

I couldn’t help but think how the AliveCor would have helped me last Winter, when Boston endured the 1-in-26,315-years-Winter-From-Hell (nope: no typo!) .  Like everyone else, I was perilously perched on my ladder, 20′ high, sticking my left hand through the ladder to pound away at an ice dam to my right with a REALLY heavy sledge hammer.  Unlike many others doing the same thing, I’m old enough (ahem..) that this counted as Risky Business.  After several hours, I started to feel chest pain.  Two days and many heart tests later, I emerged from the hospital with my own diagnosis confirmed: just a muscle strain caused by the weird position of my hammering. Couldn’t help thinking that if I’d had an AliveCor on my phone, I could have just whipped it out, taken a reading while on the ladder, and, as the web site sez,” AliveCor’s FDA-cleared Normal Detector will determine right away when your ECG is normal,” and gone back to chipping away!

Loved this quote about the AliveCor’s significance:

“Just as the introduction of thermometers and blood pressure cuffs in the past century helped patients to monitor their health, now the ability to record one’s own electrocardiogram – and get an interpretation instantly – empowers the 21st century patient to take charge of their heart health.” –Ronald Karlsberg, MD Clinical Professor of Medicine, Cedars-Sinai Heart Institute


 

*in my defense, I was mesmerized by AliveCor founder  Dr. David Albert’s colorful bowties….