OtoSense: the next level in sound-based IoT

It sounds (pardon the pun) as if the IoT may really be taking off as an important diagnostic repair tool.

I wrote a while ago about the Auguscope, which represents a great way to begin an incremental approach to the IoT because it’s a hand-held device to monitor equipment’s sounds and diagnose possible problems based on abnormalities.

Now NPR reports on a local (Cambridge) firm, OtoSense, that is expanding on this concept on the software end. Its tagline is “First software platform turning real-time machine sounds and vibrations into actionable meaning at the edge.”

Love the platform’s origins: it grows out of founder Sebastien Christian’s research on deafness (as I wrote in my earlier post, I view suddenly being able to interpret things’ sounds as a variation on how the IoT eliminates the “Collective Blindness”  that I’ve used to describe our past inability to monitor things before the IoT’s advent):

“[Christian} … is a quantum physicist and neuroscientist who spent much of his career studying deaf children. He modeled how human hearing works. And then he realized, hey, I could use this model to help other deaf things, like, say, almost all machines.”

(aside: I see this as another important application of my favorite IoT question: learning to automatically ask “who else can use this data?” How does that apply to YOUR work? But I digress).

According to Technology Review, the company is concentrating primarily on analyzing car sounds from IoT detectors on the vehicle at this point (working with a number of car manufacturers) although they believe the concept can be applied to a wide range of sound-emitting machinery:

“… OtoSense is working with major automakers on software that could give cars their own sense of hearing to diagnose themselves before any problem gets too expensive. The technology could also help human-driven and automated vehicles stay safe, for example by listening for emergency sirens or sounds indicating road surface quality.

OtoSense has developed machine-learning software that can be trained to identify specific noises, including subtle changes in an engine or a vehicle’s brakes. French automaker PSA Group, owner of brands including Citroen and Peugeot, is testing a version of the software trained using thousands of sounds from its different vehicle models.

Under a project dubbed AudioHound, OtoSense has developed a prototype tablet app that a technician or even car owner could use to record audio for automated diagnosis, says Guillaume Catusseau, who works on vehicle noise in PSA’s R&D department.”

According to NPR, the company is working to apply the same approach to a wide range of other types of machines, from assembly lines to DIY drills. As always with IoT data, handling massive amounts of data will be a challenge, so they will emphasize edge processing.

OtoSense has a “design factory” on the site, where potential customers answer a variety of questions about the sounds they must monitor (such as whether the software will be used indoors or out, whether it is to detect anomalies, etc. that will allow the company to choose the appropriate version of the program.

TechCrunch did a great article on the concept, which underscores really making sound detection precise will take a lot of time and refinement, in part because of the fact that — guess what — sounds from a variety of sources are often mingled, so the relevant ones must be determined and isolated:

“We have loads of audio data, but lack critical labels. In the case of deep learning models, ‘black box’ problems make it hard to determine why an acoustical anomaly was flagged in the first place. We are still working the kinks out of real-time machine learning at the edge. And sounds often come packaged with more noise than signal, limiting the features that can be extracted from audio data.”

In part, as with other forms of pattern recognition such as voice, this is because it will require accumulating huge data files:

“Behind many of the greatest breakthroughs in machine learning lies a painstakingly assembled dataset.ImageNet for object recognition and things like the Linguistic Data Consortium and GOOG-411 in the case of speech recognition. But finding an adequate dataset to juxtapose the sound of a car-door shutting and a bedroom-door shutting is quite challenging.

“’Deep learning can do a lot if you build the model correctly, you just need a lot of machine data,’ says Scott Stephenson, CEO of Deepgram, a startup helping companies search through their audio data. ‘Speech recognition 15 years ago wasn’t that great without datasets.’

“Crowdsourced labeling of dogs and cats on Amazon Mechanical Turk is one thing. Collecting 100,000 sounds of ball bearings and labeling the loose ones is something entirely different.

“And while these problems plague even single-purpose acoustical classifiers, the holy grail of the space is a generalizable tool for identifying all sounds, not simply building a model to differentiate the sounds of those doors.

…”A lack of source separation can further complicate matters. This is one that even humans struggle with. If you’ve ever tried to pick out a single table conversation at a loud restaurant, you have an appreciation for how difficult it can be to make sense of overlapping sounds.

Bottom line: there’s still a lot of theoretical and product-specific testing that must be done before IoT-based sound detection will be an infallible diagnostic tool for predictive maintenance, but clearly there’s precedent for the concept, and the potential payoff are great!

 


LOL: as the NPR story pointed out, this science may owe its origins to two MIT grads of an earlier era, “Click” and “Clack” of Car Talk, who frequently got listeners to contribute their own hilarious descriptions of the sounds they heard from their malfunctioning cars.   BRTTTTphssssBRTTTT…..

#IoT Sensor Breakthroughs When Lives Are On the Line!

One of my unchanging principles is always to look to situations where there’s a lot at stake — especially human lives — for breakthroughs in difficult issues.

Exhibit A of this principle for the IoT is sensor design, where needing to frequently service or recharge critical sensors that detect battlefield conditions can put soldiers’ lives at stake (yes, as long-time readers know, this is particularly of interest to me because my Army officer son was wounded in Iraq).

FedTech reports encouraging research at DARPA on how to create sensors that have ultra-low power requirements, can lie dormant for long periods of time and yet are exquisitely sensitive to critical changes in conditions (such as vehicle or troop movements) that might put soldiers at risk in battlefield conditions.

The  N-ZERO (Near Zero RF and Power Operations)  program is a three-year initiative to create new, low-energy battlefield sensors, particularly for use at forward operating bases where conditions can change quickly and soldiers are constantly at risk — especially if they have to service the sensors:

“State-of-the-art military sensors rely on “active electronics” to detect vibration, light, sound or other signals for situational awareness and to inform tactical planning and action. That means the sensors constantly consume power, with much of that power spent processing what often turns out to be irrelevant data. This power consumption limits sensors’ useful lifetimes to a few weeks or months with even the best batteries and has slowed the development of new sensor technologies and capabilities. The chronic need to service or redeploy power-depleted sensors is not only costly and time-consuming but also increases warfighter exposure to danger.”

…. (the project has) the goal of developing the technological foundation for persistent, event-driven sensing capabilities in which the sensor can remain dormant, with near-zero power consumption, until awakened by an external trigger or stimulus. Examples of relevant stimuli are acoustic signatures of particular vehicle types or radio signatures of specific communications protocols. If successful, the program could extend the lifetime of remotely deployed communications and environmental sensors—also known as unattended ground sensors (UGS)—from weeks or months to years.”

A key goal is a 20-fold battery size reduction while still having the sensor last longer.

What cost-conscious pipeline operators, large ag business or “smart city” transportation director wouldn’t be interested in that kind of product as well?

According to Signal, the three-phase project is ahead of its targets. In the first part, which ended in December, the DARPA team created “zero-power receivers that can detect very weak signals — less than 70 decibel-milliwatt radio-frequency (RF) transmissions, a measure that is better than originally expected.” This is critical to the military (and would have huge benefits to business as well, since monitoring frequently must be 24/7 but reporting of background data  (vs. significant changes) would both deplete batteries while requiring processing of huge volumes of meaningless data). Accordingly, a key goal would be to create “… radio receivers that are continuously alert for friendly radio transmissions, but with near zero power consumption when transmissions are not present.” A target is  “exploitation of the energy in the signal signature itself to detect and discriminate the events of interest while rejecting noise and interference. This requires the development of passive or event-powered sensors and signal-processing circuitry. The successful development of these techniques and components could enable deployments of sensors that can remain “off” (that is, in a state that does not consume battery power), yet alert for detecting signatures of interest, resulting in greatly extended durations of operation.”

The “exploitation of .. energy in the signal signature itself sounds reminiscent of the University of Washington research I’ve reported in the past that would harness ambient back-scatter to allow battery-less wireless transmission, another key potential advance in IoT sensor networks.

The following phrases of N-ZERO will each take a year.

Let’s hope that the project is an overall success, and that the end products will also be commercialized. I’ve always felt sensor cost and power needs were potential IoT Achilles’ heels, so that would be a major boost!

IoT: LiveBlogging PTC’s LiveWorx

Got here a little late for CEO Jim Heppelman’s keynote, so here goes!

  • Vuforia: digital twin gives you everything needed for merging digital “decorations” on the physical object
  • Unique perspective: AR takes digital back to the physical. Can understand & make better decisions.
  • Virtual reality would allow much of the same. Add in 3-D printing, etc.
  • “IoT is PLM.” Says PTC might be only company prepared to do both.
  • Says their logo captures the merger of digital and physical.
  • Case studies: they partnered with Bosch’s Rexroth division. Cytropac built-in IoT connectivity–  used Creo. Full life-cycle management. Can identify patterns of usage, etc. Using PTC’s analytics capacity, machine learning analysis. Want to improve cooling efficiency (it was high at first). Model-based digital twin to monitor product in field, then design an upgrade. How can they increase cooling efficiency 30%??  Came up with new design to optimize water channel that they will build in using 3-D printing. Cool (literally!). 43% increase in cooling efficiency. The design change results in new recommendation engine that helps in sales. Replaced operating manual with 3-D that anyone can understand. (BTW: very cool stagecraft: Heppelmann walks around stage interviewing the Rexroth design team at their workstations).
  • Ooh: getting citizen developers involved!!!  Speeds process, flexibility. App shows how products are actually operating in the field. Lets sales be much more proactive in field. Reinventing CRM.  May no longer need a physical showroom — just put on the AR headset.
  • Connectivity between all assets. The digital twin is identical, not fraternal. Brings AR into factory. They can merge new manufacturing equipment with legacy ones that didn’t have connectivity.  ABB has cloud-based retrofit sensors. Thingworx can connect almost anything, makes Industry 4.0 possible. Amazing demo of a simulated 3-D disassembly and replacement.
  • Hmmm — closing graphic of his preso is a constantly rotating circular one. Anticipating my “circular company” talk on Wednesday????

Closing the Loop With Enterprise Change Management. Lewis Lawrence of Weatherford, services to petroleum industry:

  • former engineer. In charge of Weatherford’s Windchill installation (they also use Creo).
  • hard hit by the drop in gas prices
  • constant state of flux
  • 15 years of constant evolution
  • their mantra: design anywhere, build anywhere.
  • enterprise change — not just engineering.
  • hmmm: according to his graphics, their whole change process is linear. IMHO, that’s obsolete in era of constant change: must evolve to cyclical. Ponderous process…
  • collect data: anything can be added, if it’s latest

The IoT Can Even Help You Breathe Better: GCE Group’s Zen-O portable oxygen concentrator for people with respiratory problems (not actually launched yet):

  • InVMA has built IoT application using ThingWorx to let patients, docs and service providers carefully monitor data
  • GCE made radical change from their traditional business in gas control devices. Zen-O is in the consumer markets. They were very interested in connected products — especially since their key competitor launched one!
  • Goals: predictive maintenance, improved patient care, asset management, development insight.
  • Design process very collaborative, with many partners.

The Digital Value Chain: GE’s Manufacturing Journey. Robert Ibe, global IT Engineering Leader at GE Industrial Solutions:

  • supports Brilliant Factory program.
  • they design and manufacture electrical distribution equipment, 30 factories worldwide.
  • “wing-to-wing” integrated process
  • had a highly complex, obsolete legacy
  • started in 2014: they were still running really old CAD technology. 14 CAD repositories that didn’t talk to each other. 15 year old PLM software. No confidence in any of data they had.
  • They began change with PLM — that’s where the digital thread begins.  PLM is foundation for their transformation.
  • PLM misunderstood: use it to map out cohesive, cross-functional, model-based strategy. Highlight relevance of “design anywhere — manufacture anywhere.” Make PLM master of your domain. Make it critical to commercial & manufacturing. Advertise benefits & value.
  • Whole strategy based on CAD. Windchill heart of the process.
  • Rate of implementation faster than business can keep up with!
  • Process: implementation approach:
    • design systems integration
    • model-based design
    • digital thread
    • manufacturing productivity.
  • common enterprise PLM framework
  • within Windchill, can see entire “digital bill of documents.”
  • focused on becoming critical for supply chain.
  • total shift from their paper-based legacy.
  • integrated regulatory compliance with every step of design.

It’s Not Your Grandmother’s IoT: Blockchain and IoT Morph Into An Emerging Technology Powerhouse:

  • Example of claims for fair-traded coffee that I’ve used in past

Finding Business Value in IoT panel:

  • Bayer — been in IoT (injection devices for medicine) for 7 years.  Reduced a lot of parts inventory.
  • Remote control of vending machines replaces paper & pencil
  • Your team needs to evangelize for biz benefits of IoT
  • New Opportunities:
    • vision and language
    • interacting with physical world
    • problem solving.
  • Didn’t know!  Skype can do real-time translation.
  • Google Deep Mind team worked internally, cut energy costs at its server farms. 15% energy reduction.
  • Digital progress makes economic pie bigger, BUT  most people aren’t benefitting economicallly. Some may be worse off. “Great decoupling” — mushrooming economic gap. One reason is that tech affects different groups differently.
  • “Entirely possible to create inclusive prosperity” through tech!

 

WEDNESDAY

Delivering Smart City Solutions and an Open Citywide Platform to Accelerate Economic Growth and Promote New Solution Innovation, Scott McCarley, PTC:

  • $40 trillion potential benefits from smart cities
  • 1st example & starting point for many cities, is smart lightpoles. Major savings plus value added. Real benefit is building on that, with systems of systems (water, traffic, energy, etc.) — the systems don’t operate in isolation.
  • Future buildings may have built-in batteries to add to power supply. Water reclamation, etc.
  • Cities are focused on KPIs across all target markets.
  • Cornerstone systems for a city: power & grid, water/wastewater, building management, city services & infrastructure.
  • Leveraging ThingWorx to address these needs:
    • deploy out-of-box IoT solutions from a ThingWorx Solution Provider: All examples, include Aquamatix, DEPsys (grid), Sensus, All Traffic, Smoove (bike sharing).
    • leverage ThingWorx to rapidly develop new IoT solutions.
      connect to any device, rapidly develop applications, visually model systems, quickly develop new apps. Augmented reality will play a role!
    • create role-based dashboards:
      one for your own operations, another for city.
    • bring the platform to create a citywide platform.
      Sum of connected physical assets, communication networks, and smart city solutions.

Digital Supply Networks: The Smart Factory. Steven Shepley, Deloitte:

  • 3 types of systems: 1) foundational visualization solutions:  KPIs, etc. 2) advanced analytical solutions 3) cyber-physical solutions.
  • Priority smart factory solutions:
    • advanced planning (risk-adjusted MRP), dynamic sequencing, cross network.
    • value chain integration: signal-based customer/supplies integration, dynamic distribution routing/tracking, digital twin.
    • asset efficiency: predictive maintenance, real-time asset tracking intelligence, energy management
    • labor productivity: robotic and cognitive automation, augmented reality-driven efficiency, real-time safety monitoring
    • exponential tech: 3-D printing, drones, flexible robots.
  • How to be successful: think big, start small, scale fast
  • Act differently: multi-disciplinary teams,
  • sensors getting simpler, easier to connect & retrofit. National Connectors particularly good.

Global Smart Home, Smart Enterprise, and Smart Cities IoT Use Cases. Ken Herron, Unified InBox, Pte.

  • new focus on customer
  • H2M: human to machine communication is THE key to IoT success. Respect their interests.
  • Austin TX: “robot whisperer” — industrial robot company. Their robots aging out, getting out of tune, etc. Predictive analytics anticipates problems.
  • Stuttgart: connected cow — if one cow is getting sick, may spread to entire herd. Intervene.
  • Kuala Lumpur: building bot — things such as paper towel dispensers communicating with management.
  • London: Concierge chatbot — shopper browsing can chat with assistant on combining outfits.
  • Dubai: smart camera. Help find your car in mega-shopping center: read license plates, message the camera, it gives you map to the car.
  • Singapore: Shout — for natural disasters. Walks the person making the alert through process, confirms choices.
  • Stuttgart: Feinstaubalarm — occasional very bad airborne dust at certain times. Tells people with lung problems options, such as taking mass transit.
  • Singapore: Smart appliances — I always thought smart fridge was stupid, but in-fridge camera that lets you shoot a “shelfie” does make sense
  • Fulda Germany: smart clothing for military & police: full record of personal health at the moment. Neat!
  • Noida India — smart sneakers can automatically post your run results (see connection to my SmartAging concept)

Business Impact of IoT, Eric Schaeffer, Accenture:

  • Michelin delivery trucks totally reinvented, major fuel savings, other benefits.
  • manufacturing being deconstructed
  • smart, connected products are causing it
  • industrial companies must begin transformation today

Thingworx: Platform for Management Revolution. W. David Stephenson, Stephenson Strategies:

Here are key points from my presentation about how the IoT can allow radical transformation from linear & hierarchical companies to IoT-centric “circular companies” (my entire presentation can be found here):

  • The IoT can be the platform for dramatic management change that was impossible in the past.
  • Making this change requires an extraordinary shift in management thinking: from hierarchy to collaboration.
  • The results will be worth the effort: not only more efficiency & precision, but also new creativity, revenue streams, & customer loyalty. 
  • In short, it will allow total transformation!

Kickstarting America’s Digital Transformation. Aneesh Chopra & Nicholas Thompson!

  • on day one, Our President (not the buffoon) told Chopra he wanted default to be switch from closed to open government & data.
  • National Wireless Initiative: became law 1 yr. after it was introduced.  Nationwide interoperable, secure wireless system.
  • Obama wanted to harness power of Internet to grow the economy. Talked to CIO of P & G, who was focused on opening up the company to get ideas from outside.
  • Thompson big on open data, but he thinks a lot more now is closed, we’re going wrong way.
  • Interesting example of getting down cost of solar to $1 per installed watt!!
  • Thompson: growing feeling that technology isn’t serving us economically. Chopra: need to democratize the benefits.
  • Chopra talking about opening up Labor Dept. data to lead to creative job opportunities for underserved.

 

 

 

 

ThingWorx Analytics Video: microcosm of why IoT is so transformative!

I’ll speak at PTC’s LiveWorx lollapalooza later this month (ooh: act quickly and I can get you a $300 registration discount: use code EDUCATE300) on my IoT-based Circular Company meme, so I’ve been devouring everything I can about ThingWorx to prepare.

Came across a nifty 6:09 vid about one component of ThingWorx, its Analytics feature. It seems to me this video sez it all about both how you can both launch an incremental IoT strategy (a recent focus of mine, given my webinar with Mendix) that will begin to pay immediate benefits and can serve as the basis for more ambitious transformation later, especially because you’ll already have the analytical tools such as ThingWorx Analytics already installed.

What caught my eye was that Flowserve, the pump giant involved in this case, could retrofit existing pumps with retrofit sensors from National Instruments — crucial for two reasons:

  • you may have major investments in existing, durable machinery: hard to justify scrapping it just to take advantage of the IoT
  • relatively few high-end, high-cost machinery and devices have been redesigned from the ground up to incorporate IoT monitoring and operations.

Note the screen grab: each of these sensors takes 30,000 readings per second. How’s that for real-time data?  PTC refers to this as part of the “volume, velocity and variety challenge of data” with the IoT.

As a microcosm of the IoT’s benefits, this example shows how easy it is to use those massive amounts of data and how they can be used to improve understanding and performance.

There are three major components:

  • ThingWatcher:
    This is the most critical component, because it sifts through the incredible amount of data from the edge, learns what constitutes normal performance for that sensor (creating “pop-up learning flags”), and then monitors it future performance for anomalies and, as the sample video shows, delivers real-time alerts to users (without requiring human monitoring) so they can make adjustments and/or order repairs.  Finds anomalies from edge devices in real-time. Automatically observes and learns the normal state pattern for every device or sensor. It then monitors each for anomalies and delivers re- al-time alerts to end users.
  • ThingPredictor:
    For the all-important new function of predictive maintenance, two different types of ThingPredictor indicators pop up when if anomalies are detected, predicting how long it may be until failure, allowing plenty of time for less-costly, anticipatory repairs. Because the specific deviation is identified in advance, repair crews will have the needed part with them when needed, rather than having to make an additional trip back to pick up parts.

    If you ask for a standard predictive scoring you don’t specify which performance features to include and get a simple predictive score. However, you can specify several key features to evaluate and get a more detailed (and probably more helpful) answer. For example,  “if you indicate an important feature count of three, the causal scoring output will include the three most influential features for each record and the percentage weights of each feature’s influence on the score.”

  • ThingOptimizer:
    Finally, you can use “ThingOptimizer” to do some what-if calculations to decide which possible “levers,” as ThingWorx calls the key variables, could change the projections to either maximize a positive factor or minimize the negatives. “Prescriptive scoring results include both an original score (the score before any lever attributes are changed) and an optimized score (the score after optimal values are applied to the lever attributes). In addition, for each attribute identified in your data as a lever, original and optimal values are included in the prescriptive scoring results.” It sort of reminds me how the introduction of VisiCalc allowed users, for the first time, to play around with variables to see which would have the best results.
Best of all, as the video illustrates, ThingWorx Analytics would facilitate the kind of “Circular Company” I’ll address in my speech, because the exact same real-time data could simultaneously be used by operating personnel to fine tune operations and catch a problem in time for predictive maintenance, and by senior management to get an instant overview of how operations are going at all the installations. Same data, many uses.
Bottom line: a robust IoT platform could be the key to an incremental strategy to begin by improving daily operations and reducing maintenance problems, and also be the underpinning for more radical transformation as your IoT strategy becomes more advanced!  See you at LiveWorx!

Servitization With IoT: Weird Biz-Speak, But Sound Strategy

I love it when manufacturers stop selling things — and their revenues soar!

That’s one of the things I’ll cover on May 2nd  in”Define Your Breakout IoT” strategy, (sign-up) a webinar I’m doing with Mendix. I’ll outline an incremental approach to the IoT in which you can make some early, tentative steps (such as implementing Augury’s hand-held vibration sensor as a way to start predictive maintenance) and then, as you gain experience and increase savings and efficiency, plow the savings back into more dramatic transformation.

One example of the latter that I’ll detail in the webinar is one of my four “Essential Truths” of the IoT: rethink products. By that I meant not only reinventing products to be smart (especially by building in sensors so they can report their real-time status 24/7), but, having done that, exploring new ways to market them.  Or, as one graphic I’ll use in the presentation puts it, in mangled biz-speak, “servitization.”

              Hortilux bulbs

Most of the examples I’ve written about in that regard have been from major businesses, such as GE and Rolls-Royce jet turbines, that are now leased as services (with the price determined by thrust generated), but Mendix has a smaller, niche client that also successfully made the conversion: Hortilux, a manufacturer of grow lights for greenhouses.

The Hortilux decided to differentiate itself in an increasingly competitive grow light market by evolving from simply selling bulbs to instead providing a comprehensive continuing service that helps its customers optimize availability and lifetime of grow light systems, while cut energy cost.     

Using Mendix tools, they created Hortisensehttp://www.hortidaily.com/article/31774/Hortilux-launches-Hortisense-software-suite, a digital platform that monitors and safeguards various grow light processes in the greenhouse using sensors and PLCs. Software applications interpret the data and present valuable information to the grower anytime, anywhere, and on any device.

With Mendix, Hortilux created an application to collect sensor data on light, temperature, soil, weather and more. Now users can optimize plants’ photosynthesis, energy consumption, and greenhouse maintenance. Most ambitiously, it provides comprehensive “crop yield management:” 

  • Digital cultivation schedule
  • Light strategies based on plant physiology and life cycle
  • Automatic light adjustment based on predictive analytics (e.g. weather forecast, energy prices, produce prices)

The app even allows predictive maintenance, predicting bulbs’ life expectancy and notifying maintenance to replace them in time to avoid disruptions in operations.

In the days when we suffered from what I call “Collective Blindness,” when we lacked the tools to “see” inside products to m0nitor and perhaps fix them based on real-time operating data, it made sense to sell products and provide hit-or-miss maintenance when they broke down.

Now that we can monitor them 24/7 and get early enough warning to instead provide predictive maintenance, it makes equal sense to switching to marketing them as services, with mutual benefits including:

  • increased customer satisfaction because of less down-time
  • new revenues from selling customers services based on availability of the real-time data, which in turn allows them more operating precision
  • increased customer loyalty, because the customer is less likely to actually go on the open market and buy a competing product
  • the opportunity to improve operations through software upgrades to the product.

Servitization: ugly word, but smart strategy. Hope you’ll join us on the 2nd!

Sound’s emerging IoT role

Could sound be a critical IoT tool?

I’d fixated in the past on a metaphor I called “Collective Blindness,” as a way to explain how difficult it used to be to get accurate, real-time data about how a whole range of things, from tractors to your body, were actually working (or not) because we had no way to penetrate the surface of these objects as they were used. As a result, we created some not-so-great work-arounds to cope with this lack of information.

Then along came the IoT, and no more collective blindness!

Now I’m belatedly learning about some exciting efforts to use another sense, sound, for the IoT.  Most prominent, of course, is Amazon’s Alexa and her buddies (BTW, when I ask Siri if she knows Alexa, her response was an elusive “this is about you, not me,” LOL), but I’ve found a variety of start-ups pursuing quite different aspects of sound. They nicely illustrate the variety of ways sound might be used.

technician using Auguscope to detect   sound irregularities in machinery

First is Augury.

What I particularly love about their device and accompanying smartphone app it is that they are just about the lowest-cost, easiest-to-use, rapid payback industrial IoT devices I can think of.

That makes them a great choice to begin an incremental approach to the IoT, testing the waters by some measures that can be implemented quickly, pay rapid bottom-line benefits and therefore may lure skeptical senior management who might then be willing to then try bolder measures   (this incremental approach was what I outlined in my Managing the Internet of Things Revolution e-guide for SAP, and I’ll be doing a webinar on the approach in April with Mendix, which makes a nifty no-code, low-code tool).

Instead of requiring built-in sensors, an Auguscope is a hand-held device that plant personnel can carry anywhere in the building it’s needed to analyze how the HVAC system is working. A magnetic sensor temporarily attaches to the machine and the data flows from the Auguscope to the cloud where it is analyzed to see if the sound is deviating from pre-recorded normal sounds, indicating maintenance is needed. Consistent with other IoT products that are marketed as services instead of sold, it uses a “Diagnostics as a Service” model, so there are no up-front costs and customers pay as they go. The company hopes that the technology will eventually be built into household appliances such as washers and dryers.

Presenso is the second company using sound to enable predictive maintenance.  It is sophisticated cloud-based software that takes data from a wide range of already-installed sensors and interprets any kind of data: sound, temperature, voltage, etc.  It builds a model of the machine’s normal operating data and then creates visualizations when the data varies from the norm. Presenso’s power comes from combining artificial intelligence and big data.

Finally, and most creative is Chirp (hmm: Chrome wouldn’t let me enter their site, which it said was insecure. Here’s the URL:www.chirp.io/ — try at your own risk…) , a UK company that transmits data using audio clips that really sound like chirps. It’s amazing!  Check out this video of an app in India that uses sound to pay fares on the country’s version of Uber:


Another Chirp app is a godsend to all who forget Wi-Fi passwords: your phone “chirps” a secure access code, allowing you to join the network automatically.   The company has released iOS and Android versions.  As VentureBeat reported:

“Each chirp lasts a couple of seconds, and the receiving device “listens” for a handful of notes played quickly in a certain order, in a certain range, and at a certain speed. While there are other easy ways of sharing files and data in real-time, such as Bluetooth, Chirp doesn’t require devices to pair in advance, there is no need to set up an account, and it’s ultimately a much quicker way of sharing files.

“That said, with Chirp, the file itself isn’t sent peer-to-peer, and the data doesn’t actually travel directly via audio. Chirp merely decodes and encodes the file, with the associated sound serving as the delivery mechanism. A link is generated for the recipient(s) to access it on Chirp’s servers, but the process from sending to receiving is seamless and near-instant.”

In terms of IoT applications, it could also connect with physical objects (hmm: retailing uses??). The Chirp platform is so cool that I suspect it will be a global hit (the company says it’s already used in 90 countries).

So, I’ve had my senses opened: from now on, I’ll add voice and sound in general to the list of cool IoT attributes.  Because voice and sound are so ubiquitous, they really meet the late Mark Weiser’s test:  “the most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” What could be more woven into the fabric of everyday life than sound — and, potentially, more valuable?


BTW: let me put in a plug for another IoT voice product. During the past two months, I recorded 7 hours of my voice speaking a very strange mishmash of sentences drawn from, among others, Little Women, Jack London’s Call of the Wild, The Wizard of Oz, and The Velveteen Rabbit (I worried about the she-wolf sneaking up on Meg, LOL….). Using the algorithms developed for Alexa, the Vocal ID team will slice and dice my voice and create a natural sounding one for someone who cannot speak due to a birth defect or disease.  I hope you’ll join me in volunteering for this wonderful program.

IoT Intangibles: Increased Customer Loyalty

There are so many direct, quantifiable benefits of the IoT, such as increased quality (that 99.9988% quality rate at Siemens’s Amberg plant!) and precision, that we may forget there are also potential intangible benefits.

Most important of those is customer loyalty, brought about by dramatic shifts both in product designs and how they are marketed.

Much of this results from the IoT lifting the veil of Collective Blindness to which I’ve referred before: in particular, our prior inability to document how products were actually used once they left the loading dock. As I’ve speculated, that probably meant that manufacturers got deceptive information about how customers actually used products and their degree of satisfaction. The difficulty of getting feedback logically meant that those who most liked and most hated a product were over-represented: those who kinda liked it weren’t sufficiently motivated to take the extra steps to be heard.

Now, by contrast, product designers, marketers, and maintenance staffs can share (that critical verb from my Circular Company vision!) real-time data about how a product is actually operating in the field, often from a “digital twin” they can access right at their desks.

Why’s that important?

It can give them easy insights (especially if those different departments do access and discuss the data at the same time, each offering its own unique perspectives, on issues that will build customer loyalty:

  • what new features can we add that will keep them happy?
  • can we offer upgrades such as new operating software (such as the Tesla software that was automatically installed in every single car and avoided a recall) that will provide better customer experiences and keep the product fresh?
  • what possible maintenance problems can we spot in their earliest stages, so we can put “predictive maintenance” services into play at minimal cost and bother to the customer?

I got interested in this issue of product design and customer loyalty while consulting for IBM in the 9o’s, when it introduced the IBM PS 2E (for Energy & Environmental), a CES best-of-show winner in part because of its snap-together modular design. While today’s thin-profile-at-all-costs PC and laptop designs have made user-friendly upgrades a distant memory, one of the things that appealed to me about this design was the realization that if you could keep users satisfied that they were on top of  new developments by incremental substitution of new modules, they’d be more loyal and less likely to explore other providers.

In the same vein, as GE has found, the rapid feedback can dramatically speed upgrades and new features. That’s important for loyalty: if you maintain a continuing interaction with the customer and anticipate their demands for new features, they’ll have less reason to go on the open market and evaluate all of your competitors’ products when they do want to move up.

 

Equally important for customer loyalty is the new marketing options that the continuous flow of real-time operating data offer you. For a growing number of companies, that means they’re no longer selling products, but leasing them, with the price based on actual customer usage: if it ain’t bein’ used, it ain’t costing them anything and it ain’t bringing you any revenue!

Examples include:

  • jet turbines which, because of the real-time data flow, can be marketed on the basis of thrust generated: if it’s sitting on the ground, the leasee doesn’t pay.  The same real-time data flow allows the manufacturer to schedule predictive maintenance at the earliest sign of a problem, reducing both its cost and the impact on the customer.
  • Siemens’s Mobility Services, which add in features such as 3-D manufactured spare parts that speed maintenance and reduced costs, keeping the trains running.
  • Philips’s lighting services, which are billed on the basis of use, not sold.
  • SAP’s prototype smart vending machine, which (if you opt in) may offer you a special discount based on your past purchasing habits.

At its most extreme is Caterpillar’s Reman process, where the company takes back and remanufactures old products, giving them a new life — and creating new revenues — when competitors’ products are in the landfill.

Loyalty can also be a benefit of IoT strategies for manufacturers’ own operations as well. Remember that the technological obstacles to instant sharing of real-time data have been eliminted for the supply chain as well. If you choose to share it, your resupply programs can also be automatically triggered on a M2M basis, giving an inherent advantage to the domestic supplier who can get the needed part there in a few hours, versua the low-cost supplier abroad who may take weeks to reach your loading dock.

It may be harder to quantify than quality improvements or streamlined production through the IoT, but that doesn’t mean that dependable revenue streams from loyal customers aren’t an important potential benefit as well.

Libelium: flexibility a key strategy for IoT startups

I’ve been fixated recently on venerable manufacturing firms such as 169-yr. old Siemens making the IoT switch.  Time to switch focus, and look at one of my fav pure-play IoT firms, Libelium.  I think Libelium proves that smart IoT firms must, above all, remain nimble and flexible,  by three interdependent strategies:

  • avoiding picking winners among communications protocols and other standards.
  • avoiding over-specialization.
  • partnering instead of going it alone.
Libelium CEO Alicia Asin

Libelium CEO Alicia Asin

If you aren’t familiar with Libelium, it’s a Spanish company that recently turned 10 (my, how time flies!) in a category littered with failures that had interesting concepts but didn’t survive. Bright, young, CEO Alicia Asin, one of my favorite IoT thought leaders (and do-ers!) was recently named best manager of the year in the Aragón region in Spain.  I sat down with her for a wide-ranging discussion when she recently visited the Hub of the Universe.

I’ve loved the company since its inception, particularly because it is active in so many sectors of the IoT, including logistics, industrial control, smart meters, home automation and a couple of my most favorite, agriculture (I have a weak spot for anything that combines “IoT” AND “precision”!) and smart cities.  I asked Asin why the company hadn’t picked one of those verticals as its sole focus: “it was too risky to choose one market. That’s still the same: the IoT is still so fragmented in various verticals.”

The best illustration of the company’s strategy in action is its Waspmote sensor platform, which it calls the “most complete Internet of Things platform in the market with worldwide certifications.” It can monitor up to 120 sensors to cover hundreds of IoT applications in the wide range of markets Libelium serves with this diversified strategy, ranging from the environment to “smart” parking.  The new versions of their sensors include actuators, to not simply report data, but also allow M2M control of devices such as irrigation valves, thermostats, illumination systems, motors and PLC’s. Equally important, because of the potentially high cost of having to replace the sensors, the new ones use extremely little power, so they can last        .

Equally important as the company’s refusal to limit itself to a single vertical market is its commitment to open systems and multiple communications protocols, including LoRaWAN, SIGFOX, ZigBee and 4G — a total of 16 radio technologies. It also provides both open source SDK and APIs.

Why?  As Asin told me:

 

“There is not going to be a standard. This (competiting standards and technology) is the new normal.

“I talk to some cities that want to become involved in smart cities, and they say we want to start working on this but we want to use the protocol that will be the winner.

“No one knows what will be the winner.

“We use things that are resilient. We install all the agents — if you aren’t happy with one, you just open the interface and change it. You don’t have to uninstall anything. What if one of these companies increases their prices to heaven, or you are not happy with the coverage, or the company disappears? We allow you to have all your options open.

“The problem is that this (not picking a standard) is a new message, and people don’t like to listen.  This is how we interpret the future.”

Libelium makes 110 different plug and play sensors (or as they call them, “Plug and Sense,” to detect a wide range of data from sources including gases, events, parking, energy use, agriculture, and water.  They claim the lowest power consumption in the industry, leading to longer life and lower maintenance and operating costs.

Finally, the company doesn’t try to do everything itself: Libelium has a large and growing partner network (or ecosystem, as it calls it — music to the ears of someone who believes in looking to nature for profitable business inspiration). Carrying the collaboration theme even farther, they’ve created an “IoT Marketplace,” where pre-assembled device combinations from Libelium and partners can be purchased to meet the specific needs of niches such as e-health,  vineyards, water quality, smart factories, and smart parking.  As the company says, “the lack of integrated solutions from hardware to application level is a barrier for fast adoption,” and the kits take away that barrier.

I can’t stress it enough: for IoT startups that aren’t totally focused on a single niche (a high-stakes strategy), Libelium offers a great model because of its flexibility, agnostic view of standards, diversification among a variety of niches, and eagerness to collaborate with other vendors.


BTW: Asin is particularly proud of the company’s newest offering, My Signals,which debuted in October and has already won several awards.  She told me that they hope the device will allow delivering Tier 1 medical care to billions of underserved people worldwide who live in rural areas with little access to hospitals.  It combines 15 different sensors measuring the most important body parameters that would ordinarily be measured in a hospital, including ECG, glucose, airflow, pulse, oxygen in

It combines 15 different sensors measuring the most important body parameters that would ordinarily be measured in a hospital, including ECG, glucose, airflow, pulse, blood oxygen, and blood pressure. The data is encrypted and sent to the Libelium Cloud in real-time to be visualized on the user’s private account.

It fits in a small suitcase and costs less than 1/100th the amount of a traditional Emergency Observation Unit.

The kit was created to make it possible for m-health developers to create prototypes cheaply and quickly.

Siemens’s Mobility Services: Trains Become IoT Labs on Wheels

George Stephenson's Killingworth locomotive Source: Project Gutenberg

George Stephenson’s Killingworth locomotive
Source: Project Gutenberg

As those of you who know rail history understand, with Stephenson as your last name, you’re bound to have a strong interest in railroads! Add in the fact that I was associate producer of an award-winning documentary on the subject back in the early 70’s, and it’s no wonder I was hooked when I got a chance to meet with some of Siemens’s top rail executives on my trip to Barcelona last week (Disclaimer: Siemens paid my expenses, but didn’t dictate what I covered, nor did they have editorial review of this piece).

What really excites me about railroads and the IoT is that they neatly encapsulate the dramatic transformation from the traditional industrial economy to the IoT: on one hand, the railroad was perhaps THE most critical invention making possible 19th century industry, and yet it still exists, in recognizable but radically-evolved form, in 2016. As you’ll see below, trains have essentially become laboratories on wheels!

I dwelt on the example of the Union Pacific in my e-book introduction to the IoT, SmartStuff, because to CIO Lynden Tennison was an early adopter, with his efforts focused largely on reducing the number of costly and dangerous derailments, through measures such as putting infrared sensors every twenty miles along the rail bed to spot “hotboxes,” overheating bearings. That allowed an early version of what we now know as predictive maintenance, pulling cars off at the next convenient yard so the bearings could be replaced before a serious problem. Even though the technology even five years ago was primitive compared to today, the UP cut bearing-related derailments by 75%.

Fast-forward to 2016, and Siemens’s application of the IoT to trains through its Mobility Services is yielding amazing benefits: increasing reliability, cutting costs, and even leading to possible new business models. They’ve taken over maintenance for more than 50 rail and transit programs.

While I love IoT startups with a radical new vision and no history to encumber them, Siemens is a beacon to those companies firmly rooted in manufacturing which may wonder whether to incorporate the IoT in their services and strategy. I suspect that its software products are inherently more valuable than competitors from pure-play software firms at commercial launch because the company eats its own dogfood and applies the new technology first to the products it manufactures and maintains — closing the loop.

Several of its executives emphasized that one of the advantages Siemens feels they enjoy is that their software engineers in Munich work in a corner of an old locomotive factory that Siemens still operates, so they can interact with those actually building and maintaining the engines on a daily basis. When it comes to security issues, their experience as a manufacturer means they understand the role of each component of the signaling system. Dr. Sebastian Schoning, ceo of Siemens client Gehring Technologies, which manufactures precision honing tools, told me that it was easier to sell these digital services to its own client base because so much of their current products include Siemens devices, giving them confidence in the new offerings. GE enjoys the same advantages of combining manufacturing and digital services with its Evolution Series locomotives.

The key to Siemens’s Mobility Services is Sinalytics, its platform architecture for data analysis not just for rail, but also for industries ranging from medical equipment to wind farms. More than 300,000 devices currently feed real-time data to the platform,   Consistent with my IoT-centric “Circular Company” vision, Sinalytics capitalizes on the data for multiple uses, including connectivity, data integration, analytics, and the all-important cyber security — they call the result not Big Data, but Smart Data. As with data services from jet turbine manufacturers such as Rolls Royce and GE, the platform also allows merging the data with data from sources such as weather forecasts which, in combination, can let clients optimize operating efficiency on a real-time M2M basis.  

With the new approach, trains become IoT laboratories on wheels, combining all of the key elements of an IoT system:

  • Sensing: there are sensors on the engines and gearboxes, plus vibration sensors on  microphones measure noises from bearings in commuter trains. They can even measure how engine oil is aging, so it can be changed when really needed, rather than on an arbitrary schedule.
  • Algorithms to make sense of the data and act on it. They read out patterns, record deviations & compare them with train control systems or vehicles of the same type.
  • Predictive maintenance replaces scheduled maintenance, dramatically reducing down-time and catastrophic failure.For example: “There’s a warning in one of the windows (of the control center display): engine temperature unusual. ‘We need to analyze the situation in greater depth to know what to do next  — we call it  ‘root cause analysis,” (say) Vice-President for Customer Support Herbert Padinger. ‘We look at its history and draw on comparative data from the fleet as a whole.’ Clicking on the message opens a chart showing changes in temperature during the past three months. The increased heat is gradually traced to a signal assembly. The Siemens experts talk with the customer to establish how urgent the need for action is, and then takes the most appropriate steps.”  He says that temperature and vibration analyses from the critical gearboxes gives Siemens at least three days advance notice of a breakdown — plenty of time for maintenance or replacement.  Predictive maintenance is now the norm for 70-80% of Siemens’s repairs.
  • Security (especially important given all of the miles of track and large crowds on station platforms): it includes video-based train-dispatch and platform surveillance using its SITRAIL D system, as well as cameras in the trains. The protections have to run the gamut from physical attacks to cyber attacks.  For security, the data is shared by digital radio, not networks also shared by consumers.

When operations are digitized, it allows seamlessly integrating emerging digital technologies into the services. Siemens Digital Services also included augmented reality (so repair personnel can see manuals on heads-up displays), social collaboration platforms, and — perhaps most important — 3-D printing-based additive manufacturing, so that replacement parts can be delivered with unprecedented speed. 3-D printing also allows dramatic reduction in parts inventories and allows replacement of obsolete parts that may no longer be available through conventional parts depots or even — get this — to improve on the original part’s function and/or durability, based on practical experience gained from observing the parts in use.  Siemens has used 3-D printing for the past last 3 years, and it lets them assure that they will have replacements for the locomotive’s entire lifespan, which can exceed 30 years.

The results of the new approach are dramatic.

  • None of the Velaro trains that Siemens maintains for several operators have broken down since Sinalytics was implemented. Among those in Spain only 1 has left more than 15 min. behind time in 2,300 trips: .0004%!
  • Reliability for London’s West Coast Mainline is 99.7%

  • Perhaps most impressive, because of the extreme cold conditions it must endure, the reliability rate for the Velaro service in Russia is 99.9%!

Their ultimate goal is a little higher: what Siemens calls (pardon the pun) 100% Railability (TM).

And, consistent with what other companies find when they fully implement not only IoT technology, but also what I like to call “IoT Thinking,” when it does reach those previously inconceivable quality benchmarks, the company predicts that, as the software and sensors evolve, the next stage will be new business models in which billing will be determined by guaranteeing customers availability and performance.

PS: I’ll be posting more about my interviews with Siemens officials and the Gartner event in coming days.

Smart Disposables: Could This Be Birth of Internet of Everything?

Could EVERYTHING be “smart?” It may be happening sooner we thought, and with implications that are hard to fathom today.

That’s the potential with new technology pioneered by Shyam Gollakota, an assistant professor at the University of Washington.  For the first time, it would let battery- and cordless-less devices harvest signals from Wi-Fi, radio, or TV to communicate and power themselves.

Astounding!

For a long time, the most “out there” idea about IoT sensors has been Prof. Kris Pister’s “smart dust” concept, which aimed at a complete sensor/communication system in a package only one cubic millimeter in size. Pister argued that such devices would be so small and cheap that they could be installed — or perhaps even scattered — almost everywhere. The benefits could be varied and inconceivable in the past. According to Pister, possible applications could include:

  • “Defense-related sensor networks
    • battlefield surveillance, treaty monitoring, transportation monitoring, scud hunting, …
  • Virtual keyboard
    • Glue a dust mote on each of your fingernails.  Accelerometers will sense the orientation and motion of each of your fingertips, and talk to the computer in your watch.  QWERTY is the first step to proving the concept, but you can imagine much more useful and creative ways to interface to your computer if it knows where your fingers are: sculpt 3D shapes in virtual clay, play  the piano, gesture in sign language and have to computer translate, …
    • Combined with a MEMS augmented-reality heads-up display, your entire computer I/O would be invisible to the people around you.  Couple that with wireless access and you need never be bored in a meeting again!  Surf the web while the boss rambles on and on.
  • Inventory Control
    • The carton talks to the box, the box talks to the palette, the palette talks to the truck, and the truck talks to the warehouse, and the truck and the warehouse talk to the internet.  Know where your products are and what shape they’re in any time, anywhere.  Sort of like FedEx tracking on steroids for all products in your production stream from raw materials to delivered goods.
  • Product quality monitoring
    • temperature, humidity monitoring of meat, produce, dairy products
      • Mom, don’t buy those Frosted Sugar Bombs, they sat in 80% humidity for two days, they won’t be crunchy!
    • impact, vibration, temp monitoring of consumer electronics
      • failure analysis and diagnostic information, e.g. monitoring vibration of bearings for frequency signatures indicating imminent failure (back up that hard drive now!)
  • Smart office spaces
    • The Center for the Built Environment has fabulous plans for the office of the future in which environmental conditions are tailored to the desires of every individual.  Maybe soon we’ll all be wearing temperature, humidity, and environmental comfort sensors sewn into our clothes, continuously talking to our workspaces which will deliver conditions tailored to our needs.  No more fighting with your office mates over the thermostat.
  • Interfaces for the Disabled (courtesy of Bryndis Tobin)
    • Bryndis sent me email with the following idea: put motes “on a quadriplegic’s face, to monitor blinking & facial twitches – and send them as commands to a wheelchair/computer/other device.”  This could be generalized to a whole family of interfaces for the disabled.  Thanks Bryndis!”

Now imagine that a critical component of such a tiny, ubiquitous device was removed. Because it didn’t need a battery it could be even smaller and cheaper (because of cheaper and simpler radio hardware circuitry).

The goal is having billions of disposable devices start communicating,” Gollakota said (my emphasis).

You may remember that I’ve written before about my metaphor of a pre-IoT era of “Collective Blindness,” the universal inability to peer (literally or figuratively) inside things in the past, which forced us to create all sorts of work-arounds to cope with that lack of real-time data. Imagine how precise our knowledge about just about everything will be if Gollakota’s technology becomes commonplace.

.As Technology Review reported, the critical challenge is making it possible for a device lacking a traditional power source to communicate: “Transferring power wirelessly is not a new trick. But getting a device without a conventional power source to communicate is harder, because generating radio signals is very power-intensive and the airwaves harvested from radio, TV, and other telecommunication technologies hold little energy.”

The principle making the innovation possible is “backscattering,” reflecting waves, particles or signals back in the direction they came from, which creates a new signal.

The early results are encouraging. Gollakata has made a contact lens that can connect with a smartphone. Think I’ll pass on that one, but other devices he and his team have created include brain implants and “a flexible skin patch that can sense temperature and respiration, a design that could be used to monitor hospital patients.”  Marketers will love this one: a concert poster broadcasting a bit of the featured band’s music over FM radio!

Jeeva Wireless, Gollakata’s commercial spinoff, is using a variety of the technology, “passive Wi-Fi.” Devices using it can data up to 100 feet and connect through walls.

Tiny passive devices using backscatter could be manufactured for as little as a dollar. “In tomorrow’s smart home, security cameras, temperature sensors, and smoke alarms should never need to have their batteries changed.”

Gollakata sums up the potential impact: “We can get communication for free” (my emphasis).

That’s incredible, but in light of the continuing series of major DDoS attacks made possible by weak or non-existent IoT security measures, I must remind everyone that speed, power, and ubiquity aren’t everything: we also need IoT security, so I hope the low cost and ability to function without a dedicated energy source won’t obscure that need as well.


 

BTW: a MIT profile on Gollakata mentions one of his other, related, inventions, which I think would mesh beautifully with my SmartAging vision to help seniors age in place in better health.

It’s called  WiSee, which uses wireless signals such as Wi-Fi to “enable whole-home sensing and recognition of human gestures. Since wireless signals do not require line-of-sight and can traverse through walls, WiSee can enable whole-home gesture recognition using few wireless sources (e.g., a Wi-Fi router and a few mobile devices in the living room).”

I love the concept for seniors, because (like Echo, which I’m finally getting!!) it doesn’t require technical expertise, which many seniors lack and/or find intimidating, to launch and direct automated devices. In this case, the activation is through sensing and recognition of human gestures. According to Gollakata,“’Gestures enable a whole new set of interaction techniques for always-available computing embedded in the environment. As an example, he suggests that a hand swiping motion in the air could enable a user to control the radio volume while showering – or change the song playing on the stereo in the living room while you are cooking in the kitchen.”

He goes on to explain:

“…. that the approaches offered today to enable gesture recognition – by either installing cameras throughout a home/office or outfitting the human body with sensing devices – are in most cases either too expensive or unfeasible. So he and his group members are skirting these issues by taking advantage of the slight changes in ambient wireless signals that are created by motion. Since wireless signals do not require line-of-sight and can traverse through walls, he and his group have achieved the first gesture recognition system that works in those situations. ‘We showed that this approach can extract accurate information about a rich set of gestures from multiple concurrent users.”

Combine that with speaking to Alexa, and even the most frail seniors could probably control most of the functions in a smart home. Gollakota says that the approaches offered today to enable gesture recognition – by either installing cameras throughout a home/office or outfitting the human body with sensing devices – are in most cases either too expensive or unfeasible. So he and his group members are skirting these issues by taking advantage of the slight changes in ambient wireless signals that are created by motion. Since wireless signals do not require line-of-sight and can traverse through walls, he and his group have achieved the first gesture recognition system that works in those situations. “We showed that this approach can extract accurate information about a rich set of gestures from multiple concurrent users, “he says.

Incredible work, professor!