There is always the need for computing to be more available and distributed, especially given the data volume generated from IoT. On the surface, it makes sense that most proponents of data processing have been advocating for cloud computing, to always send IoT data to the cloud.
With IoT and cloud computing (later referred to as “the cloud”), one basically pushes and processes all sensory data in the cloud. There is an ingestion module that takes in the data generated from the IoT applications and stores it in a data lake, i.e., a really large storage, and then applies parallel processing over it, which could be Spark, Azure HD Insight, Hive, etc., and then consume this fast-paced information to make decisions. There are only a few famous cloud computing platforms commercially available in the US, which are Amazon Web Services (AWS), Microsoft Azure, Google Cloud, Oracle Cloud, and IBM Watson. Benefits of merging the cloud with IoT are:
- Affordability – Customers do not have to invest in their infrastructure and only “pay as you go” depending on the demand of the needed resources, to the service provider.
- Efficient Growth – The optimal capacity of resources in the cloud will be used.
- Elasticity and Scalability – Whether it is bandwidth, storage or any other resource, everything can be increased and reduced according to the business demand.
- Avoiding Downtime or Delay – The cloud is a network of connected servers. If any node fails, then all its load will be taken away by other cloud node(s). Therefore, the business service running on the cloud will not go down.
- Disaster Recovery – The cloud provides a low cost and safe disaster recovery mechanism for any sized business.
The IoT applications are generating an unprecedented volume and variety of data. But by the time the data makes its way to the cloud for analysis, the opportunity to act on it might be gone. IoT speeds up awareness and response to events. In industries such as manufacturing, oil and gas, utilities, transportation, mining, and the public sector, faster response time can improve output, boost service levels, and increase safety. The IoT applications will challenge the cloud in the following ways:
- Bandwidth Limitation – Capitalizing on IoT requires a new kind of infrastructure. Today’s cloud models are not designed for the volume, variety, and velocity of data that IoT generates. Billions of previously unconnected devices are generating more than two exabytes of data each day. Billions of more “things” will be connected to the internet each year. Moving all data from these things to the cloud for analysis would require vast amounts of bandwidth.
- Long Latency – Milliseconds matter when it is trying to prevent manufacturing line shutdowns or restore electrical service. Analyzing data close to the devices that collected the data can make the difference between averting disaster and a cascading system failure.
The fog computing (later referred to as “the fog”) was introduced to address the timeliness of the IoT data, which:
- Analyzes the most time-sensitive data at the network edge, close to where the data is generated instead of sending vast amounts of the IoT data to the cloud.
- Acts on the IoT data in milliseconds, based on policy.
- Sends selected data to the cloud for historical analysis and longer-term storage.
The exponential growth in data collected by billions of IoT and mobile devices is driving a shift from sending data to the cloud for processing and storage to a distributed model where some computing occurs at the edge of the network, i.e., edge computing (later referred to as “the edge”), closer to where the data is created. Even though both the fog and the edge involve moving intelligence and processing capacity closer to where the data originates, the most fundamental difference between them is precisely where that intelligence and computing power is located as shown in Figure 1.
- The fog drives intelligence down to Local Area Network (LAN) level of network architecture, processing data in either a fog node or IoT gateway.
- The edge meanwhile drives the intelligence, processing power, and communication operations of an edge gateway or appliance directly into the devices themselves such as Programmable Automation Controllers (PACs).
The ultimate goal is for IoT computing to be Everything Computing Everywhere, i.e., the mist computing. The mist nodes are on the edge, in the fog, and in the cloud that all work in concert together as a single decentralized and distributed mesh computer. The mist manages routing computations to the right node, at the right time, in the right location, typically as close to the source of the data as possible.
This column concludes this series about IoT.
Author Biography
Dr. Qiang Lin is a Lead Systems Engineer at The MITRE Corporation and is a well-known expert on data architecture, data modeling, data engineering and data governance; he has assisted DoD, U.S. Army, U.S. Air Force, DHS, VA, IRS on many data management and data engineering projects for over 30 years. He has been an adjunct faculty at George Mason University since 2001. He is the co-author Internet of Things Ecosystem: 3rd Edition, published in March 2022.
‘The author’s affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to convey or imply MITRE’s concurrence with, or support for, the positions, opinions, or viewpoints expressed by the author.’
©2024 The MITRE Corporation. ALL RIGHTS RESERVED. Approved for Public Release; Distribution Unlimited. Public Release Case Number 24-3907.
