Microservice Project Architecture - Generic IoT Skeleton

Purpose

Designing an IoT solution by considering all the scenarios is very important for the long run of the product. Will describe a Microservice-based IoT Solution Architecture in this article.

Background

Before you finish reading this idea of architecting IoT solution, hundreds of new devices will be connected to the web at a breathtaking pace of 127 additions per minute. For the most part, they belong to the Internet of Things (IoT), or gadgets capable of communicating and sharing data without human interaction.

The technology will shift into an even higher gear with the wide use of fifth-generation or 5G networks supporting a million gadgets per square kilometer — ten times as many as in the 4G era. The number of active IoT connections is expected to double by 2025.

Day by day, the IoT sees wider adoption, opening new opportunities and driving more value to both businesses and their clients. For companies, incorporating the consistent IoT strategy into daily routine means continuous access to valuable data about products and processes that can be translated into reduced expenses, improved efficiency in logistics and maintenance, better products, and enhanced customer experience.

But to build and run a robust infrastructure, a manufacturer or service provider needs a solid foundation — in other words, an IoT platform that connects devices, collects data, and creates insights. This article will explain the IoT ecosystem, outline the core IoT platform functionality, and compare major players to help you make the right choice.

Problem statement (Issue or Challenge)

There are multiple layers to consider before designing the solution. With an IoT platform acting as a bridge between the physical world and business processes, the IoT infrastructure contains several key layers:

• Perception layer (hardware components such as sensors, actuators, and devices)
• Transport layer (networks and gateway)
• Processing layer (middleware or IoT platforms) and
• Application layer (software solutions for end users)

Perception layer: IoT hardware

The hardware or “things” layer includes the following gears that work with signals from the physical world.

Electronic sensors capture signals from the physical world, convert them into digital form, and feed them to the IoT system. You can monitor and manage sensors remotely, using a special application.

Actuators receive signals from the IoT system and translate them into physical actions manipulating equipment. Similar to sensors, actuators can be configured from remote computers.

Devices are connected to sensors or even have them embedded as an integral part. On the other side, devices link to a gateway or directly to an IoT platform. These hardware components cache and preprocess real-time data, reducing the burden on central storages and main processors.

Transport layer: networks and gateways

The transport layer is responsible for smooth and secure data transmission from the perception layer to the processing layer. It encompasses wired or wireless networks and a gateway — a hardware or software module that consolidates data from devices, analyzes it, performs translation between different protocols, and forwards information to the cloud. As a rule, the gateway converts all information into MQTT messages — the lightweight protocol most widely used in the IoT.

Processing layer: cloud middleware or IoT platforms

Here we come to an IoT platform or middleware that actually drives IoT, enabling you to get all components and data streams connected. On the one side, it links to gateways or devices, and on the other side integrates with third-party applications and systems via APIs.

Typically, fully-fledged platforms take care of such important tasks:

• connectivity or ensuring smooth data streaming and interactions between all IoT components;
• device management, which enables you to control and configure each piece of hardware in the IoT network as well as update software running on devices and gateways;
• data management, including data collection, processing, and storage;
• data analysis for extracting valuable patterns with machine learning, predictive analytics, and other methods;
• visualization or displaying data findings in the form of charts, graphs, 2D or 3D models;
• digital twin or creating the virtual representation of a device;
• IoT app development — platforms provide a workspace with a set of tools and templates to speed up app designing; and
• edge/fog computing — the practice of processing and storing data on devices, microcontrollers, gateways, and other IoT nodes to reduce the burden on cloud servers.

Application layer: software solutions for users

IoT software solutions allow end users to gain data insights, monitor and control devices, and, generally, manipulate the physical world through the IoT platform from computers and/or smartphones. Applications can be built on top of the IoT platform or integrated with it through APIs.

Now, with the place of IoT platforms in the connected ecosystem more clearly defined, let’s concentrate on their functions, options available on the market, and how to choose between them.

Possible solutions

The proposed architecture of a generic IoT system can be visualized as below:

There are multiple components in the architecture. A quick view of the components as the list given below:

1. Sensors / Edges
   • Sensor / Edge Driven
     • Broker (EMQX / KAFKA)
     • HTTP
   • Host Driven
     • Trigger (i.e: serverless azure function) / Scheduler (hangfire, Quartz)
2. Data Source Gateway
3. Extract Transform Load (ETL) engine
4. Database
   • Relational: (PostgreSQL with Timescale DB / SQL Server)
   • No-SQL: Redis
5. App Tier
   • API
   • Redis
   • Lazy Cache
6. Web Tier
   • Web Apps / Mobile Apps
   • Grafana
   • Angular / React
7. Separate Internal Messaging Broker
   • EMQX / MQTT / Kafka
8. Notification Hub
   • Redis as its own DB
   • Alert Rules
   • Notify
     • Email
     • SMS
     • Push Notification
9. End User

Future Considerations:

1. IP whitelist from the sources to tighten the data flow.
2. Compress and encrypt the payload during communication.

Conclusion

Above architecture is a proposed generic solution architecture to start with, but based on domain and requirements the actual system, design components may vary.