IoT technical platform changes
This section describes some of the changes which are important for the architect to understand on the technical side of the design of IoT systems. Many of these areas have changed significantly over the last ten years or are new areas unfamiliar to architects who have worked primarily on software-only projects.
The IoT system (as shown above) is a composed system of both hardware, software and networking sub-systems. The hardware components of the actual device are small board-sized elements and require electrical engineering as well as embedded systems skills. Some typical backgrounds for architects in this area would include experience with Windows Embedded devices, small AMD or Intel boards (e.g., Galileo), or maker experience with hobbyist boards like Arduino or Raspberry Pi.
The networking experience required ranges quite broadly depending on the use case: if the operator of the device is using wireless home or business-provided internet routing, then traditional TCP/IP methods are all that is required. If the device has peer-to-peer capabilities, then familiarity with the selected standard is useful. If the device has WAN communication capability – as found in automotive or robotics applications – then the architect will need to understand private networking as provided by AT&T, Orange, Verizon, Deutsche Telekom, Cisco, etc.
The architect will also need to select much of the software platform for the datacenter, including ingress or ingestion of telemetry from the large number of devices, parsing and routing of this telemetry to command and control systems and back to devices, interaction with user and datacenter (possibly by mobile phone), and any higher level analytical capabilities.
Many of these skills may be new to the architect, and by virtue of selecting many of these technologies, the architect may play a role in training the broader team on these.
Architecturally significant requirements
As the business case for IoT is often different from previous product implementations, the architect should evaluate the quality attributes for the ones which are most important in this specific development. Some of the attributes which can be significant for IoT projects include:
Cost: the architect may have to build a new cost model for the IoT device, including both initial deployment costs (which may include previously inconsequential costs like new product marketing, logistics, etc.), and operating costs. The parameterization of this cost model should be revisited frequently as the IoT application goes from prototype to selection of version 1.0 vendors and more is known about the adoption curve.
Security: since the device is connected either to other devices, the Internet or back-end datacenter assets, and possibly contains PII, a full security risk profile and remediation effort should be undertaken. The device could be hijacked, leading to PII loss, corporate data loss, or potentially modifying the object’s behavior in illegal ways.
Scalability: One of the goals for most IoT projects is the large number of connected devices, often numbering in the millions. The architect plays a critical role in identifying and correcting scale issues as the system is developed; these could arise in any implementation area from the on-device performance, to the network traffic to the ingest and message routing in the datacenter.
Performance: Often being a consumer-facing device, performance can be measured with HMI metrics. There may be other, domain-specific metrics required such as safety-critical response timing or verification of command transmission.
Reliability: If the product is a new offering, the usability and reliability of the device for purpose affects market perception and ultimately sales of the device.
Usability: Also, if the product is new in market, the purchaser or owner of the device should be able to quickly determine that it is working initially. Prior internal product offerings may not have been subject to rigorous user-acceptance criteria and some “productization” may be required in this area. For example, if the onboarding of the device requires network access, the device may have to alert the user to enter a passcode, which – if a complex process – could lead to dissatisfaction.
Manageability: Speaking to later in the lifecycle, there will ultimately need to be at least software changes to the device either to fix known issues or to add new capabilities. The architect should be able to sketch out how these updates will happen.
Technical areas of architecture
This section attempts to describe the minimum architecture and its artifacts for a “standard” IoT program. The Internet-of-Things starts with the design of the “thing” at the heart of the diagram, itself a system. The “thing” or device hereafter will have interfaces to a number of other systems, including a datacenter and optionally other things (local mesh), or owner HMI. The software systems (generally in the device, the datacenter, and optionally mobile platforms) will have software architecture artifacts. The device will have a hardware specification. The networking segment(s) may have a specification. Each interface will have a interface architecture comprised of message formats, sequence diagrams, translations and units-of-measure, and error handling. It is important that system quality attributes are traceable to subsystems from the beginning, so that overall system quality attributes and service level can be determined through regular scale and other tests. Your IoT project may require significantly more technical architecture design, but this section should cover a scalable set of deliverables and considerations to reach that goal.
The technical architecture will also include views on the consumer aspects of the project:
- Legal use and sharing of data
- Sales channel integration
- Production capacity (especially hardware)
- Pricing (COGS)
- Security standards and evaluation by third-parties/regulators
The initial set of architecture artifacts comprises:
- Overall system conceptual architecture
- Device architecture
- Hardware selection, prototyping, final design
- Software platform selection, including message passing infrastructure components
- Custom software application architecture
- Network architecture
- May include provisioning, certificate issuance/identity verification of devices, local contracts and availability
- Datacenter software architecture (pilot, “scale unit” and production sizings)
- Device I/O, telemetry and commands (via network)
- Administrative / management HMI
- Data storage (including time frame for retention)
- Billing and provisioning interface
- Partner or internal data aggregation facility (reporting, optional)
- Advanced software architectural components (“smart” services, optional)
- Owner/operator HMI
- Web portal for owner
- Mobile application (optional)
- Peer device interfaces and subsystems (optional)
- May include local networking capabilities (non-WiFi or non-WAN)
The following three sections describe some of the process for the hardware device, the communications and networking, and the datacenter services for IoT projects.
 HMI, or Human-machine interface, is a generic term for how the owner/operator may interact with the device. This may be a touchscreen on the device itself or it may be a remote application such as from mobile phone.
Embedded technology – the device
The device itself has a complete stack of capabilities, from hardware, through software platform to custom applications which are deployed onto the “thing.” Often, separate hardware and software teams must coordinate their activities in a series of integrated builds. From the hardware perspective, the builds may start as prototype boards (see Raspberry Pi image below), and then move to either a custom PCB fabricator/assembler or a completely outsourced hardware model. The hardware will have a software kernel running onboard, for deployment of applications, including the client side of the message passing stack. This kernel could be a primitive OS such as Arduino, or a more featured LINUX kernel (such as Blackberry’s automotive QNX kernel) or Windows Embedded, or even a more full-featured OS which has a full TCP/IP stack, a standard development platform and web server, and multithreading capabilities.
Image courtesy of Raspberry Pi 2 team, https://www.raspberrypi.org/products/raspberry-pi-2-model-b/, 2016
Communication protocols and the network
The device communicates with other devices, owner/operator HMI and back-end datacenter services over one or more network subsystems. Each communication endpoint may require a different network but often there is incentive to reduce cost and consolidate to one or two networks for this communication; wireless already in place with the owner/operator of the device is preferred over for-fee private telco networks or public phone connectivity (tethered).
Sample RabbitMQ topology, image courtesy of http://image.slidesharecdn.com/consumingrabbitmq-150219062454-conversion-gate01/95/consuming-rabbitmq-at-ttl-10-638.jpg?cb=1424851986
Once the communication network is selected, the protocol for message passing should also be selected. This may be a domain protocol (such as NGTP or AMQP in telematics) or may be a generic message passing protocol such as MQTT or simple MQ. The domain protocols offer two advantages: the message passing sequences are pre-defined (and testable) for domain operations, and they are often built into higher-order software components for easy implementation (for example, Apache Kafka and Azure IoT Hub support MQTT with only minor configuration required). A sample AMQP frame is shown below, indicating the wire packaging required for these byte-ordered protocols:
Sample AMQP frame, image courtesy of https://paolopatierno.wordpress.com/2015/07/23/amqp-on-the-wire-messages-content-framing
Peer-to-peer (device-to-other device) or mobile phone to device patterns may use custom protocols or mirror the datacenter communications protocols for convenience. The architect will be responsible for selecting these communication protocols, possibly from RESTful API/HTTP, AMQP, CoAP, OMA-DM, or MQTT, or as built in to the software stack,such as Azure, .NET Framework, IBM MessageSight, Apache Kafka, RabbitMQ, or others.
Sample AMQP workflow/interaction diagram, courtesy http://www.openamq.org/tutorial:soa
The protocol layer is important since it is often implemented as a whole subsystem and quite likely bought rather than custom developed. This provides higher reliability with (ideally) reduced testing effort. For example, a single library may be able to implement on both the device and the datacenter message passing using MQTT. On the datacenter, this will provide operations to perform:
- Decryption of message
- Identification of sender and message type (by certificate)
- Unpacking or translation of message body
- Routing and storage
- Hand-off to processing elements (also known as message ingestion)
Note that testing is further reduced by selecting matching pairs of versions of the protocol on both the device and the datacenter stack.
Internet-available services from the datacenter
As the datacenter aggregates messages to and from the array of devices fielded, it has itself many possible services to offer:
- On the device-facing side, this is generally a telemetry and command/control operations interface through a device gateway or scalable aggregation mechanism
- To the consumer, the datacenter can provide mobile access to the telemetry and C&C functions directly or through a device owner portal (think “Manage My Device.com”). This may involve other authentication sources from what the device uses and may involve push notifications as well.
- To the device manufacturer (or partners), CRM functionality, usage functionality (like NewRelic), and advanced services are provided by the datacenter. Often the advanced services involve the “smart” part of the device – it’s adaptability to function and recommendations to tune its usage: this is often done through data analytics (HD Insight/Hadoop, Machine Learning, Azure Search, Elasticsearch, etc.).
- To the sales channel, a means of provisioning, deprovisioning and upgrading devices already deployed as well as tacking sales contacts (CRM-like functions).
Each of these is typically determined through the business plan and varies significantly between IoT implementations.
Sample Azure IoT services, courtesy Microsoft.com 2016
Business Technology Strategy
BTS is probably the most important aspect for decision makers to work towards clarity on during the course of any IoT implementation, since it combines information from simultaneously evolving business models and technical platforms. Both the business model and the underlying technology enablers (and costs) will change radically during the course of a project and changes occurring on one side also impact the other. Similar to other aspects of the system design, the business technology strategy incorporates change and also has specific business models which should be visible to the team for directional guidance. Specifically, aids like the business model canvas and SWOT analysis – though confidential from competitors – should summarize the current business thinking well enough for implementation. Regular synchronization points should be planned for after each iteration, so that any changes to the business model are conveyed to implementation teams.
Minimum Viable Product and other approaches may be appropriate for an IoT project.
As noted in the technical description above, the generic IoT design consists of multiple sub-systems, requiring versioned designs themselves. Often a tool like swagger.io or even well-documented WSDL can provide versioning for the multiple interfaces required in the system. There will also be software and hardware architecture design documents and drawings which will be versioned. Since message passing is a fundamental construct, some auxiliary diagrams or tables may be required for translations of messages.
Since the designs are versioned, we expect that the implementation artifacts will be as well, so multiple deployments should be planned for in the schedule. All versioned designs should be kept in a repository where they are visible to all partners. Automated deployments should be pursued for IoT projects, whether following a specific DevOps approach or simpler tooling.
These are described in detail in the section above on ASR’s, and IoT projects generally need to well-define and manage the following quality attributes:
These tend to be inter-dependent and dynamic so the approach should define the service level for each (in quantitative estimated terms like $$$ for cost, or mixed terms as in system performance), for the initial prototype version, the “model” or “stamp” or “scale unit” sized deployment, and the full-up production deployment footprint.
Since early information may be incomplete, multiple estimates and verification (through testing and/or market research) should be performed on a regular basis.
The IoT environment is very complex and it might even be questioned as to whether the “IT” environment is the right home for this set of discussions. Should IT or even a product team with technical implementation skill be the focal point of the multi-vendor coordination role that is required for an IoT project? Fundamentally this is a system-of-systems integration role among multiple coordinated implementation teams with very different deliverables. There will be a set of tools, partner(s) and technologies for the network segment, for the IoT hardware itself, and probably multiple software implementers. The key for this role is to constantly know where the overall system state is… that is, where each sub-system is accurately at, and what changes to scope are required, and provide a forum for resolving conflicts and issues that arise.
Human dynamics is probably the least changed through an IoT project. Two specific areas should be addressed: discussions between coordinated partners on the implementation teams, and how to bring the voice of the customer into the effort. This latter one may take the form of an advisory board, early adopter customers or the like, but pay attention to how much the advisor knows (some details may remain confidential until after launch), and the advisors may not understand the impact of changes to technical platform, or scope of changes they could suggest. This lends itself to a guided advisory board model.