Key Takeaways
- Hardware augmented software means optimizing the performance of software through custom hardware modifications or solutions.
- Hardware augmented software has a range of applications, but is particularly common in AI, machine learning, augmented reality and IoT/Smart Home applications.
- Hardware augmented reality is a crucial part of modern smart home design, including its use in S0Cs, Digital Signal Processors and Trusted Platform Modules (TPMs).
When we think about improving software, we usually think about ways in which the code itself can be optimized to improve the final product. But another way of improving software performance is by altering the hardware itself.
Here we look at the concept of hardware augmented software, some of its applications, and how it is used within the modern smart home.
The distinction between hardware and software
Explaining the possible ways in which hardware augmentation can improve software performance requires clarity on the distinction between hardware and software. This distinction may seem too obvious to mention — the hardware is the physical part of the computer or device that we ‘see’, while the software is the invisible instructions telling the computer what to do. But the distinction is not so clear-cut: For example, in the first half of the twentieth century the computer’s ‘code’ was physical punch cards, yet we would usually conceptualize this as early software, not hardware. Similarly, modern chips and processor components may be too small to see with the naked eye, but we wouldn’t say that that feature makes them software.
The better distinction is that hardware is hardware because it is ‘hard-wired’: In most cases, it is not easy to change their configuration. Thought of in that way, hardware covers the usual physical components of a computer system: This includes the motherboard, processor, memory, storage devices, input/output devices, and any peripheral devices.
Going back to the early 20th century, computing devices were all hardware. For example, one of the oldest computers, the Atanasoff–Berry computer (ABC) from 1937 had its one set of instructions hard-wired into it, and was not programmable.
Software, standing next to this definition of hardware, refers to that feature of the computer or device that can be more readily changed and is programmable. This covers all programs, including the operating system itself. Software is stored in the memory of storage devices, such as hard drives, and ultimately translated into machine language (the only instructions that a computer can actually read). Software is not hard-wired and, in principle, can be altered to get the computer to perform different tasks, though, in some cases (e.g., operating systems), it may not be easy to
Straddling the divide between software and hardware lies firmware: This is a type of software programmed directly into a device’s read-only memory (ROM), controlling the hardware. It is a permanent form of programming, not intended to be readily altered or deleted. Common in embedded systems, such as Routers and Internet of Things (IoT) devices, firmware can be updated to fix bugs or add features, but specialized tools are required in order to do so.
What is hardware augmented software?
With this distinction between hardware and software in mind, hardware augmented software means altering those elements of the system that would traditionally be fixed (such as the physical components), in order to improve the operation of the software. Consider some common examples of hardware augmented software below:
- Hardware acceleration — using specialized hardware to make processing more efficient than in a conventional Central Processing Unit (CPU). This is not a new concept: 90s gamers will remember adding an external Sound Card (e.g., ‘Sound Blaster’) to elevate the sound effects in Doom or Duke Nukem from the simple beeps of the built-in speaker. A more sophisticated example is the recent rise of ‘System on a chip’ (SoC) — an integrated circuit that contains within it many of the hardware components of a traditional computer. The embedded SoC reduces the size of a device, improves performance, and reduces power consumption.
- Custom or alternate instruction sets — changing the fundamental instructions that the computer or device runs on. For example, the RISC (Reduced Instruction Set Computer) architecture simplifies the instructions to the computer to optimize performance. Custom instruction set extensions add new instructions to optimize for a particular application: For example, ARMv8M adds instructions to improve performance and efficiency in IoT applications.
- Special-purpose processors — where a custom processor is designed for a specific task to optimize overall software performance. For example, Graphics Processing Units (GPUs) handle complex graphical processing, including rendering 3D graphics and video. Digital Signal Processors (DSPs) are used in audio and video processing. They improve processing efficiency, allowing for real-time audio and video processing. In both cases, there is a significant performance improvement over software running on a conventional CPU.
- Hardware-based security features — a trusted platform module (TPM), for example, is a secure microcontroller that provides secure storage of critical data, such as an encryption key. They are more secure than software-only solutions as, being embedded directly into the device, they are less vulnerable to hacking and malware.
How does hardware-augmented software form part of the modern smart home?
Hardware-augmented software is a crucial feature of the modern smart home. Some examples of how hardware-enabled software solutions on the smart home market today include:
- SoCs in WiFi sensing — WiFi sensing smart home solutions interpret the way in which human movement interrupts WiFi waves in order to bolster home security, optimize energy use and improve the well-being of occupants. While WiFi sensing can be implemented through a variety of hardware enablers, SoCs reduce the size, cost, and power consumption of WiFi sensing modules.
- TPM in smart homes — TPM can be used to secure remote access to smart home systems. The TPM can store authentication credentials and verify the identity of remote devices, ensuring that only those who should have access, do have access.
- Digital Signal Processors (DSPs) in Smart Home Security. DSPs are often used in smart home security cameras to enhance video quality and perform real-time video processing. The DSPs can be hardware-enabled to perform tasks such as motion detection, object recognition, and video compression, making it easier to monitor and secure a smart home
Hardware augmented software — the way forward
Hardware augmented software is increasingly a core part of IoT and the smart home infrastructure: There are some software efficiency gains which require the hardware itself to be optimized.
In the future, we can expect to see further hardware augmented software innovations in the field, as embedded systems shrink and energy efficiency concerns push for better energy consumption.
FAQ
Indirectly, yes. Augmented reality hardware covers those components such as sensors, digital cameras, GPS and wireless sensors that are crucial to facilitating a virtual reality or augmented reality environment.
It is increasingly common for augmented reality to use hardware-enabled software (such as DSPs and SoCs) to optimize virtual reality applications.
It’s debatable. Firmware provides low-level control of a device’s hardware and is stored in a device’s non-volatile memory, such as read-only memory (ROM). The sense in which it is difficult to alter makes it close to hardware. However, it is arguably not an augmentation of hardware in order to support software performance. Some would say that it is a fundamental software augmentation.
