Sensor fusion gathers data from multiple sensors together, allowing users to have a more detailed and detailed understanding of the system's operation or environment. In many cases, the weaknesses of a certain sensor technology can be overcome by adding (fusing) information from a second sensor technology. Adding artificial intelligence (AI) and machine learning (ML) capabilities can enhance the ability of sensor fusion.

When implementing sensor fusion, several challenges need to be addressed. For example, developing a balanced solution rather than favoring one technology can be difficult to achieve. The result of doing so may lead to a lack of scalability and reduced performance. Integrating multiple sensor technologies into a single package is one of the methods to address this challenge. Sensor fusion is not limited to using multiple discrete sensors.

Regardless of the level of sensor integration, adding AI or ML can improve performance, but the training process can be complex and time-consuming. Given this, designers can switch to using self training sensors with embedded AI and ML capabilities.

This article first introduces sensor fusion achieved through discrete sensors, 32-bit MCUs, and ML software. Then introduce some integrated sensor fusion solutions and application examples in logistics facilities, data centers, process automation, material handling, and agricultural equipment.

Finally, an integrated environment sensor fusion scheme incorporating AI software is introduced. Throughout the discussion, devices from Renesas Electronics, Sensirion, TE Connectivity, ACEINNA, Bosch Sensortec, and TDK InvenSense will be used as examples for discussion.

Designers can use Renesas' reference design board to explore various fusion options for sensors. This design board is based on a 32-bit MCU and provides multiple interfaces and connection options. Among them, the core of the MCU is a 120 MHz Arm ® Cortex ®- M4， Has up to 2 MB of code flash and 640 KB of SRAM.

The relevant evaluation kit has been optimized for multi-sensor and sensor fusion design. This kit includes air quality sensors, light sensors, temperature and humidity sensors, 6-axis inertial measurement unit (IMU), microphone, and low-power Bluetooth (BLE) connectivity (Figure 1). The reference design also includes an automatic ML platform for edge device and sensor fusion applications.

IoT sensor fusion evaluation and development board (click to enlarge)
Figure 1: IoT sensor fusion evaluation and development board with automatic ML development software and BLE connection function. (Image source: Renesas Electronics)

Stable tilt sensor
Tilt sensors are specialized IMUs used in various applications including agricultural machinery, off-road vehicles, material handling, and heavy construction equipment. Safety standards sometimes require the use of tilt sensors to ensure a safe operating environment. Tilt sensors can be assembled from multiple discrete components, but this may be more complex.

The core of most tilt sensor designs is the gyro sensor. This type of sensor is used to measure angular velocity or rotational speed around an axis. If the platform is in motion, it means the state is good, but if the platform stops moving, such as tilting 20 degrees, the sensor output returns to zero. In addition, over time, the gyroscope will experience significant drift and errors will accumulate, ultimately leading to measurement results that are no longer accurate or useful.

To address the limitations of gyroscopes, the dynamic tilt sensor solution adds an accelerometer to measure motion. This can tell the system when to stop moving and enable it to estimate the tilt angle using the final output of the gyroscope. The last piece of the puzzle is the temperature sensor. Temperature sensors can compensate for the impact of constantly changing temperatures on gyroscopes and accelerometers.

Kalman filters are commonly used for sensor fusion in tilt sensors. If the sensor operates within a linear range, a standard Kalman filter based on linear quadratic estimation can be used. The Kalman filter can generate relatively accurate state estimates, even for systems like tilt sensors that have inherent uncertainty and cumulative errors.

The extended Kalman filter can linearize the estimated values using the current mean and covariance, which is beneficial for tilt sensors operating in nonlinear regions.

Tilt sensors such as TE Connectivity's AXISENSE-G-700 and ACEINNA's MTLT305D have six degrees of freedom (6 DoF) motion sensing capabilities, with three from gyroscopes and three from accelerometers, and sensor fusion using Kalman filtering technology (Figure 2).