The increasing demand for the Internet of Things (IoT) and user-based positioning and tracking services in the 5G era necessitates more efficient target tracking solutions. Traditional radar-based systems, while effective, suffer from complexity, high cost, and large size. This paper explores the use of digital programmable metasurfaces (DPMs) as a more efficient and cost-effective alternative. DPMs offer powerful and flexible control over electromagnetic waves due to their subwavelength unit cells and functional arrangements, making them suitable for dynamic beam steering and communication. This research proposes an intelligent system integrating computer vision with DPMs to achieve autonomous target tracking and wireless communication. Computer vision, specifically a convolutional neural network (CNN), is employed for real-time target detection and localization. The dual-polarized DPM, controlled by a pre-trained artificial neural network (ANN), enables smart beam tracking, directing the signal towards the detected target while simultaneously facilitating wireless communication. The system's effectiveness is validated through a series of experiments demonstrating target detection, RF signal detection, and real-time wireless communication.

Metamaterials and their two-dimensional counterparts, metasurfaces, have garnered significant attention due to their ability to manipulate electromagnetic (EM) waves. Past research has explored various applications of metasurfaces, including holography, spectroscopy, nonlinear photonics, and quantum photonics. Programmable metasurfaces, especially those utilizing digital coding elements with discretized reflection phases, provide dynamic control over EM waves through the integration of active devices like PIN diodes and varactors. These have been used in polarization and amplitude modulation, as well as transmission-reflection control, finding applications in microwave imaging, space-time modulation, and wireless communication. However, most prior work focuses on human-controlled programmable metasurfaces. While self-adaptive metasurfaces exist for applications such as invisibility cloaks, most research has been confined to verifying pre-designed functions. The integration of artificial intelligence (AI), particularly deep learning (DL), has been explored to optimize metasurface coding matrices for complex scattering problems. Simultaneously, the advancements in computer vision technology provide a path towards intuitive, reliable, and cost-effective target detection and tracking, opening avenues for intelligent communication systems. This research leverages these advancements by combining computer vision and the flexible control of DPMs to create a fully autonomous system.

The proposed intelligent system comprises a dual-polarized DPM and an RGB-D camera (Intel RealSense Depth Camera D435i). The camera captures images of a moving target (a model car in the experiments) at 40 frames per second (FPS). A YOLOv4-tiny convolutional neural network (CNN) processes these images to detect the target's location, and its elevation and azimuth angles. This information is then fed into a pre-trained artificial neural network (ANN) that determines the optimal coding sequence for the DPM. This sequence is sent to the DPM via a field-programmable gate array (FPGA), which controls the reflection properties of each element in the metasurface to direct the EM beam towards the moving target. The DPM's design incorporates 1-bit dual-linearly polarized elements, each containing two PIN diodes controlled independently to manipulate the phase of both x- and y-polarized waves. The design and performance of the DPM element are simulated using CST Microwave Studio, showing good reflection efficiency and a 180° phase difference between ON and OFF states of the PIN diodes. A total of 324 (18x18) elements are used in the fabricated DPM prototype. The far-field patterns of the DPM are measured experimentally in an anechoic chamber, confirming its dynamic beam-steering capabilities controlled by the FPGA. The YOLOv4-tiny network is optimized for high-precision detection at a suitable speed, using techniques like Mosaic data enhancement, SPPNet, and a CSPDarknet53-tiny backbone. The pre-trained ANN, based on ResNet34, maps the target's angular coordinates to the corresponding coding sequence for the DPM, achieving fast and accurate beam steering. The system is tested in various scenarios, including single and multiple target tracking, target occlusion, and low-light conditions, using additional algorithms and cameras (NV-Camera) where necessary. For RF signal detection, a portable detector using the AD8317 is used, mounted on the model car, measuring the received signal strength. Real-time wireless communication is demonstrated using a video transmission module, sending video data over the 5.8 GHz band while simultaneously tracking the target.

The experiments demonstrate the successful integration of computer vision and DPM for intelligent target tracking and communication. The system accurately detects and tracks a moving target in real-time, dynamically adjusting the beam direction of the DPM to maintain signal strength. The YOLOv4-tiny network achieves reliable target detection even in challenging scenarios, like multiple targets and partial occlusion. The pre-trained ANN efficiently translates target position data into the appropriate DPM coding sequences for beam steering, demonstrating a closed-loop control system without human intervention. The experiments with the RF signal detector confirm the real-time tracking capability, showing that the received signal strength correlates directly with the target's proximity. Furthermore, real-time video transmission was successfully achieved, with high-quality video transmission when the receiver is attached to the target or when the target is within the effective range of the directed beam. The system exhibits robustness and adaptability, functioning effectively in both indoor anechoic chambers and outdoor environments. Specific findings include the DPM's ability to steer beams from -40° to 40° on the E-plane with a high reflection efficiency, the YOLOv4-tiny network’s accurate target localization and classification, and the efficient performance of the ANN in generating the correct coding sequences for beam steering. The real-time wireless transmission experiments achieved a stable bit error rate of 10⁻⁵ under suitable conditions.

This research successfully demonstrates a novel approach to intelligent target tracking and wireless communication using the combined power of computer vision and digital programmable metasurfaces. The results address the limitations of traditional radar-based systems by offering a more efficient, compact, and cost-effective solution. The integration of AI algorithms such as the YOLOv4-tiny object detection network and the ResNet34-based ANN enhances the system's adaptability and speed, allowing for real-time operation in dynamic environments. The successful implementation of real-time video transmission alongside tracking highlights the potential for this technology in various applications. The demonstrated system's robustness across different scenarios and environments underscores its practical relevance. The findings contribute significantly to the field of intelligent wireless networks and self-adaptive systems.

This paper successfully demonstrates an integrated intelligent metasurface system capable of real-time target tracking and wireless communication. This system combines computer vision for target detection with a dual-polarized DPM controlled by an ANN for beam steering and communication. Experiments show reliable target tracking, RF signal detection, and video transmission in diverse scenarios. Future work could explore improved target detection algorithms, enhanced DPM designs for wider bandwidth and beam control, and further integration with existing wireless communication systems for broader applications within IoT and beyond 5G/6G networks.

While the system demonstrates strong performance in the tested scenarios, limitations exist. The current DPM design uses a limited number of elements, resulting in sidelobes in the radiated beam. Expanding the array size and optimizing the element design would improve beam quality and reduce sidelobes. The accuracy of target tracking is dependent on the performance of the computer vision system; factors such as lighting conditions and target occlusion can impact detection accuracy. Furthermore, the experimental environment was controlled; additional testing in more complex environments with significant interference and varied signal propagation is needed to fully assess the system's robustness. Finally, scalability to a larger number of targets and more complex communication scenarios needs further investigation.