Two-dimensional (2D) semiconductors offer significant potential in optoelectronics due to their wide range of tunable bandgaps and optoelectronic properties. Their atomic thinness and transferability make them ideal for heterogeneous integration with photonic circuits and microelectronics, enabling advanced functionalities. Black phosphorus (bP), in particular, stands out with its tunable bandgap covering a wide infrared spectral range. Previous research has demonstrated high-performing bP photodetectors in various configurations, including discrete, array, and waveguide-integrated designs, showcasing their effectiveness in infrared detection. The broadband infrared response of bP arrays makes them suitable for multispectral imaging, which captures spatial images with spectral information. This technique, combined with artificial neural networks (ANNs), is a powerful tool in diverse fields such as biomedical imaging, food classification, and industrial surface damage detection. However, multispectral imaging generates large datasets, making it computationally intensive and latency-sensitive. Edge computing offers a solution by preprocessing images within the sensor, reducing the data load on servers, improving bandwidth, and lowering latency and power consumption. This has driven the development of optoelectronic edge sensors that integrate vision-sensory and computational functionalities. While such systems have been demonstrated using 2D materials for visible/UV imaging, extending this capability to the infrared range using bP would enable advancements in intelligent night vision and multispectral sensing. This paper presents a multifunctional image sensor that combines multispectral imaging with analog in-memory computing to implement an in-sensor ANN for image recognition, using an array of programmable phototransistors made from few-layer bP (bP-PPTs). The bP-PPTs are sensitive to a broad infrared spectral range (1.5–3.1 µm), and their programmability and memory capabilities arise from stored charges in a carefully designed gate dielectric stack. The sensor can be programmed and read out electrically and optically, allowing for optoelectronic in-sensor computing, electronic in-memory computing, and optical remote programming within a single device. Its application as an optical frontend for multispectral infrared image capture, processing, and classification showcases its potential for distributed and remote sensing applications.

The existing literature extensively covers the advantages of 2D materials in optoelectronics, highlighting their tunable properties and suitability for integration. Research on black phosphorus (bP) photodetectors demonstrates their effectiveness across a broad infrared spectrum, with advancements in various configurations like discrete devices, arrays, and waveguide integration. The application of bP in multispectral imaging has also been explored, showing its promise in various fields. The combination of multispectral imaging and artificial neural networks (ANNs) has proven beneficial in applications like biomedical imaging, food quality assessment, and industrial inspection. However, the computational demands of processing large multispectral datasets necessitate efficient solutions like edge computing. Recent research showcases the use of 2D materials in creating optoelectronic edge sensors for visible/UV imaging, integrating sensing and computing functionalities. This paper builds upon this existing research by extending the capabilities of in-sensor computing to the infrared range using bP, addressing the need for efficient multispectral sensing in various applications.

The study utilized an array of black phosphorus programmable phototransistors (bP-PPTs) as the core of a multifunctional image sensor. Each bP-PPT consisted of a few-layer bP flake as the channel, a stack of Al2O3/HfO2/Al2O3 (AHA) dielectrics for charge storage, and an indium-tin-oxide (ITO) transparent top gate electrode. The AHA dielectric stack was designed to store charges in the HfO2 layer, which acts as a charge trap with a high density of trapping sites. This design allows for both electrical and optical programming of the devices. Electrical programming was achieved by applying voltage pulses to the gate, causing charge tunneling between the bP channel and the HfO2 layer. Optical programming involved illuminating the device with visible light pulses (780 nm), which provided sufficient energy to release trapped charges. A 4x3 array of bP-PPTs was fabricated on a single bP flake with a uniform thickness of 11 nm. The fabrication process involved mechanical exfoliation of bP, transfer onto a silicon substrate, electron beam lithography (EBL) for patterning, inductively coupled plasma (ICP) etching, deposition of Ni/Au source and drain contacts, and atomic layer deposition (ALD) of the AHA dielectric stack. The devices were characterized using a source-measurement unit (SMU) to measure conductance and photocurrent. For optical measurements, a tunable telecom band laser and a 780 nm laser diode were used for programming and image input, respectively. A tunable infrared laser was employed to characterize the broadband photoresponse. The bP-PPT array's functionality was demonstrated through two applications: edge detection and image recognition. For edge detection, the photoresponsivity of the array was programmed to represent a convolution kernel matrix, and the photocurrent output was used to perform a multiply-accumulate (MAC) operation on the input image. Image recognition was performed by using the bP-PPT array to implement a convolutional neural network (CNN). The array's conductance was programmed to represent the weight matrix of the CNN, and the source-drain current was used to perform the convolution operation. The MNIST dataset of handwritten digits was used for training and testing the CNN. The programming precision of the bP-PPTs was experimentally verified to be greater than 5 bits.

The research successfully demonstrated a high-precision programmable black phosphorus phototransistor array (bP-PPT) with over 5-bit resolution, achieving a record-high number of conductance levels in charge storage devices. Both electrical and optical programming methods were successfully implemented, offering versatile control over the devices' conductance and photoresponsivity. The long retention time of the stored charges (over 1000 seconds for optically programmed states and >2000 seconds for electrically programmed states) ensured the stability of the programmed states for practical applications. The broadband photoresponse of the bP-PPTs, spanning from the near-infrared (NIR) to the mid-infrared (MIR) spectral range (1.5-3.1 µm), was also confirmed and demonstrated tunability. This tunable photoresponse enabled the array to function as an optical frontend capable of multispectral image capture and preprocessing, showcasing its ability to perform in-sensor edge detection with high accuracy (over 92% correlation coefficient with simulated results). The bP-PPT array was further utilized to implement a convolutional neural network (CNN) for image recognition, achieving an accuracy of 92% on the MNIST dataset, closely matching simulation results (95%). The successful implementation of the CNN demonstrates the potential of this technology for in-sensor computing tasks.

The findings directly address the need for efficient and low-power image processing in distributed systems and robotics. The developed bP-PPT array effectively combines multispectral image sensing with in-sensor computing, significantly reducing communication latency and power consumption compared to conventional approaches. The high accuracy achieved in both edge detection and image recognition tasks validates the feasibility and efficacy of the proposed approach. The 5-bit programming precision, while lower than that of digital computers, is sufficient for the analog in-sensor computing required for edge applications that prioritize low-power and low-latency operations. The demonstrated broadband infrared capabilities of the sensor open up new possibilities for multispectral sensing applications across various fields. The results are relevant to both the materials science and computer engineering communities, demonstrating a significant advancement in the development of intelligent image sensors.

This research successfully demonstrated a multifunctional black phosphorus-based image sensor capable of both multispectral imaging and in-sensor computing. The high-precision programmable bP-PPT array achieved high accuracy in edge detection and image recognition tasks, highlighting its potential for applications in edge computing. Future research could focus on scaling up the array size to create higher-resolution image sensors and exploring heterogeneous integration with other 2D materials to further enhance performance and functionality. The integration of advanced programming techniques, such as spatial light modulation and wavelength division multiplexing, could also enable the development of more complex deep neural networks for more sophisticated machine vision applications.

The current study used a relatively small 4x3 array of bP-PPTs. Scaling up to larger arrays may present challenges in maintaining uniformity and achieving consistent performance across all pixels. The 5-bit programming precision, while sufficient for many edge computing applications, is lower than that achievable with digital computers. This limitation might restrict the complexity of the implemented neural networks. The long-term stability of the optically programmed states might need further investigation for real-world deployments.