Ultra-thin endoscopes, potentially enabling cell-scale imaging in hard-to-reach areas of the body, represent a significant advancement in medical imaging. Miniaturization efforts have focused on using multimode fibers (MMFs) with diameters as small as 0.125 mm, achieving in vivo fluorescence imaging in mice brains. However, MMFs introduce substantial optical distortion, which changes with fiber perturbations, especially in longer fibers needed for deep tissue imaging. Current calibration techniques require measuring the transmission matrix (TM) at both ends of the fiber, which is impractical for ultra-thin endoscopes due to the required distal-end components. This necessitates an approach that only needs access to one end of the fiber. Several methods have been developed for single-ended TM recovery, including those utilizing guidestars, beacons, and reflective structures. Previous work proposed a single-ended TM recovery method based on a reflector stack with varying reflectance at different wavelengths, enabling recovery from reflection matrices at three wavelengths without requiring distal access. This technique avoids the need for measuring at both ends and works for non-unitary TMs. However, solving the resulting non-linear equations currently involves an iterative approach, computationally expensive for high-resolution imaging, and unsuitable for the real-time recalibration needed due to the fiber's sensitivity to environmental changes. The need for a fast and robust method to reconstruct the transmission matrix from single-ended measurements has thus motivated the development of this Neural Network approach.

Several methods aim to reduce computational time for fiber imaging, often exploiting prior knowledge about the fiber. Compressed sampling techniques, for instance, reconstruct full-size TMs from a small subset of measurements. Look-up tables can also improve speed, especially when combined with a reflective beacon. The extended Kalman filter has also been utilized for faster TM reconstruction. Deep learning methods, particularly convolutional neural networks (CNNs), have shown promise in reconstructing images through multimode fibers in both transmission and reflection modes. These methods are fast and learn prior information, but their performance usually suffers under fiber perturbation, due to the lack of reflection calibration measurements needed for unambiguous TM resolution. Furthermore, many focus on amplitude image recovery and use traditional loss functions like mean squared error (MSE), which is not suitable for complex-valued data containing phase information. Therefore, previous AI approaches, while successful at image reconstruction, lack the robustness and speed required for practical application in real-time, single-ended TM recovery from reflection-mode data.

This paper proposes a novel single-ended TM recovery method using neural networks to solve the non-linear equations derived from reflection matrices at multiple wavelengths. Two neural network architectures are explored: a fully connected neural network (FCNN) and a convolutional U-Net. To overcome the challenge of global phase degeneracy inherent in complex-valued TM data, a custom global-phase-insensitive loss function is introduced. This loss function avoids the ambiguities and large errors that conventional loss functions like MSE would introduce due to arbitrary global phase shifts. The method is validated through simulated and experimental data. The simulated dataset consists of 900,000 sets of simulated reflection matrices at three different wavelengths (850 nm, 852 nm, and 854 nm) as input, with corresponding complex-valued, non-unitary TMs at 850 nm as output. The dataset is split into training, validation, and testing sets. The models are trained using the Adam optimizer and the custom loss function. Performance is evaluated using an average loss metric, calculating the average MAE after global phase normalization. Image reconstruction performance is assessed using IMMAE (image MAE) and SSIM (structural similarity index measure). Two imaging modalities are used to test the recovered TMs: wide-field and confocal scanning. The simulation of wide-field imaging assumes a pixel basis at the fiber output, incorporating Gaussian noise to mimic real-world conditions. For confocal imaging, an LP mode basis is used, simulating a confocal scan to obtain the focused spot's power ratio and evaluate image reconstruction quality. The performance and computational resources are compared between the FCNN and U-Net models. Furthermore, the robustness to fiber perturbation is evaluated by simulating column swaps in the reflection matrix measured at the third wavelength. The methodology also includes an experiment using real-world data, where the trained model is tested on experimentally measured TMs and further refined by retraining on a subset of this data to improve its ability to handle the variations found in real-world fiber TMs. The method to generate the training data is based on the model previously implemented in literature. Random tri-diagonal matrices are initially generated to represent the transmission matrices, and then their singular value decomposition is used to create more realistic transmission matrices based on properties of real transmission matrices. Reflector matrices are generated via random complex values to ensure distinct eigenvalues.

The proposed neural network-based approach achieves significant improvements compared to prior methods. In simulated data, both the FCNN and U-Net architectures successfully recover 64 x 64 complex-valued fiber TMs with ≤4% average loss using the custom loss function. The FCNN model shows faster convergence and slightly better accuracy than the U-Net model. The FCNN model converges in 2500 epochs and requires about 182.5 hours of training. Wide-field image reconstruction based on recovered TMs using the FCNN model achieves ≤9% IMMAE and ≥83% SSIM, while confocal imaging achieves even better performance, with ≤5% IMMAE and ≥90% SSIM. The method demonstrates robustness to fiber perturbation, tolerating up to 6% of column swaps in reflection matrices while maintaining acceptable image quality. The approach is shown to be compatible with non-square TMs, achieving ≤8% average loss in the reconstruction of a 6 x 12 TM. The training time for the models depends quadratically on the size of the image dimensions and a trade-off is highlighted between accuracy and memory usage between the FCNN and U-Net models. The training of a model for 1024 x 1024 TMs (32 x 32 image) requires more than 1 TB of memory for FCNN and 1.1 TB for U-Net. The custom global phase-insensitive loss function significantly outperforms the MAE loss function, demonstrating its effectiveness in addressing the global phase degeneracy problem. Finally, the model is cross-validated on experimentally measured TMs, achieving an average loss of 3.42%, demonstrating the applicability of the method to real-world scenarios. The prediction time for TM recovery using the FCNN is around 1 second, which is approximately 4500 times faster than previous iterative approaches, making real-time imaging applications feasible.

The results demonstrate the efficacy of the proposed neural network-based approach for single-ended TM recovery in optical fibers. The significant speed improvement over iterative methods (4500x faster) is a crucial advancement, enabling real-time applications in medical endoscopy. The robustness to fiber perturbation is also a key strength, making the method more practical for in vivo use. The ability to handle non-square TMs further enhances versatility and applicability to various optical systems. The custom loss function effectively solves the global phase degeneracy problem, achieving higher accuracy than conventional loss functions. These advantages collectively establish the neural network-based approach as a superior method for fast, accurate, and robust TM recovery in single-ended fiber optic systems, particularly relevant to endomicroscopy applications. The findings suggest that the approach could be further optimized by incorporating adaptive loss functions or by developing matrix compression techniques to reduce the memory consumption required for training on high-resolution data.

This study presents a novel neural network-based approach for single-ended recovery of optical fiber transmission matrices. The method demonstrates significant improvements in speed, robustness, and adaptability compared to existing iterative methods. The use of a custom global-phase-insensitive loss function addresses the critical issue of global phase degeneracy in complex-valued data. Future research could focus on optimizing model architectures, exploring further advanced methods of data augmentation and loss functions, and implementing real-time in vivo imaging experiments to demonstrate the method’s capabilities in practical endomicroscopic applications. Further research could also be directed at reducing the high memory usage required to train the model on high-resolution images, which currently limits its applicability to larger datasets.

The primary limitation lies in the computational resource requirements for training the neural networks, particularly for high-resolution TMs. Training the model for higher resolution images (e.g., 32x32 pixels) requires substantial memory (over 1 TB). Although the U-Net model requires less memory than the FCNN, it still presents a significant computational challenge. While the model demonstrates robustness to some degree of fiber perturbation, significant changes in fiber conformation might necessitate recalibration or retraining. The reliance on a simulated dataset for initial training assumes that the simulated data adequately reflects the characteristics of real-world fibers, which might not always hold perfectly. The quality of the simulated data influences the model's performance on experimental data.