Optical neural network via loose neuron array and functional learning

The paper addresses the challenge of training and deploying programmable optical neural networks (ONNs) using real, imperfect hardware whose components are non-differentiable and difficult to model accurately. While deep learning has achieved significant success in software, hardware constraints such as power, bandwidth, and the physical imperfections of optical and electronic components hinder ONN programmability. Traditional computational optics rely on simplified analytical models that cannot capture complex micro-structures, material inconsistencies, inter-reflections, interference, and environmental effects (e.g., thermal, moisture, electronic noise). The authors introduce the concept of a loose neuron array—non-handcrafted, loosely connected physical neurons—and propose a functional learning (FL) paradigm to train such arrays end-to-end without explicit gradients or precise calibration. They instantiate this with a light field neural network (LFNN), an incoherent, programmable ONN that modulates visible light via liquid crystal (LC) layers and polarizers, targeting tasks such as classification, recognition, and depth estimation.

The work situates within efforts to realize high-bandwidth, low-latency, power-efficient ONNs, referencing prior diffractive and nanophotonic neural networks and hybrid optical-electronic approaches. Existing methods often assume idealized, differentiable models and require precise calibration, which is infeasible for complex, noisy, multimodal hardware. Alternative stochastic optimization methods (finite differences, genetic algorithms) can optimize small parameter sets but scale poorly for high-dimensional systems and are time-inefficient when each parameter update requires physical measurements. The paper argues for a new learning paradigm that bypasses explicit gradient models and leverages implicit training via functional approximation informed by measured device responses.

The authors propose Functional Learning (FL) to train a model-free, non-differentiable physical system. Let f(x; p) be the unknown mapping implemented by hardware with parameters p. FL introduces a Functional Neural Network (FNN), g(z)(x; p), to approximate f and splits the goal into two alternating subproblems: (1) z-learning: fit g to the hardware’s measured responses with p fixed; (2) p-learning: optimize p (hardware control parameters) for the target task with z fixed. Alternating minimization iteratively updates z and p. FNN architecture: a physically inspired functional basis block models dense light field connections from each input neuron to each output neuron (input-to-output connections), modulated by LC neurons that attenuate connections. This basis is followed by five 3×3×64 CNN layers with ReLU to nonlinearly combine features. Multi-resolution design down-samples inputs and LC controls by 2 and merges resolutions via trainable weights to enhance robustness and reduce noise. Training protocol: At each epoch, z-learning collects z-data by sending impulses (a random subset of 1024 samples) through the LFNN and capturing outputs; L1 loss and Adam (lr=0.001) optimize z. p-learning uses task data (e.g., MNIST/CIFAR10) and Adam (lr=0.001) to update p. For multi-layer LFNNs, outputs are re-projected to the input plane for the next layer, with simple non-digital-friendly activations: X-activation (output minus its 180° rotation) enabling effective positive/negative operations, followed by batch normalization and an amplitude clamp [0,1] in the input driver. Fine-tuning freezes earlier layers and updates the last when loss plateaus. Sparsification: To reduce memory/compute, weak LC connections are pruned via hierarchical quad-tree testing against energy-change thresholds (20% at finest, 10% at mid, keep all at coarsest), pruning ~99.96% of LC connections in the implementation. LFNN hardware: Prototype comprises an input LCD plane, two LC layers (Chimei Innolux AT070TN83 LCDs with backlights/polarizers removed), three linear polarizers, and a diffuser and polarizer at the output plane, imaged by a camera (The Imaging Source DFK 33G274; 1600×1200, 12-bit, 4.40 μm pixels; 12.5 mm f/1.6 lens). Layer spacing is 30 mm. Neuron binning: 15×13 LCD pixels per input/LC neuron; 12×15 camera pixels per output neuron. Effective resolutions for input, output, and LC are 32×32 (RGB). LFNN trainable vars per layer: 12,288 (6,144 LC controls + 6,144 input/output gains). FNN trainable vars per layer: ~28,438,144. Training uses a low-speed camera (4 fps), ~4 minutes/epoch; ~400 minutes for 100 epochs. Datasets and readout: MNIST (28×28 grayscale) and CIFAR-10 (32×32 RGB). For classification, the output plane is partitioned into spatial regions whose summed RGB intensities encode class probabilities. For single-class recognition, the output plane is split into positive/negative regions; summed intensities yield decisions. For depth estimation, last-layer RGB sums represent depth. Multi-layer experiments cycle captured outputs with activations. Comparators: (a) Forward analytical modeling with measured PSFs and per-neuron LUTs; (b) Finite differences; (c) Genetic algorithm (scikit-opt), all under comparable time/epoch budgets.

- 1-layer MNIST (physical LFNN): 91.02% test accuracy; FNN prediction 91.39%; a matched digital fully connected layer achieves 92.71%. - 2-layer MNIST (physical LFNN): text reports 94.35% (FNN 94.52%); Table 2a reports LFNN 94.77%, FNN 95.45%, digital DNN 98.32%. - 3-layer CIFAR-10: LFNN 45.62%, FNN 46.19%, digital DNN 53.62%. - Neuron array simulations (1-layer MNIST): Regular-2 91.03%, Regular-3 92.07%, Normal-3 92.40%, Uniform 92.45% (uniform random arrays perform best, surpassing handcrafted layouts). - Neuron array simulations (2-layer MNIST): Regular-2 96.61%, Regular-3 97.30%, Normal-3 97.55%, Uniform 97.65%. - Neuron array simulations (3-layer CIFAR-10): Regular-2 47.48%, Regular-3 50.61%, Normal-3 51.73%, Uniform 52.53%. - Training paradigm comparison on 1-layer MNIST under comparable budgets: forward model 23.50%; finite differences 8.594%; genetic algorithm 14.06%; functional learning ~90.78–91%. - Robustness to random LC neuron failures (Bernoulli arrays, 1-layer MNIST): Simulation accuracies for 0/20/40/60% disabled: 89.80%/82.07%/79.17%/77.29%. Physical LFNN: 89.13%/81.83%/79.36%/75.65%. - Single-class recognition at light speed (1-layer): MNIST digit ‘0’ 98.12%; CIFAR-10 ‘plane’ 77.30%. - Depth estimation (4-layer LFNN) on RGB-D ‘coffee mug’: average relative MSE 0.0073 (train 4,500; test 300). - Practicality: With low-cost, uncalibrated optics/electronics, FL achieves results comparable to equal-layer digital dense networks while using orders of magnitude fewer trainable hardware parameters.

The findings demonstrate that functional learning can train non-differentiable, uncalibrated optical hardware end-to-end to perform meaningful inference tasks, bridging the gap between imperfect physical systems and gradient-based learning. The LFNN validates a programmable incoherent ONN capable of real-time optical computation and sensing, with accuracies approaching digital baselines in shallow settings. Simulations indicate that non-handcrafted, uniformly distributed neuron arrays can surpass handcrafted, regular layouts, supporting the premise that data-driven training of flexible hardware topologies can outperform manual design. FL’s alternating z/p learning efficiently exploits measured device responses to implicitly propagate gradients, avoiding the prohibitive costs of finite-difference or genetic search in high-dimensional parameter spaces. The approach provides a methodology for hardware design and control that tolerates manufacturing and assembly imperfections, potentially reducing costs while expanding the design space. Limitations in speed and scale (camera frame rate, parameter counts) currently bound performance; however, the paradigm scales with faster opto-electronic components and improved FNN efficiency. The work outlines routes for all-optical or analog photoelectric nonlinear activations (e.g., saturable absorbers, memristors) to enable deeper, fully optical stacks.

The paper introduces functional learning (FL), a training paradigm that uses a functional neural network to approximate and learn the behavior of non-differentiable, model-free physical systems, and demonstrates it on a light field neural network (LFNN), a programmable incoherent optical neural network. The approach achieves competitive performance on MNIST and CIFAR-10 classification, single-class recognition, and RGB-D depth estimation using inexpensive, imperfect hardware, and shows robustness to random neuron failures. Simulations further suggest that non-handcrafted, uniform neuron arrays can outperform regular layouts. The contributions open avenues for low-latency, high-bandwidth, power-efficient optical computing units and programmable lenses/displays/sensors. Future work includes scaling neuron counts and layers, reducing noise via optical/electronic design, integrating high-speed sensors and micro-LEDs, developing all-optical or analog activation elements (e.g., saturable absorbers, memristive devices), optimizing FNN architectures for efficiency, and expanding to real-world, in-the-wild optical inference.

- Training throughput is limited by sensor speed; the prototype uses a 4 fps camera, yielding ~4 minutes per epoch and ~400 minutes for 100 epochs. - FNN complexity is high (~28M parameters per layer), incurring substantial training cost; improved architectures are needed. - Hardware is built from low-cost, uncalibrated components with significant noise, limited transmittance, narrow viewing angles, poor linearity, neuron inconsistency, and biased alignments; these reduce achievable accuracy. - Analytical forward modeling is impractical at scale due to system complexity; alternative stochastic methods scale poorly for many parameters. - Scaling resolution doubles both dimensions, inducing a 16× increase in light-field connections, straining compute and data collection. - Multi-layer training requires more time to gather z-data and reach convergence; all-optical nonlinear activation is not yet implemented and currently relies on electronic cycling. - Experiments are conducted in closed environments with controlled illumination; direct inference on unconstrained real-world scenes remains future work.