Generative machine learning for robust free-space communication

Free-space optical (FSO) communications offer high-capacity wireless information transfer using multiplexing techniques, including spatial multiplexing via orbital angular momentum (OAM). By transmitting superpositions of OAM states that manifest as distinct petal-pattern images, large communication alphabets are possible. A key challenge is robust demodulation at the receiver under realistic conditions: random atmospheric turbulence, attenuation, and detector dark noise degrade received images, causing cross-talk and reducing classification accuracy. Recent advances in generative models have impacted diverse fields. In FSO, supervised learning with neural-network classifiers has been applied, but these approaches typically require extensive labeled datasets of distorted receiver images, which are costly to acquire and difficult to label under random, time-varying turbulence and attenuation. This limits classification efficiency and practicality. This work develops a receiver-end communication scheme using a generative machine learning approach to mitigate turbulence, attenuation, and noise without feedback or adaptive optics. A generative neural network (GNN), implemented as a convolutional denoising autoencoder, reconstructs clean optical mode profiles from noisy, distorted receiver images. A CNN, trained only on undistorted modes with added dark noise, then demodulates the generated profiles. The system improves classification accuracy and reduces cross-talk across a range of turbulence and detector noise conditions in both simulations and experiments, and can be pre-trained for scalable deployment.

The study builds on prior use of supervised learning and CNN-based classifiers for OAM-FSO demodulation, which require large sets of pre-labeled, distorted receiver images and may not generalize well under random turbulence and attenuation. Denoising autoencoders have proven effective at reconstructing clean inputs from corrupted data in other domains. The authors extend these approaches by employing a generative denoising autoencoder to reconstruct OAM mode profiles under turbulence and attenuation, followed by a CNN trained solely on undistorted (target) modes with dark noise. This reduces reliance on extensive labeled distorted datasets and avoids adaptive optics. The work references advances in OAM multiplexing for FSO links and prior turbulence mitigation via neural networks, positioning their GNN+CNN pipeline as a robust, pre-trainable alternative focused on generative correction plus classification.

Experimental setup: A Ti:Sapphire laser at 795 nm is coupled through a fiber and a half-wave plate to two spatial light modulators (SLMs). SLM 1 encodes a desired OAM superposition (l = 0 to ±10) on the first diffraction order (zeroth order blocked). SLM 2 applies a random phase mask simulating turbulence of given Cn^2. A variable attenuator (VA) reduces signal-to-noise ratio before detection on a CCD camera. Captured images are passed to the GNN for correction, then to a CNN classifier. GNN architecture (convolutional denoising autoencoder): Encoder with convolutional layers (kernel 5×5, zero padding, ReLU activation, stride 2, 3 feature maps), followed by max-pooling (2×2), and a fully connected layer to a latent space (latent size varied; optimized often at 32×32). Decoder mirrors the encoder: fully connected layer, convolutional and deconvolutional layers (same kernel/stride settings), ending with a single-feature convolution to generate the corrected image. Dropout of 5% is applied after each layer except the encoder’s final fully connected layer and the decoder’s final convolutional layer. Learning rate for optimization set to 0.008. Reconstruction loss is mean squared error (MSE); parameters optimized with Adam. CNN classifier: Convolutional layer (5×5 kernel, zero padding, ReLU, stride 2, single feature map), max-pooling (2×2), fully connected layer (28×28 neurons), and an output layer with softmax cross-entropy loss optimized by Adam. No dropout used. The CNN is trained only on undistorted target OAM modes with added Gaussian noise (σ = 2; σ = 1 in the Fig. 5 experiment). For CNN training, 150 images per OAM value (0 to ±10) are generated; 130 used for training and 20 for testing, yielding unity test accuracy. Data generation and splits (simulations): Simulated turbulence via Kolmogorov/Von Kármán spectrum. Generate 99 random phase screens at Cn^2 = 5×10^-14 m^-2/3 and Z = 500 m; OAM superpositions l = 0 to ±10 yield 1089 images (128×128 resolution). Normalize transmitter intensity prior to propagation. Additive Gaussian noise appended to receiver images; SNR controlled by noise level. For training/testing per SNR set, use 50 images per l (total 550) for training and 49 per l (total 539) for testing. GNNs are trained separately for separate SNR sets. In additional simulations, vary link distance Z (200–800 m) and turbulence strength Cn^2 (9×10^-15 to 1×10^-13 m^-2/3) and receiver noise σ (10, 20, 50, 80). Pre-trained GNN-CNN reconstructs and classifies images in milliseconds. Long-distance turbulent propagation simulation: Place a turbulent phase screen (SLM 2) 1 m from SLM 1; propagate Gaussian beam G(x,y,w0) through SLM 1 (OAM phase mask) and SLM 2 (turbulent phase), then to receiver at Z ≥ 200 m. Parameters: w0 = 4 cm, N = 128, λ = 1550 nm, l0 = 1 mm, L0 = 200 m, Cn^2 ∈ [9×10^-15, 1×10^-13] m^-2/3. Receiver intensity I computed with Fourier propagation and additive dark noise N(0,σ). Laboratory-analog simulation for pretraining: Generate 100 random turbulent phase masks with N = 800, λ = 795 nm, l0 = 1 mm, L0 = 25 m, Cn^2 = 36.8×10^-11 m^-2/3. Propagate with w0 = 0.45 mm, distances 0.2 m (SLM 1→SLM 2) and 0.6 m (SLM 2→CCD). Add Gaussian noise (mean 0, std 0.1), normalize to 8-bit. For each OAM l = 0–10, 100 images total: 90 for training and 10 for testing (total 990 train, 110 test). GNN optimized on these simulated images is then used to reconstruct unknown experimental images at Cn^2 = 51.2×10^-11 m^-2/3, attenuation ratio −1.93 dB. Experimental datasets and protocol: For each fixed turbulence strength (Cn^2 between 8×10^-11 and 80×10^-11 m^-2/3), record 50 images per OAM ±l (0 to ±10), total 550 images. Split into 495 training (45 per ±l) and 55 testing (5 per ±l). Targets are obtained by using a zero-turbulence mask on SLM 2. For attenuation studies, fix turbulence at Cn^2 = 51.2×10^-11 m^-2/3 and vary attenuation ratio from −2.52 to 0 dB; repeat the same train/test split. MSE is computed between desired target and received or GNN-corrected images. Attenuation ratio computed as 10 log10(total intensity with attenuation / without attenuation) in dB.

Simulation results: - SNR dependence (Cn^2 = 5×10^-14 m^-2/3, Z = 500 m): Accuracy improved with GNN across SNRs. At latent size 24×24, accuracy improved from 0.87 to 0.97 (SNR 3.11 dB), from 0.84 to 0.95 (0.11 dB), from 0.77 to 0.92 (−3.87 dB); at latent size 72×72, from 0.68 to 0.86 (−5.91 dB). Even small latent sizes (e.g., 4×4) improved accuracy (e.g., from 0.87 to 0.93 at SNR 3.11 dB). Accuracy gains saturated beyond latent ≈32×32. - Link distance (SNR ≈ −3.87 dB): Accuracy improved with GNN across Z. At Z = 200 m: 0.976 → 0.998 (latent 40×40). Z = 400 m: 0.85 → 0.95 (latent 88×88). Z = 600 m: 0.72 → 0.88 (latent 40×40). Z = 800 m: 0.67 → 0.85 (latent 24×24). - Turbulence strength and detector noise: With latent 32×32, substantial improvements across Cn^2 and noise σ. Example: at σ = 80 and Cn^2 = 3×10^-14 m^-2/3, accuracy 0.83 → 0.971. At Cn^2 = 1×10^-14 m^-2/3, uncorrected accuracies 0.995 (σ=50) and 0.945 (σ=80) improved to 1.0 with GNN. At weak turbulence Cn^2 = 9×10^-15 m^-2/3, σ=80: 0.989 → 1.0. - Latent size benchmark at fixed σ = 50: For Cn^2 = 1×10^-14 m^-2/3, accuracy improved from 0.995 to 1.0 for latent sizes ≥16×16. For Cn^2 = 5×10^-14 m^-2/3: 0.77 → 0.92 at latent 24×24. For Cn^2 = 1×10^-13 m^-2/3: 0.57 → 0.74 at latent 40×40. - Throughput: Pre-trained GNN-CNN reconstructs and classifies images on the order of milliseconds. Experimental results (turbulence): - Across turbulence strengths Cn^2 from 8×10^-11 to 80×10^-11 m^-2/3, the GNN consistently reduced MSE between target and received images, typically by an order of magnitude. Examples: mean MSE reduced from 330.2 to 34.2 (Cn^2 = 8×10^-11 m^-2/3) and from 742.3 to 120.7 (Cn^2 = 80×10^-11 m^-2/3). MSE increases with OAM order, but GNN reduces MSE across all l. Experimental results (attenuation): - With fixed turbulence (Cn^2 = 51.2×10^-11 m^-2/3) and attenuation from −2.52 to 0 dB, the GNN significantly reduced MSE. Examples: at −2.52 dB, average MSE reduced from 1.22×10^4 to 2.5×10^2; at −1.18 dB, from 9.64×10^2 to 52.8. GNN restored spatial features even under strong attenuation, with MSE generally increasing with OAM order but consistently reduced by GNN. Cross-talk reduction with simulated pretraining: - Training GNN and CNN exclusively on simulated images, then applying to experimental test set at Cn^2 = 51.2×10^-11 m^-2/3 and −1.93 dB attenuation, reduced misclassification counts markedly. Out of 50 test images per mode, incorrect predictions decreased from 18 to 3 (|l|=1), 33 to 9 (|l|=3), 48 to 8 (|l|=4), and 26 to 8 (|l|=10). Overall cross-talk between neighboring OAM values was substantially mitigated after GNN correction.

The proposed GNN+CNN pipeline addresses the core challenge of demodulating OAM-based FSO signals degraded by turbulence, attenuation, and detector noise. By reconstructing near-target spatial modes at the receiver without any feedback or adaptive optics, the system substantially enhances classification accuracy across a wide range of SNRs, link distances, turbulence strengths, and detector noise levels. The CNN, trained only on clean target modes with dark noise, benefits significantly from GNN preprocessing, reducing the need for large, labeled datasets of distorted profiles that are difficult to acquire and annotate under random channel conditions. Experimental demonstrations confirm that the GNN reduces MSE by up to an order of magnitude under both increasing turbulence and attenuation, and that simulated pretraining can transfer effectively to experimental data, yielding marked cross-talk reductions. The approach is computationally efficient (millisecond-scale inference), portable, and cost-effective, and may be integrated into existing FSO systems, including prospective applications to quantum communication links where similar atmospheric impairments affect spatial-mode encodings.

This work introduces and validates a generative denoising autoencoder (GNN) paired with a CNN classifier for robust free-space optical communication using OAM modes. The system reconstructs clean spatial profiles from noisy, distorted, and attenuated receiver images, markedly improving classification accuracy and reducing cross-talk in both simulations and experiments, without feedback or adaptive optics. The CNN’s training on only undistorted targets reduces dataset requirements compared to prior supervised approaches that relied on extensive distorted data. The networks are pre-trainable and execute rapidly, facilitating practical deployment. Potential future directions include: extending the method to more complex spatial encodings and larger alphabets; adapting to broader and time-varying turbulence/attenuation conditions; and direct application and testing in quantum communication scenarios where atmospheric turbulence impacts single-photon spatial modes.

- GNNs were trained separately for different SNR image sets, indicating sensitivity to noise-level-specific training. - Experimental validation used laboratory setups with SLM-simulated turbulence and controlled attenuation; real-world field performance under dynamically varying atmospheric conditions was not reported here. - The CNN was trained only on undistorted target modes with added dark noise; performance when trained on broader distributions of distortions was not explored within this study.