Ultra-high spatio-temporal resolution imaging with parallel acquisition-readout structured illumination microscopy (PAR-SIM)

The study addresses the challenge of achieving ultra-high temporal resolution in live-cell structured illumination microscopy (SIM) without sacrificing spatial resolution or incurring high system cost and algorithmic complexity. While SIM is valued for its ~100 nm spatial resolution and fast imaging, its speed is fundamentally constrained by the tradeoff between sCMOS rolling-shutter exposure/readout timing and the switching speed of fringe modulators (e.g., SLMs). This mismatch leaves a significant portion of the sensor unexposed during high-speed acquisition and limits frame rate and ROI size, especially at low exposure times that reduce signal-to-noise ratio (SNR). The authors propose PAR-SIM, a hardware-synchronized, parallel acquisition-readout approach that exploits the rolling shutter’s row-wise timing to project and acquire multiple SIM-modulated sub-ROIs per frame in parallel. The goal is to boost spatio-temporal information throughput and frame rates while maintaining SNR and enabling faithful reconstruction for live-cell dynamics.

The introduction reviews advances in high-speed SIM: HessianSIM enabled 188 Hz imaging of fast organelle dynamics. Rolling reconstruction and Sparse-SIM (3-rolling reconstruction) achieved up to 564 Hz and resolved ~61 nm fusion pores. GI-SIM revealed organelle interactions impacting ER and mitochondrial networks, and rDL GI-SIM (deep learning-based) quantified ciliary beat frequencies in mouse ependymal cells (17–49 Hz, mean 32 Hz), with relevance to human airway epithelia and SARS-CoV-2 replication. Previous acceleration strategies include reduced raw frame requirements, GPU/parallelized reconstruction, and faster sCMOS/SLM hardware. However, a core bottleneck remains: the exposure-readout speed of sCMOS (e.g., ~9 μs/line with 2048 pixels) versus fringe modulator switching times, which leads to unused sensor area and constrained framerates/ROI. Residual crosstalk issues can persist due to imperfect synchronization in rolling-shutter modes despite optimizations.

PAR-SIM is a hardware-synchronized SIM modality that parallelizes exposure and readout by exploiting the sCMOS rolling shutter’s row-wise timing. Key elements: (1) Optical setup: Airy Polar-SIM excitation mounted on a Nikon Ti2-E microscope. The SLM (QXGA/SXGA, 4D Displays) generates 2D-SIM patterns relayed to the objective back pupil via a transform lens, polarization rotator, 0th-order stop, and a 4f relay. Emitted fluorescence is imaged through a multiband dichroic onto the camera. A custom photolithography-made sub-ROI slit selects rectangular windows. A pair of relay lenses (L1: f=100 mm; L2: f=150 mm; magnification 1.5) images the sub-ROIs onto an sCMOS camera (Hamamatsu ORCA-Flash4.0 V3). An xy galvo set (Cambridge Technology 6210H) is placed at the conjugate plane between L1 and L2 to scan sub-ROIs across the sensor. (2) Control and synchronization: An NI PCIe-6738 DAQ provides synchronized analog and digital outputs to trigger the camera, SLM (SPO1/SPO2), and galvos. Custom LabVIEW software generates the timing waveforms. The camera operates in rolling-shutter mode (center-out readout), producing a temporal slope across rows. PAR-SIM uses this slope to expose sub-ROIs on rows that are currently open while other rows are being read out, enabling simultaneous acquisition and readout of different SIM phases/angles. (3) Parallel acquisition scheme: The same sample sub-ROI, modulated by different SIM phases/angles, is projected sequentially onto different sensor areas along the same rows using galvo deflection. In each frame, 2×3 sub-ROIs (total 6) are acquired: while sub-ROIs 1–3 are exposed with phases/angles P1–P3 on rows 0–N/2 (Frame K), rows N/2–N read out sub-ROIs 7–9 (P7–P9) from Frame K−1; then sub-ROIs 4–6 (P4–P6) are exposed while P1–P3 are read out. Scan direction depends on whether the top or bottom half of the sensor is used, to match the rolling readout direction. (4) Timing model: Three timing definitions for the SLM are used: illumination window, actual pattern exposure duration (including polarity reset; twice nominal), and switching duration (fixed per SLM series). Synchronization parameters include T1 (galvo dwell), Ti (SLM duration per pattern), Ts (SLM high-voltage duration), and delays TD1 (inter-frame exposure delay), TD2 (camera exposure vs. scan start compensation), TD3 (SLM start delay after galvo settling). Typically TD1=TD2=0 to maximize speed; TD3 buffers galvo settling with constraint Ts+TD3 ≤ Ti. To avoid crosstalk, opened frame rows slightly exceed twice the sub-ROI row height to create margins, with the overall frame time ≈ TD1 + TD2 + 6·Ti. Optimal throughput occurs when per-row exposure matches per-row readout and N·t/2 ≈ 3·Ti. (5) Hardware limits and settings: The ultimate speed is limited by SLM switching. Example settings (from Table 1): for SXGA at 200 μs actual exposure, switching 745 μs, Ti≈745 μs and T1≈795 μs yield 484 occupied rows and ~209 Hz; for QXGA at 200 μs, switching 589 μs, Ti≈589 μs, T1≈649 μs yield 400 rows and ~256 Hz. Similar configurations for 400 μs exposures achieve ~167–204 Hz depending on SLM series. (6) Reconstruction: To tolerate ultra-low SNR imposed by ultra-short exposures, a physical model-based SIM reconstruction algorithm is employed to recover high-fidelity super-resolved images with ~100 nm resolution and minimal artifacts.

- PAR-SIM achieves ultra-high spatio-temporal information flux of 132.9 MPixels s−1, corresponding to 256 Hz over 1352×384 pixels, representing a 9.6× improvement over recent techniques at comparable detector sensitivity. - The system attains ~100 nm spatial resolution while operating at very low signal levels, with SNR as low as −2.11 dB in sub-ROIs, enabled by a physical model-based reconstruction. - Hardware-level parallelization yields theoretical 6× SIM frame-rate improvement by projecting 2×3 sub-ROIs per frame and exploiting simultaneous exposure-readout on rolling-shutter sCMOS sensors. - Demonstrated high-speed live-cell imaging captures mitochondrial dynamic tubulation and membrane fusion in live COS-7 cells at up to 408 Hz. - Robust reconstructions of diverse cellular structures (microtubules, actin, mitochondria) are obtained under dual excitations and ultra-short exposures, with minimal artifacts. - Practical frame rates validated with different SLM series and exposure settings: approximately 256 Hz (QXGA, 200 μs), 204 Hz (QXGA, 400 μs), 209 Hz (SXGA, 200 μs), and 167 Hz (SXGA, 400 μs), with corresponding occupied sensor rows detailed in timing Table 1.

PAR-SIM addresses the core speed bottleneck of SIM—mismatch between sCMOS rolling readout and SLM switching—by synchronizing and parallelizing exposure with readout, thereby utilizing otherwise idle sensor rows. This strategy increases ROI coverage and frame rate without requiring expensive higher-speed cameras or complex algorithmic shortcuts that may compromise SNR. By maintaining adequate photon budgets per sub-ROI and using a physics-based reconstruction, the method tolerates ultra-low SNR while preserving ~100 nm resolution and minimizing artifacts. The demonstrated ability to capture fast mitochondrial dynamics (tubulation and fusion) at several hundred Hz highlights the method’s utility for observing rapid organelle motions where temporal resolution is critical. Beyond SIM, the parallel exposure-readout control scheme could generalize to other advanced imaging modalities constrained by rolling-shutter timing, improving their throughput with strategic hardware timing control.

The work introduces PAR-SIM, a parallel acquisition-readout strategy for SIM that synchronizes SLM patterning, galvo scanning, and sCMOS rolling-shutter timing to acquire six SIM-modulated sub-ROIs per frame. It delivers state-of-the-art spatio-temporal flux (132.9 MPixels s−1), high frame rates (up to 256 Hz in validated configurations and 408 Hz demonstrated for specific live-cell dynamics), and ~100 nm resolution at very low SNR. The approach expands ROI coverage and reduces dependence on costly detectors or complex reconstruction tactics, enabling faithful, high-speed live-cell imaging of rapid organelle dynamics. Future directions include optimizing SLM technology to reduce switching durations, refining synchronization to further suppress crosstalk, extending to 3D or multi-color SIM, integrating with faster or more sensitive detectors, and adapting the parallel exposure-readout paradigm to other advanced imaging systems.

- Speed is ultimately limited by SLM switching durations and pattern exposure constraints (Ti), which bound achievable frame rates. - Rolling-shutter dependence requires precise synchronization; mis-timing risks pattern crosstalk between sub-ROIs or frames, especially if Ti exceeds the allowable margins. - The approach relies on custom hardware control (NI DAQ, LabVIEW) and careful galvo settling (TD3), increasing system complexity and sensitivity to timing jitter. - ROI sizing and layout must account for margins to prevent overlap on the sensor, potentially reducing maximal usable field for some configurations. - Performance metrics (frame rate/occupied rows) are hardware-specific (SLM series and camera line times), which may limit generalizability across different setups. - Reconstruction at ultra-low SNR, while feasible with the physical model, may still be sensitive to parameter tuning and could degrade with further reduced exposure or adverse sample conditions.