A deep unrolled neural network for real-time MRI-guided brain intervention

Real-time guidance and visualization are crucial for robot-assisted imaging, but relatively slow MRI imaging speed limits clinical translation. In neurosurgical interventional MRI (i-MRI), balancing temporal and spatial resolution remains challenging. While acceleration strategies such as bSSFP, parallel imaging, generalized series, and keyhole imaging have been explored, they do not fully meet real-time requirements. Compressed sensing and k-t methods, as well as low-rank plus sparse (L+S) decompositions and nonlinear inverse reconstructions, improve temporal resolution but often suffer from limited spatial resolution, heavy parameter tuning, and long computation times that impede online use. Unrolled deep networks offer speed and quality with improved interpretability; however, state-of-the-art low-rank+sparsity unrolled models (e.g., SLR-Net, L+S-Net) mainly target retrospective Cartesian reconstructions and are not designed for online i-MRI with non-Cartesian sampling. This study addresses the research question of whether an unrolled network that jointly leverages low-rank and sparsity priors and is tailored to multi-coil golden-angle radial, group-based acquisition can deliver real-time, high-resolution i-MRI suitable for MRI-guided brain intervention.

Prior acceleration approaches include bSSFP, parallel imaging, generalized series, and keyhole imaging, which face limitations in meeting stringent real-time constraints. Compressed sensing (CS) methods and k-t frameworks exploit sparsity and temporal redundancy to accelerate dynamic MRI, while low-rank modeling (e.g., partially separable functions, k-t PCA, joint low-rank+sparsity) and L+S/RPCA decompositions robustly separate background and dynamic components for dynamic MRI. Nonlinear inverse (NLINV) reconstructions with undersampled radial data have enabled real-time cardiovascular MRI but at lower spatial resolution than neurosurgical needs. Deep learning methods (AUTOMAP, GANs, U-Nets, transformers, diffusion models) and recurrent networks (CRNN, recurrent U-Nets) improve speed and quality but can require large datasets and may have limited generalizability. Unrolled networks (e.g., cascaded CNNs, ISTA-Net, ADMM-Net, variational networks) increase interpretability and stability. Low-rank+sparsity unrolled models (SLR-Net, L+S-Net) achieve state-of-the-art performance for dynamic MRI but are typically designed for retrospective Cartesian data and not for online non-Cartesian i-MRI. The LSFP algorithm combined low-rank and sparsity with framelets and a primal-dual fixed-point optimizer for i-MRI, but required complex parameter tuning and long computation time, motivating an unrolled, learnable, real-time solution.

- Model: LSFP-Net unrolls the LSFP (Low-rank and Sparsity with Framelet and Primal-Dual Fixed Point) iterative reconstruction into a neural network with Nb blocks (iterations). The image sequence x is decomposed into low-rank L and sparse S (x = L + S). The LSFP objective combines data fidelity with nuclear norm on L and sparsity on S, including temporal total variation (along time for S) and framelet-domain sparsity. In LSFP-Net, sparsifying transforms are learned via 3D convolutional blocks; transform pairs for low-rank and sparse components are learned separately. Complex inputs are split into real/imag channels. Conv kernel size is 3×3×3; first/last layers have 2 kernels; intermediate layers have 32; ReLU activations are used. Hyperparameters are learned during training, removing manual tuning. - Sampling and reconstruction scheme: Multi-coil golden-angle radial sampling (2D) and stack-of-stars (3D) with trajectory repeated every group of spokes to support group-wise reconstruction. Group-based online reconstruction shares overlapped spokes across groups, enabling low latency. For 3D, data are FFT-detangled along z and reconstructed slice-by-slice with 2D LSFP-Net to reduce memory while allowing anisotropic resolution. - Parameterization: Trade-offs between spokes per frame (SPF) and frames per group (FPG) with p = m × n spokes per group. Higher SPF improves per-frame quality but reduces temporal resolution; lower SPF increases temporal resolution and computation. For phantom/cadaver studies, SPF=20 and FPG=5 were selected. Iteration depth Nt and convolutional depth Nc were studied; Nt=3 and Nc=3 were chosen for experiments based on quality-time trade-off. - Training datasets: • Simulated brain intervention dataset: 10 healthy subjects on 3T (uMR 790), 8 coronal slices per subject, matrix 128×128, 11 channels (T1W FSE FLAIR). Four intervention setups (2 unilateral, 2 bilateral), 200 frames per slice. 256 sequences (8 subjects) for training, 64 sequences (2 subjects) for testing. Golden-angle radial k-space simulated via multi-coil NUFFT; fully sampled spokes ~201 for 128×128. Acceleration factor R = SPF/201. • DBS postoperative dataset: 3D T1 FSPGR from DBS patients (1.5T GE), interventional features (electrodes) extracted to simulate intervention sequences. 2400 coronal slices (5 frames each) from 23 patients for training/validation; 188 slices (5 frames each) from 6 patients for testing. Golden-angle radial sampling simulated, 8 sensitivity maps, 402 spokes/frame, 512 readout points. - Network training: Implemented in PyTorch with torchKbNufft. Adam optimizer (β1=0.9, β2=0.999, ε=1e-8), LR=1e-4, batch size 1, 100 epochs (~5 hours). Hardware: Ubuntu 20.04, i9-12900K CPU, RTX 3080 Ti GPU (12 GB). - Baselines for comparison: Iterative CS (L+S, LSFP) and DL-based methods (CRNN, ISTA-Net, SLR-Net, L+S-Net). Unrolled iterations set to have similar parameter counts: ISTA-Net 12, SLR-Net 4, L+S-Net 15, LSFP-Net 2 (training comparisons); LSFP-Net 3 used for phantom/cadaver based on trade-off. - Real-time system integration: LSFP-Net deployed via Gadgetron on a 3T diagnostic scanner (uMR 790). Workflow: k-space streamed to Gadgetron server (Ubuntu 20.04, AMD Ryzen 9 5950X CPU, NVIDIA RTX 3090 24 GB), reconstructed online, displayed immediately on MR console. - Interventional device: MR-compatible, 4 DOFs (3-DOF ball-joint ±30° for manual trajectory; 1-DOF lead screw-nut mechanism, 88 mm travel) actuated remotely by a stepper motor in control room via torque rods and flexible shaft. All scanner-room components MR-compatible; ferromagnetic parts located in control room. MR compatibility verified (no imaging effect). - Experimental protocols: • Phantom experiments (2D GRE golden-angle radial): FOV 256×256 mm², matrix 256×256, slice 5 mm, 17 channels, TR/TE=4/2.01 ms, flip=30°. 100 groups (100×100 spokes) over 40 s. Fully sampled images (402 spokes, 1.6 s) before/after. Real-time acquisition at 20 spokes/frame (80 ms/frame), 5 frames/group (400 ms/group); reconstruction time 370 ms/group. • Phantom experiments (3D GRE stack-of-stars): FOV 256×256×24 mm³, matrix 256×256×8, slice 3 mm, 17 channels, TR/TE=4.58/1.76 ms, flip=10°. 20 spokes/slice, 5 volumes/group; 11 groups over 40.3 s. Reconstruction time 2.96 s per group; 732.8 ms/volume acquisition. • Cadaver head experiment: 3D T1 FSP GRE anatomy (TR/TE=7.22/3.1 ms, flip=8°, matrix 300×320×208, FOV 240×256×166.4 mm³, 0.8 mm slices). Real-time intervention imaged with same 3D GRE stack-of-stars parameters as phantom; target lateral ventricle for drainage; trajectory planned with MR-visible phantom; positions validated with fully sampled 3D scans before/after. An MR-compatible camera monitored device motion.

- LSFP-Net reconstruction quality and speed (simulated intervention, 10 spokes/frame, R≈20): • Achieved near-highest PSNR and SSIM among all methods; visually fewest artifacts with accurate recovery of interventional features. • Reconstruction time 0.21 s per group (5 frames), about 10× faster than CS-based L+S and LSFP; comparable to other DL methods. - Acceleration factors and runtime: For 5, 8, and 20 spokes/frame (R≈40, 25, 10), LSFP-Net delivered near-optimal PSNR/SSIM; runtime <1 s and >20× faster than LSFP, ~3× faster than L+S-Net in some settings. - Depth and complexity trade-offs: Increasing iterations Nt from 1 to 11 improved PSNR/SSIM (28.42/0.93 to 39.11/0.99) with time rising 0.11 s to 1.02 s. Selected Nt=3 and Nc=3 for experiments as best trade-off. - SPF/FPG trade-offs: Higher SPF improves per-frame image quality but lowers temporal resolution; lower SPF increases temporal resolution but increases computation due to more frames per group. Chosen SPF=20, FPG=5 for experiments. - DBS postoperative dataset (simulated intervention): LSFP-Net achieved PSNR/SSIM ≈ 28.45/0.92, comparable to L+S-Net (28.25/0.95), with reconstruction time 0.23 s vs 1.30 s (≈5× faster); outperformed iterative methods in speed. - Real-time 2D i-MRI performance (phantoms): 80 ms/frame, 400 ms/group acquisition; LSFP-Net reconstruction time 370 ms/group enabling 0.4 s latency including communication and processing. In-plane resolution 1×1 mm^2. • Fruit phantom: Needle advanced −61 mm at 1.525 mm/s over 40 s; measurement error vs theoretical depth <1 mm. • Porcine-brain phantom: Needle advanced −90 mm at 2.25 mm/s over 40 s; measurement error ≈1 mm. - Real-time 3D i-MRI performance (phantoms and cadaver): 732.8 ms/volume, 3.66 s/group acquisition; reconstruction time 2.96 s per group; overall latency 3.66 s; in-plane resolution 1×1 mm^2. Interventional procedure clearly visualized in specific slices. - System integration: Demonstrated end-to-end online reconstruction with Gadgetron and MR-compatible device enabling remote-controlled intervention under real-time imaging.

The study demonstrates that unrolling the LSFP model into LSFP-Net and combining it with a group-based, golden-angle radial acquisition enables online, low-latency i-MRI with high spatial and temporal resolution appropriate for neurosurgical guidance. By exploiting both low-rankness of the background and sparsity (including spatial sparsity) of the interventional features, LSFP-Net improves reconstruction fidelity and robustness over methods that use only sparse priors. Compared with CS-based iterative methods, LSFP-Net reduces iterations and computation time, satisfying real-time constraints. Relative to prior unrolled networks optimized for retrospective Cartesian reconstructions, the proposed design addresses online reconstruction with non-Cartesian sampling and group-wise processing, enhancing motion resilience and latency. The phantom and cadaver experiments validate that the system can accurately monitor needle advancement, with sub-millimeter to ~1 mm agreement against planned depths, and 0.4 s (2D) and 3.66 s (3D) latencies. These results suggest LSFP-Net can facilitate simultaneous imaging and intervention within the MR bore, potentially improving accuracy and efficiency of MRI-guided neurosurgery.

This work introduces LSFP-Net, an unrolled deep network tailored for real-time i-MRI reconstruction using multi-coil golden-angle radial sampling and group-based processing. Integrated with a custom MR-compatible interventional device and deployed on a clinical 3T scanner via Gadgetron, the system achieved 2D imaging at 1×1 mm^2, 80 ms temporal resolution, 0.4 s latency, and 3D imaging at 1×1 mm^2 in-plane, 732.8 ms per volume, 3.66 s latency. LSFP-Net delivered superior or comparable image quality to state-of-the-art methods with markedly reduced computation time and strong generalizability across simulated intervention and DBS datasets. The approach shows promise for incorporation into diagnostic scanners for image-guided neurosurgery. Future work includes larger-scale training/testing, integration with a fully fledged robotic system, and obtaining regulatory and ethical approvals for live patient use.

- Clinical translation requires a fully integrated robotic platform and regulatory/ethical approvals; current validation includes phantoms and a cadaver head but not live patients. - Training datasets, while diverse (simulated intervention and DBS postoperative data), are of limited size; broader, multi-center datasets could further improve generalizability. - Increasing network depth (e.g., more convolutional layers) can degrade generalizability and increase runtime; careful balancing is required. - SVD and low-rank operations are computationally intensive; although mitigated by unrolling and group-based schemes, they remain a consideration for scaling.