A deep convolutional neural network for real-time full profile analysis of big powder diffraction data

Advances in X-ray sources, optics and detectors now enable acquisition of large, high-quality powder X-ray diffraction datasets in minutes (laboratory) to milliseconds (synchrotrons), making high-throughput in situ/operando and spatially resolved XRD feasible. However, analysis—not acquisition—has become the bottleneck, especially for multi-dimensional XRD-CT experiments generating terabytes of data. Conventional least-squares full-profile methods like Rietveld yield rich physico-chemical information but do not scale to big data and limit real-time decision-making during beamtime. Deep learning, particularly CNNs, offers scalability and fast inference, but prior XRD work has focused mainly on classification (e.g., phase ID, space group). This study introduces PQ-Net, a regression CNN designed to directly quantify key parameters (phase scale factors, lattice parameters, crystallite sizes) from 1D powder diffraction patterns, aiming to enable accurate, real-time analysis for complex multi-phase systems.

Recent CNN applications in materials crystallography include crystal system and space group classification, phase identification, and related tasks trained on synthetic/augmented XRD data. Examples include models by Park et al. for space-group inference and Lee et al. for pattern identification with potential fraction estimation, as well as interpretable CNNs for rapid identification using limited data. Non-neural algorithms (clustering, chemometrics) can perform phase identification but are generally less efficient/accurate than neural networks. Most prior work addresses classification (presence/absence of phases), not quantitative regression of physico-chemical parameters. PQ-Net addresses this gap by directly regressing scale factors, lattice parameters, and crystallite sizes for multi-phase systems and by incorporating uncertainty via deep ensembles.

Architecture: PQ-Net is a 1D regression CNN with three components: (1) Pattern-block to reduce pattern dimensionality and extract local features using stacked 1D convolutions (e.g., initial layers with 128 filters, kernel size 35, stride 1) and max-pooling (stride 2); (2) Phase-block replicated per crystalline phase, consisting of sequences of convolution and pooling layers ending with a flatten layer to extract phase-specific features; (3) Parameter-block comprising fully connected layers for each target parameter (scale factor, crystallite size, lattice parameter a), with dropout (10%) to mitigate overfitting. Parameter-block depth/width are tuned; larger width is used for crystallite size and lattice parameters as they are harder to predict. Training uses MAE loss (preferred over MSE due to robustness to outliers), Adam optimizer with learning rate 0.0005. Target parameters are normalized to a common range after subtracting per-parameter minima. Data generation and training: Training libraries are generated using TOPAS v7 and in-house MATLAB scripts, sampling parameter values from fixed ranges with recorded minima/maxima to allow de-normalization. Libraries include (as appropriate) Poisson noise and second-degree Chebyshev background terms. Validation split is 10%. Hardware: 3XS Data Science Workstation (2x Intel Xeon Silver 4216, 350 GB RAM, 2x Quadro RTX 8000). Single-phase study: Proof-of-concept on simulated noiseless, zero-background Ni fcc (ICSD 64989) patterns. Library size sensitivity indicates performance saturates above ~100k patterns, with marked MAE drop above ~10k; accurate results achievable with libraries as small as ~20k. A phantom XRD-CT dataset (120×120 = 14,400 patterns) is simulated using a realistic scale-factor map derived from prior experimental analysis to evaluate generalization to spatially varying properties. Multi-phase study: Architecture extended to five phases reflecting experimental chemistry: NiO (ICSD 9866), PdO (ICSD 24692), CeO2 (ICSD 72155), ZrO2 (ICSD 66781), and theta-Al2O3. Targets per phase: scale factor, lattice parameters, and crystallite size. Training libraries include Poisson noise and linear backgrounds. A deep ensemble of independently trained PQ-Net models (typically 5–10 models) is used to improve robustness and provide uncertainty (standard deviation across model predictions). A simulated multi-phase XRD-CT dataset is constructed by summing five single-phase datasets (14,400 patterns) and used as test data. Experimental dataset and benchmark: An experimental XRD-CT dataset (151×151 = 22,801 patterns) of a multi-component Ni-Pd/CeO2-ZrO2/Al2O3 catalyst is analyzed with a deep ensemble (10 models, trained on 100k patterns). Conventional Rietveld (TOPAS) serves as a benchmark and approximate ground truth. For Rietveld, preprocessing used a 2nd degree Chebyshev background; refined parameters per phase: scale factor, lattice parameters, and crystallite size. Mean pattern fitting was used to initialize batch fits. Uncertainty and evaluation: Ensemble mean provides predictions; standard deviation across ensemble gives uncertainty maps. Goodness-of-fit assessed via Rwp and comparison to ground-truth (simulations) or Rietveld (experiment).

Single-phase simulations: PQ-Net accurately recovers Ni parameters across the phantom XRD-CT dataset. Errors: scale factor relative error <5%; crystallite size error <1 nm; lattice parameter error <1×10−3 Å; Rwp <5% for the majority of particles. Example fits to average particle patterns yield Rwp of 3.144%, 1.191%, and 1.835%. Multi-phase simulations: Deep ensemble (10 models) trained with 100k patterns accurately reconstructs scale factor maps for NiO, PdO, CeO2, ZrO2, and theta-Al2O3, preserving local features and relative intensities. Crystallite size and lattice parameter maps are also accurately recovered; Rwp <10% across particles. Increasing ensemble size reduces MAE and provides usable uncertainty estimates; practical ensemble sizes are 5–10 due to diminishing returns and training overhead. A lower bound on validation MAE remains due to difficulty predicting parameters when phase scale factors approach zero. Experimental XRD-CT: PQ-Net reproduces Rietveld-derived scale factor maps, including detection of minor PdO (<1 wt.%) concentrated near particle surfaces. It captures known chemical gradients in CeO2-ZrO2: egg-shell distributions with lower lattice parameter and crystallite size at particle shells. Differences between PQ-Net and Rietveld maps: lattice parameter differences typically <2×10−2 Å; crystallite size differences ~1–2 nm. Fits to representative regional mean patterns yield Rwp of 8.353% and 7.749%. Overall, PQ-Net’s ensemble Rwp differs by <2% from Rietveld. Performance: PQ-Net analyzes ~20k XRD patterns in ~10 s, whereas state-of-the-art Rietveld required ~4.4 h for ~9k patterns, demonstrating orders-of-magnitude speedup while maintaining accuracy and providing uncertainty quantification.

PQ-Net addresses the critical bottleneck in modern diffraction experiments by enabling rapid, quantitative analysis of large 1D powder XRD datasets. Across simulated and experimental single- and multi-phase scenarios, it accurately quantifies phase scale factors, lattice parameters, and crystallite sizes, matching Rietveld within small tolerances and providing robust uncertainty estimates through deep ensembles. This capability facilitates real-time or near-real-time decision-making during in situ/operando beamtime, allowing dynamic adjustment of experimental conditions and targeted data collection. While PQ-Net is not intended to replace Rietveld, it serves as a powerful front-end for fast prediction and initialization, after which traditional least-squares refinement can fine-tune parameters. The method’s uncertainty maps and Rwp residuals also help flag evolving or unknown chemistry. Pre-generating diffraction libraries and pretraining models prior to experiments is a practical path to deploy PQ-Net for on-the-fly analysis.

This work introduces PQ-Net, a regression CNN for full-profile quantitative analysis of powder XRD patterns in multi-phase systems. Validated on progressively complex datasets—including an experimental five-phase XRD-CT dataset with ~22.8k patterns—PQ-Net delivers accurate maps of scale factors, lattice parameters, and crystallite sizes with uncertainty estimates, and achieves orders-of-magnitude speedups relative to Rietveld while maintaining comparable Rwp. Deep ensembles enhance robustness and provide actionable uncertainty for real-time use. Future directions include integrating PQ-Net into beamline workflows for live feedback, expanding to additional parameters (e.g., microstrain, occupancies), improving handling of unknown/evolving phases (e.g., hybrid classification-regression pipelines), and extending to other diffraction/spectroscopy modalities.

- Requires a priori knowledge of phases and approximate chemistry to construct training libraries; not designed to discover unknown phases. Unknown or evolving phases must be detected via elevated Rwp, residuals, or increased ensemble uncertainty. - Lower bound on achievable MAE for parameters (especially crystallite size and lattice parameters) when phase scale factors approach zero. - Training deep ensembles improves accuracy and uncertainty estimates but increases computational cost; practical ensemble sizes are limited (≈5–10). - PQ-Net does not replace detailed least-squares refinement; fine parameter tuning may still require Rietveld starting from PQ-Net predictions.