The anomalous Hall effect (AHE) and the anomalous Nernst effect (ANE), transverse transport phenomena, are typically associated with ferromagnetism. However, recent research has revealed their presence in non-magnetic and antiferromagnetic topological materials, highlighting a range of underlying mechanisms requiring further investigation. Conventional AHE and ANE in ferromagnets are directly proportional to magnetization (M), arising from both intrinsic Berry curvature and extrinsic scattering processes. In collinear antiferromagnets, despite broken time-reversal symmetry (TRS), the intrinsic contribution is usually zero due to combined symmetry operations. Nevertheless, large anomalous transport coefficients have been observed in non-collinear or non-coplanar antiferromagnets with non-trivial band topology, such as Mn₃Sn, Mn₃Ge, and YbMnBi₂. The absence of net magnetization in antiferromagnets at zero field offers advantages for device applications by minimizing stray fields. Furthermore, some antiferromagnets exhibit significant AHE and ANE that are not proportional to M, a phenomenon termed 'unconventional' anomalous transport. Possible explanations include topological responses from chiral spin textures, field-tuning of topological band crossings, interplay between spin-orbit coupling (SOC) and crystal symmetry, and domain wall effects. The relationship between material symmetry and electronic band topology is crucial for understanding these effects. This study focuses on SmMnBi₂, a member of the RMnPn₂ family, to investigate the interplay between magnetism and topology in relation to anomalous transport. Unlike other RMnPn₂ materials, SmMnBi₂ possesses a unique tetragonal structure with a more complex band structure, making it a prime candidate for studying unconventional anomalous transport.

The literature extensively documents the anomalous Hall effect (AHE) and anomalous Nernst effect (ANE) in ferromagnetic materials, where these effects are directly proportional to magnetization. However, recent studies have challenged this conventional understanding by demonstrating the existence of significant AHE and ANE in antiferromagnetic materials. The work by Nakatsuji et al. (Nature 2015) highlighted the large anomalous Hall effect in Mn₃Sn, a non-collinear antiferromagnet. Similarly, Nayak et al. (Sci. Adv. 2016) reported a large anomalous Hall effect in Mn₃Ge, also attributed to its non-collinear magnetic structure. The discovery of sizable AHE and ANE in antiferromagnets has spurred further research into the underlying mechanisms, particularly the role of Berry curvature and other topological properties. This body of work emphasizes the potential for antiferromagnetic materials in spintronics applications due to their lack of stray fields. The research on YbMnBi₂ (Pan et al., Nat. Mater. 2022) showcased a giant anomalous Nernst signal, furthering the interest in understanding these effects in this class of materials.

High-quality SmMnBi₂ single crystals were grown using a bismuth self-flux method. The crystal structure was confirmed using single-crystal X-ray diffraction (XRD), revealing a tetragonal structure (space group I4/mmm). Magnetic properties were characterized using a Quantum Design MPMS3 system, measuring temperature-dependent magnetic susceptibility and isothermal magnetization curves. These measurements revealed an antiferromagnetic transition at TN = 235 K, with a small canting of the Mn spins. Electrical transport measurements (longitudinal resistivity ρxx and Hall resistivity ρxy) were performed in a Quantum Design PPMS-9T system using a standard Hall bar configuration. Thermoelectric measurements (Seebeck coefficient Sxx and Nernst coefficient Sxy) were conducted using a home-built high-resolution thermoelectric puck within the PPMS. Angle-resolved photoemission spectroscopy (ARPES) measurements were performed at BL03U of Shanghai Synchrotron Radiation Facility and SSRL to investigate the electronic band structure. First-principles density functional theory (DFT) calculations were used to model the electronic structure and Berry curvature, considering different spin configurations (C-type and G-type antiferromagnetic structures with canting).

The study revealed an unconventional anomalous Hall effect (UAHE) in SmMnBi₂, characterized by a non-monotonic field dependence of the Hall resistivity (ρxy) that does not scale with magnetization. The Hall conductivity (σxy) showed a resonance-like peak at low fields. A scaling relation (ρxy,max ∝ ρxx²) was observed below TN, consistent with an intrinsic AHE origin. A significant anomalous Nernst effect (ANE) was also observed, reaching a maximum of 1.8 µV K⁻¹. ARPES measurements and DFT calculations corroborated the experimental findings. The DFT calculations, considering a canted G-type antiferromagnetic order, successfully reproduced the key features of the ARPES data and showed significant Berry curvature contribution to the AHE. The analysis indicated that spin canting plays a vital role in generating the observed AHE and ANE. The UAHE persists above TN, possibly due to short-range canted AFM order or extrinsic skew scattering. The large ANE in SmMnBi₂, exceeding the empirical k_B/e threshold, suggests a significant contribution from extrinsic mechanisms, despite the intrinsic AHE. The comparison of ANE in SmMnBi₂ with other ferromagnetic and antiferromagnetic materials highlights the significant magnitude of the effect in SmMnBi₂. The ANE is notably larger than that observed in other antiferromagnets except YbMnBi₂, where the measurement configuration differs.

The findings demonstrate the existence of both unconventional AHE and a large ANE in SmMnBi₂, challenging the traditional association of these effects with ferromagnetism. The intrinsic origin of the UAHE is confirmed by the scaling relation and DFT calculations, which emphasize the role of Berry curvature arising from the canted antiferromagnetic spin structure. The significantly larger-than-expected ANE suggests the presence of strong extrinsic contributions alongside the intrinsic component. The coexistence of intrinsic and extrinsic mechanisms for both effects is notable. The field-dependent modulation of the AHE and ANE can be attributed to the field-induced variation in spin canting angle. The persistence of the UAHE above TN points towards the potential role of short-range order or extrinsic mechanisms. The results highlight the complex interplay between spin canting, Berry curvature, and extrinsic scattering in determining the anomalous transport properties of SmMnBi₂.

This study provides compelling evidence of unconventional AHE and large ANE in the antiferromagnetic material SmMnBi₂. The AHE is predominantly intrinsic, originating from Berry curvature due to canted antiferromagnetism. The ANE, however, is significantly enhanced by extrinsic mechanisms. SmMnBi₂ presents an intriguing platform for further investigations into the multifaceted mechanisms governing anomalous transport in antiferromagnetic topological materials, potentially leading to advancements in spintronic and thermoelectric devices.

The study primarily focuses on the transport properties of SmMnBi₂ single crystals. Further investigations are needed to explore the effects of defects, grain boundaries, and sample preparation methods on the anomalous transport properties. While the DFT calculations provide a good qualitative understanding of the electronic structure and Berry curvature, more sophisticated calculations could offer a more detailed understanding of the complex interplay between spin canting, Berry curvature, and extrinsic effects.