A scalable synergy-first backbone decomposition of higher-order structures in complex systems

The paper addresses a central question in complexity science: to what degree can a whole system be greater than the sum of its parts, and how can higher-order, emergent interactions be quantified? In information-theoretic terms, this higher-order dependence is termed synergy—information present in the joint state of multiple variables but absent from any subset. While synergy has been observed across neuroscience, climate, social systems, and physiology, existing tools for analyzing it are limited. PID-based approaches provide a complete map of multivariate information but scale super-exponentially and are redundancy-first, whereas O-information provides scalable but coarse summaries of redundancy versus synergy dominance with a conservative definition of synergy. The purpose of this study is to introduce a synergy-first, scalable decomposition that captures how synergies distribute across scales, providing interpretable and practical tools to study part–whole relationships in complex systems.

Two major methodological families dominate synergy analysis: (1) PID and its derivatives (partial entropy decomposition, integrated information decomposition, generalized information decomposition), which deliver a complete lattice of information atoms but scale with Dedekind numbers, becoming infeasible beyond small systems and relying on redundancy-first definitions; and (2) O-information and multivariate mutual information extensions, which scale well but only indicate whether redundancy or synergy dominates, using a conservative synergy definition that often classifies sub-global joint information as redundancy. Related constructs include TSE complexity (partition-based, total-correlation-based multiscale analysis with a built-in null), connected information (requires maximum-entropy distributions under constraints, often intractable), entropy/complexity profiles, and marginal utility of information. Recent synergy-first approaches (synergistic disclosure) compress lattice information into a backbone but remain tied to lattice growth and are specific to expected mutual information. Gradients of O-information provide low-order descriptors of high-order dependencies, focusing on element-specific contributions rather than a global backbone. This work positions α-synergy as a synergy-first, lattice-free backbone spanning scales, applicable beyond information theory.

The paper defines a synergy-first backbone decomposition over scales using channel-failure logic. For a discrete multivariate source X = {X1,…,Xk} with realization x and local entropy h(x) = −log P(x), synergy is the information destroyed when access to any specified number of elements fails. For single failures, the 1-synergy is the minimum information lost across all single-element failures: h^1(x) = min_i [h(x) − h(x^{-i})] = min_i h(x^{i}|x^{-i}). This generalizes to the α-synergy for any α in {1,…,k} by minimizing over all α-subsets: h^α(x) = min_{S:|S|=α} [h(x) − h(x^{-S})] = min_{S} h(x^{S}|x^{-S}). The expected α-synergy is H^α(X) = E_x[h^α(x). The α-synergy function is non-negative and monotonically increasing with α, enabling a decomposition of the local entropy into non-negative partial atoms via bootstrapping: ∂h_α(x) = h(x^α) − h(x^α|x^{-α}), with h(x) = Σ_{α=1}^k ∂h_α(x). This yields a 1D backbone of k values ordered from most fragile (α=1) to most robust (α=k) synergistic entropy. Extensions: The same logic applies to other measures built from entropies. For Kullback-Leibler divergence D^P_Q(X) = E_P[h^Q(x) − h^P(x)], α-partial atoms are d^α_{Q,P}(x) = ∂h^Q_α(x) − ∂h^P_α(x), aggregated as ΔD^α_{Q,P}(X) = E_P[d^α_{Q,P}(x)], noting atoms can be negative. This induces α-synergy decompositions for negentropy N(X) (divergence from uniform), total correlation TC(X) (divergence from product of marginals), and single-target mutual information I(X;Y), the latter via KLD between P(X|y) and P(X) averaged over y. Alternative formulations: Replace min over α-subsets with average (expected loss) or max (worst-case loss), preserving non-negativity and monotonicity, with different interpretations and practicalities for large systems. Computation and scaling: The α-synergy decomposition requires evaluating conditional entropies over all α-subsets and complements—scaling with Bell numbers. For moderate k, exact search is feasible; for larger systems, heuristics include random sampling, simulated annealing, or Queyranne’s algorithm. Care must be taken to preserve monotonicity to avoid negative partial atoms from suboptimal minima. Beyond information theory: A generalized α-synergy can be defined for any set function f satisfying localizability, symmetry, non-negativity, and monotonicity: f^α(x) = min_{S:|S|=α} [f(x) − f(x^{-S})]. As a demonstration, the paper applies this to structural synergy in networks using communicability C(M) = E[e^M] as f, localizing to node pairs and assessing how edge failures reduce integration across scales.

- The α-synergy framework provides a synergy-first, scale-wise backbone that is non-negative and monotonic, decomposing entropy into k partial synergy atoms from fragile to robust. - Entropy decompositions for example systems: • XOR (Table 2): ∂H1=0 bit, ∂H2=1 bit, ∂H3=1 bit. Interpretation: XOR’s entropy is robust to single failures; synergy appears at higher α. • Giant bit (Table 4): ∂H1=0 bit, ∂H2=0 bit, ∂H3=1 bit. Interpretation: maximally robust, global configuration recoverable from any single node. • W (Table 6): ∂H1=0 bit, ∂H2=0 bit, ∂H3=log2(3) bit. Interpretation: configurations reconstructable unless all but one channel fail; highest-order synergy concentrates when elements are 0. • MaxEnt (Table 8): ∂H1=1 bit, ∂H2=1 bit, ∂H3=1 bit. Interpretation: despite no dependencies, entropy distributes uniformly across scales. - Total correlation decompositions (Table 9): • XOR: αTC1=1 bit, αTC2=0, αTC3=0 (all structure at fragile scale, matching synergy intuition under TC). • Giant bit: αTC1=1 bit, αTC2=1 bit, αTC3=0 (shift towards synergy under TC due to resolving high-order uncertainty from MaxEnt prior to redundant posterior). • W: αTC1=log2(3)−1 bit, αTC2=log2(3)−1 bit, αTC3=0. • MaxEnt: all 0 (no structure relative to independent prior). - Single-target mutual information decompositions I(X1,X2;Y) (Table 10): • XOR: α=1 → 0 bit; α=2 → 1 bit (pure synergy at the whole). • Giant bit: α=1 → 1 bit; α=2 → log2(3)−1 bit (pure robustness/redundancy-like profile). • W: α=1 → 1/3 bit; α=2 → 0 bit (mixed synergy/robustness). • MaxEnt: α=1 → 0; α=2 → 0 (no information). - Structural synergy example (communicability): In a 10-node, 19-edge Erdős–Rényi graph, α-synergistic communicability shows minimal fragile synergy; the gap between 1-synergy and total communicability spans ~4 orders of magnitude, reflecting high redundancy of diffusive paths. - Practicality: Compared to PID (Dedekind growth), the backbone scales more gracefully (Bell growth) and yields interpretable, per-scale summary statistics while remaining localizable and extendable to multiple information-theoretic measures and beyond.

The study reframes the measurement of higher-order structure via a synergy-first, failure-based definition, directly aligning with the intuitive notion of information present only in the whole. By constructing a 1D backbone of α-partial synergies, it addresses PID’s scalability limits and O-information’s coarse summaries. Results across entropy, total correlation, and mutual information highlight that synergy is function-dependent: the same distribution can appear robust or synergistic depending on whether one analyzes entropy (what is present) or information gain relative to a prior (what is learned), and whether analysis is directed or undirected. The XOR, giant bit, W, and MaxEnt examples illustrate these contrasts, providing guidance on selecting appropriate measures for specific questions. The approach also extends beyond information theory to structural properties in networks, enabling assessment of how higher-order combinations of edges contribute to integration. This contributes a tractable, interpretable perspective on part–whole relationships in complex systems while maintaining flexibility to different definitions (min/avg/max) and measures (entropy, KL, TC, MI).

The paper introduces the α-synergy decomposition, a synergy-first, scalable backbone that decomposes entropy and derived measures (KL divergence, negentropy, total correlation, single-target mutual information) into non-negative, monotonic partial synergy atoms across scales. It balances trade-offs between PID (complete but intractable and redundancy-first) and O-information (scalable but coarse), providing per-scale insight without element-specific lattices. Demonstrations on canonical distributions and a network communicability example showcase interpretability and breadth. Future directions include: extending to differential entropy and continuous variables; robust optimization heuristics preserving monotonicity at scale; developing an analogue to the integrated information decomposition; combining with element-specific gradients (e.g., gradients of α-synergy); and applying to dynamic processes and real systems (e.g., supply chains) to study robustness–efficiency trade-offs and cascading failures.

- Loss of element-specific detail: the backbone homogenizes contributions across elements and subsets, obscuring which variables or sets carry synergy. - Computational burden: although improved over PID, exhaustive backbone computation scales with Bell numbers and becomes intractable for large k; heuristic optimization or sampling may violate monotonicity and yield negative partial atoms. - Measure dependence: interpretations of synergy differ across entropy, total correlation, and mutual information; careful selection and framing are required. - Differential entropy issues: for continuous variables, local differential entropy can be negative, complicating assumptions (non-negativity/monotonicity) underlying the decomposition. - Framework gaps: no current analogue for integrated information decomposition (IID) within the α-synergy framework. - Choice of min/avg/max introduces methodological flexibility that can affect results and interpretability.