Intervention studies typically utilize randomization to estimate average treatment effects (ATEs), comparing the mean outcomes of different interventions. However, this approach overlooks individual variation in treatment response, known as heterogeneous treatment effects (HTE). HTE represents systematic variability in the direction and magnitude of individual treatment effects (ITEs). While RCTs provide an unbiased ATE, they do not identify which intervention is optimal for a given individual. This study argues that the focus should shift from ATE estimation to optimal treatment assignment, especially in personalized settings where knowing which treatment works best for whom is critical. The authors introduce the Recurrent Individual Treatment Assignment (RITA) algorithm as a solution to improve overall intervention response by addressing the issue of unobserved HTE. Existing methods, including frequentist approaches (incorporating moderator effects) and bandit/Bayesian approaches (updating assignment probabilities based on observed characteristics), suffer from limitations such as bias due to unobserved characteristics and the reference class problem. RITA bypasses these issues by focusing directly on learning optimal individual treatment assignments over time, leveraging sequential RCTs or A/B tests.

The authors review existing literature highlighting the limitations of using ATEs to guide individual treatment decisions when HTE exists. They discuss the challenges posed by HTE, including the reference class problem (generalization concerns due to unobserved characteristics), statistical overfitting, false discovery rates, and biased predictions for new populations. They also address the limitations of observational studies due to potential internal validity issues. The authors note the potential of longitudinal RCTs to learn over time through exploration (random assignment to infer optimal interventions) and exploitation (using previous observations to guide assignments). Existing algorithmic approaches, such as frequentist and stochastic methods, are discussed, noting their reliance on assumptions like strong ignorability or no unmeasured confounding, which often are not met in real-world applications. The limitations of these approaches in fully addressing unobserved HTE are emphasized, paving the way for the introduction of the RITA algorithm.

This study develops and evaluates the RITA algorithm, a longitudinal individual assignment algorithm based on sequential RCTs. RITA updates assignment decisions based on observed variation in intervention response, incorporating both random variation within individuals and variation across interventions. The algorithm operates in a multi-period setting, sequentially assigning individuals to interventions A and B. The algorithm uses four different simulated "worlds" to evaluate its performance under varying conditions of HTE, including scenarios with and without observed and unobserved heterogeneity and different ATEs for interventions A and B. The RITA algorithm is compared to a baseline model where individuals are assigned to treatments based on an RCT-derived ATE. The simulation parameters are established as follows: 1000 individuals, 60 periods, two alternative interventions, initial treatment assignment probability P(A) = P(B) = 0.5, initial outcome y₀ ~ N(10,1), and treatment response variation including an error term ε ~ N(0, 0.1). The structure of heterogeneity is modeled as h(LX) = u(L) + o(X) representing the impact of unobserved (L) and observed (X) factors. Four different treatment effect settings are simulated based on this structure: World 1 (no heterogeneity), World 2 (unobserved heterogeneity), World 3 (observed and unobserved heterogeneity), and World 4 (same ATE, but unobserved heterogeneity leading to different optimal interventions for subgroups). The performance of both algorithms is evaluated using treatment assignment, cumulative treatment response, and individual treatment response. Finally, A technical description of the baseline model and the RITA algorithm is provided including the detailed steps involved in the RITA algorithm such as Random Assignment, determining treatment response, determining the rank, determining the individual mean rank (IMR), updating the treatment assignment probability, and finally ensuring the desired level of exploration.

The simulation results demonstrate that RITA outperforms the baseline model in all simulated worlds except World 1 (no heterogeneity). In the presence of heterogeneity (Worlds 2, 3, and 4), RITA rapidly learns the optimal assignment, exhibiting a clear advantage over the baseline model. Figure 3 shows that RITA exhibits differing assignment patterns compared to the baseline model which is increasing in the presence of HTE and absence of ATE difference. Figure 4 displays average cumulative outcome gains, revealing RITA's superior performance across all worlds except World 1, where the baseline's deterministic assignment was optimal. In Worlds 2 and 3, RITA converges to similar performance as the baseline after approximately 7-8 periods, exceeding it thereafter. World 4, with equivalent ATEs but unobserved heterogeneity, reveals a significant advantage for RITA, with an average increase in cumulative gains of 11.45 (SD = 21.08). Figure 5 illustrates individual cumulative outcome gains, showing that while RITA can harm individuals with unfavorable random assignments, it offers substantial benefits to those for whom the exploration proves beneficial. This asymmetry in benefits and losses contributes to the overall superior performance of RITA. The paper explores the potential performance of RITA against more advanced models that address observed heterogeneity—a baseline model with an interaction effect and a Bayesian/bandit model—reasoning that RITA’s effectiveness stems from its ability to handle unobserved heterogeneity, a feature absent in the other models. The study emphasizes that the relative performance of these models depends on the complexity of observed heterogeneity and the prevalence of unobserved heterogeneity.

The study's findings challenge the dominance of RCTs and ATEs in guiding individual treatment decisions, especially in the presence of substantial HTE. RITA offers a novel approach that prioritizes learning optimal individual treatment assignments over explicit ITE estimation. By focusing on treatment response variation, it effectively addresses both observed and unobserved HTE. The asymmetry between potential small losses for some individuals and large gains for others is a key driver of RITA’s overall superior performance. While the specific RITA algorithm presented here is one instance, future research is needed to explore optimal algorithm designs, investigate asymptotic properties and convergence, and apply the approach to real-world data. The potential for undesirable oscillating behavior in treatment assignments is acknowledged and suggestions for mitigating this are proposed.

This study demonstrates the effectiveness of RITA, a novel algorithm that improves overall treatment response by learning optimal individual treatment assignments. Its ability to accommodate unobserved HTE surpasses existing methods. Future research should focus on optimization of the algorithm, extending it to dynamic treatment regimes, and applying it to real-world datasets. RITA's potential for revealing underlying treatment mechanisms offers new avenues for diagnostic insights.

The study relies on simulated data, which might not fully capture the complexities of real-world scenarios. The specific RITA algorithm used is just one example, and its optimal design may vary depending on specific contexts. The potential for undesirable oscillating behavior in treatment assignments exists, although mitigation strategies are suggested. Further investigation into the algorithm's asymptotic properties and convergence behavior is warranted.