Timbral effects on consonance disentangle psychoacoustic mechanisms and suggest perceptual origins for musical scales

The perception of consonance and dissonance is a fundamental aspect of music across diverse cultures. Western music theory traditionally attributes consonance to simple frequency ratios between tones, assuming timbre plays a negligible role. However, existing psychoacoustic theories, such as Helmholtz's interference theory, suggest that timbre, specifically the harmonic structure of tones, significantly influences consonance judgments. This paper addresses the unresolved question of timbre's impact on consonance perception. The research seeks to determine if consonance is solely dependent on harmonic frequency ratios or if timbral characteristics significantly alter consonance preferences. The importance of this research lies in its potential to shed light on the psychological mechanisms underlying consonance perception and its role in the cultural evolution of musical scales and styles worldwide. Centuries of research, from Pythagoras to modern psychologists and ethnomusicologists, have grappled with the nature of consonance. While psychoacoustic mechanisms like fusion and combination tones have been proposed, the dominant theories focus on either interference between partials or harmonicity detection. Both predict that harmonic intervals should sound most pleasant, aligning with the observation that many, but not all, global scale systems favor these intervals. However, cultural familiarity with specific musical styles also plays a crucial role, leading to biases in consonance perception. The present study aims to resolve the conflicting perspectives on the role of timbre in consonance perception by conducting a series of large-scale experiments that systematically manipulate timbral features while measuring consonance judgments. This approach allows for a comprehensive investigation into the interplay between psychoacoustic mechanisms and cultural influences on consonance.

Previous research on consonance has yielded contradictory findings regarding the influence of timbre. While traditional Western music theory suggests timbre independence, psychoacoustic theories like Helmholtz's interference theory predict that consonance judgments should vary significantly based on the harmonic structure of the tones involved. Empirical studies have produced mixed results, with some claiming to find timbral effects while others report no significant influence. This inconsistency necessitates a more rigorous and comprehensive investigation using large-scale behavioral experiments. Existing computational models of consonance, such as those based on interference between partials or harmonicity detection, offer frameworks for interpreting the potential effects of timbral manipulations. These models, however, have not been extensively tested with comprehensive timbral manipulations in large-scale studies. The current research aims to address these gaps by using advanced experimental techniques and computational modeling to provide a more nuanced understanding of the relationship between timbre and consonance.

The study employs a series of 23 large-scale behavioral experiments, involving 4272 online participants and 235,440 judgments. The experiments utilize two primary paradigms: dense rating and Gibbs Sampling with People (GSP). The dense rating paradigm involves presenting participants with continuous ranges of pitch intervals and evaluating their pleasantness ratings. This method allows for a detailed mapping of consonance across a continuous intervallic space, avoiding the limitations of relying solely on discrete scales commonly used in previous research. This approach allows for a more nuanced exploration of the continuous relationships in consonance across various intervals rather than focusing on traditional, culturally-influenced, discrete sets. GSP, on the other hand, is particularly useful for exploring higher-dimensional stimulus spaces, such as triads. It involves collaborative exploration of the stimulus space where participants guide the process towards regions of higher consonance. The study systematically manipulates three primary aspects of timbral features: (1) changing the frequencies of harmonics (spectral stretching/compression), (2) changing the amplitudes of harmonics (spectral roll-off), and (3) deleting individual harmonics. These manipulations allow for a focused examination of the differential effects of timbre on consonance perception. The use of both paradigms allows for mapping of consonance across both 2-tone chords (dyads) and 3-tone chords (triads). The synthetic tones used in the experiments are created with fine-grained control, using additive synthesis, allowing manipulation of specific aspects of the harmonic spectrum. This precise control is vital for testing specific predictions of the competing consonance models. In addition to experiments with synthetic tones, experiments also include naturalistic instrument tones to test the applicability of findings to sounds from real instruments. To interpret the behavioral data, two main psychoacoustic models are employed: the Hutchinson-Knopoff interference model and the Harrison-Pearce harmonicity model. These models represent the leading theoretical accounts of consonance perception. The models' predictions for different spectral manipulations are compared to the experimental results to elucidate which mechanisms better explain the observed consonance preferences. The study also incorporates a composite model, combining elements of both interference and harmonicity models, to assess if a unified explanation can provide a better account of the data. Importantly, the experimental design includes a control group of Korean participants to test the generalizability of the results across cultures.

Study 1 establishes a baseline consonance profile for harmonic dyads, showing peaks aligning with Western scale systems. Studies 2A and 2B demonstrate that stretching or compressing the harmonic series leads to correspondingly stretched or compressed consonance profiles. This finding supports interference models over harmonicity models, indicating that the interaction of partials is an important determinant of consonance perception. The findings also reveal a cross-cultural replication with Korean participants. Study 2C uses a synthetic bonang tone (Javanese gamelan instrument) to create an inharmonic consonance profile aligned with the slendro scale, suggesting a link between instrument timbre and scale development in non-Western music. Study 3 investigates the effect of spectral roll-off (amplitude decay of harmonics). Results show a main effect of roll-off (higher roll-off is more pleasant), but the shape of the consonance profile remains largely unaffected, suggesting harmonicity as the dominant factor in this case and pointing to necessary revisions to the interference model. Studies 4A and 4B examine harmonic deletion. Study 4A reveals that deleting harmonics reduces the number of peaks in the consonance profiles. This result highlights the importance of both interference and harmonicity, as different models predict specific peak eliminations. Study 4B investigates tuning preferences and reveals that for tones with strong upper harmonics, listeners prefer slight deviations from just intonation. This phenomenon, possibly due to preferred slow beat frequencies, is successfully incorporated into the interference model. Study 5 extends the findings to triads using GSP, replicating the stretching/compressing effect observed in dyads. The overall findings show that timbral manipulations can alter consonance judgments, challenging the traditional view of timbre independence and revealing the complexity of underlying psychoacoustic mechanisms. The results are most effectively explained by a composite model integrating aspects of both interference and harmonicity, along with a preference for slow beats, highlighting the need for more comprehensive models of consonance perception.

The findings challenge the traditional view of consonance as solely determined by harmonic frequency ratios, independent of timbre. The results strongly suggest a significant role for timbre in shaping consonance preferences. The study’s large-scale design and systematic timbral manipulations provide a robust basis for these conclusions. The dissociation of interference and harmonicity effects through spectral manipulations has significant implications for our understanding of the underlying mechanisms of consonance perception. The results highlight the limitations of solely relying on either interference or harmonicity models, emphasizing the necessity of integrated models that consider both mechanisms. Furthermore, the findings provide insights into the cultural evolution of musical scales and the influence of instrument timbre on scale development, offering explanations for the emergence of inharmonic scales in non-Western music traditions. The preference for slight deviations from just intonation points to an additional factor beyond the established models: the appreciation of slow beats, offering an additional dimension to our understanding of consonance.

This research demonstrates that timbre significantly influences consonance perception, challenging the traditional view of its independence. The findings reveal the complex interplay of interference, harmonicity, and a preference for slow beats in shaping consonance judgments. The study provides a comprehensive model integrating these factors and offers valuable insights into the cultural evolution of musical scales. Future research could explore the generalizability of these findings across a wider range of musical cultures and investigate other timbral aspects beyond those addressed in this study. Exploring the neural correlates of these timbral effects would further enhance our understanding of the underlying mechanisms.

The study primarily relies on online data collection, potentially introducing some limitations on the control of listening environments. While a headphone screening test was implemented to mitigate this, individual variations in listening conditions remain a possibility. The focus on Western and Korean participants limits the generalizability of the findings to other musical cultures. Although the study included a cross-cultural comparison with Korean participants, further research across diverse cultures is needed to fully explore the universality of these effects. The limited number of trials per participant in some analyses affected participant-level reliability, though group-level reliability remained high.