Instruments measuring motivation to learn Arabic as a second language: evidence of validity and reliability

Research on second language (L2) motivation predominantly focuses on English as a foreign language (EFL), neglecting the specific factors influencing Arabic L2 learning. This study addresses this gap by evaluating the validity and reliability of instruments designed to measure L2 Arabic motivation. Motivation, encompassing internal and external factors driving goal attainment (Dörnyei, 2001; Harmer, 2015), is a critical element in L2 acquisition. Existing L2 motivation theories, such as Gardner's (2000) instrumental and integrative motivation and Dörnyei's (2009) L2 Motivational Self System (L2MSS), primarily apply to EFL contexts. Learning Arabic presents a unique dimension, incorporating religious and cultural integration motives. The scarcity of research on L2 Arabic motivation hinders educators' ability to effectively identify factors that enhance or diminish motivation in Arabic learning settings, particularly at the post-secondary level. This study aims to assess the validity of two quantitative instruments designed to measure L2 motivation in Arabic learners at Arabic learning institutes (ALIs) in Saudi Arabia (KSA), bridging this critical research gap. The study focuses on three primary sources of L2 motivation: identity-related motivation (L2MSS), L2LE-influenced motivation, and integrative motivation (Gardner, 2000), specifically the desire for integration with the local Saudi population. Draft quantitative instruments were developed and piloted, leading to refined instruments focusing on identity motivation and the Arabic L2LE.

The study extensively reviews existing literature on L2 motivation, highlighting the dominance of EFL research and the limited focus on Arabic L2 motivation. It distinguishes between Classical Arabic (CA), used in religious contexts, and Modern Standard Arabic (MSA) used in formal settings, alongside colloquial dialects. The review examines previous research on L2 Arabic motivation, noting varying factors across diverse contexts (e.g., religious motivation among Muslims learning Arabic in non-Arabic speaking countries, functional motivation among expatriates in non-Arabic-speaking countries). It then introduces Saudi Arabia's ALIs, their standardized curriculum (emphasizing CA), and their role in supporting non-Saudi learners in higher education. The review incorporates findings from previous studies conducted at KSA's ALIs, highlighting learner satisfaction with religious aspects of Arabic learning but also challenges in social communication and speaking anxiety. The review also covers the process of developing the instruments used in the current study, including a pilot study at three ALIs, which identified four subscales for identity motivation (cultural exposure, Islamic identity, Middle East politics/problems, and instrumental needs) and three for L2LE (classroom environment, teacher/curriculum, and personal anxiety about using the L2).

This mixed-methods study employed a three-cohort design to assess the reliability and validity of the developed instruments. Cohort 1 involved a reanalysis of pilot study data from one ALI (a public women's university) focusing on the identity motivation and L2LE instruments. Cohorts 2 and 3 involved administering the refined instruments to Diploma Program participants at the same ALI using online surveys distributed via WhatsApp. The surveys were anonymous, except for Cohort 2, where participants who expressed interest were invited to participate in semi-structured interviews to assess the validity of the instruments. Six participants from Cohort 2 participated in interviews conducted in Arabic and translated into English. Quantitative data analysis involved calculating subscale scores for each cohort and comparing means across cohorts to assess reliability. The interviews were analyzed using thematic analysis, identifying themes related to the instrument subscales. This approach aimed to understand the apparent contradictions identified in the pilot study, particularly the unexpectedly low instrumental motivation scores and the high personal anxiety scores despite high satisfaction with the classroom environment and instruction. The quantitative data were standardized by dividing each mean score by its maximum potential score, allowing for comparisons of standardized mean scores across the three cohorts. Interview data were analyzed separately for long-term and short-term residents of KSA, to explore if residency length influenced thematic patterns.

The quantitative analysis revealed consistent mean subscale scores across the three cohorts, supporting the reliability of the instruments. High scores were consistently observed for cultural exposure (CE) and Islamic identity subscales of the identity motivation instrument, and for classroom environment (CEnv) and teacher/curriculum (TC) subscales of the L2LE instrument, mirroring the pilot study. Lower-than-expected scores were consistently observed for the instrumental needs and Middle East politics/problems subscales, along with consistently high scores for the personal anxiety (PersAnx) subscale of the L2LE instrument. Qualitative data provided insights into these patterns. Interviewees revealed that their strong motivation stemmed from a desire to integrate into Saudi society and deepen their understanding of Islam. Cultural exposure was linked to adopting Saudi norms and behaviors, and interest in Middle East politics reflected a desire to understand Saudi perspectives as a representation of the ideal Islamic society. The unexpectedly low instrumental scores reflected the fact that the instrument items didn't clearly differentiate between CA and conversational Arabic, leading to interpretations focused on learning the local dialect rather than CA. The high personal anxiety scores, despite high satisfaction with the learning environment, were linked to the lack of conversational Arabic instruction in the curriculum. Learners faced difficulties communicating with classmates and others in KSA, hindering their progress and causing anxiety. The long-term residents described overcoming these communication challenges through interactions with Saudi community members, indicating a need for the integration of conversational Arabic in the curriculum. The qualitative analysis found that learners conflated Saudi culture and society with ideal Islamic practices.

The findings demonstrate the reliability of the developed instruments and provide evidence for the validity of the identity motivation subscales and the personal anxiety subscale. The study resolved the apparent contradictions from the pilot study by revealing the conflation of Saudi culture and ideal Islamic society in learners' motivations. The low instrumental scores are attributed to unclear item wording not adequately differentiating between CA and conversational Arabic. The high personal anxiety scores, despite satisfaction with the classroom, arise from the lack of conversational Arabic instruction. The results highlight a significant gap in the curriculum, as learners need conversational skills to integrate with Saudi society and enhance their learning of CA. This underscores the importance of incorporating conversational Arabic instruction, aligning with previous recommendations from similar studies.

This study confirms the reliability and validity of the developed instruments in measuring L2 Arabic motivation, particularly identity-related aspects and speaking anxiety. The findings emphasize the importance of including conversational Arabic training in the ALI curriculum to improve learner integration, reduce anxiety, and enhance the learning of Classical Arabic. Future research should examine the generalizability of these findings to different learner populations (e.g., men, learners in non-Arabic-speaking countries). Further investigation is needed to validate the classroom environment, teacher, and curriculum subscales. The study's data are publicly available to facilitate further research and replication.

The study's sample was limited to female learners at a single ALI, potentially limiting the generalizability of findings to male learners or other ALIs. The interview sample size was relatively small, possibly affecting the richness of qualitative insights. The reliance on self-report measures might introduce biases in participants’ responses. The interview method, conducted by the Dean of the ALI, might have influenced participant responses, limiting candid feedback on specific aspects of the L2LE. Finally, the study did not directly measure the effectiveness of different teaching methodologies on the measured variables.