Instruments measuring motivation to learn Arabic as a second language: evidence of validity and reliability

The study addresses the limited research on motivation to learn Arabic as a second language compared with extensive work on English L2 motivation. Grounded in Gardner’s integrative/instrumental framework and Dörnyei’s L2 Motivational Self System, the authors target two key motivational sources among learners in Saudi Arabia’s Arabic Language Institutes (ALIs): identity-related motivation and the Arabic second-language learning environment (L2LE). Prior pilot work showed strong motivation tied to Islamic identity and cultural exposure, but unexpectedly revealed notable speaking anxiety despite positive classroom and teacher/curriculum experiences. The study aims to evaluate validity evidence for subscale measurements from two instruments (identity motivation; L2LE) and to use this evidence to resolve apparent contradictions in earlier findings. It is important because Arabic presents unique religious and cultural dimensions (e.g., CA vs MSA/ammiyah) that shape learner motivation and classroom dynamics in ways not captured by typical EFL-focused research.

- Motivation and L2 motivation: Gardner distinguishes instrumental and integrative motives, with integrative often more predictive of achievement. Dörnyei’s L2 Motivational Self System emphasizes ideal/ought-to L2 selves and language learning experience. Both frameworks have empirical support. - L2 Arabic motivation: Far fewer studies exist than for EFL. Arabic functions as both a heritage and liturgical language, with CA central to religious practice and MSA/dialects to everyday communication. Studies outside Arab countries show religious motives are prominent among Muslim learners, while some expatriate learners emphasize functional/instrumental goals. Motivation and persistence can be context-dependent. - Saudi ALIs context: KSA’s ALIs serve international learners, emphasizing CA and literacy, with limited conversational practice. Prior research in KSA ALIs reported high religiously driven motivation but challenges in oral communication and learner requests for more MSA/colloquial practice. Tensions between religiously focused CA curricula and learners’ communicative needs have been noted. - Measurement development: Draft instruments were created to assess identity-related motivation (CE, Islam, MEP, Instrumental) and L2LE (Classroom environment, Teacher/Curriculum, Personal Anxiety). Pilot psychometrics informed item reduction and subscale structure. Apparent contradictions emerged (high satisfaction with class/teacher yet elevated speaking anxiety), motivating the present validity and reliability study.

Design: Mixed-methods with three overlapping cross-sectional cohorts at a public women’s university ALI in KSA, combining repeated quantitative survey administrations and qualitative interviews to assess reliability and validity of subscale scores. Instruments: - Identity motivation instrument: 12 items across four subscales: Cultural Exposure (CE, 4 items), Islamic identity (Islam, 2 items), Middle East problems/politics (MEP, 2 items), Instrumental (Instrum, 4 items). - L2 Learning Environment (L2LE) instrument: 15 items across three subscales: Classroom Environment (CEnv, 7 items), Teacher/Curriculum (TC, 5 items), Personal Anxiety/speaking anxiety (PersAnx, 3 items). Sampling and data collection: - Cohort 1: Reanalysis of pilot data (Mar–Apr 2022). Survey via SurveyMonkey link distributed on WhatsApp groups to Diploma Program learners. From 63 initial responses, n=49 included after removing rows with missing instrument data. - Cohort 2: New cross-sectional survey (Jun–Jul 2022) and qualitative interviews. From 65 initial responses, n=29 included in quantitative analysis. Six volunteers were interviewed (30–35 minutes each) via videoconference in Arabic, recorded, transcribed, and translated. Interviewees included three long-term non-Saudi residents raised in KSA and three recent arrivals (∼2 years). Demographic questions preceded 10 semi-structured questions designed to probe L2 self, instrumental motives, barriers, and the role of context. - Cohort 3: New cross-sectional survey (Oct–Nov 2022). From 72 initial responses, n=39 included. Analysis: - Quantitative: Subscale scores computed for each instrument and cohort; means and SDs summarized. Standardized mean scores (mean/max possible) compared across cohorts to assess reliability and to examine apparent contradictions (e.g., high CEnv/TC vs elevated PersAnx; high CE/Islam vs lower Instrum/MEP). - Qualitative: Thematic analysis using Burnard et al. (2008). Coding frames developed per participant, then merged to derive themes. Themes were examined separately for long-term versus short-term residents to explore context effects and to provide validity evidence for subscales. Ethics: Protocol exempted by Princess Nourah bint Abdulrahman University (Exemption #HAP-01-R-059). Informed consent obtained. Data collected anonymously for surveys; interviewees provided contact details voluntarily.

- Reliability across time: Mean subscale scores were stable across Cohorts 1, 2, and 3, supporting reliability of the instruments. - Identity motivation (Table 4): - Cultural Exposure (max 28): All mean 21.5±6.1; Coh1 20.8±6.6; Coh2 22.7±4.9; Coh3 21.6±6.4. - Islamic (max 14): All 13.1±2.3; Coh1 13.2±2.1; Coh2 13.5±0.9; Coh3 12.5±3.2 (high, near ceiling). - Middle East problems/politics (max 14): All 7.6±3.8; Coh1 7.2±3.5; Coh2 8.0±4.1; Coh3 8.0±4.1 (moderate). - Instrumental (max 28): All 20.0±7.0; Coh1 18.4±7.7; Coh2 22.1±6.3; Coh3 20.6±6.1 (lower than expected given program context). - L2 Learning Environment (Table 4): - Classroom Environment (max 49): All 43.8±6.9; Coh1 44.8±4.1; Coh2 43.6±10.5; Coh3 42.9±9.3 (high satisfaction). - Teacher/Curriculum (max 35): All 28.5±6.9; Coh1 27.6±5.9; Coh2 30.4±7.9; Coh3 28.1±6.9 (high satisfaction). - Personal Anxiety (max 21): All 13.2±4.9; Coh1 11.7±4.4; Coh2 13.5±5.2; Coh3 14.4±4.9. Standardized PersAnx (≈56–69%) exceeded standardized MEP (≈51–57%), indicating notable speaking anxiety despite positive class/teacher ratings. - Qualitative themes: - Strong Islamic identity drives motivation; CE is framed as adopting Saudi cultural norms and practices perceived as ideal Islamic behavior. - Learners (both long- and short-term residents) sought social and cultural integration in KSA tied to becoming closer to Islam. - Communication barriers: Studying CA without sufficient proficiency in conversational Arabic (MSA/local dialect) impeded practical communication, fostering speaking anxiety and reduced self-confidence. - Long-term residents recounted early experiences of ridicule when speaking CA in colloquial contexts; short-term residents reported grammar/listening difficulties and actively sought interaction to improve. - Item wording and diglossia: Instrument items did not explicitly differentiate CA vs conversational Arabic, likely contributing to lower Instrumental scores and highlighting the need to refine items. - Recommendations emerging from findings: Incorporate targeted conversational Arabic (MSA/dialect) practice into the CA-focused curriculum (e.g., language exchange sessions) to reduce speaking anxiety and support integration goals.

The study’s research questions focused on (a) validity evidence for subscale measurements and (b) reconciling contradictions observed in pilot results. Stable subscale means across three cohorts provide reliability evidence. Qualitative data substantiate the constructs measured by identity subscales (CE, Islam, MEP, Instrumental): learners’ strong Islamic identity underpins motivation, and CE is conceived as adopting Saudi norms associated with ideal Islamic conduct. MEP interest emerged but was secondary. Lower-than-expected Instrumental scores appear linked to ambiguity in item wording regarding CA vs conversational Arabic—learners often interpreted instrumental/communication items as referring to dialectal use, which they lacked. For the L2LE instrument, high CEnv and TC scores coexisted with elevated PersAnx. Interviews clarify this tension: instruction centered on CA did not equip learners for everyday conversational interaction in KSA, generating speaking anxiety and reduced confidence in class and social contexts. Thus, PersAnx captures a real and impactful barrier consistent with learner narratives, supporting its validity. The diglossic context (CA vs MSA/ammiyah) is crucial for interpreting motivation and classroom experiences; distinguishing these varieties in measurement is necessary. Practical implications include integrating limited, structured conversational Arabic activities into ALI programs (e.g., language exchange panels with native speakers) to bolster communicative competence, lower anxiety, and better align curricula with learners’ identity-driven goals.

This study provides evidence of reliability over time for subscale scores from two instruments measuring L2 Arabic motivation (identity motivation; L2 learning environment) among ALI Diploma Program learners at a public women’s university in KSA. Validity is supported for all four identity subscales and for the L2LE speaking-anxiety subscale; the latter explains previous contradictions in light of diglossia-related communicative barriers. The work highlights the need to refine instrument wording to distinguish CA from conversational Arabic and recommends incorporating minimal conversational practice into CA-focused curricula to reduce anxiety and facilitate cultural integration aligned with learners’ Islamic identity motivations. Future research should: (1) further validate the L2LE classroom and teacher/curriculum subscales with targeted qualitative probes; (2) test instruments with male learners and in non-religious/non-Arabic-speaking contexts; and (3) evaluate the impact of integrating conversational Arabic activities on PersAnx and overall motivation.

- Sample drawn from a single public women’s university ALI; findings may not generalize to male learners or other institutions. - Strong Islamic motivations in this context may not reflect motivations of learners in non-Arabic-speaking or non-religious settings. - Interview protocol did not explicitly probe classroom/teacher/curriculum issues, limiting qualitative validation of CEnv and TC subscales; interviews conducted by the ALI Dean may have inhibited critical feedback. - Cross-sectional cohorts likely overlapped; substantial missing data reduced analyzable samples (final N=117/200), which may affect precision. - Instrument items did not distinguish CA vs conversational Arabic, potentially affecting interpretation of Instrumental subscale responses.