Uncovering the failure of Agile framework implementation using SSM-based action research

The paper addresses why many organizations fail to properly implement Agile software development (SD) despite its widespread adoption and intended benefits (meeting business needs, improving customer satisfaction, reducing bugs, shortening cycles, and adapting to change). Organizational learning and knowledge management are framed as central to understanding Agile implementation, emphasizing the role of individuals, teams, and organizations in creating, sharing, and retaining knowledge. Prior literature highlights the importance of tacit and explicit knowledge in SD and how close interactions and informal communications help teams understand requirements and build shared mental models. Despite Agile’s promise and apparent success rates relative to waterfall (as per the CHAOS report), implementation remains challenging, suggesting a need for frameworks that make the learning process explicit and actionable. The study poses two questions: whether current supporting frameworks can uncover actual issues in Agile SD implementation, and how such frameworks can support exploration of the learning process during implementation. It proposes using Soft Systems Methodology (SSM)-based Action Research (AR) to identify and address the real issues in Agile SD implementation by leveraging both tacit and explicit knowledge in a human-centric, experience-based approach.

The literature connects individual, team, and organizational learning, positioning individuals as agents of organizational knowledge creation (Argyris & Schön, Nonaka & Takeuchi). Tacit knowledge, largely internal, context-rich, and difficult to articulate, contrasts with explicit knowledge, which is codified and easily communicated. Disseminating tacit knowledge is challenging due to its complexity and context-dependence, yet it is central to SD where informal communication and close collaboration are pivotal.

• Types and levels of learning: Individual learning (intuition, reflection, interaction), team learning (feedback, alignment, group reflection), and organizational learning (learning from experience, continuous improvement) are interrelated. Close interactions within teams are vital for co-evolution and collective learning.
• Bridging individual and organizational learning: Differences in objectives, factors (e.g., personal characteristics vs. structure/culture/technology), and processes (tacit vs. explicit) require mechanisms that connect personal tacit knowledge to organizational explicit knowledge. Frameworks should capture team learning as teams act, retrieve, and reflect on feedback, adapting over time.
• Knowledge management in SD: SD depends on specialized knowledge to resolve complex, time-sensitive issues; both tacit and explicit knowledge are exchanged through close, often informal interactions among developers and with customers. Tacit knowledge underpins Agile learning processes.
• Existing frameworks: A systematic review across ProQuest, IEEE, ScienceDirect, and Scopus initially identified 1,359 records; 87 papers matched Agile implementation using knowledge management approaches; 22 discussed supporting frameworks; 14 focused on knowledge management/learning approaches. While several works consider learning or human-centric factors (e.g., Qumer & Henderson-Sellers 2008; McAvoy & Butler 2009; Hoda & Noble 2017; Pries-Heje & Baskerville 2017), they generally lack clear explication of how frameworks were developed, how learning processes are discovered and maintained, and practical application linking conceptual models to problem solving. Thus, the literature does not provide a sufficiently interpretivist, practice-grounded framework to uncover real implementation issues in Agile SD.

The study adopts Soft Systems Methodology (SSM)-based Action Research (AR) to uncover and address issues in Agile SD implementation by capturing and converting tacit knowledge into explicit knowledge through structured, participatory inquiry. Rationale for SSM-based AR:

• Addresses complex, human-centric problem situations with multiple worldviews.
• Emphasizes accommodation among stakeholders’ perspectives and supports systemic learning.
• Enables two learning modes: (1) comparison between conceptual and reality models (tacit to explicit), and (2) learning by doing during action (explicit to tacit/internalization). Research design and steps (SSM 7-step process):

1. Enter situation considered problematical (unstructured problems): Collect information via observation, informal interviews, and secondary data (e.g., meeting minutes) to identify perceived Agile implementation issues.
2. Express the problem situation (structured problems): Use rich pictures and cultural, social, and political analyses to structure stakeholder perspectives and identify problem owners (head of business, head of department, product owners/scrum masters).
3. Formulate root definitions (RDs) of relevant systems of purposeful activity: Apply PQR (what/how/why), CATWOE (Customers, Actors, Transformation, Weltanschauung, Owners, Environment), and 3E (Efficacy, Efficiency, Effectiveness) to define transformations. Six RDs were developed addressing: optimal squad working process; PO alignment forums; improved interaction/collaboration to understand business requirements; clarity of PO/SM roles and responsibilities; quality of requirements; and storing/retrieving information supporting face-to-face communication.
4. Build conceptual models: For each RD, construct a purposeful activity model depicting activity groups, dependencies, and monitoring/control elements. Conceptual models included processes for evaluating current tasks and performance, establishing forums, enhancing collaboration, defining roles, improving requirements quality, and creating storage/retrieval processes aligned with Agile values.
5. Compare models with real-world situations: Conduct structured discussions to contrast conceptual models with current practice, posing “Who does what, how, when?” to surface gaps and facilitate accommodation among differing worldviews.
6. Define feasible and desirable changes: Summarize analyses to propose actions that are both culturally acceptable and practical.
7. Take action to improve: Execute agreed actions; perform critical reflection to capture learning-by-doing (planned for future phases). The present study reports primarily on steps 1–6 and the first learning mode. Case context and data collection: Conducted within a financial institution’s IT department after over a year of Agile SD adoption. Data were gathered via informal observation, informal interviews with heads of business and department, POs/SMs, and review of internal documentation. The rich picture captured key issues: perceived IT unproductivity, knowledge/communication gaps between squads and business, unclear PO/SM roles, minimal documentation leading to unclear requirements, and difficulty recalling context from prior user stories. Analysis artifacts: CATWOE tables for each RD (six in total) specifying stakeholders, transformations, worldviews, ownership, environmental constraints, and 3E criteria; six corresponding conceptual models detailing activities and monitoring controls; comparison matrices/discussions leading to identified learnings and recommended actions (Tables 12–13).

From the step-5 comparisons (conceptual vs. real-world) using Uchiyama’s first learning mode, the team surfaced six key issues in Agile SD implementation (Table 12) and corresponding actions (Table 13):

1. Optimal working process: Comparing squads to other lines of business (LOBs) is inappropriate due to differing objectives, strategies, and systems. Action: Define, agree, and execute an expected working process within squads.
2. Product Owner (PO) alignment: Coordination exists informally but lacks a specific, disciplined forum for knowledge sharing. Action: Create a recurring PO forum to discuss and resolve current issues.
3. Interaction and collaboration for requirements: POs report insufficient business-IT interaction, creating knowledge gaps in understanding end-to-end processes. Action: Strengthen interaction and collaboration practices within Scrum events and beyond.
4. Roles and responsibilities: Absence of clear rules/guidelines for POs (and role overlaps with SMs) hinders performance. Action: Define, socialize, agree, and enforce clear PO and SM roles and responsibilities.
5. Requirements quality and documentation: Misunderstanding that Agile entails “minimal documentation” led to unclear requirements. Action: Define and socialize standards for clear, quality requirements to ensure working software and reduce defects.
6. Knowledge capture and recall: Existing documentation tools are insufficient to support recall of knowledge shared face-to-face; teams require more time to reconstruct context. Action: Enforce use of existing tools at every meeting to capture outcomes of face-to-face communication; adopting new tools deemed infeasible due to cost. These findings demonstrate that SSM-based AR exposed tacit assumptions and misunderstandings (e.g., about documentation and benchmarking across LOBs) and translated them into explicit organizational learning and actionable improvements.

The study demonstrates that SSM-based AR effectively uncovers real issues in Agile SD implementation by organizing and comparing stakeholders’ tacit understandings against conceptual models. The first learning mode (step 5) converts tacit knowledge into explicit knowledge by debating gaps between idealized processes and current practice, revealing misconceptions (e.g., minimal documentation), structural gaps (unclear PO/SM roles), and missing mechanisms (formal PO forums, consistent knowledge capture). This addresses the research questions by: (1) providing a supporting framework that discovers actual implementation issues grounded in participants’ worldviews; and (2) enabling exploration of the learning process through structured, iterative reflection and accommodation. The approach integrates human-centric considerations and knowledge management principles, capturing team learning and linking individual tacit knowledge to organizational explicit practices. The second learning mode (step 7)—internalization during action—remains for future work but is anticipated to reinforce and embed new practices across the organization.

This study fills an empirical gap in understanding the learning processes underlying Agile SD implementation failures. It proposes and applies SSM-based action research as a supportive framework that captures tacit knowledge, facilitates team learning through accommodation of multiple worldviews, links individuals’ knowledge within the organization, and structures exploration of the learning process to improve implementation outcomes. In a corporate case, the method surfaced six concrete issues and feasible actions related to working processes, PO alignment, collaboration, role clarity, requirements quality, and knowledge capture. Contributions include: (a) clarifying how a framework can be developed and validated using both tacit and explicit knowledge; (b) demonstrating an interpretivist, experience-based approach to Agile implementation; and (c) integrating human-centric and knowledge management concepts to enhance organizational learning. Future research should complete the AR cycle by executing agreed changes (step 7), studying the second learning mode (explicit-to-tacit/internalization), and evaluating outcomes of the recommended actions (including cultural feasibility and performance impacts).

The study reports only the first learning mode (step 5) of SSM; the action phase (step 7) and associated second learning mode (internalization of explicit knowledge through learning-by-doing) have not yet been conducted. Thus, the embedding and longer-term effects of the recommended changes remain unassessed. Additionally, data are not publicly available due to privacy constraints.