Unit 5: Agile Foundations and Frameworks

1. Introduction to Agile

Agile is an iterative approach to project management and software development that helps teams deliver value to their customers faster and with fewer headaches. Instead of a single, large "big bang" launch, an agile team delivers work in small, but consumable, increments. Requirements, plans, and results are evaluated continuously, so teams have a natural mechanism for responding to change quickly.

The Agile Manifesto (2001)

In 2001, 17 software developers met in Snowbird, Utah, to discuss lightweight development methods. They published the "Manifesto for Agile Software Development," which is based on four core values and twelve supporting principles.

Core Idea: Agile is a mindset, not a specific methodology. It's about embracing uncertainty and using iterative feedback loops to navigate complexity.

2. Agile Project Management

Agile Project Management (APM) is a paradigm shift from traditional, predictive project management (like Waterfall).

• Traditional PM: Focuses on upfront planning, detailed requirements gathering, and sequential execution (analyze -> design -> build -> test -> deploy). The goal is to control the project to meet the plan. It is best suited for projects where requirements are well-understood and unlikely to change.
• Agile PM: Embraces change and uncertainty. It uses iterative and incremental cycles (called "sprints" or "iterations") to develop and deliver parts of the project. The goal is to "inspect and adapt" based on feedback. It is best suited for complex projects where requirements are expected to evolve.

Key Characteristics of APM:

• Iterative: Work is done in repeating cycles.
• Incremental: Completed work is delivered in small, usable pieces.
• Empirical: Decisions are based on observation, experience, and experimentation rather than detailed upfront planning.
• Customer-Centric: A high degree of customer/stakeholder collaboration is essential.
• Team-Oriented: Relies on self-organizing, cross-functional teams.

3. Agile Values and Principles

The Four Core Values

The Agile Manifesto prioritizes certain values over others, stated as:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

1. Individuals and interactions over processes and tools
   • Explanation: While tools and processes are important, the people doing the work and their collaboration are more critical to success. A good team with a bad tool will outperform a bad team with a good tool.
2. Working software over comprehensive documentation
   • Explanation: Documentation has its place, but the primary measure of progress is a functional product that delivers value. Agile prioritizes creating the product itself over creating extensive documentation about it.
3. Customer collaboration over contract negotiation
   • Explanation: Projects succeed when developers and customers work together continuously. A collaborative partnership is more flexible and effective than rigid, adversarial contracts that try to predict every eventuality.
4. Responding to change over following a plan
   • Explanation: The business environment, market conditions, and user needs change. Agile embraces this by building change into the process, allowing the team to pivot and deliver what is truly needed, rather than blindly following an outdated plan.

Note: While there is value in the items on the right, Agile values the items on the left more.

The Twelve Supporting Principles

1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
4. Business people and developers must work together daily throughout the project.
5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
7. Working software is the primary measure of progress.
8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
9. Continuous attention to technical excellence and good design enhances agility.
10. Simplicity—the art of maximizing the amount of work not done—is essential.
11. The best architectures, requirements, and designs emerge from self-organizing teams.
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

4. Agile vs. Waterfall/Predictive vs. Hybrid

5. Agile Frameworks

Agile is a mindset. Frameworks are specific, structured approaches for implementing that mindset. No single framework is inherently "better"; the best choice depends on the team, organization, and project.

• Scrum: A prescriptive framework focused on time-boxed iterations (Sprints) for completing complex work.
• Kanban: A method focused on visualizing work, limiting work in progress (WIP), and maximizing flow.
• Extreme Programming (XP): A software development framework focused on technical excellence and engineering practices.
• Lean: A philosophy focused on maximizing value by eliminating waste.

6. Venn Diagram: Agile, Scrum, and Kanban

This textual diagram illustrates the relationship between the Agile mindset and two popular implementations.

+-------------------------------------------------------------+
| Agile Mindset (Values & Principles)                         |
|                                                             |
|   +--------------------------+                              |
|   | Scrum (Framework)        |                              |
|   | * Roles (PO, SM, Devs)   |      +---------------------+   |
|   | * Events (Sprints, etc.) |      | Kanban (Method)     |   |
|   | * Artifacts (Backlogs)   |      | * Visualize Workflow|   |
|   |                          |      | * Limit WIP         |   |
|   |        (Prescriptive)    |      | * Manage Flow       |   |
|   +--------------------------+      |    (Adaptive)       |   |
|                                     +---------------------+   |
|             (Both are Agile implementations that          |
|              emphasize transparency, feedback loops,       |
|              and continuous improvement)                   |
|                                                             |
+-------------------------------------------------------------+

• Agile is the overarching umbrella of values and principles.
• Scrum is a structured framework within Agile. It has specific roles, events, and rules.
• Kanban is a less prescriptive method within Agile that can be applied to existing processes.
• Teams can also combine practices, leading to Scrumban, which uses Scrum roles and events with a Kanban board and WIP limits.

7. Lean Agile

Lean thinking originates from the Toyota Production System. In software, Lean Agile focuses on delivering maximum customer value by relentlessly eliminating waste and optimizing the entire value stream.

Core Lean Principles

1. Eliminate Waste: Identify and remove anything that does not add value for the customer.
2. Amplify Learning: Use short iteration cycles, feedback, and continuous integration to learn faster.
3. Decide as Late as Possible: Defer irreversible decisions until you have more information to reduce risk.
4. Deliver as Fast as Possible: Reduce cycle time to get value and feedback to the customer quickly.
5. Empower the Team: The people closest to the work are best equipped to make decisions about it.
6. Build Integrity In: Ensure quality is part of the process from the start (e.g., automated testing, refactoring).
7. See the Whole: Optimize the entire value stream from concept to delivery, not just individual parts.

The 7 Wastes of Software Development (Muda)

• Partially Done Work: Code that isn't tested, documented, or deployed. It represents a sunk cost with no value.
• Extra Features: Building features the customer doesn't need ("gold plating").
• Relearning: Solving the same problem multiple times due to poor knowledge sharing.
• Handoffs: Transferring work between people or teams, which introduces delays and potential for miscommunication.
• Delays: Time spent waiting for approvals, reviews, or dependencies.
• Task Switching: Context switching between multiple projects or tasks, which reduces focus and efficiency.
• Defects: Bugs that require rework, testing, and redeployment.

8. Kanban

Kanban is a visual method for managing workflow. It is not as prescriptive as Scrum and focuses on continuous flow rather than fixed-length iterations.

Core Principles

1. Start with what you do now: Don't make drastic changes. Apply Kanban to your existing process.
2. Agree to pursue incremental, evolutionary change: Favor small, continuous improvements over radical shifts.
3. Respect the current process, roles, responsibilities & titles: Kanban does not prescribe new roles.

Core Practices

1. Visualize the Workflow: Use a Kanban Board with columns representing stages of your process (e.g., Backlog, To Do, In Progress, In Review, Done). Work items are represented as cards that move across the board.
2. Limit Work In Progress (WIP): Each "in-progress" column has a numerical limit on how many cards it can hold at one time. This prevents bottlenecks, exposes process issues, and encourages a "pull" system where new work is only started when capacity is available.
3. Manage Flow: Measure and optimize the movement of work through the system. The goal is a smooth, predictable flow.
4. Make Process Policies Explicit: Clearly define the rules for moving a card from one column to the next (your "Definition of Done" for each stage).
5. Implement Feedback Loops: Use regular meetings (like stand-ups or retrospectives) to review the flow and identify areas for improvement.
6. Improve Collaboratively, Evolve Experimentally: Use data and team collaboration to make changes to the process.

Key Kanban Metrics

• Lead Time: The total time from when a request is made to when it is delivered.
• Cycle Time: The time it takes for a work item to move through a specific part of the process (e.g., from In Progress to Done).

9. XP (Extreme Programming)

XP is an agile framework that emphasizes high-quality software development through specific engineering practices. It aims to improve software quality and responsiveness to changing customer requirements.

XP Values

• Simplicity: Do the simplest thing that could possibly work. Avoid over-engineering.
• Communication: Foster direct, face-to-face communication among the team and with the customer.
• Feedback: Seek rapid feedback at all levels (from code, from the customer, from the team).
• Courage: The courage to refactor code, discard failing ideas, and speak truth to power.
• Respect: Team members respect each other's expertise and contributions.

Key XP Practices

• The Planning Game: Quickly determine the scope of the next release based on business priorities and technical estimates.
• Small Releases: Release small, valuable increments of working software frequently.
• Metaphor: A simple shared story of how the whole system works.
• Simple Design: Keep the design as simple as possible for the current requirements.
• Test-Driven Development (TDD): Write a failing automated test before writing the code to make it pass.
• Refactoring: Continuously improve the internal structure of the code without changing its external behavior.
• Pair Programming: Two developers work together at one workstation to write code. One "drives" (writes code) while the other "navigates" (reviews and thinks strategically).
• Continuous Integration (CI): Developers integrate their work frequently (at least daily). Each integration is verified by an automated build and test sequence.
• Collective Code Ownership: Anyone on the team can improve any piece of code at any time.
• Sustainable Pace: Work at a pace that can be maintained indefinitely without burnout (e.g., 40-hour week).
• On-site Customer: A dedicated customer representative is available full-time to the team to answer questions and set priorities.

10. Introduction to Scrum

Scrum is a lightweight yet powerful framework for developing and sustaining complex products. It is founded on empirical process control theory, meaning knowledge comes from experience and decisions are based on what is known. Empiricism rests on three pillars:

• Transparency: Significant aspects of the process must be visible to those responsible for the outcome.
• Inspection: The team must frequently inspect the Scrum artifacts and progress toward a goal to detect undesirable variances.
• Adaptation: If an inspector determines that one or more aspects of a process deviate outside acceptable limits, the process must be adjusted.

The 3-5-3 of Scrum

Scrum is defined by 3 Roles, 5 Events, and 3 Artifacts.

The 3 Scrum Roles (The Scrum Team)

1. The Product Owner (PO):
   • The "voice of the customer."
   • Responsible for maximizing the value of the product resulting from the work of the Development Team.
   • Solely responsible for managing the Product Backlog, including creating, prioritizing, and clearly communicating backlog items.
2. The Scrum Master (SM):
   • A servant-leader for the Scrum Team.
   • Helps everyone understand Scrum theory, practices, rules, and values.
   • Removes impediments (blockers) to the team's progress.
   • Facilitates Scrum events as needed.
   • Coaches the team in self-organization and cross-functionality.
3. The Development Team:
   • A self-organizing, cross-functional group of professionals who do the work of delivering a potentially releasable Increment of "Done" product at the end of each Sprint.
   • Typically 3-9 members.
   • Owns the Sprint Backlog and has autonomy over how they turn Product Backlog items into a finished Increment.

The 5 Scrum Events

1. The Sprint:
   • A time-box of one month or less during which a "Done," usable, and potentially releasable product Increment is created.
   • Sprints are of consistent duration. A new Sprint starts immediately after the conclusion of the previous Sprint.
2. Sprint Planning:
   • Time-boxed event to plan the work to be performed in the Sprint.
   • Answers two questions: "What can be delivered in the Increment resulting from the upcoming Sprint?" and "How will the work needed to deliver the Increment be achieved?"
   • The output is the Sprint Goal and the Sprint Backlog.
3. Daily Scrum (Daily Stand-up):
   • A 15-minute time-boxed event for the Development Team to synchronize activities and create a plan for the next 24 hours.
   • Often structured around three questions: "What did I do yesterday?", "What will I do today?", and "Do I see any impediments?"
4. Sprint Review:
   • Held at the end of the Sprint to inspect the Increment and adapt the Product Backlog if needed.
   • The Scrum Team and stakeholders collaborate on what was done in the Sprint. This is a demo of the working software, not a status meeting.
5. Sprint Retrospective:
   • An opportunity for the Scrum Team to inspect itself and create a plan for improvements to be enacted during the next Sprint.
   • Focuses on the process: what went well, what could be improved, and what the team will commit to changing.

The 3 Scrum Artifacts

1. Product Backlog:
   • An ordered, living list of everything that is known to be needed in the product.
   • It is the single source of requirements for any changes to be made to the product.
   • Managed by the Product Owner. Items at the top are more detailed and have higher priority.
2. Sprint Backlog:
   • The set of Product Backlog items selected for the Sprint, plus a plan for delivering the product Increment and realizing the Sprint Goal.
   • It is a forecast by the Development Team about what functionality will be in the next Increment and the work needed to deliver that functionality.
   • Managed by the Development Team.
3. The Increment:
   • The sum of all the Product Backlog items completed during a Sprint and the value of the increments of all previous Sprints.
   • At the end of a Sprint, the new Increment must be "Done," which means it is in a usable condition and meets the Scrum Team's Definition of Done.

11. Deeper Dives into Key Concepts

Scrum Master

The Scrum Master is not a project manager. They do not manage people or tasks. Their primary role is to serve the team, the Product Owner, and the organization.

• Service to the Product Owner: Helping find techniques for effective Product Backlog management; facilitating stakeholder collaboration.
• Service to the Development Team: Coaching in self-organization; helping create high-value products; removing impediments.
• Service to the Organization: Leading and coaching the organization in its Scrum adoption; planning Scrum implementations; causing change that increases the productivity of the Scrum Team.

Sprint Charter

A Sprint Charter is a helpful but not official Scrum artifact. It's a document created at the beginning of a Sprint (often during Sprint Planning) to align the team.

• Purpose: To provide a clear, shared understanding of the Sprint's objectives and boundaries.
• Typical Contents:
  • Sprint Goal: A one-sentence summary of what the Sprint aims to achieve.
  • Scope: The list of Sprint Backlog items the team has committed to.
  • Team Members: List of who is on the team for the Sprint.
  • Risks & Dependencies: Any identified risks that could impact the Sprint Goal.

Daily Stand-up

The primary purpose is not a status report to a manager. It is a planning and synchronization meeting for the Development Team.

• Goal: To inspect progress toward the Sprint Goal and adapt the Sprint Backlog for the next 24 hours.
• Common Pitfalls (Anti-Patterns):
  • Turning into a status report for the Scrum Master or a manager.
  • Exceeding the 15-minute time-box.
  • Problem-solving during the meeting (these should be taken "offline" by the relevant people after the stand-up).
  • Team members not listening to each other.

Simplicity Principle

From the 12 Agile Principles: "Simplicity—the art of maximizing the amount of work not done—is essential."

• Meaning: This principle fights against "gold plating" and over-engineering. It encourages teams to find the simplest, most direct solution to a problem.
• Practical Application:
  • Ask: "What is the simplest thing we can build to meet this need right now?"
  • Avoid building features "just in case" they might be needed in the future (YAGNI - "You Ain't Gonna Need It").
  • Focus on delivering the core value and iterating based on feedback, rather than trying to build the "perfect" solution upfront.

User Story Points & Team Velocity

User Story Points

• What they are: A unitless, relative measure of the effort required to implement a user story. "Effort" includes complexity, uncertainty, and the volume of work.
• How they work: Instead of estimating in hours or days (which is notoriously inaccurate), teams assign points. A story estimated at 2 points should be roughly half the effort of a story estimated at 4 points. The Fibonacci sequence (1, 2, 3, 5, 8, 13...) is commonly used to reflect the increasing uncertainty in larger estimates.
• Estimation Technique (Planning Poker):
  1. The Product Owner explains a user story.
  2. The team discusses it and asks questions.
  3. Each team member privately chooses a card with a story point value.
  4. Everyone reveals their card simultaneously.
  5. If estimates differ, the high and low estimators explain their reasoning.
  6. The process is repeated until the team reaches a consensus.

Team Velocity

• What it is: The average number of story points a team completes in a Sprint. It is a measure of a team's capacity for work, not their productivity or performance.
• How to calculate it: Sum the story points of all fully "Done" user stories at the end of a Sprint. After 3-4 Sprints, calculate the average.
  • Example: Sprint 1 = 25 points, Sprint 2 = 30 points, Sprint 3 = 28 points. The average velocity is (25+30+28)/3 = 27.7, or ~28 points per Sprint.
• How it's used:
  • Forecasting: If the remaining Product Backlog has 140 points, and the team's velocity is 28, they can forecast that it will take approximately 140 / 28 = 5 more Sprints to complete the work.
• Important Caveat: Velocity should never be used to compare different teams. Each team's story point scale is unique to them.

Setting Priorities

Effective prioritization is crucial for maximizing value. The Product Owner is responsible for this, but they use input from stakeholders and the team.

MoSCoW Method

A simple technique for categorizing requirements:

• M - Must-have: Critical for the current delivery time-box to be a success. The system is not viable without them.
• S - Should-have: Important but not vital. The system is still viable without them, but they may be painful to leave out.
• C - Could-have: Desirable but not necessary. A "nice to have" that will be included if time and resources permit.
• W - Won't-have (this time): Explicitly stated as being out of scope for the current time-box. This helps manage expectations.

Value vs. Effort Matrix

A 2x2 grid used to plot features:

• The Y-axis represents Value (High/Low).
• The X-axis represents Effort (Low/High).

Kano Model

A more advanced model that classifies customer preferences into five categories:

1. Must-be Quality: Expected features. If they are absent, customers are very dissatisfied (e.g., brakes on a car).
2. One-dimensional Quality: The more you provide, the more satisfied customers become (e.g., gas mileage).
3. Attractive Quality: Unexpected features that cause delight when present but no dissatisfaction when absent (e.g., the first time a car had cup holders).
4. Indifferent Quality: Features that customers don't care about.
5. Reverse Quality: Features that cause dissatisfaction when present (e.g., too many complicated buttons on a remote).

Prioritization should focus on fulfilling all Must-be requirements, maximizing One-dimensional attributes, and including a few Attractive "delighters."