Scrum and The Illusion of Control

Scrum is very popular with managers and agile consultants. Consultants often advocate for the use of Scrum, especially in organisations new to Agile because the proscriptive nature of Scrum and its ceremonies is easy to understand and implement. Developers, on the other hand, often dislike Scrum because it is proscriptive nature is often difficult to adapt to the real world pressures of changing priorities and deadlines, and the reality of imperfect estimation. (Here, here, and here for example.) Developers may see Scrum as being weaponized by the product owners and managers as a way to force them to estimate work that isn't actually ready for estimation. If teams, and their leaders, fail to enforce the rules of Scrum, then the ceremonies can become artificial and team's can lose faith in the process. For example, if teams plan out sprints but are then force to accept changes intra-sprint without real acceptance of the consequence of those changes. Conflicts about Scrum can lead to a lack of trust between teams, their leaders, and their stakeholders.

Conflicts about Scrum can lead to a lack of trust between the team and their leaders and stakeholders.

But why did Scrum come about? A common meme suggests that Scrum was invented by managers to control Agile teams. It is clear that Agile, and especially its predecessor eXtreme Programming (XP), was designed to empower teams to own their own development process. Scrum was developed around the same time as XP and itself also predates the Agile Manifesto by several years. While it is true that Scrum was largely developed as a management tool, the clear goal of Scrum was team alignment, using the rugby concept of a Scrum as a metaphor for the team working together to achieve a common goal. Scrum casts itself in sharp contrast to the traditional waterfall approach where teams are often siloed and work in isolation from each other, and managers may assign tasks to individuals as if they are all interchangeable cogs in a machine.

But does Scrum really provide control? And if it does, is it the kind of control that managers want?

The root of the problems with Scrum and Agile, especially in larger organisations, is that management (i.e. executive management that is removed from the day-to-day life of software development) often thinks Agile is a silver bullet for all that ails their software development processes. Managers what software development to be predictable, repeatable, and largely pain free, especially where cost and budgeting are concerned. Yet agile and Scrum are not designed to provide this kind of control. Yes, they are intended to make the process better and more predictable, but this is largely a consequence of increased human interaction. One of the 12 principles of the agile manifesto is: "Business people and developers must work together daily throughout the project."
Another is: "Working software is the primary measure of progress." Equally important is understanding what is not in the agile manifesto.
There is no mention of cost, budget, or schedule. There is no mention of metrics or KPIs. If senior leaders believe that Scrum is going to deliver scheduled and KPI's they are misled.

A strong cohort of consultants is selling "Agile at scale" as a way to deliver these things, but this is usually just a rebranding of waterfall programme management with incremental delivery of requirements. It is not Agile. It is not Scrum. It is not even Lean. Agile at scale should be about aligning teams and stakeholders, and building organisational and project structures that are designed to accommodate flexibility and change. It can be done, but too often this is not what is actually being sold. (Hint: if your agile at scale programme is talking about 3 month "sprints" - it's not doing agile at any scale.)

Agile = People, empathy, and communication

Where teams go wrong is when everyone forgets or ignores what Agile is really about. People, empathy and communication. If customer-team interactions are relegated to being "every two weeks in a massive sprint planning session," then it should be no surprise that the team and the customer are not aligned. If managers are asking teams to estimate work for the first time when they come to a planning session it will not be effective or useful, and yes, it will be a waste of everyone's time.

When developers complain about Scrum being a waste of time they demonstrate either a lack of understanding of what Scrum and Agile are really about, or an unwillingness to provide feedback and participate in continuous improvement. Developer complaints may be a strong telltale that something is actually wrong with the process and needs to be changed.

Developers that complain about tne need for daily stand-ups are missing the point of collaborating and working as a team. If the team doesn't get good value from the stand-up, if it is actually just a status meeting for their manager, than there is a lack of collaboration and communication. Is the team really aligned on a single objective? Are team members standing up to help each-other when someone is struggling or needs an extra pair of hands? How often are teams interacting with their customers and stakeholders? Are they getting feedback on their work regularly? If the answers to these questions are not positive, then we should ask if the team is really doing Scrum or Agile at all? It's probably time for a deep retrospective discussion to understand what is going wrong and brainstorm ideas to fix it.

Scrum is not the only Agile methodology of course. Nor is Scrum even one methodology, as many Scrum experts will attest Scrum that works is the Scrum that a team adopts for itself and refines through continuous improvement. Many teams may find a Kanban process may work better. Teams should be encouraged to experiment and find a process that works for themselves and their stakeholders. But sides, and especially management need to be clear on what they expect from Agile. This should be clearly communicated and also realistic.

If teams (developers, managers, product owners) don't really understand or believe in Agile, then it is unlikely to work with any methodology. Start with education, and focus on the value of communication to all the involved stakeholders. Once teams are communicating, iterate.