Engineering culture beyond scrum / SAFe / agile with capital A

Agile manifesto and Agiles with capital A

Agile manifesto is still as true today as it was when it was published: https://agilemanifesto.org/ (also note the 12 principles https://agilemanifesto.org/principles.html)

For years now what is most often associated to to the word 'agile' is not the manifesto though but rather specific implementations of agility with their own doctrines. You've heard of these and most likely experienced one or several: XP, scrum and the later organizationally scaled up versions SAFe, lean business etc.

One thing that has often bothered me about many of these methodologies is the dogmatism their practitioners often preach which easily goes against the foundation on which it is built.

Agile manifesto and the core principles it and its implementations are based on is that of the realization of human imperfection. We cannot make detailed plans for the future - especially in a context with significant unknown unknowns (i.e. things we don't even know that we don't know). Thus an iterative model is forced upon us whether we want or not - and the more we align our planning with human realities the better we're likely to do.

This iterative and somewhat unpredictable nature of building software naturally goes against traditional annual budgetary planning and everything that has traditionally followed from that (I'm looking at you waterfall planning) and the most modern variants attempt to reconcile these two worlds at the scale of large enterprises (e.g. SAFe - scaled agile framework).

A framework is only a framework though and eventually all the details are up to individuals.

The principles on which the entire frameworks are built are so easily forgotten in the heat of immediate priorities and restrictions. Many of the principles from the manifesto are so much easier to say than to actualize.

One of the core factors that these frameworks and individual companies often struggle with and try to balance is the control between individual teams and centralized functions (applying to operations, architectural guidelines, budgeting, prioritization and many other factors).

Spotify's squad model

Spotify provides a different model, having noticed the constraints of scrum and moved to a new model of their own making.

https://www.youtube.com/watch?v=4GK1NDTWbkY

In Spotify model teams are entirely autonomous, being only given mission, architectural guidelines, product strategy and short-term goals from outside.

This model enables teams to make decisions locally without the overhead of centralization and minimize handoffs to enable higher scaling.

Following the team's mission is very important i.e. "be autonomous but don't suboptimize".

Goal is "loosely coupled, tightly aligned squads".

You might think alignment and autonomy are hopelessly intertwined but they can be thought of as a two-dimensional grid where the sections are:

• Low alignment + low autonomy = micromanagement with no high level purpose, just shut up and follow orders
• High alignment + low autonomy = leaders are good at communicating what needs to be solved but are also telling you how to solve it
• High alignment + high autonomy = leaders focus on what needs to be solved but let teams figure out how to solve them
• Low alignment + high autonomy = teams do whatever they want without supervision or direction

Strong alignment affords higher autonomy. This is an important thought - trust has to be earned. It's impossible (for a multitude of reasons) to jump from a culture of low alignment + low autonomy into a the opposite on the grid. Even if that is the direction, the progress must be made in steps so that the teams on the other hand get the opportunity to show they're up to the challenge and on the other hand leadership can display they can stay they hand off the micromanagement wheel enough to give teams sufficient room to grow.

In the squad model architectural guidelines and standardization don't work in the usual way - it's more of a cross-pollination leading to standardization model (i.e. if enough teams think it's a good idea to start doing it then at some point a critical mass is encountered and the rest of the teams should start following along the same lines as well).

People are something that cannot be over-emphasized in this model. While high quality recruitment is essential for any company, in a place shooting for autonomous teams it becomes paramount. Having a positive, constructive and highly skilled team is essential to building the necessary trust for your team mates and other teams to be able to deliver on their mission and promises. It's a combination of competence and attitude.

In the Spotify model the cross-cutting concerns' information sharing is accomplished via forming chapters and guilds which collect together people performing similar responsibilities from multiple teams in larger groups (i.e. chapters for "tribes" that are a collection of teams and guilds for the entire company). These groups then have bi-annual conferences, meetups and other communal organization methods for information exchange and idea pollination.

One interesting note was that "most organizational charts are an illusion so we focus on community rather than hierarchical structure". This very much reflects my personal experience - hierarchy can only capture one perspective of the organization while all the others are usually left up to people to figure out impromptu (SAFe and some other methodologies understand this problem and propose some solutions to it).

"We found that a strong enough community can get away with informal decision structure: if you need to know exactly who is making decisions, you are in the wrong place"

This overall model is an enabler for the practice promoted by most agile ideologies - that is small frequent releases. In the Spotify model this is accomplished by the true decentralization of decisions.

This model also doesn't mean that everyone does everything - there are still squads that focus on infrastructure, squads that focus on client apps and squads that focus on features but even then handoffs are avoided like plague.

For inevitably required synchronization between releases the squad model relies on release train and converges with SAFe on this front albeit from a different angle of approach. Feature toggles are interestingly mentioned as an enabler for being able to keep simplified version control structures and "release" unfinished features (while not being active yet). I'll have to write a post on feature toggles in the future since that is a subject which is popping up on increasing frequency.

This model is not something you can just transition to over a few months. It's something you could set as a long term goal and start building the requisite team trust along the way (and inevitably exchange some people incompatible with the model along the way).