Software Developers Are Humans Too

(2023-07-27)

Assessing the value created by our teams is a key piece of every manager's job. Good managers are constantly evaluating their teams and course correcting throughout the year. Eventually, we have to codify assessments in annual reviews and bonus checks. During feedback sessions, we don’t want our employees to feel that our reviews are not factually based (SMART feedback). Unfortunately, when we evaluate technical teams, too often we are forced to fall back on gut feelings and fuzzy feedback because we lack good tools for quantitatively measuring software.

Even where tools exist, they tend to be flawed in one way or another. When used as measuring sticks, especially when incentives are on the line, poor metrics can backfire. (For example, incentivising new features can lead to ignoring platform stability.) Additionally, many common metrics tend to be focused solely on developers ignoring the value contributed by other members of the teams. Ideally, metrics should be of value to every member of the team. As an example, Including metrics focussed on system reliability can help broaden the scope and include wider members of the team.

Measuring the value of any creative profession is hard, but unlike many frontline jobs that primarily involve creative thought (e.g. lawyers, salespeople, marketing, traders) software is often relegated to a supporting role. Unfortunately, in many companies, software is still considered a cost, even in industries that are heavily dependent on technology. When software is not considered a driver of revenue it is often more difficult to directly tie the output of software teams to the bottom-line results because the organisational focus will be on the cost. (This is not the case at all companies. Younger organisations, even non-tech ones, are more likely to see technology as directly aligned with the business. I am also not suggesting that all knowledge work is front-line profit; many professions are as difficult to measure as software.)

Why is measuring the productivity of software developers and software development teams so hard? I’ve rarely seen this problem explained as well as in this article: IEEE: A Human-Centered Approach to Developer Productivity (Jaspan & Green 2023). While the authors don’t offer any answers, they do provide a thorough examination of why this problem is so hard to solve.

A few points struck home for me.

How do you explain the cost of estimating and planning work to a user? If I had a dollar for every time I was told “Estimating is free,” I would be on the beach somewhere today.

Decomposition, though, is itself an engineering task and is arguably much more difficult and time consuming than the completion of the decomposed work. (Jaspan & Green)

When we need to justify time that is not just "writing code":

Problem solving (because that’s what this really is) involves a bunch of work that doesn’t result in an immediate objective output, is hard to measure, and may pay long-term dividends in productivity. (Jaspan & Green)

Why are estimates often wrong? (Besides pointing users at the dictionary definition of estimate).

The output of software development is not known from the start: In part, that’s because the solution is not known ahead of time. It may be well constrained, and it may look a lot like solutions that have been built in the past, but even where code reuse is extensive and a developer draws on bits and pieces from others’ solutions, the output of a software development task is unique. (Jaspan & Green)

Measuring the productivity of software developers is hard, but it is not impossible. The ideal metrics will be of value to the developers themselves, their managers, and their business. Taking time to produce good metrics will pay dividends in the long run.