Software Delivery That Makes Sense

The decline and possible restoration of reason in organizations that make software against their will or better judgment.

Now that every company is a software company, recent discussions with people around me (including those not in the software business) made me rediscover the abysmal state that software delivery is in, at many places.

This article will share some of my reflections on that, as well as let me present some non-conclusive high-level advice that I believe can form a solid foundation around which such problematic organizations might be able to be rewired into actual software orgs.

Note that when I write about organizations and businesses, they will therefore not be pure-play tech companies, i.e. Meta, Google, Spotify, or such — rather the long tail of orgs (i.e. IT-enhanced legacy companies) who unfortunately have had to adapt to software being part of their output and product.

Where I am writing this from

Some context first. I have experience and learnings from several “worlds”, both currently as a Technical Standards Lead at Polestar, being an open source maker, being an overall avid technology reader, and being the founder of a tiny brand-new tech startup (since defunct, as all ideas are not worth pursuing). Finally, together with an old colleague of mine, I am also thinking about setting up a local, free fractional CTO service for young value-driven orgs.

I started coding HTML and CSS back in 1997, have explored quite some areas since then, and while taking lots of tours and detours in life, my strict professional career started in 2016. Already the first position was a complex, messy one and I’ve loved having positions where I have to balance the hard and soft sides, up and down from C-level to other developers. I’ve run two small businesses in design and technology, perhaps not extremely successfully, but I do have experience with all that it entails. My experience with clients goes from the super small to the biggest companies in Sweden, and from everyday consumer goods to the most extremely niched services. I’ve been fortunate and very happy to help innovate and platform how software delivery can or at least should work in many organizations, with various needs and depths of skill in technology across many domains. In closing, that’s a lot of sentences starting with “I…”.🤭

OK, so why do I write the above? Singularly because I want to be fully transparent in that what I will attack comes from having experienced the pain and often “unnaturalness” of software in many organizations — it just doesn’t seem to gel in most places! There is a metric ton of reading — well enough to occupy your lifespan and beyond — if you want to dig into ideas, concepts, and philosophies of software, even before you get to the business side, how to operate startups, how to make MVPs, how to approach quality software, and so on. To be fair — none of us have the time to take it all in, and it wouldn’t be very good time management if we don’t prune our inspiration sources and intake of new ideas to some extent.

So, with this storytelling out of the way, I am going to write about three things in particular:

• That we have a problem with software in non-software orgs.
• Why developers themselves cannot make your org into a software business.
• What some actual, real-life changes might look like to regrow a foundation around.

Let’s start with some of the issues as I happen to think of them.

Software engineering, from an orbital perspective

We are an alien spaceship hovering over Earth, looking down at all those wannabe (i.e. IT) organizations.

Some facts our instruments have gathered together with reflections by our chief Human-Alien Scientist:

• Tech (especially Big Tech) has been majorly over-employing. The aim seems to be, somewhat cynically, not to put the people to work, but to remove them from someone else’s productive output.
• This goes to the natural consequence of making it more than possible to have multiple jobs at the same time. Ironic but true, given that you now have two paychecks for doing twice as much nothing.
• At the company scale, as Victor Ronin writes, “you can fire 80% of software engineers, and the company will survive”. Not that strange given that no one works in the first place, or that the typical IT organization tends to be a quagmire even if they tried.
• Personal work experience clearly tells that the “title inflation” is severe (at least in Sweden, but it seems to be a global phenomenon) — I can’t evidence this but if you have been in (or helped out with) recruiting or hiring you will most likely have seen this. In effect, people want more money for sometimes less qualified labor.
• Nationally, in Sweden, the degree of qualified candidates in IT/CS-oriented programs has been on the decline for some time. It seems some of the vocational non-theoretical education programs do scoop up some candidates though. Ergo, the market does not seem to be increasing over time — it’s shrinking, at least in terms of the most qualified juniors.
• We can generalize and say “it was easier before”, as IT was a cost center and essentially in many companies relegated to vending licenses for commercial software (that someone else built) and ensuring your printer and email worked. Now, somewhat, unfortunately, every company is a software company. This hasn’t panned out super well in reality, as it requires restructuring and rethinking — most likely the hardest nut for regular people to crack.
• IT is depressingly over-managed and underqualified. There’s most likely a long story (or many of them) underpinning that statement, but it’s quite clear that IT is attracting a lot of unqualified talent and ideas, often (but not only) in the (mis)management parts. Manager roles seem to be exploding off the vines of the great hierarchy tree, while there is less (good) output to actually manage in the first place. We need to have something that puts product (and of course people!) first, not management or shelf-stored ideas of “scaling” product management that just isn’t working.
• Product thinking, lateral thinking, and T-shaped people are hard to come by. We should ensure we have mechanisms that make our people have stronger such skills, in the day-to-day immersion as well as with regular and expected training. But we should not take in those lacking all of those in the first place.
• Process is not the enemy, contrary to what agile zealots would want you to believe. Process is great if it meaningfully keeps employees effective and happy while keeping the output at a competitive level, in both functional and cross-functional terms. The problem is that traditional orgs are not adapting processes around the much higher cadence of software delivery.
• What seems to be occupying time for software engineers in many places? It seems to be around relatively modest integration concerns, process automation, and system maintenance. Almost like a digital plumber… interesting… 🤔

“I have nothing against the professionalisation of programming; it would be a good thing. But the fact remains that the great majority of programmers are not engineers. We are tradesmen, a bit like plumbers or domestic electricians. There’s nothing to be ashamed of in being a skilled tradesman; for a good part of my career, a plumber earned more than I did.”

— User “denton-scratch”https://news.ycombinator.com/item?id=33961785

• How lucky for those poor manager souls that there is some kind of answer to that digital plumber scenario: No-code/low-code is an approach that is prescient in that it recognizes that there are not enough competent software engineers available, either in training or on the market. There are some awesome products out like Zapier, Autocode, AppSheet, AirTable, Webflow, Retool, and Carrd. Hell, levels.fyi scaled to millions of users with only Google Sheets as their backend…
• …but it’s not ripe yet for all areas of business, and it still requires somewhat of an engineering mindset to get working in a good way. So we still seem to need those engineers/developers to use the “no code” tool, when management and the rest are horrified by the fact that it still has to do with software.

Identity crisis in the age of digital capitalism

The speed of innovation in the software and development sector is blazing right now. I am writing that statement as part apologia and part love letter to address just how easy and “democratized” technology can be for companies and organizations, no matter their technical maturity or competency. Apology because in my reality, it’s actually very hard to bring large companies to reap the benefits of our time, and love letter as I know and cherish the impact developers and business people can have, if given the context of the “right” organization.

For a new start-up, this has never been as true as it is today given the plethora of well-UX’ed, well-functioning tools that can create experiences and applications that, not so long ago, were only the remit of masters of the arcane.

It seems that it hasn’t yet sunk in deep enough in many orgs that they need to define (and stratify accordingly) if they are either an organization with IT enhancements or if they are a software company with its rapid-loop product mindset of building software front-and-center of operations. The cargo cult around software and how everyone thinks it’s critical to be one means that we’ve invited wolves and charlatans to orgs with grand visions. Grand visions with inexperienced and nervous leaders, in orgs that cannot always fulfill the lofty castles in the sky, given inherent or ingrained limits and constraints that might make it fundamentally impossible to restructure around software, rather than, say, cars, shipping, ball bearings, medical products or what have you…

Our hunger to fit in with the times has clearly, in many occurrences, lead to bad decision making and ill-functioning conditions that fail to deliver high-quality, stable, and safe software… It’s fine to be a non-software company! Being a poser is much worse.

Reminder: Survival is not mandatory. Someone will always have to play the role of the dodo.

A key separator between traditional orgs and software orgs is that software is the product. Duh. Stating that, it’s assumed that we have also come to some understandings:

• Software as being extremely malleable (changeable, adaptable);
• The rate of change in software is significantly higher than in the material world;
• Competition in software is global by default and ultra-antagonistic by nature;
• Software is continuously improved, it’s not a deliverable artifact with a date stamped on it.

Deviating from or forgetting those factoids is a hazardous proposition. A matrix organization with a waterfall mindset won’t be apt to deliver well on these new game-changing circumstances. Yet that’s how the numerary majority of orgs building software seem to actually look like — keeping the old and putting software (the “new”) as an odd, quaint bird in its IT/digital department cage.

The problem is — inevitably — somewhere, sometime, your luck is going to run out if you can’t improve this situation. Your primary competition does not come from your well-known, mapped-out legacy competitors — it comes from some guy or girl writing your obituary in code in their sublet in some completely different country. And you won’t see it coming before it’s too late.

An organization centered around software caters to, adapts to, and optimizes for such facts — by being cross-functional, by having short loops in the product development cycle, by understanding the rest of the digital landscape (from legislation to user patterns, etc.), and having the user as a stakeholder, not a mindless, abstract bag of money hidden out of sight. A software company is not shielded from reality, the market, or anything else — but they adapt by having their customers close, a form of somewhat-untrustworthy Oracle of Delphi.

Developers won’t save a bad org

I’ve been known to be controversial, so I’ll go ahead and let that satanic thought out right away, so we can unpack this together… You probably shouldn’t get any (more) developers, because:

1. You can’t outsource business and “knowing the plan” to engineers. Not because they are daft, but because that’s another instrument in the big orchestra. Given that your business is not deeply ingrained in development, then that’s not where you’ll find those who drive business. Start by setting the plan and vision for your org and (possible) product. Invite devs, though!
2. The mystery and magic around developers just needs to die. Developers are not superheroes, nor are they all the mythical 10x kind. The ultimate value in creating anything at all — within the context of business — is that it supports your customer-perceived value. Developers are not the ideal people to fully understand what that value is. Development is often driven reactively and after the fact, which directly translates into code being only a measure of production rather than a measure of value. If this is what you are doing, then no, more developers is not the right way to go about business.
3. You are probably banking on the belief that making that software into something tangible (including all the non-digital processes etc.) will work out well in your legacy hierarchy. Experience says that this is hardly ever the case and that LinkedIn is full of jackals who are not speaking the truth. Or when you read a victory was “hard-won” (i.e. Pyrrhic victory), ask yourself if you want to find out what that really meant in actuality. Many times it likely translates to a project utterly whipped to bits by marketing, design, development, business, and management who had completely conflicting ideas on how to proceed and what it was all about even. Also, a large org will take its time to get work packages from idea to reality, so this will only ever happen once in a blue moon. And the rest of the world will perhaps already have moved on when the light of day shines on that burden-shaped project. Unless software is designed and built as a vertical slice that includes all the required people and processes, then it’s almost bound to fail or perform poorly given the cost and effort involved.

In short: If you don’t have an organization that can support development as it’s done today — then don’t get developers. This is perhaps the single most important thing you might want to consider and reflect on, perhaps with some hard liquor, if you are a business leader.

I know it may sting, but I’m actually trying to be your friend here.

Big organization? Probably big problems. If people are the problem, what is the solution? Logically the answer is Fewer of them and better qualified. Consequently, headcount is a really poor way to go and scale is probably not as important as you think. Instead, aim for a smaller number of highly qualified and motivated developers engaging in an organization that values transparency and cross-functional interactions.

Explosive growth in what one competent developer can do with contemporary tech offerings

Improvements in technology mean competent individuals can significantly produce more than developers who lack cutting-edge skills. From individual services to the public cloud, there are countless tales of companies running on the work of only 1–5 developers, much thanks to the serverless paradigm and its forebears.

For the more tech-heavy companies, I am seeing (and have seen) how one or a small number of developers can create things that were impossible to do without hiring the best the market had to offer, and paying with your organs. No small part of this comes from the arms race between the big clouds, but also since other tech companies, often specializing in their own field, are pouring gasoline on the fire. As “consumers” of the services, we are clear winners. Not utilizing the momentum will leave your org far behind very quickly.

When should you simply not employ developers at all?

If you’re a small business or in a context that is not deeply affected by digital. I’m thinking of restaurants and similar. If margins are too low and you have a decent enough idea of your business requirements, then you could be better served by something like Carrd, Webflow, Retool or something other low-code/no-code-oriented and no one can blame you for it, given you have a bit of a knack to think in programmatic terms. You’d not be the first or only one to successfully drive business with these commercially available and user-friendly tools.

Better software delivery now

We’ve seen that the real problem in IT organizations is less about technology and a lot more concerning people and processes. Therefore, while some of the below will be about technology, quite a lot isn’t.

Not to be harsh, but that’s also ample reason to ask yourself: “If I can’t help affect or change those aspects, what are the realistic odds of my company being good at delivering software?”. I’d wager low. Sorry, not sorry.

Let’s get to it.

Care about the customer/user

This is without a doubt number #1 on your list.

I’ve never worked at a company (no, never; not today, not before) where the company has truly cared about the customer or user. Don’t be like that.

Without the customer perspective, eventually, you have fuck all of anything. That’s the truth.

It’s a regression beyond stupidity to not have your product organization have direct contact with users (as and when appropriate). Ensure it has such capabilities!

Ethics

Read the following, regardless of your position:

• Forsgren, Humble, and Kim’s Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
• Robert C. Martin’s Clean Craftsmanship: Disciplines, Standards, and Ethics
• Dave Farley’s Modern Software Engineering: Doing What Works to Build Better Software Faster

Use these books to fuel the ethics and understanding of delivering software in your engineering organization.

We hear a lot about empowerment, such as empowering teams with the vertical responsibilities in a DevOps role, but with freedom comes responsibility — also from the rest of the org towards the engineering teams. DevOps won’t work in a feature factory. On a related note, read Product vs. Feature Teams.

Your number #2 goal should be increasing the organizational efficiency by shorter loops from idea to implementation. This is impeded by large groups or structures, rigid procedures not adding value, and not trusting smaller groups to do their work autonomously. The shorter and more intense this loop is, the better.

Have a clear and communicated vision for your product, service, or whatever your org is doing. Make it clear what everyone can do to get there. “Higher sales” does not count as an adult’s vision or guidance, but it’s certainly good in the context of a Scrooge McDuck cartoon.

Transparency in goals and true ownership and decision-making: Without vision, direction, and ownership you can have all the DevOps you could ever shake a stick at, but it will be literally for nothing. Having a meaningful, concrete stake and responsibility in the product increases work satisfaction, sense of purpose, and many other factors. In the development world, this would be something like well-oiled DevOps which also happens to work better in smaller teams with tighter, clearer scopes.

Don’t build, support, invest in or fund a management layer. Everyone needs to have a stake in the product. Management does not typically cut that requirement.

Have a clear concept of how accountability and responsibility work. While it’s common in many countries to be hard on the workers, in countries like Sweden, it’s in fact the other way around — it can be incredibly hard to put clear pressure when expectations are not met. Don’t get me as a dominatrix or taskmaster here, but if you flip the script and get inspired by any of the multitudinous psychology books available or findings from for example DORA, you’ll see that high investment and caretaking are natural consequences of a well-functioning team just doing their day-to-day work. So paradoxically — by being transparent, clear, and working together (rather than only as individuals, i.e. against each other) — you’ll create the conditions for high-performing teams where accountability tends to come automatically.

Be engaged and invested in the work. Allow remote work. People won’t cheat if they are well-selected, dedicated, and take pride in their work.

First: Trust, expectations, and high aims… next comes empowerment.

Hiring the right people

Get a few, but highly enthusiastic, developers with experience in pulling off their own projects (or other companies) in the cloud. Give these new engineers a starting task to migrate or re-platform some legacy applications you may have. It shouldn’t take too long before you can start using those exit clauses for any of the staff who are not up to par.

Don’t hire people on LinkedIn with unclear remits or kindergarten titles like “Keynote speaker” or “Visionary”.

Don’t hire “bulk”/”volume” developers, regardless of your own company size.

Don’t hire anyone over a coffee. Let them be vetted on all levels: Technical competence, product thinking, lateral problem-solving skills, emotional intelligence, previous demonstrated work, and clarity of their own goals. What do you both gain by working together? (not “for you”!) Let them meet the teams and work together for an hour or two before you pull the trigger and commit.

Don’t run a school. Invest in people with solid foundational abilities or suffer the consequences. Conceptual tardiness is probably one of the key reasons teams and organizations never grow to be “great”. For example, teaching continuous integration in an organization in 2023 isn’t setting it up for success — we should expect such knowledge as a basic core competency. The same goes for innovation orgs, where the same goes, but the newness of concepts is radically higher. In short — build an organization from people who are already well-established in the foundational basis of your operations, or else it’s most certainly doomed.

Keep it agile and lean

See if this resonates with you first:

Anyway… You probably don’t really know Agile, but you might have a lot of opinions on it. Everyone seems to have! For some realignment, I’d recommend Robert C. Martin’s Clean Agile: Back to Basics or at least the below video with him presenting on the history of agile and letting that sink in first.

Planning? Go ahead and use something like GitHub projects; whatever is close to the code. Avoid Jira and management-for-managements-sake happening external to the actual money maker — the code.

Scaling? Agile isn’t meant to scale — you need to descale the problem and make it more tangible. Agile isn’t the answer to everything, but if you go to some of the core literature and throw away whatever SAFe stuff you have in the bookshelf, at least you’ll have an actual start. Also, do check Gojko Adzic’s presentation on Descaling Agile. Descale your organization if needed.

Don’t buy into a framework. Start from literally nothing: No meetings, no ceremonies, no anything. Then add a few sprinkles as needed. Avoid ceremonies that add nothing — for example, maybe keep a short 10-minute morning huddle to get the day in order (or at the end of the day!). Kick out the SAFe consultants if you are misfortunate to have them hanging around. Also, look at operational models that make more sense in a fast-moving setting, such as Team Topologies.

Have meetings that “aren’t meetings”. Meetings need to be outcomes-oriented, and therefore should only do one of the following, but nothing else:

• Decide or plan: We need to do certain meta activities as well. As long as they are actionable and everyone has context it’s worth having these. But stay wary of poorly planned planning meetings! :)
• Work: Yep! Work. Setting up an uninterrupted pair/mob working block for a few hours is a great way to use a “meeting that’s not a meeting”.
• Learn or share: Peer learning, competence groups, retrospective…
• Socialize: Shoot the breeze, meet new colleagues, help people find their way…

Use Kanban and its truly no-bullshit do, doing, done model until you know for a fact you need something more elaborate. Use basic local prioritization if necessary, no more (i.e. today’s or the week’s big priorities).

Don’t trust me? Check out Dave Farley’s presentation on Agile and Kanban.

While there are several models with variations on the above, some decisive factors should cut out all unnecessary chaff.

Continuously deliver

“Continuous delivery improves both delivery performance and quality, and also helps improve culture and reduce burnout and deployment pain.”

– Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations

Continuously deliver software. The “minimum viable” model for that requires:

• Trunk-based development
• Work integrates to the trunk at a minimum daily
• Work has automated testing before merging to trunk
• Work is tested with other work automatically on merge
• All feature work stops when the build is red
• New work does not break delivered work

You’d be amazed at this probably not happening wherever you work today, but in a feature factory or software company wannabe setting, you won’t see a lot of the above going on.

And yes, use trunk-based development: It’s the easiest branching strategy of all, is directly tied to continuous delivery, and avoids the meaningless opining around branching. If you think you need complex branching, the problem isn’t tech, it’s the way you pose the situation. Merge hell is not an actual problem when continuously integrating.

Don’t do pull requests unless logically meaningful — do pairing or mobbing instead, whenever possible. Async reviews break the flow and are built on distrust. Make sure people work together and build the right things.

Use a good, versatile off-the-shelf CI/CD platform like GitHub, GitLab, Harness, or Azure DevOps. Don’t run your own Jenkins setup or reinvent this particular wheel. I really can’t think of anything more useless to spend your time building.

Be iterative. Build something that you deliver in some way or form today. Hopefully many times today, many times tomorrow.

Use a single production environment and only ephemeral, short-lived testing environments (if you ever need them!). Don’t make things over-complicated like you’ve learned them in your previous career.

Use serverless technology as a default, and also default to SaaS options — don’t overinvest in what is already readily and cheaply available. It’s safer, cheaper, and easier if you have people who know the cloud. Too many don’t and will complain that serverless is bad. Don’t trust them. Instead, get people who know the cloud.

Use feature toggles once you start needing to segregate feature sets, such as for unfinished features, user groups, beta testers… As Charity Majors writes, “Deploys Are the Wrong Way to Change User Experience”.

Quality

My thesis is this, if an engineering approach to software development doesn’t help us to create better software faster, then it’s wrong and doesn’t qualify as “Engineering”.

— Dave Farley’s weblog, What is Modern Software Engineering?

Write tests, all of them — focusing on unit tests and integration tests. It’s OK to not have 100% coverage on day 1, but don’t let that debt creep in for too many days.

Don’t hire QA engineers, testers, or anything in a similar vein. Expect your engineers to be able to quantify their quality with tests, and make sure they have time to do so.

Learn to use the public cloud. Don’t hire anyone who has no experience in the cloud.

Set tasks for every last bit of technical debt.

Don’t forget legal, security, and other cross-functional requirements. They should help inform your SDLC or minimum internal criteria for the definition of done.

In closing

My attempt here, futile or not, has been to be as frank as I can about the problems that you, as a traditional organization leveraging “IT”, have when you try to become a more software-centric company. In no way is the last word said or written, but I hope that you have an idea of where I’d recommend making some changes, what they look like, and what some of the cultural differences are between an IT org and a true software business.