You Sound Like an Idiot When You Call Software Engineering “IT”

Save your business by distinguishing between baseline IT and making novel software.

As in all fields, there are heretical scriptures even in the paper-dry world of IT. One of these is Nicholas Carr’s “IT Doesn’t Matter”.

To put the article in context, it was published in 2003 as a short piece in Harvard Business Review and immediately sparked huge controversy for claiming, among other things, that:

• It’s time to cut back on IT spending: After years of substantial IT investments, he said the naughty thing that riled up countless vendors—it’s time to reduce spending on IT. Just imagine Steve “Developers Developers Developers” Ballmer, but angry.
• IT is a commodity: Carr claimed that IT had become commoditized, meaning it was widely accessible and no longer provided a competitive advantage simply by having more of it.
• Generic software replaces custom development: Software would become more standardized and capable of meeting business needs without extensive custom development.

I highly recommend reading the PDF version which also includes some very interesting letters to the editor.

The main point of my article is that he, as so many others, is muddling up software development capabilities—which aim at generating new value—with generic or “baseline” IT, which is basic hygiene, i.e. enabling the business to function at all.

That’s to say:

If you create (or aim to create) value through novel software, then that’s what you have to make and support. And that software creation has nothing to do with “IT”. Ergo, the person fixing your printer is probably not the person creating the foundations for earning your next billion dollars. No offense meant.

I’m going to pick one long quote from his text that we’ll use as a backdrop for the present article. It goes like this:

Myriad other companies have gained important advantages through the innovative deployment of IT. Some, like American Airlines with its Sabre reservation system, Federal Express with its package-tracking system, and Mobil Oil with its automated Speedpass payment system, used IT to gain particular operating or marketing advantages — to leapfrog the competition in one process or activity. Others, like Reuters with its 1970s financial information network or, more recently, eBay with its Internet auctions, had superior insight into the way IT would fundamentally change an industry and were able to stake out commanding positions. […]

But the opportunities for gaining IT-based advantages are already dwindling. Best practices are now quickly built into software or otherwise replicated. And as for IT-spurred industry transformations, most of the ones that are going to happen have likely already happened or are in the process of happening. Industries and markets will continue to evolve, of course, and some will undergo fundamental changes — the future of the music business, for example, continues to be in doubt. But history shows that the power of an infrastructural technology to transform industries always diminishes as its buildout nears completion.
— Nicholas Carr, “IT Doesn’t Matter”, Harvard Business Review magazine, May (2003)

My gripe is that there is a lack, not just in his article, but in general understanding—even 20-odd years later—that IT is not the same thing as a software engineering capability in an organisation. That’s to say that the IT department—often providing primarily communication tools and company hardware assets—is functionally completely different from a capability that delivers software (hopefully novel, new software). Sounds obvious? Should be, but reality says otherwise.

Back to Carr’s article. First, let’s assert that all software is built and running on some kind of infrastructure (hardware). You’ll likely have noticed that the world got more, not less, software after the extreme industrialization and commoditization of infrastructure, from virtual private servers through the cloud to serverless and then beyond. The same goes for open-source software — practically all software uses some degree of open-source software to deliver new value. Commoditization of software did not limit the production of software—it led to more software.

All of Carr’s examples of companies, and probably most of those more recent cases you can think of, all boil down to technology being outright created or at least reconstituted and deployed (in the physical sense) in a novel way. The technology itself was not configured and ready to use for these new value-generating ways out of the box: Their possibilities were latent but not exploited (until they actually were). So, simply buying new racks loaded with servers, getting MacBooks, or committing to bucketfuls of Slack licenses won’t magically transform your business. Unfortunately, this same delusion lives on in various forms today, as seen from my own experience: “Hey, let’s cosplay Silicon Valley!” 🤡

A decade later, Robert Plant came close to specifying the root cause of this problem:

IT-related problems aren’t rooted in technology but in leadership failings. The people in the C-suite don’t understand IT problems, don’t provide adequate resources to solve them, and don’t approach the issues as members of unified technology-literate teams.
 — Robert Plant, “IT Doesn’t Matter (to CEOs)”, Harvard Business Review online, August 15 (2013)

I sincerely believe it’s too vague and lazy to simply blame the management—even if, frankly, they are probably rightfully being blamed. The more action-oriented approach would instead be to delineate in the organization these very different capabilities into separate arms and mobilize resources to make them flourish. What I’d do to have such great powers…

Finding the words

What, then, is a better term for this software-making capability, to distinguish it from more generic IT? Let’s just call it what it is: Software engineering.

Baseline IT (trad. “IT”) focuses on maintaining, among other possible things, hardware, networks, and communication tools. Their primary role is to ensure the reliability, security, and efficiency of organization-wide generic technical solutions. They enable the business to function as intended but do not drive the business intent. Frameworks like ITIL and ITSM could be perfectly fine here, offering conventional, formal guidelines for IT governance, service management, and operational efficiency.

The value of baseline IT is primarily internal and is therefore quite different from the external value provided by a software engineering arm. While it’s still critical to the day-to-day functioning of most organizations that there is a functioning network, printers, and good sets of hardware distributed to the staff, none of this constitutes any form of competitive advantage. If anything, these aspects are no more than table stakes.

On the other hand, an organizational software engineering capability revolves around creating and launching novel software products. Creating novel value through software drives business value (all or some of it). The best fit here tends to be some agile variant, as these powerfully emphasize and support flexibility, iterative development, and collaboration.

The stricter frameworks that may make sense for baseline IT are directly counter-productive for software engineering teams and their circumstances.

A hybrid between these two is created when you set up supporting functions like platform teams, as they indeed do create software, but as a value-add to the software engineering arm to expedite their work. This should therefore be seen resolutely as part of the mobilization of a software engineering capability, rather than as a baseline IT function.

A basic definition of business and value

First, I’ll share my trivial view of business (and I’m not an economist so sorry about that) so that we can agree (or not) on the basic premises.

A business:

• Has some kind of value (service, product, right…) that it can provide;
• Has the means (processes, assets, IP, staff, capital…) to reproduce said value;
• Can reproduce the value;
• Can provide that value to a market;
• Can be packaged in a format that contains everything needed to reproduce that value, e.g. processes, IP, patents, standard operating procedures, etc.

Value is contingent and variant, and not absolute. Value is contingent partly because the perceived value and the production of it hinges on uncontrollable, even unknowable, factors outside the realm of the business (e.g. global politics), as well as in the internal constitution and stability (e.g. company culture) of the business that produces the value.

IT and technology doesn’t have any intrinsic value. Like anything else in the context of business, they aren’t strictly needed unless they can be exploited to extract a form of value that you can trade. This is not specific to tech: It’s the same story with any other good, asset, or bit of knowledge.

Technology offers possibilities to provide and/or create value in richer and/or more efficient ways, and this is why it’s important to organizations. Because technology progresses fast and independently of any given organization, it’s a constantly changing field of opportunities to derive new value from—convenient, since someone else is paying for the heavy lifting while your job is to find the nuggets of gold. Thus, you have an opportunity to effectively exploit and extract value from technology that is not of your own making. In the market, novel or unique value tends to command greater attention, power, and earnings.

Technology-assisted capabilities in business

Many organizations seem able to comprehend the generic exploitable abilities of IT and technology. We could speak of these in increasing order of “difficulty levels”. I do believe that they follow mostly linearly (1–2–3…) but that an organization can have a variable ability to execute on each independent capability.

These four general capabilities could be stated as…

1. Communicate: “Easy mode” 👶; where you provide means of communicating one-way and two-way. No organization can exist without communication.
2. Systematize: “Medium mode” 🗃️; where you turn papers or other knowledge into digits, store the data in relevant tools and systems, make the data accessible and collaboration-enabled, and map processes from minds to replicable formats. This is the first step of moving human knowledge into technological means or “records”.
3. Automate: “Hard mode” 🤖; where you put computers to work, making the data and processes you’ve systematized into a leveraged ecosystem that allows you to produce vastly more with vastly fewer resources. This is where the labor is entirely, or mostly, done by machines rather than by human workers.
4. Innovate: Now you’re just making it hard on yourself 🧪☠️; where you use new and existing tools and processes to identify and derive new value that can be exploited. This is where you, for example, create new value streams or can operate existing value streams at a materially lower cost.

Or in the parlance of Doom, the classic first-person shooter from 1993:

The first three capabilities are well-known, industrialized facts. The communicate capability is widely adopted, likely because it operates on a simple and known set of metaphors: Letters and the telephone. The same goes for video chat. Practically everyone should be able to understand these concepts from lived experience.

Conventional “baseline IT” offers clear, known ways to provide the foundations for the capabilities communicate, systematize, and automate. Depending on the needs around automation, you could quite easily cover that entire capability with commercially available tools. Nicholas Carr got this, too, quite right.

Wardley Maps are great at plotting out our thinking in a visual way, to understand the value chain involved in a given scenario. Extremely briefly, the Y-axis tells how visible the perceived value is to the user/actor/customer: The higher, the more visible. But, alas, everything is connected. Your online photo storage requires a web site, CRM, platform, compute… so much!

Now look at the bottom (X-axis) line. It shows how evolved something is. If it’s truly volatile, unique, special, speculative (and so on), then it’s to the far left. And the further right you go, the more common, utility-oriented, and off-the-shelf it becomes. “Power” is a good example in the above diagram.

Imagine how we could plot out the first three levels from far right (off-the-shelf commodity) to left, and for the automation solutions, probably stopping somewhere in the range of the product stage or a tiny bit into the custom-built evolution. Remember that being lower and further to the right is not a bad thing, it just tells that the value is further away from the explicit need, and that it’s more commodified than something further to the left. You should be getting signals that the further right something is, the more it makes sense to “buy” rather than “build”, consider outsourcing or renting, etc. Conversely, the further left you go, the more you are losing by not “owning” and centering on that ability, as it’s increasingly singular.

Leaving “easy mode” and welcoming software engineering

So that’s the easy stuff done and dusted.

Why systematize and automate are significantly harder than communicate is because they are knowledge-intensive, and to be frank, organizations are not commonly built around cognitive ability and labor — many times they are built, for example, around a manufacturing ability, which is a wholly different set of requirements, culture, needs, and expectations.

Think of any place of (relevant) work you’ve ever been at: Has everything been truly and systematically documented and formalized? Has everything that made sense to automate really been automated? Has there ever been problems when Person X left, and you all understood that “shit, we don’t know how we are going to deal with this?”

In fact, this reality check tells us vividly about one of the great failures of the digitization and digitalization “revolution” that began in the 90s: The belief that we could make legacy organizations—for example in the extremely physical manufacturing industry—into knowledge organizations by simply throwing in some desktops in the middle manager offices and some more PLC hardware out on the factory floor. This is part of why I claim that digitalization is still nowhere near done, even after 30 years of it unfolding. We must continue this work.

One way of starting to frame the problem is to get acquainted with the Cynefin framework, so we can place our problem domain into a “box” from which we can reflect on its properties. For example, the problem space of baseline IT should be mostly clear.

The majority of problems and decisions should revolve around milquetoast questions such as what the best tool for the job is, how they compare on feature sets and price, and so on. I don’t mean to trivialize this work at all. There are lots of people able to work with this type of problem space and you can find any number of strategies and tools to work adeptly within the area, utilizing reviews, trials, in-house tests and opinion data, comparison sheets, Gartner/Forrester research, and much more to solve typical problems in that space. The risk for unholy complication should be universally minimal here, is the point.

Now, software development capabilities that aim at generating new value are instead much more typically a complicated, or sometimes even a complex, problem space; You can trade-off-analyze yourself to death just over details (note: not worth it). Technology and innovation are both emergent and cannot readily be boxed in terms of time, space, money, and what it will do, as this is perhaps not even known at the time. The number of unknown factors that are in play can be staggering (though this is not always the case!). This fact only becomes truer the further you push into uncharted territory, and less true the more conventionally you use technology and the more conventional the tech itself is. Using Office 365 won’t ever stand as an act of rebellious innovation, nor is there a necessary innovation implication in writing a few hundred lines of Python code to massage some scraps of data. Ergo, code itself is not a guarantee of innovation.

Knowledge as core of software

So, why is software so complicated, then? It’s a question of many dimensions, but for this article, my angle is that software is always predicated on some sort of formalized and materialized knowledge. Phillip Armour wrote succinctly about that particular angle almost 25 years ago:

If software is not a product, then what is the product of our efforts to produce software? It is the knowledge contained in the software. It’s rather easy to produce software. It’s much more difficult to produce software that works, because we have to understand the meaning of “works.” It’s easy to produce simple software because it doesn’t contain much knowledge. Software is easier to produce using an application generator, because much of the knowledge is already stored in the application generator. Software is easy to produce if I’ve already produced this type of system before, because I have already obtained the necessary knowledge.

So, the hard part of building systems is not building them, it’s knowing what to build — it’s in acquiring the necessary knowledge. This leads us to another observation: if software is not a product but a medium for storing knowledge, then software development is not a product-producing activity, it is a knowledge-acquiring activity.
 — Phillip G. Armour, “The Five Orders of Ignorance”, Communications of the ACM, Oct 1 (2000)

It should be quite evident here that also Armour seems to subscribe implicitly to the view that some degree of useful software can be suitably produced by “application generators”, or what I expect maps more or less to Carr’s notion of “generic software”. However, if the knowledge is unique — to our team, at the very least — then capturing it and agreeing on what it is, becomes the constituent problem to solve.

You can also see this notion in much newer directions, like in the Manifesto for Knowledge-Centric Software Development:

Also consider seeing Kevlin Henney’s presentation on this theme:

Let’s take a practical example: Why’s Google still around after all these years if they perfected their mythical “search algorithm” (if such a thing even exists today) way back when? It’s because they are much more than that; they are everything from video conferences to smart cities to digital maps. The notion of Google as a simple holistic whole — based on a single algorithm — does not embed all the work and IP and patents of the actual, real corporation called Google. It does not account for all the innovation they’ve done (and are doing) nor the countless things they’ve failed at or put in their graveyard. You also can’t forget how different the collaboration models at software (or really, tech innovation companies) can be; see for example Valve. These companies are continuously searching for value and leveraging technology and collaboration models to find it — this work doesn’t just stop. They’ve learned something and moved on. Many times.

Again: Knowledge in organization A may be completely useless to organization B. Knowledge is not necessarily transferable, so I posit that it has at least additional social and contextual dimensions; Knowledge has no intrinsic value. Any value can only come through exploiting it, which requires necessary resources to be deployed, as well as skilled staff.

Innovation is also harder than other work because it’s something not everyone has actively worked with, in the concrete sense of that word. Further, there is a lack of metaphors and useful models when operating a knowledge-centric IT organization, because few people have a lived, shared reality of similar meaningful metaphors. Perhaps those with deep university research or advanced R&D have some idea, but it’s not peddled as part of the regular IT consultancy way of working.

Next, by definition innovation is discriminatory:

• It takes technical skills that few have;
• It takes communication skills that few have;
• It takes analytical and deductive skills that few have;
• It takes an aptitude for knowledge-seeking that is not common among all individuals;
• It takes acceptance of risks and uncertainty (personal and organizational) to a degree that is uncommon.

These skills aren’t readily taught (if all can even be taught), nor are the “soft sides” of software engineering very prevalent in the majority of workplaces, thus not letting engineers build gradually layered experience.

Nothing about innovation and deploying technology for new value screams that it’s something every organization can, wants, or should do. Not every organization can foster and support such conditions. Nor do they necessarily have people to do the work, nor is it guaranteed they will be able to attract those who are able to do it.

You don’t have to read studies, such as “A comprehensive overview of software product management challenges” by Springer & Muler (2022) to know that most traditional organizations are notoriously poor at product management (which I’d say begets innovation). Just ask people for an honest opinion in your own organization!

The data is certainly discouraging…

The 5th ranked problem even explicitly calls out reactive work taking precedence over innovation work!

Read the full paper at:

The above data provides some evidence for one of the key reasons I believe software doesn’t work as a non-dysfunctional part of organizations:

Because knowledge-centricity must fundamentally change how work is conducted and the organization is constructed.

This same experience is what you may have from seeing why, for example, agile frameworks are not working. While the agile framework can be picked as a box-ready product and be “rolled out” by certified consultants, it’s not so uncommonly plainly degraded to a ritualized corporate circus sideshow. Of course it fails!

Indeed, that fact has also been recorded by Lugnet, Ericson, and Larsson. Moving to an actual agile model is challenging, and they conclude their paper by summarizing why that is:

[T]here might be a lack of approval, not only from top managers but also from critical peers. Second, the organizational culture has to be prepared to allow self-organization, trust, and different leadership styles that truly support teamwork. Third, there might be a lack of company- specific success factors to support agile methods. In its current state, the theory mainly suggests that agile methods are useful, but there is limited support for how to fully implement them, i.e., how to be agile and how to manage resistance to change in project work. Only the latter perspective is addressed by, e.g., psychology.
— Johan Lugnet, Åsa Ericson, Andreas Larsson: “Realization of Agile Methods in Established Processes: Challenges and Barriers” (2021), Applied Sciences, February 2021

In the agile world this leads to the distinction of “being agile” versus “doing agile”. Correspondingly, this is where I draw the line between “making software” and “cosplaying software” (or Silicon Valley). Real effort is needed to do this stuff.

Do you need software engineers?

First: The question is not “Do I need software?” as the answer is yes.

Next: This is not a pride or money question. It really isn’t.

As the unnamed man in Swedish filmmaker Roy Andersson’s fantastic World of Glory plainly states:

There are many who want an apartment like this. Especially youths. But… they don’t have any money. And if there is no money, then there will be no apartment. (Medium author’s translation from Swedish)
— Unnamed man, “World of Glory” (1991; Roy Andersson)

For these simple examples, it’s worth stating that I’m trying to make a point, not cover all angles. In reality, depending on our context and the complexity of your reality, you want to consider a much richer set of trade-offs and constraints. And there are certainly many more constraints than the usual money shortage! One of these constraints is, as I’ve mentioned, if you have the organizational setup to actually provide value (and make money) from a software engineering arm.

Anyway, the very simple cases.

Case 1: Transactional emails

Context: For your SaaS service you need to send an assortment of different transactional emails to people, for example when signing up, canceling appointments, being late with payments, etc.

Tech stability: Very stable.

Business value: Medium.

Who owns the work? IT.

Short rationale: This is a must-have, but it’s also heavily generic. As a capability, it fills an important need for day-to-day needs, but you don’t have to go reinvent the wheel. Buy a service with the characteristics you need (such as API support) and make it available through your organization to your internal users.

Case 2: Website

Context: The charity you represent needs a website with fairly common features, but the majority is static text and images.

Tech stability: Stable.

Business value: Very high.

Who owns the work? In this case, the equivalent of IT (i.e. using off-the-shelf products, tools, templates). In other, bigger cases, it may be your software development group/department, depending on strategy and needs.

Short rationale: In this case, a realistic choice could be something as simple as a modern templatized site builder like Carrd or something more elaborate like Webflow. A driving factor here is the level of uniqueness and nature of the features required. While even simple website projects can get botched, the level of unknownness is quite limited and the risk is manageable.

Case 3: Bi-directional charging for cars

Context: You’re part of a car company and need to figure out how to build and operate (including monetizing) a solution for bi-directional charging.

Tech stability: Low.

Business value: Unknown, speculative, identified as potentially high.

Who owns the work? Your software development group/department.

Short rationale: This one is simple, as this case requires many layers of collaboration (hardware, R&D, software, operations, business development…) and since there exists no complete (and adapted) solution you quite literally have to build new things to make this idea even remotely viable. Going in, you just don’t know the full scope, but as they say “high risk, high reward” (maybe…)

Summary

So, what do I think of Carr’s article? Long story short: I think he’s right in all three paraphrased aspects, but maybe not always in his reasoning.

• Cut costs: I’m too young to have the full IT cost management context that Carr referred to, but I’m well aware of this being important in a contemporary context and why that is. In fact, more should be done here (FinOps, frugal architecture, and so on, even if these lean into the software-making side).
• IT commoditization: Agreed. I can’t see that “IT” itself was an advantage unless you go way back in time. Everyone has access to the same basic tools and hardware, now more than ever.
• Generic software over custom: I think it’s highly likely that forming and conforming (internal) business processes around standards, and to some extent tools, can be beneficial in terms of innovation ability, cycle time, and onboarding of new staff. Don’t die from Unique Snowflake Syndrome. As I’ve written before, I’m entirely convinced new paradigms of making software (including low-code/no-code tools) are part of making “generic software” replace the need for many types of traditional, expensive-to-make custom software.

In this article, I’ve attempted to semantically distinguish between baseline IT, which focuses on maintaining essential business operations, and software engineering, which aims to create novel software for new business value. I’ve pointed out that misunderstanding and conflating these fields leads to missed opportunities for innovation. Lastly, organizations need to clearly separate and understand these distinct capabilities to maximize their potential, and I’ve tried to clarify why it is hard for traditional organizations to actually do this in reality.

Many organizations would be better off to understand their generic nature of IT and embrace that, unless they want to make a committed move and become knowledge-centric software engineering organizations.

If you liked this article, you might also like my other writings, such as: