The Universal Developer™
The troubled parable of Software Delivery Nirvana, or: “Why your business will not be digital-native just because you hire a few developers”
Situation: It’s the year 2020. Every organization scrambles to be “digital-native”, possessing the capability of moving legacy business models and procedures, often based in physical space, to always-available, globally distributed, rel="noopener" target="_blank">This is the context in which the adage “you build it, you run it” was minted at Amazon. Following Gene Kim’s “three ways of DevOps”, teams should aim at automating processes, contextualizing developer choices in reality (ops), and continuously improving their processes.The Swiss-Army Developer
It’s pretty common that most developers (certainly front-end devs) will be fully capable of running code locally, but it’s less common that they have competence in CI/CD, backend deployment and anything else that goes on “behind the curtain”. With the cloud it’s becoming easier than ever to gain access to the full chain of tools needed for committing, coding and deploying almost anything, combined with (mostly) easy integrations from commit to deploy.
In an ideal world an individual developer would be able to exercise full control and insight into the whole set (“stack”) of technologies required to operate a system or application. This is, the developer would be able to produce new value as well as react to unforeseen events equally easily with complete transparency. Anything that happens as an error (or other type of issue) would be automatically mitigated by the system itself, maybe through recipes, rules, schemas, or machine learning inference, which continuously scans the current state. Further, since any system requires assembly in terms of both original source code, pre-existing software and libraries (and their integrations), as well as infrastructure such as compute capacity, storage, and networking, all those concerns would also be elegantly pieced together, both from the initial architecture into actual provisioning and coding without delays, stops or unknowns.
The limits of individual competency
The above does not happen very often, for several reasons. First and foremost is competency: As an example, how many developers have deep knowledge in both current frontend technology and monitoring distributed systems, and also the chops to build machine learning pipelines? With the exodus to the big public hyperscale clouds (like Google Cloud) it’s become easier than ever to get started in new fields and to accumulate skills, but for most companies and individual devs, there is no real reason to explosively grow your skillset across domains. There is more than work in life, after all!
It’s as Mike Pfeiffer wrote recently:
“We need to do a better job of recognizing that the average IT organization does not have the same skillset as the engineering teams at Netflix.” — https://twitter.com/mike_pfeiffer/status/1221909277780017153There’s a few things to read into that. I’d say it’s a bit of validation on what I just wrote myself above. But it’s also telling something about how convenient it’s become for organizations/businesses trying to ride the hype train of DevOps, no-ops, cloud-native development and so forth, while forgetting the reality of the workers: That reality will necessarily involve how an organization can enforce skill-building, how individuals partake in that, how the organization wants to grow and learn, and how it operates from business plan to tangible output. Organizations must support learning, rather than hoping each developer will just magically do it on their own.
If development capability and business goals/plans do not move equally fast, one or the other will be constantly over-producing, and will likely result in people leaving because of frustration, lack of traction, or lack of actual real-life results.
As a manager or BA or similar, trust the team, not the individuals. DevOps as applied to individual persons creates an impossibly high expectation on what that one single person can deliver. It’s not enough to say that “hey we do devops” nor is it enough (in the long run) to do a kind of consultant aquihire either, where you buy your way in if you lack the competency in-house today. Give them the possibility to learn, also by doing mistakes.
Responsibility and trust are obvious factors that are needed to elevate the dev organization closer to a DevOps way-of-working. But equally important is that teams should have a fuller insight and ownership so they can feel that they own both new development work, as well as any bug fixes. Few things are worse for devs than getting angry complaints from management about bugs for features that managers have down-prioritized; thus developers were never the initial root cause of those issues. In a digital-native organization, the ongoing work will never stop, so bugs must be seen as “improvement possibilities” rather than showstoppers, as they often are when an org delivers single big releases. Trust developers to prioritize their own work and help them as a manager by providing smaller tasks continuously. Get developers in early in projects to give them a sense of ownership. Share learnings between dev and business.
Culture shock
The second problem, alluded to above, of getting right into a DevOps way-of-working is the sometimes radical change in culture. This is my highly subjective picture of some key benefits that implementing DevOps brings but also some problems in getting there:
Increased innovation speed
By using microservices, decoupled architectures, or Platforms-as-a-service (or similar), improvement cycle times will be drastically shorter.
This puts stress on management and anyone trying to enforce lock-in platforms (like many integration platforms and CRMs are). Innovation becomes hampered by integration necessities and requiring new greenfield development (that has a fundamentally different operating model) to being in sync with the worse-performing legacy that is sidelined into the greenfield.
Increased product delivery speed
By doing Continuous Deployment instead of big releases, it becomes easier to provide high customer satisfaction “on time”.
Creates sometimes unbearable stress for any managers coming from “big bang” release organizations. Orgs will often think they are “customer-centric” yet are sometimes reluctant to accepting that direct customer improvements through rapid code changes can only happen if customer feedback is heard in the team, and that the team is allowed to work in small batches.
Increased capabilities to own and deploy all layers of a system
Teams should have wider, or even full, possibility to deploy anything from databases to APIs to front-end applications to own and maintain their solution.
Depending on how the company is structured, this can be complicated if there are lots of groups or people with overlapping mandates or responsibilities to deploy and monitor infrastructure and other resources. The development org may need to rethink how they work with cloud segmentation through IAM (access management) and how teams can own their part of the overall pie.
Increased insight to support decreased failure rate
Let teams have full control of monitoring and other accesses to system health so they can effectively fix errors and other problems in their daily work.
This is similar to the above: IT (or similar) needs to have a very clear idea of who gets access to what.
Quality is regulated through guardrails rather than through gate-keeping
Provide baseline implementations, cloud-backed guardrails, and readable documents that show patterns that conform to quality and security benchmarks. Ensure high visibility of these, and make it possible for teams to adjust and improve on them.
It’s not unheard that senior staff, like architects or engineers, are babysitting or guarding parts of the system. If you want developers to be able to do things on their owns, there needs to be clear documentation and requirements to form the basis of how to do things right. Then those can be improved, automated and shared like boilerplates, benchmarks or guardrails like AWS’s Service Control Policies (which, again, require good segmentation of devs, groups, accounts, projects…).
I don’t have multiple decades of experience in this industry (yet), but let me be frank that I’ve never seen anything similar to DevOps work well unless the organization around it—business, marketing, and so on—also adapts accordingly. DevOps culture should be used as a way of easing into that vaunted broader insight, by naturally opening interfaces for developers to witness their work in context. With agile principles and lean, many product teams got closer to requirements and the business side. The impact has been that those are less abstract, strange and “from the top” than in classical IT organizations with waterfall principles.
“Since the early days of our Technology Radar, we’ve warned against tools and techniques that isolate members of software teams from one another, hampering feedback and collaboration. Often, when new specializations come along, practitioners, vendors and tools insist that some part of development must be done in an isolated environment, away from the chaos of “regular” development. We reject that claim and constantly look for new ways to reengage software development as a team sport. Feedback is critical when developing something as complex as software. While projects increasingly require specialization, we strive to fit them into regular collaboration and feedback […] (where areas are) integrated into cross-functional teams and supported with solid automation. The next frontier is bringing more governance and compliance activities into the fold.”
— ThoughtWorks, “Software Development as a Team Sport” from Technology Radar #21
When working with Polestar, I was one of the founding members of the DevOps/architecture/platform team responsible for quality, processes and guidance for—among others—the numerous product teams. We have let the teams have almost complete freedom when it comes to interpreting or working with the assignments and architectural recommendations, but have usually been involved in validating their solutions, and transforming them into reproducible infrastructure-as-code assets in Terraform. While they are indeed in complete control of their APIs and backend functions, they are constrained in terms of other resources. We are now looking into ways for devs to more easily (and safely) step into provisioning their own infrastructure, with only guardrails and no manual intervention from us.
This very brief example shows how a small degree of centralized governance can balance autonomy with architectural and enterprise adherence. For sure even our model is under constant scrutiny and improvement, but it is a lot faster than legacy IT orgs while still retaining quality.
Personally, I believe that DevOps does not by itself create any transformation: It is a way of working that can only have any real value or place when the organization can leverage what DevOps can enhance—speed of delivery, innovation rate, increasing degree of work satisfaction through insight and ownership, as some examples.
All in all, there is really no good reason in 2020 to not have product teams or IT organizations working according to agile principles with a DevOps mindset. Just don’t forget that if your digital output truly matters, it will cause ripples everywhere else in your workplace and that it’s not “just another tech thing”. I hope the above is not scaring anybody off—on the contrary I hope it just highlights that becoming a digital leader today is a steep incline that technology is not the answer to: To deliver tech, you must solve the people and process problems first.