Member post originally published on the Syntasso blog by Cat Morris
While building an internal developer platform sounds like something an engineering organisation would do – and often tries to do – from scratch, the reality is, most are working with a complex web of legacy systems and tools.
Organisations often turn to platform engineering to alleviate inefficiencies in or limitations of existing systems. This means that most of the time, a platform initiative isn’t a greenfield endeavour, and it isn’t in most cases realistic to build something brand new or replace existing solutions wholesale.
Platforms evolve from the responses to challenges identified in existing systems, making most such projects brownfield in nature – and this can, in fact, be advantageous.
As I shared in my PlatformCon 2024 talk, “The Power of a Brownfield Mindset when Building Your Platforms“, I’ve learned a number of lessons in my years working with developer platforms in brownfield environments, and this article will cover five of the key realisations I’ve had in building successful platforms:
- Lesson 1: Most platforms are brownfield
- Lesson 2: Adapt platform requirements for complexity and scale
- Lesson 3: Platforms are emergent
- Lesson 4: Change how you think about MVPs
- Lesson 5: Build a sense of platform ownership
Lesson 1: Most platforms are brownfield
Platforms emerge from what already exists. I’ve been on a number of platform development journeys with different organisations, and almost invariably, I’ve discovered that most platforms are brownfield. In the software development space, “brownfield development” describes areas that need the development and deployment of new software systems and solutions alongside existing software applications and systems.
The term “brownfield” was borrowed from civil engineering where brownfield developments require you to work with or around existing structures, environments and hazards. And this is where the majority of platform development takes place. It flies in the face of conventional greenfield thinking, which imagines that platforms can be developed as brand-new, standalone tools to help developers ship software consistently, swiftly and more safely.
The greenfield way of thinking doesn’t account for the reality facing most organisations: the considerable footprint of existing systems and legacy software around which most platforms will end up being built. And most platforms aren’t built from scratch and brought in to replace whole systems in their entirety. Instead, the old and new co-exist. I’ve routinely supported organisations in adapting their platform development “to meet the existing complexities of the legacy enterprise and its infrastructure and systems.” Starting with a brownfield mindset and planning accordingly creates more realistic expectations and conditions for building a platform that will actually serve the needs of its users.
Lesson 2: Adapt platform requirements for complexity and scale
As an example of my own experience, I worked as a part of a consulting team helping a financial company/neobank build their developer platform. The company had a DevOps team that struggled to scale the necessary governance and security checks across teams. As the needs of the business grew, DevOps could not hire people fast enough to meet the new demands. They saw platform engineering as an efficient way to meet those needs and offload the DevOps burden without having to double or triple the size of their team.
This isn’t a unique or even unusual situation. Many organisations balance growing scalability, compliance and governance demands against legacy infrastructure and systems that won’t fit neatly into a shiny new platform. Naturally, this introduces unintentional layers of complexity and is an automatic ticket to brownfield development. What becomes important in these cases is figuring out how to work with what you have alongside mapping out what you actually need. What actually needs to get done, and how can you make this happen with what you have and what you can adopt and adapt?
To illustrate the complexity in real terms, when I worked with the neobank to create their platform, we started mapping the processes for what we wanted to build. We went in enthusiastically, thinking we could begin with a lightweight, minimum viable product (MVP), add a bit of observability, deploy one service and include a few governance checks. Digging a little deeper, we discovered that there were entrenched interests we would need to consider first. For example, the team had no capacity to do anything new. Also, each of the development teams followed slightly different processes and would not or could not change what they were doing. For example, some teams were using obscure load-balancing features and would not migrate to a new platform without bringing those along. Another had super admin permissions to all the databases and used them daily, meaning we couldn’t get rid of them.
The bottom line: We spent three months trying to free up DevOps capacity by automating existing systems and processes before even beginning to tackle platform building. And we realised we are not actually building something new (see “Most platforms are brownfield” above). The organisation didn’t have a platform or platform team, but what it did have was a host of existing ways of doing things a platform was going to do and embedded preferences and a lot of accompanying complexity. We also could see that continuing what we were doing would lead to the all-too-common and almost comical problem of multiple competing platforms all trying to accomplish the same things.
We needed to rethink and adapt platform requirements to accommodate this complexity and legacy while still scaling up. At the same time, we needed to create something fit for purpose (ideally, we thought, a universal platform that would work for everyone) and not just another thing that we would have to support and maintain without the payoff of actually delivering value for its users.
Lesson 3: Platforms are emergent
Related to the idea that most platforms are not built as blank canvases, one key lesson is that platforms are emergent. They emerge from a need to scale and a need for consistency in a complex environment. And what they emerge from is stuff that already exists. What this means is that developers are already deploying codes onto environments that already exist. This code is being maintained somewhere. Errors are already being found and fixed in production. Accounts are being configured and managed. All of this is already happening now, and the need for a unifying platform emerges from these circumstances – potentially highlighting the chaos and inconsistency (and the need for scalability) that necessitates a platform.
With the neobank’s platform project, we reached a critical mass of services and necessary business capabilities that needed to be supported by someone but were unique to the organisation, so it was not possible to just buy a platform off the shelf. Keeping in mind that a big part of scaling up hinges on getting the product into the hands of customers – fast, the real needs of the platform take shape. Consider that once an organisation starts putting products out there, the products are out there. They can’t be reeled back in. With that in mind, it’s clear that the initial processes and implementations needed to be supported or migrated to the new platform, which is where working around existing structures, environments and hazards becomes important.
Lesson 4: Change how you think about MVPs
Because of the brownfield, emergent nature of platforms, we need to change how we think about MVPs. The common way we think about building an MVP is often through a lens that doesn’t take real-world considerations into account. For example, if we are building a car, we might start with a skateboard that becomes a scooter that becomes a bike, which eventually becomes a car. With an MVP, we may be testing our iterations in perfect conditions – a beautiful day, clear road, pothole-free, no traffic. But with a platform, you’re driving into a veritable spaghetti junction overloaded with cars driving in the wrong lane at speed, trying to merge into the right lane and get to where you want to go on time.
What this tells us is that the standard approach to building an MVP isn’t going to cut it. They just don’t work in the platform context. But why? One explanation is loss aversion.
Daniel Kahneman and Amos Tversky published a paper on prospect theory and loss aversion that showed that when offered two options, people choose a guaranteed but lower amount of cash over a higher amount at a lower probability of winning. To illustrate, a person could choose between either a 99% chance of winning $1,400 and a 1% chance of winning nothing OR a guaranteed $900. The bias toward avoidance of loss (also detailed in Kahneman’s book Thinking, Fast and Slow) is significant.
In the context of platform building, we draw on the idea of the pain of loss versus positive gains. What is the smallest gain that would need to be made to balance out an equal chance of loss? More specifically, for platform builders, even if people don’t really use or like a feature, they still feel the pain of giving it up more than any potential gain they get from the new platform. If we frame this again using our MVP example, it is like building a bike that may do everything your users need and have other benefits as well – but users don’t want to give up the heated seats in their car, even though they never turn the seat heating on.
Instead, we need to reframe what we are doing and focus on Tony Ulwick’s jobs-to-be-done theory, which is an approach to innovation that argues that customers buy products and services to get a specific job or jobs done. This helps companies streamline the innovation process by uncovering the true purpose of a product or service. In our platform, that means understanding all the steps needed for users to get their job done. The important takeaway for us is that we want to build the platform to allow a user to complete one job from beginning to end. This means that all the capabilities needed to do one thing in its entirety, from configuration to integration checks and rules and so on, need to be included. We create in this way more incrementally than if we tried to build a whole platform – but we are still thinking end to end, only focusing on a smaller but complete task.
Lesson five: Build a sense of platform ownership
Even once you have built a platform that should be embraced and used widely, there will be people who will continue to use all the tools they used before. Partly, this is attributable to loss aversion and partly to the endowment principle, which dictates that people tend to perceive what they already have as more valuable than something similar that they don’t have. For example, a brand-loyal iPhone user believes that the iPhone is inherently more valuable than a comparable Android phone.
In terms of our platform, users will see what they already have or built as more valuable than the new platform, regardless of whether this is objectively true. Organisations investing in platforms can avoid this by helping users see the platform as something they own – not as competition for their existing solution. Users can embrace both the old and the new ways of getting their job done, but by fostering a sense of platform ownership, the organisation can get users on board and provide active ways for users to contribute to and make the platform better, such as pathways for giving feedback, ways to contribute code or resources and active platform support.
In a brownfield world, we are building a platform with live systems and processes that need to be supportable in the long term. The platform has to scale exponentially, which is where fleet management can come in, that is, your platform can manage all of your services as one unit, upgrading them seamlessly and automatically. If the platform manages these things, it builds user trust and develops the sense of ownership we’re looking for.
Five brownfield lessons: Working with what you have
Building a platform is typically a brownfield exercise. By the time an organisation is intentionally looking to build a platform, developers are already shipping applications to production and all the mechanisms that support that are already in place. Serve these users and build trust in a platform by taking this into account and eschewing the standard approach to MVP development. Think about the jobs to be done and create a thin but complete solution for one discrete job, and invite users to contribute and give feedback to the platform to make it both more valuable and successful in the long run.