Guest post originally published on Mia-Platform’s blog by the Mia-Platform team
The world of software development is constantly and rapidly changing. New technologies emerge every day to solve problems that slow down the daily work of IT departments. So while innovation is a positive factor in that it helps simplify work, there is also a proliferation of new tools and services that need to be managed by developers, thus increasing the cognitive load needed to develop and release software. Internal Developer Platforms (IDPs) aim to solve this problem and simplify the use of tools and services by providing a single point of access and interaction. Those familiar with the topic of Platform Engineering will have heard this statement before but not always is clear which tools are orchestrated by the Internal Developer Platforms.
With this article we want to answer this question, using Mia‑Platform’s experience over the years. In this way, we want to provide a term of comparison for those already involved in Platform Engineering. In this article, we will outline 7 components that we believe are indispensable in an Internal Developer Platform. For each one we will show the real underlying problem, and how the component responds to it.
7 components of Internal Developer Platforms
Before we begin, however, a few brief premises should be made. First, these 7 components do not cover all the capabilities that an IDP can offer, and the order in which we will mention them does not necessarily represent their importance. In addition, we have chosen to mention only 7 components for the sake of article length, not because these are necessarily the most important in every IDP. The very choice of these 7 components may not apply in all cases, where perhaps others are more relevant, e.g., to the target business or the context of the organization.
However, we have experienced that these 7 components have proven to be crucial in most cases.
Pipelines
Since we are talking about software development, attention should initially be directed toward the code. Taking for granted the practice of versioning code, which is now a de facto standard, the next step concerns pipeline management. Once the code has been written and versioned, developers need to automate the processes of validating the code itself, and possibly even its release. Pipelines first and foremost meet this need.
There are several tools for pipeline management. To mention just a few of the most widely used, we find Azure Pipeline, GitHub Actions, GitLab CI Runner, Jenkins.
Artifact Repository
Another important issue involves artifact management. Having versioned images of already compiled, containerized, ready‑to‑use code available is very important for governance and security reasons, as well as providing a quick fallback in case of downtime.
Examples of tools for versioning and saving containerized artifacts are containerd, CRI‑O, Docker Engine, Dragonfly, Harbor, Nexus.
Runtime
Taking further steps in the software lifecycle, without dwelling on very important topics such as Continuous Deployment, we come to its heart, namely the runtime phase. For the purpose of this article, we consider the runtime phase in its entirety, thus including security, permission management, networking, etc.
Considering its broadness, for the runtime phase we find a large number of tools, each dedicated to a particular aspect. For the orchestration of containerized applications, one cannot fail to mention Kubernetes, which can now be considered the de facto standard. In particular, its autoscaling, self‑healing, and load-balancing capabilities (not to mention the many others it guarantees) make Kubernetes one of the best tools for scaling the runtime.
To ensure security, a noteworthy tool is Rönd, an open‑source project developed and maintained by Mia-Platform. Rönd is a container sidecar designed to protect APIs through simple security policies. It also allows roles and permissions to be created and managed using an RBAC/ABAC approach.
In addition to these two tools, we also mention Cilium for networking and Falco for security.
API Gateway / Service Proxy
Another essential component that ensures the products and services can communicate with the outside world is the API Gateway. Depending on the type of product and its use, we also include in this section Service Proxies, which can be implemented for a similar purpose.
To further explore this topic, please refer to this video. Some of the most widely used tools in the Service Proxy and API Gateway category are Gloo Edge, Emissary Ingress, Envoy, NGINX, and Traefik.
Monitoring
One of the most important aspects of the software life cycle is related to monitoring and log tracking. Indeed, monitoring is crucial to know the status of systems so that proactive action can be taken to resolve problems and ensure service continuity. By collecting metrics, displaying them in dedicated dashboards, and triggering alerts, monitoring tools help development teams identify and resolve critical issues. In the context of microservices and distributed applications, collecting logs can be complex but is still a crucial part of successful projects.
Among the most widely used tools for ensuring software observability are Elasticsearch, Fluentd, Grafana, Kibana, and Prometheus.
FinOps and Sustainability
Going beyond the realm of development in the narrow sense, two very important and closely related topics are FinOps and sustainability. The main need from which the FinOps discipline arose is to manage cloud-related operations from a financial perspective. Controlling and optimizing cloud costs also brings sustainability benefits.
Sustainability, primarily environmental but not limited to this aspect, should not only be a benefit that is gained reflexively from other optimization activities but should be seen as a goal to be achieved. The issue is increasingly relevant, so much so that energy efficiency is becoming a real architectural requirement, according to Thoughtworks. Platform Engineering can contribute to making software more sustainable, as Michel Murabito, Developer Advocate at Mia-Platform, explains in this article.
One of the most relevant tools in this category is kube‑green, an open‑source project developed by Davide Bianchi, Senior Technical Leader at Mia‑Platform. kube‑green significantly reduces energy consumption by automatically shutting down some unused resources according to a predetermined schedule, also bringing big cost savings.
Data Management
Last, but certainly not least, there is the issue of data management, as applications need it to run. Databases are needed for data storage: depending on the type of application, databases can be relational or documental databases (SQL and NoSQL). Examples of relational databases are MySQL, Oracle, PostgreSQL, while one of the most popular NoSQL databases is MongoDB.
But data management is not limited to databases alone. Data can initially be collected from the various systems in which they reside and be made available in real‑time: this task is performed by data stream systems, such as Confluent or Kafka.
Finally, it is very important to handle data differently depending on its temperature, that is, how often it is used. In this way, it is possible to optimize costs by leveraging the storage capabilities of storage buckets such as Amazon S3 and Oracle Object Storage.
An Internal Developer Platform for a better Developer Experience
The components illustrated above are fundamental to application software development, and as noted above, each requires knowledge and management of many different services. Performing the various steps independently and manually connecting all the various components is time‑consuming, difficult, and error‑prone.
For this reason, Internal Developer Platforms have emerged and are spreading. Thanks to them, developers have a single point of access, namely the platform, which is responsible for managing the entire software lifecycle, simplifying and speeding up their work. The Developer Experience improves significantly by using an Internal Developer Platform: the delivered software will be of higher quality and productivity will increase significantly as developers can focus on code and new features. Secondary activities, namely those related to the management of development tools, will in fact be automated and simplified by the platform, easing the cognitive load of developers.
Conclusion
In this article, we have outlined some of the components for application software development. Based on our experience in Mia‑Platform, these components are critical for most Internal Developer Platforms, but they certainly do not cover all the capabilities that platforms can provide. In addition, considering that new tools emerge almost daily to simplify development or extend functionality, it is important that platforms are built with a modular perspective so that new services and capabilities can be easily integrated.
Download our free white paper to find out how to evolve your company into a Platform Company!