How DeFacto migrated to an event-driven architecture
Challenge
DeFacto is a fashion retail company that operates across 33 countries with over 500 stores. The company develops all the applications for its core retail business in-house, encompassing an extensive range of Enterprise Resource Planning (ERP) modules including Point of Sales (POS), Store Operations, Customer Relationship Management (CRM), B2B, HR, Finance, integrations and many more.
However, DeFacto’s existing legacy application framework follows a monolithic approach, which has become increasingly challenging to maintain due to rapid growth in metrics including millions of lines of code, a growing number of developers across different teams, large numbers of application modules, servers, change requests, and deployment requests. Moreover, the costs associated with development are not the only concern; the infrastructure scaling costs are also a significant factor and have been consistently growing.
Solution
To address these business and technical issues, DeFacto made the strategic decision to adopt an event-driven architecture using microservices, and, wherever feasible, embrace asynchronous methodologies. They encouraged their developers to create and deploy distributed application components instead of relying on an overly complex monolithic codebase. To accomplish this, DeFacto set out to develop a platform called DFApi, which includes new technology strategies and a modern approach for developer operations, with a CLI, and a core runtime library to facilitate the development of distributed applications. As part of the creation of the core developer library, DeFacto investigated the Dapr project and determined that Dapr’s ability to seamlessly integrate with the application layer using its APIs was an ideal fit for their requirements
Impact
In a short time, as a result of adopting Dapr, DeFacto’s developers rapidly deployed a significant number of services into all environments. With multiple development teams working on different scopes and solution areas, interdependencies between services under the control of different teams added further complexity to the development process. However, Dapr’s service invocation API building block streamlined the development process by simplifying the discoverability and calling between multiple services.
By the numbers
27.5 Million
Events published per 100 hours
26 Million
Events consumed per 100 hours
300
Deployments
How DeFacto migrated to an event-driven architecture leveraging Dapr
DeFacto is a fashion retail company that operates across 33 countries with over 500 stores. The company develops all the applications for its core retail business in-house, encompassing an extensive range of Enterprise Resource Planning (ERP) modules including Point of Sales (POS), Store Operations, Customer Relationship Management (CRM), B2B, HR, Finance, integrations and many more.
However, DeFacto’s existing legacy application framework follows a monolithic approach, which has become increasingly challenging to maintain due to rapid growth in metrics including millions of lines of code (LOC), a growing number of developers across different teams, large numbers of application modules, servers, change requests, and deployment requests. As these numbers continue to rise, this has resulted in an inevitable decline in quality, agility, and importantly velocity when delivering these solutions.
Moreover, the costs associated with development are not the only concern; the infrastructure scaling costs are also a significant factor and have been consistently growing. As the volume of usage, LOC, and data increases, the expenses related to scaling infrastructure have escalated even further. As a result of these business challenges, DeFacto set about finding a solution to its growing needs.
Adopting an event-driven architecture
To address these business and technical issues, DeFacto made the strategic decision to adopt an event-driven architecture (EDA) using microservices, and wherever feasible embrace asynchronous methodologies. They encouraged their developers to create and deploy distributed application components instead of relying on an overly complex monolithic codebase. To accomplish this, DeFacto set out to develop a platform called DFApi, which includes new technology strategies and a modern approach for developer operations, with a CLI, and a core runtime library to facilitate the development of distributed applications.
As part of the creation of the core developer library, DeFacto investigated the Dapr project and determined that Dapr’s ability to seamlessly integrate with the application layer using its APIs was an ideal fit for their requirements. By leveraging the Dapr .Net SDK, DeFacto was able to effortlessly encapsulate the necessary functionality to develop an EDA required by their underlying scenarios.
To implement an asynchronous approach, DeFacto opted to utilize the Dapr publish & subscribe API building block to decouple various backends notably NATS, RabbitMQ and Kafka from the application code, enabling a technology-independent integration in the architecture. This approach ensures flexibility and scalability of the DFApi developer platform, whilst promoting efficient communication between services.
Impact
As a result of using Dapr, DeFacto has gained many advantages, along with the ability to effortlessly perform the following operations in a distributed environment:
- Discover and call target APIs for both synchronous and asynchronous communication.
- moothly transition between different environment configurations and dependencies: Dev, Test, Prod and User Acceptance Testing (UAT)
- Isolate application API calls in the same Kubernetes namespace.
- Seamlessly publish and subscribe to messaging events treating them similarly to regular API requests and responses.
- Troubleshoot performance issues and errors in a distributed environment using tracing, logs, and metrics.
In a short time, as a result of adopting Dapr, DeFacto’s developers rapidly deployed a significant number of services into all these environments. With multiple development teams working on different scopes and solution areas, interdependencies between services under the control of different teams added further complexity to the development process. However, Dapr’s service invocation API building block streamlined the development process by simplifying the discoverability and calling between multiple services.
“With Dapr, it was much easier to transition from our monolithic applications to a distributed environment. Especially for developers. The adaptation process to a distributed environment took much less time with Dapr.”
Tugay Ersoy, Software Architect at DeFacto
Red Hat OpenShift Clusters and Namespacing with Dapr
DeFacto uses two Red Hat OpenShift clusters—one for production and a second for pre-production. Since the teams use multiple environments (Dev, Test, UAT) within the pre-production cluster, DeFacto decided to utilize namespaces (OpenShift projects) to organize their deployments. This decision addressed the need for environment separation and contributed to an improved development experience.
In this design, a service may have multiple instances within the same cluster. For example, an application named ServiceA, which depends on ServiceB, should call the instance of ServiceB residing in the same namespace. Fortunately, Dapr handles this scenario seamlessly, automatically routing requests to the service within the same namespace.
However, what about the case where the request originates externally from the cluster and needs to be fulfilled by ServiceB in the UAT environment? DeFacto’s design incorporates an API Gateway deployed with its own Dapr sidecar in the cluster. The API Gateway utilizes path-based routing rules to construct the desired Dapr service invocation URL, including the addition of the necessary namespace. The Dapr sidecar on the API Gateway then routes the request to ServiceB residing in the requested namespace, enabling seamless communication across environments. Allowing Dapr to handle incoming requests via an API Gateway, which are then routed to the correct service, and then enabling these services to send messages between applications using a message broker such as NATs or Kafka, has created an easy-to-manage and scalable EDA.
Designing and developing business logic in an asynchronous manner can be very time-consuming and complex compared to creating synchronous services. This complexity is exacerbated for inexperienced developers. Fortunately, Dapr simplifies and standardizes the consumption and publishing of events. Developers can now create methods and easily bind them to desired message topics, eliminating the need to learn specific stream connection methods or underlying protocols. This streamlined approach allowed DeFacto to quickly deploy event-driven business logic without prior experience or examples. In a microservices environment with numerous deployments, these operations would typically require repeated implementation with different messaging systems and for different application scopes and teams. Dapr significantly contributed to simplifying the management, development, maintenance, stabilization, and learning efforts in such a complex environment.
Dapr also provides built-in features for collecting distributed telemetry using Open Telemetry, which can be visualized using tools like Zipkin or others. DeFacto configured Elasticsearch as the backend for Zipkin and used Dapr to send telemetry data to the Zipkin endpoint. These configurations were created seamlessly through a simple YAML configuration file, enabling the DeFacto team to save time and focus on developing additional features such as Grafana dashboards.
In summary, Dapr has played a pivotal role in enhancing DeFacto’s development process and improving maintenance, stability, and manageability in its microservices environment.
“Dapr has made it extremely easy to adopt new approaches. Thanks to Dapr, in a short time we could create a new platform and start to produce distributed asynchronous applications. Dapr helped us to reach our goal very fast in a well-architected way.”
Sedat Eyüboğlu, Software Architecture Consultant at DeFacto & CTO at Codelytic
Dapr Applications
At the time this article was written, DeFacto had over 300 deployments running in its OpenShift production cluster. Below are two example applications built using the Dapr APIs and components running in the DFApi platform.
Email Service API
The Email API receives mail events from NATS via the Dapr Pub/Sub API, saves the email body to Couchbase DB using the Dapr State API, and saves attachments to OpenShift Container Storage.
Customer Contact Permissions
The Contact Permission Management service distributes contact permissions set by customers to third-party providers using NATS via the Dapr Pub/Sub API and saves customer state information to Couchbase DB using the Dapr State API.