What is an Internal Developer Platform
An internal developer platform (IDP) comprises the tools and services that developers need to build, deploy and manage infrastructure and processes, in a simplified self-service capability form that reduces the cognitive load on developers and eliminates TicketOps. The platform includes infrastructure, cloud services, and other essential components of the software development lifecycle (SDLC).
This means that rather than having to work directly within third-party tools or infrastructure (eg. cloud providers, incident management, AppSec, CI/CD, etc), the developer can access self-service capabilities provided through the platform, which provides a level of abstraction, making it easier to use. This abstraction helps developers to focus on coding and innovation rather than the intricacies of deployment and operations. Typically developers access the platform through the Internal Developer Portal, which is also sometimes called IDP, but they can also make use of the internal developer platform using other self-service interfaces, such as CI tools.
IDPs and platform engineering
The IDP is a result of the shift towards platform engineering. As companies have matured, their tech stacks have become more complex, with the introduction of cloud-native, microservices, Kubernetes, and more. This complexity has created inefficiencies in the SDLC, where developers are overly reliant on DevOps, SREs and IT for approvals, deployment and management.
It has also placed a burden on developers, who are being asked to do more and know more. Platform engineering aims to better manage workflows, and the tools and technologies within the SDLC. It’s a movement that puts developers at the forefront - treating them as ‘customers’ to the product to provide them with what they need to do their jobs better. As a result, it has been crucial to create an IDP as part of platform engineering.
IDPs provide developers with standardization, autonomy through reusable self-service actions for configuring, deploying, creating, monitoring, and maintaining applications. This boosts their efficiency and productivity, and subsequently streamlines the SDLC and improves developer experience.
Key Components of an Internal Developer Platform
An IDP is built, configured and maintained by the platform engineering team, sometimes called a developer experience or developer productivity team.
Its key components are:
Source Control Management (SCM): Tools such as GitHub, GitLab, and Bitbucket enable version control, facilitating developers to track changes, collaborate effectively, and manage code repositories with ease.
Continuous Integration/Continuous Deployment (CI/CD): These tools handle building, testing, and deploying code. Deployment management includes debugging support, consolidated logs, and comprehensive versioning for audit trails, simplifying operations and boosting productivity. Tools like Jenkins, CircleCI, and GitHub Actions automate code integration and deployment. These tools ensure that changes are automatically tested and deployed, minimizing manual effort and reducing errors.
Infrastructure Orchestration via Infrastructure as Code (IaC): Tools like Terraform, Ansible, and Pulumi enable the provisioning and management of infrastructure through code, otherwise known as infrastructure orchestration. This approach ensures consistent and repeatable provisioning of environments across development, testing, and production stages. Infrastructure orchestration streamlines processes, reduces manual intervention, and enhances efficiency.
Control Plane: This manages all resources, be it applications running in a K8s cluster or infrastructure running in various cloud providers. It is an entry point API that acts as the central point for managing resources in the platform.
Control Plane Interface: To enhance user experience, an internal developer portal can simplify the interface by hiding unnecessary details like database storage information. Direct interaction with the control plane or cluster should be minimized, favoring Git for state management.
Containerization: Technologies such as Docker and Kubernetes are vital for packaging applications and managing containerized workloads, ensuring scalability and consistent environments.
Observability and Monitoring: Observability offers insights into the performance, health, and behavior of platform components and applications. This involves collecting and analyzing data from logs, metrics, traces, and events. Tools like Prometheus, Grafana, and Datadog are integral to this process, enabling quick detection, diagnosis, and resolution of issues, thus ensuring the platform’s reliability and performance.
Internal Developer Portal: An Internal Developer Portal provides a user-friendly interface to the platform. It abstracts the complexities of the software development environment, offering a unified interface designed to meet the needs of various development teams. By further simplifying and refining the IDP, the portal also provides developer self-service directly, while maintaining best practices and necessary guardrails to provide developers with golden paths. It consolidates all DevOps resources in a service catalog (also known as a software catalog), ensuring developers have the necessary tools and information without being overwhelmed.
Gartner says “internal developer portals serve as the interface through which developers can discover and access internal developer platform capabilities”.
Application Configuration Management: A standardized way to manage configuration, which has presented challenges for engineering teams in the past because of version control and a lack of autonomy. IDPs enable engineers to manage application configurations dynamically, scalably and reliably.
Self-Service Environment Management: An IDP streamlines the ability for developers to create fully provisioned environments as needed, by providing self-service options that cut down manual wait times and prevent idle resources. This approach enhances the developer experience, speeds up delivery, and manages costs effectively. Another term for self-service environment management is Platform Orchestration.
Database Management: Efficiently manage database schemas as part of application definitions in Git.
Secrets Management: Employ a secrets manager for sensitive information, such as passwords, that shouldn't be stored in Git.
Benefits of Using an Internal Developer Platform
Implementing an internal developer platform provides numerous advantages for developer teams and organizations. Here are the key benefits:
Compliance with Security, Standards and Governance: An IDP ensures that both software and infrastructure adhere to standards for production readiness, cost management, and application security. This guarantees high-quality, secure software delivery and streamlined operations. Compliance is ensured both through golden paths when entities are set up and through initiatives and dashboards that provide a birds-eye view of compliance over time. Features like RBAC and audit logs enhance security and compliance.
Increased Productivity: By automating repetitive tasks and providing a self-service environment, developers have fewer bottlenecks and can concentrate on coding and delivering features more quickly, resulting in shorter development cycles and faster time-to-market.
Standardization: An IDP enforces standardized practices across teams for code integration, deployment, and infrastructure management. This consistency reduces errors and enhances the overall quality of software delivery.
Reduced Cognitive Load: By abstracting infrastructure complexities and offering user-friendly tools, an IDP allows developers to focus on their core responsibilities. This reduces the learning curve for new team members and minimizes context switching, making development more efficient.
Enhanced Collaboration: A centralized platform with integrated tools and shared practices promotes better collaboration between development, operations, and other stakeholders. This improves communication and overall efficiency within the organization.
Autonomy: With access to the right tools and resources, developers can efficiently complete their tasks.
Efficiency: Automating routine tasks speeds up processes and reduces the time taken for inter-team interactions. This allows platform engineers to focus on strategic initiatives.
Best Practices for Implementing an Internal Developer Platform
Internal Developer Platforms are intended to streamline development, but often fall short in accessibility and scalability, and this can impact adoption. The root cause is failing to treat these platforms as products. Here are some best practices that enable platform engineers to take on a product management approach:
- Understand Developer Needs: Identify what developers aim to achieve, such as creating environments or scaffolding microservices. Use a ‘Jobs to be Done’ framework to define, categorize and manage these needs. Conduct user interviews and research to gather this information. Three standout questions are: some text
- Who is the portal actually for?
- What do developers need to know to ship software safely, while using golden paths?
- What functionality does a platform need to facilitate this?
- Assess User Knowledge: Determine what users need to know, avoiding overwhelming details. For instance, clarify the necessary knowledge about Kubernetes for self-service without causing cognitive overload.
- User-Centric Design: Build platforms based on developer needs and knowledge. When providing platform elements for managers that need to measure appsec compliance, engineering metrics and more, ensure the data is presented in an actionable way.
- Abstract Away Complexity: Use internal developer portals as a unified interface to abstract complexities, provide self-service options, and centralize resources for a seamless developer experience. The portal helps engineers to visualize what the developer experience will be.
- Drive Business Value: Establish roadmaps to ensure the platform delivers value to the business and improves developer experience.
- Consider End-to-End Experiences: think about the SDLC as a whole; from planning your day, to development, to shipping features, and operating production. This helps engineers to focus less on individual elements or features.
- Continuously Refine: Keep asking for feedback on new features and additions, keep iterating the platform and portal, customize the rollout to align with the goals and structures of your team.
