Using Backstage’s C4 Model Adaptation to Visualize Software - Creating a Software Catalog in Port

Using Backstage’s C4 Model Adaptation to Visualize Software - Creating a Software Catalog in Port

Standardizing architecture diagrams for developer portals

It all started when spotify engineers decided to visualize software in order to standardize architecture diagrams there, to make the information clear and consumable to everyone within the engineering organization. This model is based on the C4 model with some adaptations. The model’s metadata is presented in a software catalog and the best part is that the question of whether what’s in the diagram and software model is up to date no longer matters, since it automatically updates. 

At Port, we like this model since it goes way beyond a microservices catalog and embarks on cataloging all elements related to the software - the microservices, the resources they are deployed on, the specific deployments and the relationships and dependencies between all of them. This is best said by Renato Kalman and Johan Wallin here:

“By expressing this model as metadata, we have been able to create a software catalog that keeps track of components, ownership, dependencies, and lifecycles in our ecosystem”.

I agree that this is what every engineering organization should do (once it reaches a certain size). But can this be accomplished outside backstage? As you’re probably guessing the answer is yes. This matters because not everyone can or wants to implement spotify’s backstage. Backstage requires some extensive coding and a team around it. Why not go for a SaaS based internal developer portal - which will let you accomplish the same with a builder based approach?

But first, let’s see the logic of the model, before we actually show how to apply it.

Why the backstage software catalog is the best there is 

This subtitle is a pretty bold statement, but at Port we truly believe in it. The only way to make sense and derive value from a software catalog is to take a similar approach. Engineering needs a unified and consistent view of everything, and it needs to include dependencies, relations, and a lot of additional metadata.

Backstage’s model begins with three core entities: API, component and resource, as in the C4 model. To make it easier to consume in large and complex organizations, two additional entities were added: Systems and Domains.

Systems are collections of entities that work together in performing a certain function and domains are systems and entities that “belong” to different parts of the business. 

To best understand the C4 model and its adaptation, let’s imagine you have to model a software design for your organization's new food delivery application. You will have to consider the following:

  • A RESTful Cart Service that is responsible for adding an order to the customer’s cart

    • The Cart Service depends on the libraries

      • Core Payment Library.

      • Core Kafka Library.

    • The Cart service also needs an SQL database to operate.

  • A Products Service that consumes Cart’s Service REST API is responsible for presenting the items that were ordered.

Now let’s begin the walkthrough: how can this be modeled in a builder-based software catalog?

Software Catalog - Modeling the structure

Port uses entities that support this adaptation of the C4 model. Entities are defined using a Port Blueprint, which is the primary building block of Port. Blueprints represent assets that can be managed in Port, such as microservices, environments, packages, clusters, databases, and many more.

Blueprints are completely customizable, and they support any number of properties the user chooses, all of which can be modified as you go.

Using simple JSON files, I have created five blueprints corresponding with the backstage entities with the exact relations that should be reflected

The software catalog model elements

Components

Components are Service or Library, which are differentiated by the type, a simple enum of “service” or “library.”

Components are also connected to themselves by many relations, because a service might be related to multiple libraries, like in our case. Or, it can also be related to another service.

{{component-blueprint}}

Resources

Resources are any Infrastructure needed to run a component (S3 buckets, SQL databases, etc..)

{{resource-blueprint}}

API

API is a simple software catalog item that can be consumed or provided by a component.

API also has a type, which tells the exact API type, REST, GRPC, protobuf, etc.

{{api-blueprint}}

Domain

A domain is a collection of systems representing a distinct area of influence, activity, and decision-making within an organization (business).

{{domain-blueprint}}

System

A system is a sub-domain that is focused on a specific branch (business) within the organization.

{{system-blueprint}}

Ingesting the data into the catalog

Once the blueprints are ready, we need to ingest the data into the software catalog, creating entities. Backstage requires putting a YAML inside a git repository or writing a custom entity processor. Port makes it easier, allowing you to ingest entities from your pipelines, K8S, Git, Terraform, API, and more.

For the sake of simplicity, we will use the JSON files below to reflect the entities within the catalog and the relations between them. You can copy and paste them into the UI, or via any other method mentioned above.

{{order-domain}}

{{cart-system}}

{{products-system}}

{{cart-resource}}

{{cart-api}}

{{ckl}}

{{cpl}}

{{cart-service-json}}

{{product-service-json}}

Once the data is ingested, you will gain complete visibility of the service catalog. You can see all the related entities for each entity. For example, let's look at the Cart Service entity page. We can see the Component it interacts with and the API it provides alongside the business-related domain of the service in a single united view.

Congratulations! You have created your own Developer Portal using the C4 adaptations Backstage use Port.

Conclusion

When it comes to an internal developer portal, every organization is different and has a unique set of individual needs and requirements. Backstage’s adaptation of the C4 model is the best way to create a software catalog (far better than most “microservices catalogs) but it probably isn’t the fastest…

{{cta}}
Developer Portal
DevOps
Developer Experience

Book a demo right now to check out Port's developer portal yourself

{{jenkins}}
Self-Service
Jenkins
Microservices
DevOps
Developer Experience

It's a Trap - Jenkins as Self service UI

{{gitops}}
Developer Experience
GitOps
DevOps
Internal Developer Platform

How do GitOps affect developer experience?

{{ebook}}
Self-Service
Jenkins
Microservices
DevOps
Developer Experience

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

{{cyberark}}
Developer Portal
Developer Velocity
Developer Experience

Learning from CyberArk - building an internal developer platform in-house

{{dropdown}}

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}
{{tabel-1}}

Microservices SDLC

  • Scaffold a new microservice

  • Deploy (canary or blue-green)

  • Feature flagging

  • Revert

  • Lock deployments

  • Add Secret

  • Force merge pull request (skip tests on crises)

  • Add environment variable to service

  • Add IaC to the service

  • Upgrade package version

Development environments

  • Spin up a developer environment for 5 days

  • ETL mock data to environment

  • Invite developer to the environment

  • Extend TTL by 3 days

Cloud resources

  • Provision a cloud resource

  • Modify a cloud resource

  • Get permissions to access cloud resource

SRE actions

  • Update pod count

  • Update auto-scaling group

  • Execute incident response runbook automation

Data Engineering

  • Add / Remove / Update Column to table

  • Run Airflow DAG

  • Duplicate table

Backoffice

  • Change customer configuration

  • Update customer software version

  • Upgrade - Downgrade plan tier

  • Create - Delete customer

Machine learning actions

  • Train model

  • Pre-process dataset

  • Deploy

  • A/B testing traffic route

  • Revert

  • Spin up remote Jupyter notebook

{{tabel-2}}

Engineering tools

  • Observability

  • Tasks management

  • CI/CD

  • On-Call management

  • Troubleshooting tools

  • DevSecOps

  • Runbooks

Infrastructure

  • Cloud Resources

  • K8S

  • Containers & Serverless

  • IaC

  • Databases

  • Environments

  • Regions

Software and more

  • Microservices

  • Docker Images

  • Docs

  • APIs

  • 3rd parties

  • Runbooks

  • Cron jobs

Let us walk you through the platform and catalog the assets of your choice.

I’m ready, let’s start