Announcing the GitHub and Bitbucket Exporters

As 2023 begins, things are really warming up at Port.

‍

Our GitHub and BitBucket exporters are our newest features. To understand them in context, we wanted to first explain the interplay of GitOps and internal developer portals and then the power of using exporters to populate the software catalog with GitOps data.

‍

Git and an Internal Developer Portal

Git contains a lot of information about the different entities and components you manage. From readmes, microservice names (repo names if you operate in poly repos), IaC files, K8S manifests, ArgoCD files, issues, API Documentation and pull requests. The list goes on.

On our mission to bring everything into one Internal Developer Portal, for context, self-service and even workflow automations, Git providers are a key data source to deliver on this promise.

‍

Ingesting data into the software catalog in the developer portal from your git provider

A simple ingestion of data from GitHub or Bitbucket would essentially sync the relevant YAML about the relevant component (any of service, IaC or k8s related) to the developer portal. It’s almost manual - whatever was written into the YAML in Git is sent to the software catalog.

This isn’t necessarily a bad practice, since requiring a service owner to maintain a YAML file is a good driver of “you build it, you own it”, since it forces the service owners to put the relevant data in the YAML file. But it potentially misses out on the valuable data contained in the Git provider.

‍

GitHub and Bitbucket contain very valuable data that would enrich any software catalog. This information (which our GitHub and Bitbucket exporters fetch) is made of two types:

1. Data about user activity: think of issues, pull requests, who ran which workflow. Perhaps hundreds of these data points are created each and every day. This is valuable real time data which isn’t truly accessible and is at no-one's fingertips.

2. Data about services, IaC, Kubernetes & everything GitOps. The Git provider contains multiple files with important data about repos, environments and how they are structured. This appears in Terraform files for the environment, helm charts with the K8s setup and the values files for K8s, as well as the argo app. Reflecting this data in the developer portal can provide much needed insight about the state of services and infrastructure - in real-time.

‍

The GitHub and the Bitbucket Exporters - what are they?

Our new GitHub and Bitbucket exporters are integrations that allow you to automatically collect live data to be digested by Port’s software catalog. These integrations export GitHub or Bitbucket objects (like pull requests, issues, workflows, and repositories) to Port. They also let you map your GitHub or Bitbucket resources into Port’s software catalog.

This means that you can:

List and export existing Git Provider data from your organization, such as pull requests or repositories and more
Watch for diff changes in specific git files (commit new file / change / delete) in real-time. The changes will be automatically applied to the relevant entities in the software catalog;
Configure which objects are relevant, and how to map and transform objects to the relevant entities in the software catalog.

‍

How does it work?

Port collects data across your entire stack, processes it and shows it just where you want it to appear in the software catalog.

‍

Why This Matters

We’ve already explained our approach of streaming exporter data into the right place in the software catalog instead of to the microservice. Similarly, this means that the relevant Bitbucket or GitHub data will appear in the right place in Port’s software catalog.

And the best news? We’re working on a GitLab exporter too.

Home

Blog

Announcing the GitHub and Bitbucket Exporters

This is some text inside of a div block.

Developer Portal

DevOps

Developer Experience

Book a demo right now to check out Port's developer portal yourself

It's a Trap - Jenkins as Self service UI

Developer Experience

GitOps

DevOps

Internal Developer Platform

How do GitOps affect developer experience?

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

Developer Portal

Developer Velocity

Developer Experience

Learning from CyberArk - building an internal developer platform in-house

Example JSON block

{
  "foo": "bar"
}

Order Domain

{
  "properties": {},
  "relations": {},
  "title": "Orders",
  "identifier": "Orders"
}

Cart System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Cart",
  "title": "Cart"
}

Products System

{
  "properties": {},
  "relations": {
    "domain": "Orders"
  },
  "identifier": "Products",
  "title": "Products"
}

Cart Resource

{
  "properties": {
    "type": "postgress"
  },
  "relations": {},
  "icon": "GPU",
  "title": "Cart SQL database",
  "identifier": "cart-sql-sb"
}

Cart API

{
 "identifier": "CartAPI",
 "title": "Cart API",
 "blueprint": "API",
 "properties": {
   "type": "Open API"
 },
 "relations": {
   "provider": "CartService"
 },
 "icon": "Link"
}

Core Kafka Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Kafka Library",
  "identifier": "CoreKafkaLibrary"
}

Core Payment Library

{
  "properties": {
    "type": "library"
  },
  "relations": {
    "system": "Cart"
  },
  "title": "Core Payment Library",
  "identifier": "CorePaymentLibrary"
}

Cart Service JSON

{
 "identifier": "CartService",
 "title": "Cart Service",
 "blueprint": "Component",
 "properties": {
   "type": "service"
 },
 "relations": {
   "system": "Cart",
   "resources": [
     "cart-sql-sb"
   ],
   "consumesApi": [],
   "components": [
     "CorePaymentLibrary",
     "CoreKafkaLibrary"
   ]
 },
 "icon": "Cloud"
}

Products Service JSON

{
  "identifier": "ProductsService",
  "title": "Products Service",
  "blueprint": "Component",
  "properties": {
    "type": "service"
  },
  "relations": {
    "system": "Products",
    "consumesApi": [
      "CartAPI"
    ],
    "components": []
  }
}

Component Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

Resource Blueprint

{
 "identifier": "Component",
 "title": "Component",
 "icon": "Cloud",
 "schema": {
   "properties": {
     "type": {
       "enum": [
         "service",
         "library"
       ],
       "icon": "Docs",
       "type": "string",
       "enumColors": {
         "service": "blue",
         "library": "green"
       }
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "system": {
     "target": "System",
     "required": false,
     "many": false
   },
   "resources": {
     "target": "Resource",
     "required": false,
     "many": true
   },
   "consumesApi": {
     "target": "API",
     "required": false,
     "many": true
   },
   "components": {
     "target": "Component",
     "required": false,
     "many": true
   },
   "providesApi": {
     "target": "API",
     "required": false,
     "many": false
   }
 }
}

API Blueprint

{
 "identifier": "API",
 "title": "API",
 "icon": "Link",
 "schema": {
   "properties": {
     "type": {
       "type": "string",
       "enum": [
         "Open API",
         "grpc"
       ]
     }
   },
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "provider": {
     "target": "Component",
     "required": true,
     "many": false
   }
 }
}

Domain Blueprint

{
 "identifier": "Domain",
 "title": "Domain",
 "icon": "Server",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {}
}

System Blueprint

{
 "identifier": "System",
 "title": "System",
 "icon": "DevopsTool",
 "schema": {
   "properties": {},
   "required": []
 },
 "mirrorProperties": {},
 "formulaProperties": {},
 "calculationProperties": {},
 "relations": {
   "domain": {
     "target": "Domain",
     "required": true,
     "many": false
   }
 }
}

Microservices SDLC

Scaffold a new microservice
Deploy (canary or blue-green)
Feature flagging
Revert
Lock deployments
Add Secret
Force merge pull request (skip tests on crises)
Add environment variable to service
Add IaC to the service
Upgrade package version

Development environments

Spin up a developer environment for 5 days
ETL mock data to environment
Invite developer to the environment
Extend TTL by 3 days

Cloud resources

Provision a cloud resource
Modify a cloud resource
Get permissions to access cloud resource

SRE actions

Update pod count
Update auto-scaling group
Execute incident response runbook automation

Data Engineering

Add / Remove / Update Column to table
Run Airflow DAG
Duplicate table

Backoffice

Change customer configuration
Update customer software version
Upgrade - Downgrade plan tier
Create - Delete customer

Machine learning actions

Train model
Pre-process dataset
Deploy
A/B testing traffic route
Revert
Spin up remote Jupyter notebook

Engineering tools

Observability
Tasks management
CI/CD
On-Call management
Troubleshooting tools
DevSecOps
Runbooks

Infrastructure

Cloud Resources
K8S
Containers & Serverless
IaC
Databases
Environments
Regions

Software and more

Microservices
Docker Images
Docs
APIs
3rd parties
Runbooks
Cron jobs

Announcing the GitHub and Bitbucket Exporters

Git and an Internal Developer Portal

Ingesting data into the software catalog in the developer portal from your git provider

The GitHub and the Bitbucket Exporters - what are they?

Why This Matters

Book a demo right now to check out Port's developer portal yourself

It's a Trap - Jenkins as Self service UI

How do GitOps affect developer experience?

It's a Trap - Jenkins as Self service UI. Click her to download the eBook

Learning from CyberArk - building an internal developer platform in-house

Example JSON block

Order Domain

Cart System

Products System

Cart Resource

Cart API

Core Kafka Library

Core Payment Library

Cart Service JSON

Products Service JSON

Component Blueprint

Resource Blueprint

API Blueprint

Domain Blueprint

System Blueprint

Microservices SDLC

Development environments

Cloud resources

SRE actions

Data Engineering

Backoffice

Machine learning actions

Engineering tools

Infrastructure

Software and more

Our blog

How to Drive a Culture of Kubernetes Standards With Platform Engineering

A Package Catalog in Your Internal Developer Portal

Announcing the GitHub and Bitbucket Exporters

Let us walk you through the platform and catalog the assets of your choice.