Infrastructure as Code: Manage apps using the Ably Control API GitHub Action

I'm excited to announce the Ably Control API GitHub Action! With this Action you can create Ably apps and API keys straight from your GitHub workflow, ensuring a reliable and repeatable process of creating your cloud infrastructure.

Managing applications & infrastructure

With the adoption of DevOps, software development teams are becoming responsible for managing their applications and cloud infrastructure. Instead of handing over a zip file and a list of instructions to a system administrator, developers now use continuous integration & delivery (CI/CD) pipelines, an automated way to build and deploy software.

The benefits of automation include:

• A repeatable process.
• Reduced human error.
• Shorter delivery cycle times.

Source control

These days, the CI/CD pipeline definitions are put in source control, which provides yet another benefit. Pipeline changes can be reviewed using pull requests (PRs), reducing the risk of one person accidentally breaking production systems. Many organizations require the 'four eyes principle' for production changes, and PRs facilitate this well.

Infrastructure as Code

The next step in the automation process is managing the cloud infrastructure that the application requires. This can include Platform as a Service (PaaS) components from cloud service providers, or Software as a Service (SaaS) components from other vendors. When considering a new SaaS component, ensure that it provides a management/provisioning API. Otherwise, you might be stuck using a web interface that is not easy to automate.

The process of automating your cloud infrastructure and putting it in source control is called Infrastructure as Code (IaC) and has become very popular in recent years (see Figure 1). Armon Dadgar, co-founder of HashiCorp, gives a great explanation on IaC in this video:

Cloud service providers usually have their own IaC formats and tooling. Microsoft has ARM and Bicep templates, AWS has CloudFormation, and Google has Deployment Manager. However, many other vendors offer IaC tools that are cloud agnostic such as Terraform and Pulumi.

Regardless of the format and tooling, a deployment pipeline that runs the IaC to deploy the infrastructure is needed.

GitHub

GitHub, by far the most popular source control service, offers CI/CD capabilities in the form of Workflows and Actions. This feature was announced at GitHub Universe 2018.

Workflows & Actions

A Github Workflow defines a CI/CD process and exists as a YAML file inside the .github/workflows folder inside a GitHub repo. The YAML file has the following structure:

name: GitHub Actions Demo   # Name of the workflow 
on: [push]                  # How is this workflow triggered
jobs:
  Explore-GitHub-Actions:   # Name of the job
    runs-on: ubuntu-latest  # Defines the OS for the runner
    steps:                  # A job contains one or many steps
    - run: echo "? The job was automatically triggered by a ${{ github.event_name }} event." # Run a command or script
    - name: Check out repository code
      uses: actions/checkout@v2 # Use a GitHub Action
    - run: echo "? The ${{ github.repository }} repository has been cloned to the runner."

The sample above is very minimal; the workflow syntax is quite extensive - see the GitHub Docs for more details.

Although you can use a run step to execute a command to create and manage your infrastructure, for example, call the Terraform CLI, GitHub has a Marketplace that contains thousands of Actions to automate your infrastructure more conveniently.

The benefit of using Actions over the generic run step is that Actions usually combine several commands into one package that is easily configurable in the workflow file. Actions hide some complexity that you would otherwise have to deal with by writing a script. On the other hand, since Actions are usually a wrapper around an existing API or CLI, they might not have all the features the API/CLI offers, and might not be suitable if you need more control.

Custom Actions

GitHub provides detailed information and tools that help in creating custom Actions.

There are three types of Actions that you can build:

• JavaScript Actions; these run directly on a runner machine (either Ubuntu, Windows, or macOS).
• Container Actions; these are Docker containers where you have full control of the OS version and tools/dependencies. These Actions require a bit of time to be built and downloaded to the runner and are therefore slower to start than JavaScript-based Actions.
• Composite Actions; these are scripts or several workflow steps that are combined into an Action for easier re-use.

Custom Action for the Ably Control API

For our Ably FFS Chat App project there initially was a CI/CD pipeline to only build and deploy the Static Web App that hosts our chat application. It was missing the creation of the Ably app and an API key. So we weren't doing full infrastructure as code just yet.

Luckily we've recently developed a Control API that allows you to create and manage your Ably apps using a REST client.

This Control API is ideal for creating/managing Ably apps in CI/CD pipelines. However, instead of making cURL calls to the REST endpoints directly in the GitHub workflow file, we made a Control API GitHub Action, so everyone who is using Ably can now create apps and API keys as part of their CI/CD pipeline.

The Control API Action is a JavaScript-based Action since it doesn't require special tools or dependencies what can't be installed through npm. GitHub provides detailed instructions on how to create a JavaScript Action. So that is what we followed.

Using the Ably Control API GitHub Action

The YAML example below shows a minimal GitHub workflow that uses the Ably Control API Action to create an Ably app and an API key.

- name: Create Ably App
  id: ablyapp
  uses: ably-labs/[email protected]
  with:
    account-id: '${{ secrets.ABLY_ACCOUNT_ID }}'
    control-api-key: '${{ secrets.ABLY_CONTROL_API_KEY }}'
- name: Get the output
  run: |
    echo "App Name: ${{ steps.ablyapp.outputs.app-name }}"
    echo "App ID: ${{ steps.ablyapp.outputs.app-id }}"
    echo "API Key Name: ${{ steps.ablyapp.outputs.api-key-name }}"

Action Inputs

The Action has the following inputs:

Action Outputs

The Action has the following outputs:

(*) Output is marked as a secret, so it won't be visible in the GitHub workflow logs.

For a full description of the Control API Action please see the project README.

We're currently using the Action in our FFS project to create our Ably App and API key. The output of the Action is used to set the API key as an app setting of the Azure Static Web App.

This is the log output of the Action:

Run ably-labs/ably-control-api-action@v0.1.4
  with:
    account-id: ***
    control-api-key: ***
    app-name: fully-featured-scalable-chat
    create-key: true
    key-name: all-capabilities-1
    key-capabilities: channel-metadata, history, presence, publish, push-admin, push-subscribe, statistics, subscribe
Generating Ably Keys...
Checking credentials for account: ***
GET: https://control.ably.net/v1/accounts/***/apps returned status: 200
Listing all apps registered on account...
GET: https://control.ably.net/v1/accounts/***/apps returned status: 200
Listing API keys available for app: xv_W-A
GET: https://control.ably.net/v1/apps/xv_W-A/keys returned status: 200
Account, application and keys all exist - storing secrets...

Completed

As can be seen in the log, sensitive values, such as IDs and keys are marked as secrets and therefore not visible in the logs.

We'd love your feedback

Infrastructure as Code is a great way to get reliable and repeatable deployments. The Workflows and Actions by GitHub offer a great way to deploy your applications wherever they are hosted. If none of the existing Actions on the GitHub Marketplace fulfill your needs, you can create and publish your own Action instead. This is will not only benefit you but also others who are looking for a similar solution.

If you haven't tried the Ably Control API Action yet, please give it a go and let us know if you find it useful. Feature requests or issues can be reported via GitHub. If you want more information about this topic feel free to contact me on Twitter.

Further reading

• Introducing Control API: provision & configure Ably programmatically
• Squid game: how we load-tested Ably’s Control API
• Conform and monitor...with humanity
• Ask these questions before you buy from a PaaS provider