How We Continually Deliver Software

We currently have five different web applications in production and they all share a very similar stack - Django/Vue/Docker/PostgreSQL (some with Redis/django-rq for background tasks).

We have developed a set of Github Actions for Continuous Integration / Continuous Delivery that take care the this basic workflow:

1. Every commit either on main or a feature branch, runs:
   1. Python Linting
   2. Vue/JS Testing
   3. Build Docker Image and then on that image run:
      1. Python tests
      2. Check for missing migrations
      3. Push image / tags after being rebuilt without the dev mode flag
2. Then if on main it follows through with a deployment to a QA app on Heroku.

We have a second workflow for handling releases.

When a release is generated/published in Github:

1. Pulls latest image from the Github Container Repository
2. Pushes the tagged image to Heroku
3. Executes release commands, but this time to a Production app on Heroku

Results

These two pipelines enable us to work really fast. It speeds up code reviews as most of the testing is done automatically allowing us to focus on just the business rules and architecture getting put into place. It speeds up end to end testing and getting user feedback having code automatically deployed to a QA test instance that won't interfere / interrupt production. And finally it speeds up getting releases out to production which we do as needed, often a few times a day!

Open Source

The two yaml files configuring these were hundreds of lines long with lots of duplication except for a few things. We were copying them around when we'd start a new web app, and then tweak. They'd invariably get out of sync and it was becoming a burden to maintain.

So we extracted actions and workflows into wedgworth/actions which is now open source so if you like our workflow you can feel free to use (or fork and tweak to suit your needs).

Now each project looks like this:

ci.yaml

name: Test / Build / Deploy to QA
on:
  push:
    branches: "**"
    tags-ignore: "**"

jobs:
  test-and-build:
    name: CI
    uses: wedgworth/actions/.github/workflows/test.yml@v7.0.0
    with:
      python-src-dir: myapp
    secrets:
      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
      CR_UN: ${{ secrets.CR_UN }}
      CR_PAT: ${{ secrets.CR_PAT }}
      SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }}

  deploy-qa:
    name: CD
    needs: [test-and-build]
    if: ${{ github.event.ref == 'refs/heads/main' }}
    uses: wedgworth/actions/.github/workflows/deploy.yml@v7.0.0
    with:
      app-name: my-heroku-app-qa
      processes: web release
    secrets:
      HEROKU_API_KEY: ${{ secrets.HEROKU_API_KEY }}
      CR_UN: ${{ secrets.CR_UN }}
      CR_PAT: ${{ secrets.CR_PAT }}

release.yaml

name: Publish and Release Image
on:
  release:
    types: [published]

jobs:
  release:
    name: Release
    uses: wedgworth/actions/.github/workflows/release.yml@v7.0.0
    with:
      app-name: my-heroku-app-prod
      processes: web release
    secrets:
      HEROKU_API_KEY: ${{ secrets.HEROKU_API_KEY }}
      CR_UN: ${{ secrets.CR_UN }}
      CR_PAT: ${{ secrets.CR_PAT }}

We still copy and paste these but they are extremely stable.

We just need to set python-src-dir, app-name, and processes .

These do use runners from namespace.so which are not free (but cheap!) and run much faster especially when doing Docker builds than the Github runners.

There might be a way to make these configurable so if you like what you see but want to use the Github runners, we'd welcome a pull request to make this more generally useful, otherwise feel free to fork it and run your own copies.

Happy building!