Steampipe-powered GitOps

The GitHub Setup and Check actions simplify installing Steampipe in a GitHub workflow, installing and configuring plugins, installing mods, and running benchmarks. You can also automate pushing snapshots to Turbot Pipes and annotating pull requests when controls raise alarms. Let's see how it all works.

Setup Steampipe, as a GitHub Actions

It's easy enough to install Steampipe in a GitHub action, the command is just a one-liner. But when you need to install and configure a plugin, things can get messy if you have to manipulate your .spc file in a GitHub runner. The Setup action handles everything in a clean and simple way.

- name: Setup Steampipe
  uses: turbot/steampipe-action-setup@v1
  with:
    plugin-connections: |
      connection "aws_dev" {
        plugin     = "aws"
        secret_key = "${{ secrets.AWS_ACCESS_KEY_ID_DEV }}"
        access_key = "${{ secrets.AWS_SECRET_ACCESS_KEY_DEV }}"
        regions    = ["*"]
      }

- name: Run queries
  run: |
    steampipe query "select account_id from aws_dev.aws_account"

That's it! Steampipe is installed, and the AWS plugin is installed and configured. If you need it, the Steampipe binary is available — for example, to run a query as shown here. But you won't need it to run benchmarks.

Thanks to François de Metz who built the first version of this action, then generously donated it to Turbot!

Steampipe Check, as a GitHub Action

There are powerful mods available to check for compliance and to display insights dashboards. The Check action neatly manages all the moving parts required to install a mod, choose a branch, specify which checks to run, specify exports, create an artifact, create and upload a Pipes snapshot, and display the report's markdown export in GitHub. This example installs, configures, and runs Terraform AWS Compliance.

name: Check terraform_aws_compliance.benchmark.ec2
on:
  workflow_dispatch

jobs:
  aws_tf_compliance:
    runs-on: ubuntu-latest
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v3
      - name: Steampipe Setup
        uses: turbot/steampipe-action-setup@v1
        with:
          plugin-connections: |
            connection "tf" {
              plugin = "terraform"
              paths  = [ "./path/to/dir/*.tf" ]
            }
      - name: Steampipe Check
        uses: turbot/steampipe-action-check@v1
        with:
          mod-url: https://github.com/turbot/steampipe-mod-terraform-aws-compliance
          checks: terraform_aws_compliance.benchmark.ec2
          snapshot-visibility: anyone_with_link
          pipes-token: ${{ secrets.PIPES_TOKEN }}

That's it! After the Setup action loads the Terraform plugin and points it at a set of Terraform files in the repo, the Check action loads the mod, runs the EC2 benchmark, and pushes a snapshot to Pipes.

Here's the GitHub view of the result. The artifact contains the action's default output (check results in CSV format) and also includes the check's markdown.

And here's the dashboard it uploaded to Pipes.

Annotations on pull requests

You can also trigger the Check action on pull requests. If your code changes raise alarms, the Check action will attach them to the pull request as annotations. Here's a Terraform resource that improperly sets block_public_acls to false.

resource "aws_s3_bucket_public_access_block" "public_access_bucket_1" {
  bucket = aws_s3_bucket.my_bucket_1.id

  block_public_acls   = false # should be true
  block_public_policy = true
  ignore_public_acls = true
  restrict_public_buckets = true
}


resource "aws_s3_bucket" "my_bucket_1" {
  bucket = "test-project-foo-bucket-01"
}

And here's the workflow file we'll use.

name: Annotate Pull Request Test
on:
  pull_request:
    types:
      - opened
    branches:
      - 'main'
    paths:
      - tf/*

jobs:
  compliance_with_annotations:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      checks: write
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v3
      - name: Steampipe Setup
        uses: turbot/steampipe-action-setup@v1
        with:
          plugin-connections: |
            connection "tf" {
              plugin = "terraform"
              paths  = [ "./path/to/dir/*.tf" ]
            }
      - name: Steampipe Check
        uses: turbot/steampipe-action-check@v1
        with:
          mod-url: https://github.com/turbot/steampipe-mod-terraform-aws-compliance
          github-token: ${{ secrets.GITHUB_TOKEN }}
          snapshot-visibility: anyone_with_link
          pipes-token: ${{ secrets.PIPES_TOKEN }}

The key difference: it triggers on pull requests, specifically those that affect Terraform files. It also adds required permissions and a GitHub token. And because it omits the checks argument, all the controls in the Terraform AWS Compliance will run.

When the pull request lands, the Check action runs. Again it reports outputs, creates an artifact, and posts a snapshot. But now it also annotates the PR!

The misconfiguration we introduced in the PR shows up first, anchored to the place we made the change: line 4 of s3_bucket.tf. But if there are more alarms — as in this case — you'd like to know about them, so the Check action posts those as annotations too.

See it in action

Get started with Steampipe Setup and Steampipe Check

Steampipe is happy to run in any CI/CD pipeline in order to enable queries that use the growing family of plugins, and to enable checks that use the parallel family of mods. If GitHub is your game, you can now take it to the next level with actions that make it easy to install, configure, and run Steampipe, then run compliance checks that integrate with both GitHub itself and with Turbot Pipes. If you haven't tried these new actions yet, give them a whirl and let us know how it goes!