New! Filter and export controls, plus lots of new mods and plugins. →
Announcement

v0.5.0: Controls, compliance benchmarks (CIS + PCI + custom) and lots of Hub updates

Learn more about the new open source features in Steampipe v0.5.0

Steampipe Team
5 min. read - May 20, 2021

What is Steampipe?

Steampipe is open source software for interrogating your cloud. Run SQL queries, compliance controls and full governance benchmarks from the comfort of your CLI.

Steampipe’s codified operations framework gives you the power to test your cloud resources against security, compliance and cost benchmarks, and to build your own custom control frameworks.

Our multi-threaded Golang CLI makes your custom SQL controls blazing fast with unlimited integration options via our embedded PostgreSQL database.

steampipe cli
>
select
region,
instance_state as state,
instance_type as type
from
aws_ec2_instance;

+-----------+---------+-----------+
| region    | state   | type      |
+-----------+---------+-----------+
| eu-west-1 | running | t3.medium |
| eu-west-2 | running | m5a.large |
| us-east-1 | running | t3.large  |
+-----------+---------+-----------+

tl;dr

Define and run custom Controls.
Run our CIS and PCI DSS Compliance Benchmarks or code your own.
Discover existing and collaborate on new mods using the Steampipe Hub.
Join our new Slack Channel!
Even more goodies in the full release notes.


Controls

Running queries interactively is a powerful way to explore your resources and answer adhoc questions of your cloud configuration, but often we know the exact question we would like answered (e.g. are all my cloud storage buckets private?). In those situations, we simply want to check if our currently deployed resources meet a pre-defined condition, Steampipe controls (new in v0.5.0) allow you to do just that. To run a control we use the new CLI action check:

bash
$ steampipe check all

⠏ Running 63 controls. (10 complete, 53 pending, 0 errors): executing...
    

Define a custom control

The evaluation logic for a control is (of course) SQL. In fact, controls are built from named queries that adhere to a simple interface. The required elements of the control query are:

  • resource: a short resource identifying name/id
  • status: the evaluated current status for the resource [ok, alarm, error, info, skip]
  • reason: a description of the reason for the current status.

Queries can also return optional results called dimensions. Dimensions are metadata that can be returned from a control query to provide additional context for a given resource (i.e. region and account are common dimensions for AWS resources).

Controls are named and defined using a domain specific HCL syntax:

my_controls.sp
control "bucket_versioning" {
title = "S3 buckets should have versioning enabled."
sql = <<EOT
select
arn as resource,
case
when versioning_enabled then 'ok'
else 'alarm'
end status,
name || ' versioning is ' || versioning_enabled || '.' as reason,
region,
account_id
from
aws_s3_bucket
EOT
}

Multiple controls can be defined in a single .sp file. All .sp files in the current workspace are loaded when the CLI (or service) are running, and named controls can be run with the steampipe check {control-name} command (see above).

The SQL query can be included inline (as above) or it can be a reference to a named query in the same workspace (additional info on named queries and workspaces).

Our control returns a single row for each bucket, with a status of skip if the bucket does not have the “env”:”prod” tag, ok if tagged and versioning is enabled and alarm if prod with no versioning.

CIS, PCI and Custom Benchmarks

Manually running many individual controls would be tedious and time consuming. Benchmarks are used to organize controls into hierarchies for aggregation and reporting; these collections of controls can then be run with a single command.

In our example above we see the power of organizing controls into a hierarchy. Steampipe automatically aggregates the subtotals for each level of control hierarchy in the benchmark, providing a birds eye view of your current compliance state.

Open source benchmarks for AWS, Azure & GCP

To show off the power of mods to codify compliance and security controls, the Turbot team has built out a series of compliance mods for AWS, GCP and Azure that include Center for Internet Security (CIS) and PCI DSS benchmarks.

At launch, these mods define 200+ controls as code, covering the most common cloud environment misconfigurations (encryption, public access, networking and identity management). These controls are fully documented, annotated and ready for download. Installing these benchmarks is as simple as cloning a repository:

steampipe cli
$ steampipe plugin install aws
$ git clone git@github.com:turbot/steampipe-mod-aws-compliance.git
$ cd steampipe-mod-aws-compliance
$ steampipe check all
⠏ Running 63 controls. (10 complete, 53 pending, 0 errors): executing...

The Steampipe Hub

A Steampipe mod is a collection of related Steampipe resources such as queries, controls, and benchmarks. Steampipe mods are defined using HCL, SQL and Markdown, and are distributed as simple text files. These files are designed to be portable, and version controlled via git. The Steampipe Hub provides a central place to discover and document mods shared from any public git repository.

Mods can be discovered and examined in the Steampipe Hub. The Hub uses introspection, metadata and markdown docs from the mod repository to allow searching, and browsing of mod resources. Here is an example from the excellent AWS compliance mod:

The folder structure for a given mod should define a complete steampipe workspace. You can simply cd into the directory and run steampipe check all to immediately run all defined benchmarks and controls in the mod.

Installing an existing published mod to your local machine is accomplished by cloning a repo:

git clone https://github.com/turbot/steampipe-mod-aws-compliance.git

The Steampipe Hub and the mod repositories are deeply integrated. The hub uses introspection of the mod metadata to allow for killer search, hierarchical browsing of controls and deep links between controls, queries and the source files in the repository.

Let’s get building!

Steampipe now delivers a full suite of tools to build, execute and share cloud configuration, compliance, and security frameworks using SQL, HCL and a little elbow grease! We would love your help to expand the open source documentation and control coverage for CIS, PCI, HIPAA, NIST… and the best way to get started is to join our new Slack workspace and raise your hand; we would love to talk to you!

For even more good stuff in v0.5.0, checkout the full release notes on GitHub.