What is Steampipe?

Steampipe is open source software for interrogating your cloud. Run SQL queries, compliance controls and full governance benchmarks from the comfort of your CLI.

Steampipe’s codified operations framework gives you the power to test your cloud resources against security, compliance and cost benchmarks, and to build your own custom control frameworks.

Our multi-threaded Golang CLI makes your custom SQL controls blazing fast with unlimited integration options via our embedded PostgreSQL database.

{` select region, instance_state as state, instance_type as type from aws_ec2_instance; `} {` +-----------+---------+-----------+ | region | state | type | +-----------+---------+-----------+ | eu-west-1 | running | t3.medium | | eu-west-2 | running | m5a.large | | us-east-1 | running | t3.large | +-----------+---------+-----------+ `}

## tl;dr ** → ** **[New output formats](#new-output-formats)**.
** → ** **[Templates for control outputs](#templates-for-control-outputs)**.
** → ** **[4 new plugins](#new-plugins)**.
** → ** Even more goodies in the [full release notes](https://github.com/turbot/steampipe/blob/main/CHANGELOG.md#v0120-2022-01-20). ## New output formats We've added two new formats for control outputs: `asff` and `nunit3`. ### ASFF ASFF is the [Amazon Security Finding Format](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings-format.html): > AWS Security Hub consumes, aggregates, organizes, and prioritizes [findings](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-findings.html) from AWS security services and from the third-party product integrations. Security Hub processes these findings using a standard findings format called the AWS Security Finding Format (ASFF), which eliminates the need for time-consuming data conversion efforts. Then it correlates ingested findings across products to prioritize the most important ones. Here's an example of `asff` output for an [AWS Compliance](https://hub.steampipe.io/mods/turbot/aws_compliance) run. It's the finding for the [foundational_security_ec2_2 control](https://hub.steampipe.io/mods/turbot/aws_compliance/controls/control.foundational_security_ec2_2?context=benchmark.foundational_security/benchmark.foundational_security_ec2) (The VPC default security group should not allow inbound and outbound traffic). ```json { "SchemaVersion": "2018-10-08", "Id": "aws_compliance.control.foundational_security_ec2_2", "ProductArn": "arn:aws:securityhub:ap-south-1:453319552164:product/453319552164/default", "ProductFields": { "ProviderName": "Steampipe", "ProviderVersion": "0.12.0-rc.1" }, "GeneratorId": "steampipe-foundational_security_ec2_2", "AwsAccountId": "453319552164", "Types": [ "automated" ], "UpdatedAt": "2022-01-20T22:33:52+05:30", "CreatedAt": "2022-01-20T22:33:52+05:30", "Title": "2 The VPC default security group should not allow inbound and outbound traffic", "Description": "This control checks that the default security group of a VPC does not allow inbound or outbound traffic. The rules for the default security group allow all outbound and inbound traffic from network interfaces (and their associated instances) that are assigned to the same security group.", "Severity": { "Label": "HIGH" }, "Resources": [ { "Type": "Other", "Id": "arn:aws:ec2:ap-south-1:453319552164:security-group/sg-79245001" } ], "Compliance": { "Status": "FAILED" } } ``` Here's one way to integrate this finding into Security Hub. ```shell aws securityhub batch-import-findings --findings "{\"SchemaVersion\":\"2018-10-08\", ... {\"Status\":\"FAILED\"}}" ``` Now the finding appears in Security Hub under the Findings section.

### nunit3 This is the XML format for [NUnit 3.0 test results](https://docs.nunit.org/articles/nunit/technical-notes/usage/Test-Result-XML-Format.html). Here's an example of `nunit3` output for the [CIS v1.40](https://hub.steampipe.io/mods/turbot/aws_compliance/controls/benchmark.cis_v140) benchmark. ```xml steampipe:status info steampipe:reason Manual verification required. steampipe:dimension:account_id 899206412154 ``` Here is a visualization made with AWS CodeBuild.

## Templates for control outputs This release adds a templating mechanism for control outputs, based on the golang [text/template](https://pkg.go.dev/text/template) package. For each output format there's now a directory, in `~/.steampipe/check/templates/`, which minimally includes a file called `output.tmpl`. The name of the directory defines the name of an output format. The content of `output.tmpl` defines how to unpack and format a golang struct, called `Data`, that's passed to the template. For example, let's create `~/.steampipe/check/templates/summary/output.tmpl` like so. ``` {{ define "output" }} {{ range .Data.Root.Groups }} Summary for {{ .Title }} {{ end }} total: {{ .Data.Root.Summary.Status.TotalCount }} passed: {{ .Data.Root.Summary.Status.PassedCount }} failed: {{ .Data.Root.Summary.Status.FailedCount }} skipped: {{ .Data.Root.Summary.Status.Skip }} {{ end }} ``` Now there's a new `summary` output format. ```shell steampipe check --output=summary all ``` ``` Summary for Zoom Compliance total: 185 passed: 119 failed: 66 skipped: 0 ``` This command puts the above output into the file `output.summary`. ```shell steampipe check --export=output.summary all ``` This command produces an inferred filename like `all-20220119-111307.summary`. ```shell steampipe check --export=summary all ``` Along with `asff` and `nunit3`, the `csv`, `html`, `json`, and `md` formats now use this mechanism. So you can explore those templates in `~/.steampipe/check/templates` to find more advanced examples that you can adapt for your own needs. ## New plugins Since our [last release](https://steampipe.io/blog/release-0-11-0), we've added 4 new plugins: - [Confluence](https://hub.steampipe.io/plugins/ellisvalentiner/confluence) - query spaces and contents - [Hypothesis](https://hub.steampipe.io/plugins/turbot/hypothesis) - query annotations, user profile - [PagerDuty](https://hub.steampipe.io/plugins/turbot/pagerduty) - query incidents, schedules, services, users, etc - [Prometheus](https://hub.steampipe.io/plugins/turbot/prometheus) - query alerts, metrics, rules, targets, etc We are always improving the suite of plugins. During this cycle we added tables to [AWS](https://hub.steampipe.io/plugins/turbot/aws), [Azure](https://hub.steampipe.io/plugins/turbot/azure), [Cloudflare](https://hub.steampipe.io/plugins/turbot/cloudflare), [Okta](https://hub.steampipe.io/plugins/turbot/okta), and [Oracle Cloud](https://hub.steampipe.io/plugins/turbot/oci). ## Let’s get building! Steampipe delivers tools to build, execute and share cloud configuration, compliance, and security frameworks using SQL, HCL and a little elbow grease. To support those tools, it maps a growing suite of APIs to tables that you can query, and join across, in Postgres. Do you want to help us expand the open source documentation and control coverage for CIS, PCI, HIPAA, and NIST? Add tables to existing plugins? Create plugins to bring new APIs into the mix? The best way to get started is to [join our Slack workspace](https://steampipe.io/community/join) and raise your hand. We would love to talk to you! For even more good stuff in v0.12.0, check out the [full release notes on GitHub](https://github.com/turbot/steampipe/blob/main/CHANGELOG.md).

v0.12.0: Templates for control outputs, two new formats