# v0.7.0: AWS multi-account queries & Docker support

> Learn what's new in Steampipe's open source v0.7.0 release

By Steampipe Team
Published: 2021-07-22


<div className="row mb-5 mt-1">
  <div className="col col-12 col-lg-6">
    <h2>What is Steampipe?</h2>
    <p>Steampipe is <strong>open source software for interrogating your cloud</strong>. Run SQL queries, compliance controls and full governance benchmarks from the comfort of your CLI.</p>
    <p>Steampipe’s codified operations framework gives you the power to <strong>test your cloud resources against security, compliance and cost benchmarks</strong>, and to build your own custom control frameworks. </p>
    <p>Our <strong>multi-threaded Golang CLI</strong> makes your custom SQL controls blazing fast with unlimited integration options via our <strong>embedded PostgreSQL database</strong>.</p>
  </div>
  <div className="col col-12 col-lg-1"></div>
  <div className="col col-12 col-lg-5 mt-5">
    <Terminal title="steampipe cli">
      <TerminalCommand enableCopyToClipboard={false}>
        {`
select
  region,
  instance_state as state,
  instance_type as type
from
  aws_ec2_instance;
        `}
      </TerminalCommand>
      <TerminalResult>
        {`
+-----------+---------+-----------+
| region    | state   | type      |
+-----------+---------+-----------+
| eu-west-1 | running | t3.medium |
| eu-west-2 | running | m5a.large |
| us-east-1 | running | t3.large  |
+-----------+---------+-----------+
        `}
      </TerminalResult>
    </Terminal>
  </div>
</div>

## tl;dr

** &rarr; ** **[Connection aggregators](#connection-aggregators)** for AWS multi-account &amp; more. <br />
** &rarr; ** AWS **[multi-region wildcard](#multi-region-wildcards)** support. <br />
** &rarr; ** Steampipe **[support for Docker](#docker-support)**. <br />
** &rarr; ** **[Fast startup](#fast-startup-and-smoother-services)** and smoother services. <br />
** &rarr; ** **[7 new plugins and mods](#new-plugins-and-mods)**. <br />
** &rarr; ** **[Join our new Slack Channel](https://steampipe.io/community/join)**! <br />
** &rarr; ** Even more goodies in the [full release notes](https://github.com/turbot/steampipe/blob/main/CHANGELOG.md#v070-2021-07-22).

<br />

## Connection Aggregators

Connection aggregators are a simple new way to combine results from multiple connections into a single schema. With this multi-account setup for AWS:

```hcl
connection "all_dmi" {
  plugin      = "aws"
  type        = "aggregator"
  connections = ["dmi*"]
}
 
connection "dmi_prod" {
  plugin      = "aws"
  profile     = "profile1"
  regions     = ["us-east-1", "us-east-2"]
}
 
connection "dmi_dev" {
  plugin      = "aws"
  profile     = "profile2"
  regions     = ["us-west-1", "us-west-2"]
}
```

We can run a single query to instantly combine results from all accounts and regions:

```sql
select
  title,
  instance_type,
  region,
  account_id
from
  all_dmi.aws_ec2_instance;
```
```text
+-----------------+---------------+-----------+--------------+
| title           | instance_type | region    | account_id   |
+-----------------+---------------+-----------+--------------+
| DMI IT Bastion  | t3.large      | us-east-1 | 111111111111 |
| DMI Sales DB    | m5.xlarge     | us-east-2 | 111111111111 |
| DMI ActiveDir   | m5.large      | us-west-1 | 222222222222 |
| wufph.com       | m3.medium     | us-west-2 | 222222222222 |
+-----------------+---------------+-----------+--------------+
```

Aggregator connections works like all the connections you are used to, so can
be used immediately to run benchmarks and controls across multiple accounts!

<div className="row mb-4 mt-4 text-center">
  <div className="col col-12 col-lg-10">
    <img width="100%" className="center-block" src="/images/blog/control-run-aws-multi-account.png" />
  </div>
  <div className="col col-0 col-lg-1"></div>
</div>

### Multi-region wildcards

Accompanying the new multi-connection support in Steampipe, the AWS mod has
also added support for using wildcards in the region field of the AWS
connection configuration. Previously, multiple regions would need to be
specified individually, which can be unwieldly in the configuration file if
teams want to support all regions. Using wildcards greatly simplifies the
configuration and allows for graceful addition of new regions without need for
changing your config. Here are a few examples:

```hcl
# All US regions
connection "dmi_us" {
  plugin  = "aws"
  profile = "default"
  regions = ["us-*"]
}
 
# All EMEA
connection "dmi_emea" {
  plugin  = "aws"
  profile = "default"
  regions = ["eu-*", "af-*", "me-*"]
}
 
# All Regions
connection "dmi_global" {
  plugin  = "aws"
  profile = "default"
  regions = ["*"]
}
```

### More than just AWS

While AWS multi-account is a common case, connection aggregators work with any
plugin type so you can easily query across credential boundaries. Maybe you
ended up with multiple GitHub accounts for work and personal projects and want
to query them together:

```hcl
connection "gh" {
  plugin      = "github"
  type        = "aggregator"
  connections = ["dwight_dmi", "schrute_farms"]
}
 
connection "dwight_dmi" {
  plugin      = "github"
  token       = "30a98976e098798c9807609876c789675a"
}
 
connection "scrute_farms" {
  plugin      = "github"
  profile     = "30a4242424d8c983076093876c7238942c"
}
```

### (Un)Paralleled Performance

The quality of life change to allow for aggregate connections & queries is cool in itself, but the developers didn’t stop there. Steampipe’s FDW layer is designed to run all these queries in parallel, so when you execute an aggregate query, Steampipe uses its multi-threaded core to begin fetching results from all connections in parallel and immediately streaming back the results. In practice multi-connection queries will have the same performance as the slowest single connection in the group.

<div className="row mb-4 mt-4 text-center">
  <div className="col col-12 col-lg-10">
    <img width="100%" className="center-block" src="/images/blog/async.png" />
  </div>
  <div className="col col-0 col-lg-1"></div>
</div>

Caching behavior is just as smart. Running an aggregate query pre-loads the cache for subsequent multi-connection queries and for each of the individual connections as well.

## Fast startup and smoother services

"Time to first query" is something we focus on a lot. Too many people are stuck
building warehouses, or installing databases, etc when all they want to do is
run a query. That's why Steampipe automatically detects your credentials when
it can and works with any data source with no synchronization steps required.

Startup is faster in this release, noticably faster. We've worked hard to drop
you into the SQL prompt as fast as possible.

Other service improvements in this release:
- Run multiple `steampipe query` terminals in parallel.
- Improved cancellation handling for queries you regret.

## Docker Support

Once you get the feel for running queries interactively with Steampipe it is a
natural progression to want to automate queries that you run frequently or
share preconfigured workspaces with other members of your team. Containers are
a great way to isolate applications from the environment they run in, giving
you the ultimate flexibility to consistently run Steampipe on your desktop or
in your favorite cloud.

While some members of the community have had success getting Steampipe to run
in Docker, we wanted to implement some core changes to make use of Steampipe in
containerized environments more elegant, as we added support to query Docker
container configuration.

So, we've published an image to [Docker Hub](https://hub.docker.com/r/turbot/steampipe)!

```bash
docker run -it --rm --name steampipe turbot/steampipe
```

**[Using Containers &rarr;](/docs/develop/containers)**

Because of the Steampipe configuration, embedded database, etc there are a
number of different scenarios for using Steampipe in containers. Please check
out our [Using Containers doc](/docs/develop/containers) for all the details
and share your experiences in the [Steampipe Slack Community](https://steampipe.io/community/join).

Psst: We've also published a [Docker plugin](https://hub.steampipe.io/plugins/turbot/docker)!


## New plugins and mods

Beyond the CLI, since [our last release](/blog/release-0-6-0), we've added 6 new plugins:

- [AbuseIPDB](https://hub.steampipe.io/plugins/turbot/abuseipdb) - query IP address security data
- [Bitbucket](https://hub.steampipe.io/plugins/turbot/bitbucket) - query projects, issues, branches, etc
- [Docker](https://hub.steampipe.io/plugins/turbot/docker) - query Dockerfile data, containers, etc
- [Stripe](https://hub.steampipe.io/plugins/turbot/stripe) - query customers, invoices, etc
- [urlscan.io](https://hub.steampipe.io/plugins/turbot/urlscan) - query website metadata
- [VirusTotal](https://hub.steampipe.io/plugins/turbot/virustotal) - query virus signatures

We've also expanded our mod coverage, including:

- [AWS Compliance mod](https://hub.steampipe.io/mods/turbot/aws_compliance) controls for [NIST CSF](https://hub.steampipe.io/mods/turbot/aws_compliance/controls/benchmark.nist_csf) and [NIST 800-53](https://hub.steampipe.io/mods/turbot/aws_compliance/controls/benchmark.nist_800_53_rev_4)
- [GCP Thrifty mod](https://hub.steampipe.io/mods/turbot/gcp_thrifty) — Cost/usage checking


## Let’s get building!

Steampipe now delivers a full suite of tools to build, execute and share cloud configuration, compliance, and security frameworks using SQL, HCL and a little elbow grease! We would love your help to expand the open source documentation and control coverage for CIS, PCI, HIPAA, NIST… and the best way to get started is to [join our new Slack workspace](https://steampipe.io/community/join) and raise your hand; we would love to talk to you!

For even more good stuff in v0.7.0, checkout the [full release notes on GitHub](https://github.com/turbot/steampipe/blob/main/CHANGELOG.md#v070-2021-07-22).