# v0.20.0: Faster startup & revamped mod dependencies

> Discover the great new features in Steampipe's open source v0.20.0 release!

By Steampipe Team
Published: 2023-05-18


## Faster startup

v0.20.0 supercharges Steampipe's startup and connection sync time. No more waiting minutes for hundreds of connections to sync up, you're now set to query in seconds!

<div style={{"width":"80%"}}>

| Connections | v0.19.5 setup (seconds) | v0.20.0 setup (seconds) | Speed up |
|-------------|-------------------|-------------------|-------|
| 50          | 31                | 7                 | 4.4x  |
| 200         | 120               | 7                 | 17.1x |
| 500         | 306               | 8                 | 43.7x |
| 1000        | 714               | 11                | 79.3x |

</div>

Connection setup is done in priority order (by [search_path](https://steampipe.io/docs/managing/connections#setting-the-search-path)) and is now fully asynchronous. You can start querying your most important connections and aggregators instantly, with full autocomplete, even for your largest [aggregators](https://steampipe.io/docs/managing/connections#querying-multiple-connections).

## Schema optional

Until v0.20.0, Steampipe required each connection to be set up as a separate schema in Postgres. So, if you have 1000 AWS accounts in an aggregator, Steampipe would create 1000 schemas and ~400,000 tables in Postgres! That's a lot of overhead, and it's now optional.

Using `import_schema = "disabled"` we can tell Steampipe to include connections in the aggregator (`aws_all`) but not create separate schemas for them. For the example above, that means only 1 schema and 400 tables are created in Postgres!

```hcl
connection "aws_all" {
  type          = "aggregator"
  plugin        = "aws"
  connections   = ["aws_*"]
}

connection "aws_1" {
  plugin        = "aws"
  import_schema = "disabled"
}

connection "aws_2" {
  plugin        = "aws"
  import_schema = "disabled"
}
```

## Improved mod dependencies

Mod dependencies enable you to [reuse and remix](https://steampipe.io/blog/remixing-dashboards) existing mods. In v0.20.0 we've refined this mechanism to make it more potent for mod authors.

Steampipe's new [AWS Well-Architected](https://hub.steampipe.io/mods/turbot/aws_well_architected) mod, our first major mod to depend on another mod ([AWS Compliance](https://hub.steampipe.io/mods/turbot/aws_compliance)), inspired us to enhance this feature.

A mod may specify dependencies on other mods. While you can manually edit the mod dependencies in the `mod.sp`, they are more commonly managed by Steampipe when you install, update, or uninstall mods via the Steampipe [mod commands](https://steampipe.io/docs/reference/cli/mod). The version can be an exact version or a semver string:

```hcl
require {
  mod "github.com/turbot/steampipe-mod-aws-compliance" {
    version = "^0.10"
  }
  mod "github.com/turbot/steampipe-mod-aws-insights" {
    version = "2.0"
  }
  mod "github.com/turbot/steampipe-mod-gcp-compliance" {
    version = "*"
  }
}
```

You may pass args to set variables defined in the dependency mods. You can pass hard-coded values (literals), however it is more common to define variables in your mod that encapsulate the variables and optionality of your dependencies:

```hcl
variable "common_dimensions" {
  type        = list(string)
  description = "A list of common dimensions to add to each control."
  default     = [ "account_id", "region" ]
}

variable "tag_dimensions" {
  type        = list(string)
  description = "A list of tags to add as dimensions to each control."
  default     = []
}

mod "aws_well_architected" {
  require {
    mod "github.com/turbot/steampipe-mod-aws-compliance" {
      version = "^0.63.0"
      args = {
        common_dimensions = var.common_dimensions,
        tag_dimensions    = var.tag_dimensions
      }
    }
  }
}
```

We've also tightened up some constraints that weren't previously enforced. When a mod refers to a resource in another mod that it depends on, it now must fully qualify the reference: `<modname>.<resourcetype>.<resourcename>`. And a mod can no longer refer to a [local variable](https://steampipe.io/docs/reference/mod-resources/locals#locals) defined in another mod. Note that if your mods were taking advantage of the prior non-enforcement of these rules, you'll need to adjust to them in v0.20.0.

## Time-axis charts

Prior to v0.20.0 it was hard to chart time-series data when there were were gaps in the data: missing hours, days, or years. You could massage the data with Postgres' `generate_series` function but that was an awkward and verbose solution.

Enter José Reyes, a prolific explorer of Steampipe and author of a [remarkable series of deep dives](https://jreyesr.github.io/tags/steampipe/) into various aspects of the system. In [part 3](https://jreyesr.github.io/posts/steampipe-part-int-pi-mods/) he noticed the issue, dug into the server- and client-side machinery, and worked out how to specify when the x axis is of type `time`. In that case, the [chart library](https://echarts.apache.org/en/index.html) can render the series with appropriate gaps. And as a bonus, the labels become cleaner and more adaptive. He writes:

> All that code has been contributed back to Steampipe, in [PR 3214](https://github.com/turbot/steampipe/pull/3214). Hopefully, it will get merged soon!

Indeed it has been. On several internal dashboards we've already taken advantage of the new time axis to simplify queries. One 30-line query that used the `generate_series` method, for example, now clocks in at just 15 lines.

Here's a chart of incidents by day, with appropriate gaps for days with no incidents.

<div style={{"width":"80%"}}>
  <img alt="time-axis chart" src="/images/blog/2023-05-release-0-20-0/incidents_by_day.png" />
</div>

Thanks José!

## … and more

* [Cache optimizations & options](https://steampipe.io/docs/guides/caching)
* Expanded [workspace configuration options](https://steampipe.io/docs/reference/config-files/workspace)
* Other [bug fixes & improvements](https://github.com/turbot/steampipe/blob/main/CHANGELOG.md#v0200-2023-05-18)


## Start your engines!

We're on a journey to make Steampipe ever more performant. The startup optimization in this release is a major milestone, please [let us know](https://steampipe.io/community/join) how it goes for you!
