Major memory reduction, new benchmarks, secure AWS regions →

STEAMPIPE_CACHE_MAX_SIZE_MB

Set the maximum size (in MB) of the query cache across all plugins.

If STEAMPIPE_CACHE_MAX_SIZE_MB is set, Steampipe will limit the query cache across all plugins to the specified size. Each plugin version runs in a separate process, and each plugin process has its own cache. When STEAMPIPE_CACHE_MAX_SIZE_MB is set, Steampipe divides the cache based on the total number of connections and allocates memory shares to each plugin process based on the number of connections for that plugin.

For example, consider the following case:

  • A Steampipe instance has:
    • 10 aws connections
    • 5 azure connections
    • 5 gcp connections
    • 1 github connection
    • 1 rss connection
    • 1 net connection
    • 2 csv connections
  • STEAMPIPE_CACHE_MAX_SIZE_MB is set to 4000

In this example, there are 25 total connections, so each connection's share is 4000 / 25 = 160 MB. The plugin query caches will be capped as follows:

PluginMax Cache Size
aws1600 MB (10 connections x 160 MB)
azure800 MB (5 connections x 160 MB)
gcp800 MB (5 connections x 160 MB)
github160 MB (1 connection x 160 MB)
rss160 MB (1 connection x 160 MB)
net160 MB (1 connection x 160 MB)
csv320 MB (2 connections x 160 MB

By default, Steampipe does not limit the size of the query cache.

STEAMPIPE_CACHE_MAX_SIZE_MB only works with plugins compiled with Steampipe Plugin SDK version 4.0.0 and later.

Usage

Limit cache to 4GB:

export STEAMPIPE_CACHE_MAX_SIZE_MB=4096

Reset caching to unlimited:

unset STEAMPIPE_CACHE_MAX_SIZE_MB