Plugin Release Checklist
As of October 2023, we've absorbed 135+ plugins into the Hub. If you want to contribute one -- and we hope you do! -- here are the most common things we ask contributors to check to prepare for the plugin's release. Feel free to tick the boxes as you go through the list!
- Basic Configuration
- Configuration File
- Table and Column Names
- Table and Column Descriptions
- Table and Column Design
- Final Review
Basic ConfigurationRepository name
The repository name should use the format
steampipe-plugin-microsoft365. The plugin name should be one word, so there are always 3 parts in the repository name.
To help with discoverability in GitHub, the repository topics should include:
The repository website/homepage should link to the Hub site. The URL is composed of the GitHub organization and plugin name, for instance:
- https://github.com/turbot/steampipe-plugin-aws results in https://hub.steampipe.io/plugins/turbot/aws
- https://github.com/francois2metz/steampipe-plugin-airtable results in https://hub.steampipe.io/plugins/francois2metz/airtable
The Go version in
go.mod and any workflows is 1.21.
.goreleaser.yml file uses the standard format, e.g., AWS plugin .goreleaser.yml.
CHANGELOG.md is included and contains release notes for the upcoming version (typically v0.0.1).
The plugin uses the Apache License 2.0.Makefile
Makefile file is present and builds to the correct plugin path.
Configuration File.spc examples
config/PLUGIN.spc file is neatly formatted, and explains each argument with links as appropriate, using realistic values, e.g., "xoxp-abcads…" instead of "TOKEN_HERE".
Arguments that can also be set via environment variable include the environment variable name(s) in their descriptions.
If there's a Terraform provider for your API, the plugin supports the same credential methods as the provider.Existing CLI credentials
When there are commonly used CLI credentials, like
.aws/credentials, the plugin works with them.
When credentials expire, and the API's SDK does not automatically refresh them, the plugin alerts the user and tells them how to refresh.Environment variables
It's possible to set credentials using an environment variable if the API's SDK also supports using environment variables.
Table and Column NamesStandard names
All table and column names follow our Table & Column Naming Standards.
Table and Column DescriptionsDescriptions
Every table and column has a description. These are consistent across tables.Other standards
All descriptions adhere to the Table and Column Descriptions Standards.
Table and Column DesignGlobal and per-authenticated-user data
Many plugins can return both global data, e.g., all GitHub repos or Google Drive files, and data only for the authenticated user (my repos, my files). If that's the case, there are separate tables, e.g.
If tables share columns, these are abstracted as shown in the AWS plugin's common_columns.go.Required configuration arguments
The plugin checks required configuration arguments are set once at load time.
When the plugin returns an error, it includes the location and any related args, along with the error itself. See example.
Data IngestionDefault transform
The plugin sets a preferred transform as the default. For example, the GitLab plugin uses DefaultTransform: transform.FromGo().NullIfZero(). Please see Transform Functions for a full list of transform functions.Pagination
The plugin implements pagination in each table's List function supported by the API's SDK. If pagination is implemented, the plugin sets the page size per request to the maximum allowed; however, if
QueryContext.Limit is smaller than that page size, the page size should be set to the limit. See example.
If a non-List hydrate function requires paging, consider separating that data into a separate table. Columns that require separate hydrate data that uses paging can lead to throttling and rate limiting errors unexpectedly.Backoff and retry
If the API SDK doesn't automate backoff and retry, the plugin leverages capabilities of the Steampipe plugin SDK's RetryHydrate function. For instance, the
github_issue table uses this function when listing issues due to the strict throttling of the GitHub API.
If the API has strict rate limiting, the table sets HydrateConfig.MaxConcurrency for the relevant hydrate functions. For instance, the
googleworkspace_gmail_message table limits the number of getGmailMessage calls.
Each table's list hydrate function checks for remaining rows from the API SDK, and aborts inside loops (e.g., while streaming items) if there are none. (See example.)
Money is represented as a string, not a double which is never exact.
Dynamic TablesSpecifying tables to generate
If the plugin can generate dynamic tables, a configuration argument should allow users to specify which tables the plugin will generate. This configuration argument typically accepts a list of strings and should support filesystem glob patterns like in the CSV plugin.
If this configuration argument is not set or is explicitly empty, e.g.,
paths = , then no dynamic tables should be generated.
The plugin should determine if it will generate dynamic tables by default after plugin installation based on if the configuration argument mentioned above is commented by default. For instance, in the Prometheus plugin, the
metrics configuration argument is commented. After plugin installation, the plugin will not generate dynamic tables unless the user adds a non-commented value for
You may not want to load dynamic tables by default if it drastically increases the plugin initialization time due to the number of tables.Table name prefixes
When naming dynamic tables, the plugin name prefix, e.g.,
kubernetes_, should be added if it helps avoid namespace collisions or if it helps group them with static tables that share the same prefix.
Index DocumentationFront matter
The index document contains a front matter block, like the one below:
Front matter: category
---organization: Turbotcategory: ["security"]icon_url: "/images/plugins/turbot/duo.svg"brand_color: "#6BBF4E"display_name: Duo Securityname: duodescription: Steampipe plugin for querying Duo Security users, logs and more.og_description: Query Duo Security with SQL! Open source CLI. No DB required.og_image: "/images/plugins/turbot/duo-social-graphic.png"---
The category is an appropriate choice from the list at hub.steampipe.io/plugins.Front matter: icon_url
The icon URL is a link to an
.svg file hosted on hub.steampipe.io. Please request an icon through the Turbot Community Slack and a URL will be provided to use in this variable.
The color matches the provider's brand guidelines, typically stated on a page like this one for Twilio.Plugin description
The description in
docs/index.md is appropriate for the provider. The AWS plugin, for example, uses:
AWS provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.
The opening sentence of the Wikipedia page for the provider can be a good source of guidance here.Credentials
Credentials are the most important piece of documentation. The plugin:
- Explains scopes and required permissions
- Links to provider documentation
- Explains how to use existing CLI creds when that's possible
For plugins that benefit from using multiple connections and aggregators, like the AWS plugin, one or more H2 sections with examples should be added to the index document so users can easily reference it. For instance, the AWS plugin has examples in AWS Multi-Account Connections.
If a plugin doesn't strongly benefit from aggregator connections, an H2 section called
Multiple Connections should be added that briefly talks about aggregators and has a link to Using Aggregators.
Table DocumentationUseful examples
Each table document shows 4 - 5 useful examples that reflect real-world scenarios. Please see Writing Example Queries for common patterns and samples.Column specificity
Most examples specify columns. Using
SELECT * is OK for one or two things, but generally not preferred as it can produce too much data to be helpful. See also When Not to SELECT *.
If some columns are required, these are called out and explained.
The plugin has been tested on a real account with substantial data. Please note that errors and API throttling issues may not appear when using a test account with little data.Matching query examples
The example in
README.md matches the one in
The example in
config/PLUGIN.spc matches the one in
The social graphic is included at the top of the README file and is uploaded to the Social preview feature in the GitHub repository. Please request a social graphic through the Steampipe Slack.Ease of first use
The plugin really nails easy setup, there's a short path to a first successful query, and it runs quickly.Pre-mortem
You've considered, and addressed, reasons why this plugin could fail to delight its community.