Plugin Release Checklist
As of May 2022, we've absorbed 80+ plugins into the hub. If you want to contribute one -- and we hope you do! -- here are the most common things we ask contributors to check to prepare for the plugin's release. Feel free to tick the boxes as you go through the list!
- Basic Configuration
- Configuration File
- Table and Column Names
- Table and Column Descriptions
- Table and Column Design
- Final Review
Basic ConfigurationRepository name
The repository name should use the format
steampipe-plugin-microsoft365. The plugin name should be one word, so there are always 3 parts in the repository name.
The Go version in
go.mod and any workflows is 1.19.
.goreleaser.yml file uses the standard format, e.g., AWS plugin .goreleaser.yml.
CHANGELOG.md is included and contains release notes for the upcoming version (typically v0.0.1).
The plugin uses the Apache License 2.0.Makefile
Makefile file is present and builds to the correct plugin path.
Configuration File.spc examples
config/PLUGIN.spc file is neatly formatted, and explains each argument with links as appropriate, using realistic values, e.g., "xoxp-abcads…" instead of "TOKEN_HERE".
Arguments that can also be set via environment variable include the environment variable name(s) in their descriptions.
If there's a Terraform provider for your API, the plugin supports the same credential methods as the provider.Existing CLI credentials
When there are commonly used CLI credentials, like
.aws/credentials, the plugin works with them.
When credentials expire, and the API's SDK does not automatically refresh them, the plugin alerts the user and tells them how to refresh.Environment variables
It's possible to set credentials using an environment variable if the API's SDK also supports using environment variables.
Table and Column NamesStandard names
All table and column names follow our Table & Column Naming Standards.
Table and Column DescriptionsDescriptions
Every table and column has a description. These are consistent across tables.Other standards
All descriptions adhere to the Table and Column Descriptions Standards.
Table and Column DesignGlobal and per-authenticated-user data
Many plugins can return both global data, e.g., all GitHub repos or Google Drive files, and data only for the authenticated user (my repos, my files). If that's the case, there are separate tables, e.g.
If tables share columns, these are abstracted as shown in the AWS plugin's common_columns.go.Required configuration arguments
The plugin checks required configuration arguments are set once at load time.
When the plugin returns an error, it includes the location and any related args, along with the error itself. See example.
Data IngestionDefault transform
The plugin sets a preferred transform as the default. For example, the GitLab plugin uses DefaultTransform: transform.FromGo().NullIfZero(). Please see Transform Functions for a full list of transform functions.Pagination
The plugin implements pagination in each table's List function supported by the API's SDK. If pagination is implemented, the plugin sets the page size per request to the maximum allowed; however, if
QueryContext.Limit is smaller than that page size, the page size should be set to the limit. See example.
If a non-List hydrate function requires paging, consider separating that data into a separate table. Columns that require separate hydrate data that uses paging can lead to throttling and rate limiting errors unexpectedly.Backoff and retry
If the API SDK doesn't automate backoff and retry, the plugin leverages capabilities of the Steampipe plugin SDK's RetryHydrate function. For instance, the
github_issue table uses this function when listing issues due to the strict throttling of the GitHub API.
If the API has strict rate limiting, the table sets HydrateConfig.MaxConcurrency for the relevant hydrate functions. For instance, the
googleworkspace_gmail_message table limits the number of getGmailMessage calls.
Each table's list hydrate function checks for remaining rows from the API SDK, and aborts inside loops (e.g., while streaming items) if there are none. (See example.)
Money is represented as a string, not a double which is never exact.
Index DocumentationFront matter
The index document contains a front matter block, like the one below:
Front matter: category
---organization: Turbotcategory: ["security"]icon_url: "/images/plugins/turbot/duo.svg"brand_color: "#6BBF4E"display_name: Duo Securityname: duodescription: Steampipe plugin for querying Duo Security users, logs and more.og_description: Query Duo Security with SQL! Open source CLI. No DB required.og_image: "/images/plugins/turbot/duo-social-graphic.png"---
The category is an appropriate choice from the list at hub.steampipe.io/plugins.Front matter: icon_url
The icon URL is a link to an
.svg file hosted on hub.steampipe.io. Please request an icon through the Steampipe Slack and a URL will be provided to use in this variable.
The color matches the provider's brand guidelines, typically stated on a page like this one for Twilio.Plugin description
The description in
docs/index.md is appropriate for the provider. The AWS plugin, for example, uses:
AWS provides on-demand cloud computing platforms and APIs to authenticated customers on a metered pay-as-you-go basis.
The opening sentence of the Wikipedia page for the provider can be a good source of guidance here.Credentials
Credentials are the most important piece of documentation. The plugin:
- Explains scopes and required permissions
- Links to provider documentation
- Explains how to use existing CLI creds when that's possible
Table DocumentationUseful examples
Each table document shows 4 - 5 useful examples that reflect real-world scenarios. Please see Writing Example Queries for common patterns and samples.Column specificity
Most examples specify columns. Using
SELECT * is OK for one or two things, but generally not preferred as it can produce too much data to be helpful. See also When Not to SELECT *.
If some columns are required, these are called out and explained.
The plugin has been tested on a real account with substantial data. Please note that errors and API throttling issues may not appear when using a test account with little data.Matching query examples
The example in
README.md matches the one in
The example in
config/PLUGIN.spc matches the one in
The social graphic is included at the top of the README file and is uploaded to the Social preview feature in the GitHub repository. Please request a social graphic through the Steampipe Slack.Ease of first use
The plugin really nails easy setup, there's a short path to a first successful query, and it runs quickly.Pre-mortem
You've considered, and addressed, reasons why this plugin could fail to delight its community.