v0.18.0: Relationship graphs →

flow

A flow allows visualization of queries using types such as sankey. Flows are node/edge visualizations. The data to be displayed is specified using a series of nodes and edges. The nodes define the vertices of the graph, and the edges define the connections between them.

Flow blocks can be declared as named resources at the top level of a mod, or can be declared as anonymous blocks inside a dashboard or container, or be re-used inside a dashboard or container by using a flow with base = <mod>.flow.<flow_resource_name>.

Example Usage

dashboard "flow_ex_node_edge" {
input "vpc_input" {
width = 4
sql = <<-EOQ
select
title as label,
vpc_id as value
from
aws_vpc
EOQ
}
flow {
title = "AWS VPC Subnets by AZ"
node {
sql = <<-EOQ
select
vpc_id as id,
vpc_id as title
from
aws_vpc
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
node {
sql = <<-EOQ
select
distinct on (availability_zone)
availability_zone as id,
availability_zone as title
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
node {
sql = <<-EOQ
select
subnet_id as id,
subnet_id as title
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
edge {
sql = <<-EOQ
select
distinct on (availability_zone)
vpc_id as from_id,
availability_zone as to_id
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
edge {
sql = <<-EOQ
select
availability_zone as from_id,
subnet_id as to_id
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
}
}

Argument Reference

ArgumentTypeOptional?Description
argsMapOptionalA map of arguments to pass to the query.
baseflow ReferenceOptionalA reference to a named flow resource that this flow should source its definition from. title and width can be overridden after sourcing via base.
categoryBlockOptionalcategory blocks that specify display options for nodes with that category.
edgeBlockOptionaledge blocks that define the edges in the flow.
nodeBlockOptionalnode blocks that define the nodes in the flow.
paramBlockOptionalparam blocks that defines the parameters that can be passed in to the query. param blocks may only be specified for hierarchies that specify the sql argument.
queryQuery ReferenceOptionalA reference to a query resource that defines the query to run. A flow may either specify the query argument or the sql argument, but not both.
sqlStringOptionalAn SQL string to provide data for the flow. A flow may either specify the query argument or the sql argument, but not both.
titleStringOptionalA plain text title to display for this flow.
typeStringOptionalThe type of the flow. Can be sankey or table.
widthNumberOptionalThe width as a number of grid units that this item should consume from its parent.
withBlockOptionalwith blocks that define prerequisite queries to run. with blocks may only be specified when the flow is defined as a top-level (mod level), named resource.

Common Flow Properties

category

PropertyTypeDefaultValuesDescription
colorstringThe matching color from the default theme for the data series index.A valid color value. This may be a named color, RGB or RGBA string, or a control status color.The color to display for this category.

Data Format

Flow data must be provided in a format where each row represents a node (vertex), an edge (connecting 2 vertices), or both.

Note that both column names and their relative position are important in flow queries; Steampipe looks for columns by name in the result set, however Postgres union queries will append the rows based on the column's position, not the name of the column. All the union queries must return the same columns, in the same order.

Significant columns are:

NameDescription
idA unique identifier for the node. Nodes have an id, edges do not.
titleA title to display for the node.
categoryA display category name. Both nodes and edges may specify a category to dictate how the item is displayed. By default, items of the same category are displayed with the same appearance (color), distinct from other categories. You can specify display options with a category block.
depthAn integer to set the position of the node. The layout of the nodes is inferred from the query, however you can force placement with the depth column if you need to override the default behavior.
from_idThe id of the source side of an edge.
to_idThe id of the destination side of an edge.

Generally speaking, there are 2 data formats commonly used for flows. If the data is hierarchical, it is often simpler to specify results where each row species a node (with an id, and optionally title, category, and/or depth) and an edge, by specifying a from_id:

from_ididtitlecategory
<null>1fooroot
12barwidget
13bazwidget
24foobarfidget

For flows that do not conform to a single-parent hierarchical structure, its usually easier to specify nodes and edges as separate rows. In this case, nodes will have an id and optionally title, category, and/or depth, but to_id and from_id will be null. Edges will populate to_id and from_id and optionally category, and will have null id, depth, and title:

from_idto_ididtitlecategory
<null><null>1fooroot
<null><null>2barwidget
<null><null>3bazwidget
<null><null>4foobarfidget
12<null><null>widget
13<null><null>widget
24<null><null>fidget
34<null><null>fidget

More Examples

Sankey with color by category

dashboard "flow_ex_node_edge_category" {
input "vpc_input" {
width = 4
sql = <<-EOQ
select
title as label,
vpc_id as value
from
aws_vpc
EOQ
}
flow {
title = "AWS VPC Subnets by AZ"
node {
category = category.aws_vpc
sql = <<-EOQ
select
vpc_id as id,
vpc_id as title
from
aws_vpc
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
node {
category = category.aws_availability_zone
sql = <<-EOQ
select
distinct on (availability_zone)
availability_zone as id,
availability_zone as title
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
node {
category = category.aws_vpc_subnet
sql = <<-EOQ
select
subnet_id as id,
subnet_id as title
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
edge {
sql = <<-EOQ
select
distinct on (availability_zone)
vpc_id as from_id,
availability_zone as to_id
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
edge {
sql = <<-EOQ
select
availability_zone as from_id,
subnet_id as to_id
from
aws_vpc_subnet
where
vpc_id = $1
EOQ
args = [self.input.vpc_input.value]
}
}
}
category "aws_vpc" {
color = "orange"
}
category "aws_availability_zone" {
color = "tan"
}
category "aws_vpc_subnet" {
color = "green"
}

sankey with monolithic query

flow {
type = "sankey"
title = "AWS VPC Subnets by AZ"
width = 6
sql = <<-EOQ
with vpc as
(select 'vpc-9d7ae1e7' as vpc_id)
select
null as from_id,
vpc_id as id,
vpc_id as title,
0 as depth,
'aws_vpc' as category
from
aws_vpc
where
vpc_id in (select vpc_id from vpc)
union all
select
distinct on (availability_zone)
vpc_id as from_id,
availability_zone as id,
availability_zone as title,
1 as depth,
'aws_availability_zone' as category
from
aws_vpc_subnet
where
vpc_id in (select vpc_id from vpc)
union all
select
availability_zone as from_id,
subnet_id as id,
subnet_id as title,
2 as depth,
'aws_vpc_subnet' as category
from
aws_vpc_subnet
where
vpc_id in (select vpc_id from vpc)
EOQ
}

Sankey with color by category (monolithic query)

flow {
type = "sankey"
title = "AWS VPC Subnets by AZ"
width = 6
category "aws_vpc" {
color = "orange"
}
category "aws_availability_zone" {
color = "tan"
}
category "aws_vpc_subnet" {
color = "green"
}
sql = <<-EOQ
with vpc as
(select 'vpc-9d7ae1e7' as vpc_id)
select
null as from_id,
vpc_id as id,
vpc_id as title,
0 as depth,
'aws_vpc' as category
from
aws_vpc
where
vpc_id in (select vpc_id from vpc)
union all
select
distinct on (availability_zone)
vpc_id as from_id,
availability_zone as id,
availability_zone as title,
1 as depth,
'aws_availability_zone' as category
from
aws_vpc_subnet
where
vpc_id in (select vpc_id from vpc)
union all
select
availability_zone as from_id,
subnet_id as id,
subnet_id as title,
2 as depth,
'aws_vpc_subnet' as category
from
aws_vpc_subnet
where
vpc_id in (select vpc_id from vpc)
EOQ
}

Sankey with node / edge data format, color by category, depth

flow {
type = "sankey"
title = "AWS IAM Managed Policies for User"
width = 6
category "direct" {
color = "alert"
}
category "indirect" {
color = "ok"
}
sql = <<-EOQ
with user_list as
(select 'arn:aws:iam::111111111111:user/jsmyth' as arn)
-- User Nodes
select
arn as id,
name as title,
0 as depth,
'user' as category,
null as from_id,
null as to_id
from
aws_iam_user
where
arn in (select arn from user_list)
-- Group Nodes
union select
g ->> 'Arn' as id,
g ->> 'GroupName' as title,
1 as depth,
'group' as category,
null as from_id,
null as to_id
from
aws_iam_user,
jsonb_array_elements(groups) as g
where
arn in (select arn from user_list)
-- Policy Nodes (attached to groups)
union select
p.arn as id,
p.name as title,
2 as depth,
'policy' as category,
null as from_id,
null as to_id
from
aws_iam_user as u,
jsonb_array_elements(groups) as g,
aws_iam_group as grp,
jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn,
aws_iam_policy as p
where
g ->> 'Arn' = grp.arn
and pol_arn = p.arn
and u.arn in (select arn from user_list)
-- Policy Nodes (attached to user)
union select
p.arn as id,
p.name as title,
2 as depth,
'policy' as category,
null as from_id,
null as to_id
from
aws_iam_user as u,
jsonb_array_elements_text(attached_policy_arns) as pol_arn,
aws_iam_policy as p
where
pol_arn = p.arn
and u.arn in (select arn from user_list)
-- User-> Group Edge
union select
null as id,
null as title,
null as depth,
'indirect' as category,
arn as from_id,
g ->> 'Arn' as to_id
from
aws_iam_user,
jsonb_array_elements(groups) as g
where
arn in (select arn from user_list)
-- User -> Policy Edge
union select
null as id,
null as title,
null as depth,
'direct' as category,
arn as from_id,
pol_arn
from
aws_iam_user,
jsonb_array_elements_text(attached_policy_arns) as pol_arn
where
arn in (select arn from user_list)
-- Group -> Policy Edge
union select
null as id,
null as title,
null as depth,
'indirect' as category,
grp.arn as from_id,
pol_arn
from
aws_iam_user as u,
jsonb_array_elements(groups) as g,
aws_iam_group as grp,
jsonb_array_elements_text(grp.attached_policy_arns) as pol_arn,
aws_iam_policy as p
where
g ->> 'Arn' = grp.arn
and pol_arn = p.arn
and u.arn in (select arn from user_list)
EOQ
}