Skip to content

Commit ebaa653

Browse files
bschaatsbergenbschaatsbergen
committed
add data product, teams and users
1 parent f53219c commit ebaa653

34 files changed

Lines changed: 2440 additions & 160 deletions

‎README.md‎

Lines changed: 56 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# Terraform Provider: Marmot
22

3-
The [Marmot Terraform provider](https://registry.terraform.io/providers/marmotdata/marmot/latest/docs)
4-
lets you manage your [Marmot](https://marmotdata.io) instance as code. Use it to
5-
declare assets, the lineage between them, and glossary terms alongside the
6-
rest of your infrastructure.
3+
The [Marmot Terraform provider](https://registry.terraform.io/providers/marmotdata/marmot/0.3.0/docs)
4+
lets you manage your [Marmot](https://marmotdata.io) instance as code. It populates
5+
Marmot from Terraform, letting you declare your Marmot resources alongside the
6+
infrastructure they describe.
77

8-
* [Terraform Registry](https://registry.terraform.io/providers/marmotdata/marmot/latest/docs)
8+
* [Terraform Registry](https://registry.terraform.io/providers/marmotdata/marmot/0.3.0/docs)
99
* [Marmot documentation](https://marmotdata.io/docs)
1010

1111
## Usage
@@ -92,6 +92,57 @@ resource "marmot_glossary_term" "active_customer" {
9292
}
9393
```
9494

95+
## Teams and Users
96+
97+
Manage the teams and users that own catalog entities. A user's password goes
98+
through the write-only `password_wo` attribute (Terraform >= 1.11), so it never
99+
lands in state:
100+
101+
```hcl
102+
resource "marmot_team" "analytics" {
103+
name = "analytics"
104+
}
105+
106+
ephemeral "random_password" "svc" {
107+
length = 24
108+
}
109+
110+
resource "marmot_user" "svc" {
111+
name = "Catalog Service"
112+
username = "svc-catalog"
113+
password_wo = ephemeral.random_password.svc.result
114+
password_wo_version = "1"
115+
}
116+
```
117+
118+
## Data Products
119+
120+
Group related assets into a data product. Add assets directly, or match them
121+
with a rule:
122+
123+
```hcl
124+
resource "marmot_data_product" "orders" {
125+
name = "orders"
126+
description = "Order events and the tables derived from them"
127+
tags = ["orders"]
128+
129+
owner_team_ids = [marmot_team.analytics.id]
130+
}
131+
132+
resource "marmot_data_product_asset" "orders_table" {
133+
data_product_id = marmot_data_product.orders.id
134+
asset_id = marmot_asset.orders_table.id
135+
}
136+
137+
resource "marmot_data_product_rule" "order_datasets" {
138+
data_product_id = marmot_data_product.orders.id
139+
140+
name = "order-datasets"
141+
type = "query"
142+
query_expression = "tag:orders"
143+
}
144+
```
145+
95146
## Requirements
96147

97148
* [Terraform](https://developer.hashicorp.com/terraform/downloads) >= 1.0

‎docs/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,14 @@
33
page_title: "marmot Provider"
44
description: |-
55
Manage your Marmot https://marmotdata.io catalog as code. Marmot is the open-source context layer for agents and humans: it catalogs every service, API, queue, topic, database, and pipeline in your organization, storing only metadata such as schemas, ownership, descriptions, and lineage.
6-
This provider populates Marmot from Terraform, letting you declare assets, the lineage between them, and glossary terms alongside the infrastructure they describe.
6+
This provider populates Marmot from Terraform, letting you declare your Marmot resources alongside the infrastructure they describe.
77
---
88

99
# marmot Provider
1010

1111
Manage your [Marmot](https://marmotdata.io) catalog as code. Marmot is the open-source context layer for agents and humans: it catalogs every service, API, queue, topic, database, and pipeline in your organization, storing only metadata such as schemas, ownership, descriptions, and lineage.
1212

13-
This provider populates Marmot from Terraform, letting you declare assets, the lineage between them, and glossary terms alongside the infrastructure they describe.
13+
This provider populates Marmot from Terraform, letting you declare your Marmot resources alongside the infrastructure they describe.
1414

1515
## Usage
1616

‎docs/resources/asset.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,3 +127,14 @@ Optional:
127127

128128
- `priority` (Number) Priority of the source
129129
- `properties` (Map of String) Properties of the source
130+
131+
## Import
132+
133+
Import is supported using the following syntax:
134+
135+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
136+
137+
```shell
138+
# Assets are imported by their ID.
139+
terraform import marmot_asset.example 018e1234-5678-7abc-def0-123456789abc
140+
```

‎docs/resources/data_product.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
# generated by https://github.com/hashicorp/terraform-plugin-docs
3+
page_title: "marmot_data_product Resource - marmot"
4+
subcategory: ""
5+
description: |-
6+
Groups related assets into a data product. Add assets to it directly with marmot_data_product_asset, or match them dynamically with marmot_data_product_rule.
7+
---
8+
9+
# marmot_data_product (Resource)
10+
11+
Groups related assets into a data product. Add assets to it directly with `marmot_data_product_asset`, or match them dynamically with `marmot_data_product_rule`.
12+
13+
## Example Usage
14+
15+
```terraform
16+
resource "marmot_team" "analytics" {
17+
name = "analytics"
18+
description = "Owns the reporting datasets"
19+
}
20+
21+
resource "marmot_data_product" "orders" {
22+
name = "orders"
23+
description = "Order events and the tables derived from them"
24+
25+
tags = ["orders"]
26+
27+
owner_team_ids = [marmot_team.analytics.id]
28+
29+
metadata = {
30+
domain = "commerce"
31+
}
32+
}
33+
34+
resource "marmot_asset" "orders_table" {
35+
name = "orders"
36+
type = "dataset"
37+
services = ["PostgreSQL"]
38+
}
39+
40+
# Add an asset directly.
41+
resource "marmot_data_product_asset" "orders_table" {
42+
data_product_id = marmot_data_product.orders.id
43+
asset_id = marmot_asset.orders_table.id
44+
}
45+
46+
# Or pull assets in by query.
47+
resource "marmot_data_product_rule" "order_datasets" {
48+
data_product_id = marmot_data_product.orders.id
49+
50+
name = "order-datasets"
51+
type = "query"
52+
query_expression = "tag:orders"
53+
}
54+
```
55+
56+
<!-- schema generated by tfplugindocs -->
57+
## Schema
58+
59+
### Required
60+
61+
- `name` (String) Name of the data product
62+
63+
### Optional
64+
65+
- `description` (String) Description of the data product
66+
- `metadata` (Map of String) Key/value metadata for the data product. The API can't clear metadata on update: once set, removing every key leaves the old values in place until the product is replaced.
67+
- `owner_team_ids` (Set of String) IDs of teams that own the data product.
68+
- `owner_user_ids` (Set of String) IDs of users that own the data product. Defaults to the calling user when no owners are set.
69+
- `tags` (Set of String) Tags associated with the data product
70+
71+
### Read-Only
72+
73+
- `created_at` (String) Creation timestamp
74+
- `id` (String) Data product ID
75+
- `updated_at` (String) Last update timestamp
76+
77+
## Import
78+
79+
Import is supported using the following syntax:
80+
81+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
82+
83+
```shell
84+
# Data products are imported by their ID.
85+
terraform import marmot_data_product.example 018e1234-5678-7abc-def0-123456789abc
86+
```

‎docs/resources/data_product_asset.md‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
---
2+
# generated by https://github.com/hashicorp/terraform-plugin-docs
3+
page_title: "marmot_data_product_asset Resource - marmot"
4+
subcategory: ""
5+
description: |-
6+
Adds a single asset to a data product by hand. For rule-based membership, use marmot_data_product_rule.
7+
---
8+
9+
# marmot_data_product_asset (Resource)
10+
11+
Adds a single asset to a data product by hand. For rule-based membership, use `marmot_data_product_rule`.
12+
13+
## Example Usage
14+
15+
```terraform
16+
resource "marmot_data_product" "orders" {
17+
name = "orders"
18+
}
19+
20+
resource "marmot_asset" "orders_table" {
21+
name = "orders"
22+
type = "dataset"
23+
services = ["PostgreSQL"]
24+
}
25+
26+
# Add the asset to the data product.
27+
resource "marmot_data_product_asset" "orders_table" {
28+
data_product_id = marmot_data_product.orders.id
29+
asset_id = marmot_asset.orders_table.id
30+
}
31+
```
32+
33+
<!-- schema generated by tfplugindocs -->
34+
## Schema
35+
36+
### Required
37+
38+
- `asset_id` (String) ID of the asset to add to the data product
39+
- `data_product_id` (String) ID of the data product
40+
41+
## Import
42+
43+
Import is supported using the following syntax:
44+
45+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
46+
47+
```shell
48+
# Memberships are imported with the composite ID "<data_product_id>/<asset_id>".
49+
terraform import marmot_data_product_asset.example 018e1234-5678-7abc-def0-123456789abc/018eabcd-1234-7def-8901-23456789abcd
50+
```

‎docs/resources/data_product_rule.md‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
---
2+
# generated by https://github.com/hashicorp/terraform-plugin-docs
3+
page_title: "marmot_data_product_rule Resource - marmot"
4+
subcategory: ""
5+
description: |-
6+
Pulls assets into a data product dynamically, either with a search query or by matching a metadata field against a pattern. A data product can have up to 10 rules.
7+
---
8+
9+
# marmot_data_product_rule (Resource)
10+
11+
Pulls assets into a data product dynamically, either with a search query or by matching a metadata field against a pattern. A data product can have up to 10 rules.
12+
13+
## Example Usage
14+
15+
```terraform
16+
resource "marmot_data_product" "orders" {
17+
name = "orders"
18+
}
19+
20+
# Match assets with a search query.
21+
resource "marmot_data_product_rule" "by_query" {
22+
data_product_id = marmot_data_product.orders.id
23+
24+
name = "order-datasets"
25+
description = "Datasets tagged orders"
26+
type = "query"
27+
query_expression = "tag:orders"
28+
}
29+
30+
# Match assets on a metadata field.
31+
resource "marmot_data_product_rule" "by_metadata" {
32+
data_product_id = marmot_data_product.orders.id
33+
34+
name = "commerce-domain"
35+
type = "metadata_match"
36+
metadata_field = "domain"
37+
pattern_type = "exact"
38+
pattern_value = "commerce"
39+
priority = 10
40+
}
41+
```
42+
43+
<!-- schema generated by tfplugindocs -->
44+
## Schema
45+
46+
### Required
47+
48+
- `data_product_id` (String) ID of the data product the rule belongs to
49+
- `name` (String) Name of the rule
50+
- `type` (String) `query` to match assets with a search query, or `metadata_match` to match a metadata field against a pattern.
51+
52+
### Optional
53+
54+
- `description` (String) Description of the rule
55+
- `enabled` (Boolean) Whether the rule is evaluated. Defaults to `true`.
56+
- `metadata_field` (String) Metadata field to match for 'metadata_match' rules
57+
- `pattern_type` (String) How the pattern is matched for 'metadata_match' rules: 'exact', 'wildcard', 'regex', or 'prefix'
58+
- `pattern_value` (String) Pattern to match the metadata field against for 'metadata_match' rules
59+
- `priority` (Number) Priority of the rule. Defaults to `0`.
60+
- `query_expression` (String) Search query for 'query' rules
61+
62+
### Read-Only
63+
64+
- `created_at` (String) Creation timestamp
65+
- `id` (String) Rule ID
66+
- `updated_at` (String) Last update timestamp
67+
68+
## Import
69+
70+
Import is supported using the following syntax:
71+
72+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
73+
74+
```shell
75+
# Rules are imported with the composite ID "<data_product_id>/<rule_id>".
76+
terraform import marmot_data_product_rule.query 018e1234-5678-7abc-def0-123456789abc/018eabcd-1234-7def-8901-23456789abcd
77+
```

‎docs/resources/glossary_term.md‎

Lines changed: 34 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,31 @@ description: |-
1010

1111
Glossary term resource for defining business terminology
1212

13+
## Example Usage
1314

15+
```terraform
16+
resource "marmot_team" "analytics" {
17+
name = "analytics"
18+
}
19+
20+
resource "marmot_glossary_term" "active_customer" {
21+
name = "Active Customer"
22+
definition = "A customer with at least one order in the last 90 days."
23+
24+
owner_team_ids = [marmot_team.analytics.id]
25+
26+
metadata = {
27+
domain = "sales"
28+
}
29+
}
30+
31+
# Terms can be organized hierarchically.
32+
resource "marmot_glossary_term" "churned_customer" {
33+
name = "Churned Customer"
34+
definition = "An active customer who has not ordered in the last 90 days."
35+
parent_term_id = marmot_glossary_term.active_customer.id
36+
}
37+
```
1438

1539
<!-- schema generated by tfplugindocs -->
1640
## Schema
@@ -24,7 +48,8 @@ Glossary term resource for defining business terminology
2448

2549
- `description` (String) Additional description for the glossary term
2650
- `metadata` (Map of String) Metadata associated with the glossary term
27-
- `owners` (Attributes List) Owners of the glossary term (see [below for nested schema](#nestedatt--owners))
51+
- `owner_team_ids` (Set of String) IDs of teams that own the term.
52+
- `owner_user_ids` (Set of String) IDs of users that own the term. Defaults to the calling user when no owners are set.
2853
- `parent_term_id` (String) ID of the parent glossary term for hierarchical organization
2954

3055
### Read-Only
@@ -33,10 +58,13 @@ Glossary term resource for defining business terminology
3358
- `id` (String) Glossary term ID
3459
- `updated_at` (String) Last update timestamp
3560

36-
<a id="nestedatt--owners"></a>
37-
### Nested Schema for `owners`
61+
## Import
62+
63+
Import is supported using the following syntax:
3864

39-
Required:
65+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
4066

41-
- `id` (String) ID of the owner (user or team ID)
42-
- `type` (String) Type of owner: 'user' or 'team'
67+
```shell
68+
# Glossary terms are imported by their ID.
69+
terraform import marmot_glossary_term.active_customer 018e1234-5678-7abc-def0-123456789abc
70+
```

‎docs/resources/lineage.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,14 @@ resource "marmot_asset" "target" {
4242
### Read-Only
4343

4444
- `id` (String) Lineage ID
45+
46+
## Import
47+
48+
Import is supported using the following syntax:
49+
50+
The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
51+
52+
```shell
53+
# Lineage edges are imported by their ID.
54+
terraform import marmot_lineage.example 018e1234-5678-7abc-def0-123456789abc
55+
```

0 commit comments

Comments
 (0)