You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* docs: specify cost guardrails and global tagging
* feat: add cost and tagging guardrails
* fix(sdk): make global tagging opt-in
Keep account-wide tagging available for explicit enablement while preserving the supported local-only Resource Explorer setup for default discovery.
* refactor(sdk): share resource explorer plumbing
Deduplicate Resource Explorer view resolution, aggregator requirements, paginated resource collection, and discovery initialization finalization.
---------
Co-authored-by: Danny Steenman <danny@towardsthecloud.com>
Copy file name to clipboardExpand all lines: README.md
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -61,7 +61,7 @@ npx cloudburn scan ./main.tf
61
61
62
62
### Config
63
63
64
-
Config is optional. By default, CloudBurn runs all checks for the mode you use.
64
+
Config is optional. By default, CloudBurn runs the AWS Core preset for the mode you use. Account-wide rules such as `CLDBRN-AWS-TAGGING-1` are opt-in through `enabled-rules` because they require an accessible Resource Explorer aggregator.
The CLI targets one region at a time. Multi-region discovery remains available through the SDK.
133
+
`CLDBRN-AWS-TAGGING-1`is opt-in and requires an accessible Resource Explorer aggregator; a local-only setup cannot run account-wide tagging discovery.
132
134
Use `--debug` to relay SDK and provider execution trace details to `stderr` while keeping normal command output on `stdout`.
133
135
134
136
Generate a starter config with `cloudburn config --init`. Full details in the [config reference](docs/reference/config-schema.md).
135
137
136
138
## AWS Permissions
137
139
138
-
CloudBurn needs Resource Explorer read/write access plus read-only permissions for the services behind the rules you enable (EC2, EBS, RDS, S3, Lambda, CloudTrail, CloudWatch, etc.). Which permissions you need depends on which rules you're running.
140
+
CloudBurn needs Resource Explorer read/write access plus read-only permissions for the services behind the rules you enable (EC2, EBS, RDS, S3, Lambda, CloudTrail, CloudWatch, etc.). `discover init` uses `resource-explorer-2:UpdateView` to enable tag visibility for the opt-in global tagging rule. Which permissions you need depends on which rules you're running.
|**1. Export surface**|`exports.test.ts`|`awsRules` is non-empty, preset inclusion policy holds, `azureRules`/`gcpRules` are empty |
18
+
|**2. Metadata contract**|`rule-metadata.test.ts`| Every rule has non-empty `id`, `name`, `description`, and `supports`|
19
+
|**3. Evaluator behavior**|`{rule-name}.test.ts`| Full finding payloads for both `evaluateLive` and `evaluateStatic`, plus negative cases |
20
20
21
21
For static IaC rules, evaluator coverage must include both Terraform-shaped and CloudFormation-shaped resources. A passing test suite for only one source kind is incomplete.
Copy file name to clipboardExpand all lines: docs/architecture/rules.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -71,6 +71,8 @@ graph LR
71
71
Preset --> Export
72
72
```
73
73
74
+
`awsRules` is the complete public AWS rule pack. `awsCorePreset` is the runtime default and may exclude opt-in rules whose infrastructure requirements are not guaranteed by a standard setup.
75
+
74
76
## Authoring Rules
75
77
76
78
See [`docs/guides/adding-a-rule.md`](../guides/adding-a-rule.md) for the full end-to-end guide and [`docs/reference/rule-ids.md`](../reference/rule-ids.md) for the ID convention and complete rule table.
Copy file name to clipboardExpand all lines: docs/architecture/sdk.md
+4-2Lines changed: 4 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -78,15 +78,17 @@ Current live-discovery behavior:
78
78
- Explicit single-region discovery uses the selected region as the Resource Explorer control plane instead of the ambient current region.
79
79
- Explicit multi-region discovery requires an aggregator index and fails fast when one is not enabled.
80
80
- Discovery resolves the explicit default Resource Explorer view in the chosen search region and fails if no default view exists or if that default view applies additional filters.
81
+
-`discover init` ensures the default Resource Explorer view includes the optional `tags` property. Tag-aware datasets fail with an actionable error when an existing view does not expose tags.
81
82
- Discovery setup returns existing local indexes without forcing aggregator creation, and `discover init` retries as local-only setup when cross-region aggregator creation is denied.
82
83
- Catalog collection uses Resource Explorer `ListResources` with filter strings instead of `Search`, which avoids the 1,000-result ceiling on filter-only queries.
83
-
- Resource Explorer catalog seeding batches `resourcetype:` and `region:` filters into the smallest possible query set, raises`MaxResults` to `1000`, and retries throttled `ListResources` calls before failing.
84
+
- Resource Explorer catalog seeding batches `resourcetype:` and `region:` filters into the smallest possible query set, uses`MaxResults: 999` so AWS reliably returns pagination tokens, and retries throttled `ListResources` calls before failing.
84
85
- Account-scoped or fallback-backed datasets can bypass Resource Explorer seeding entirely by declaring no `resourceTypes`; the loader then receives `[]` and owns the account-level API call.
85
86
- Account-scoped loaders share one lazy STS account-ID resolution per discovery run; the cache is discarded between runs so ambient credential contexts cannot leak identity.
86
87
- Each discovery run shares one AWS call budget: concurrent datasets fanning out to the same service in the same region are capped at a combined in-flight limit, on top of each loader's own bounded batches. Route 53 operations additionally share an account-wide sliding-window limit of five request starts per second across hosted zones, record sets, health checks, pagination, retries, and simultaneous discovery runs in the same process. Hydrators called directly outside a discover run stay unbounded.
87
88
-`CloudBurnClient.discover({ onProgress })` streams `AwsDiscoveryProgressEvent` values (catalog ready, per-dataset completion counts) while the run executes, so callers can render live feedback without waiting for the final result.
88
89
- Resource Explorer catalog failures degrade when the run also requested account-scoped datasets: the SDK records one catalog diagnostic, marks every catalog-backed dataset unavailable (skipping the rules that need them), and still evaluates account-scoped datasets. When every requested dataset needs the catalog, the failure stays fatal so the actionable Resource Explorer error reaches the user.
89
90
- Dataset loader failures are non-fatal: the SDK records diagnostics, marks the affected datasets unavailable, skips rules that require them, and returns findings from datasets that loaded successfully.
91
+
- Global tagging discovery is opt-in because it requires an accessible aggregator, and uses one paginated `ListResources` filter query (`resourcetype.supports:tags tag:none`) instead of per-service tagging APIs.
90
92
- Missing Lambda `Architectures` values from AWS are normalized to `['x86_64']`, matching the AWS default architecture.
91
93
- Lambda hydrators limit in-flight `GetFunctionConfiguration` calls per region to avoid API throttling in large accounts.
92
94
- Live scans require Resource Explorer access plus narrow hydrator permissions such as `apigateway:GetStage`, `application-autoscaling:DescribeScalableTargets`, `application-autoscaling:DescribeScalingPolicies`, `ce:GetCostAndUsage`, `cloudfront:GetDistribution`, `cloudfront:ListDistributions`, `cloudtrail:DescribeTrails`, `cloudwatch:GetMetricData`, `dynamodb:DescribeTable`, `ecs:DescribeContainerInstances`, `ecs:DescribeServices`, `ec2:DescribeInstances`, `ec2:DescribeNatGateways`, `ec2:DescribeVolumes`, `eks:ListNodegroups`, `eks:DescribeNodegroup`, `lambda:GetFunctionConfiguration`, `rds:DescribeDBInstances`, `route53:ListHealthChecks`, `route53:ListHostedZones`, `route53:ListResourceRecordSets`, `s3:GetLifecycleConfiguration`, `s3:GetIntelligentTieringConfiguration`, `sagemaker:DescribeEndpoint`, `sagemaker:DescribeEndpointConfig`, `sagemaker:DescribeNotebookInstance`, and `secretsmanager:DescribeSecret`.
@@ -123,7 +125,7 @@ absolute filesystem paths.
123
125
124
126
## Provider Layer
125
127
126
-
`buildRuleRegistry(config, mode)` decides which rules are active for the requested mode.
128
+
`buildRuleRegistry(config, mode)` decides which rules are active for the requested mode. When `enabledRules` is unset, it starts from `awsCorePreset`; an explicit `enabledRules` list replaces that selection and can activate opt-in rules from the complete `awsRules` export.
127
129
128
130
Config behavior: see [`docs/reference/config-schema.md`](../reference/config-schema.md) for full field definitions, merge behavior, and config loading semantics. Registry filtering is mode-aware and only activates rules that support the requested source.
`awsCorePreset` in `packages/rules/src/presets/aws-core.ts`uses `toRuleIds(awsRules)`, so new rules are automatically included when added to `awsRules`. No manual preset change is needed.
105
+
`awsCorePreset` in `packages/rules/src/presets/aws-core.ts`normally includes IDs from `awsRules`. Rules that require account-wide infrastructure or other explicit setup can be excluded from the preset and enabled by users through `enabled-rules`. Document any opt-in requirement in [`rule-ids.md`](../reference/rule-ids.md).
|`CLDBRN-AWS-CLOUDWATCH-3`| Reviews only log groups storing at least `1 GiB` and flags them when no metric filters are configured. | cloudwatch | discovery |
|`CLDBRN-AWS-COSTGUARDRAILS-3`| Flags configured AWS Budgets only when normalized actual spend is strictly greater than the same-unit budget limit. Malformed and unit-mismatched spend details are skipped. | costguardrails | discovery |
29
30
|`CLDBRN-AWS-COSTEXPLORER-1`| Compares the last two full months and flags only services with an existing prior-month baseline and a cost increase greater than `10` cost units. | costexplorer | discovery |
30
31
|`CLDBRN-AWS-DYNAMODB-1`| Flags only tables whose parsed `latestStreamLabel` is older than `90` days. Tables without a stream label are skipped. | dynamodb | discovery |
31
32
|`CLDBRN-AWS-DYNAMODB-2`| Reviews only provisioned-capacity tables and flags them when no table-level read or write autoscaling targets are configured. | dynamodb | discovery, iac |
|`CLDBRN-AWS-SAGEMAKER-1`| Flags only notebook instances whose normalized status remains `InService`. | sagemaker | discovery |
92
93
|`CLDBRN-AWS-SAGEMAKER-2`| Flags only endpoints whose normalized status remains `InService`, whose parsed `creationTime` is at least `14` days old, and whose complete 14-day `Invocations` total stays at `0`. Endpoints with incomplete metrics are skipped. | sagemaker | discovery |
93
94
|`CLDBRN-AWS-SECRETSMANAGER-1`| Flags secrets with no `lastAccessedDate` and secrets whose parsed last access is at least `90` days old. | secretsmanager | discovery |
95
+
|`CLDBRN-AWS-TAGGING-1`| Opt-in. Uses an accessible Resource Explorer aggregator and `resourcetype.supports:tags tag:none` to flag taggable resources without user-created tags across the account. AWS-managed tags do not satisfy the rule. | tagging | discovery |
|`CLDBRN-AWS-LAMBDA-2`| Uses 7-day CloudWatch totals and flags only functions whose observed `Errors / Invocations` ratio is greater than `10%`. | lambda | discovery |
96
98
|`CLDBRN-AWS-LAMBDA-3`| Reviews only functions with configured timeouts of at least `30` seconds and flags when the timeout is at least `5x` the observed 7-day average duration. | lambda | discovery |
0 commit comments