mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-10-30 07:14:20 +00:00
Merge master
This commit is contained in:
768 files changed
+62058
-7161
No files matched your search
@@ -19,7 +19,7 @@ A compliance framework must represent the **complete state** of the source catal
|
||||
Requirement coverage feeds the compliance percentage calculations and the metadata surfaces (dashboards, widgets, exports). Missing requirements skew those metrics and break the report as a faithful snapshot of the framework.
|
||||
</Warning>
|
||||
|
||||
### Two supported schemas
|
||||
### Two Supported Schemas
|
||||
|
||||
| Schema | When to use | File location | Discovered as |
|
||||
| --- | --- | --- | --- |
|
||||
@@ -46,7 +46,7 @@ Before adding a new framework, complete the following checks:
|
||||
|
||||
## Universal Compliance Framework
|
||||
|
||||
### Where the file lives
|
||||
### Where the File Lives
|
||||
|
||||
Place the file at the top level of the compliance directory:
|
||||
|
||||
@@ -58,7 +58,7 @@ Examples in the repository: `prowler/compliance/csa_ccm_4.0.json`, `prowler/comp
|
||||
|
||||
The file is auto-discovered — there is **no** need to register it in any `__init__.py`, modify `prowler/lib/outputs/`, or update any other Python module. The framework key Prowler CLI accepts via `--compliance` is the basename of the JSON file without `.json` (`dora_2022_2554.json` → `dora_2022_2554`).
|
||||
|
||||
### Top-level structure
|
||||
### Top-Level Structure
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -199,7 +199,7 @@ Per requirement:
|
||||
|
||||
For MITRE-style frameworks, additional optional fields are available on the requirement: `tactics`, `sub_techniques`, `platforms`, `technique_url` (these are populated automatically when adapting a legacy MITRE JSON to the universal model).
|
||||
|
||||
### Multi-provider frameworks
|
||||
### Multi-Provider Frameworks
|
||||
|
||||
A single universal file can cover any number of providers. The framework appears under each provider's `--list-compliance` output as long as **at least one** requirement has that provider key in its `checks` dict.
|
||||
|
||||
@@ -227,7 +227,7 @@ The legacy schema spans **four layers** — a complete contribution must touch e
|
||||
|
||||
The universal schema collapses Layers 3 and 4 into declarative configuration inside the JSON — that is the main reason it is preferred for new contributions.
|
||||
|
||||
### Directory structure and file naming
|
||||
### Directory Structure and File Naming
|
||||
|
||||
Compliance frameworks live at:
|
||||
|
||||
@@ -260,7 +260,7 @@ prowler/lib/outputs/compliance/<framework>/
|
||||
└── __init__.py
|
||||
```
|
||||
|
||||
### JSON schema reference
|
||||
### JSON Schema Reference
|
||||
|
||||
Every legacy compliance file is a JSON document with the following top-level keys. `Framework`, `Name` and `Provider` are validated non-empty by the root validator `framework_and_provider_must_not_be_empty` (`compliance_models.py`).
|
||||
|
||||
@@ -363,7 +363,7 @@ For the remaining attribute classes (`AWS_Well_Architected_Requirement_Attribute
|
||||
The `Attributes` field is a Pydantic `Union`. The generic attribute model **must** remain the last element of that Union — otherwise Pydantic v1 silently coerces every framework into the generic shape and your specialized fields are dropped. Adding a brand-new attribute shape requires inserting the Pydantic class **before** `Generic_Compliance_Requirement_Attribute`.
|
||||
</Note>
|
||||
|
||||
#### Minimal working example
|
||||
#### Minimal Working Example
|
||||
|
||||
The following snippet is a complete, valid framework file named `my_framework_1.0_aws.json`, saved at `prowler/compliance/aws/my_framework_1.0_aws.json`. It uses the generic attribute shape for simplicity.
|
||||
|
||||
@@ -409,7 +409,7 @@ The following snippet is a complete, valid framework file named `my_framework_1.
|
||||
}
|
||||
```
|
||||
|
||||
### Mapping checks to requirements
|
||||
### Mapping Checks to Requirements
|
||||
|
||||
Each requirement links to the Prowler checks that, together, produce a PASS or FAIL verdict for that control.
|
||||
|
||||
@@ -426,7 +426,7 @@ To discover available checks:
|
||||
uv run python prowler-cli.py <provider> --list-checks
|
||||
```
|
||||
|
||||
### Supporting multiple providers (legacy)
|
||||
### Supporting Multiple Providers (Legacy)
|
||||
|
||||
The legacy schema binds each file to a single provider. To cover several providers with the same framework, ship one JSON file per provider:
|
||||
|
||||
@@ -440,7 +440,7 @@ Keep the `Framework` and `Version` values identical across the files so the disp
|
||||
|
||||
For a brand-new framework that spans several providers, **prefer the universal schema** — it covers every provider from a single file. If you must use the legacy schema, add one transformer per provider in `prowler/lib/outputs/compliance/<framework>/` and extend the summary-table dispatcher accordingly. See [Output Formatter](#output-formatter).
|
||||
|
||||
### Output formatter
|
||||
### Output Formatter
|
||||
|
||||
Legacy frameworks render in two forms: a detailed CSV report written to disk, and a summary table printed in the CLI. Both are produced by the output formatter package for the framework. Universal frameworks do **not** need a Python output formatter — the `outputs` config inside the JSON drives rendering — so this section applies only to the legacy schema.
|
||||
|
||||
@@ -454,19 +454,19 @@ prowler/lib/outputs/compliance/my_framework/
|
||||
└── models.py # CSV row Pydantic model
|
||||
```
|
||||
|
||||
#### Step 1 — Define the CSV row model
|
||||
#### Step 1 — Define the CSV Row Model
|
||||
|
||||
In `models.py`, declare a Pydantic v1 model with one field per CSV column. Use existing models such as `AWSCISModel` in `prowler/lib/outputs/compliance/cis/models.py` as the reference. Fields typically include `Provider`, `Description`, `AccountId`, `Region`, `AssessmentDate`, `Requirements_Id`, `Requirements_Description`, one `Requirements_Attributes_*` field per attribute key, plus the finding fields `Status`, `StatusExtended`, `ResourceId`, `ResourceName`, `CheckId`, `Muted`, `Framework`, `Name`.
|
||||
|
||||
#### Step 2 — Implement the transformer
|
||||
#### Step 2 — Implement the Transformer
|
||||
|
||||
In `my_framework_aws.py`, subclass `ComplianceOutput` from `prowler.lib.outputs.compliance.compliance_output` and implement `transform(findings, compliance, compliance_name)`. Iterate over `findings`, match each finding to the requirements it satisfies through `finding.compliance.get(compliance_name, [])`, and append one row per attribute to `self._data`.
|
||||
|
||||
#### Step 3 — Add the summary-table dispatcher
|
||||
#### Step 3 — Add the Summary-Table Dispatcher
|
||||
|
||||
In `my_framework.py`, implement `get_my_framework_table(findings, bulk_checks_metadata, compliance_framework, output_filename, output_directory, compliance_overview)` following the pattern in `prowler/lib/outputs/compliance/cis/cis.py`.
|
||||
|
||||
#### Step 4 — Register the framework in the dispatchers
|
||||
#### Step 4 — Register the Framework in the Dispatchers
|
||||
|
||||
- Add the dispatcher call in `prowler/lib/outputs/compliance/compliance.py`, inside `display_compliance_table`, with a branch such as `elif "my_framework" in compliance_framework:`.
|
||||
- Register the CSV model and transformer in `prowler/lib/outputs/compliance/compliance_output.py` so the CSV file is emitted during the scan.
|
||||
@@ -475,7 +475,7 @@ In `my_framework.py`, implement `get_my_framework_table(findings, bulk_checks_me
|
||||
For NIST-style catalogs that use `Generic_Compliance_Requirement_Attribute`, no custom formatter is needed. The generic formatter in `prowler/lib/outputs/compliance/generic/` handles them automatically, provided the JSON validates against the generic attribute schema.
|
||||
</Note>
|
||||
|
||||
### Legacy-to-universal adapter
|
||||
### Legacy-to-Universal Adapter
|
||||
|
||||
At load time, every legacy file is transparently adapted to a `ComplianceFramework` via `adapt_legacy_to_universal()` (`compliance_models.py`), which: (a) flattens the first element of `Attributes` into a flat `attributes` dict, (b) wraps `Checks` as `{provider_lower: [...]}`, (c) infers `attributes_metadata` from the matched Pydantic class via `_infer_attribute_metadata()`. The rest of Prowler (CSV/OCSF/PDF output, CLI table) then treats both formats identically.
|
||||
|
||||
@@ -498,7 +498,7 @@ Configuration guardrails close that gap. A requirement declares the configuratio
|
||||
Guardrails are an **optional** safety net for configurable checks. A requirement that maps only to non-configurable checks does not need them. When the field is absent, behavior is unchanged.
|
||||
</Note>
|
||||
|
||||
### Where guardrails are declared
|
||||
### Where Guardrails Are Declared
|
||||
|
||||
The field is attached to each requirement and exists in both schemas:
|
||||
|
||||
@@ -507,7 +507,7 @@ The field is attached to each requirement and exists in both schemas:
|
||||
|
||||
When a legacy file is adapted to the universal model, `adapt_legacy_to_universal()` copies `ConfigRequirements` into `config_requirements` (`compliance_models.py`), so downstream code only ever reads one shape.
|
||||
|
||||
### Constraint schema
|
||||
### Constraint Schema
|
||||
|
||||
Each entry in the list is a single constraint with the following fields:
|
||||
|
||||
@@ -534,7 +534,7 @@ Each entry in the list is a single constraint with the following fields:
|
||||
`subset` / `superset` require both the applied value and `Value` to be lists; any other type is treated as not satisfied. For `eq` against a boolean, declare `Value` as a JSON boolean (`false`, not `0`) — the model keeps booleans distinct from integers.
|
||||
</Note>
|
||||
|
||||
### How guardrails are evaluated
|
||||
### How Guardrails Are Evaluated
|
||||
|
||||
All evaluation lives in one shared module, `prowler/lib/check/compliance_config_eval.py`, consumed by every compliance output (CSV, OCSF, and the CLI tables) and reused by the Prowler API backend so the rule is defined exactly once.
|
||||
|
||||
@@ -548,7 +548,7 @@ All evaluation lives in one shared module, `prowler/lib/check/compliance_config_
|
||||
Guardrails only ever make a result **stricter** (they can turn PASS into FAIL); they never relax a real FAIL into PASS. A requirement with no constraints, or whose keys all use defaults, is reported exactly as before.
|
||||
</Warning>
|
||||
|
||||
### Example: legacy framework
|
||||
### Example: Legacy Framework
|
||||
|
||||
From `prowler/compliance/aws/cis_6.0_aws.json`, requirement 2.11 declares two guardrails — one per configurable check it maps to:
|
||||
|
||||
@@ -591,7 +591,7 @@ A boolean guardrail from the same file: requirement 2.5 (IAM Access Analyzer) on
|
||||
]
|
||||
```
|
||||
|
||||
### Example: universal framework
|
||||
### Example: Universal Framework
|
||||
|
||||
The universal schema uses the lowercase `config_requirements` key with the identical object shape:
|
||||
|
||||
@@ -617,7 +617,7 @@ The universal schema uses the lowercase `config_requirements` key with the ident
|
||||
|
||||
Each constraint declares the `Provider` it targets so the guardrail is only evaluated on scans of that provider — essential for universal frameworks like CSA CCM and DORA, where one requirement maps checks across `aws`, `azure`, `gcp` and more. Because the operator is `subset`, adding `"TLS 1.0"` to `recommended_minimal_tls_versions` widens the allowlist beyond `["TLS 1.2", "TLS 1.3"]` and the requirement is forced to FAIL.
|
||||
|
||||
### What the user sees
|
||||
### What the User Sees
|
||||
|
||||
With a loosened config, the affected requirement's findings report:
|
||||
|
||||
@@ -631,7 +631,7 @@ StatusExtended: Configuration not valid for this requirement. The check
|
||||
|
||||
The same `Configuration not valid for this requirement.` message appears identically across the CSV, OCSF, and console-table outputs.
|
||||
|
||||
### Authoring guidelines
|
||||
### Authoring Guidelines
|
||||
|
||||
- Declare a guardrail only for keys whose value actually changes whether the requirement is met. Most configurable checks do not need one.
|
||||
- Set `Value` to the **strictest** configuration the control tolerates — the same number the control text cites (CIS 45 days, NIST ≤90, and so on).
|
||||
@@ -640,7 +640,7 @@ The same `Configuration not valid for this requirement.` message appears identic
|
||||
- Pick the operator from the value's role: a max threshold is `lte`, a min threshold is `gte`, a toggle is `eq`, an allowlist is `subset`, a denylist is `superset`.
|
||||
- An unrecognized operator does **not** block the requirement — a malformed constraint is treated as satisfied rather than failing the whole framework. Validate your JSON with the tests below.
|
||||
|
||||
### Testing guardrails
|
||||
### Testing Guardrails
|
||||
|
||||
The shared evaluator and the per-output integration are covered by:
|
||||
|
||||
@@ -671,7 +671,7 @@ Prowler matches frameworks by concatenating `Framework` and `Version`. A missing
|
||||
|
||||
Before opening a PR, validate the JSON loads cleanly against the model and that every referenced check actually exists.
|
||||
|
||||
### 1. Schema validation
|
||||
### 1. Schema Validation
|
||||
|
||||
For **universal** frameworks, load the file and inspect what was parsed. The framework key inside `bulk` is the **basename of the JSON file** (without `.json`); for `prowler/compliance/dora_2022_2554.json` that key is `dora_2022_2554`, for `prowler/compliance/aws/cis_5.0_aws.json` it is `cis_5.0_aws`.
|
||||
|
||||
@@ -689,7 +689,7 @@ bulk = get_bulk_compliance_frameworks_universal("aws")
|
||||
assert "<your_framework_filename_without_json>" in bulk
|
||||
```
|
||||
|
||||
### 2. Check existence cross-check
|
||||
### 2. Check Existence Cross-Check
|
||||
|
||||
There is **no automatic check-existence validation** at load time. Cross-check that every check name in your framework maps to a real check directory:
|
||||
|
||||
@@ -709,7 +709,7 @@ missing = referenced - real
|
||||
assert not missing, f"checks referenced in framework but not found in repo: {sorted(missing)}"
|
||||
```
|
||||
|
||||
### 3. CLI smoke test
|
||||
### 3. CLI Smoke Test
|
||||
|
||||
```bash
|
||||
uv run python prowler-cli.py <provider> --list-compliance
|
||||
@@ -729,7 +729,7 @@ Verify that:
|
||||
- The CLI summary table lists every section / pillar of the framework.
|
||||
- Findings roll up under the expected requirements.
|
||||
|
||||
### 4. Inspect the CSV output
|
||||
### 4. Inspect the CSV Output
|
||||
|
||||
Open the generated CSV and confirm:
|
||||
|
||||
|
||||
Reference in new issue
Block a user