Compare commits

..
Author SHA1 Message Date
Daniel Barranquero 4cc2710418 feat(ui): show linked Jira issues on findings and report already-ticketed sends
The finding detail drawer shows the Jira issue linked to the finding (key
as a link to Jira, last known status) fetched from GET /jira-issues after
the panel paints, cached per finding and silently absent on APIs without
the endpoint. Sending findings to Jira now reads skipped_count / skipped
from the task result: findings that already have an open issue are reported
with their keys instead of counting as failures, and an all-skipped dispatch
is a success.
2026-08-25 17:40:00 +02:00
Daniel Barranquero 1ced1af07e chore(api): flag the temporary SDK pin in pyproject.toml
Reminder next to the pinned line so the master lock bump is not forgotten
before the PR leaves draft.
2026-08-25 17:39:53 +02:00
Daniel Barranquero d8a9878a2c chore(api): pin the SDK to the Jira issue reference branch while in draft
Temporary: points api/uv.lock at prowler-cloud/prowler#12539 so this draft
can run against Jira.send_finding() returning the issue key and
Jira.get_issues_status(). To be replaced by a master lock bump once #12539
is merged, before this PR leaves draft.
2026-08-25 16:59:17 +02:00
Daniel Barranquero 8097b8e012 feat(api): track Jira issues per finding and skip already-ticketed findings
New RLS table jira_issues keyed on (integration, provider, finding uid) that
stores the Jira issue key, URL and last observed status for every finding
sent to Jira. send_findings_to_jira() pre-checks the batch in bounded
chunks, refreshes the status of linked issues in bulk, skips findings whose
issue is still open (reported as skipped_count / skipped), creates a
replacement issue when the linked one is closed or deleted in Jira, and
reserves the slot before calling Jira so concurrent runs cannot duplicate.
Read-only GET /api/v1/jira-issues exposes the links with provider-scoped
visibility. Rows are removed with the provider.
2026-08-25 16:58:40 +02:00
Daniel Barranquero 53be73c2ac feat(api): add finding labels, finding URL and tenant info to Jira issues
send_findings_to_jira() now passes issue_labels (prowler, prowler-{provider},
prowler-{severity}, prowler-{check_id} and prowler-finding-{sanitized uid}),
a finding_url built from the new DJANGO_UI_BASE_URL setting (empty by
default, so self-hosted deployments emit no link unless configured) and the
tenant name as tenant_info. The label sanitizer is local to the API so it
does not depend on an unreleased SDK symbol.
2026-08-25 16:29:19 +02:00
674 changed files with 10500 additions and 49960 deletions
+1 -1
View File
@@ -158,7 +158,7 @@ SENTRY_RELEASE=local
# REO_DEV_CLIENT_ID=
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.43.0
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.40.0
# Social login credentials
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
+13 -13
View File
@@ -1,23 +1,23 @@
# SDK
/* @prowler-cloud/engineering
/prowler/ @prowler-cloud/engineering
/tests/ @prowler-cloud/engineering
/dashboard/ @prowler-cloud/engineering
/docs/ @prowler-cloud/engineering
/examples/ @prowler-cloud/engineering
/util/ @prowler-cloud/engineering
/contrib/ @prowler-cloud/engineering
/permissions/ @prowler-cloud/engineering
/codecov.yml @prowler-cloud/engineering
/* @prowler-cloud/detection-remediation
/prowler/ @prowler-cloud/detection-remediation
/tests/ @prowler-cloud/detection-remediation
/dashboard/ @prowler-cloud/detection-remediation
/docs/ @prowler-cloud/detection-remediation
/examples/ @prowler-cloud/detection-remediation
/util/ @prowler-cloud/detection-remediation
/contrib/ @prowler-cloud/detection-remediation
/permissions/ @prowler-cloud/detection-remediation
/codecov.yml @prowler-cloud/detection-remediation @prowler-cloud/api
# API
/api/ @prowler-cloud/engineering
/api/ @prowler-cloud/api
# UI
/ui/ @prowler-cloud/engineering
/ui/ @prowler-cloud/ui
# AI
/mcp_server/ @prowler-cloud/engineering
/mcp_server/ @prowler-cloud/detection-remediation
# Platform
/.github/ @prowler-cloud/platform
@@ -46,17 +46,6 @@ runs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
if grep -q "prowler-cloud/prowler" uv.lock; then
:
else
status=$?
if [ "$status" -ne 1 ]; then
echo "::error::grep failed reading uv.lock (exit code $status)."
exit "$status"
fi
echo "No prowler-cloud/prowler entry in uv.lock, nothing to update."
exit 0
fi
LATEST_COMMIT=$(curl -sf --retry 3 --retry-all-errors --retry-delay 2 --retry-max-time 60 \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
@@ -77,17 +66,6 @@ runs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
if grep -q "prowler-cloud/prowler" uv.lock; then
:
else
status=$?
if [ "$status" -ne 1 ]; then
echo "::error::grep failed reading uv.lock (exit code $status)."
exit "$status"
fi
echo "No prowler-cloud/prowler entry in uv.lock, nothing to update."
exit 0
fi
LATEST_COMMIT=$(curl -sf --retry 3 --retry-all-errors --retry-delay 2 --retry-max-time 60 \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
+1 -1
View File
@@ -39,7 +39,7 @@ jobs:
- name: Check labels
id: label_check
uses: agilepathway/label-checker@c324842522fbd012e4f590afe3b4e591301322ed # v1.6.66
uses: agilepathway/label-checker@c3d16ad512e7cea5961df85ff2486bb774caf3c5 # v1.6.65
with:
allow_failure: true
prefix_mode: true
@@ -114,8 +114,6 @@ jobs:
egress-policy: block
allowed-endpoints: >
auth.docker.io:443
dl-cdn.alpinelinux.org:443
dualstack.j.sni.global.fastly.net:443
files.pythonhosted.org:443
ghcr.io:443
github.com:443
@@ -81,8 +81,6 @@ jobs:
pkg-containers.githubusercontent.com:443
files.pythonhosted.org:443
pypi.org:443
dl-cdn.alpinelinux.org:443
dualstack.j.sni.global.fastly.net:443
api.github.com:443
mirror.gcr.io:443
check.trivy.dev:443
@@ -44,10 +44,7 @@ jobs:
cache: 'pip'
- name: Install dependencies
# Pinned to the versions in pyproject.toml: the ISO partitions region
# data comes from the endpoints.json bundled with botocore, so the
# botocore version is itself a data source and must be deterministic
run: pip install boto3==1.40.61 botocore==1.40.61
run: pip install boto3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
-34
View File
@@ -17,40 +17,6 @@ ignore:
- vulnerability: CVE-2026-71556
package:
name: github.com/go-git/go-git/v5
# CVE-2026-84304 is the same temporary exception documented in .trivyignore.yaml:
# Trivy 0.74.0 still embeds grpc 1.82.1, while the 1.83.1 fix is not in any release.
# Prowler only runs `trivy image` / `trivy fs`, never client/server mode, so no gRPC
# endpoint exists in the image. Pinned to the embedded version so the rule stops
# matching on its own once Trivy bumps grpc. Remove with the Trivy exception by 2026-10-15.
# https://github.com/aquasecurity/trivy/pull/11176
- vulnerability: CVE-2026-84304
package:
name: google.golang.org/grpc
version: v1.82.1
# CVE-2026-84445 is the same temporary exception documented in .trivyignore.yaml:
# Trivy 0.74.0 still embeds grpc 1.82.1, while the 1.82.2 / 1.83.2 fix is not in any
# release. The panic needs a gRPC server built with `xds.NewGRPCServer()`; Prowler only
# runs `trivy image` / `trivy fs`, so the image serves no gRPC at all. Pinned to the
# embedded version so the rule stops matching on its own once Trivy bumps grpc. Remove
# with the Trivy exception by 2026-10-15.
# https://github.com/advisories/GHSA-2v4p-qf9q-27wj
- vulnerability: CVE-2026-84445
package:
name: google.golang.org/grpc
version: v1.82.1
# CVE-2026-56855 / CVE-2026-78662 are the same temporary exception documented in
# .trivyignore.yaml: Trivy 0.74.0 still embeds golang.org/x/crypto v0.55.0, while the
# 0.56.0 fix (published 2026-09-02) hasn't reached any Trivy release, or even Trivy
# main, yet. Pinned to the embedded version so the rule stops matching on its own once
# Trivy bumps it. Remove with the Trivy exception by 2026-10-15.
- vulnerability: CVE-2026-56855
package:
name: golang.org/x/crypto
version: v0.55.0
- vulnerability: CVE-2026-78662
package:
name: golang.org/x/crypto
version: v0.55.0
- vulnerability: CVE-2026-56852
package:
name: golang.org/x/text
-68
View File
@@ -113,18 +113,6 @@ vulnerabilities:
purls:
- "pkg:npm/fast-uri"
expired_at: 2027-01-31
- id: CVE-2026-75899
purls:
- "pkg:npm/fast-uri"
expired_at: 2027-01-31
- id: CVE-2026-75975
purls:
- "pkg:npm/fast-uri"
expired_at: 2027-01-31
- id: CVE-2026-76172
purls:
- "pkg:npm/fast-uri"
expired_at: 2027-01-31
- id: CVE-2026-69192
purls:
- "pkg:npm/ip-address"
@@ -160,62 +148,6 @@ vulnerabilities:
- "pkg:golang/github.com/go-git/go-git/v5"
expired_at: 2026-09-15
# CVE-2026-84304 is a DoS in grpc-go <= 1.83.0: a peer fragments a gRPC stream into
# millions of tiny HTTP/2 DATA frames until the receiver runs out of heap. Fixed in
# 1.83.1 (published 2026-09-01). Trivy 0.74.0, the latest published release and the
# version the images ship, pins 1.82.1 as an indirect dependency:
# https://github.com/aquasecurity/trivy/blob/v0.74.0/go.mod
# Upstream bump still open: https://github.com/aquasecurity/trivy/pull/11176
# Trivy only speaks gRPC in client/server mode (`trivy server`, `--server`). Prowler
# invokes it exclusively as `trivy image` and `trivy fs` on a local path, so no gRPC
# listener or connection ever exists in the image and the affected path is not
# reachable. Remove this temporary suppression as soon as a Trivy release pins
# grpc >= 1.83.1.
- id: CVE-2026-84304
purls:
- "pkg:golang/google.golang.org/grpc"
expired_at: 2026-10-15
# CVE-2026-84445 is a DoS in grpc-go servers built with `xds.NewGRPCServer()`: a request
# carrying neither `:authority` nor `Host` reaches the xDS routing interceptor, which
# indexes an empty slice of authorities and panics. The per-RPC goroutine does not
# recover, so the whole server process dies. Fixed in 1.82.2 and 1.83.2 (published
# 2026-09-08). Trivy 0.74.0, the latest published release and the version the images
# ship, pins 1.82.1 as an indirect dependency:
# https://github.com/aquasecurity/trivy/blob/v0.74.0/go.mod
# Trivy main already carries 1.83.2, but no published release includes it yet.
# The reachability argument is the one made for CVE-2026-84304 above, only narrower:
# this panic needs an xDS-managed gRPC server. Prowler invokes Trivy exclusively as
# `trivy image` and `trivy fs` on a local path, never `trivy server`, so the image runs
# no gRPC server at all, xDS or otherwise. Remove this temporary suppression as soon as
# a Trivy release pins grpc >= 1.83.2.
# https://github.com/advisories/GHSA-2v4p-qf9q-27wj
- id: CVE-2026-84445
purls:
- "pkg:golang/google.golang.org/grpc@v1.82.1"
expired_at: 2026-10-15
# CVE-2026-56855 and CVE-2026-78662 are DoS deadlocks in x/crypto/ssh: a malicious peer
# can flood or misuse channel messages (RFC 4254) to block the whole connection.
# Fixed in golang.org/x/crypto v0.56.0 (published 2026-09-02). Trivy 0.74.0, the latest
# published release and the version the images ship, still pins v0.55.0, and Trivy main
# has not bumped it either:
# https://github.com/aquasecurity/trivy/blob/v0.74.0/go.mod
# x/crypto/ssh is pulled in transitively through go-git's ssh transport, the same
# dependency chain as the CVE-2026-71556 entry above. Prowler invokes Trivy only with
# `fs` on an existing local path or with `image`; it never asks Trivy to clone over SSH
# or to run `trivy server`, so no SSH connection -- as client or server -- ever exists in
# the image and the affected code path is not reachable. Remove this temporary
# suppression as soon as a fixed Trivy release is available.
- id: CVE-2026-56855
purls:
- "pkg:golang/golang.org/x/crypto@v0.55.0"
expired_at: 2026-10-15
- id: CVE-2026-78662
purls:
- "pkg:golang/golang.org/x/crypto@v0.55.0"
expired_at: 2026-10-15
- id: CVE-2026-56852
purls:
- "pkg:golang/golang.org/x/text"
+1 -15
View File
@@ -22,25 +22,11 @@ ARG POWERSHELL_SHA256_ARM64=2503b71da3e83635592b092df59a0aca4c3606b4d9b068217bb0
ARG ZIZMOR_SHA256_AMD64=a8000f3c683319a523d3b20df0e75457ba591f049cfcbfa98966631b56733c03
ARG ZIZMOR_SHA256_ARM64=d66e37ef8a375fb07939c630ebf9709a6e0f20242bdc3faf672a7ed97e0b768d
# High CVEs fixed in Debian trixie but not yet in the pinned base image:
# openssl/libssl3t64/openssl-provider-legacy 3.5.7-1~deb13u2 CVE-2026-14456,
# -14457, -18798, -54874, -63072, -63073, -63074, -63075, -63076, -75803
# libsqlite3-0 3.46.1-7+deb13u2 CVE-2026-11822, -11824
# gzip 1.13-1+deb13u1 CVE-2026-41992
# perl-base 5.40.1-6+deb13u1 CVE-2026-42497, -48962, -57432
# libssh2-1t64 1.11.1-1+deb13u2 CVE-2026-58050
# libpcre2-8-0 10.46-1~deb13u2 CVE-2026-86145, -89161
# Taken as a targeted --only-upgrade rather than by moving the digest: the newest
# published python:3.12-slim-trixie carries the same vulnerable versions. The three
# openssl packages are flagged separately, so all are named.
# Drop each one once the base image ships its fixed version.
# hadolint ignore=DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
wget libicu76 libunwind8 libssl3 libcurl4 ca-certificates apt-transport-https gnupg \
build-essential pkg-config libzstd-dev zlib1g-dev \
&& apt-get install -y --no-install-recommends --only-upgrade \
util-linux libssl3t64 openssl openssl-provider-legacy \
libsqlite3-0 gzip perl-base libssh2-1t64 libpcre2-8-0 \
&& apt-get install -y --no-install-recommends --only-upgrade util-linux \
&& rm -rf /var/lib/apt/lists/*
# Install PowerShell
+1 -1
View File
@@ -139,7 +139,7 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
| MongoDB Atlas | 10 | 3 | 1 | 8 | Official | UI, API, CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | CLI |
| Image | N/A | N/A | N/A | N/A | Official | UI, API, CLI |
| Google Workspace | 65 | 11 | 4 | 6 | Official | UI, API, CLI |
| Google Workspace | 65 | 11 | 3 | 6 | Official | UI, API, CLI |
| OpenStack | 34 | 5 | 1 | 9 | Official | UI, API, CLI |
| Vercel | 26 | 6 | 1 | 8 | Official | UI, API, CLI |
| Okta | 29 | 8 | 2 | 2 | Official | UI, API, CLI |
-33
View File
@@ -4,39 +4,6 @@ All notable changes to the **Prowler API** are documented in this file.
<!-- changelog: release notes start -->
## [1.43.0] (Prowler v5.42.0)
### 🔄 Changed
- Speed up compliance overview ingestion by reading ThreatScore mappings from the compliance template instead of each finding, generating time-ordered `uuid7` row ids and grouping inserted rows by framework and requirement [(#12738)](https://github.com/prowler-cloud/prowler/pull/12738)
---
## [1.42.0] (Prowler v5.41.0)
### 🚀 Added
- Jira issues created from Prowler Cloud now carry the `prowler`, `prowler-<provider>`, `prowler-<severity>`, `prowler-<check-id>`, and `prowler-finding-<finding-uid>` labels, a link back to the finding when `DJANGO_UI_BASE_URL` is configured, and the tenant name [(#12540)](https://github.com/prowler-cloud/prowler/pull/12540)
### 🐞 Fixed
- `POST /api/v1/mute-rules` now updates only each affected provider's latest completed scan and future scans, preventing historical reaggregation from flooding Celery queues [(#12681)](https://github.com/prowler-cloud/prowler/pull/12681)
---
## [1.41.0] (Prowler v5.40.0)
### 🐞 Fixed
- `FINDINGS_TABLE_PARTITION_MAX_AGE_MONTHS` is now applied in months instead of days, and negative values are rejected [(#12580)](https://github.com/prowler-cloud/prowler/pull/12580)
### 🔐 Security
- `sqlparse` upgraded to 0.6.0, patching CVE-2026-54284, CVE-2026-59893, and CVE-2026-71491 [(#12509)](https://github.com/prowler-cloud/prowler/pull/12509)
- `openssl`, `libssl3t64` and `openssl-provider-legacy` upgraded to 3.5.7-1~deb13u2 in the API container image, patching ten high OpenSSL CVEs [(#12549)](https://github.com/prowler-cloud/prowler/pull/12549)
---
## [1.40.1] (Prowler v5.39.1)
### 🔄 Changed
+1 -15
View File
@@ -21,18 +21,6 @@ ARG POWERSHELL_SHA256_ARM64=2503b71da3e83635592b092df59a0aca4c3606b4d9b068217bb0
ARG ZIZMOR_SHA256_AMD64=a8000f3c683319a523d3b20df0e75457ba591f049cfcbfa98966631b56733c03
ARG ZIZMOR_SHA256_ARM64=d66e37ef8a375fb07939c630ebf9709a6e0f20242bdc3faf672a7ed97e0b768d
# High CVEs fixed in Debian trixie but not yet in the pinned base image:
# openssl/libssl3t64/openssl-provider-legacy 3.5.7-1~deb13u2 CVE-2026-14456,
# -14457, -18798, -54874, -63072, -63073, -63074, -63075, -63076, -75803
# libsqlite3-0 3.46.1-7+deb13u2 CVE-2026-11822, -11824
# gzip 1.13-1+deb13u1 CVE-2026-41992
# perl-base 5.40.1-6+deb13u1 CVE-2026-42497, -48962, -57432
# libssh2-1t64 1.11.1-1+deb13u2 CVE-2026-58050
# libpcre2-8-0 10.46-1~deb13u2 CVE-2026-86145, -89161
# Taken as a targeted --only-upgrade rather than by moving the digest: the newest
# published python:3.12-slim-trixie carries the same vulnerable versions. The three
# openssl packages are flagged separately, so all are named.
# Drop each one once the base image ships its fixed version.
# hadolint ignore=DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
wget \
@@ -48,9 +36,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
libtool \
libxslt1-dev \
python3-dev \
&& apt-get install -y --no-install-recommends --only-upgrade \
util-linux libssl3t64 openssl openssl-provider-legacy \
libsqlite3-0 gzip perl-base libssh2-1t64 libpcre2-8-0 \
&& apt-get install -y --no-install-recommends --only-upgrade util-linux \
&& rm -rf /var/lib/apt/lists/*
# Install PowerShell
@@ -1 +0,0 @@
`libsqlite3-0`, `gzip`, `perl-base` and `libpcre2-8-0` upgraded in the API container image, patching high Debian CVEs
@@ -0,0 +1 @@
Jira issues created from Prowler Cloud now carry `prowler-*` labels (provider, severity, check id and a sanitized finding UID), a link back to the finding when `DJANGO_UI_BASE_URL` is configured, and the tenant name
@@ -0,0 +1 @@
Jira issues created from findings are now tracked per finding UID in the new `jira_issues` table and exposed through `GET /api/v1/jira-issues`; sending a finding that already has an open Jira issue skips it (reported as `skipped_count`), and findings whose issue was closed or deleted in Jira get a new issue that replaces the link
+1
View File
@@ -0,0 +1 @@
`sqlparse` upgraded to 0.6.0, patching CVE-2026-54284, CVE-2026-59893, and CVE-2026-71491
+5 -2
View File
@@ -45,7 +45,10 @@ dependencies = [
"gunicorn==26.0.0",
"uvloop==0.22.1",
"lxml==6.1.0",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@master",
# TEMPORARY (draft only): pinned to the head of prowler-cloud/prowler#12539 so this
# branch runs against the new Jira SDK helpers. Before this PR leaves draft: restore
# "@master", run `uv lock --upgrade-package prowler`, and delete this comment.
"prowler @ git+https://github.com/prowler-cloud/prowler.git@23e048fcd20738b68b54be6adc8b43f8a26d0d43",
"psycopg2-binary==2.9.9",
"pytest-celery[redis] (==1.3.0)",
"sentry-sdk[django] (==2.56.0)",
@@ -71,7 +74,7 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.44.0"
version = "1.41.0"
# Shared ruff baseline (kept in sync with mcp_server/pyproject.toml).
# target-version tracks this project's lowest supported Python.
+28
View File
@@ -17,6 +17,7 @@ from api.models import (
FindingGroupDailySummary,
Integration,
Invitation,
JiraIssue,
LighthouseProviderConfiguration,
LighthouseProviderModels,
Membership,
@@ -1900,3 +1901,30 @@ class ComplianceWatchlistFilter(BaseProviderFilter):
class Meta(BaseProviderFilter.Meta):
model = ProviderComplianceScore
class JiraIssueFilter(BaseProviderFilter):
finding_uid = CharFilter(field_name="finding_uid", lookup_expr="exact")
finding_uid__in = CharInFilter(field_name="finding_uid", lookup_expr="in")
finding_id = UUIDFilter(field_name="finding_id", lookup_expr="exact")
finding_id__in = UUIDInFilter(field_name="finding_id", lookup_expr="in")
integration = UUIDFilter(field_name="integration__id", lookup_expr="exact")
integration__in = UUIDInFilter(field_name="integration__id", lookup_expr="in")
issue_key = CharFilter(field_name="issue_key", lookup_expr="exact")
issue_key__in = CharInFilter(field_name="issue_key", lookup_expr="in")
issue_status_category = ChoiceFilter(
choices=JiraIssue.StatusCategoryChoices.choices
)
issue_status_category__in = ChoiceInFilter(
choices=JiraIssue.StatusCategoryChoices.choices,
field_name="issue_status_category",
lookup_expr="in",
)
class Meta:
model = JiraIssue
fields = {
"inserted_at": ["date", "gte", "lte"],
"updated_at": ["date", "gte", "lte"],
"project_key": ["exact", "in"],
}
@@ -0,0 +1,104 @@
import uuid
import api.rls
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0097_attack_paths_scan_db_defaults"),
]
operations = [
migrations.CreateModel(
name="JiraIssue",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
("finding_uid", models.CharField(max_length=300)),
("finding_id", models.UUIDField()),
(
"issue_key",
models.CharField(blank=True, default="", max_length=64),
),
(
"issue_id",
models.CharField(blank=True, default="", max_length=64),
),
(
"issue_url",
models.URLField(blank=True, default="", max_length=2048),
),
("project_key", models.CharField(max_length=64)),
(
"issue_status",
models.CharField(blank=True, default="", max_length=64),
),
(
"issue_status_category",
models.CharField(
blank=True,
choices=[
("new", "New"),
("indeterminate", "In progress"),
("done", "Done"),
],
default="",
max_length=16,
),
),
("status_synced_at", models.DateTimeField(blank=True, null=True)),
(
"integration",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="jira_issues",
to="api.integration",
),
),
(
"provider",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="jira_issues",
to="api.provider",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "jira_issues",
"abstract": False,
},
),
migrations.AddConstraint(
model_name="jiraissue",
constraint=models.UniqueConstraint(
fields=("tenant_id", "integration_id", "provider_id", "finding_uid"),
name="unique_jira_issue_per_finding",
),
),
migrations.AddConstraint(
model_name="jiraissue",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_jiraissue",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
@@ -0,0 +1,17 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0098_jira_issues"),
]
operations = [
migrations.AddIndex(
model_name="jiraissue",
index=models.Index(
fields=["tenant_id", "provider_id", "finding_uid"],
name="ji_tenant_prov_uid_idx",
),
),
]
+74
View File
@@ -3113,3 +3113,77 @@ class TenantComplianceSummary(RowLevelSecurityProtectedModel):
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
class JiraIssue(RowLevelSecurityProtectedModel):
"""Jira issue created from a finding through a Jira integration.
One row per (integration, provider, finding uid). Keyed on the finding ``uid``
rather than the per-scan finding id so the link survives rescans, which is
what lets a repeated send be recognised as already ticketed. Only the latest
ticket is kept: when a linked issue is closed or deleted in Jira and the
finding is sent again, the row is updated to point at the new issue.
"""
class StatusCategoryChoices(models.TextChoices):
NEW = "new", _("New")
INDETERMINATE = "indeterminate", _("In progress")
DONE = "done", _("Done")
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
integration = models.ForeignKey(
Integration, on_delete=models.CASCADE, related_name="jira_issues"
)
provider = models.ForeignKey(
Provider, on_delete=models.CASCADE, related_name="jira_issues"
)
finding_uid = models.CharField(max_length=300)
# Last finding record that was sent; informational, findings are partitioned
# and rotate per scan so this is not a foreign key
finding_id = models.UUIDField()
# Empty while the issue is being created (reservation), filled after Jira
# confirms the creation
issue_key = models.CharField(max_length=64, blank=True, default="")
issue_id = models.CharField(max_length=64, blank=True, default="")
issue_url = models.URLField(max_length=2048, blank=True, default="")
project_key = models.CharField(max_length=64)
issue_status = models.CharField(max_length=64, blank=True, default="")
issue_status_category = models.CharField(
max_length=16, choices=StatusCategoryChoices.choices, blank=True, default=""
)
status_synced_at = models.DateTimeField(null=True, blank=True)
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "jira_issues"
constraints = [
models.UniqueConstraint(
fields=("tenant_id", "integration_id", "provider_id", "finding_uid"),
name="unique_jira_issue_per_finding",
),
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
indexes = [
models.Index(
fields=["tenant_id", "provider_id", "finding_uid"],
name="ji_tenant_prov_uid_idx",
),
]
class JSONAPIMeta:
resource_name = "jira-issues"
@property
def is_linked(self) -> bool:
"""Whether the row points at a confirmed Jira issue (not a reservation)."""
return bool(self.issue_key)
@property
def is_done(self) -> bool:
return self.issue_status_category == self.StatusCategoryChoices.DONE
+5 -13
View File
@@ -6,7 +6,6 @@ from api.rls import RowLevelSecurityConstraint
from api.uuid_utils import datetime_to_uuid7
from dateutil.relativedelta import relativedelta
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from psqlextra.partitioning import (
PostgresPartitioningError,
PostgresPartitioningManager,
@@ -154,17 +153,10 @@ class PostgresUUIDv7PartitioningStrategy(PostgresRangePartitioningStrategy):
)
def relative_months_or_none(value):
# A negative value would set the cutoff in the future and delete every
# partition, so it is rejected rather than silently ignored.
if value is not None and value < 0:
raise ImproperlyConfigured(
"FINDINGS_TABLE_PARTITION_MAX_AGE_MONTHS must not be negative; "
"leave it unset or use 0 to keep partitions indefinitely"
)
if not value:
def relative_days_or_none(value):
if value is None:
return None
return relativedelta(months=value)
return relativedelta(days=value)
#
@@ -181,7 +173,7 @@ manager = PostgresPartitioningManager(
months=settings.FINDINGS_TABLE_PARTITION_MONTHS
),
count=settings.FINDINGS_TABLE_PARTITION_COUNT,
max_age=relative_months_or_none(
max_age=relative_days_or_none(
settings.FINDINGS_TABLE_PARTITION_MAX_AGE_MONTHS
),
name_format="%Y_%b",
@@ -197,7 +189,7 @@ manager = PostgresPartitioningManager(
months=settings.FINDINGS_TABLE_PARTITION_MONTHS
),
count=settings.FINDINGS_TABLE_PARTITION_COUNT,
max_age=relative_months_or_none(
max_age=relative_days_or_none(
settings.FINDINGS_TABLE_PARTITION_MAX_AGE_MONTHS
),
name_format="%Y_%b",
+537 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.0.3
info:
title: Prowler API
version: 1.44.0
version: 1.41.0
description: |-
Prowler API specification.
@@ -7107,6 +7107,405 @@ paths:
schema:
$ref: '#/components/schemas/OpenApiResponseResponse'
description: ''
/api/v1/jira-issues:
get:
operationId: api_v1_jira_issues_list
description: Retrieve the Jira issues created from findings through Jira integrations.
Each entry links a finding UID to the latest Jira issue created for it, with
the last status observed in Jira. Use `filter[finding_uid__in]` and `filter[provider_id]`
to check whether specific findings already have a ticket.
summary: List Jira issues linked to findings
parameters:
- in: query
name: fields[jira-issues]
schema:
type: array
items:
type: string
enum:
- inserted_at
- updated_at
- finding_uid
- finding_id
- issue_key
- issue_id
- issue_url
- project_key
- issue_status
- issue_status_category
- status_synced_at
- integration
- provider
- url
description: endpoint return only specific fields in the response on a per-type
basis by including a fields[TYPE] query parameter.
explode: false
- in: query
name: filter[finding_id]
schema:
type: string
format: uuid
- in: query
name: filter[finding_id__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[finding_uid]
schema:
type: string
- in: query
name: filter[finding_uid__in]
schema:
type: array
items:
type: string
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[inserted_at__date]
schema:
type: string
format: date
- in: query
name: filter[inserted_at__gte]
schema:
type: string
format: date-time
- in: query
name: filter[inserted_at__lte]
schema:
type: string
format: date-time
- in: query
name: filter[integration]
schema:
type: string
format: uuid
- in: query
name: filter[integration__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[issue_key]
schema:
type: string
- in: query
name: filter[issue_key__in]
schema:
type: array
items:
type: string
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[issue_status_category]
schema:
type: string
x-spec-enum-id: 6e5c623f6bbdd92d
enum:
- done
- indeterminate
- new
description: |-
* `new` - New
* `indeterminate` - In progress
* `done` - Done
- in: query
name: filter[issue_status_category__in]
schema:
type: array
items:
type: string
x-spec-enum-id: 6e5c623f6bbdd92d
enum:
- done
- indeterminate
- new
description: |-
Multiple values may be separated by commas.
* `new` - New
* `indeterminate` - In progress
* `done` - Done
explode: false
style: form
- in: query
name: filter[project_key]
schema:
type: string
- in: query
name: filter[project_key__in]
schema:
type: array
items:
type: string
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_groups]
schema:
type: string
format: uuid
- in: query
name: filter[provider_groups__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_id]
schema:
type: string
format: uuid
- in: query
name: filter[provider_id__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_type]
schema:
type: string
x-spec-enum-id: 203afc16daac9b64
enum:
- alibabacloud
- aws
- azure
- cloudflare
- gcp
- github
- googleworkspace
- iac
- image
- kubernetes
- m365
- mongodbatlas
- okta
- openstack
- oraclecloud
- vercel
description: |-
* `aws` - AWS
* `azure` - Azure
* `gcp` - GCP
* `kubernetes` - Kubernetes
* `m365` - M365
* `github` - GitHub
* `mongodbatlas` - MongoDB Atlas
* `iac` - IaC
* `oraclecloud` - Oracle Cloud Infrastructure
* `alibabacloud` - Alibaba Cloud
* `cloudflare` - Cloudflare
* `openstack` - OpenStack
* `image` - Image
* `googleworkspace` - Google Workspace
* `vercel` - Vercel
* `okta` - Okta
- in: query
name: filter[provider_type__in]
schema:
type: array
items:
type: string
x-spec-enum-id: 203afc16daac9b64
enum:
- alibabacloud
- aws
- azure
- cloudflare
- gcp
- github
- googleworkspace
- iac
- image
- kubernetes
- m365
- mongodbatlas
- okta
- openstack
- oraclecloud
- vercel
description: |-
Multiple values may be separated by commas.
* `aws` - AWS
* `azure` - Azure
* `gcp` - GCP
* `kubernetes` - Kubernetes
* `m365` - M365
* `github` - GitHub
* `mongodbatlas` - MongoDB Atlas
* `iac` - IaC
* `oraclecloud` - Oracle Cloud Infrastructure
* `alibabacloud` - Alibaba Cloud
* `cloudflare` - Cloudflare
* `openstack` - OpenStack
* `image` - Image
* `googleworkspace` - Google Workspace
* `vercel` - Vercel
* `okta` - Okta
explode: false
style: form
- name: filter[search]
required: false
in: query
description: A search term.
schema:
type: string
- in: query
name: filter[updated_at__date]
schema:
type: string
format: date
- in: query
name: filter[updated_at__gte]
schema:
type: string
format: date-time
- in: query
name: filter[updated_at__lte]
schema:
type: string
format: date-time
- in: query
name: include
schema:
type: array
items:
type: string
enum:
- provider
description: include query parameter to allow the client to customize which
related resources should be returned.
explode: false
- name: page[number]
required: false
in: query
description: A page number within the paginated result set.
schema:
type: integer
- name: page[size]
required: false
in: query
description: Number of results to return per page.
schema:
type: integer
- name: sort
required: false
in: query
description: '[list of fields to sort by](https://jsonapi.org/format/#fetching-sorting)'
schema:
type: array
items:
type: string
enum:
- inserted_at
- -inserted_at
- updated_at
- -updated_at
- issue_key
- -issue_key
- project_key
- -project_key
- issue_status
- -issue_status
- status_synced_at
- -status_synced_at
explode: false
tags:
- Integration
security:
- JWT or API Key: []
responses:
'200':
content:
application/vnd.api+json:
schema:
$ref: '#/components/schemas/PaginatedJiraIssueList'
description: ''
/api/v1/jira-issues/{id}:
get:
operationId: api_v1_jira_issues_retrieve
description: Fetch the Jira issue linked to a finding by the link ID.
summary: Retrieve a Jira issue link
parameters:
- in: query
name: fields[jira-issues]
schema:
type: array
items:
type: string
enum:
- inserted_at
- updated_at
- finding_uid
- finding_id
- issue_key
- issue_id
- issue_url
- project_key
- issue_status
- issue_status_category
- status_synced_at
- integration
- provider
- url
description: endpoint return only specific fields in the response on a per-type
basis by including a fields[TYPE] query parameter.
explode: false
- in: path
name: id
schema:
type: string
format: uuid
description: A UUID string identifying this jira issue.
required: true
- in: query
name: include
schema:
type: array
items:
type: string
enum:
- provider
description: include query parameter to allow the client to customize which
related resources should be returned.
explode: false
tags:
- Integration
security:
- JWT or API Key: []
responses:
'200':
content:
application/vnd.api+json:
schema:
$ref: '#/components/schemas/JiraIssueResponse'
description: ''
/api/v1/lighthouse-configurations:
get:
operationId: api_v1_lighthouse_configurations_list
@@ -18618,6 +19017,134 @@ components:
$ref: '#/components/schemas/InvitationUpdate'
required:
- data
JiraIssue:
type: object
required:
- type
- id
additionalProperties: false
properties:
type:
type: string
description: The [type](https://jsonapi.org/format/#document-resource-object-identification)
member is used to describe resource objects that share common attributes
and relationships.
enum:
- jira-issues
id:
type: string
format: uuid
attributes:
type: object
properties:
inserted_at:
type: string
format: date-time
readOnly: true
updated_at:
type: string
format: date-time
readOnly: true
finding_uid:
type: string
readOnly: true
finding_id:
type: string
format: uuid
readOnly: true
issue_key:
type: string
readOnly: true
issue_id:
type: string
readOnly: true
issue_url:
type: string
format: uri
readOnly: true
project_key:
type: string
readOnly: true
issue_status:
type: string
readOnly: true
issue_status_category:
enum:
- new
- indeterminate
- done
type: string
description: |-
* `new` - New
* `indeterminate` - In progress
* `done` - Done
x-spec-enum-id: 6e5c623f6bbdd92d
readOnly: true
status_synced_at:
type: string
format: date-time
readOnly: true
nullable: true
relationships:
type: object
properties:
integration:
type: object
properties:
data:
type: object
properties:
id:
type: string
format: uuid
type:
type: string
enum:
- integrations
title: Resource Type Name
description: The [type](https://jsonapi.org/format/#document-resource-object-identification)
member is used to describe resource objects that share common
attributes and relationships.
required:
- id
- type
required:
- data
description: The identifier of the related object.
title: Resource Identifier
readOnly: true
provider:
type: object
properties:
data:
type: object
properties:
id:
type: string
format: uuid
type:
type: string
enum:
- providers
title: Resource Type Name
description: The [type](https://jsonapi.org/format/#document-resource-object-identification)
member is used to describe resource objects that share common
attributes and relationships.
required:
- id
- type
required:
- data
description: The identifier of the related object.
title: Resource Identifier
readOnly: true
JiraIssueResponse:
type: object
properties:
data:
$ref: '#/components/schemas/JiraIssue'
required:
- data
LighthouseConfig:
type: object
required:
@@ -20227,6 +20754,15 @@ components:
$ref: '#/components/schemas/Invitation'
required:
- data
PaginatedJiraIssueList:
type: object
properties:
data:
type: array
items:
$ref: '#/components/schemas/JiraIssue'
required:
- data
PaginatedLighthouseConfigList:
type: object
properties:
+53
View File
@@ -4,6 +4,7 @@ import pytest
from allauth.socialaccount.models import SocialApp
from api.db_router import MainRouter
from api.models import (
JiraIssue,
ProviderComplianceScore,
Resource,
ResourceTag,
@@ -524,3 +525,55 @@ class TestTenantComplianceSummaryModel:
assert summary1.id != summary2.id
assert summary1.requirements_passed != summary2.requirements_passed
@pytest.mark.django_db
class TestJiraIssueModel:
def test_create_jira_issue(
self, jira_integration_fixture, aws_provider, findings_fixture
):
finding = findings_fixture[0]
issue = JiraIssue.objects.create(
tenant_id=jira_integration_fixture.tenant_id,
integration=jira_integration_fixture,
provider=aws_provider,
finding_uid=finding.uid,
finding_id=finding.id,
issue_key="TEST-1",
project_key="TEST",
)
assert issue.is_linked
assert not issue.is_done
assert issue.issue_status_category == ""
def test_reservation_is_not_linked(
self, jira_integration_fixture, aws_provider, findings_fixture
):
finding = findings_fixture[0]
issue = JiraIssue.objects.create(
tenant_id=jira_integration_fixture.tenant_id,
integration=jira_integration_fixture,
provider=aws_provider,
finding_uid=finding.uid,
finding_id=finding.id,
project_key="TEST",
)
assert not issue.is_linked
def test_unique_per_integration_provider_and_finding_uid(
self, jira_integration_fixture, aws_provider_pair, findings_fixture
):
provider, provider2 = aws_provider_pair
finding = findings_fixture[0]
common = {
"tenant_id": jira_integration_fixture.tenant_id,
"integration": jira_integration_fixture,
"finding_uid": finding.uid,
"finding_id": finding.id,
"project_key": "TEST",
}
JiraIssue.objects.create(provider=provider, issue_key="TEST-1", **common)
# Same finding uid on another provider is a different finding
JiraIssue.objects.create(provider=provider2, issue_key="TEST-2", **common)
with pytest.raises(IntegrityError):
JiraIssue.objects.create(provider=provider, issue_key="TEST-3", **common)
@@ -1,63 +0,0 @@
from datetime import UTC, datetime
from itertools import islice
import pytest
from api.partitions import (
PostgresUUIDv7PartitioningStrategy,
relative_months_or_none,
)
from dateutil.relativedelta import relativedelta
from django.core.exceptions import ImproperlyConfigured
from psqlextra.partitioning import PostgresTimePartitionSize
def build_strategy(max_age):
return PostgresUUIDv7PartitioningStrategy(
size=PostgresTimePartitionSize(months=1),
count=1,
start_date=datetime.now(UTC),
max_age=max_age,
name_format="%Y_%b",
)
class TestRelativeMonthsOrNone:
@pytest.mark.parametrize("value", [None, 0])
def test_unset_or_zero_keeps_partitions_indefinitely(self, value):
assert relative_months_or_none(value) is None
@pytest.mark.parametrize("months", [1, 3, 12])
def test_value_is_interpreted_as_months(self, months):
assert relative_months_or_none(months) == relativedelta(months=months)
def test_value_is_not_interpreted_as_days(self):
assert relative_months_or_none(12) != relativedelta(days=12)
def test_negative_is_rejected(self):
with pytest.raises(ImproperlyConfigured):
relative_months_or_none(-12)
class TestToDelete:
@pytest.mark.parametrize("max_age", [None, relative_months_or_none(0)])
def test_nothing_is_deleted_without_max_age(self, max_age):
strategy = build_strategy(max_age)
assert list(islice(strategy.to_delete(), 5)) == []
def test_first_deleted_partition_is_max_age_old(self):
months = 3
strategy = build_strategy(relative_months_or_none(months))
first = next(strategy.to_delete())
expected = strategy.get_start_datetime() - relativedelta(months=months)
assert first.name() == expected.strftime("%Y_%b").lower()
def test_deleted_partitions_go_further_back_in_time(self):
strategy = build_strategy(relative_months_or_none(3))
names = [p.name() for p in islice(strategy.to_delete(), 3)]
starts = [datetime.strptime(n, "%Y_%b") for n in names]
assert starts == sorted(starts, reverse=True)
+13
View File
@@ -1591,6 +1591,19 @@ class TestLimitedVisibility:
assert response.status_code == status.HTTP_204_NO_CONTENT
def test_jira_issues_limited_to_visible_providers(
self, authenticated_client_rbac_limited, jira_issues_fixture
):
linked, other_provider_issue, _ = jira_issues_fixture
response = authenticated_client_rbac_limited.get(reverse("jiraissue-list"))
assert response.status_code == status.HTTP_200_OK
assert [item["id"] for item in response.json()["data"]] == [str(linked.id)]
response = authenticated_client_rbac_limited.get(
reverse("jiraissue-detail", kwargs={"pk": other_provider_issue.id})
)
assert response.status_code == status.HTTP_404_NOT_FOUND
def test_jira_issue_types_allowed_without_unlimited_visibility(
self, authenticated_client_rbac_limited, jira_integration_fixture
):
+166 -18
View File
@@ -34,6 +34,7 @@ from api.models import (
Integration,
Invitation,
InvitationRoleRelationship,
JiraIssue,
LighthouseProviderConfiguration,
LighthouseProviderModels,
LighthouseTenantConfiguration,
@@ -13557,6 +13558,144 @@ class TestScheduleViewSet:
assert response.status_code == status.HTTP_409_CONFLICT
@pytest.mark.django_db
class TestJiraIssueViewSet:
def test_list_hides_reservations(self, authenticated_client, jira_issues_fixture):
linked, other_provider_issue, reservation = jira_issues_fixture
response = authenticated_client.get(reverse("jiraissue-list"))
assert response.status_code == status.HTTP_200_OK
ids = {item["id"] for item in response.json()["data"]}
assert ids == {str(linked.id), str(other_provider_issue.id)}
assert str(reservation.id) not in ids
def test_retrieve(self, authenticated_client, jira_issues_fixture):
linked, *_ = jira_issues_fixture
response = authenticated_client.get(
reverse("jiraissue-detail", kwargs={"pk": linked.id})
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert data["type"] == "jira-issues"
attributes = data["attributes"]
assert attributes["finding_uid"] == linked.finding_uid
assert attributes["finding_id"] == str(linked.finding_id)
assert attributes["issue_key"] == "TEST-1"
assert attributes["issue_url"] == "https://test.atlassian.net/browse/TEST-1"
assert attributes["project_key"] == "TEST"
assert attributes["issue_status"] == "To Do"
assert attributes["issue_status_category"] == "new"
assert attributes["status_synced_at"] is not None
relationships = data["relationships"]
assert relationships["provider"]["data"]["id"] == str(linked.provider_id)
assert relationships["integration"]["data"]["id"] == str(linked.integration_id)
def test_retrieve_reservation_returns_404(
self, authenticated_client, jira_issues_fixture
):
*_, reservation = jira_issues_fixture
response = authenticated_client.get(
reverse("jiraissue-detail", kwargs={"pk": reservation.id})
)
assert response.status_code == status.HTTP_404_NOT_FOUND
def test_retrieve_other_tenant_returns_404(
self, authenticated_client, jira_issues_fixture, tenants_fixture
):
linked, *_ = jira_issues_fixture
with rls_transaction(str(tenants_fixture[2].id)):
JiraIssue.objects.filter(id=linked.id).update(
tenant_id=tenants_fixture[2].id
)
response = authenticated_client.get(
reverse("jiraissue-detail", kwargs={"pk": linked.id})
)
assert response.status_code == status.HTTP_404_NOT_FOUND
@pytest.mark.parametrize(
"filter_name, filter_value, expected_keys",
[
("finding_uid", "test_finding_uid_1", {"TEST-1"}),
(
"finding_uid__in",
"test_finding_uid_1,test_finding_uid_other_provider",
{"TEST-1", "TEST-2"},
),
("finding_uid__in", "does-not-exist", set()),
("issue_key", "TEST-2", {"TEST-2"}),
("issue_status_category", "done", {"TEST-2"}),
("issue_status_category__in", "new,indeterminate", {"TEST-1"}),
("project_key", "TEST", {"TEST-1", "TEST-2"}),
("search", "TEST-1", {"TEST-1"}),
],
)
def test_filters(
self,
authenticated_client,
jira_issues_fixture,
filter_name,
filter_value,
expected_keys,
):
response = authenticated_client.get(
reverse("jiraissue-list"), {f"filter[{filter_name}]": filter_value}
)
assert response.status_code == status.HTTP_200_OK
keys = {item["attributes"]["issue_key"] for item in response.json()["data"]}
assert keys == expected_keys
def test_filter_by_provider(self, authenticated_client, jira_issues_fixture):
linked, other_provider_issue, _ = jira_issues_fixture
response = authenticated_client.get(
reverse("jiraissue-list"),
{"filter[provider_id]": str(other_provider_issue.provider_id)},
)
assert response.status_code == status.HTTP_200_OK
assert [item["id"] for item in response.json()["data"]] == [
str(other_provider_issue.id)
]
def test_filter_by_integration_and_finding_id(
self, authenticated_client, jira_issues_fixture
):
linked, *_ = jira_issues_fixture
response = authenticated_client.get(
reverse("jiraissue-list"),
{
"filter[integration]": str(linked.integration_id),
"filter[finding_id]": str(linked.finding_id),
},
)
assert response.status_code == status.HTTP_200_OK
assert [item["id"] for item in response.json()["data"]] == [str(linked.id)]
def test_invalid_filter(self, authenticated_client, jira_issues_fixture):
response = authenticated_client.get(
reverse("jiraissue-list"), {"filter[invalid]": "x"}
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_include_provider(self, authenticated_client, jira_issues_fixture):
response = authenticated_client.get(
reverse("jiraissue-list"), {"include": "provider"}
)
assert response.status_code == status.HTTP_200_OK
included_types = {item["type"] for item in response.json()["included"]}
assert included_types == {"providers"}
def test_read_only(self, authenticated_client, jira_issues_fixture):
linked, *_ = jira_issues_fixture
response = authenticated_client.post(
reverse("jiraissue-list"),
data=json.dumps({"data": {"type": "jira-issues", "attributes": {}}}),
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_405_METHOD_NOT_ALLOWED
response = authenticated_client.delete(
reverse("jiraissue-detail", kwargs={"pk": linked.id})
)
assert response.status_code == status.HTTP_405_METHOD_NOT_ALLOWED
@pytest.mark.django_db
class TestIntegrationViewSet:
def test_integrations_list(self, authenticated_client, integrations_fixture):
@@ -18333,14 +18472,19 @@ class TestMuteRuleViewSet:
assert len(data) == 2
assert data[0]["id"] == str(mute_rules_fixture[first_index].id)
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("api.v1.views.chain")
@patch("api.v1.views.reaggregate_all_finding_group_summaries_task.si")
@patch("api.v1.views.mute_historical_findings_task.si")
@patch("api.v1.views.transaction.on_commit", side_effect=lambda fn: fn())
def test_mute_rules_create_valid(
self,
_mock_on_commit,
mock_mute_task,
mock_mute_signature,
mock_reaggregate_signature,
mock_chain,
authenticated_client,
findings_fixture,
create_test_user,
):
"""Test creating a valid mute rule."""
finding_ids = [str(findings_fixture[0].id)]
@@ -18367,20 +18511,24 @@ class TestMuteRuleViewSet:
assert response_data["attributes"]["name"] == "New Mute Rule"
assert response_data["attributes"]["reason"] == "Security exception approved"
# Verify the finding was immediately muted
from api.models import Finding
finding = Finding.objects.get(id=findings_fixture[0].id)
assert finding.muted is False
assert finding.muted_at is None
assert finding.muted_reason is None
assert finding.muted is True
assert finding.muted_at is not None
assert finding.muted_reason == "Security exception approved"
mock_mute_task.assert_called_once_with(
kwargs={
"tenant_id": str(finding.tenant_id),
"mute_rule_id": response_data["id"],
"provider_ids": [str(finding.scan.provider_id)],
}
# Verify background task chain was called: mute → reaggregate all
mock_mute_signature.assert_called_once()
mock_reaggregate_signature.assert_called_once()
mock_chain.assert_called_once_with(
mock_mute_signature.return_value,
mock_reaggregate_signature.return_value,
)
mock_chain.return_value.apply_async.assert_called_once()
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("tasks.tasks.mute_historical_findings_task.apply_async")
def test_mute_rules_create_converts_finding_ids_to_uids(
self,
mock_task,
@@ -18416,7 +18564,7 @@ class TestMuteRuleViewSet:
]
assert set(mute_rule.finding_uids) == set(expected_uids)
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("tasks.tasks.mute_historical_findings_task.apply_async")
def test_mute_rules_deduplicates_uids(
self,
mock_task,
@@ -18483,10 +18631,10 @@ class TestMuteRuleViewSet:
finding1.refresh_from_db()
finding2.refresh_from_db()
assert finding1.muted is False
assert finding2.muted is False
assert finding1.muted is True
assert finding2.muted is True
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("tasks.tasks.mute_historical_findings_task.apply_async")
def test_mute_rules_create_overlap_detection_active(
self,
mock_task,
@@ -18519,7 +18667,7 @@ class TestMuteRuleViewSet:
"already muted" in error_detail.lower() or "overlap" in error_detail.lower()
)
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("tasks.tasks.mute_historical_findings_task.apply_async")
def test_mute_rules_create_no_overlap_with_inactive(
self,
mock_task,
@@ -18575,7 +18723,7 @@ class TestMuteRuleViewSet:
== "/data/attributes/finding_ids"
)
@patch("api.v1.views.mute_findings_in_latest_scans_task.apply_async")
@patch("tasks.tasks.mute_historical_findings_task.apply_async")
def test_mute_rules_create_invalid_finding_ids(
self, mock_task, authenticated_client
):
+36
View File
@@ -14,6 +14,7 @@ from api.models import (
IntegrationProviderRelationship,
Invitation,
InvitationRoleRelationship,
JiraIssue,
LighthouseConfiguration,
LighthouseProviderConfiguration,
LighthouseProviderModels,
@@ -4149,6 +4150,41 @@ class LighthouseProviderModelsUpdateSerializer(BaseWriteSerializer):
# Mute Rules
class JiraIssueSerializer(RLSSerializer):
"""
Read-only view of a Jira issue linked to a finding by a Jira integration.
Rows are keyed on the finding ``uid`` so the same finding maps to the same
issue across scans. ``issue_status`` is the last status Prowler observed in
Jira (refreshed whenever a dispatch touches the finding), not a live value.
"""
class Meta:
model = JiraIssue
fields = [
"id",
"inserted_at",
"updated_at",
"finding_uid",
"finding_id",
"issue_key",
"issue_id",
"issue_url",
"project_key",
"issue_status",
"issue_status_category",
"status_synced_at",
"integration",
"provider",
"url",
]
read_only_fields = fields
included_serializers = {
"provider": "api.v1.serializers.ProviderIncludeSerializer",
}
class MuteRuleSerializer(RLSSerializer):
"""
Serializer for reading MuteRule instances.
+2
View File
@@ -14,6 +14,7 @@ from api.v1.views import (
IntegrationViewSet,
InvitationAcceptViewSet,
InvitationViewSet,
JiraIssueViewSet,
LighthouseConfigViewSet,
LighthouseProviderConfigViewSet,
LighthouseProviderModelsViewSet,
@@ -107,6 +108,7 @@ router.register(
basename="lighthouse-models",
)
router.register(r"mute-rules", MuteRuleViewSet, basename="mute-rule")
router.register(r"jira-issues", JiraIssueViewSet, basename="jiraissue")
tenants_router = routers.NestedSimpleRouter(router, r"tenants", lookup="tenant")
tenants_router.register(
+81 -19
View File
@@ -54,6 +54,7 @@ from api.filters import (
IntegrationFilter,
IntegrationJiraFindingsFilter,
InvitationFilter,
JiraIssueFilter,
LatestFindingFilter,
LatestFindingGroupFilter,
LatestFindingGroupSummaryFilter,
@@ -89,6 +90,7 @@ from api.models import (
Integration,
Invitation,
InvitationRoleRelationship,
JiraIssue,
LighthouseConfiguration,
LighthouseProviderConfiguration,
LighthouseProviderModels,
@@ -179,6 +181,7 @@ from api.v1.serializers import (
InvitationCreateSerializer,
InvitationSerializer,
InvitationUpdateSerializer,
JiraIssueSerializer,
LighthouseConfigCreateSerializer,
LighthouseConfigSerializer,
LighthouseConfigUpdateSerializer,
@@ -244,6 +247,7 @@ from api.v1.serializers import (
UserUpdateSerializer,
)
from botocore.exceptions import ClientError, NoCredentialsError, ParamValidationError
from celery import chain
from celery.result import AsyncResult
from config.custom_logging import BackendLogger
from config.env import env
@@ -341,7 +345,8 @@ from tasks.tasks import (
enqueue_scan_execution_on_commit,
get_active_provider_scan,
jira_integration_task,
mute_findings_in_latest_scans_task,
mute_historical_findings_task,
reaggregate_all_finding_group_summaries_task,
refresh_lighthouse_provider_models_task,
)
@@ -7486,6 +7491,56 @@ class TenantApiKeyViewSet(BaseRLSViewSet):
return Response(data=serializer.data, status=status.HTTP_200_OK)
# Jira issues
@extend_schema_view(
list=extend_schema(
tags=["Integration"],
summary="List Jira issues linked to findings",
description=(
"Retrieve the Jira issues created from findings through Jira integrations. "
"Each entry links a finding UID to the latest Jira issue created for it, "
"with the last status observed in Jira. Use `filter[finding_uid__in]` "
"and `filter[provider_id]` to check whether specific findings already "
"have a ticket."
),
),
retrieve=extend_schema(
tags=["Integration"],
summary="Retrieve a Jira issue link",
description="Fetch the Jira issue linked to a finding by the link ID.",
),
)
class JiraIssueViewSet(BaseRLSViewSet):
queryset = JiraIssue.objects.all()
serializer_class = JiraIssueSerializer
filterset_class = JiraIssueFilter
http_method_names = ["get"]
search_fields = ["finding_uid", "issue_key"]
ordering = ["-inserted_at"]
ordering_fields = [
"inserted_at",
"updated_at",
"issue_key",
"project_key",
"issue_status",
"status_synced_at",
]
# RBAC required permissions (implicit -> MANAGE_PROVIDERS enables unlimited
# visibility or check visibility via provider group, like findings)
required_permissions = []
def get_queryset(self):
if getattr(self, "swagger_fake_view", False):
return JiraIssue.objects.none()
# Rows without an issue key are in-flight reservations, not links
queryset = JiraIssue.objects.filter(tenant_id=self.request.tenant_id).exclude(
issue_key=""
)
if not self.user_role.unlimited_visibility:
queryset = queryset.filter(provider__in=get_providers(self.user_role))
return queryset.select_related("provider", "integration")
# MuteRules
@extend_schema_view(
list=extend_schema(
@@ -7549,28 +7604,35 @@ class MuteRuleViewSet(BaseRLSViewSet):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
tenant_id = str(request.tenant_id)
finding_ids = serializer.validated_data["finding_ids"]
provider_ids = list(
dict.fromkeys(
Finding.all_objects.filter(
id__in=finding_ids, tenant_id=tenant_id
).values_list("scan__provider_id", flat=True)
)
)
# Create the mute rule
mute_rule = serializer.save()
transaction.on_commit(
lambda: mute_findings_in_latest_scans_task.apply_async(
kwargs={
"tenant_id": tenant_id,
"mute_rule_id": str(mute_rule.id),
"provider_ids": [str(provider_id) for provider_id in provider_ids],
}
)
tenant_id = str(request.tenant_id)
finding_ids = request.data.get("finding_ids", [])
# Immediately mute the selected findings
Finding.all_objects.filter(
id__in=finding_ids, tenant_id=tenant_id, muted=False
).update(
muted=True,
muted_at=mute_rule.inserted_at,
muted_reason=mute_rule.reason,
)
# Launch background task for historical muting + reaggregation
transaction.on_commit(
lambda: chain(
mute_historical_findings_task.si(
tenant_id=tenant_id,
mute_rule_id=str(mute_rule.id),
),
reaggregate_all_finding_group_summaries_task.si(
tenant_id=tenant_id,
),
).apply_async()
)
# Return the created mute rule
serializer = self.get_serializer(mute_rule)
return Response(
data=serializer.data,
+47
View File
@@ -20,6 +20,7 @@ from api.models import (
Integration,
IntegrationProviderRelationship,
Invitation,
JiraIssue,
LighthouseConfiguration,
Membership,
MuteRule,
@@ -1470,6 +1471,52 @@ def jira_integration_fixture(tenants_fixture):
)
@pytest.fixture
def jira_issues_fixture(jira_integration_fixture, aws_provider_pair, findings_fixture):
"""Two linked issues (one per provider) and one in-flight reservation."""
provider, provider2 = aws_provider_pair
finding1, finding2 = findings_fixture
tenant_id = jira_integration_fixture.tenant_id
with rls_transaction(str(tenant_id)):
linked = JiraIssue.objects.create(
tenant_id=tenant_id,
integration=jira_integration_fixture,
provider=provider,
finding_uid=finding1.uid,
finding_id=finding1.id,
issue_key="TEST-1",
issue_id="10001",
issue_url="https://test.atlassian.net/browse/TEST-1",
project_key="TEST",
issue_status="To Do",
issue_status_category=JiraIssue.StatusCategoryChoices.NEW,
status_synced_at=datetime.now(UTC),
)
hidden_provider_issue = JiraIssue.objects.create(
tenant_id=tenant_id,
integration=jira_integration_fixture,
provider=provider2,
finding_uid="test_finding_uid_other_provider",
finding_id=finding2.id,
issue_key="TEST-2",
issue_id="10002",
issue_url="https://test.atlassian.net/browse/TEST-2",
project_key="TEST",
issue_status="Done",
issue_status_category=JiraIssue.StatusCategoryChoices.DONE,
status_synced_at=datetime.now(UTC),
)
reservation = JiraIssue.objects.create(
tenant_id=tenant_id,
integration=jira_integration_fixture,
provider=provider,
finding_uid=finding2.uid,
finding_id=finding2.id,
project_key="TEST",
)
return linked, hidden_provider_issue, reservation
@pytest.fixture
def backfill_scan_metadata_fixture(scans_fixture, findings_fixture):
for scan_instance in scans_fixture:
+2
View File
@@ -5,6 +5,7 @@ from api.db_utils import batch_delete, rls_transaction
from api.models import (
AttackPathsScan,
Finding,
JiraIssue,
Provider,
ProviderComplianceScore,
Resource,
@@ -86,6 +87,7 @@ def delete_provider(tenant_id: str, pk: str):
deletion_steps = [
("Scan Summaries", ScanSummary.all_objects.filter(scan__provider=instance)),
("Jira Issues", JiraIssue.objects.filter(provider=instance)),
("Findings", Finding.all_objects.filter(scan__provider=instance)),
("Resources", Resource.all_objects.filter(provider=instance)),
("Scans", Scan.all_objects.filter(provider=instance)),
+325 -58
View File
@@ -1,25 +1,25 @@
import os
import time
from datetime import UTC, datetime
from datetime import UTC, datetime, timedelta
from glob import glob
from urllib.parse import quote
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.db_utils import REPLICA_MAX_ATTEMPTS, REPLICA_RETRY_BASE_DELAY, rls_transaction
from api.models import Finding, Integration, Provider
from api.models import Finding, Integration, JiraIssue, Provider
from api.rls import Tenant
from api.utils import initialize_prowler_integration, initialize_prowler_provider
from celery.utils.log import get_task_logger
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
from django.conf import settings
from django.db import OperationalError
from django.db import IntegrityError, OperationalError
from django.utils import timezone
from prowler.lib.outputs.asff.asff import ASFF
from prowler.lib.outputs.compliance.generic.generic import GenericCompliance
from prowler.lib.outputs.csv.csv import CSV
from prowler.lib.outputs.finding import Finding as FindingOutput
from prowler.lib.outputs.html.html import HTML
from prowler.lib.outputs.jira.exceptions.exceptions import JiraBaseException
from prowler.lib.outputs.jira.jira import Jira
from prowler.lib.outputs.ocsf.ocsf import OCSF
from prowler.providers.aws.aws_provider import AwsProvider
from prowler.providers.aws.lib.s3.s3 import S3
@@ -482,6 +482,32 @@ def upload_security_hub_integration(
JIRA_LABEL_PREFIX = "prowler"
JIRA_LABEL_MAX_LENGTH = 255
def sanitize_jira_label(label: str) -> str:
"""Make a value safe to use as a Jira label.
Jira rejects labels containing whitespace or longer than 255 characters. The
transformation is deterministic so the same finding always yields the same
label: whitespace runs become a single underscore, control characters are
dropped and the result is truncated. Mirrors ``Jira.sanitize_label`` in the
SDK; kept local so the API does not depend on an unreleased SDK symbol.
"""
if not label:
return ""
cleaned = "".join(ch for ch in str(label) if ch.isprintable() or ch.isspace())
return "_".join(cleaned.split())[:JIRA_LABEL_MAX_LENGTH]
def sanitize_jira_labels(labels: list[str]) -> list[str]:
"""Sanitize a list of labels, dropping empties and duplicates (order kept)."""
result: list[str] = []
for label in labels or []:
sanitized = sanitize_jira_label(label)
if sanitized and sanitized not in result:
result.append(sanitized)
return result
def build_jira_finding_url(finding_uid: str) -> str:
@@ -510,9 +536,9 @@ def build_jira_issue_labels(
f"{JIRA_LABEL_PREFIX}-{provider}" if provider else "",
f"{JIRA_LABEL_PREFIX}-{severity}" if severity else "",
f"{JIRA_LABEL_PREFIX}-{check_id}" if check_id else "",
Jira.build_finding_label(finding_uid),
f"{JIRA_LABEL_PREFIX}-finding-{finding_uid}" if finding_uid else "",
]
return Jira.sanitize_labels(raw_labels)
return sanitize_jira_labels(raw_labels)
def get_tenant_name(tenant_id: str) -> str:
@@ -530,6 +556,182 @@ def get_tenant_name(tenant_id: str) -> str:
return ""
# Findings are pre-checked against existing Jira issues in chunks so the IN list
# stays bounded however many findings a dispatch carries
JIRA_DEDUP_CHUNK_SIZE = 500
# A reservation (row without issue key) older than this belongs to a run that
# died mid-send and can be reclaimed
JIRA_RESERVATION_TTL = timedelta(minutes=15)
# Cap on the per-finding detail returned in the task result; counts are exact
JIRA_SKIPPED_REPORT_LIMIT = 100
def _load_finding_refs(finding_ids: list[str]) -> dict[str, tuple[str, str]]:
"""Map finding id -> (provider id, finding uid) for the batch, in one query."""
refs = {}
for finding_id, provider_id, uid in Finding.all_objects.filter(
id__in=finding_ids
).values_list("id", "scan__provider_id", "uid"):
refs[str(finding_id)] = (str(provider_id), uid)
return refs
def _load_existing_jira_issues(
tenant_id: str, integration_id: str, refs: dict[str, tuple[str, str]]
) -> dict[tuple[str, str], JiraIssue]:
"""Load the Jira issue rows already linked to the batch's findings.
Grouped by provider and chunked so each query is a bounded index lookup on
(tenant, integration, provider, finding_uid).
"""
uids_by_provider: dict[str, list[str]] = {}
for provider_id, uid in refs.values():
uids_by_provider.setdefault(provider_id, []).append(uid)
existing: dict[tuple[str, str], JiraIssue] = {}
for provider_id, uids in uids_by_provider.items():
for start in range(0, len(uids), JIRA_DEDUP_CHUNK_SIZE):
chunk = uids[start : start + JIRA_DEDUP_CHUNK_SIZE]
for row in JiraIssue.objects.filter(
tenant_id=tenant_id,
integration_id=integration_id,
provider_id=provider_id,
finding_uid__in=chunk,
):
existing[(str(row.provider_id), row.finding_uid)] = row
return existing
def _refresh_jira_issue_statuses(
tenant_id: str, jira_integration, rows: list[JiraIssue]
) -> dict[str, dict] | None:
"""Fetch the current Jira status of linked rows and cache it on them.
Returns the statuses keyed by issue key (keys missing from the result no
longer exist in Jira), or None when Jira could not be queried, in which case
the cached values are left untouched.
"""
keys = [row.issue_key for row in rows if row.issue_key]
if not keys:
return {}
try:
statuses = jira_integration.get_issues_status(keys)
except JiraBaseException as error:
logger.warning(
"Could not refresh Jira issue statuses, keeping cached values: %s",
error.message or error,
)
return None
except Exception:
logger.exception("Could not refresh Jira issue statuses, keeping cached values")
return None
now = timezone.now()
for row in rows:
status = statuses.get(row.issue_key)
if status is None:
# The issue is gone: keep the key for reference but mark it as done so
# the next send creates a fresh issue
row.issue_status = ""
row.issue_status_category = JiraIssue.StatusCategoryChoices.DONE
else:
row.issue_status = status.get("status", "")[:64]
row.issue_status_category = status.get("status_category", "")[:16]
row.status_synced_at = now
with rls_transaction(tenant_id):
JiraIssue.objects.bulk_update(
rows, ["issue_status", "issue_status_category", "status_synced_at"]
)
return statuses
def _reserve_jira_issue(
tenant_id: str,
integration_id: str,
provider_id: str,
finding_uid: str,
finding_id: str,
project_key: str,
) -> JiraIssue | None:
"""Claim the (integration, provider, finding uid) slot before calling Jira.
The unique constraint makes this the arbiter between concurrent runs: only
the run that inserts the row (or reclaims an expired reservation) sends the
finding. Returns None when another run owns the slot.
"""
with rls_transaction(tenant_id):
try:
row, created = JiraIssue.objects.get_or_create(
tenant_id=tenant_id,
integration_id=integration_id,
provider_id=provider_id,
finding_uid=finding_uid,
defaults={"finding_id": finding_id, "project_key": project_key},
)
except IntegrityError:
return None
if created:
return row
if row.issue_key:
# Linked by a concurrent run between the pre-check and now
return None
if timezone.now() - row.updated_at < JIRA_RESERVATION_TTL:
# Another run is sending this finding right now
return None
# Expired reservation from a run that died mid-send: reclaim it
row.finding_id = finding_id
row.project_key = project_key
row.save(update_fields=["finding_id", "project_key", "updated_at"])
return row
def _link_jira_issue(
tenant_id: str, row: JiraIssue, issue: dict, finding_id: str, project_key: str
) -> None:
"""Point the row at the issue Jira just created."""
with rls_transaction(tenant_id):
row.issue_key = (issue.get("key") or "")[:64]
row.issue_id = str(issue.get("id") or "")[:64]
row.issue_url = (issue.get("url") or "")[:2048]
row.project_key = project_key
row.finding_id = finding_id
row.issue_status = ""
row.issue_status_category = JiraIssue.StatusCategoryChoices.NEW
row.status_synced_at = None
row.save(
update_fields=[
"issue_key",
"issue_id",
"issue_url",
"project_key",
"finding_id",
"issue_status",
"issue_status_category",
"status_synced_at",
"updated_at",
]
)
def _release_jira_issue(tenant_id: str, row: JiraIssue) -> None:
"""Drop a reservation whose send failed so the finding can be retried."""
if row.issue_key:
# A previously linked (now closed) issue stays linked; the send failed so
# there is nothing newer to point at
return
with rls_transaction(tenant_id):
JiraIssue.objects.filter(id=row.id, issue_key="").delete()
def _skipped_entry(finding_id: str, row: JiraIssue) -> dict:
return {
"finding_id": str(finding_id),
"issue_key": row.issue_key,
"issue_url": row.issue_url,
"issue_status": row.issue_status,
}
def send_findings_to_jira(
tenant_id: str,
integration_id: str,
@@ -537,14 +739,50 @@ def send_findings_to_jira(
issue_type: str,
finding_ids: list[str],
):
"""Create one Jira issue per finding, skipping findings that already have one.
Findings are matched to existing issues by (integration, provider, finding
uid), so a finding that was already sent in a previous scan is recognised.
Findings whose linked issue is still open are skipped and reported; findings
whose issue is closed or was deleted in Jira get a new issue that replaces
the link.
"""
with rls_transaction(tenant_id):
integration = Integration.objects.get(id=integration_id)
jira_integration = initialize_prowler_integration(integration)
tenant_info = get_tenant_name(tenant_id)
finding_refs = _load_finding_refs(finding_ids)
existing = _load_existing_jira_issues(tenant_id, integration_id, finding_refs)
# Refresh the status of the linked issues in bulk so closed/deleted ones can be
# replaced. If Jira cannot be queried the linked findings are skipped as-is.
linked_rows = [row for row in existing.values() if row.issue_key]
statuses = (
_refresh_jira_issue_statuses(tenant_id, jira_integration, linked_rows)
if linked_rows
else {}
)
num_tickets_created = 0
error_messages = []
skipped: list[dict] = []
error_messages: list[str] = []
created_rows: list[JiraIssue] = []
for finding_id in finding_ids:
finding_id = str(finding_id)
provider_id, finding_uid = finding_refs.get(finding_id, (None, None))
row = existing.get((provider_id, finding_uid)) if provider_id else None
if row is not None:
if row.issue_key:
if statuses is None or not row.is_done:
# Still open (or status unknown): already ticketed
skipped.append(_skipped_entry(finding_id, row))
continue
# Closed or deleted in Jira: create a replacement below
elif timezone.now() - row.updated_at < JIRA_RESERVATION_TTL:
# Another run is sending this finding right now
skipped.append(_skipped_entry(finding_id, row))
continue
with rls_transaction(tenant_id):
finding_instance = (
Finding.all_objects.select_related("scan__provider")
@@ -574,69 +812,98 @@ def send_findings_to_jira(
remediation_code = remediation.get("code", {})
provider_type = finding_instance.scan.provider.provider
if provider_id is None:
provider_id = str(finding_instance.scan.provider_id)
finding_uid = finding_instance.uid
issue_labels = build_jira_issue_labels(
finding_uid=finding_instance.uid,
finding_uid=finding_uid,
provider=provider_type,
severity=finding_instance.severity,
check_id=finding_instance.check_id,
)
finding_url = build_jira_finding_url(finding_instance.uid)
finding_url = build_jira_finding_url(finding_uid)
try:
# Send the individual finding to Jira
result = jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=provider_type,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get("nativeiac", ""),
remediation_code_terraform=remediation_code.get("terraform", ""),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
issue_labels=issue_labels,
finding_url=finding_url,
tenant_info=tenant_info,
)
except JiraBaseException as error:
error_message = error.message or JIRA_GENERIC_SEND_ERROR
logger.exception(
"Failed to send finding %s to Jira: %s", finding_id, error_message
)
error_messages.append(error_message)
continue
except Exception:
logger.exception("Failed to send finding %s to Jira", finding_id)
error_messages.append(JIRA_GENERIC_SEND_ERROR)
if row is None:
row = _reserve_jira_issue(
tenant_id,
integration_id,
provider_id,
finding_uid,
finding_id,
project_key,
)
if row is None:
skipped.append({"finding_id": finding_id})
continue
if result:
num_tickets_created += 1
logger.info(
"Finding %s sent to Jira as %s",
finding_id,
result.get("key") if isinstance(result, dict) else result,
)
else:
error_message = JIRA_GENERIC_SEND_ERROR
logger.error(error_message)
error_messages.append(error_message)
try:
# Send the individual finding to Jira
result = jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=provider_type,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get("nativeiac", ""),
remediation_code_terraform=remediation_code.get("terraform", ""),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
issue_labels=issue_labels,
finding_url=finding_url,
tenant_info=tenant_info,
)
except JiraBaseException as error:
error_message = error.message or JIRA_GENERIC_SEND_ERROR
logger.exception(
"Failed to send finding %s to Jira: %s", finding_id, error_message
)
error_messages.append(error_message)
_release_jira_issue(tenant_id, row)
continue
except Exception:
logger.exception("Failed to send finding %s to Jira", finding_id)
error_messages.append(JIRA_GENERIC_SEND_ERROR)
_release_jira_issue(tenant_id, row)
continue
if result:
num_tickets_created += 1
issue = result if isinstance(result, dict) else {}
logger.info(
"Finding %s sent to Jira as %s", finding_id, issue.get("key") or result
)
_link_jira_issue(tenant_id, row, issue, finding_id, project_key)
created_rows.append(row)
else:
error_message = JIRA_GENERIC_SEND_ERROR
logger.error(error_message)
error_messages.append(error_message)
_release_jira_issue(tenant_id, row)
# Record the initial status of the issues just created, in bulk
if created_rows:
_refresh_jira_issue_statuses(
tenant_id, jira_integration, [row for row in created_rows if row.issue_key]
)
result = {
"created_count": num_tickets_created,
"failed_count": len(finding_ids) - num_tickets_created,
"skipped_count": len(skipped),
"failed_count": len(finding_ids) - num_tickets_created - len(skipped),
}
if skipped:
result["skipped"] = skipped[:JIRA_SKIPPED_REPORT_LIMIT]
if error_messages:
result["error"] = "; ".join(dict.fromkeys(error_messages))
+46 -87
View File
@@ -1,104 +1,63 @@
from collections.abc import Iterable
from api.db_utils import rls_transaction
from api.models import Finding, MuteRule, Scan, StateChoices
from api.models import Finding, MuteRule
from celery.utils.log import get_task_logger
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
from tasks.utils import batched
logger = get_task_logger(__name__)
def _mute_findings_for_rule(
*,
tenant_id: str,
scan_id: str,
finding_uids: Iterable[str],
muted_at,
muted_reason: str,
) -> int:
finding_uids = list(finding_uids)
if not finding_uids:
return 0
def mute_historical_findings(tenant_id: str, mute_rule_id: str):
"""
Mute historical findings that match the given mute rule.
return Finding.all_objects.filter(
tenant_id=tenant_id,
scan_id=scan_id,
uid__in=finding_uids,
muted=False,
).update(
muted=True,
muted_at=muted_at,
muted_reason=muted_reason,
)
This function processes findings in batches, updating their muted status
and adding the mute reason.
Args:
tenant_id (str): The tenant ID for RLS context
mute_rule_id (str): The ID of the mute rule to apply
def mute_findings_in_latest_scans(
tenant_id: str, mute_rule_id: str, provider_ids: list[str]
) -> dict:
"""Apply a mute rule to the latest completed scan of each provider."""
provider_ids = list(dict.fromkeys(provider_ids))
Returns:
dict: Summary of the muting operation with findings_muted count
"""
findings_muted_count = 0
# Get the list of UIDs to mute and the reason
with rls_transaction(tenant_id):
mute_rule = MuteRule.objects.get(id=mute_rule_id, tenant_id=tenant_id)
latest_scans = list(
Scan.objects.filter(
tenant_id=tenant_id,
provider_id__in=provider_ids,
state=StateChoices.COMPLETED,
completed_at__isnull=False,
)
.order_by("provider_id", "-completed_at", "-inserted_at", "-id")
.distinct("provider_id")
.values_list("id", flat=True)
)
changed_scan_ids = []
findings_muted = 0
for scan_id in latest_scans:
updated = _mute_findings_for_rule(
tenant_id=tenant_id,
scan_id=str(scan_id),
finding_uids=mute_rule.finding_uids,
muted_at=mute_rule.inserted_at,
muted_reason=mute_rule.reason,
)
if updated:
findings_muted += updated
changed_scan_ids.append(str(scan_id))
logger.info(
"Muted %d findings in %d latest scans for rule %s",
findings_muted,
len(changed_scan_ids),
mute_rule_id,
)
return {
"findings_muted": findings_muted,
"rule_id": mute_rule_id,
"scan_ids": changed_scan_ids,
}
def reconcile_scan_mute_rules(tenant_id: str, scan_id: str) -> dict:
"""Apply the current enabled mute rules to one completed scan."""
findings_muted = 0
finding_uids = mute_rule.finding_uids
mute_reason = mute_rule.reason
muted_at = mute_rule.inserted_at
# Query findings that match the UIDs and are not already muted
with rls_transaction(tenant_id):
mute_rules = MuteRule.objects.filter(tenant_id=tenant_id, enabled=True).values(
"finding_uids", "reason", "inserted_at"
findings_to_mute = Finding.objects.filter(
tenant_id=tenant_id, uid__in=finding_uids, muted=False
)
total_findings = findings_to_mute.count()
logger.info(
f"Processing {total_findings} findings for mute rule {mute_rule_id}"
)
for mute_rule in mute_rules:
findings_muted += _mute_findings_for_rule(
tenant_id=tenant_id,
scan_id=scan_id,
finding_uids=mute_rule["finding_uids"],
muted_at=mute_rule["inserted_at"],
muted_reason=mute_rule["reason"],
)
if total_findings > 0:
for batch, is_last in batched(
findings_to_mute.iterator(), DJANGO_FINDINGS_BATCH_SIZE
):
batch_ids = [f.id for f in batch]
updated_count = Finding.all_objects.filter(
id__in=batch_ids, tenant_id=tenant_id
).update(
muted=True,
muted_at=muted_at,
muted_reason=mute_reason,
)
findings_muted_count += updated_count
logger.info(
"Reconciled mute rules for scan %s; muted %d findings",
scan_id,
findings_muted,
)
return {"findings_muted": findings_muted, "scan_id": str(scan_id)}
logger.info(f"Muted {findings_muted_count} findings for rule {mute_rule_id}")
return {
"findings_muted": findings_muted_count,
"rule_id": mute_rule_id,
}
+36 -57
View File
@@ -5,6 +5,7 @@ import json
import random
import re
import time
import uuid
from collections import defaultdict
from collections.abc import Callable, Iterable
from datetime import UTC, datetime
@@ -72,7 +73,6 @@ from tasks.jobs.queries import (
COMPLIANCE_UPSERT_TENANT_SUMMARY_SQL,
)
from tasks.utils import CustomEncoder, batched
from uuid6 import uuid7
logger = get_task_logger(__name__)
@@ -1756,27 +1756,32 @@ def aggregate_findings(tenant_id: str, scan_id: str):
def _aggregate_findings_by_region(
tenant_id: str,
scan_id: str,
normalized_threatscore_id: str,
threatscore_requirements_by_check: dict[str, list[str]],
tenant_id: str, scan_id: str, modeled_threatscore_compliance_id: str
) -> tuple[dict, dict]:
"""
Aggregate findings by region using streaming, column-scoped ORM reads.
Reads only the consumed columns as tuples via ``values_list`` and streams
them with ``.iterator()``, using the denormalized ``resource_regions`` array
instead of ``prefetch_related("resources")``. ThreatScore requirement ids
are resolved per ``check_id`` from ``threatscore_requirements_by_check``.
instead of ``prefetch_related("resources")``. ``resource_regions`` mirrors the
regions of a finding's related resources, so it yields the same per-region
tally without joining the resource table.
Args:
tenant_id: Tenant UUID
scan_id: Scan UUID
modeled_threatscore_compliance_id: ID for ThreatScore compliance framework
Returns:
tuple: (check_status_by_region, findings_count_by_compliance)
- check_status_by_region: {region: {check_id: status}}
- findings_count_by_compliance: {region: {normalized_threatscore_id: {requirement_id: {total, pass}}}}
- findings_count_by_compliance: {region: {normalized_id: {requirement_id: {total, pass}}}}
"""
check_status_by_region: dict = {}
findings_count_by_compliance: dict = {}
normalized_id = re.sub(r"[^a-z0-9]", "", modeled_threatscore_compliance_id.lower())
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
findings = (
Finding.all_objects.filter(
@@ -1785,12 +1790,14 @@ def _aggregate_findings_by_region(
muted=False,
status__in=["PASS", "FAIL"],
)
.values_list("check_id", "status", "resource_regions")
.values_list("check_id", "status", "resource_regions", "compliance")
.iterator(chunk_size=DJANGO_FINDINGS_BATCH_SIZE)
)
for check_id, status, resource_regions in findings:
threatscore_requirements = threatscore_requirements_by_check.get(check_id)
for check_id, status, resource_regions, compliance in findings:
threatscore_requirements = (compliance or {}).get(
modeled_threatscore_compliance_id
)
for region in resource_regions or ():
# Priority: FAIL > any other status
@@ -1802,7 +1809,7 @@ def _aggregate_findings_by_region(
if threatscore_requirements:
compliance_key = findings_count_by_compliance.setdefault(
region, {}
).setdefault(normalized_threatscore_id, {})
).setdefault(normalized_id, {})
for requirement_id in threatscore_requirements:
requirement_stats = compliance_key.setdefault(
@@ -1841,28 +1848,15 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
provider_instance.provider
]
normalized_threatscore_id = _normalized_compliance_key(
"ProwlerThreatScore", "1.0"
)
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
requirement_lookup: dict[str, list[tuple[str, str]]] = {}
threatscore_requirements_by_check: dict[str, list[str]] = {}
for compliance_id, compliance in compliance_template.items():
is_threatscore = (
_normalized_compliance_key(
compliance["framework"], compliance["version"]
)
== normalized_threatscore_id
)
for requirement_id, requirement in compliance["requirements"].items():
for check_id in requirement["checks"].keys():
requirement_lookup.setdefault(check_id, []).append(
(compliance_id, requirement_id)
)
if is_threatscore:
threatscore_requirements_by_check.setdefault(
check_id, []
).append(requirement_id)
regions = []
requirements_created = 0
@@ -1875,10 +1869,7 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
# Aggregate findings by region using SQL for optimal performance
check_status_by_region, findings_count_by_compliance = (
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_threatscore_id,
threatscore_requirements_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
)
@@ -1943,35 +1934,23 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
# Yield rows lazily (consumed batch-by-batch by COPY) so peak memory
# stays bounded; tally requirement_statuses in the same pass. The
# ORM fallback re-iterates from scratch, so the tally resets first.
# Region is the innermost loop so consecutive rows share the leading
# columns of the table's secondary indexes.
def _iter_compliance_requirement_rows():
requirement_statuses.clear()
for (
compliance_id,
framework,
version,
modeled_compliance_id,
requirements,
) in compliance_plan:
stats_by_region = [
(
region,
region_requirement_stats.get(region, {}).get(
compliance_id, {}
),
findings_count_by_compliance.get(region, {}).get(
modeled_compliance_id, {}
),
for region in regions:
region_stats = region_requirement_stats.get(region, {})
region_findings = findings_count_by_compliance.get(region, {})
for (
compliance_id,
framework,
version,
modeled_compliance_id,
requirements,
) in compliance_plan:
compliance_stats = region_stats.get(compliance_id, {})
compliance_findings = region_findings.get(
modeled_compliance_id, {}
)
for region in regions
]
for requirement_id, description, total_checks in requirements:
for (
region,
compliance_stats,
compliance_findings,
) in stats_by_region:
for requirement_id, description, total_checks in requirements:
stats = compliance_stats.get(requirement_id)
if stats:
passed_checks = stats["passed_checks"]
@@ -2002,7 +1981,7 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
requirement_statuses[key]["pass_count"] += 1
yield {
"id": uuid7(),
"id": uuid.uuid4(),
"tenant_id": tenant_id_str,
"inserted_at": utc_datetime_now,
"compliance_id": compliance_id,
+100 -46
View File
@@ -73,10 +73,7 @@ from tasks.jobs.lighthouse_providers import (
check_lighthouse_provider_connection,
refresh_lighthouse_provider_models,
)
from tasks.jobs.muting import (
mute_findings_in_latest_scans,
reconcile_scan_mute_rules,
)
from tasks.jobs.muting import mute_historical_findings
from tasks.jobs.orphan_recovery import reconcile_orphans
from tasks.jobs.report import (
STALE_TMP_OUTPUT_MAX_AGE_HOURS,
@@ -529,7 +526,6 @@ def perform_scan_task(
provider_id=provider_id,
checks_to_execute=checks_to_execute,
)
reconcile_scan_mute_rules(tenant_id, scan_id)
_perform_scan_complete_tasks(tenant_id, scan_id, provider_id)
return result
finally:
@@ -639,7 +635,6 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
scan_id=str(scan_instance.id),
provider_id=provider_id,
)
reconcile_scan_mute_rules(tenant_id, str(scan_instance.id))
_perform_scan_complete_tasks(tenant_id, str(scan_instance.id), provider_id)
return result
finally:
@@ -1193,48 +1188,85 @@ def aggregate_finding_group_summaries_task(tenant_id: str, scan_id: str):
return aggregate_finding_group_summaries(tenant_id=tenant_id, scan_id=scan_id)
def _dispatch_scan_summary_reaggregation(tenant_id: str, scan_ids: list[str]) -> None:
if not scan_ids:
return
logger.info(
"Reaggregating overview/finding summaries for %d latest scans",
len(scan_ids),
)
group(
chain(
perform_scan_summary_task.si(tenant_id=tenant_id, scan_id=scan_id),
group(
aggregate_daily_severity_task.si(tenant_id=tenant_id, scan_id=scan_id),
aggregate_finding_group_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_scan_resource_group_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_scan_category_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_attack_surface_task.si(tenant_id=tenant_id, scan_id=scan_id),
),
)
for scan_id in scan_ids
).apply_async()
@shared_task(base=RLSTask, name="findings-mute-latest-scans", queue="overview")
@shared_task(
base=RLSTask, name="reaggregate-all-finding-group-summaries", queue="overview"
)
@set_tenant(keep_tenant=True)
def mute_findings_in_latest_scans_task(
tenant_id: str, mute_rule_id: str, provider_ids: list[str]
):
"""Apply a mute rule to current scans and rebuild only changed summaries."""
result = mute_findings_in_latest_scans(
tenant_id=tenant_id,
mute_rule_id=mute_rule_id,
provider_ids=provider_ids,
def reaggregate_all_finding_group_summaries_task(tenant_id: str):
"""Reaggregate every pre-aggregated summary table for this tenant.
Mirrors the unbounded scope of `mute_historical_findings_task`: that task
rewrites every Finding row whose UID matches a mute rule, with no time
limit. To keep the pre-aggregated tables consistent with that update,
this task re-runs the same per-scan aggregation pipeline that scan
completion runs on the latest completed scan of every (provider, day)
pair, rebuilding the tables that power the read endpoints:
- `ScanSummary` and `DailySeveritySummary` -> `/overviews/findings`,
`/overviews/findings-severity`, `/overviews/services`.
- `FindingGroupDailySummary` -> `/finding-groups` and
`/finding-groups/latest`.
- `ScanGroupSummary` -> `/overviews/resource-groups` (resource
inventory).
- `ScanCategorySummary` -> `/overviews/categories`.
- `AttackSurfaceOverview` -> `/overviews/attack-surfaces`.
Per-scan pipelines are dispatched in parallel via a Celery group so
wallclock scales with the worker pool.
"""
completed_scans = list(
Scan.objects.filter(
tenant_id=tenant_id,
state=StateChoices.COMPLETED,
completed_at__isnull=False,
)
.order_by("-completed_at")
.values("id", "completed_at", "provider_id")
)
_dispatch_scan_summary_reaggregation(tenant_id, result["scan_ids"])
return result
# Keep the latest scan per (provider, day) pair so the daily summary row
# the aggregator writes is the most recent snapshot of that day for that
# provider. Iterating from most recent to oldest means the first scan we
# see for a given key wins.
latest_scans: dict[tuple, str] = {}
for scan in completed_scans:
key = (scan["provider_id"], scan["completed_at"].date())
if key not in latest_scans:
latest_scans[key] = str(scan["id"])
scan_ids = list(latest_scans.values())
if scan_ids:
logger.info(
"Reaggregating overview/finding summaries for %d scans (provider x day)",
len(scan_ids),
)
# DailySeveritySummary reads from ScanSummary, so ScanSummary must be
# recomputed first; the other aggregators read Finding directly and
# can run in parallel with the severity step.
group(
chain(
perform_scan_summary_task.si(tenant_id=tenant_id, scan_id=scan_id),
group(
aggregate_daily_severity_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_finding_group_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_scan_resource_group_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_scan_category_summaries_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
aggregate_attack_surface_task.si(
tenant_id=tenant_id, scan_id=scan_id
),
),
)
for scan_id in scan_ids
).apply_async()
return {"scans_reaggregated": len(scan_ids)}
@shared_task(base=RLSTask, name="lighthouse-connection-check")
@@ -1435,3 +1467,25 @@ def generate_compliance_reports_task(tenant_id: str, scan_id: str, provider_id:
generate_csa=True,
generate_cis=True,
)
@shared_task(name="findings-mute-historical")
def mute_historical_findings_task(tenant_id: str, mute_rule_id: str):
"""
Background task to mute all historical findings matching a mute rule.
This task processes findings in batches to avoid memory issues with large datasets.
It updates the Finding.muted, Finding.muted_at, and Finding.muted_reason fields
for all findings whose UID is in the mute rule's finding_uids list.
Args:
tenant_id (str): The tenant ID for RLS context.
mute_rule_id (str): The primary key of the MuteRule to apply.
Returns:
dict: A dictionary containing:
- 'findings_muted' (int): Total number of findings muted.
- 'rule_id' (str): The mute rule ID.
- 'status' (str): Final status ('completed').
"""
return mute_historical_findings(tenant_id, mute_rule_id)
+17 -1
View File
@@ -2,7 +2,7 @@ from unittest.mock import MagicMock, call, patch
import pytest
from api.attack_paths import database as graph_database
from api.models import Provider, Tenant, TenantComplianceSummary
from api.models import JiraIssue, Provider, Tenant, TenantComplianceSummary
from django.core.exceptions import ObjectDoesNotExist
from tasks.jobs.deletion import delete_provider, delete_tenant
@@ -33,6 +33,22 @@ class TestDeleteProvider:
str(instance.id),
)
def test_delete_provider_removes_jira_issues(self, jira_issues_fixture):
linked, other_provider_issue, reservation = jira_issues_fixture
provider = linked.provider
tenant_id = str(provider.tenant_id)
with (
patch("tasks.jobs.deletion.graph_database.get_database_name"),
patch("tasks.jobs.deletion.graph_database.drop_subgraph"),
):
delete_provider(tenant_id, provider.id)
remaining = set(JiraIssue.objects.values_list("id", flat=True))
assert linked.id not in remaining
assert reservation.id not in remaining
# Issues of other providers are untouched
assert other_provider_issue.id in remaining
def test_delete_provider_does_not_exist(self, tenants_fixture):
with (
patch(
+344 -55
View File
@@ -1,25 +1,30 @@
from datetime import UTC, datetime
import itertools
from datetime import UTC, datetime, timedelta
from unittest.mock import MagicMock, patch
import pytest
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.models import Integration
from api.db_utils import rls_transaction
from api.models import Integration, JiraIssue
from api.utils import prowler_integration_connection_test
from django.db import OperationalError
from django.test import override_settings
from django.utils import timezone
from prowler.lib.outputs.jira.exceptions.exceptions import (
JiraRefreshTokenError,
JiraRequiredCustomFieldsError,
)
from prowler.lib.outputs.jira.jira import Jira
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHubConnection
from prowler.providers.common.models import Connection
from tasks.jobs.integrations import (
JIRA_RESERVATION_TTL,
build_jira_finding_url,
build_jira_issue_labels,
get_s3_client_from_integration,
get_security_hub_client_from_integration,
get_tenant_name,
sanitize_jira_label,
sanitize_jira_labels,
send_findings_to_jira,
upload_s3_integration,
upload_security_hub_integration,
@@ -1655,6 +1660,27 @@ class TestSecurityHubIntegrationUploads:
@pytest.mark.django_db
class TestJiraIntegration:
"""Sending findings to Jira, with the dedup bookkeeping stubbed out.
These tests use fake tenant ids and fully mocked findings; the dedup helpers
are exercised with real rows in TestJiraIssueDedup.
"""
@pytest.fixture(autouse=True)
def no_dedup_bookkeeping(self):
reservation = MagicMock()
reservation.issue_key = ""
with patch.multiple(
"tasks.jobs.integrations",
_load_finding_refs=MagicMock(return_value={}),
_load_existing_jira_issues=MagicMock(return_value={}),
_refresh_jira_issue_statuses=MagicMock(return_value={}),
_reserve_jira_issue=MagicMock(return_value=reservation),
_link_jira_issue=MagicMock(),
_release_jira_issue=MagicMock(),
):
yield
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@@ -1764,7 +1790,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 2, "failed_count": 0}
assert result == {"created_count": 2, "skipped_count": 0, "failed_count": 0}
# Verify Jira integration was initialized
mock_initialize_integration.assert_called_once_with(integration)
@@ -1884,6 +1910,7 @@ class TestJiraIntegration:
# Assertions
assert result == {
"created_count": 2,
"skipped_count": 0,
"failed_count": 1,
"error": "Failed to create Jira issue.",
}
@@ -1950,6 +1977,7 @@ class TestJiraIntegration:
assert result == {
"created_count": 0,
"skipped_count": 0,
"failed_count": 1,
"error": error_message,
}
@@ -2018,6 +2046,7 @@ class TestJiraIntegration:
assert result == {
"created_count": 0,
"skipped_count": 0,
"failed_count": 1,
"error": error_message,
}
@@ -2082,6 +2111,7 @@ class TestJiraIntegration:
assert result == {
"created_count": 0,
"skipped_count": 0,
"failed_count": 1,
"error": "Failed to create Jira issue.",
}
@@ -2159,7 +2189,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 1, "failed_count": 0}
assert result == {"created_count": 1, "skipped_count": 0, "failed_count": 0}
# Verify send_finding was called with empty resource fields
call_kwargs = mock_jira_integration.send_finding.call_args.kwargs
@@ -2222,7 +2252,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 1, "failed_count": 0}
assert result == {"created_count": 1, "skipped_count": 0, "failed_count": 0}
# Verify send_finding was called with default/empty values
call_kwargs = mock_jira_integration.send_finding.call_args.kwargs
@@ -2240,6 +2270,29 @@ class TestJiraIntegration:
class TestJiraFindingReference:
"""Helpers that give Jira issues a stable reference back to the finding."""
def test_sanitize_jira_label(self):
assert sanitize_jira_label("") == ""
assert sanitize_jira_label(None) == ""
assert sanitize_jira_label(" ") == ""
assert sanitize_jira_label("simple") == "simple"
assert sanitize_jira_label("with space") == "with_space"
assert sanitize_jira_label(" many spaces \t tabs\nnewline ") == (
"many_spaces_tabs_newline"
)
assert sanitize_jira_label("ctrl\x00char\x07here") == "ctrlcharhere"
assert sanitize_jira_label("arn:aws:iam::123456789012:role/Admin") == (
"arn:aws:iam::123456789012:role/Admin"
)
assert sanitize_jira_label("x" * 300) == "x" * 255
# Deterministic and idempotent
once = sanitize_jira_label("a b\tc")
assert sanitize_jira_label(once) == once
def test_sanitize_jira_labels(self):
assert sanitize_jira_labels([]) == []
assert sanitize_jira_labels(None) == []
assert sanitize_jira_labels(["b", "a b", "b", "", "a_b"]) == ["b", "a_b"]
def test_build_jira_issue_labels(self):
assert build_jira_issue_labels(
finding_uid="prowler-aws-check-123-eu-west-1-hub/unknown",
@@ -2259,55 +2312,12 @@ class TestJiraFindingReference:
finding_uid="", provider="", severity="", check_id=""
) == ["prowler"]
def test_build_jira_issue_labels_sanitizes_metadata(self):
assert build_jira_issue_labels(
finding_uid=" uid\x00 with spaces ",
provider="aws cloud",
severity="high severity",
check_id="check id",
) == [
"prowler",
"prowler-aws_cloud",
"prowler-high_severity",
"prowler-check_id",
"prowler-finding-uid_with_spaces",
]
def test_build_jira_issue_labels_preserves_maximum_length_uid(self):
finding_uid = "u" * (Jira.LABEL_MAX_LENGTH - len(Jira.FINDING_LABEL_PREFIX) - 1)
finding_label = build_jira_issue_labels(
finding_uid=finding_uid,
provider="gcp",
severity="low",
check_id="check",
)[-1]
assert finding_label == f"{Jira.FINDING_LABEL_PREFIX}-{finding_uid}"
assert len(finding_label) == Jira.LABEL_MAX_LENGTH
def test_build_jira_issue_labels_distinguishes_long_uids(self):
common_prefix = "u" * 300
first_uid = f"{common_prefix}-first"
second_uid = f"{common_prefix}-second"
first_label = build_jira_issue_labels(
finding_uid=first_uid,
provider="gcp",
severity="low",
check_id="check",
)[-1]
second_label = build_jira_issue_labels(
finding_uid=second_uid,
provider="gcp",
severity="low",
check_id="check",
)[-1]
assert first_label == Jira.build_finding_label(first_uid)
assert second_label == Jira.build_finding_label(second_uid)
assert first_label != second_label
assert len(first_label) == Jira.LABEL_MAX_LENGTH
assert len(second_label) == Jira.LABEL_MAX_LENGTH
def test_build_jira_issue_labels_truncates_long_uid(self):
labels = build_jira_issue_labels(
finding_uid="u" * 300, provider="gcp", severity="low", check_id="c"
)
assert labels[-1] == ("prowler-finding-" + "u" * 300)[:255]
assert all(len(label) <= 255 for label in labels)
@override_settings(UI_BASE_URL="")
def test_build_jira_finding_url_without_base_url(self):
@@ -2333,3 +2343,282 @@ class TestJiraFindingReference:
def test_get_tenant_name_unknown_or_invalid(self):
assert get_tenant_name("00000000-0000-0000-0000-000000000000") == ""
assert get_tenant_name("not-a-uuid") == ""
@pytest.mark.django_db
class TestJiraIssueDedup:
"""send_findings_to_jira with real JiraIssue rows: skip, replace, reserve."""
@pytest.fixture
def jira_mock(self):
jira = MagicMock()
counter = itertools.count(1)
def _create_issue(**kwargs):
number = next(counter)
return {
"key": f"TEST-{number}",
"id": str(10000 + number),
"url": f"https://test.atlassian.net/browse/TEST-{number}",
}
jira.send_finding.side_effect = _create_issue
jira.get_issues_status.return_value = {}
return jira
@pytest.fixture
def send(self, jira_mock, jira_integration_fixture):
def _send(finding_ids):
with patch(
"tasks.jobs.integrations.initialize_prowler_integration",
return_value=jira_mock,
):
return send_findings_to_jira(
str(jira_integration_fixture.tenant_id),
str(jira_integration_fixture.id),
"TEST",
"Task",
[str(finding_id) for finding_id in finding_ids],
)
return _send
@staticmethod
def _rows(integration):
with rls_transaction(str(integration.tenant_id)):
return {
row.finding_uid: row
for row in JiraIssue.objects.filter(integration=integration)
}
def test_first_send_links_findings(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, finding2 = findings_fixture
jira_mock.get_issues_status.return_value = {
"TEST-1": {"id": "10001", "status": "To Do", "status_category": "new"},
"TEST-2": {"id": "10002", "status": "To Do", "status_category": "new"},
}
result = send([finding1.id, finding2.id])
assert result == {"created_count": 2, "skipped_count": 0, "failed_count": 0}
assert jira_mock.send_finding.call_count == 2
rows = self._rows(jira_integration_fixture)
assert set(rows) == {finding1.uid, finding2.uid}
row = rows[finding1.uid]
assert row.issue_key == "TEST-1"
assert row.issue_id == "10001"
assert row.issue_url == "https://test.atlassian.net/browse/TEST-1"
assert row.project_key == "TEST"
assert row.finding_id == finding1.id
assert row.provider_id == finding1.scan.provider_id
# Status of the new issues is fetched once, in bulk, after creation
jira_mock.get_issues_status.assert_called_once_with(["TEST-1", "TEST-2"])
assert row.issue_status == "To Do"
assert row.issue_status_category == "new"
assert row.status_synced_at is not None
def test_second_send_skips_open_issue(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, _ = findings_fixture
send([finding1.id])
jira_mock.send_finding.reset_mock()
jira_mock.get_issues_status.reset_mock()
jira_mock.get_issues_status.return_value = {
"TEST-1": {
"id": "10001",
"status": "In Progress",
"status_category": "indeterminate",
}
}
result = send([finding1.id])
assert result == {
"created_count": 0,
"skipped_count": 1,
"failed_count": 0,
"skipped": [
{
"finding_id": str(finding1.id),
"issue_key": "TEST-1",
"issue_url": "https://test.atlassian.net/browse/TEST-1",
"issue_status": "In Progress",
}
],
}
jira_mock.send_finding.assert_not_called()
jira_mock.get_issues_status.assert_called_once_with(["TEST-1"])
row = self._rows(jira_integration_fixture)[finding1.uid]
assert row.issue_key == "TEST-1"
# The refreshed status is cached on the row
assert row.issue_status == "In Progress"
assert row.issue_status_category == "indeterminate"
@pytest.mark.parametrize(
"status_response",
[
{"TEST-1": {"id": "10001", "status": "Done", "status_category": "done"}},
{}, # deleted in Jira
],
ids=["closed", "deleted"],
)
def test_closed_or_deleted_issue_is_replaced(
self,
send,
jira_mock,
jira_integration_fixture,
findings_fixture,
status_response,
):
finding1, _ = findings_fixture
send([finding1.id])
jira_mock.send_finding.reset_mock()
jira_mock.get_issues_status.return_value = status_response
result = send([finding1.id])
assert result == {"created_count": 1, "skipped_count": 0, "failed_count": 0}
jira_mock.send_finding.assert_called_once()
rows = self._rows(jira_integration_fixture)
assert len(rows) == 1
assert rows[finding1.uid].issue_key == "TEST-2"
assert (
rows[finding1.uid].issue_url == "https://test.atlassian.net/browse/TEST-2"
)
def test_status_lookup_failure_skips_linked_findings(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, _ = findings_fixture
send([finding1.id])
jira_mock.send_finding.reset_mock()
jira_mock.get_issues_status.side_effect = JiraRefreshTokenError(
message="token expired"
)
result = send([finding1.id])
assert result["created_count"] == 0
assert result["skipped_count"] == 1
jira_mock.send_finding.assert_not_called()
# Cached status untouched
row = self._rows(jira_integration_fixture)[finding1.uid]
assert row.issue_key == "TEST-1"
def test_failed_send_releases_reservation(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, _ = findings_fixture
jira_mock.send_finding.side_effect = JiraRequiredCustomFieldsError(
message="custom fields"
)
result = send([finding1.id])
assert result["created_count"] == 0
assert result["failed_count"] == 1
assert result["error"] == "custom fields"
assert self._rows(jira_integration_fixture) == {}
def test_failed_replacement_keeps_previous_link(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, _ = findings_fixture
send([finding1.id])
jira_mock.get_issues_status.return_value = {
"TEST-1": {"id": "10001", "status": "Done", "status_category": "done"}
}
jira_mock.send_finding.side_effect = Exception("boom")
result = send([finding1.id])
assert result["failed_count"] == 1
row = self._rows(jira_integration_fixture)[finding1.uid]
assert row.issue_key == "TEST-1"
assert row.issue_status_category == "done"
def test_fresh_reservation_is_skipped_and_stale_one_reclaimed(
self, send, jira_mock, jira_integration_fixture, findings_fixture
):
finding1, _ = findings_fixture
tenant_id = str(jira_integration_fixture.tenant_id)
with rls_transaction(tenant_id):
reservation = JiraIssue.objects.create(
tenant_id=tenant_id,
integration=jira_integration_fixture,
provider_id=finding1.scan.provider_id,
finding_uid=finding1.uid,
finding_id=finding1.id,
project_key="TEST",
)
# Another run is sending this finding right now
result = send([finding1.id])
assert result == {
"created_count": 0,
"skipped_count": 1,
"failed_count": 0,
"skipped": [
{
"finding_id": str(finding1.id),
"issue_key": "",
"issue_url": "",
"issue_status": "",
}
],
}
jira_mock.send_finding.assert_not_called()
# The run died: the reservation expired and is reclaimed
with rls_transaction(tenant_id):
JiraIssue.objects.filter(id=reservation.id).update(
updated_at=timezone.now() - JIRA_RESERVATION_TTL - timedelta(minutes=1)
)
result = send([finding1.id])
assert result == {"created_count": 1, "skipped_count": 0, "failed_count": 0}
rows = self._rows(jira_integration_fixture)
assert len(rows) == 1
assert rows[finding1.uid].id == reservation.id
assert rows[finding1.uid].issue_key == "TEST-1"
def test_other_integration_does_not_dedup(
self,
send,
jira_mock,
jira_integration_fixture,
findings_fixture,
tenants_fixture,
):
finding1, _ = findings_fixture
tenant_id = str(jira_integration_fixture.tenant_id)
with rls_transaction(tenant_id):
other = Integration.objects.create(
tenant_id=tenant_id,
enabled=True,
connected=True,
integration_type=Integration.IntegrationChoices.JIRA,
configuration={"projects": {"OTHER": "Other"}},
credentials={
"domain": "other",
"user_mail": "a@b.com",
"api_token": "t",
},
)
send([finding1.id])
jira_mock.send_finding.reset_mock()
with patch(
"tasks.jobs.integrations.initialize_prowler_integration",
return_value=jira_mock,
):
result = send_findings_to_jira(
tenant_id, str(other.id), "OTHER", "Task", [str(finding1.id)]
)
assert result["created_count"] == 1
jira_mock.send_finding.assert_called_once()
with rls_transaction(tenant_id):
assert JiraIssue.objects.filter(finding_uid=finding1.uid).count() == 2
+505 -179
View File
@@ -1,205 +1,531 @@
from datetime import UTC, datetime, timedelta
from datetime import UTC, datetime
from uuid import uuid4
import pytest
from api.models import Finding, MuteRule, Scan, StateChoices
from api.models import Finding, MuteRule
from django.core.exceptions import ObjectDoesNotExist
from prowler.lib.check.models import Severity
from prowler.lib.outputs.finding import Status
from tasks.jobs.muting import (
mute_findings_in_latest_scans,
reconcile_scan_mute_rules,
)
def _create_finding(scan: Scan, uid: str) -> Finding:
return Finding.objects.create(
tenant_id=scan.tenant_id,
uid=uid,
scan=scan,
status=Status.FAIL,
status_extended="Test finding",
impact=Severity.high,
severity=Severity.high,
raw_result={},
check_id="test_check",
check_metadata={"CheckId": "test_check"},
muted=False,
)
def _create_mute_rule(tenant_id, user, finding_uids, *, enabled=True) -> MuteRule:
return MuteRule.objects.create(
tenant_id=tenant_id,
name=f"Mute rule {uuid4()}",
reason="Approved exception",
enabled=enabled,
created_by=user,
finding_uids=finding_uids,
)
from tasks.jobs.muting import mute_historical_findings
@pytest.mark.django_db
class TestMuteFindingsInLatestScans:
def test_mutes_latest_scan_and_leaves_older_scan_unchanged(
self, scans_fixture, create_test_user
):
latest_scan = scans_fixture[0]
older_scan = Scan.objects.create(
tenant_id=latest_scan.tenant_id,
provider=latest_scan.provider,
name="Older scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
started_at=datetime.now(UTC) - timedelta(days=1),
completed_at=datetime.now(UTC) - timedelta(days=1),
)
uid = "latest-scan-only"
older_finding = _create_finding(older_scan, uid)
latest_finding = _create_finding(latest_scan, uid)
mute_rule = _create_mute_rule(latest_scan.tenant_id, create_test_user, [uid])
class TestMuteHistoricalFindings:
"""
Test suite for the mute_historical_findings function.
result = mute_findings_in_latest_scans(
str(latest_scan.tenant_id),
str(mute_rule.id),
[str(latest_scan.provider_id)],
)
This class tests the batch processing of findings to update their muted status
based on MuteRule criteria.
"""
older_finding.refresh_from_db()
latest_finding.refresh_from_db()
assert older_finding.muted is False
assert latest_finding.muted is True
assert latest_finding.muted_at == mute_rule.inserted_at
assert latest_finding.muted_reason == mute_rule.reason
assert result == {
"findings_muted": 1,
"rule_id": str(mute_rule.id),
"scan_ids": [str(latest_scan.id)],
}
@pytest.fixture(scope="function")
def test_user(self, create_test_user):
"""Create a test user for mute rule creation."""
return create_test_user
def test_mutes_one_latest_scan_per_provider(self, scans_fixture, create_test_user):
first_scan, second_scan, _ = scans_fixture
uid = "shared-selected-uid"
first_finding = _create_finding(first_scan, uid)
second_finding = _create_finding(second_scan, uid)
mute_rule = _create_mute_rule(first_scan.tenant_id, create_test_user, [uid])
result = mute_findings_in_latest_scans(
str(first_scan.tenant_id),
str(mute_rule.id),
[str(first_scan.provider_id), str(second_scan.provider_id)],
)
first_finding.refresh_from_db()
second_finding.refresh_from_db()
assert first_finding.muted is True
assert second_finding.muted is True
assert result["findings_muted"] == 2
assert set(result["scan_ids"]) == {str(first_scan.id), str(second_scan.id)}
def test_provider_without_completed_scan_does_nothing(
self, tenants_fixture, provider_factory, create_test_user
):
@pytest.fixture(scope="function")
def mute_rule_with_findings(self, tenants_fixture, findings_fixture, test_user):
"""
Create a mute rule that targets the first finding in the fixture.
"""
tenant = tenants_fixture[0]
provider = provider_factory()
mute_rule = _create_mute_rule(
tenant.id, create_test_user, ["future-scan-finding"]
finding = findings_fixture[0]
mute_rule = MuteRule.objects.create(
tenant_id=tenant.id,
name="Test Mute Rule",
reason="Testing mute functionality",
enabled=True,
created_by=test_user,
finding_uids=[finding.uid],
)
result = mute_findings_in_latest_scans(
str(tenant.id), str(mute_rule.id), [str(provider.id)]
)
return mute_rule
assert result == {
"findings_muted": 0,
"rule_id": str(mute_rule.id),
"scan_ids": [],
}
def test_retry_does_not_report_changed_scans_twice(
self, scans_fixture, create_test_user
):
@pytest.fixture(scope="function")
def mute_rule_multiple_findings(self, scans_fixture, test_user):
"""
Create multiple unmuted findings and a mute rule targeting all of them.
"""
scan = scans_fixture[0]
finding = _create_finding(scan, "idempotent-mute")
mute_rule = _create_mute_rule(scan.tenant_id, create_test_user, [finding.uid])
args = (
str(scan.tenant_id),
str(mute_rule.id),
[str(scan.provider_id)],
tenant_id = scan.tenant_id
# Create 5 unmuted findings
finding_uids = []
for i in range(5):
finding = Finding.objects.create(
tenant_id=tenant_id,
uid=f"test_finding_uid_mute_{i}",
scan=scan,
status=Status.FAIL,
status_extended=f"Test status {i}",
impact=Severity.high,
severity=Severity.high,
raw_result={
"status": Status.FAIL,
"impact": Severity.high,
"severity": Severity.high,
},
check_id=f"test_check_id_{i}",
check_metadata={
"CheckId": f"test_check_id_{i}",
"Description": f"Test description {i}",
},
muted=False,
)
finding_uids.append(finding.uid)
# Create mute rule targeting all findings
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Multiple Findings Mute Rule",
reason="Testing batch muting",
enabled=True,
created_by=test_user,
finding_uids=finding_uids,
)
first_result = mute_findings_in_latest_scans(*args)
second_result = mute_findings_in_latest_scans(*args)
return mute_rule, finding_uids
assert first_result["scan_ids"] == [str(scan.id)]
assert second_result["findings_muted"] == 0
assert second_result["scan_ids"] == []
@pytest.fixture(scope="function")
def mute_rule_already_muted(self, findings_fixture, test_user):
"""
Create a mute rule that targets an already-muted finding.
"""
tenant_id = findings_fixture[1].tenant_id
already_muted_finding = findings_fixture[1]
def test_does_not_cross_tenant_boundary(
self, tenants_fixture, provider_factory, create_test_user
):
tenant = tenants_fixture[0]
other_tenant = tenants_fixture[2]
other_provider = provider_factory(tenant=other_tenant)
other_scan = Scan.objects.create(
tenant_id=other_tenant.id,
provider=other_provider,
name="Other tenant scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
started_at=datetime.now(UTC),
completed_at=datetime.now(UTC),
)
other_finding = _create_finding(other_scan, "tenant-isolated-uid")
mute_rule = _create_mute_rule(tenant.id, create_test_user, [other_finding.uid])
result = mute_findings_in_latest_scans(
str(tenant.id), str(mute_rule.id), [str(other_provider.id)]
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Already Muted Rule",
reason="Testing already muted findings",
enabled=True,
created_by=test_user,
finding_uids=[already_muted_finding.uid],
)
other_finding.refresh_from_db()
assert other_finding.muted is False
assert result["scan_ids"] == []
return mute_rule
def test_nonexistent_rule_raises(self, tenants_fixture):
with pytest.raises(MuteRule.DoesNotExist):
mute_findings_in_latest_scans(str(tenants_fixture[0].id), str(uuid4()), [])
@pytest.mark.django_db
class TestReconcileScanMuteRules:
def test_applies_only_enabled_rules_to_requested_scan(
self, scans_fixture, create_test_user
):
@pytest.fixture(scope="function")
def mute_rule_mixed_findings(self, scans_fixture, test_user):
"""
Create a mute rule with a mix of muted and unmuted findings.
"""
scan = scans_fixture[0]
active_finding = _create_finding(scan, "active-rule-uid")
disabled_finding = _create_finding(scan, "disabled-rule-uid")
active_rule = _create_mute_rule(
scan.tenant_id, create_test_user, [active_finding.uid]
)
_create_mute_rule(
scan.tenant_id,
create_test_user,
[disabled_finding.uid],
enabled=False,
)
older_scan = Scan.objects.create(
tenant_id=scan.tenant_id,
provider=scan.provider,
name="Older matching scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
started_at=datetime.now(UTC) - timedelta(days=1),
completed_at=datetime.now(UTC) - timedelta(days=1),
)
older_finding = _create_finding(older_scan, active_finding.uid)
tenant_id = scan.tenant_id
result = reconcile_scan_mute_rules(str(scan.tenant_id), str(scan.id))
# Create 3 unmuted findings
unmuted_uids = []
for i in range(3):
finding = Finding.objects.create(
tenant_id=tenant_id,
uid=f"unmuted_finding_{i}",
scan=scan,
status=Status.FAIL,
status_extended=f"Unmuted status {i}",
impact=Severity.medium,
severity=Severity.medium,
raw_result={
"status": Status.FAIL,
"impact": Severity.medium,
"severity": Severity.medium,
},
check_id=f"unmuted_check_{i}",
check_metadata={
"CheckId": f"unmuted_check_{i}",
"Description": f"Unmuted description {i}",
},
muted=False,
)
unmuted_uids.append(finding.uid)
active_finding.refresh_from_db()
disabled_finding.refresh_from_db()
older_finding.refresh_from_db()
assert active_finding.muted is True
assert active_finding.muted_at == active_rule.inserted_at
assert disabled_finding.muted is False
assert older_finding.muted is False
assert result == {"findings_muted": 1, "scan_id": str(scan.id)}
# Create 2 already muted findings
muted_uids = []
for i in range(2):
finding = Finding.objects.create(
tenant_id=tenant_id,
uid=f"muted_finding_{i}",
scan=scan,
status=Status.FAIL,
status_extended=f"Muted status {i}",
impact=Severity.low,
severity=Severity.low,
raw_result={
"status": Status.FAIL,
"impact": Severity.low,
"severity": Severity.low,
},
check_id=f"muted_check_{i}",
check_metadata={
"CheckId": f"muted_check_{i}",
"Description": f"Muted description {i}",
},
muted=True,
muted_at=datetime.now(UTC),
muted_reason="Already muted",
)
muted_uids.append(finding.uid)
# Create mute rule targeting all findings
all_uids = unmuted_uids + muted_uids
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Mixed Findings Rule",
reason="Testing mixed muted/unmuted findings",
enabled=True,
created_by=test_user,
finding_uids=all_uids,
)
return mute_rule, unmuted_uids, muted_uids
@pytest.fixture(scope="function")
def mute_rule_batch_test(self, scans_fixture, test_user):
"""
Create enough findings to test batch processing (>1000 for default batch size).
"""
scan = scans_fixture[0]
tenant_id = scan.tenant_id
# Create 1500 findings to exceed default batch size of 1000
finding_uids = []
for i in range(1500):
finding = Finding.objects.create(
tenant_id=tenant_id,
uid=f"batch_test_finding_{i}",
scan=scan,
status=Status.FAIL,
status_extended=f"Batch test status {i}",
impact=Severity.critical,
severity=Severity.critical,
raw_result={
"status": Status.FAIL,
"impact": Severity.critical,
"severity": Severity.critical,
},
check_id=f"batch_test_check_{i}",
check_metadata={
"CheckId": f"batch_test_check_{i}",
"Description": f"Batch test description {i}",
},
muted=False,
)
finding_uids.append(finding.uid)
# Create mute rule targeting all findings
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Batch Processing Rule",
reason="Testing batch processing functionality",
enabled=True,
created_by=test_user,
finding_uids=finding_uids,
)
return mute_rule, finding_uids
def test_mute_historical_findings_single_finding(
self, mute_rule_with_findings, findings_fixture
):
"""
Test muting a single historical finding.
"""
mute_rule = mute_rule_with_findings
tenant_id = str(mute_rule.tenant_id)
finding = findings_fixture[0]
# Ensure the finding is not muted before execution
finding.refresh_from_db()
assert finding.muted is False
assert finding.muted_at is None
assert finding.muted_reason is None
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify return value
assert result["findings_muted"] == 1
assert result["rule_id"] == str(mute_rule.id)
# Verify the finding was muted
finding.refresh_from_db()
assert finding.muted is True
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_reason == mute_rule.reason
def test_mute_historical_findings_multiple_findings(
self, mute_rule_multiple_findings
):
"""
Test muting multiple historical findings.
"""
mute_rule, finding_uids = mute_rule_multiple_findings
tenant_id = str(mute_rule.tenant_id)
# Verify all findings are unmuted
findings = Finding.objects.filter(tenant_id=tenant_id, uid__in=finding_uids)
assert findings.count() == 5
for finding in findings:
assert finding.muted is False
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify return value
assert result["findings_muted"] == 5
assert result["rule_id"] == str(mute_rule.id)
# Verify all findings were muted
findings = Finding.objects.filter(tenant_id=tenant_id, uid__in=finding_uids)
for finding in findings:
assert finding.muted is True
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_reason == mute_rule.reason
def test_mute_historical_findings_already_muted(
self, mute_rule_already_muted, findings_fixture
):
"""
Test that already-muted findings are not counted or updated.
"""
mute_rule = mute_rule_already_muted
tenant_id = str(mute_rule.tenant_id)
finding = findings_fixture[1]
# Verify the finding is already muted
finding.refresh_from_db()
assert finding.muted is True
original_muted_at = finding.muted_at
original_muted_reason = finding.muted_reason
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify no findings were muted
assert result["findings_muted"] == 0
assert result["rule_id"] == str(mute_rule.id)
# Verify the finding's mute status did not change
finding.refresh_from_db()
assert finding.muted is True
assert finding.muted_at == original_muted_at
assert finding.muted_reason == original_muted_reason
def test_mute_historical_findings_mixed_status(self, mute_rule_mixed_findings):
"""
Test muting when some findings are already muted and others are not.
"""
mute_rule, unmuted_uids, muted_uids = mute_rule_mixed_findings
tenant_id = str(mute_rule.tenant_id)
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify only unmuted findings were counted
assert result["findings_muted"] == 3
assert result["rule_id"] == str(mute_rule.id)
# Verify unmuted findings are now muted
unmuted_findings = Finding.objects.filter(
tenant_id=tenant_id, uid__in=unmuted_uids
)
for finding in unmuted_findings:
assert finding.muted is True
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_reason == mute_rule.reason
# Verify already-muted findings remained unchanged
already_muted_findings = Finding.objects.filter(
tenant_id=tenant_id, uid__in=muted_uids
)
for finding in already_muted_findings:
assert finding.muted is True
assert finding.muted_reason == "Already muted"
def test_mute_historical_findings_nonexistent_rule(self, tenants_fixture):
"""
Test that a nonexistent mute rule raises ObjectDoesNotExist.
"""
tenant_id = str(tenants_fixture[0].id)
nonexistent_rule_id = str(uuid4())
with pytest.raises(ObjectDoesNotExist):
mute_historical_findings(tenant_id, nonexistent_rule_id)
def test_mute_historical_findings_no_matching_findings(
self, tenants_fixture, test_user
):
"""
Test muting when no findings match the rule's UIDs.
"""
tenant_id = str(tenants_fixture[0].id)
# Create a mute rule with non-existent finding UIDs
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test No Match Rule",
reason="Testing no matching findings",
enabled=True,
created_by=test_user,
finding_uids=[
"nonexistent_uid_1",
"nonexistent_uid_2",
"nonexistent_uid_3",
],
)
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify no findings were muted
assert result["findings_muted"] == 0
assert result["rule_id"] == str(mute_rule.id)
def test_mute_historical_findings_batch_processing(self, mute_rule_batch_test):
"""
Test that large numbers of findings are processed in batches correctly.
"""
mute_rule, finding_uids = mute_rule_batch_test
tenant_id = str(mute_rule.tenant_id)
# Verify all findings exist and are unmuted
findings = Finding.objects.filter(tenant_id=tenant_id, uid__in=finding_uids)
assert findings.count() == 1500
for finding in findings:
assert finding.muted is False
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify return value
assert result["findings_muted"] == 1500
assert result["rule_id"] == str(mute_rule.id)
# Verify all findings were muted
findings = Finding.objects.filter(tenant_id=tenant_id, uid__in=finding_uids)
for finding in findings:
assert finding.muted is True
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_reason == mute_rule.reason
def test_mute_historical_findings_preserves_muted_at_timestamp(
self, mute_rule_with_findings, findings_fixture
):
"""
Test that muted_at is set to the rule's inserted_at, not the current time.
"""
mute_rule = mute_rule_with_findings
tenant_id = str(mute_rule.tenant_id)
finding = findings_fixture[0]
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify the finding was muted
assert result["findings_muted"] == 1
# Verify muted_at matches the rule's inserted_at timestamp
finding.refresh_from_db()
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_at is not None
def test_mute_historical_findings_partial_match(self, scans_fixture, test_user):
"""
Test muting when only some of the rule's UIDs exist as findings.
"""
scan = scans_fixture[0]
tenant_id = str(scan.tenant_id)
# Create 3 findings
existing_uids = []
for i in range(3):
finding = Finding.objects.create(
tenant_id=tenant_id,
uid=f"partial_match_finding_{i}",
scan=scan,
status=Status.FAIL,
status_extended=f"Partial match status {i}",
impact=Severity.high,
severity=Severity.high,
raw_result={
"status": Status.FAIL,
"impact": Severity.high,
"severity": Severity.high,
},
check_id=f"partial_match_check_{i}",
check_metadata={
"CheckId": f"partial_match_check_{i}",
"Description": f"Partial match description {i}",
},
muted=False,
)
existing_uids.append(finding.uid)
# Create a mute rule with both existing and non-existing UIDs
all_uids = existing_uids + [
"nonexistent_uid_1",
"nonexistent_uid_2",
]
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Partial Match Rule",
reason="Testing partial matching",
enabled=True,
created_by=test_user,
finding_uids=all_uids,
)
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify only existing findings were muted
assert result["findings_muted"] == 3
assert result["rule_id"] == str(mute_rule.id)
# Verify the existing findings were muted
findings = Finding.objects.filter(tenant_id=tenant_id, uid__in=existing_uids)
assert findings.count() == 3
for finding in findings:
assert finding.muted is True
assert finding.muted_at == mute_rule.inserted_at
assert finding.muted_reason == mute_rule.reason
def test_mute_historical_findings_empty_uids(self, tenants_fixture, test_user):
"""
Test muting when the rule has an empty finding_uids array.
"""
tenant_id = str(tenants_fixture[0].id)
# Create a mute rule with empty finding_uids
mute_rule = MuteRule.objects.create(
tenant_id=tenant_id,
name="Test Empty UIDs Rule",
reason="Testing empty UIDs",
enabled=True,
created_by=test_user,
finding_uids=[],
)
# Execute the muting function
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify no findings were muted
assert result["findings_muted"] == 0
assert result["rule_id"] == str(mute_rule.id)
def test_mute_historical_findings_return_format(self, mute_rule_with_findings):
"""
Test that the return value has the correct format and fields.
"""
mute_rule = mute_rule_with_findings
tenant_id = str(mute_rule.tenant_id)
result = mute_historical_findings(tenant_id, str(mute_rule.id))
# Verify return value structure
assert isinstance(result, dict)
assert "findings_muted" in result
assert "rule_id" in result
assert isinstance(result["findings_muted"], int)
assert isinstance(result["rule_id"], str)
assert result["rule_id"] == str(mute_rule.id)
+66 -229
View File
@@ -1,5 +1,6 @@
import csv
import json
import re
import uuid
from collections.abc import MutableMapping
from contextlib import contextmanager
@@ -9,7 +10,6 @@ from unittest.mock import MagicMock, patch
import pytest
from api.db_router import MainRouter
from api.db_utils import rls_transaction
from api.exceptions import ProviderConnectionError, ProviderDeletedException
from api.models import (
Finding,
@@ -2795,167 +2795,6 @@ class TestCreateComplianceRequirements:
assert count_after_first > 0
assert count_after_second == count_after_first
with rls_transaction(tenant_id):
row_versions = {
row_id.version
for row_id in ComplianceRequirementOverview.objects.filter(
scan_id=scan_id
).values_list("id", flat=True)
}
assert row_versions == {7}
def test_create_compliance_requirements_threatscore_counts_from_template(
self,
tenants_fixture,
scans_fixture,
aws_provider,
findings_fixture,
):
"""ThreatScore finding counts are derived from the template mapping,
not from each finding's stored ``compliance`` payload."""
from api.models import ComplianceRequirementOverview
with patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template:
tenant_id = str(tenants_fixture[0].id)
scan_id = str(scans_fixture[0].id)
mock_compliance_template.__getitem__.return_value = {
"prowler_threatscore_aws": {
"framework": "ProwlerThreatScore",
"version": "1.0",
"requirements": {
"1.1.1": {
"description": "ThreatScore requirement",
"checks": {"test_check_id": None},
},
"1.1.2": {
"description": "Unrelated requirement",
"checks": {"other_check_id": None},
},
},
},
"other_framework": {
"framework": "Other",
"version": "2.0",
"requirements": {
"a": {
"description": "Same check, other framework",
"checks": {"test_check_id": None},
},
},
},
}
create_compliance_requirements(tenant_id, scan_id)
with rls_transaction(tenant_id):
counted = sum(
len(finding.resource_regions or [])
for finding in Finding.all_objects.filter(
scan_id=scan_id,
muted=False,
status__in=["PASS", "FAIL"],
check_id="test_check_id",
)
)
rows = list(
ComplianceRequirementOverview.objects.filter(
scan_id=scan_id
).values_list("compliance_id", "requirement_id", "total_findings")
)
assert counted > 0
assert (
sum(
total
for compliance_id, requirement_id, total in rows
if (compliance_id, requirement_id)
== ("prowler_threatscore_aws", "1.1.1")
)
== counted
)
assert all(
total == 0
for compliance_id, requirement_id, total in rows
if (compliance_id, requirement_id) == ("prowler_threatscore_aws", "1.1.2")
)
assert all(
total == 0
for compliance_id, _, total in rows
if compliance_id == "other_framework"
)
def test_create_compliance_requirements_rows_across_regions_and_frameworks(
self,
tenants_fixture,
scans_fixture,
aws_provider,
):
from api.models import ComplianceRequirementOverview
tenant_id = str(tenants_fixture[0].id)
scan_id = str(scans_fixture[0].id)
check_status_by_region = {
"us-east-1": {"check_a": "FAIL", "check_b": "PASS"},
"eu-west-1": {"check_a": "PASS"},
}
template = {
"fw_one": {
"framework": "One",
"version": "1",
"requirements": {
"r1": {"description": "a", "checks": {"check_a": None}},
"r2": {
"description": "a+b",
"checks": {"check_a": None, "check_b": None},
},
},
},
"fw_two": {
"framework": "Two",
"version": "2",
"requirements": {
"m1": {"description": "manual", "checks": {}},
},
},
}
with (
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan._aggregate_findings_by_region",
return_value=(check_status_by_region, {}),
),
):
mock_compliance_template.__getitem__.return_value = template
result = create_compliance_requirements(tenant_id, scan_id)
assert result["requirements_created"] == 6
with rls_transaction(tenant_id):
rows = set(
ComplianceRequirementOverview.objects.filter(
scan_id=scan_id
).values_list(
"compliance_id",
"requirement_id",
"region",
"requirement_status",
"passed_checks",
"failed_checks",
"total_checks",
)
)
assert rows == {
("fw_one", "r1", "us-east-1", "FAIL", 0, 1, 1),
("fw_one", "r1", "eu-west-1", "PASS", 1, 0, 1),
("fw_one", "r2", "us-east-1", "FAIL", 1, 1, 2),
("fw_one", "r2", "eu-west-1", "PASS", 1, 0, 2),
("fw_two", "m1", "us-east-1", "MANUAL", 0, 0, 0),
("fw_two", "m1", "eu-west-1", "MANUAL", 0, 0, 0),
}
def test_create_compliance_requirements_kubernetes_provider(
self,
@@ -4884,11 +4723,17 @@ class TestAggregateFindingsByRegion:
"""Test function returns correct data structure."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
# (check_id, status, resource_regions) tuples
finding_rows = [("check1", "FAIL", ["us-east-1"])]
threatscore_by_check = {"check1": ["req1", "req2"]}
# (check_id, status, resource_regions, compliance) tuples
finding_rows = [
(
"check1",
"FAIL",
["us-east-1"],
{modeled_threatscore_compliance_id: ["req1", "req2"]},
)
]
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -4902,16 +4747,13 @@ class TestAggregateFindingsByRegion:
check_status_by_region, findings_count_by_compliance = (
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
@@ -4932,14 +4774,13 @@ class TestAggregateFindingsByRegion:
"""Test that FAIL status takes priority over other statuses."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
# Same check/region: PASS first, then FAIL — FAIL must win
finding_rows = [
("check1", "PASS", ["us-east-1"]),
("check1", "FAIL", ["us-east-1"]),
("check1", "PASS", ["us-east-1"], {}),
("check1", "FAIL", ["us-east-1"], {}),
]
threatscore_by_check = {}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -4952,15 +4793,12 @@ class TestAggregateFindingsByRegion:
mock_findings_filter.return_value = mock_queryset
check_status_by_region, _ = _aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
@@ -4975,9 +4813,8 @@ class TestAggregateFindingsByRegion:
"""Test that muted findings are filtered out (muted=False in query)."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
threatscore_by_check = {}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
mock_queryset.iterator.return_value = []
@@ -4989,15 +4826,12 @@ class TestAggregateFindingsByRegion:
mock_findings_filter.return_value = mock_queryset
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
@@ -5017,14 +4851,23 @@ class TestAggregateFindingsByRegion:
"""Test that ThreatScore compliance counts are processed correctly."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
# PASS and FAIL findings mapped to the same ThreatScore requirement
finding_rows = [
("check1", "PASS", ["us-east-1"]),
("check2", "FAIL", ["us-east-1"]),
(
"check1",
"PASS",
["us-east-1"],
{modeled_threatscore_compliance_id: ["req1"]},
),
(
"check2",
"FAIL",
["us-east-1"],
{modeled_threatscore_compliance_id: ["req1"]},
),
]
threatscore_by_check = {"check1": ["req1"], "check2": ["req1"]}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -5037,19 +4880,19 @@ class TestAggregateFindingsByRegion:
mock_findings_filter.return_value = mock_queryset
_, findings_count_by_compliance = _aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
# Verify compliance counts
normalized_id = re.sub(
r"[^a-z0-9]", "", modeled_threatscore_compliance_id.lower()
)
assert "us-east-1" in findings_count_by_compliance
assert normalized_id in findings_count_by_compliance["us-east-1"]
assert "req1" in findings_count_by_compliance["us-east-1"][normalized_id]
@@ -5066,14 +4909,13 @@ class TestAggregateFindingsByRegion:
"""Test aggregation across multiple regions."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
# One finding per region
finding_rows = [
("check1", "FAIL", ["us-east-1"]),
("check1", "PASS", ["us-west-2"]),
("check1", "FAIL", ["us-east-1"], {}),
("check1", "PASS", ["us-west-2"], {}),
]
threatscore_by_check = {}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -5086,15 +4928,12 @@ class TestAggregateFindingsByRegion:
mock_findings_filter.return_value = mock_queryset
check_status_by_region, _ = _aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
@@ -5112,10 +4951,16 @@ class TestAggregateFindingsByRegion:
"""A finding with multiple resource_regions is tallied in every region."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
finding_rows = [("check1", "FAIL", ["us-east-1", "eu-west-1"])]
threatscore_by_check = {"check1": ["req1"]}
finding_rows = [
(
"check1",
"FAIL",
["us-east-1", "eu-west-1"],
{modeled_threatscore_compliance_id: ["req1"]},
)
]
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -5129,19 +4974,19 @@ class TestAggregateFindingsByRegion:
check_status_by_region, findings_count_by_compliance = (
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
normalized_id = re.sub(
r"[^a-z0-9]", "", modeled_threatscore_compliance_id.lower()
)
for region in ("us-east-1", "eu-west-1"):
assert check_status_by_region[region]["check1"] == "FAIL"
req_stats = findings_count_by_compliance[region][normalized_id]["req1"]
@@ -5155,13 +5000,12 @@ class TestAggregateFindingsByRegion:
"""A finding with no denormalized regions contributes nothing."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
finding_rows = [
("check1", "FAIL", []),
("check2", "PASS", None),
("check1", "FAIL", [], {modeled_threatscore_compliance_id: ["req1"]}),
("check2", "PASS", None, {}),
]
threatscore_by_check = {"check1": ["req1"]}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
@@ -5175,16 +5019,13 @@ class TestAggregateFindingsByRegion:
check_status_by_region, findings_count_by_compliance = (
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
@@ -5199,9 +5040,8 @@ class TestAggregateFindingsByRegion:
"""Test with no findings - should return empty dicts."""
tenant_id = str(uuid.uuid4())
scan_id = str(uuid.uuid4())
normalized_id = "prowlerthreatscore10"
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
threatscore_by_check = {}
mock_queryset = MagicMock()
mock_queryset.values_list.return_value = mock_queryset
mock_queryset.iterator.return_value = []
@@ -5214,16 +5054,13 @@ class TestAggregateFindingsByRegion:
check_status_by_region, findings_count_by_compliance = (
_aggregate_findings_by_region(
tenant_id,
scan_id,
normalized_id,
threatscore_by_check,
tenant_id, scan_id, modeled_threatscore_compliance_id
)
)
# Streaming query contract: column-scoped values_list + iterator
mock_queryset.values_list.assert_called_once_with(
"check_id", "status", "resource_regions"
"check_id", "status", "resource_regions", "compliance"
)
mock_queryset.iterator.assert_called_once()
+124 -69
View File
@@ -1,6 +1,6 @@
import uuid
from contextlib import contextmanager
from datetime import UTC, datetime
from datetime import UTC, datetime, timedelta
from unittest.mock import MagicMock, patch
import httpx
@@ -33,10 +33,10 @@ from tasks.tasks import (
check_integrations_task,
check_lighthouse_provider_connection_task,
generate_outputs_task,
mute_findings_in_latest_scans_task,
perform_attack_paths_scan_task,
perform_scan_task,
perform_scheduled_scan_task,
reaggregate_all_finding_group_summaries_task,
refresh_lighthouse_provider_models_task,
s3_integration_task,
security_hub_integration_task,
@@ -2959,7 +2959,6 @@ class TestPerformScheduledScanTask:
with (
patch("tasks.tasks.perform_prowler_scan", side_effect=_complete_scan),
patch("tasks.tasks._perform_scan_complete_tasks"),
patch("tasks.tasks.reconcile_scan_mute_rules") as mock_reconcile,
self._override_task_request(perform_scheduled_scan_task, id=task_id),
):
perform_scheduled_scan_task.run(
@@ -2983,13 +2982,6 @@ class TestPerformScheduledScanTask:
).count()
== 1
)
completed_scan = Scan.objects.get(
tenant_id=tenant.id,
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.COMPLETED,
)
mock_reconcile.assert_called_once_with(str(tenant.id), str(completed_scan.id))
assert (
Scan.objects.filter(
tenant_id=tenant.id,
@@ -3184,10 +3176,7 @@ class TestPerformScanTask:
task=queued_task,
)
events = []
def _complete_scan(tenant_id, scan_id, provider_id, checks_to_execute=None):
events.append("scan")
scan_instance = Scan.objects.get(id=scan_id)
scan_instance.state = StateChoices.COMPLETED
scan_instance.save()
@@ -3195,14 +3184,7 @@ class TestPerformScanTask:
with (
patch("tasks.tasks.perform_prowler_scan", side_effect=_complete_scan),
patch(
"tasks.tasks.reconcile_scan_mute_rules",
side_effect=lambda *_args: events.append("reconcile"),
),
patch(
"tasks.tasks._perform_scan_complete_tasks",
side_effect=lambda *_args: events.append("summaries"),
),
patch("tasks.tasks._perform_scan_complete_tasks"),
patch("tasks.tasks.perform_scan_task.apply_async") as mock_apply_async,
):
with django_capture_on_commit_callbacks(execute=True):
@@ -3214,7 +3196,6 @@ class TestPerformScanTask:
queued_task_result.refresh_from_db()
assert result == {"status": "ok"}
assert events == ["scan", "reconcile", "summaries"]
assert queued_task_result.status == states.PENDING
mock_apply_async.assert_called_once_with(
kwargs={
@@ -3260,7 +3241,10 @@ class TestPerformScanTask:
@pytest.mark.django_db
class TestMuteFindingsInLatestScansTask:
class TestReaggregateAllFindingGroupSummaries:
def setup_method(self):
self.tenant_id = str(uuid.uuid4())
@patch("tasks.tasks.chain")
@patch("tasks.tasks.group")
@patch("tasks.tasks.aggregate_attack_surface_task")
@@ -3269,10 +3253,10 @@ class TestMuteFindingsInLatestScansTask:
@patch("tasks.tasks.aggregate_finding_group_summaries_task")
@patch("tasks.tasks.aggregate_daily_severity_task")
@patch("tasks.tasks.perform_scan_summary_task")
@patch("tasks.tasks.mute_findings_in_latest_scans")
def test_reaggregates_only_changed_scans(
@patch("tasks.tasks.Scan.objects.filter")
def test_dispatches_subtasks_for_each_provider_per_day(
self,
mock_mute_findings,
mock_scan_filter,
mock_scan_summary_task,
mock_daily_severity_task,
mock_finding_group_task,
@@ -3281,36 +3265,49 @@ class TestMuteFindingsInLatestScansTask:
mock_attack_surface_task,
mock_group,
mock_chain,
tenants_fixture,
):
tenant_id = str(tenants_fixture[0].id)
mute_rule_id = str(uuid.uuid4())
provider_ids = [str(uuid.uuid4()), str(uuid.uuid4())]
scan_ids = [str(uuid.uuid4()), str(uuid.uuid4())]
result = {
"findings_muted": 2,
"rule_id": mute_rule_id,
"scan_ids": scan_ids,
}
mock_mute_findings.return_value = result
provider_id_1 = uuid.uuid4()
provider_id_2 = uuid.uuid4()
scan_id_today_p1 = uuid.uuid4()
scan_id_yesterday_p1 = uuid.uuid4()
scan_id_today_p2 = uuid.uuid4()
today = datetime.now(tz=UTC)
yesterday = today - timedelta(days=1)
mock_outer_group_result = MagicMock()
# The first `group()` call wraps the inner parallel step; subsequent
# calls wrap the outer per-scan generator.
mock_group.side_effect = lambda *args, **kwargs: (
list(args[0]) if args and hasattr(args[0], "__iter__") else None,
mock_outer_group_result,
)[1]
task_result = mute_findings_in_latest_scans_task(
tenant_id=tenant_id,
mute_rule_id=mute_rule_id,
provider_ids=provider_ids,
)
mock_scan_filter.return_value.order_by.return_value.values.return_value = [
{
"id": scan_id_today_p1,
"completed_at": today,
"provider_id": provider_id_1,
},
{
"id": scan_id_today_p2,
"completed_at": today,
"provider_id": provider_id_2,
},
{
"id": scan_id_yesterday_p1,
"completed_at": yesterday,
"provider_id": provider_id_1,
},
]
assert task_result == result
mock_mute_findings.assert_called_once_with(
tenant_id=tenant_id,
mute_rule_id=mute_rule_id,
provider_ids=provider_ids,
)
result = reaggregate_all_finding_group_summaries_task(tenant_id=self.tenant_id)
assert result == {"scans_reaggregated": 3}
expected_scan_ids = {
str(scan_id_today_p1),
str(scan_id_today_p2),
str(scan_id_yesterday_p1),
}
for task_mock in (
mock_scan_summary_task,
mock_daily_severity_task,
@@ -3319,35 +3316,93 @@ class TestMuteFindingsInLatestScansTask:
mock_category_task,
mock_attack_surface_task,
):
assert task_mock.si.call_count == 2
assert {
assert task_mock.si.call_count == 3
dispatched = {
call.kwargs["scan_id"] for call in task_mock.si.call_args_list
} == set(scan_ids)
assert mock_chain.call_count == 2
}
assert dispatched == expected_scan_ids
for call in task_mock.si.call_args_list:
assert call.kwargs["tenant_id"] == self.tenant_id
assert mock_chain.call_count == 3
mock_outer_group_result.apply_async.assert_called_once()
@patch("tasks.tasks.chain")
@patch("tasks.tasks.group")
@patch("tasks.tasks.mute_findings_in_latest_scans")
def test_skips_reaggregation_when_no_scan_changed(
self, mock_mute_findings, mock_group, mock_chain, tenants_fixture
@patch("tasks.tasks.aggregate_attack_surface_task")
@patch("tasks.tasks.aggregate_scan_category_summaries_task")
@patch("tasks.tasks.aggregate_scan_resource_group_summaries_task")
@patch("tasks.tasks.aggregate_finding_group_summaries_task")
@patch("tasks.tasks.aggregate_daily_severity_task")
@patch("tasks.tasks.perform_scan_summary_task")
@patch("tasks.tasks.Scan.objects.filter")
def test_dedupes_scans_to_latest_per_provider_per_day(
self,
mock_scan_filter,
mock_scan_summary_task,
mock_daily_severity_task,
mock_finding_group_task,
mock_resource_group_task,
mock_category_task,
mock_attack_surface_task,
mock_group,
mock_chain,
):
tenant_id = str(tenants_fixture[0].id)
mute_rule_id = str(uuid.uuid4())
result = {
"findings_muted": 0,
"rule_id": mute_rule_id,
"scan_ids": [],
}
mock_mute_findings.return_value = result
"""When several scans run on the same day for the same provider, only
the latest one is dispatched (matching the daily summary unique key)."""
provider_id = uuid.uuid4()
latest_scan_today = uuid.uuid4()
earlier_scan_today = uuid.uuid4()
today_late = datetime.now(tz=UTC)
today_early = today_late - timedelta(hours=4)
task_result = mute_findings_in_latest_scans_task(
tenant_id=tenant_id,
mute_rule_id=mute_rule_id,
provider_ids=[],
)
mock_outer_group_result = MagicMock()
mock_group.side_effect = lambda *args, **kwargs: (
list(args[0]) if args and hasattr(args[0], "__iter__") else None,
mock_outer_group_result,
)[1]
assert task_result == result
# Returned ordered by `-completed_at`, so the most recent comes first.
mock_scan_filter.return_value.order_by.return_value.values.return_value = [
{
"id": latest_scan_today,
"completed_at": today_late,
"provider_id": provider_id,
},
{
"id": earlier_scan_today,
"completed_at": today_early,
"provider_id": provider_id,
},
]
result = reaggregate_all_finding_group_summaries_task(tenant_id=self.tenant_id)
assert result == {"scans_reaggregated": 1}
for task_mock in (
mock_scan_summary_task,
mock_daily_severity_task,
mock_finding_group_task,
mock_resource_group_task,
mock_category_task,
mock_attack_surface_task,
):
task_mock.si.assert_called_once_with(
tenant_id=self.tenant_id, scan_id=str(latest_scan_today)
)
mock_chain.assert_called_once()
mock_outer_group_result.apply_async.assert_called_once()
@patch("tasks.tasks.chain")
@patch("tasks.tasks.group")
@patch("tasks.tasks.Scan.objects.filter")
def test_no_completed_scans_skips_dispatch(
self, mock_scan_filter, mock_group, mock_chain
):
mock_scan_filter.return_value.order_by.return_value.values.return_value = []
result = reaggregate_all_finding_group_summaries_task(tenant_id=self.tenant_id)
assert result == {"scans_reaggregated": 0}
mock_group.assert_not_called()
mock_chain.assert_not_called()
Generated
+4 -14
View File
@@ -4835,8 +4835,8 @@ wheels = [
[[package]]
name = "prowler"
version = "5.41.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=master#f05a490cd74a2c0f11a5d66d8ce29d03fa5c64a2" }
version = "5.40.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=23e048fcd20738b68b54be6adc8b43f8a26d0d43#23e048fcd20738b68b54be6adc8b43f8a26d0d43" }
dependencies = [
{ name = "alibabacloud-actiontrail20200706" },
{ name = "alibabacloud-credentials" },
@@ -4930,7 +4930,6 @@ dependencies = [
{ name = "stackit-resourcemanager" },
{ name = "stackit-ske" },
{ name = "tabulate" },
{ name = "truststore" },
{ name = "tzlocal" },
{ name = "uuid6" },
{ name = "zstandard" },
@@ -4938,7 +4937,7 @@ dependencies = [
[[package]]
name = "prowler-api"
version = "1.44.0"
version = "1.41.0"
source = { virtual = "." }
dependencies = [
{ name = "cartography" },
@@ -5038,7 +5037,7 @@ requires-dist = [
{ name = "matplotlib", specifier = "==3.10.8" },
{ name = "neo4j", specifier = "==6.1.0" },
{ name = "openai", specifier = "==1.109.1" },
{ name = "prowler", git = "https://github.com/prowler-cloud/prowler.git?rev=master" },
{ name = "prowler", git = "https://github.com/prowler-cloud/prowler.git?rev=23e048fcd20738b68b54be6adc8b43f8a26d0d43" },
{ name = "psycopg2-binary", specifier = "==2.9.9" },
{ name = "pytest-celery", extras = ["redis"], specifier = "==1.3.0" },
{ name = "reportlab", specifier = "==4.4.10" },
@@ -6243,15 +6242,6 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl", hash = "sha256:26445eca388f82e72884e0d580d5464cd801a3ea01e63e5601bdff9ba6a48de2", size = 78540, upload-time = "2024-11-24T20:12:19.698Z" },
]
[[package]]
name = "truststore"
version = "0.10.4"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/53/a3/1585216310e344e8102c22482f6060c7a6ea0322b63e026372e6dcefcfd6/truststore-0.10.4.tar.gz", hash = "sha256:9d91bd436463ad5e4ee4aba766628dd6cd7010cf3e2461756b3303710eebc301", size = 26169, upload-time = "2025-08-12T18:49:02.73Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/19/97/56608b2249fe206a67cd573bc93cd9896e1efb9e98bce9c163bcdc704b88/truststore-0.10.4-py3-none-any.whl", hash = "sha256:adaeaecf1cbb5f4de3b1959b42d41f6fab57b2b1666adb59e89cb0b53361d981", size = 18660, upload-time = "2025-08-12T18:49:01.46Z" },
]
[[package]]
name = "typer"
version = "0.21.1"
+1 -279
View File
@@ -4,284 +4,6 @@ description: "New features and improvements in each Prowler release"
rss: true
---
<Update label="v5.42.0" description="September 11, 2026">
### ☁️ AWS — ISO Partitions
Prowler now resolves regions and services for the AWS ISO partitions (`aws-iso`, `aws-iso-b`, `aws-iso-e` and `aws-iso-f`) the same way it does for the commercial, China, European Sovereign Cloud and GovCloud partitions. The region matrix is filled from the endpoint metadata bundled with botocore, which needs no credentials or network access, so it covers partitions that are air-gapped from the internet. Scanning them no longer requires a hand-edited `aws_regions_by_service.json`: ISO regions such as `us-isob-east-1` are accepted by `--region` and `--excluded-region`.
Deployments that declare `PROWLER_AWS_PARTITION` also keep their bootstrap STS calls in the configured region when it belongs to that partition. An install in `us-gov-west-1` that reaches AWS only through its own VPC endpoints is no longer sent to `us-gov-east-1`, where the connection check and the scan used to time out.
Read more in the [AWS Regions and Partitions documentation](https://docs.prowler.com/user-guide/providers/aws/regions-and-partitions).
### ⏱️ AWS — Configurable Timeouts for Restricted Networks
Scans from networks with restricted egress (VPC endpoints for only some services, GovCloud or private deployments) could take hours: Boto3 waits 60 seconds to connect by default and retries connection errors, so every service without a reachable endpoint cost up to four 60-second attempts in every region. Prowler now lowers the default connect timeout to 10 seconds, keeps the read timeout at 60 seconds, and exposes both through `--aws-connect-timeout` and `--aws-read-timeout`, or through the `PROWLER_AWS_BOTO3_CONNECT_TIMEOUT` and `PROWLER_AWS_BOTO3_READ_TIMEOUT` environment variables for deployments without a CLI. `--aws-retries-max-attempts 0` now disables retries instead of silently falling back to three, leaving a single attempt per call.
Read more in the [Boto3 configuration documentation](https://docs.prowler.com/user-guide/providers/aws/boto3-configuration).
### 🐳 Image Provider — Reusable Vulnerability Database
The Image provider now honors `TRIVY_CACHE_DIR`. When the variable names a directory, Trivy keeps its vulnerability database there and Prowler leaves the directory in place after the scan, so the database is downloaded once instead of on every scan. Hosts without internet access can now scan images by pointing `TRIVY_CACHE_DIR` at a pre-populated database and setting `TRIVY_SKIP_DB_UPDATE=true`. Without the variable, the temporary cache is created and removed as before.
Read more in the [Image provider documentation](https://docs.prowler.com/user-guide/providers/image/getting-started-image#vulnerability-database-cache).
### 🎫 Jira Integration — Faster Connection Test
Testing a Jira integration no longer reports a false failure on accounts with many projects. The connection test fetched the issue types of every project one request at a time, which could outlast the wait in the UI even when the check was about to succeed. Issue types are now fetched concurrently, a project whose issue types the integration user cannot see is no longer logged as an error, and the Integrations page keeps following the connection test instead of giving up after about a minute.
Read more in the [Jira integration documentation](https://docs.prowler.com/user-guide/tutorials/prowler-app-jira-integration).
### 📚 Compliance — Catalog Integrity Fixes
A new integrity test runs over every compliance framework, asserting unique requirement IDs, no check listed twice within a requirement, and that every referenced check exists for its provider. The fixes it drove span 42 frameworks across AWS, Azure, GCP, GitHub, Kubernetes and Microsoft 365:
- **Duplicate requirement IDs:** identical copies are removed, and distinct requirements that shared an ID get their own, such as `1.10` in CIS AWS 5.0 and `rc_rp_1` for RC.RP-1 in NIST CSF 1.1. In Prowler ThreatScore for Azure, SQL auditing retention moves from `3.2.1` to `3.2.4`, and requirement `1.2.1` of Prowler ThreatScore for GCP now points to `iam_sa_no_user_managed_keys`.
- **Stale check references:** checks that no longer exist are replaced with their current name when there is a direct equivalent, or removed so the requirement reports as manual. Most of these were in the FedRAMP 20x KSI frameworks.
Renamed requirement IDs appear as new requirements for scans run after the upgrade.
The compliance overview task that runs after every scan is also faster: ThreatScore mappings are read once from the compliance template instead of from every finding, and rows are inserted with time-ordered `uuid7` IDs grouped by framework and requirement.
Read more in the [Compliance documentation](https://docs.prowler.com/user-guide/compliance/tutorials/compliance).
### 🔍 Checks
`rolesanywhere_profile_restricts_session_permissions`, `iam_role_service_trust_restricts_source_to_account` and `codebuild_project_uses_allowed_github_organizations` no longer crash with `TypeError` when the scanning role is denied `iam:ListRoles`, which dropped every finding of those checks for the account. Without the role inventory, an enabled IAM Roles Anywhere profile without session scoping reports `MANUAL`, and CodeBuild projects whose service role cannot be resolved are skipped.
Explore all AWS checks at [Prowler Hub](https://hub.prowler.com/check?provider=aws).
### 🔐 Security Updates
- `next` upgraded to 16.3.3 in the UI, patching unauthenticated remote code execution through AVIF image optimization ([GHSA-2xp9-vwfh-vxw4](https://github.com/advisories/GHSA-2xp9-vwfh-vxw4)) and on Windows-hosted servers ([GHSA-p293-qw3h-jr36](https://github.com/advisories/GHSA-p293-qw3h-jr36)).
- `sharp` upgraded to 0.35.4 in the UI, patching libheif image-decoding vulnerabilities ([GHSA-rgj7-g3m4-5g8c](https://github.com/advisories/GHSA-rgj7-g3m4-5g8c)).
- `nanoid`, `js-yaml` and `postcss`, plus eleven transitive UI dependencies, upgraded to patched versions, resolving 40 npm audit advisories (21 high, 15 moderate, 4 low).
- `libuuid` upgraded to 2.41.6-r1 in the MCP Server image, patching CVE-2026-53612, CVE-2026-53613, CVE-2026-53614, CVE-2026-76642, CVE-2026-78408 and CVE-2026-78410.
See the [full release notes on GitHub](https://github.com/prowler-cloud/prowler/releases/tag/5.42.0) for the complete list of changes.
</Update>
<Update label="v5.41.0" description="September 2, 2026">
### 📥 Scans — Import Findings from the Browser
<Note>
This feature is available exclusively in **Prowler Cloud** and **Prowler Private Cloud** with a [subscription](https://prowler.com/pricing).
</Note>
Findings produced outside the platform, by the Prowler CLI or a CI pipeline, can now be brought into the app without leaving the browser. The Scans page gains an "Import Findings" dialog that takes a Prowler `.ocsf.json` report by drag-and-drop or file picker, hands it to the ingestion API and tracks the job to completion, reporting how many records were processed and how many were invalid. Files that are not a `.ocsf.json` report, or are empty, are refused before any upload starts, and a rejected upload or a failed status poll can be retried in place. The dialog is available to roles holding the Manage Ingestions permission.
![Import Findings button on the Scans page](/images/prowler-app/import-findings/import-findings-button.png)
![Import findings dialog with the drag-and-drop area](/images/prowler-app/import-findings/import-findings-dialog.png)
Read more in the [Import Findings documentation](https://docs.prowler.com/user-guide/tutorials/prowler-import-findings#using-the-ui).
### 🎫 Jira Integration — Finding Reference in Every Issue
Every Jira issue created from a finding now carries a stable reference back to it. Issues are labeled `prowler`, `prowler-<provider>`, `prowler-<severity>`, `prowler-<check-id>` and `prowler-finding-<finding-uid>`, so they can be filtered, searched with JQL or matched by automation; labels are sanitized to Jira's limits so a long or unusual value never blocks issue creation. The issue also links back to the finding in Prowler, filtered by its UID so the link keeps working after later scans, and names the Prowler organization that sent it. Prowler Cloud always includes the link; Prowler Local Server enables it by setting `DJANGO_UI_BASE_URL` in the API environment.
Read more in the [Jira integration documentation](https://docs.prowler.com/user-guide/tutorials/prowler-app-jira-integration).
### 📚 Compliance — CIS Google Workspace Foundations Benchmark v1.4.0
Prowler now ships the CIS Google Workspace Foundations Benchmark v1.4.0. Alongside the new framework, the Google Workspace checks mapped to CIS were reworked to evaluate the benchmark's full audit procedure instead of a single condition, so Gmail spoofing actions, 2-Step Verification, password expiration and alert severity left on Google's defaults no longer pass. Expect new `FAIL` findings on domains that rely on those defaults. Three accuracy fixes also land:
- `security_2sv_enforced` and `security_2sv_hardware_keys_admins` report `MANUAL` instead of judging domain-wide values that a group or a sub-organizational unit overrides; a domain-wide failure is still reported as such, with the override noted.
- `rules_*_alert_configured` no longer passes a rule whose delivery to the alert center is disabled.
- `security_password_policy_strong` no longer fails a domain that never touched the password strength setting, since Google enforces strong passwords by default.
`security_login_challenges_configured` was unmapped from CIS Google Workspace requirement 4.1.4.1 (Post-SSO verification) and `security_2sv_enforced` from CISA SCuBA `GWS.COMMONCONTROLS.1.1` (phishing-resistant MFA), because neither check can prove what those requirements ask for.
Read more in the [Compliance documentation](https://docs.prowler.com/user-guide/compliance/tutorials/compliance).
### 🔍 Checks
Ten new AWS checks land in this release, eight of them contributed by @tamg-aws. Thank you!
#### Amazon Bedrock AgentCore
- `iam_policy_passrole_to_bedrock_agentcore_restricted` flags customer-managed IAM policies that allow `iam:PassRole` over every role where the passed role can reach Bedrock AgentCore, so any principal holding the policy could run agent code under any role in the account.
- `iam_policy_no_agentcore_workload_access_token_wildcard` flags customer-managed IAM policies that allow `bedrock-agentcore:GetWorkloadAccessToken`, `GetWorkloadAccessTokenForJWT` or `GetWorkloadAccessTokenForUserId` on resources reaching workload identities other than the caller's own.
- `cloudwatch_log_group_agentcore_data_protection_policy_enabled` verifies that Bedrock AgentCore log groups mask sensitive data with a CloudWatch Logs data protection policy. The log group prefixes are configurable through `agentcore_log_group_name_prefixes` in `config.yaml`.
#### Amazon GuardDuty
- `guardduty_runtime_monitoring_enabled` flags detectors without unified Runtime Monitoring, the only feature that covers Amazon EC2 instances and Amazon ECS on AWS Fargate tasks in addition to Amazon EKS.
- `guardduty_ai_protection_enabled` flags detectors without AI Protection, which analyzes CloudTrail data events from Amazon Bedrock, Amazon Bedrock AgentCore and Amazon SageMaker AI. A detector that does not report the feature is `MANUAL` rather than `FAIL`.
`guardduty_eks_runtime_monitoring_enabled` no longer reports `FAIL` for detectors that use unified Runtime Monitoring, which is mutually exclusive with `EKS_RUNTIME_MONITORING` and already covers Amazon EKS.
#### Amazon ECR and EKS
- `ecr_registry_enhanced_scanning_enabled` verifies that the ECR registry scan type is enhanced (Amazon Inspector, covering programming language packages and continuous rescanning) instead of basic, reporting `MANUAL` when the registry scanning configuration cannot be read.
- `eks_cluster_vpc_cni_network_policy_enforced` flags EKS clusters whose Amazon VPC CNI managed add-on does not enable Kubernetes network policy enforcement, reporting `MANUAL` where the EKS API cannot show the setting.
#### AWS IAM, Elastic Beanstalk and MemoryDB
- `iam_role_service_trust_restricts_source_to_account` flags IAM roles whose trust policy lets an AWS service principal assume the role without confining the request to a specific source account, including trust policies that `iam_role_cross_service_confused_deputy_prevention` does not evaluate.
- `elasticbeanstalk_environment_no_secrets_in_configuration` scans the option settings of every Elastic Beanstalk environment for hardcoded secrets. Thanks to @haneul-24!
- `memorydb_cluster_in_transit_encryption_enabled` verifies that MemoryDB clusters have in-transit encryption (TLS) enabled. Thanks to @UTKARSH698!
Explore all AWS checks at [Prowler Hub](https://hub.prowler.com/check?provider=aws).
### 🐳 Image Provider — On-Premises Registries
Scanning registries that live on private networks is now supported end to end. `PROWLER_IMAGE_PROVIDER_ALLOWED_PRIVATE_NETWORKS` takes a comma-separated list of IPs and CIDRs the provider may reach, while every other non-public address, including link-local and loopback, stays blocked by the SSRF guard. Authentication negotiation is also more resilient: the provider falls back to Basic when a registry such as Harbor rejects the negotiated bearer token, and switches to a bearer token when the server answers a Basic or anonymous request with a Bearer challenge. `--registry-insecure` now propagates to Trivy through `TRIVY_INSECURE`, so images behind self-signed certificates can be pulled and scanned, not just enumerated. The flag now disables certificate validation for the image pull too, so keep it for trusted internal registries only.
Registry scans also skip non-image OCI artifacts (Helm charts, cosign signatures, SBOM attestations), no longer abort the whole scan when Trivy fails on a single image, and enumerate repositories in parallel instead of one request at a time.
Read more in the [Image provider documentation](https://docs.prowler.com/user-guide/providers/image/getting-started-image#on-premises-registries-and-private-networks).
### 🛠️ Prowler MCP Server — Tool Failures Reported as Errors
Prowler Local Server tools now report a failure as an MCP tool execution error (`isError: true`, with the explanation in `content`) instead of a successful result carrying an `{"error": ...}` object, which clients and models read as a success. The Prowler Documentation and Prowler Hub tools follow the same rule: `prowler_docs_search` no longer reports a failed search as zero matches, `prowler_docs_get_document` no longer reports a failed fetch as a missing page, and `prowler_hub_get_check_code` and `prowler_hub_get_check_fixer` now name the provider a check ID actually belongs to instead of reporting it as nonexistent. `prowler_get_compliance_framework_state_details` also rejects a call that passes both `scan_id` and `provider_id` instead of silently ignoring the provider.
Read more in the [Prowler MCP documentation](https://docs.prowler.com/getting-started/products/prowler-mcp).
### 🙌 External Contributors
Thank you to our community contributors for this release!
- @tamg-aws: GuardDuty unified Runtime Monitoring and AI Protection checks ([#12564](https://github.com/prowler-cloud/prowler/pull/12564)), EKS VPC CNI network policy check ([#12661](https://github.com/prowler-cloud/prowler/pull/12661)), ECR enhanced scanning check ([#12660](https://github.com/prowler-cloud/prowler/pull/12660)), Bedrock AgentCore IAM and service trust checks ([#12664](https://github.com/prowler-cloud/prowler/pull/12664)), AgentCore log group data protection check ([#12662](https://github.com/prowler-cloud/prowler/pull/12662)), and fixes to ECR scan frequency ([#12560](https://github.com/prowler-cloud/prowler/pull/12560)), CloudWatch metric filters ([#12561](https://github.com/prowler-cloud/prowler/pull/12561)) and SageMaker direct internet access ([#12659](https://github.com/prowler-cloud/prowler/pull/12659))
- @haneul-24: AWS `elasticbeanstalk_environment_no_secrets_in_configuration` check ([#12378](https://github.com/prowler-cloud/prowler/pull/12378))
- @UTKARSH698: AWS `memorydb_cluster_in_transit_encryption_enabled` check ([#12246](https://github.com/prowler-cloud/prowler/pull/12246))
- @ye11oc4t: GitHub repository discovery pagination for unscoped scans ([#12460](https://github.com/prowler-cloud/prowler/pull/12460))
See the [full release notes on GitHub](https://github.com/prowler-cloud/prowler/releases/tag/5.41.0) for the complete list of changes.
</Update>
<Update label="v5.40.0" description="August 28, 2026">
### 💬 Slack Integration — Alert Channel Destinations
<Note>
This feature is available exclusively in **Prowler Cloud** and **Prowler Private Cloud** with a [subscription](https://prowler.com/pricing).
</Note>
Alerts can now reach Slack. Connect a Slack workspace from the Integrations page, authorize one or several destination channels (the connection check confirms each channel with a one-time message and names any channel Slack refuses), and pick those channels in the alert modal's "Destination channels" selector, next to the "Recipients" selector for email. The alerts list summarizes both in a single "Destinations" column, showing a rule's email recipients and Slack channels at a glance. Disconnecting the workspace and recovering from revoked credentials are handled from the same page.
![Connected Slack workspace on the Integrations page](/images/prowler-app/slack/connected-workspace.png)
![Alert rule with Slack channel destinations](/images/prowler-app/alerts/create-alert-modal.png)
Read more in the [Slack integration documentation](https://docs.prowler.com/user-guide/tutorials/prowler-app-slack-integration) and the [Alerts documentation](https://docs.prowler.com/user-guide/tutorials/prowler-alerts).
### 🤖 Lighthouse AI — Answer Feedback
<Note>
This feature is available exclusively in **Prowler Cloud** and **Prowler Private Cloud** with a [subscription](https://prowler.com/pricing).
</Note>
Every Lighthouse AI answer can now be rated with a thumbs up or thumbs down, with an optional field to describe what worked or what did not. Feedback is collected per answer, directly in the chat, and tells the team where Lighthouse should improve next.
Read more in the [Lighthouse AI documentation](https://docs.prowler.com/getting-started/products/prowler-cloud-lighthouse).
### 📥 Providers — Imported Findings Indicator
<Note>
This feature is available exclusively in **Prowler Cloud** and **Prowler Private Cloud** with a [subscription](https://prowler.com/pricing).
</Note>
Providers whose findings were imported with the Prowler CLI now show an "Imported provider" indicator next to their connection status in the providers table. In accounts that mix connected providers with Import Findings uploads, the table now tells them apart at a glance.
![Provider row with the Imported provider indicator and its tooltip](/images/changelog/v5.40.0-imported-provider-indicator.png)
Read more in the [Import Findings documentation](https://docs.prowler.com/user-guide/tutorials/prowler-import-findings).
### 📚 Compliance — NCSC Cyber Essentials 3.3
Cyber Essentials is the UK National Cyber Security Centre (NCSC) scheme certifying the baseline technical controls an organization must implement, and cloud services are explicitly in scope and cannot be excluded from an assessment. Prowler now includes NCSC Cyber Essentials: Requirements for IT Infrastructure v3.3 (April 2026) as a universal framework, with its 28 requirements organized in the five control themes: Firewalls, Secure Configuration, Security Update Management, User Access Control, and Malware Protection.
Sixteen requirements map to Azure checks covering the controls the applicant organization owns under the shared responsibility model. The remaining twelve apply to end-user devices, on-premises network appliances, or organizational process, which cloud control-plane evidence cannot observe, so they are reported as Manual.
Contributed by @m-khan-97. Thank you!
Read more in the [Compliance documentation](https://docs.prowler.com/user-guide/compliance/tutorials/compliance).
### 🔍 Checks
Fifteen new checks land across seven providers in this release.
#### AWS
- `ecr_repository_image_no_secrets` scans the latest image of each ECR repository, both its configuration and its filesystem layers, for hardcoded secrets. Thanks to @esquaredsec!
- Four new Amazon Bedrock checks, thanks to @tamg-aws!
- `bedrock_guardrail_contextual_grounding_filter_enabled` verifies that guardrails enable both contextual grounding filters, blocking responses that are not supported by the retrieved source or do not answer the question asked.
- `bedrock_custom_model_encrypted_with_cmk` verifies that custom models are encrypted at rest with a customer-managed KMS key instead of an AWS-owned key the organization cannot audit, rotate, or revoke.
- `bedrock_knowledge_base_encrypted_with_cmk` verifies that each knowledge-base data source encrypts with a customer-managed KMS key the transient storage used while documents are chunked and embedded.
- `bedrock_agent_role_not_shared_across_agents` verifies that every agent has a dedicated execution role, so no agent inherits another's permissions.
- `rolesanywhere_profile_restricts_session_permissions` flags IAM Roles Anywhere profiles that reference an administrative role without scoping down the vended session with a session policy or managed policies.
Explore all AWS checks at [Prowler Hub](https://hub.prowler.com/check?provider=aws).
#### GCP
- `iam_workload_identity_pool_provider_attribute_condition` flags Workload Identity Federation providers that trust a multi-tenant issuer without an attribute condition restricting which external identities can impersonate federated principals.
Explore all GCP checks at [Prowler Hub](https://hub.prowler.com/check?provider=gcp).
#### GitHub
Three new checks harden GitHub Actions defaults, all contributed by @Edneam. Thank you!
- `organization_default_workflow_permissions_read_only` and `repository_default_workflow_permissions_read_only` verify that workflows get a read-only default `GITHUB_TOKEN` at the organization and repository level.
- `organization_actions_pull_request_approval_disabled` verifies that organizations prevent GitHub Actions from creating and approving pull requests.
Explore all GitHub checks at [Prowler Hub](https://hub.prowler.com/check?provider=github).
#### Microsoft 365
- `defender_domain_dmarc_records_published` checks that every Exchange Online domain publishes a DMARC record with an enforcing policy (`p=quarantine` or `p=reject`). Thanks to @Rishi943!
Explore all Microsoft 365 checks at [Prowler Hub](https://hub.prowler.com/check?provider=m365).
#### Alibaba Cloud
- `oss_bucket_versioning_enabled` verifies that OSS buckets keep versioning enabled, allowing recovery from accidental or malicious object overwrite and deletion. Thanks to @abidedavana!
- `oss_bucket_server_side_encryption_enabled` verifies that OSS buckets define a default server-side encryption rule, either AES256 or KMS. Thanks to @alexchen-sys!
OSS bucket logging, versioning, default encryption, and ACL configurations are also now read correctly from the Alibaba Cloud SDK, so the checks reading them no longer report every bucket as unconfigured.
Explore all Alibaba Cloud checks at [Prowler Hub](https://hub.prowler.com/check?provider=alibabacloud).
#### Huawei Cloud
- `vpc_security_group_open_egress` flags VPC security groups that allow open egress to the internet. Thanks to @tomitobio!
Explore all Huawei Cloud checks at [Prowler Hub](https://hub.prowler.com/check?provider=huaweicloud).
#### STACKIT
- `ske_cluster_no_public_endpoint` flags SKE clusters whose Kubernetes API endpoint is reachable from the whole internet, because the ACL extension is disabled or its allowed CIDR list contains `0.0.0.0/0` or `::/0`. Thanks to @johannes-engler-mw!
Explore all STACKIT checks at [Prowler Hub](https://hub.prowler.com/check?provider=stackit).
### 🔐 Security Updates
- The API and SDK container images upgrade OpenSSL to 3.5.7-1~deb13u2, patching ten high CVEs; the UI image upgrades `libcrypto3` and `libssl3` to 3.5.8-r0, patching seven high CVEs; the MCP Server image patches CVE-2026-14456 (OpenSSL), CVE-2026-11822, and CVE-2026-11824 (SQLite).
- `sqlparse` upgraded to 0.6.0 in the API, patching CVE-2026-54284, CVE-2026-59893, and CVE-2026-71491.
### 🙌 External Contributors
Thank you to our community contributors for this release!
- @Edneam: GitHub `organization_default_workflow_permissions_read_only` ([#12122](https://github.com/prowler-cloud/prowler/pull/12122)), `repository_default_workflow_permissions_read_only` ([#12143](https://github.com/prowler-cloud/prowler/pull/12143)), and `organization_actions_pull_request_approval_disabled` ([#12394](https://github.com/prowler-cloud/prowler/pull/12394)) checks
- @tamg-aws: four AWS Bedrock checks covering guardrail grounding, CMK encryption, and agent role isolation ([#12459](https://github.com/prowler-cloud/prowler/pull/12459))
- @esquaredsec: AWS `ecr_repository_image_no_secrets` check ([#12123](https://github.com/prowler-cloud/prowler/pull/12123))
- @Rishi943: Microsoft 365 `defender_domain_dmarc_records_published` check ([#11936](https://github.com/prowler-cloud/prowler/pull/11936))
- @abidedavana: Alibaba Cloud `oss_bucket_versioning_enabled` check ([#11913](https://github.com/prowler-cloud/prowler/pull/11913))
- @alexchen-sys: Alibaba Cloud `oss_bucket_server_side_encryption_enabled` check ([#11981](https://github.com/prowler-cloud/prowler/pull/11981))
- @tomitobio: Huawei Cloud `vpc_security_group_open_egress` check ([#12209](https://github.com/prowler-cloud/prowler/pull/12209))
- @johannes-engler-mw: STACKIT `ske_cluster_no_public_endpoint` check ([#11943](https://github.com/prowler-cloud/prowler/pull/11943))
- @gabrielfrdev: cluster name in Kubernetes compliance report outputs ([#12506](https://github.com/prowler-cloud/prowler/pull/12506))
- @jfgmesquita: AWS FSBP compliance mapping fix for IAM.9 and EKS.1 ([#12372](https://github.com/prowler-cloud/prowler/pull/12372))
- @hackertwinten: `ec2_securitygroup_not_used` no longer flags security groups held only by scaled-down AWS Batch compute environments ([#12458](https://github.com/prowler-cloud/prowler/pull/12458))
- @0xTaoZ: ECS task-definition checks no longer report PASS when `DescribeTaskDefinition` fails, shipped early in v5.39.1 ([#12217](https://github.com/prowler-cloud/prowler/pull/12217))
- @ye11oc4t: `ses_identity_not_publicly_accessible` now evaluates every identity authorization policy, shipped early in v5.39.1 ([#12464](https://github.com/prowler-cloud/prowler/pull/12464))
- @Zuhef: IaC provider raises typed exceptions instead of `sys.exit` when cloning the scanned repository or running Trivy fails ([#12227](https://github.com/prowler-cloud/prowler/pull/12227)), Kubernetes kubelet checks no longer disappear from the scan when a `kubelet-config` ConfigMap is broken ([#12225](https://github.com/prowler-cloud/prowler/pull/12225)), and the CLI `--slack` summary is sent for scans with no findings instead of failing with `ZeroDivisionError` ([#12229](https://github.com/prowler-cloud/prowler/pull/12229))
- @m-khan-97: NCSC Cyber Essentials 3.3 compliance framework with Azure provider coverage across the five Cyber Essentials themes ([#11588](https://github.com/prowler-cloud/prowler/pull/11588))
See the [full release notes on GitHub](https://github.com/prowler-cloud/prowler/releases/tag/5.40.0) for the complete list of changes.
</Update>
<Update label="v5.39.0" description="August 13, 2026">
### 🤖 Lighthouse AI — Finding Skills
@@ -554,7 +276,7 @@ rss: true
All checks are fully passive, using AWS APIs and CloudTrail with no instance access or SSM agent required, and are mapped across 23 compliance frameworks, including NIST 800-53 Rev 5, PCI-DSS v4.0, ISO 27001:2022, SOC 2, HIPAA, and MITRE ATT&CK.
Read more about it in this [blog post](https://prowler.com/blog/your-llm-runs-in-a-nitro-enclave-who-is-checking-the-enclave).
Read more about it this [blog post](https://prowler.com/blog/your-llm-runs-in-a-nitro-enclave-who-is-checking-the-enclave).
Try them out now at [cloud.prowler.com](https://cloud.prowler.com/sign-up)!
+2 -2
View File
@@ -57,7 +57,7 @@ The AWS provider implementation follows the general [Provider structure](/develo
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/aws/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used across all AWS services.
@@ -131,7 +131,7 @@ def _get_email_identities(self, identity):
The AWS checks pattern is described in [checks page](/developer-guide/checks). You can find all the right now implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/aws/services/s3/s3_bucket_acl_prohibited/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services/s3/s3_bucket_acl_prohibited))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new check is following the [check creation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
+1 -32
View File
@@ -129,42 +129,12 @@ Each check **must** populate the `report.status` and `report.status_extended` fi
- Status field: `report.status`
- `PASS` – Assigned when the check confirms compliance with the configured value.
- `FAIL` – Assigned when the check detects non-compliance with the configured value.
- `MANUAL` – This status must not be used unless manual verification is necessary to determine whether the status (`report.status`) passes (`PASS`) or fails (`FAIL`). This includes the case where Prowler could not retrieve the data needed to evaluate the resource (see below).
- `MANUAL` – This status must not be used unless manual verification is necessary to determine whether the status (`report.status`) passes (`PASS`) or fails (`FAIL`).
- Status extended field: `report.status_extended`
- It **must** end with a period (`.`).
- It **must** include the audited service, the resource, and a concise explanation of the check result, for instance: `EC2 AMI ami-0123456789 is not public.`.
### Permission and Data-Availability Errors Are Not Findings
A `FAIL` must only be emitted when an insecure condition has actually been detected. A check **must never** report `FAIL` because the underlying API call failed: missing permissions or scopes on the scanning identity, an API that is not enabled, a feature that is not licensed, or data that could not be retrieved are scan-configuration problems, not security issues. Reporting them as `FAIL` surfaces a misleading (and often high-severity) finding to the user and skews compliance scores.
When the service layer cannot obtain the data a check depends on, the check must:
1. Emit a single `MANUAL` finding scoped to the widest affected resource (the tenant, account, project or subscription), not one finding per resource. For example, if user registration details cannot be read, emit one tenant-level `MANUAL` instead of one per user.
2. Explain in `status_extended` that the check could not be evaluated and what to fix, naming the permission, scope, API or license required, for instance: `Cannot evaluate credential exposure for privileged users: unable to query Microsoft Defender XDR Advanced Hunting. Verify that the ThreatHunting.Read.All permission is granted to the scanning application.`
3. Leave the check's severity untouched. Do not override `report.check_metadata.Severity` to hide the problem.
The service layer must make the distinction possible: log the error and expose it to checks in a way that cannot be confused with a legitimate empty result. Common patterns already used in Prowler are:
- Defaulting the attribute to `None` (data could not be read) instead of `[]`/`{}` (data was read and is empty), e.g. the `metric_filters is not None` guard in `prowler/providers/aws/services/cloudwatch/lib/metric_filters.py`.
- Keeping an availability flag raised on any denied listing, e.g. `logs_client.metric_filters_unavailable` consumed by the AWS CloudWatch metric filter checks.
- Keeping an error flag or message next to the data, e.g. `entra_client.user_registration_details_error` in M365 or `*_scan_errors` in AWS Bedrock.
- Keeping a set of resources whose lookup failed, e.g. `accessapproval_client.settings_lookup_failed` in GCP.
Make sure the error branch only captures real access errors. A `404`/not-found response frequently means the feature is simply not configured, which **is** a legitimate `FAIL`; a `403` or an unexpected exception is not. An "API not enabled" error is usually a scan-configuration problem too — **except** when the API's activation is itself the control being audited (e.g. GCP Access Approval: with `accessapproval.googleapis.com` disabled the feature provably cannot be enabled, so a definitive API-disabled state is a legitimate `FAIL`, while an undetermined state stays `MANUAL`).
```python
if <service>_client.<data> is None:
report = CheckReport<Provider>(metadata=self.metadata(), resource={})
report.resource_name = "<Tenant/Account-level resource>"
report.resource_id = "<stable-id>"
report.status = "MANUAL"
report.status_extended = "Cannot evaluate <requirement>: <data> could not be retrieved. Verify that <permission/API/license> is granted to the scanning identity."
findings.append(report)
return findings
```
### Prowler's Check Severity Levels
The severity of each check is defined in the metadata file using the `Severity` field. Severity values are always lowercase and must be one of the predefined categories below.
@@ -467,7 +437,6 @@ The metadata structure is enforced in code using a Pydantic model. For reference
- Use clear, actionable, and user-friendly language in `status_extended` to explain the result. Always provide information to identify the resource.
- Use helper functions/utilities for repeated logic to avoid code duplication. Save them in the `lib` folder of the service.
- Handle exceptions gracefully: catch errors per resource, log them, and continue processing other resources.
- Never report `FAIL` because data could not be retrieved (missing permissions, API not enabled, feature not licensed). Emit a single `MANUAL` finding explaining what is required instead; see [Permission and Data-Availability Errors Are Not Findings](#permission-and-data-availability-errors-are-not-findings).
- Document the check with a class and function level docstring explaining what it does, what it checks, and any caveats or provider-specific behaviors.
- Use type hints for the `execute()` method (e.g., `-> list[CheckReport<Provider>]`) for clarity and static analysis.
- Ensure checks are efficient; avoid excessive nested loops. If the complexity is high, consider refactoring the check.
+1 -1
View File
@@ -10,7 +10,7 @@ Visual Studio Code (also referred to as VSCode) provides an integrated debugger
### Debugging Configuration Example
The following file is an example of a [debugging configuration](https://code.visualstudio.com/docs/editor/debugging#_launch-configurations) file for [Visual Studio Code](https://code.visualstudio.com/).
The following file is an example of a [debugging configuration](https://code.visualstudio.com/docs/editor/debugging#_launch-configurations) file for [Virtual Studio Code](https://code.visualstudio.com/).
This file must be placed inside the *.vscode* directory and named *launch.json*:
+3 -3
View File
@@ -50,7 +50,7 @@ When adding or maintaining E2E tests for Prowler Local Server, follow these guid
```
5. **Tag and document scenarios**
- Follow the existing naming convention for suites and test cases (for example, `SCANS-E2E-001`, `PROVIDER-E2E-003`) and use tags such as `@e2e`, `@serial` and feature tags (for example, `@providers`, `@scans`, `@aws`) to filter and organize tests.
- Follow the existing naming convention for suites and test cases (for example, `SCANS-E2E-001`, `PROVIDER-E2E-003`) and use tags such as `@e2e`, `@serial` and feature tags (for example, `@providers`, `@scans`,`@aws`) to filter and organize tests.
**Example:**
```typescript
@@ -71,7 +71,7 @@ When adding or maintaining E2E tests for Prowler Local Server, follow these guid
}
);
```
- Document each one in the Markdown files under `ui/tests`, including **Priority**, **Tags**, **Description**, **Preconditions**, **Flow steps**, **Expected results**, **Key verification points** and **Notes**.
- Document each one in the Markdown files under `ui/tests`, including **Priority**, **Tags**, **Description**, **Preconditions**, **Flow steps**, **Expected results**,**Key verification points** and **Notes**.
**Example**
```Markdown
@@ -256,7 +256,7 @@ To execute E2E tests for Prowler Local Server:
pnpm run test:e2e
```
This command runs Playwright with the configured projects.
This command runs Playwright with the configured projects
2. **Run E2E tests with the Playwright UI runner**
+13 -57
View File
@@ -120,14 +120,14 @@ class NewFeatureTools(BaseTool):
Returns complete feature details including configuration and metadata.
"""
response = await self.api_client.get(f"/api/v1/features/{feature_id}")
return DetailedFeature.from_api_response(response["data"]).model_dump()
try:
response = await self.api_client.get(f"/api/v1/features/{feature_id}")
return DetailedFeature.from_api_response(response["data"]).model_dump()
except Exception as e:
self.logger.error(f"Failed to get feature {feature_id}: {e}")
return {"error": str(e), "status": "failed"}
```
There is no `try`/`except` here on purpose. A failed request raises, and
[Error Handling](#error-handling) explains what turns that raise into a message
the agent can act on.
### Step 2: Create the Models
Create corresponding models in `prowler_app/models/`:
@@ -369,62 +369,18 @@ async def search_items(self, status: str = Field(...)) -> dict:
### Error Handling
**Raise, never return.** A returned `{"error": ...}` dict is reported to the
client as `isError: false` -- a *successful* tool call whose payload happens to
mention a failure. Clients and models read that as success. A raised exception
becomes a spec-correct tool execution error instead.
The common case therefore needs no handler at all:
Return structured error responses instead of raising exceptions:
```python
async def get_item(self, item_id: str) -> dict:
response = await self.api_client.get(f"/api/v1/items/{item_id}")
return DetailedItem.from_api_response(response["data"]).model_dump()
try:
response = await self.api_client.get(f"/api/v1/items/{item_id}")
return DetailedItem.from_api_response(response["data"]).model_dump()
except Exception as e:
self.logger.error(f"Failed to get item {item_id}: {e}")
return {"error": str(e), "status": "failed"}
```
`prowler_mcp_server/lib/errors.py` classifies the failures every tool shares --
a rejected credential, a missing permission, a rate limit, an outage, an
unreachable API, a bad argument -- and gives each one a message that says what
went wrong and what to do about it. Anything it does not recognise is masked,
because `mask_error_details=True` is set on every sub-server and upstream
response bodies must never be replayed into a model's context.
Three ways to raise, in the order to reach for them:
```python
from fastmcp.exceptions import ToolError
from prowler_mcp_server.lib.errors import InvalidArgument
# 1. An argument this server rejected before any request went out. The message
# is repeated to the agent verbatim, so write it for one to read.
if not 1 <= page_size <= 1000:
raise InvalidArgument("page_size must be between 1 and 1000.")
# 2. A request the API answered or never answered: let it propagate untouched.
# `ProwlerAPIError` and `ProwlerAPIUnreachable` are what the classifier keys
# on, and the second one is what stops a retry from duplicating a write.
response = await self.api_client.get(f"/api/v1/items/{item_id}")
data = response["data"]
# 3. A sentence the classifier cannot know -- a resource name, a precondition,
# the next tool to call. NOTE the absent `from` clause: it is what marks the
# message as already final. With `from e` the classifier would replace it.
if not data:
raise ToolError(
f"No item with the ID {item_id!r} exists. Use prowler_list_items to "
"find a valid one."
)
```
The one thing that still *returns* rather than raises is a write whose outcome is
genuinely unknown. `prowler_send_findings_to_jira` is the worked example: work
items are created one at a time and Prowler cannot delete them, so a dispatch
that stopped halfway answers with a result object carrying
`safe_to_retry: false`. "This may have been applied" is a fact about the world,
not an error, and squashing it into one loses the only thing that stops a retry
from duplicating the write.
### Parameter Descriptions
Use Pydantic `Field()` with clear descriptions. This also helps LLMs understand
+7 -7
View File
@@ -25,7 +25,7 @@ For providers supported by Prowler, refer to [Prowler Hub](https://hub.prowler.c
Prowler supports several types of providers, each with its own implementation pattern and use case. Understanding these differences is key to designing your provider correctly.
### Classifying Your Provider
### Classifying your Provider
Before implementing a new provider, you need to determine which type it belongs to. This classification will guide your implementation approach and help you choose the right patterns and libraries.
@@ -1090,7 +1090,7 @@ Main registration makes your provider discoverable by Prowler's core system. It'
cis.batch_write_data_to_file()
```
#### Step 11: Register in the List of Providers
#### Step 11: Register in the list of providers
**Explanation:**
This is needed to be able to use the provider in the generic checks. The provider must be registered in the `init_global_provider` method to handle CLI arguments and initialization.
@@ -1966,7 +1966,7 @@ Main registration makes your provider discoverable by Prowler's core system. It'
This step is the same as the [SDK providers](#step-10-register-in-main).
#### Step 11: Register in the List of Providers
#### Step 11: Register in the list of providers
**Explanation:**
This is needed to be able to use the provider in the generic checks. The provider must be registered in the `init_global_provider` method to handle CLI arguments and initialization.
@@ -2648,7 +2648,7 @@ Main registration makes your provider discoverable by Prowler's core system. It'
This step is the same as the [SDK providers](#step-10-register-in-main).
#### Step 7: Register in the List of Providers
#### Step 7: Register in the list of providers
**Explanation:**
This is needed to be able to use the provider in the generic checks. The provider must be registered in the `init_global_provider` method to handle CLI arguments and initialization.
@@ -2808,7 +2808,7 @@ def validate_your_provider_uid(value):
**Provider Model:**
The `Provider` model already exists and supports all provider types. Ensure your provider type is included in the choices.
### 2.2. Add the Provider to the Provider Choices
### 2.2. Add the provider to the Provider Choices
Update the `return_prowler_provider` function to include your provider. This function is crucial for the API to instantiate the correct provider class.
@@ -3209,7 +3209,7 @@ class YourProviderAPITestCase(APITestCase):
self.assertEqual(response.status_code, 201)
```
#### 2.6.1. Add Your Mocked Provider to the Tests
#### 2.6.1. Add your mocked provider to the tests
If needed, add a named provider fixture or extend the provider factory defaults so tests can request only the provider they need.
@@ -3272,7 +3272,7 @@ Your provider will be available through these endpoints:
- `DELETE /api/v1/providers/{id}/` - Delete provider
- `POST /api/v1/providers/secrets/` - Add provider credentials
### 2.9. Update the Provider If Needed
### 2.9. Update the provider if needed
Depending on your provider's authentication requirements, you may need to add new authentication methods that are compatible with the API. This involves updating the provider class to support additional credential types beyond the basic ones.
@@ -18,7 +18,7 @@ A compliance framework must represent the **complete state** of the source catal
Requirement coverage feeds the compliance percentage calculations and the metadata surfaces (dashboards, widgets, exports). Missing requirements skew those metrics and break the report as a faithful snapshot of the framework.
</Warning>
### Two Supported Schemas
### Two supported schemas
| Schema | When to use | File location | Discovered as |
| --- | --- | --- | --- |
@@ -45,7 +45,7 @@ Before adding a new framework, complete the following checks:
## Universal Compliance Framework
### Where the File Lives
### Where the file lives
Place the file at the top level of the compliance directory:
@@ -57,7 +57,7 @@ Examples in the repository: `prowler/compliance/csa_ccm_4.0.json`, `prowler/comp
The file is auto-discovered — there is **no** need to register it in any `__init__.py`, modify `prowler/lib/outputs/`, or update any other Python module. The framework key Prowler CLI accepts via `--compliance` is the basename of the JSON file without `.json` (`dora_2022_2554.json` → `dora_2022_2554`).
### Top-Level Structure
### Top-level structure
```json
{
@@ -198,7 +198,7 @@ Per requirement:
For MITRE-style frameworks, additional optional fields are available on the requirement: `tactics`, `sub_techniques`, `platforms`, `technique_url` (these are populated automatically when adapting a legacy MITRE JSON to the universal model).
### Multi-Provider Frameworks
### Multi-provider frameworks
A single universal file can cover any number of providers. The framework appears under each provider's `--list-compliance` output as long as **at least one** requirement has that provider key in its `checks` dict.
@@ -226,7 +226,7 @@ The legacy schema spans **four layers** — a complete contribution must touch e
The universal schema collapses Layers 3 and 4 into declarative configuration inside the JSON — that is the main reason it is preferred for new contributions.
### Directory Structure and File Naming
### Directory structure and file naming
Compliance frameworks live at:
@@ -259,7 +259,7 @@ prowler/lib/outputs/compliance/<framework>/
└── __init__.py
```
### JSON Schema Reference
### JSON schema reference
Every legacy compliance file is a JSON document with the following top-level keys. `Framework`, `Name` and `Provider` are validated non-empty by the root validator `framework_and_provider_must_not_be_empty` (`compliance_models.py`).
@@ -362,7 +362,7 @@ For the remaining attribute classes (`AWS_Well_Architected_Requirement_Attribute
The `Attributes` field is a Pydantic `Union`. The generic attribute model **must** remain the last element of that Union — otherwise Pydantic v1 silently coerces every framework into the generic shape and your specialized fields are dropped. Adding a brand-new attribute shape requires inserting the Pydantic class **before** `Generic_Compliance_Requirement_Attribute`.
</Note>
#### Minimal Working Example
#### Minimal working example
The following snippet is a complete, valid framework file named `my_framework_1.0_aws.json`, saved at `prowler/compliance/aws/my_framework_1.0_aws.json`. It uses the generic attribute shape for simplicity.
@@ -408,7 +408,7 @@ The following snippet is a complete, valid framework file named `my_framework_1.
}
```
### Mapping Checks to Requirements
### Mapping checks to requirements
Each requirement links to the Prowler checks that, together, produce a PASS or FAIL verdict for that control.
@@ -425,7 +425,7 @@ To discover available checks:
uv run python prowler-cli.py <provider> --list-checks
```
### Supporting Multiple Providers (Legacy)
### Supporting multiple providers (legacy)
The legacy schema binds each file to a single provider. To cover several providers with the same framework, ship one JSON file per provider:
@@ -439,7 +439,7 @@ Keep the `Framework` and `Version` values identical across the files so the disp
For a brand-new framework that spans several providers, **prefer the universal schema** — it covers every provider from a single file. If you must use the legacy schema, add one transformer per provider in `prowler/lib/outputs/compliance/<framework>/` and extend the summary-table dispatcher accordingly. See [Output Formatter](#output-formatter).
### Output Formatter
### Output formatter
Legacy frameworks render in two forms: a detailed CSV report written to disk, and a summary table printed in the CLI. Both are produced by the output formatter package for the framework. Universal frameworks do **not** need a Python output formatter — the `outputs` config inside the JSON drives rendering — so this section applies only to the legacy schema.
@@ -453,19 +453,19 @@ prowler/lib/outputs/compliance/my_framework/
└── models.py # CSV row Pydantic model
```
#### Step 1 — Define the CSV Row Model
#### Step 1 — Define the CSV row model
In `models.py`, declare a Pydantic v1 model with one field per CSV column. Use existing models such as `AWSCISModel` in `prowler/lib/outputs/compliance/cis/models.py` as the reference. Fields typically include `Provider`, `Description`, `AccountId`, `Region`, `AssessmentDate`, `Requirements_Id`, `Requirements_Description`, one `Requirements_Attributes_*` field per attribute key, plus the finding fields `Status`, `StatusExtended`, `ResourceId`, `ResourceName`, `CheckId`, `Muted`, `Framework`, `Name`.
#### Step 2 — Implement the Transformer
#### Step 2 — Implement the transformer
In `my_framework_aws.py`, subclass `ComplianceOutput` from `prowler.lib.outputs.compliance.compliance_output` and implement `transform(findings, compliance, compliance_name)`. Iterate over `findings`, match each finding to the requirements it satisfies through `finding.compliance.get(compliance_name, [])`, and append one row per attribute to `self._data`.
#### Step 3 — Add the Summary-Table Dispatcher
#### Step 3 — Add the summary-table dispatcher
In `my_framework.py`, implement `get_my_framework_table(findings, bulk_checks_metadata, compliance_framework, output_filename, output_directory, compliance_overview)` following the pattern in `prowler/lib/outputs/compliance/cis/cis.py`.
#### Step 4 — Register the Framework in the Dispatchers
#### Step 4 — Register the framework in the dispatchers
- Add the dispatcher call in `prowler/lib/outputs/compliance/compliance.py`, inside `display_compliance_table`, with a branch such as `elif "my_framework" in compliance_framework:`.
- Register the CSV model and transformer in `prowler/lib/outputs/compliance/compliance_output.py` so the CSV file is emitted during the scan.
@@ -474,7 +474,7 @@ In `my_framework.py`, implement `get_my_framework_table(findings, bulk_checks_me
For NIST-style catalogs that use `Generic_Compliance_Requirement_Attribute`, no custom formatter is needed. The generic formatter in `prowler/lib/outputs/compliance/generic/` handles them automatically, provided the JSON validates against the generic attribute schema.
</Note>
### Legacy-to-Universal Adapter
### Legacy-to-universal adapter
At load time, every legacy file is transparently adapted to a `ComplianceFramework` via `adapt_legacy_to_universal()` (`compliance_models.py`), which: (a) flattens the first element of `Attributes` into a flat `attributes` dict, (b) wraps `Checks` as `{provider_lower: [...]}`, (c) infers `attributes_metadata` from the matched Pydantic class via `_infer_attribute_metadata()`. The rest of Prowler (CSV/OCSF/PDF output, CLI table) then treats both formats identically.
@@ -497,7 +497,7 @@ Configuration guardrails close that gap. A requirement declares the configuratio
Guardrails are an **optional** safety net for configurable checks. A requirement that maps only to non-configurable checks does not need them. When the field is absent, behavior is unchanged.
</Note>
### Where Guardrails Are Declared
### Where guardrails are declared
The field is attached to each requirement and exists in both schemas:
@@ -506,7 +506,7 @@ The field is attached to each requirement and exists in both schemas:
When a legacy file is adapted to the universal model, `adapt_legacy_to_universal()` copies `ConfigRequirements` into `config_requirements` (`compliance_models.py`), so downstream code only ever reads one shape.
### Constraint Schema
### Constraint schema
Each entry in the list is a single constraint with the following fields:
@@ -533,7 +533,7 @@ Each entry in the list is a single constraint with the following fields:
`subset` / `superset` require both the applied value and `Value` to be lists; any other type is treated as not satisfied. For `eq` against a boolean, declare `Value` as a JSON boolean (`false`, not `0`) — the model keeps booleans distinct from integers.
</Note>
### How Guardrails Are Evaluated
### How guardrails are evaluated
All evaluation lives in one shared module, `prowler/lib/check/compliance_config_eval.py`, consumed by every compliance output (CSV, OCSF, and the CLI tables) and reused by the Prowler API backend so the rule is defined exactly once.
@@ -547,7 +547,7 @@ All evaluation lives in one shared module, `prowler/lib/check/compliance_config_
Guardrails only ever make a result **stricter** (they can turn PASS into FAIL); they never relax a real FAIL into PASS. A requirement with no constraints, or whose keys all use defaults, is reported exactly as before.
</Warning>
### Example: Legacy Framework
### Example: legacy framework
From `prowler/compliance/aws/cis_6.0_aws.json`, requirement 2.11 declares two guardrails — one per configurable check it maps to:
@@ -590,7 +590,7 @@ A boolean guardrail from the same file: requirement 2.5 (IAM Access Analyzer) on
]
```
### Example: Universal Framework
### Example: universal framework
The universal schema uses the lowercase `config_requirements` key with the identical object shape:
@@ -616,7 +616,7 @@ The universal schema uses the lowercase `config_requirements` key with the ident
Each constraint declares the `Provider` it targets so the guardrail is only evaluated on scans of that provider — essential for universal frameworks like CSA CCM and DORA, where one requirement maps checks across `aws`, `azure`, `gcp` and more. Because the operator is `subset`, adding `"TLS 1.0"` to `recommended_minimal_tls_versions` widens the allowlist beyond `["TLS 1.2", "TLS 1.3"]` and the requirement is forced to FAIL.
### What the User Sees
### What the user sees
With a loosened config, the affected requirement's findings report:
@@ -630,7 +630,7 @@ StatusExtended: Configuration not valid for this requirement. The check
The same `Configuration not valid for this requirement.` message appears identically across the CSV, OCSF, and console-table outputs.
### Authoring Guidelines
### Authoring guidelines
- Declare a guardrail only for keys whose value actually changes whether the requirement is met. Most configurable checks do not need one.
- Set `Value` to the **strictest** configuration the control tolerates — the same number the control text cites (CIS 45 days, NIST ≤90, and so on).
@@ -639,7 +639,7 @@ The same `Configuration not valid for this requirement.` message appears identic
- Pick the operator from the value's role: a max threshold is `lte`, a min threshold is `gte`, a toggle is `eq`, an allowlist is `subset`, a denylist is `superset`.
- An unrecognized operator does **not** block the requirement — a malformed constraint is treated as satisfied rather than failing the whole framework. Validate your JSON with the tests below.
### Testing Guardrails
### Testing guardrails
The shared evaluator and the per-output integration are covered by:
@@ -670,7 +670,7 @@ Prowler matches frameworks by concatenating `Framework` and `Version`. A missing
Before opening a PR, validate the JSON loads cleanly against the model and that every referenced check actually exists.
### 1. Schema Validation
### 1. Schema validation
For **universal** frameworks, load the file and inspect what was parsed. The framework key inside `bulk` is the **basename of the JSON file** (without `.json`); for `prowler/compliance/dora_2022_2554.json` that key is `dora_2022_2554`, for `prowler/compliance/aws/cis_5.0_aws.json` it is `cis_5.0_aws`.
@@ -688,7 +688,7 @@ bulk = get_bulk_compliance_frameworks_universal("aws")
assert "<your_framework_filename_without_json>" in bulk
```
### 2. Check Existence Cross-Check
### 2. Check existence cross-check
There is **no automatic check-existence validation** at load time. Cross-check that every check name in your framework maps to a real check directory:
@@ -708,7 +708,7 @@ missing = referenced - real
assert not missing, f"checks referenced in framework but not found in repo: {sorted(missing)}"
```
### 3. CLI Smoke Test
### 3. CLI smoke test
```bash
uv run python prowler-cli.py <provider> --list-compliance
@@ -728,7 +728,7 @@ Verify that:
- The CLI summary table lists every section / pillar of the framework.
- Findings roll up under the expected requirements.
### 4. Inspect the CSV Output
### 4. Inspect the CSV output
Open the generated CSV and confirm:
+9 -9
View File
@@ -12,7 +12,7 @@ This guide explains how to add a **Server-Sent Events (SSE)** endpoint to the Pr
The platform ships the SSE **infrastructure** (`api.sse`) and wiring. No feature endpoint streams over SSE out of the box — this guide shows how to build one on top of the shared base.
</Info>
## When to Use SSE
## When to use SSE
| Need | Use |
|------|-----|
@@ -22,7 +22,7 @@ The platform ships the SSE **infrastructure** (`api.sse`) and wiring. No feature
SSE is the right tool when the **client only consumes**: scan progress, long-running job checkpoints, streamed LLM tokens, cross-client resource-sync notifications. It rides on plain HTTP, reconnects automatically in the browser via the native [`EventSource`](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) API, and needs no extra protocol.
## How It Works
## How it works
SSE is wired through [`django-eventstream`](https://github.com/fanout/django_eventstream) and a small platform layer in `api/src/backend/api/sse/`:
@@ -34,11 +34,11 @@ SSE is wired through [`django-eventstream`](https://github.com/fanout/django_eve
| `make_channel_name` / `tenant_id_from_channel` | `api/sse/utils.py` | Single source of truth for the channel-name format, so publishers and the channel manager agree byte-for-byte. |
| Settings | `config/settings/eventstream.py` | Valkey Pub/Sub backend (dedicated DB), channel manager, allowed headers. |
### Transport: The Server Runs on ASGI
### Transport: the server runs on ASGI
SSE connections are long-lived. Holding one open per synchronous worker would exhaust the worker pool, so the API runs under Gunicorn's native **`asgi` worker** (`config.asgi:application`). Streams are parked on the event loop while ordinary CRUD endpoints keep their synchronous execution (Django runs sync views in a thread-sensitive executor under ASGI). This is configured in `config/guniconf.py` and used by both the dev and production entrypoints — no separate server process is needed.
### The Data Flow
### The data flow
```
publisher (Celery task / view) subscriber (browser, CLI)
@@ -53,7 +53,7 @@ publisher (Celery task / view) subscriber (browser, CLI)
A publisher anywhere in the system (most often a Celery task) calls `send_event(channel, event_type, payload)`. `django-eventstream` fans it out over Valkey Pub/Sub to every connection subscribed to that channel.
## Adding an SSE Endpoint to Your Feature
## Adding an SSE endpoint to your feature
The example below streams progress for a long-running **scan**. Adapt the resource, prefix, and event names to your feature.
@@ -161,7 +161,7 @@ publish_end(channel, scan_id=str(scan.id))
</Steps>
## Event Naming Convention
## Event naming convention
Every event uses an event type of the form **`<resource>.<verb>`** (lowercased, dot-separated). The verb comes from this platform-wide vocabulary — if you need a verb that is not listed, document the addition in this guide so the catalog stays discoverable.
@@ -197,7 +197,7 @@ curl -N -H "Authorization: Bearer $JWT" \
https://<host>/api/v1/scans/$SCAN_ID/event-stream
```
## Tenant Isolation & Security Model
## Tenant isolation & security model
Authorization is enforced at two layers:
@@ -206,7 +206,7 @@ Authorization is enforced at two layers:
Because the tenant id lives inside the channel name, this gate works for any feature without the platform knowing anything about it.
## Reconnect & State Recovery
## Reconnect & state recovery
The platform deliberately ships **without server-side replay** (`is_channel_reliable` returns `False`). When a client reconnects, it does **not** receive missed events. Instead:
@@ -215,7 +215,7 @@ The platform deliberately ships **without server-side replay** (`is_channel_reli
Design your event payloads accordingly: deltas are ephemeral and concatenated in-flight; the durable truth always lives behind a REST resource.
## Local Development
## Local development
- The dev and production entrypoints both launch Gunicorn with the `asgi` worker (`config.asgi:application`). In dev, `DJANGO_DEBUG=True` enables hot reload; `preload_app` is automatically disabled under debug so edited code is picked up.
- SSE uses a **dedicated Valkey database** (`EVENTSTREAM_VALKEY_DB`, default `2`) kept separate from the Celery broker so a noisy broker cannot crowd out streaming traffic. It reuses the same `VALKEY_*` connection settings as the rest of the platform.
+1 -1
View File
@@ -537,7 +537,7 @@ This architecture allows Prowler to efficiently scan AWS accounts with resources
## Best Practices
- When available in the provider, use threading or parallelization utilities for all methods that can be parallelized to maximize performance and reduce scan time.
- When available in the provider, use threading or parallelization utilities for all methods that can be parallelized by to maximize performance and reduce scan time.
- Define a Pydantic `BaseModel` for every resource you manage, and use these models for all resource data handling.
- Log every major step (start, success, error) in resource discovery and attribute collection for traceability and debugging; include as much context as possible.
- Catch and log all exceptions, providing detailed context (region, subscription, resource, error type, line number) to aid troubleshooting.
+4 -4
View File
@@ -154,7 +154,7 @@ Failing to update this table when adding cross-service dependencies may result i
For AWS provider, different testing approaches apply based on API coverage based on several criteria.
<Note>
Prowler leverages and contributes to the [Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
</Note>
- AWS API Calls Covered by [Moto](https://github.com/getmoto/moto):
@@ -408,7 +408,7 @@ In all above scenarios, check execution must occur within the context of mocked
When a service requires API calls that are partially covered by the Moto decorator, additional mocking is necessary. In such cases, custom mocked API calls must be implemented alongside Moto to ensure full coverage.
To achieve this, mock the `botocore.client.BaseClient._make_api_call` function—the method responsible for making actual API requests to AWS—using [`mock.patch`](https://docs.python.org/3/library/unittest.mock.html#patch):
To achieve this, mock the `botocore.client.BaseClient._make_api_call` function—the method responsible for making actual API requests to AWS—using `mock.patch <https://docs.python.org/3/library/unittest.mock.html#patch>`:
```python
@@ -475,7 +475,7 @@ However, if additional `moto` decorators are applied alongside the patch, Moto w
</Note>
<Note>
The source of the above implementation can be found here: [Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching_other_services.html)
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
</Note>
#### Mocking Several Services
@@ -603,7 +603,7 @@ with mock.patch(
will cause that the service is initialized only once—at the moment of mocking out `set_mocked_aws_provider([<region>])` using `mock.patch`.
Later, when Python attempts to import the client at the check level, the execution continues using `from prowler.providers.<provider>.services.<service>.<service>_client`. As a result of it being already mocked out, the execution will continue using `service_client` without getting into `<service>_client.py`.
Later, when Python attempts to import the client at the check level, the execution continues using`from prowler.providers.<provider>.services.<service>.<service>_client`. As a result of it being already mocked out, the execution will continue using `service_client` without getting into `<service>_client.py`.
### Testing AWS Services
+1 -12
View File
@@ -182,8 +182,7 @@
"pages": [
"user-guide/tutorials/prowler-app-s3-integration",
"user-guide/tutorials/prowler-app-security-hub-integration",
"user-guide/tutorials/prowler-app-jira-integration",
"user-guide/tutorials/prowler-app-slack-integration"
"user-guide/tutorials/prowler-app-jira-integration"
]
},
{
@@ -302,15 +301,6 @@
{
"group": "Providers",
"pages": [
{
"group": "Organizations",
"pages": [
"user-guide/organizations",
"user-guide/providers/aws/organizations",
"user-guide/providers/gcp/organization",
"user-guide/providers/azure/management-groups"
]
},
{
"group": "Alibaba Cloud",
"pages": [
@@ -340,7 +330,6 @@
"user-guide/providers/azure/getting-started-azure",
"user-guide/providers/azure/authentication",
"user-guide/providers/azure/use-non-default-cloud",
"user-guide/providers/azure/management-groups",
"user-guide/providers/azure/subscriptions",
"user-guide/providers/azure/resource-groups",
"user-guide/providers/azure/create-prowler-service-principal"
@@ -2,11 +2,9 @@
title: 'Basic Usage'
---
import { VersionBadge } from "/snippets/version-badge.mdx"
## Running Prowler
Running Prowler requires specifying the provider (e.g. `aws`, `gcp`, `azure`, `kubernetes`, `m365`, `github`, `iac` or `mongodbatlas`):
Running Prowler requires specifying the provider (e.g `aws`, `gcp`, `azure`, `kubernetes`, `m365`, `github`, `iac` or `mongodbatlas`):
<Note>
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
@@ -93,18 +91,6 @@ By default, `prowler` will scan all AWS regions.
</Note>
See more details about AWS Authentication in the [Authentication Section](/user-guide/providers/aws/authentication) section.
- **AWS Retrier and Timeout Configuration**
<VersionBadge version="5.42.0" />
Tune the Boto3 standard retrier and the endpoint timeouts when AWS throttles the scan or when some endpoints are unreachable from the network Prowler runs in:
```console
prowler aws --aws-retries-max-attempts 5 --aws-connect-timeout 5 --aws-read-timeout 30
```
See the [Boto3 configuration](/user-guide/providers/aws/boto3-configuration) page for defaults and environment variables.
## Azure
Azure requires specifying the auth method:
@@ -128,8 +128,8 @@ To update the environment file:
Edit the `.env` file and change version values:
```env
PROWLER_UI_VERSION="5.42.0"
PROWLER_API_VERSION="5.42.0"
PROWLER_UI_VERSION="5.39.0"
PROWLER_API_VERSION="5.39.0"
```
<Note>
@@ -161,7 +161,7 @@ The Prowler MCP Server enables powerful workflows through AI assistants:
- "What authentication methods does Prowler support for Azure?"
- "How can I contribute with a new security check to Prowler?"
### Example: Creating a Custom Dashboard with Prowler Extracted Data
### Example: Creating a custom dashboard with Prowler extracted data
In the next example you can see how to create a dashboard using Prowler MCP Server and Claude Desktop.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 74 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 118 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 145 KiB

After

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 194 KiB

After

Width:  |  Height:  |  Size: 210 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 187 KiB

After

Width:  |  Height:  |  Size: 160 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 169 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 156 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 186 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 93 KiB

-1
View File
@@ -92,7 +92,6 @@ li[id="/user-guide/tutorials/prowler-alerts"] a > div > div > span:first-child::
li[id="/user-guide/tutorials/prowler-app-attack-paths-active-queries"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-app-findings-triage"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-app-scan-configuration"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-app-slack-integration"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-cloud-aws-organizations"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-cloud-azure-management-groups"] a > div > div > span:first-child::after,
li[id="/user-guide/tutorials/prowler-cloud-gcp-organizations"] a > div > div > span:first-child::after,
+1 -1
View File
@@ -22,7 +22,7 @@ See section [Logging](/user-guide/cli/tutorials/logging) for further information
Common issues with the Docker Compose installation of Prowler Local Server.
### Problem Adding AWS Provider Using "Connect assuming IAM Role" in Docker
### Problem adding AWS Provider using "Connect assuming IAM Role" in Docker
See [GitHub Issue #7745](https://github.com/prowler-cloud/prowler/issues/7745) for more details.
@@ -51,7 +51,6 @@ The following list includes all the AWS checks with configurable variables that
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_actions` | List of Strings | See `config.yaml` |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_minutes` | Integer | `1440` |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_threshold` | Float | `0.2` |
| `cloudwatch_log_group_agentcore_data_protection_policy_enabled` | `agentcore_log_group_name_prefixes` | List of Strings | See `config.yaml` |
| `cloudwatch_log_group_no_secrets_in_logs` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `cloudwatch_log_group_retention_policy_specific_days_enabled` | `log_group_retention_days` | Integer | `365` |
| `codebuild_project_no_secrets_in_variables` | `excluded_sensitive_environment_variables` | List of Strings | `[]` |
+1 -1
View File
@@ -105,7 +105,7 @@ def fixer(resource_id: str) -> bool:
return True
```
## Fixer Config File
## Fixer Config file
For some fixers, you can have configurable parameters depending on your use case. You can either use the default config file in `prowler/config/fixer_config.yaml` or create a custom config file and pass it to the fixer with the `--fixer-config` flag. The config file should be a YAML file with the following structure:
+2 -2
View File
@@ -4,7 +4,7 @@ title: 'Miscellaneous'
## Prowler Version
### Showing the Prowler Version
### Showing the Prowler version:
```console
prowler <provider> -V/-v/--version
@@ -22,7 +22,7 @@ To enable verbose mode in Prowler, similar to Version 2, use:
prowler <provider> --verbose
```
### Filter Findings by Status
### Filter findings by status
Prowler allows filtering findings based on their status, ensuring reports and CLI display only relevant findings:
+1 -1
View File
@@ -268,7 +268,7 @@ Accounts:
## AWS Mutelist
### Muting Specific AWS Regions
### Muting specific AWS regions
If you want to mute failed findings only in specific regions, create a file with the following syntax and run it with `prowler aws -w mutelist.yaml`:
+4 -1
View File
@@ -43,7 +43,8 @@ prowler <provider> --categories secrets
Several checks analyse resources that are exposed to the Internet, these are:
- apigateway\_restapi\_public
1. apigateway\_restapi\_public
- appstream\_fleet\_default\_internet\_access\_disabled
- awslambda\_function\_not\_publicly\_accessible
- ec2\_ami\_public
@@ -57,6 +58,8 @@ Several checks analyse resources that are exposed to the Internet, these are:
- ecr\_repositories\_not\_publicly\_accessible
- eks\_control\_plane\_endpoint\_access\_restricted
- eks\_endpoints\_not\_publicly\_accessible
- eks\_control\_plane\_endpoint\_access\_restricted
- eks\_endpoints\_not\_publicly\_accessible
- elbv2\_internet\_facing
- kms\_key\_not\_publicly\_accessible
- opensearch\_service\_domains\_not\_publicly\_accessible
-58
View File
@@ -1,58 +0,0 @@
---
title: 'Organizations Across Cloud Providers'
description: 'Understand organization hierarchies and onboarding across AWS, Google Cloud, and Azure'
---
Cloud providers use organization-level hierarchies to group accounts, projects, or subscriptions and apply access and governance consistently. Prowler uses these hierarchies to discover cloud targets and help configure multi-account or multi-project scanning.
This guide explains the shared lifecycle and the differences between AWS Organizations, Google Cloud organizations, and Azure Management Groups. Use the provider-specific guides for commands, permissions, and limitations.
## Organization Lifecycle
Organization-level onboarding generally follows these steps:
1. **Identify the hierarchy:** Locate the organization, management account, management group, folder, organizational unit, or equivalent parent node in the cloud provider.
2. **Grant access:** Assign the provider permissions required to enumerate the hierarchy and read the resources that Prowler scans.
3. **Discover members:** Use Prowler to retrieve accounts, projects, or subscriptions under the selected hierarchy.
4. **Select scan targets:** Choose the cloud targets to connect or scan. Discovery does not necessarily make every discovered target a Prowler provider.
5. **Test access:** Confirm that Prowler can authenticate to each selected target and read its resources.
6. **Scan and maintain:** Run scans, review findings, and repeat discovery when the provider hierarchy changes.
<Note>
Organization membership changes are not automatically synchronized in every Prowler workflow. Follow the provider-specific guide to learn when manual rediscovery is required.
</Note>
## Capability Matrix
| Capability | AWS Organizations | Google Cloud organization | Azure Management Groups |
| --- | --- | --- | --- |
| Hierarchy members | AWS accounts grouped in organizational units (OUs) | Projects grouped in folders and nested folders | Subscriptions grouped in management groups |
| Organization-level discovery | Supported through AWS Organizations APIs | Supported through the Cloud Asset API | Supported through Azure management-group and subscription APIs |
| Primary scan target | AWS account | Google Cloud project | Azure subscription |
| Common organization-level permission | IAM role in the management or delegated administrator account | Cloud Asset Viewer or Cloud Asset Owner at the organization node | Appropriate Azure role assignment at the management-group or subscription scope |
| Provider-specific onboarding | AWS account discovery and optional StackSet role deployment | Project discovery under an organization ID | Subscription discovery under a management group; role assignments inherit to subscriptions |
| Membership maintenance | Repeat the discovery flow when accounts are added or removed | Re-run organization discovery when projects or folders change | Refresh discovery when subscriptions move between management groups |
## Provider Guides
### AWS Organizations
The [AWS Organizations guide](/user-guide/providers/aws/organizations) covers account details, delegated administration, IAM roles, CloudFormation StackSets, and CLI scanning. For Prowler Cloud onboarding, see [AWS Organizations in Prowler Cloud](/user-guide/tutorials/prowler-cloud-aws-organizations).
### Google Cloud Organization
The [Google Cloud organization guide](/user-guide/providers/gcp/organization) covers scanning projects under an organization ID, organization-level permissions, and Cloud Asset API requirements. For Prowler Cloud onboarding, see [Google Cloud organizations in Prowler Cloud](/user-guide/tutorials/prowler-cloud-gcp-organizations).
### Azure Management Groups
The [Azure Management Groups guide](/user-guide/providers/azure/management-groups) covers hierarchy setup, role assignment, subscription scope, and Azure-specific limitations. For Prowler Cloud onboarding, see [Azure Management Groups in Prowler Cloud](/user-guide/tutorials/prowler-cloud-azure-management-groups).
## Scope Boundaries
The organization concepts in this guide refer only to cloud-provider resource hierarchies:
- **GitHub organizations** group repositories and GitHub resources. They are a separate provider concept and are not part of AWS, Google Cloud, or Azure organization discovery.
- **MongoDB Atlas organizations** group Atlas projects and teams. They use a separate provider API and authentication model.
- **Prowler Cloud organizations** are internal tenants that isolate providers, scans, findings, users, and permissions. They are not the same as a cloud-provider organization and do not replace one.
Choose the guide that matches the hierarchy being configured, then use the relevant Prowler Cloud or CLI workflow for the scan targets.
@@ -144,25 +144,25 @@ prowler alibabacloud --ecs-ram-role RoleName
### Step 2: Run the First Scan
#### Scan All Regions
#### Scan all regions
```bash
prowler alibabacloud
```
#### Scan Specific Regions
#### Scan specific regions
```bash
prowler alibabacloud --region cn-hangzhou cn-shanghai
```
#### Run Specific Checks
#### Run specific checks
```bash
prowler alibabacloud --checks ram_no_root_access_key ram_user_mfa_enabled_console_access
```
#### Run a Compliance Framework
#### Run a compliance framework
```bash
prowler alibabacloud --compliance cis_2.0_alibabacloud
@@ -1,39 +1,14 @@
---
title: "Boto3 Retrier and Timeout Configuration in Prowler"
title: "Boto3 Retrier Configuration in Prowler"
---
import { VersionBadge } from "/snippets/version-badge.mdx"
Prowler's AWS Provider leverages Boto3's [Standard](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html) retry mode to automatically retry client calls to AWS services when encountering errors or exceptions.
## Timeout Configuration
<VersionBadge version="5.42.0" />
Every AWS API call is bounded by two timeouts:
- Connect timeout: seconds to wait to establish a connection (TCP, proxy tunnel and TLS handshake) to the AWS endpoint. Prowler's default is 10 seconds, configurable via `--aws-connect-timeout 5`.
- Read timeout: seconds to wait for a response once connected. Prowler's default is 60 seconds, configurable via `--aws-read-timeout 30`.
Both timeouts can also be set through environment variables, which is the way to tune them in Prowler Cloud and other deployments without a CLI:
```console
export PROWLER_AWS_BOTO3_CONNECT_TIMEOUT=5
export PROWLER_AWS_BOTO3_READ_TIMEOUT=30
```
CLI flags take precedence over the environment variables. Prowler sets both timeouts explicitly, so `AWS_DEFAULTS_MODE` and a `connect_timeout` in `~/.aws/config` are ignored; use the flag or the environment variable instead.
<Note>
Boto3 defaults both timeouts to 60 seconds. In networks with restricted egress (for example VPC endpoints for a subset of services, GovCloud or private deployments), every AWS service without a reachable endpoint used to cost up to 4 attempts × 60 seconds (the first call plus the 3 retries) for each region. Prowler lowers the connect timeout to 10 seconds so unreachable endpoints fail fast; lower it further together with `--aws-retries-max-attempts 0`, which disables retries and leaves a single attempt per call, if a scan still spends most of its time waiting on unreachable services.
</Note>
## Retry Behavior Overview
Boto3's Standard retry mode includes the following mechanisms:
- Maximum Retry Attempts: Default value set to 3, configurable via the `--aws-retries-max-attempts 5` argument. `0` disables retries.
- Maximum Retry Attempts: Default value set to 3, configurable via the `--aws-retries-max-attempts 5` argument.
- Expanded Error Handling: Retries occur for a comprehensive set of errors.
@@ -12,8 +12,6 @@ See [AWS Organizations](/user-guide/tutorials/prowler-cloud-aws-organizations) i
Prowler can integrate with AWS Organizations to manage the visibility and onboarding of accounts centrally.
For the cross-provider organization lifecycle and capability comparison, see [Organizations Across Cloud Providers](/user-guide/organizations).
When trusted access is enabled with the Organization, Prowler can discover accounts as they are created and even automate deployment of the Prowler Scan IAM Role.
> ℹ️ Trusted access can be enabled in the Management Account from the AWS Console under **AWS Organizations → Settings → Trusted access for AWS CloudFormation StackSets**.
@@ -167,7 +165,7 @@ Include the `ExternalId` parameter in the StackSet if required by the organizati
When encountering issues during deployment or needing to target specific OUs or environments (e.g., dev/staging/prod), reach out to the Prowler team via [Slack Community](https://prowler.com/slack) or [Support](mailto:support@prowler.com).
## Extra: Run Prowler Across All Accounts in AWS Organizations by Assuming Roles
## Extra: Run Prowler across all accounts in AWS Organizations by assuming roles
### Running Prowler Across All AWS Organization Accounts
@@ -21,28 +21,10 @@ When scanning the China (`aws-cn`), European Sovereign Cloud (`aws-eusc`) or Gov
- Specify the regions to audit within that partition using the `-f/--region` flag.
- Declare the partition with the `PROWLER_AWS_PARTITION` environment variable, set to `aws`, `aws-cn`, `aws-eusc` or `aws-us-gov`.
<Note>
Refer to: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html#configuring-credentials for more information about the AWS credential configuration.
</Note>
### Declaring the Partition
`PROWLER_AWS_PARTITION` tells Prowler which partition the scan runs against, without relying on a region being configured:
```bash
export PROWLER_AWS_PARTITION="aws-us-gov"
```
It matters most where nothing else says. Resolving an identity means calling STS before anything is known about the credentials, and with no region configured Prowler would otherwise start from the commercial endpoints. Declaring the partition makes that first call go to the right place, which is the difference between a scan that starts and one that fails on an endpoint the credentials cannot use.
A region configured for the session still wins when it belongs to the declared partition, so a deployment in `us-gov-west-1` is not sent to `us-gov-east-1`. A region belonging to a different partition is ignored, since a partition that has been declared explicitly is the more deliberate statement of the two.
<Note>
Set it wherever the scan runs. For deployments that scan from containers, that means the environment of the containers doing the scanning, not only the one accepting the request.
</Note>
### Scanning Specific Regions
To scan a particular AWS region with Prowler, use:
@@ -1,53 +0,0 @@
---
title: 'Azure Management Groups in Prowler'
---
Azure Management Groups provide a hierarchy above subscriptions. They allow Azure role assignments and governance policies to apply to multiple subscriptions through a shared scope.
For the cross-provider concepts and lifecycle, see [Organizations Across Cloud Providers](/user-guide/organizations).
## Azure Hierarchy
Azure resources are organized in the following order:
1. Tenant
2. Management groups
3. Subscriptions
4. Resource groups
5. Resources
Prowler scans Azure subscriptions. Management groups help organize those subscriptions and provide a scope where permissions can be assigned, but a management group is not itself a scan target.
## Create a Management Group
To create a management group, follow the [official Azure guide](https://learn.microsoft.com/en-us/azure/governance/management-groups/create-management-group-portal).
![Create management group](/images/create-management-group.gif)
After creating the management group, add the subscriptions that Prowler should access and scan.
![Add Subscription to Management Group](/images/add-sub-to-management-group.gif)
## Assign Roles
Assign the roles required by Prowler at the management-group scope instead of assigning them separately to every subscription. Role assignments at a management group can inherit to its child subscriptions, subject to Azure role-assignment and inheritance rules.
Use the [subscription scope permissions](/user-guide/providers/azure/authentication#subscription-scope-permissions) guide to identify the permissions required for scans. The identity used by Prowler must be able to read the management-group hierarchy and access each subscription selected for scanning.
## Subscription Scope
Management groups organize subscriptions, but Azure scan results remain scoped to individual subscriptions:
- Prowler Cloud scans one subscription per scan.
- Prowler CLI can scan multiple subscriptions by using the `--subscription-ids` option.
- A subscription must be accessible to the configured identity before Prowler can scan it.
- Moving a subscription between management groups can change the permissions it inherits and may require a connection test or rediscovery.
See [Azure Subscription Scope](/user-guide/providers/azure/subscriptions) for subscription selection and CLI options.
## Limitations
- Management groups do not replace subscription providers in Prowler.
- Azure role inheritance depends on the management-group hierarchy and the scope of each assignment; verify access on every subscription selected for scanning.
- The Prowler Cloud workflow is designed around Azure management-group discovery and subscription onboarding. The Prowler CLI workflow still requires explicit subscription selection when restricting scans.
- Changes to management-group membership or role assignments may not be reflected until the hierarchy is refreshed and access is tested again.
@@ -25,4 +25,14 @@ Check the [Authentication > Subscription Scope Permissions](/user-guide/provider
## Recommendation for Managing Multiple Subscriptions
Scanning multiple subscriptions requires creating and assigning roles for each, which can be a time-consuming process. To streamline subscription management and auditing, use [Azure Management Groups](/user-guide/providers/azure/management-groups) to organize subscriptions and assign permissions collectively.
Scanning multiple subscriptions requires creating and assigning roles for each, which can be a time-consuming process. To streamline subscription management and auditing, use management groups in Azure. This approach allows Prowler to efficiently organize and audit multiple subscriptions collectively.
1. **Create a Management Group**: Follow the [official guide](https://learn.microsoft.com/en-us/azure/governance/management-groups/create-management-group-portal) to create a new management group.
![Create management group](/images/create-management-group.gif)
2. **Assign Roles**: Assign necessary roles to the management group, similar to the [role assignment process](#assigning-permissions-for-subscription-scans).
Role assignment should be done at the management group level instead of per subscription.
3. **Add Subscriptions**: Add all subscriptions you want to audit to the newly created management group. ![Add Subscription to Management Group](/images/add-sub-to-management-group.gif)
@@ -4,8 +4,6 @@ title: 'Scanning a Specific GCP Organization'
By default, Prowler scans all Google Cloud projects accessible to the authenticated user.
For the cross-provider organization lifecycle and capability comparison, see [Organizations Across Cloud Providers](/user-guide/organizations).
To limit the scan to projects within a specific Google Cloud organization, use the `--organization-id` option with the GCP organization’s ID:
```console
@@ -42,7 +42,7 @@ Required for scanning repository security settings:
| Permission | Access Level | Purpose | Checks Enabled |
|------------|-------------|---------|----------------|
| **Administration** | Read | Branch protection, security and Actions settings | All branch protection checks, secret scanning status, `repository_default_workflow_permissions_read_only` |
| **Administration** | Read | Branch protection, security settings | All branch protection checks, secret scanning status |
| **Contents** | Read | File existence checks | `repository_public_has_securitymd_file`, `repository_has_codeowners_file` |
| **Metadata** | Read | Basic repository information | All checks (automatically granted) |
| **Dependabot alerts** | Read | Dependency vulnerability scanning | `repository_dependency_scanning_enabled` |
@@ -63,7 +63,7 @@ Required for scanning organization-level security settings:
| Permission | Access Level | Purpose | Checks Enabled |
|------------|-------------|---------|----------------|
| **Administration** | Read | Organization security policies and Actions settings | `organization_members_mfa_required`, `organization_repository_creation_limited`, `organization_default_repository_permission_strict`, `organization_default_workflow_permissions_read_only`, `organization_actions_pull_request_approval_disabled` |
| **Administration** | Read | Organization security policies | `organization_members_mfa_required`, `organization_repository_creation_limited`, `organization_default_repository_permission_strict` |
| **Members** | Read | Member access reviews | Organization membership auditing |
#### Account Permissions (Fine-Grained PAT only)
@@ -83,8 +83,8 @@ With the **Read-only permissions** listed above, Prowler can run:
| Check Category | Coverage | Notes |
|----------------|----------|-------|
| Branch protection checks (12 checks) | ✅ Full | Signed commits, status checks, PR reviews, etc. |
| Repository security checks | ✅ Full | Secret scanning, Dependabot, SECURITY.md, CODEOWNERS, default workflow permissions |
| Organization checks (5 checks) | ✅ Full | MFA, repo creation policies, default permissions, Actions workflow permissions |
| Repository security checks | ✅ Full | Secret scanning, Dependabot, SECURITY.md, CODEOWNERS |
| Organization checks (3 checks) | ✅ Full | MFA, repo creation policies, default permissions |
| Compliance frameworks | ✅ Full | CIS GitHub Benchmark and others |
| Merge settings (`delete_branch_on_merge`) | ⚠️ MANUAL | Requires write permission (see below) |
@@ -171,7 +171,6 @@ Use OAuth App Tokens when building applications that need delegated user permiss
- `repo`: Full control of repositories
- `read:org`: Read organization and team membership
- `admin:org`: Required by `organization_default_workflow_permissions_read_only` and `organization_actions_pull_request_approval_disabled` to read the organization Actions workflow permissions
- `read:user`: Read user profile data
**Create an OAuth App:**
@@ -215,7 +214,7 @@ If a GitHub App is required:
| Permission | Access Level | Purpose | Checks Enabled |
|------------|-------------|---------|----------------|
| **Administration** | Read | Branch protection, security and Actions settings | All branch protection checks, `repository_secret_scanning_enabled`, `repository_default_workflow_permissions_read_only` |
| **Administration** | Read | Branch protection, security settings | All branch protection checks, `repository_secret_scanning_enabled` |
| **Contents** | Read | File existence checks | `repository_public_has_securitymd_file`, `repository_has_codeowners_file` |
| **Metadata** | Read | Basic repository information | All checks (automatically granted) |
| **Dependabot alerts** | Read | Dependency vulnerability scanning | `repository_dependency_scanning_enabled` |
@@ -224,7 +223,7 @@ If a GitHub App is required:
| Permission | Access Level | Purpose | Checks Enabled |
|------------|-------------|---------|----------------|
| **Administration** | Read | Organization security policies and Actions settings | `organization_members_mfa_required`, `organization_repository_creation_limited`, `organization_default_repository_permission_strict`, `organization_default_workflow_permissions_read_only`, `organization_actions_pull_request_approval_disabled` |
| **Administration** | Read | Organization security policies | `organization_members_mfa_required`, `organization_repository_creation_limited`, `organization_default_repository_permission_strict` |
| **Members** | Read | Member access reviews | Organization membership auditing |
**Create a GitHub App:**
@@ -96,29 +96,6 @@ Install Trivy using one of the following methods:
For additional installation methods, see the [Trivy installation guide](https://trivy.dev/latest/getting-started/installation/).
### Vulnerability Database Cache
<VersionBadge version="5.42.0" />
Trivy keeps its vulnerability database in a cache directory. By default Prowler gives it a temporary one and removes it when the scan ends, so the database is downloaded again for every scan.
Set `TRIVY_CACHE_DIR` to a directory that persists and the database is downloaded once and reused:
```bash
export TRIVY_CACHE_DIR="$HOME/.cache/trivy"
prowler image --image <image>
```
Prowler never deletes a directory you supply. Trivy still creates and updates its cache and database files inside it.
<Note>
A host with no internet access needs a pre-populated vulnerability database in a persistent directory, with `TRIVY_CACHE_DIR` pointing at it. Populate the directory on a machine that does have access and copy it across.
Trivy tries to refresh the database when it considers it stale, and that download fails without network access. Set `TRIVY_SKIP_DB_UPDATE=true` (and `TRIVY_SKIP_JAVA_DB_UPDATE=true` if Java scanning is enabled) so it uses the supplied database as is.
The database ages. A scan run against an old one reports only the vulnerabilities known when it was built, and nothing in the output says so, so keep track of when it was last refreshed.
</Note>
### Supported Scanners
@@ -329,21 +306,9 @@ prowler image --registry internal-registry.local --registry-insecure
```
<Warning>
Skipping TLS verification disables certificate validation for registry connections, including the Trivy image pull (`TRIVY_INSECURE`). Use this flag only for trusted internal registries with self-signed certificates.
Skipping TLS verification disables certificate validation for registry connections. Use this flag only for trusted internal registries with self-signed certificates.
</Warning>
#### On-Premises Registries and Private Networks
<VersionBadge version="5.41.0" />
By default, Prowler rejects registry-provided URLs (token endpoints, pagination links) that resolve to non-public addresses, as an SSRF defense. On-premises registries live on private networks by definition, so to scan them declare the trusted ranges explicitly:
```bash
export PROWLER_IMAGE_PROVIDER_ALLOWED_PRIVATE_NETWORKS="192.168.65.254/32,10.20.0.0/16"
```
The value is a comma-separated list of IPs and CIDRs. A resolved address inside an allowlisted range is permitted; every other non-public address stays blocked, so link-local (`169.254.169.254`), loopback, and the rest of the internal network remain protected. The variable applies to registry enumeration and to the connection test. Malformed entries fail at startup, and a non-empty allowlist is logged as a relaxed security control. When unset, behavior is unchanged: only public addresses are followed.
#### Supported Registries
Registry Scan Mode supports the following registry types:
@@ -36,7 +36,7 @@ If **Require IP Access List for the Atlas Administration API** is enabled in the
<VersionBadge version="5.15.0" />
### Step 1: Add the Provider
### Step 1: Add the provider
1. Navigate to **Providers** and click **Add Provider**.
![Add provider list](./img/add-provider-list.png)
@@ -45,13 +45,13 @@ If **Require IP Access List for the Atlas Administration API** is enabled in the
![Add organization ID](./img/add-org-id.png)
4. (Optional) Add a friendly alias to identify this organization in dashboards.
### Step 2: Provide API Credentials
### Step 2: Provide API credentials
1. Click **Next** to open the credentials form.
2. Paste the **Atlas Public Key** and **Atlas Private Key** generated in the Atlas console.
![Add credentials](./img/add-credentials.png)
### Step 3: Test the Connection and Start Scanning
### Step 3: Test the connection and start scanning
1. Click **Test connection** to ensure Prowler Cloud can reach the Atlas API.
2. Save the credentials. The provider will appear in the list with its current connection status.
@@ -66,11 +66,11 @@ If **Require IP Access List for the Atlas Administration API** is enabled in the
You can also run MongoDB Atlas assessments directly from the CLI. Both command-line flags and environment variables are supported.
### Step 1: Select an Authentication Method
### Step 1: Select an authentication method
Choose one of the following authentication methods:
#### Command-Line Arguments
#### Command-line arguments
```bash
prowler mongodbatlas \
@@ -78,7 +78,7 @@ prowler mongodbatlas \
--atlas-private-key <private_key>
```
#### Environment Variables
#### Environment variables
```bash
export ATLAS_PUBLIC_KEY=<public_key>
@@ -86,9 +86,9 @@ export ATLAS_PRIVATE_KEY=<private_key>
prowler mongodbatlas
```
### Step 2: Run the First Scan
### Step 2: Run the first scan
#### Scan All Projects and Clusters
#### Scan all projects and clusters
```bash
prowler mongodbatlas
@@ -96,7 +96,7 @@ prowler mongodbatlas
This command enumerates all projects accessible to the API key and scans every cluster.
#### Scan a Specific Project
#### Scan a specific project
Add the `--atlas-project-id` flag when you only want to assess one project:
@@ -104,7 +104,7 @@ Add the `--atlas-project-id` flag when you only want to assess one project:
prowler mongodbatlas --atlas-project-id <project-id>
```
### Additional Tips
### Additional tips
- Combine flags (for example, `--checks` or `--services`) just like with other providers.
- Use `--output-modes` to export findings in JSON, CSV, ASFF, etc.
@@ -67,7 +67,7 @@ The service application must be assigned **one** of the following Okta admin rol
Okta's Management API enforces a two-layer authorization model: an OAuth **scope** decides which API endpoints the token can call, and an **admin role** decides whether the call returns data. With only a scope granted, the token mint succeeds but every read returns `403 Forbidden`. Read-Only Administrator is the minimum role that lets the granted `okta.*.read` scopes return configuration data to Prowler's checks; without it, the credential probe at provider startup fails and the scan never gets to evaluate any check.
#### When Super Administrator Is Required
#### When Super Administrator is required
Four checks need to resolve the Authentication Policy bound to Okta's first-party apps (Okta Admin Console, Okta Dashboard) and depend on `/api/v1/apps` returning those system apps — which Okta restricts to Super Administrator:
@@ -92,17 +92,17 @@ Read-Only Administrator stays the recommended default for the least-privilege fr
## Step-by-Step Setup
### 1. Go to the Admin Console
### 1. Go to the admin console
![Okta — admin console page](/user-guide/providers/okta/images/select-admin-console.png)
### 2. [Optional] - Disable the Privilege-Escalation Bypass (Org-Wide, One-Time)
### 2. [Optional] - Disable the privilege-escalation bypass (org-wide, one-time)
In the Okta Admin Console, go to **Settings → Account → Public client app admins** and ensure it is **off**. When enabled, every API Services app can be auto-assigned the Super Administrator role after scopes are granted, which would invalidate the read-only premise of this integration.
![Okta — disable Public client app admins](/user-guide/providers/okta/images/public-client-app-admins.png)
### 3. Create the API Services App
### 3. Create the API Services app
1. Go to **Applications → Applications**.
@@ -118,7 +118,7 @@ In the Okta Admin Console, go to **Settings → Account → Public client app ad
![Okta — copy client id](/user-guide/providers/okta/images/copy-client-id.png)
### 4. Switch to Private-Key Authentication and Generate a Keypair
### 4. Switch to private-key authentication and generate a keypair
On the new app's **General** tab, scroll to **Client Credentials**:
@@ -136,13 +136,13 @@ Okta displays the private key **only once**. If you close the modal without copy
![Okta — create Public Key](/user-guide/providers/okta/images/create-public-key.png)
### 5. Grant the Required OAuth Scopes
### 5. Grant the required OAuth scopes
On the app, open the **Okta API Scopes** tab and click **Grant** on every scope Prowler needs. The bundled checks require `okta.policies.read`, `okta.brands.read`, `okta.apps.read`, `okta.authenticators.read`, `okta.networkZones.read`, `okta.apiTokens.read`, `okta.roles.read`, `okta.groups.read`, `okta.logStreams.read`, and `okta.idps.read`.
![Okta — grant OAuth scopes](/user-guide/providers/okta/images/grant-permissions.png)
### 6. Assign an Admin Role
### 6. Assign an admin role
On the app, open the **Admin roles** tab and click **Edit assignments → Add assignment**:
@@ -155,7 +155,7 @@ To additionally evaluate the first-party application checks (Okta Admin Console
![Okta — grant Read-Only role](/user-guide/providers/okta/images/grant-roles.png)
### 7. [Optional] Verify DPoP Setting
### 7. [Optional] Verify DPoP setting
Prowler sends DPoP (Demonstrating Proof of Possession) proofs on every token request. The integration works whether the **Require Demonstrating Proof of Possession (DPoP) header in token requests** setting on the service app is on or off — but enabling it is the more secure default.
@@ -206,20 +206,20 @@ The org domain must be `<org>.okta.com` (or `.oktapreview.com` / `.okta-emea.com
The file at `OKTA_PRIVATE_KEY_FILE` is missing, unreadable, or empty. Confirm the path and that the file contains a non-empty PEM block or JWK JSON document.
### `OktaInvalidCredentialsError` at Provider Init
### `OktaInvalidCredentialsError` at provider init
Prowler validates credentials at startup by listing one sign-on policy. This error indicates the credential material itself was rejected:
- **`invalid_client`** — the public key registered in Okta does not match the private key on disk. Generate a fresh keypair and try again.
### `OktaInsufficientPermissionsError` at Provider Init
### `OktaInsufficientPermissionsError` at provider init
Raised when the credential probe succeeds at the OAuth layer but the request is rejected because the service app lacks the required scope or admin role:
- **`invalid_scope`** — one of the requested scopes (`okta.policies.read`, `okta.brands.read`, `okta.apps.read`, `okta.authenticators.read`, `okta.networkZones.read`, `okta.apiTokens.read`, `okta.roles.read`, `okta.groups.read`, `okta.logStreams.read`, and `okta.idps.read`) is not granted on the service app. Grant the missing scope from **Okta API Scopes**.
- **`Forbidden` / `not authorized`** — no admin role is assigned to the service app. Assign **Read-Only Administrator** (or **Super Administrator** for the first-party application checks) from **Admin roles**.
### Application-Service Checks Return MANUAL on First-Party Apps
### Application-service checks return MANUAL on first-party apps
When the service app runs with Read-Only Administrator, the five application-service checks targeting the Okta Admin Console and Okta Dashboard return MANUAL. This is by design — Okta restricts the underlying endpoints (`/api/v1/first-party-app-settings/{appName}` and `/api/v1/apps` for first-party app `name` values `saasure` / `okta_enduser`) to **Super Administrator**. Assign the Super Administrator role to the service app to evaluate those checks. See [Required Admin Role](#required-admin-role) for the full list.
@@ -4,8 +4,6 @@ title: 'AWS Organizations Bulk Provisioning in Prowler'
Prowler offers an automated tool to discover and provision all AWS accounts within an AWS Organization. This streamlines onboarding for organizations managing multiple AWS accounts by automatically generating the configuration needed for bulk provisioning.
For the cross-provider organization lifecycle and terminology, see [Organizations Across Cloud Providers](/user-guide/organizations).
The tool, `aws_org_generator.py`‎, complements the [Bulk Provider Provisioning](./bulk-provider-provisioning) tool and is available in the Prowler repository at: [util/prowler-bulk-provisioning](https://github.com/prowler-cloud/prowler/tree/master/util/prowler-bulk-provisioning)
<Note>
+6 -46
View File
@@ -1,7 +1,7 @@
---
title: 'Alerts'
sidebarTitle: 'Alerts'
description: 'Create alerts from Prowler Cloud findings, deliver them to email recipients and Slack channels, and monitor relevant security changes after scans or in daily digests.'
description: 'Create email alerts from Prowler Cloud findings to monitor relevant security changes after scans or in daily digests.'
---
import { VersionBadge } from "/snippets/version-badge.mdx"
@@ -9,7 +9,7 @@ import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
<VersionBadge version="5.26.0" />
Alerts notify their destinations — email recipients, Slack channels, or both — when security findings match saved filter conditions. Use Alerts to track high-priority findings, monitor specific providers or services, and keep teams informed about scan results that match defined criteria.
Alerts notify recipients by email when security findings match saved filter conditions. Use Alerts to track high-priority findings, monitor specific providers or services, and keep teams informed about scan results that match defined criteria.
<SubscriptionBanner />
@@ -19,13 +19,12 @@ Before creating Alerts, ensure that:
* At least one scan has completed and produced findings.
* The user role includes the `manage_alerts` permission.
* To deliver Alerts to Slack channels, a Slack workspace is connected, at least one channel is authorized on it, and the integration's connection check has confirmed that channel. See [Slack Integration](/user-guide/tutorials/prowler-app-slack-integration).
The `manage_alerts` permission is required to create, edit, test, enable, disable, and delete Alerts. See [RBAC Administrative Permissions](/user-guide/tutorials/prowler-app-rbac#rbac-administrative-permissions) for details.
## How Alerts Work
Alerts are created from Findings filters. When an Alert runs, Prowler Cloud evaluates the saved conditions against findings and notifies the Alert's destinations when matching findings exist: an email digest to each recipient, a message to each Slack channel, or both. Destination kinds are independent — neither requires the other, and neither displaces the other.
Alerts are created from Findings filters. When an Alert runs, Prowler Cloud evaluates the saved conditions against findings and sends an email digest when matching findings exist.
<Note>
Alerts evaluate findings with status `FAIL` only. Findings with status `PASS` or `MANUAL`, and muted findings, never trigger an Alert regardless of the saved filters.
@@ -54,7 +53,6 @@ To create an Alert:
* **Description:** Add optional context for the Alert.
* **Frequency:** Select when Prowler Cloud should evaluate the Alert.
* **Recipients:** Select the recipients who should receive the email digest.
* **Destination channels:** Select the Slack channels that should receive the Alert. See [Slack Channel Destinations](#slack-channel-destinations).
![Create Alert Modal](/images/prowler-app/alerts/create-alert-modal.png)
@@ -88,18 +86,11 @@ Navigate to **Alerts** to review and manage existing Alerts.
![Alerts List](/images/prowler-app/alerts/alerts-list.png)
The **Destinations** column summarizes where each Alert delivers, without the Alert being opened:
* **Email recipients:** The first address, plus a count of the rest, such as `security@example.com +2 more`.
* **Slack channels:** The first channel, plus a count of the rest, such as `#sec-alerts +1 more`.
Each summary is omitted when that destination kind is empty, and the column reads **No destinations** when an Alert has neither.
Each Alert provides these actions:
| Action | Description |
|--------|-------------|
| Edit | Update name, description, recipients, Slack channels, frequency, or filters. |
| Edit | Update name, description, recipients, frequency, or filters. |
| Enable/Disable | Start or stop Alert evaluation without deleting the Alert. |
| Delete | Permanently remove the Alert. |
@@ -134,37 +125,6 @@ By default, the **organization owner** receives a **daily digest** for **critica
If a recipient unsubscribes from Alerts, that address stops receiving digests until it is reconfirmed.
An Alert does not require email recipients: an Alert that targets Slack channels only is accepted with those channels as its sole destinations. An Alert with no destinations at all stays valid and keeps evaluating its filters, but it delivers nothing.
## Slack Channel Destinations
<VersionBadge version="5.40.0" />
An Alert can post to Slack channels alongside its email recipients, or instead of them. The **Destination channels** field sits directly below **Recipients** in the Alert form, both when creating an Alert and when editing one. When the Alert matches findings, Prowler Cloud posts a message to each of its channels and sends the email digest to each of its recipients, independently of each other.
The channels offered are the confirmed channels of the connected Slack integration, never the whole Slack workspace. Widening the pool takes two steps on the integration: authorize the channel there, then run its connection check, which confirms the channel by posting a one-time confirmation message to it. Once confirmed, the channel is selectable on every Alert. See [Slack Integration](/user-guide/tutorials/prowler-app-slack-integration) for connecting a workspace, authorizing its channels, and confirming them.
A channel that was authorized a moment ago but does not appear in the Alert form has not been confirmed yet. Run **Test connection** on the Slack integration, then reopen the Alert form.
Private channels are identified as **Private** both in the open channel list and on the selected channels once the list is closed, so a private destination is never mistaken for a public one.
### When Slack Channels Cannot Be Selected
The field is always present, so channel delivery is never silently missing. It reports why it cannot be used:
| State | What the Alert form shows |
|-------|---------------------------|
| No Slack workspace connected | The field is visible but cannot be edited, explaining that posting Alerts to Slack channels needs a connected Slack workspace, with a link to the Slack integration. |
| Workspace connected, no confirmed channels | A notice that no channels are available yet and that they are authorized and confirmed on the Slack integration, with the same link. |
In both states the rest of the Alert is unaffected: it can still be created or saved with its filters, frequency, and email recipients.
<Note>
Slack destinations stay in step with the integration. Removing a channel from the integration's authorized set — or disconnecting the Slack integration altogether — removes that channel from every Alert that targeted it, so an Alert never keeps a destination Prowler can no longer deliver to. The Alert keeps its filters, frequency, and email recipients, and future delivery to that channel stops: nothing is posted to announce the removal, and the notifications already delivered stay in the channel. Restoring delivery means authorizing and confirming the channel again on the integration, then selecting it again on the Alert.
</Note>
Saving an Alert that names a channel which is not a confirmed channel of a connected Slack integration is refused, and the reason is reported on the Alert form. Authorize and confirm the channel on the Slack integration, or remove it from the Alert, and save again.
## Email Notifications
When an Alert matches findings, Prowler Cloud sends a security alert email that summarizes the matching findings. The email includes:
@@ -181,6 +141,6 @@ When an Alert matches findings, Prowler Cloud sends a security alert email that
* **Start with focused filters:** Create Alerts for specific high-priority scopes, such as critical findings, production providers, or important services.
* **Use clear names:** Choose names that explain the intent of the Alert.
* **Review destinations regularly:** Keep recipient lists and channel selections aligned with current ownership.
* **Review recipients regularly:** Keep recipient lists aligned with current ownership.
* **Test before saving edits:** Use **Test** after changing filters to confirm that the Alert matches the expected findings.
* **Disable instead of deleting during tuning:** Disable Alerts temporarily when adjusting filters or destinations.
* **Disable instead of deleting during tuning:** Disable Alerts temporarily when adjusting filters or recipients.
@@ -141,18 +141,18 @@ Muting a finding does not fix the underlying configuration. Review the finding b
## Troubleshooting
### Triage Controls Do Not Appear
### Triage controls do not appear
Make sure the row is an individual finding row. Finding Groups rows do not show triage controls. Expand a group to see affected resources and their triage controls.
### Changes Cannot Be Saved
### Changes cannot be saved
Confirm that the user role has **Manage Scans** permission. Prowler Local Server does not support Findings Triage writes.
### Resolved or Reopened Is Missing from the Selector
### Resolved or Reopened is missing from the selector
**Reopened** is always automatic. **Resolved** is set automatically from scan result changes and appears as a selector option only on `MANUAL` findings, where it records a [Manual Pass](#verify-a-manual-finding-as-pass). On findings with any other status, this is expected.
### Risk Accepted or False Positive Muted a Finding
### Risk Accepted or False Positive muted a finding
This is expected. Those statuses create a mute rule through Mutelist.
@@ -28,7 +28,7 @@ Source: [`prowler-cloud/prowler`](https://github.com/prowler-cloud/prowler) · M
## Usage
### AWS Scan
### AWS scan
```yaml
- uses: prowler-cloud/prowler@5.25
@@ -41,7 +41,7 @@ Source: [`prowler-cloud/prowler`](https://github.com/prowler-cloud/prowler) · M
AWS_SESSION_TOKEN: ${{ secrets.AWS_SESSION_TOKEN }}
```
### Push Findings to Prowler Cloud
### Push findings to Prowler Cloud
Send scan results directly to [Prowler Cloud](/user-guide/tutorials/prowler-import-findings) for centralized visibility, compliance tracking, and team collaboration.
@@ -97,7 +97,7 @@ jobs:
- GitHub Code Scanning is free for public repositories. Private repositories require a [GitHub Code Security](https://docs.github.com/en/get-started/learning-about-github/about-github-advanced-security) license.
</Warning>
### Combine Push-to-Cloud with SARIF Upload
### Combine push-to-cloud with SARIF upload
```yaml
- uses: prowler-cloud/prowler@5.25
@@ -114,7 +114,7 @@ jobs:
PROWLER_CLOUD_API_KEY: ${{ secrets.PROWLER_CLOUD_API_KEY }}
```
### Scan the Current Repository with the GitHub Provider
### Scan the current repository with the GitHub provider
```yaml
name: Prowler GitHub Scan
@@ -142,7 +142,7 @@ jobs:
`--repository` scans a single repo. Use `--organization <name>` instead to include org-level checks (MFA, security policies, etc.). See the [GitHub provider authentication](/user-guide/providers/github/authentication) for required token permissions.
</Info>
### Fail the PR on Findings
### Fail the PR on findings
By default the action tolerates findings (exit code 3) and succeeds. Set `fail-on-findings: true` to fail the workflow step when Prowler detects findings. Combine with `--severity` to control which severity levels trigger the failure:
@@ -258,7 +258,7 @@ Scan results are written to `output/` in the workspace and uploaded as artifacts
When `upload-sarif` is enabled, SARIF results are also uploaded to GitHub Code Scanning and appear on the repository's **Security → Code scanning** tab, filtered by the branch that ran the scan.
### Step Summary
### Step summary
The action writes a summary to the run page with a per-severity breakdown of failing checks, artifact and Code Scanning links, and (when `push-to-cloud: false`) a pointer to [Prowler Cloud](https://cloud.prowler.com) for continuous monitoring.
@@ -126,15 +126,22 @@ To manually send individual Findings to Jira:
### Finding Reference in the Jira Issue
<VersionBadge version="5.41.0" />
Every Jira issue created from a single Finding carries a stable reference back to that Finding, so issues can be filtered, searched with Jira Query Language (JQL), or matched by automation:
* **Labels**: `prowler`, `prowler-<provider>`, `prowler-<severity>`, `prowler-<check-id>` and `prowler-finding-<finding-uid>`. Labels are sanitized deterministically: whitespace becomes `_`, control characters are removed, and values are truncated to Jira's 255-character label limit.
* **Finding URL**: a link that opens the Finding in Prowler, filtered by its unique identifier (UID) so it keeps working after later scans.
* **Finding URL**: a link that opens the Finding in Prowler App, filtered by its unique identifier (UID) so it keeps working after later scans.
* **Tenant Info**: the name of the Prowler organization that sent the Finding.
Prowler Cloud always includes the Finding URL. In Prowler Local Server, set `DJANGO_UI_BASE_URL` in the API environment (for example, `https://prowler.example.com`) to enable it. When the variable is empty, the issue is created without the link.
Prowler Cloud always includes the Finding URL. In a self-hosted Prowler App, set `DJANGO_UI_BASE_URL` in the API environment (for example, `https://prowler.example.com`) to enable it. When the variable is empty, the issue is created without the link.
### Sending a Finding That Already Has a Jira Issue
Prowler remembers the Jira issue created for each Finding, keyed by the Finding UID, so sending the same Finding again does not create a duplicate issue:
* If the linked issue is still open in Jira, the Finding is skipped and the existing issue key is reported in the task result (`skipped_count`, `skipped`).
* If the linked issue is closed (any Jira status in the **Done** category) or was deleted, a new issue is created and becomes the linked issue for that Finding.
The link, and the last status observed in Jira, are available through the API at `GET /api/v1/jira-issues` (filter by `finding_uid`, `finding_uid__in`, `provider_id`, `integration` or `issue_key`). Each Jira integration keeps its own links, so the same Finding can have one issue per integration.
## Integration Status
@@ -171,13 +178,13 @@ Support for custom field mapping is planned for a future release.
## Troubleshooting
### Connection Test Fails
### Connection test fails
* Verify Jira instance domain is correct and accessible
* Confirm API token or credentials are valid
* Ensure API access is enabled in Jira settings and the needed scopes are granted
### Check Task Status (API)
### Check task status (API)
If the Jira issue does not appear in your Jira project, follow these steps to verify the export task status via the API.
@@ -257,11 +257,11 @@ The **Scope** column indicates where each permission applies. **All** means the
</Note>
To grant all administrative permissions, select the **Grant all admin permissions** option.
### Prowler Cloud Exclusive Permissions
### Prowler Cloud exclusive permissions
The following permissions are available exclusively in **Prowler Cloud**:
**Manage Ingestions:** Submit and manage findings ingestion jobs. Required to upload OCSF scan results from the Scans page, with the `--push-to-cloud` CLI flag or through the ingestion endpoints. See [Import Findings](/user-guide/tutorials/prowler-import-findings) for details.
**Manage Ingestions:** Submit and manage findings ingestion jobs via the API. Required to upload OCSF scan results using the `--push-to-cloud` CLI flag or the ingestion endpoints. See [Import Findings](/user-guide/tutorials/prowler-import-findings) for details.
**Manage Billing:** Access and manage billing settings, subscription plans, and payment methods.
@@ -1,197 +0,0 @@
---
title: "Slack Integration"
sidebarTitle: 'Slack'
description: 'Connect a Slack workspace to Prowler Cloud or Prowler Private Cloud, authorize the channels Prowler posts to, and verify the connection.'
---
import { VersionBadge } from "/snippets/version-badge.mdx"
import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
<VersionBadge version="5.40.0" />
<SubscriptionBanner />
Prowler Cloud and Prowler Private Cloud connect to a Slack workspace so security updates arrive where teams already work. Connecting takes one approval in Slack — there is no bot token to create, copy, or store by hand — and Prowler records the set of channels it is authorized to post to.
Integrating Prowler Cloud or Prowler Private Cloud with Slack provides:
* **Approval-based setup:** Approve Prowler once in Slack instead of building a Slack app and pasting a token.
* **Confirmed destinations:** The connection check verifies every authorized channel and confirms each new one in the channel itself, so a channel Prowler cannot reach is reported before anything depends on it.
* **Controlled reach:** Prowler posts only to the channels authorized on the integration, and private channels stay invisible until the Prowler app is invited to them.
<Note>
This guide covers the Slack integration in Prowler Cloud and Prowler Private Cloud. It is unrelated to the Prowler CLI `--slack` flag, which posts a scan summary from the command line using a self-created Slack app and the `SLACK_API_TOKEN` and `SLACK_CHANNEL_NAME` environment variables — see [CLI Integrations](/user-guide/cli/tutorials/integrations) for that feature.
</Note>
## How the Slack Integration Works
When connected and configured:
1. A Slack workspace is approved once through Slack's app install flow, and Prowler stores the resulting credential encrypted.
2. Prowler reads the channels it can post to: the workspace's public channels, plus the private channels the Prowler app has been invited to.
3. Several of those channels are selected and saved as the integration's authorized channels.
4. The connection check verifies the credential and every authorized channel, and posts a one-time confirmation message to each channel it has not confirmed yet.
5. Features that deliver to Slack, such as [Alerts](/user-guide/tutorials/prowler-alerts), choose their destinations from the confirmed channels.
6. Disconnecting removes the integration from Prowler and attempts to revoke Prowler's access at Slack.
## Prerequisites
The Slack integration is available only in **Prowler Cloud** and **Prowler Private Cloud**. Prowler Local Server does not serve the Slack endpoints at all, so the Slack card does not appear on the Integrations page and the management page redirects away.
Configuring and using the Slack integration requires the **Manage Integrations** permission. The integration is tenant-wide, so it does not require **Unlimited Visibility** or any specific Provider Group.
One Slack workspace connects per tenant. Approving Prowler again in the same workspace refreshes the stored credential and keeps the authorized channels, but it resets their confirmations and the connection state — the connection check has to be run again. Approving Prowler in a *different* workspace is refused until the current workspace is disconnected: a workspace is never swapped out silently.
## Permissions Prowler Requests in Slack
Slack shows a consent screen listing everything the Prowler app asks for. Prowler requests exactly four bot scopes:
| Scope | Why Prowler Requests It |
|-------|-------------------------|
| `chat:write` | Post the confirmation message, and any later notification, to the authorized channels. |
| `chat:write.public` | Post to a public channel without first inviting the Prowler app to it. |
| `channels:read` | List public channels for the channel selection and resolve the chosen ones. |
| `groups:read` | List the private channels the Prowler app has been invited to, so they appear in the channel selection. |
Two of these read more broadly than they behave, and both are worth understanding before approving the app.
### What `chat:write.public` Does Not Grant
On the consent screen, `chat:write.public` reads as permission to post in any public channel. Prowler never uses it that way: **Prowler only ever posts to the channels authorized on the integration.** The scope exists so that authorizing a public channel does not also require someone to invite the Prowler app to it first.
{/* The Prowler UI deep-links to this heading's anchor, so rewording the heading breaks that link. */}
### Why a Private Channel Is Missing From the Channel List
`groups:read` reveals only the private channels the Prowler app is already a member of. A private channel therefore appears in the channel list only after someone invites `@Prowler Cloud` to it in Slack:
```text
/invite @Prowler Cloud
```
That invite is issued in Slack, by that channel's own members, and **the invite itself is the permission grant** — no scope bypasses it. Prowler ships no in-product flow to get the app invited, because the decision belongs to the channel's members. After inviting the app, click **Refresh channels** to re-read the list.
## Connecting a Slack Workspace
To connect a Slack workspace to Prowler Cloud or Prowler Private Cloud:
1. In either product, navigate to **Integrations**.
2. Locate the **Slack** card and click **Manage**.
![Slack card on the Integrations page in Prowler Cloud or Prowler Private Cloud](/images/prowler-app/slack/integrations-tab.png)
3. Click **Add to Slack**.
![Slack management page before a workspace is connected, showing the Add to Slack action](/images/prowler-app/slack/no-workspace-connected.png)
4. In Slack, select the workspace to connect and approve the permissions listed on the consent screen.
5. Slack returns to Prowler Cloud or Prowler Private Cloud, which completes the install and shows the connected workspace.
![Connected Slack workspace with no channels authorized yet](/images/prowler-app/slack/connected-workspace.png)
The connected card reports the workspace name and a **Not checked yet** status: the connection is checked against the authorized channels, and none are authorized at this point. Authorizing them is the next step. Once at least one channel is authorized, **Test connection** verifies the credential and every authorized channel, and confirms the ones not confirmed yet.
<Note>
Declining the consent screen creates nothing. Prowler reports that the workspace was not connected and offers to start again.
</Note>
## Authorizing Destination Channels
Prowler posts to the channels authorized on the integration. Several channels can be authorized at once, and once the connection check has confirmed them they are the pool every consumer of the integration draws from: an [Alert](/user-guide/tutorials/prowler-alerts) picks its Slack destinations from the confirmed channels, never from the whole workspace.
1. Open the **Destination channels** selection. It lists the workspace's public channels, plus the private channels the Prowler app has been invited to, each marked **Private**.
![Destination channels selection listing public channels and an invited private channel marked Private](/images/prowler-app/slack/channel-picker.png)
2. Select one or more channels. A selected private channel keeps the same **Private** marking with the list closed, so the authorized set stays readable at a glance.
3. Click **Save channels**.
Prowler validates the selection against Slack and derives each channel name itself, so a recorded name can never drift from the channel it belongs to. Once the set is saved, the page reports where Prowler posts and runs the connection check over it.
If the selection reports that no channels are available, the workspace exposes nothing Prowler can see. Create a public channel, or invite `@Prowler Cloud` to a private one, then click **Refresh channels**.
A workspace can hold more channels than Prowler reads in one go. When that happens, the selection says so and lists what was read: every listed channel is usable, and a channel missing from a partial list is not necessarily one `@Prowler Cloud` has to be invited to. Only listed channels can be selected: **Refresh channels** repeats the same bounded read rather than reading further, and the selection's search filters what was already read, so neither surfaces a channel the read left out.
Saving a new selection replaces the authorized set: channels left out of it stop being authorized, and channels added to it are authorized but not yet confirmed. Changing which channels are in the set also resets the integration's connection state, so the check runs again over the new set — reordering the same channels does not. Saving an empty selection leaves the integration with no authorized channels, and **Test connection** cannot be run again until at least one channel is authorized.
<Warning>
Removing a channel from the authorized set also removes it from every Alert that targeted it. Those Alerts keep their filters, frequency, and email recipients, and future delivery to that channel simply stops: nothing is posted to announce the removal, and the notifications already delivered stay in the channel. Disconnecting the integration has the same effect on every channel it had authorized. Restoring delivery means authorizing and confirming the channel again here, then selecting it again on each Alert.
</Warning>
### Confirming the Authorized Channels
A channel becomes usable as a destination once the connection check has confirmed it. Click **Test connection**: it verifies the stored credential and every authorized channel, and posts a one-time message to each channel it has not confirmed yet.
```text
✅ Prowler connection verified. Notifications will be delivered to this channel.
```
Later checks never post that message again to a channel that is already confirmed, so it arrives once per channel. The integration reports as connected only when every check and every required confirmation succeeded; a failure names the channel that failed. The check needs at least one authorized channel — with none authorized, it cannot be run yet.
Confirmation is what makes a channel selectable elsewhere in Prowler Cloud. A channel authorized a moment ago is missing from an Alert's channel list until a connection check confirms it.
## Disconnecting a Slack Workspace
Disconnecting removes the integration from Prowler **and** attempts to revoke Prowler's access at Slack.
1. On the Slack management page, click **Disconnect**.
2. Review the confirmation, then click **Disconnect workspace**.
![Disconnect Slack workspace confirmation dialog](/images/prowler-app/slack/disconnect-confirmation.png)
The page returns to its unconnected state, ready for a new install.
### What Revocation Means
Revocation is attempted at Slack, and it is best-effort:
* **Revocation succeeded:** The stored credential no longer grants Prowler anything, and the integration is gone from Prowler.
* **Revocation failed:** The integration and the stored credential are gone from Prowler either way, so there is nothing to retry. Slack did not confirm the revocation, which means the Prowler app may still be installed in the workspace. Remove it from that workspace's Slack app settings.
* **Revocation unreported:** Slack's answer carried no outcome either way. The integration is gone from Prowler, and the disconnect is reported without any claim about revocation. When certainty matters, check the workspace's Slack app settings and remove the Prowler app if it is still installed.
Prowler reports the outcome it received: a failed revocation always names the manual cleanup step, and an unreported one is never presented as revoked.
<Warning>
Disconnecting cannot be undone, and it removes the Slack channels from every Alert that targeted them. Reconnecting means approving Prowler in Slack again, authorizing the destination channels again, confirming them with a connection check, and selecting them again on each Alert that posts to Slack.
</Warning>
## Integration Status
The Slack management page reports the state of the connection and offers these actions:
| Button | Purpose | Notes |
|--------|---------|-------|
| **Test connection** | Verify the credential and every authorized channel, and confirm the ones not confirmed yet | Posts the confirmation message once per channel and updates the last-checked time. Cannot be run until at least one channel is authorized |
| **Refresh channels** | Re-read the workspace's channel list | Use after inviting `@Prowler Cloud` to a private channel |
| **Save channels** | Record the selected channels as the integration's authorized set | Enabled once the selection differs from the authorized set |
| **Disconnect** | Remove the integration and attempt to revoke access at Slack | ⚠️ **Cannot be undone** — confirm before disconnecting |
## Troubleshooting
### Slack Is Not Available in This Environment Yet
The Prowler Slack app is not configured for the deployment being used, so no workspace can be connected. This resolves without any action on the tenant's side — the page starts working as soon as the app is configured.
### A Private Channel Does Not Appear in the Channel List
The Prowler app has not been invited to it. In Slack, run `/invite @Prowler Cloud` in that channel, then click **Refresh channels**. Membership is the permission: no scope reveals a private channel the app is not in.
### Connection Test Fails
* Confirm every authorized channel still exists and has not been archived. A failure names the channel Slack refused, and the integration reports as connected only when every authorized channel passes.
* For a private authorized channel, confirm the Prowler app is still a member of it.
* Confirm the Prowler app is still installed in the workspace.
### A Channel Is Missing From an Alert's Channel List
The channel is authorized here but not confirmed yet. Click **Test connection**: it confirms every authorized channel it has not confirmed, and confirmed channels become selectable on Alerts.
### Prowler's Access Has Been Revoked
When Slack stops accepting the stored credential — because a workspace administrator revoked it, or the app was removed from the workspace — Prowler reports the workspace as disconnected and offers **Reconnect to Slack**. Approving Prowler in Slack again restores access.
### The Connection Check Fails on a Channel
* Check the outcome reported on the page: it names the channel Slack refused and the reason Slack gave — an archived or deleted channel surfaces here rather than failing silently.
* Confirm that channel is still one of the intended destinations, and that it has not been archived or deleted in Slack.
* For a private channel, confirm the Prowler app is still a member of it.
* One unreachable channel is enough to report the integration as not connected, so removing a retired channel from the authorized set clears the failure — bearing in mind that removing it also removes it from every Alert that targeted it.
+1 -1
View File
@@ -166,6 +166,6 @@ Once your scan has finished, you don’t need to grab the entire ZIP—just pull
<Note>
**API Note**
To fetch a single compliance report via API, see the Retrieve compliance report as CSV endpoint in the Prowler API Reference. [Prowler API Reference - Retrieve compliance report as CSV](https://api.prowler.com/api/v1/docs#tag/Scan/operation/scans_compliance_retrieve)
To fetch a single compliance report via API, see the Retrieve compliance report as CSV endpoint in the Prowler API Reference.[Prowler API Reference - Retrieve compliance report as CSV](https://api.prowler.com/api/v1/docs#tag/Scan/operation/scans_compliance_retrieve)
</Note>
@@ -10,8 +10,6 @@ import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
Prowler Cloud onboards every AWS account in your Organization through a single guided wizard. Instead of connecting accounts one by one, you can discover every account in your AWS Organization, select the ones you want to monitor, test connectivity, and launch scans — all from the Prowler Cloud UI.
For the cross-provider organization lifecycle and terminology, see [Organizations Across Cloud Providers](/user-guide/organizations).
<SubscriptionBanner>
For CLI-based multi-account scanning, see [AWS Organizations in Prowler CLI](/user-guide/providers/aws/organizations).
</SubscriptionBanner>
@@ -268,7 +266,7 @@ Click **Save**, **Save and launch scan**, or **Launch scan**, depending on the s
After launching:
- Scans appear in the **Scans** page as they start and complete.
- Results populate the **Overview** and **Findings** pages.
- To detect accounts added to or removed from the AWS Organization, repeat the discovery flow described in [Add or Remove Organization Accounts](#add-or-remove-organization-accounts).
- Prowler runs an **automatic sync every 6 hours** to detect accounts added to or removed from your Organization. New accounts under the targeted OU or root are onboarded automatically.
## Manage Your Organization After Onboarding
@@ -290,24 +288,6 @@ Open the row actions menu on the organization row on the **Providers** page.
Organizational unit rows carry the same **Test Connections** and **Delete Organizational Unit** actions, scoped to the accounts beneath them.
### Add or Remove Organization Accounts
To refresh the account membership of an existing AWS Organization, repeat the same discovery flow used during onboarding:
1. Navigate to **Providers**, click **Add Provider**, and select **Amazon Web Services**.
2. Choose **Add Multiple Accounts With AWS Organizations**.
3. Enter the existing **Organization ID**, proceed to **Authentication Details**, and use the existing deployment account **Role ARN**.
4. Confirm that the stack is deployed and click **Authenticate**. Prowler reuses the existing organization and starts a new discovery instead of creating a duplicate.
The refreshed tree shows the accounts currently returned by AWS Organizations:
- **New accounts** appear in the tree. Select them, test their connections, and save the configuration to connect them as providers. Existing providers and their historical data are preserved.
- **Accounts that left the Organization** no longer appear in the tree. Discovery does not automatically delete their existing providers. To remove one, return to the **Providers** page, open the account provider actions, and select **Delete Provider**.
<Danger>
Deleting a provider permanently removes its scans, findings, resources, and other stored data. Confirm that the account has left the AWS Organization and that its historical data is no longer required before deleting it.
</Danger>
### Update Organization Credentials
Choosing **Update Credentials** re-enters the Authentication Details step. Because the organization already holds a credential, Prowler warns before overwriting it and names how many providers re-authenticate with the new one:
@@ -478,17 +458,16 @@ Deploy the ProwlerScan role to every member account with a [CloudFormation Stack
</Note>
1. In your management account, navigate to **CloudFormation > StackSets > Create StackSet** ([open directly](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacksets/create)).
2. Choose **Service-managed permissions**.
3. Enable **Automatic deployment** so CloudFormation deploys the role to accounts added to the targeted root or OUs. Configure the account removal behavior based on whether the stack and its resources should be retained when an account leaves the target.
4. Select **Amazon S3 URL** as the template source and paste:
2. Choose **Service-managed permissions** so AWS Organizations deploys the role automatically across current and future member accounts.
3. Select **Amazon S3 URL** as the template source and paste:
```
https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml
```
5. Set the **ExternalId** parameter to the External ID shown in the Prowler wizard.
6. Choose your deployment targets (entire organization or specific OUs) and regions, then click **Create StackSet**.
7. Open the **Stack instances** tab and confirm every instance shows **Status: CURRENT** and **Stack status: CREATE_COMPLETE**. Deployment typically takes **2–5 minutes**; large organizations (500+ accounts) may take longer.
4. Set the **ExternalId** parameter to the External ID shown in the Prowler wizard.
5. Choose your deployment targets (entire organization or specific OUs) and regions, then click **Create StackSet**.
6. Open the **Stack instances** tab and confirm every instance shows **Status: CURRENT** and **Stack status: CREATE_COMPLETE**. Deployment typically takes **2–5 minutes**; large organizations (500+ accounts) may take longer.
The StackSet role uses read-only access only (`SecurityAudit`, `ViewOnlyAccess`, plus a small set of additional read-only permissions). Prowler makes no changes to your accounts. See the [CloudFormation template](https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml) for the full list. When **Automatic deployment** is enabled, the StackSet deploys the role to new accounts under the targeted OU or root. Repeat the [organization discovery flow](#add-or-remove-organization-accounts) to connect those accounts in Prowler Cloud.
The StackSet role uses read-only access only (`SecurityAudit`, `ViewOnlyAccess`, plus a small set of additional read-only permissions). Prowler makes no changes to your accounts. See the [CloudFormation template](https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml) for the full list. When you add new accounts under the targeted OU or root, the StackSet deploys the role automatically, and Prowler's 6-hour sync onboards them end-to-end.
## Key Concepts
@@ -1,7 +1,7 @@
---
title: 'Import Findings'
sidebarTitle: 'Import Findings'
description: 'Upload OCSF scan results to Prowler Cloud from the UI, the CLI or the API'
description: 'Upload OCSF scan results to Prowler Cloud from external sources or the CLI'
---
import { VersionBadge } from "/snippets/version-badge.mdx"
@@ -9,7 +9,7 @@ import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
<VersionBadge version="5.19.0" />
Findings Ingestion enables uploading OCSF (Open Cybersecurity Schema Framework) scan results to Prowler Cloud. This feature supports importing findings from Prowler CLI output files that use the [Detection Finding](https://schema.ocsf.io/classes/detection_finding) class. Reports can be imported from the Scans page in the Prowler Cloud UI, pushed by the CLI with `--push-to-cloud`, or submitted through the API.
Findings Ingestion enables uploading OCSF (Open Cybersecurity Schema Framework) scan results to Prowler Cloud. This feature supports importing findings from Prowler CLI output files that use the [Detection Finding](https://schema.ocsf.io/classes/detection_finding) class.
<SubscriptionBanner />
@@ -132,32 +132,10 @@ Only **Detection Finding** (`class_uid: 2004`) records are accepted. Other OCSF
## Required Permissions
The **Manage Ingestions** RBAC permission controls access to the ingestion endpoints. Without this permission, findings cannot be submitted from the Scans page, via the API or with `--push-to-cloud`.
The **Manage Ingestions** RBAC permission controls access to the ingestion endpoints. Without this permission, findings cannot be submitted via the API or `--push-to-cloud`.
For more information about RBAC permissions, refer to the [Prowler Cloud RBAC documentation](/user-guide/tutorials/prowler-app-rbac).
## Using the UI
<VersionBadge version="5.41.0" />
The Scans page imports a Prowler OCSF report from the browser, with no CLI or API key involved. The import runs as a regular ingestion job, so the [status values](#ingestion-status-values), the [billing impact](#billing-impact) and the [errors endpoint](#get-ingestion-errors) apply as they do for the CLI and the API.
1. Go to **Scans** and click **Import Findings**. The button is shown only to roles with the **Manage Ingestions** permission.
![Import Findings button on the Scans page](/images/prowler-app/import-findings/import-findings-button.png)
2. Drag a `.ocsf.json` report onto the drop area, or click **Select File** to pick one. The dialog takes one file per import. A file whose name does not end in `.ocsf.json`, or an empty file, is rejected before the upload starts.
![Import findings dialog with the drag-and-drop area](/images/prowler-app/import-findings/import-findings-dialog.png)
3. Click **Start import**. The dialog uploads the report, creates the ingestion job and follows its status until the job finishes. On completion it reports the total number of records, how many were processed and how many were invalid.
Closing the dialog while an import is running does not cancel the job. When the job completes in the background, a notification confirms it and the imported findings appear in Scans.
If the upload is rejected or the job fails, the dialog shows the reason and a **Retry import** button that sends the same file again. A failed job also shows the progress it reported before failing. A different file can be selected instead of retrying. If the status check fails after the upload was accepted, **Retry status** resumes tracking the same job without uploading the file again.
Invalid records are counted in the summary but not listed in the dialog. To see why each one was rejected, [list the ingestion jobs](#list-ingestion-jobs) through the API and query the [errors endpoint](#get-ingestion-errors) for that job.
## Using the CLI
The `--push-to-cloud` flag uploads scan results directly to Prowler Cloud after a scan completes. This approach automates the ingestion process without manual file uploads.
@@ -175,18 +153,6 @@ export PROWLER_CLOUD_API_KEY="pk_your_api_key_here"
prowler aws --push-to-cloud
```
### TLS Certificate Trust
For `--push-to-cloud` uploads, Prowler CLI creates one ingestion-scoped TLS context and validates HTTPS certificates with one handshake and one POST request. The upload does not retry or fall back to another TLS configuration. Redirect responses are rejected.
The ingestion context combines the operating system roots with the default certificate authority (CA) roots bundled with Requests. If `REQUESTS_CA_BUNDLE` is configured, Prowler CLI also loads that file or directory into the ingestion context. Otherwise, Prowler CLI loads `CURL_CA_BUNDLE` when configured.
`REQUESTS_CA_BUNDLE` and `CURL_CA_BUNDLE` can also affect other Requests-based connections throughout the Prowler CLI process, including provider authentication. A bundle containing only a private CA can cause connections to public services to fail before the upload starts. Installing the private CA in the operating system or container trust store is recommended. If a custom bundle is required, it must include both the public CA roots, such as the certifi bundle, and the required private CA certificates.
For Prowler Private Cloud deployments that use an organization CA or a TLS-intercepting corporate proxy, installing the required root CA in the operating system or container store remains the recommended approach. Containers have an isolated system CA store, so add the organization or proxy CA to the container image or runtime, then run the operating system's CA update command, such as `update-ca-certificates`, before starting Prowler CLI. Installing a CA on the container host does not automatically install it inside the container.
Prowler CLI does not create, modify, or remove these environment variables. The custom TLS context created by `--push-to-cloud` applies only to the temporary ingestion session and does not change API, provider, integration, global SSL, or unrelated Requests session behavior.
### Combining with Output Formats
When using `--push-to-cloud` with custom output formats that exclude OCSF, Prowler generates a temporary OCSF file for upload:
@@ -466,7 +432,7 @@ For pricing details, see [Prowler Cloud Pricing](https://prowler.com/pricing).
- The user associated with the API key lacks the **Manage Ingestions** permission
- Contact the tenant administrator to grant the required permission
### Ingestion Job Status Is "failed"
### Ingestion job status is "failed"
- Check the `/api/v1/ingestions/{id}/errors` endpoint for details
- Verify the OCSF file format is valid
+1 -2
View File
@@ -113,8 +113,7 @@ make test-mcp # Run the MCP test suite exactly as CI does
- [ ] Models use `MinimalSerializerMixin`
- [ ] API responses transformed to simplified models
- [ ] No hardcoded secrets
- [ ] Failures are raised, not returned (see `prowler_mcp_server/lib/errors.py`);
a returned error dict is reported to the client as a success
- [ ] Error handling returns structured responses
- [ ] Parameter descriptions use Pydantic `Field()`
- [ ] Tests added under `mcp_server/tests/`, mirroring the source path below the
package root (`prowler_mcp_server/prowler_app/tools/` -> `tests/prowler_app/tools/`),

Some files were not shown because too many files have changed in this diff Show More