Compare commits

..

51 Commits

Author SHA1 Message Date
Adrián Jesús Peña Rodríguez 9e5df42222 Merge branch 'master' into implement-provider-groups-filter 2026-06-12 19:13:47 +02:00
Adrián Jesús Peña Rodríguez 93e510db28 docs(api): update changelog for provider group filters 2026-06-12 19:06:34 +02:00
Adrián Jesús Peña Rodríguez 055befe94c feat(api): add provider group filters
- Add provider group filters alongside provider type filters

- Support exact and comma-separated provider group filtering

- Cover provider group filtering across API views and overviews
2026-06-12 19:04:14 +02:00
Prowler Bot dc3433aaf0 feat(aws): Update regions for AWS services (#11570)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-12 14:15:02 +02:00
Pedro Martín 25fc285966 chore(banner): update info (#11568) 2026-06-12 13:45:34 +02:00
s1ns3nz0 9022a3a138 feat(azure): add cosmosdb_account_automatic_failover_enabled check (#11031)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-12 13:18:08 +02:00
Josema Camacho ca443b8ff1 chore: prepare API and UI changelogs for 5.30.1 release (#11562) 2026-06-12 12:07:31 +02:00
s1ns3nz0 79e066d3f5 feat(gcp): add cloudsql_instance_high_availability_enabled check (#11024)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
Co-authored-by: lydiavilchez <114735608+lydiavilchez@users.noreply.github.com>
2026-06-12 11:51:13 +02:00
Hugo Pereira Brito 56831a7392 feat(oci): add storage admin delete exclusion check (#11523) 2026-06-12 11:10:46 +02:00
Alejandro Bailo 2e82f1564f fix(ui): show threat map data for okta and google workspace accounts (#11542) 2026-06-12 10:07:56 +02:00
Josema Camacho a394c0fdf6 fix(api): drop_subgraph deletes relationships then nodes to cut Neo4j memory (#11557) 2026-06-11 18:32:35 +02:00
Pedro Martín 20eca78767 fix(compliance): resolve provider from scan in attributes endp (#11546) 2026-06-11 18:00:36 +02:00
Oleksandr_Sanin bba594a1db feat(aws/sagemaker): add sagemaker_clarify_exists check (#11211)
Signed-off-by: Oleksandr Sanin <alexaaander.sanin@gmail.com>
Signed-off-by: Oleksandr Yizchak Sanin <alexaaander.sanin@gmail.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-11 17:40:41 +02:00
Hugo Pereira Brito 65f00a197b fix(api): normalize OCI scan region credentials (#11558) 2026-06-11 17:32:28 +02:00
Zeus Almightee ce27053c2d feat(aws): add securityhub + config org-wide delegated admin checks (#11259)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-11 16:53:28 +02:00
Pedro Martín 610febb5d5 fix(api): bump prowler SDK lock to v5.30.0 for okta_idaas_stig (#11553) 2026-06-11 15:53:44 +02:00
Prowler Bot c4378d5992 chore(release): Bump versions to v5.31.0 (#11548)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-11 15:28:25 +02:00
Hugo Pereira Brito f1d741214a fix(ui): adapt risk pipeline sankey layout (#11527) 2026-06-11 09:44:17 +02:00
Pepe Fagoaga 285974b7d4 chore(changelog): v5.30.0 (#11540)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Daniel Barranquero <74871504+danibarranqueroo@users.noreply.github.com>
2026-06-11 09:08:25 +02:00
Daniel Barranquero 989c3b174e fix(bedrock): per-finding severity for long-term API key check (#11526) 2026-06-11 08:31:08 +02:00
Pedro Martín 75f95559d6 fix(api): warm compliance caches when starting the worker (#11530) 2026-06-10 19:04:40 +02:00
sahil-sols e085e14247 fix(aws): order-independent CloudWatch metric filter pattern checks (#11345)
Co-authored-by: Sahil Pugalia <sahil-sols@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-10 18:49:06 +02:00
Johannes Engler 368d3a2661 feat(stackit): add objectstorage checks (#11397)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-06-10 18:43:24 +02:00
Pedro Martín 3c8fde25ee chore(cli): add banner about Prowler Cloud (#11528) 2026-06-10 18:19:50 +02:00
Aryan Bhaskar ec0bb53839 feat(bedrock): add bedrock_agent_role_least_privilege check (#11335)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-10 12:40:54 +02:00
Pedro Martín bfb3fcea4c fix(e2e): use branch SDK changes to create the container (#11522) 2026-06-10 11:34:35 +02:00
Pedro Martín 61cd4aea3f feat(compliance): add Okta IDaaS STIG V1R2 framework (#11428)
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-10 11:22:42 +02:00
StylusFrost 01b49f0743 feat(dashboard): render dynamic-provider compliance frameworks (#11503)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-06-10 11:16:39 +02:00
Pedro Martín 4a5a49b5bb fix(api): store and refresh Resource.name on every scan (#11476)
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-06-10 10:55:31 +02:00
Alan Buscaglia a21cb64a94 fix(ui): extend integration poll timeouts to 60s (#11519) 2026-06-10 10:34:50 +02:00
Hugo Pereira Brito 9a50dffaa0 feat(gcp): split kms_key_rotation_enabled into enabled and max-90-days checks (#11516) 2026-06-09 16:52:49 +02:00
Jasmine e710ebff1c feat(m365): add exchange_mailbox_primary_smtp_custom_domain check (#11215)
Co-authored-by: Jasmine Sullivan <20147180@tafe.wa.edu.au>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-09 16:24:25 +02:00
Hugo Pereira Brito b3caee88e4 fix(m365): skip future hires in MFA capable check (#11511) 2026-06-09 15:42:06 +02:00
Hugo Pereira Brito d9f90e50b8 fix(m365): paginate admincenter group enumeration (#11510) 2026-06-09 15:23:35 +02:00
Alan Buscaglia 58efb719fa docs(skills): correct setup symlink paths in README (#11514) 2026-06-09 14:41:18 +02:00
Alan Buscaglia 355b7071aa docs: add skills installation and usage guide (#11513) 2026-06-09 14:41:13 +02:00
Pepe Fagoaga b994b0b14e chore(ui): rename customer support to support desk (#11508) 2026-06-09 13:53:21 +02:00
StylusFrost 6c559fbb8d feat(sdk): discover external universal compliance frameworks via entry points (#11490) 2026-06-09 13:45:34 +02:00
César Arroba b2d74711d9 chore(deps): bump dulwich to 1.2.5 and pyjwt to 2.13.0 for osv-scanner (#11499) 2026-06-09 13:01:46 +02:00
Ashishraymajhi 7e60e8f8da feat(m365): add entra_service_prinicipal_privileged_role_no_owners_check (#11189)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-09 11:29:03 +02:00
Hugo Pereira Brito 62955dd16b feat(okta): add authenticator STIG checks (#11465)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-09 10:17:23 +02:00
Adrián Peña 1f7caa6394 feat(api): make orphan-task recovery configurable and drop the Jira idempotency table (#11472) 2026-06-09 09:16:48 +02:00
Pepe Fagoaga 662e7e9e18 chore(changelog): prepare for v5.29.3 (#11505) 2026-06-09 08:13:12 +02:00
StylusFrost e3013d9918 feat(sdk): Dynamic provider loading and compliance framework (#10700)
Co-authored-by: Pedro Martín <pedromarting3@gmail.com>
2026-06-08 17:47:22 +02:00
Hugo Pereira Brito 0ea2f6d67e feat(okta): add API token STIG checks (#11464)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-08 17:11:54 +02:00
Hugo Pereira Brito 7692a1d76a feat(okta): add network zone STIG check (#11463)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-08 16:51:58 +02:00
Aline Almeida 1c9afc714e fix(gcp): honour org-aggregated sinks in metric-filter checks (#11488)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-08 16:46:48 +02:00
Daniel Barranquero 466f1a3d73 feat(okta): add user, systemlog, and idp services with DISA STIG checks (#11496)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-06-08 14:59:50 +02:00
César Arroba 061fbaa7bb feat(api): label Postgres connections with application_name per component and alias (#11494) 2026-06-08 13:45:06 +02:00
Josema Camacho 28b045302f fix(api): create Neo4j driver lazily so an outage can't block API startup (#11491) 2026-06-08 13:30:18 +02:00
Alejandro Bailo 5a2226c02c fix(ui): preserve active tab styling with tooltips (#11493) 2026-06-08 11:54:51 +02:00
402 changed files with 24835 additions and 3412 deletions
+1 -1
View File
@@ -145,7 +145,7 @@ SENTRY_RELEASE=local
NEXT_PUBLIC_SENTRY_ENVIRONMENT=${SENTRY_ENVIRONMENT}
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.30.0
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.31.0
# Social login credentials
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
+11 -1
View File
@@ -134,7 +134,17 @@ jobs:
# docker-compose.yml references prowlercloud/prowler-api:latest from the registry,
# which lags behind PR changes; build locally so E2E exercises the API image
# produced by this PR.
run: docker build -t prowlercloud/prowler-api:latest ./api
#
# The image installs the SDK from git@master (api/uv.lock), so a PR changing BOTH the SDK
# and the API would run against the OLD SDK and crash on startup. Overlay the checkout's
# SDK source so both run together. New SDK dependencies still need an api/uv.lock bump.
run: |
docker build -t prowlercloud/prowler-api:pr-base ./api
docker build -t prowlercloud/prowler-api:latest -f - prowler <<'DOCKERFILE'
FROM prowlercloud/prowler-api:pr-base
RUN rm -rf /home/prowler/.venv/lib/python3.12/site-packages/prowler
COPY --chown=prowler:prowler . /home/prowler/.venv/lib/python3.12/site-packages/prowler
DOCKERFILE
- name: Start API services
run: |
+1 -1
View File
@@ -122,7 +122,7 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
| Vercel | 26 | 6 | 0 | 8 | Official | UI, API, CLI |
| Okta | 1 | 1 | 0 | 1 | Official | CLI |
| Scaleway [Contact us](https://prowler.com/contact) | 1 | 1 | 0 | 1 | Unofficial | CLI |
| StackIT [Contact us](https://prowler.com/contact) | 4 | 1 | 0 | 1 | Unofficial | CLI |
| StackIT [Contact us](https://prowler.com/contact) | 7 | 2 | 0 | 3 | Unofficial | CLI |
| NHN | 6 | 2 | 1 | 0 | Unofficial | CLI |
> [!Note]
+36 -5
View File
@@ -2,23 +2,54 @@
All notable changes to the **Prowler API** are documented in this file.
## [1.31.0] (Prowler UNRELEASED)
## [1.32.0] (Prowler UNRELEASED)
### 🚀 Added
- Automatic recovery of allowlisted idempotent background tasks whose worker died during a deploy or crash: stuck scan and summary tasks are detected and re-run instead of staying pending forever, with a `reconcile_orphan_tasks` management command for on-demand recovery [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- Jira integration no longer creates duplicate issues on a retried send; findings already ticketed are skipped [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- Provider group filters for API endpoints that support cloud provider filtering, including exact and `__in` variants [(#11573)](https://github.com/prowler-cloud/prowler/pull/11573)
---
## [1.31.1] (Prowler v5.30.1)
### 🐞 Fixed
- `compliance-overviews/attributes` now resolves the provider from the scan, so multi-provider universal frameworks (e.g. CSA CCM) return the check IDs of the scan's provider and Azure/GCP requirement details show their findings instead of appearing empty [(#11546)](https://github.com/prowler-cloud/prowler/pull/11546)
- Attack Paths: `drop_subgraph` now deletes relationships first and then nodes in batches, using less memory on Neo4j when clearing a dense provider graph [(#11557)](https://github.com/prowler-cloud/prowler/pull/11557)
- OCI scans now use API key credentials with the configured region instead of falling back to `/home/prowler/.oci/config` [(#11558)](https://github.com/prowler-cloud/prowler/pull/11558)
---
## [1.31.0] (Prowler v5.30.0)
### 🚀 Added
- Opt-in automatic recovery of allowlisted idempotent background tasks whose worker died during a deploy or crash: when enabled via `DJANGO_TASK_RECOVERY_ENABLED` (off by default), stuck summary and deletion tasks are detected and re-run instead of staying pending forever (scan and Jira tasks are excluded), with a `reconcile_orphan_tasks` management command for on-demand recovery [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- DORA compliance framework support [(#11131)](https://github.com/prowler-cloud/prowler/pull/11131)
- Label Postgres connections with `application_name="<component>:<alias>"` (component injected per process via `DJANGO_APP_COMPONENT`) so connections are attributable by component in `pg_stat_activity` [(#11494)](https://github.com/prowler-cloud/prowler/pull/11494)
- DISA Okta IDaaS STIG V1R2 compliance framework export support for the Okta provider [(#11428)](https://github.com/prowler-cloud/prowler/pull/11428)
### 🔄 Changed
- Allowlisted idempotent background tasks are no longer lost when a worker is stopped or crashes mid-task; tasks with external side effects are marked terminal instead of blindly re-running [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- A recovered scan rewrites its findings, summaries, attack surface, and compliance data instead of appending to the previous run, so recovery never leaves stale or duplicate materialized rows [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- Provider type is validated against the SDK's available providers instead of a static enum, so the API accepts any installed provider (built-in or external); `Provider.provider` is stored as `varchar` and the native PostgreSQL enum is removed [(#11399)](https://github.com/prowler-cloud/prowler/pull/11399)
### 🐞 Fixed
- Workers now shut down gracefully on deploy or restart, finishing or re-queueing in-flight tasks instead of being force-killed and leaving them stuck [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- Resource `name` is now stored and refreshed on every scan, so resources no longer keep an empty name [(#11476)](https://github.com/prowler-cloud/prowler/pull/11476)
- Compliance catalog now warms in background during startup. `compliance-overviews/attributes` returns `503` while warming, so the first request after a deploy no longer trips the API timeout [(#11530)](https://github.com/prowler-cloud/prowler/pull/11530)
### 🔐 Security
- `dulwich` from 0.23.0 to 1.2.5 and `pyjwt` from 2.12.1 to 2.13.0, patching `GHSA-897w-fcg9-f6xj` (arbitrary file write) and `PYSEC-2026-179` (HMAC/JWK key confusion) [(#11499)](https://github.com/prowler-cloud/prowler/pull/11499)
---
## [1.30.3] (Prowler v5.29.3)
### 🐞 Fixed
- API startup no longer crashes when Neo4j is unreachable, as the Neo4j driver now connects lazily on first use rather than during app initialization [(#11491)](https://github.com/prowler-cloud/prowler/pull/11491)
---
+9
View File
@@ -68,6 +68,15 @@ manage_db_partitions() {
fi
}
# Identify this process to Postgres (application_name=<component>:<alias>) so
# connections are attributable by component in pg_stat_activity. Web tiers
# report "api"; everything else uses the launch subcommand.
case "$1" in
prod|dev) DJANGO_APP_COMPONENT="api" ;;
*) DJANGO_APP_COMPONENT="$1" ;;
esac
export DJANGO_APP_COMPONENT
case "$1" in
dev)
apply_migrations
+37 -18
View File
@@ -1,10 +1,11 @@
# Orphan Celery task recovery
When a worker is terminated mid-task (a deploy, an OOM kill, a node eviction), the
task it was running can be left non-terminal forever: the `Scan` stays `EXECUTING`,
the `TaskResult` stays `STARTED`, and nothing re-runs it. This page describes the
mechanisms that detect and recover allowlisted idempotent orphans so users never
see a stuck scan and pending-task alerts do not fire.
task it was running can be left non-terminal forever: the `TaskResult` stays
`STARTED` and nothing re-runs it. This page describes the mechanisms that detect and
recover allowlisted idempotent orphans so pending-task alerts do not fire. Scan tasks
are not auto-recovered (re-running a scan is not safe to do automatically); the
watchdog covers the summary/aggregation and deletion tasks.
## How recovery works
@@ -13,29 +14,35 @@ see a stuck scan and pending-task alerts do not fire.
(`worker_prefetch_multiplier = 1`), and an abruptly-lost worker re-queues its task
(`task_reject_on_worker_lost`). On `SIGTERM` the worker is given a soft-shutdown
window (`worker_soft_shutdown_timeout`) to finish or re-queue in-flight work
before it is force-killed.
before it is force-killed. `scan-perform`, `scan-perform-scheduled` and
`integration-jira` opt out of redelivery with `acks_late=False`, so a crash drops
them rather than re-running and duplicating findings or Jira issues. Other
non-recovered side-effect tasks keep `acks_late=True`, so the broker can still
re-deliver them after a worker loss: the S3 upload rebuilds from worker-local files
that did not survive the crash and so no-ops, but Security Hub re-reads findings from
the DB and re-sends them to AWS.
2. **Periodic watchdog.** A Beat task, `reconcile-orphan-tasks`, runs every couple of
minutes (a `django_celery_beat` periodic task created by migration). For each
in-flight task result with an allowlisted idempotent task name, it pings the
worker recorded on the task's `TaskResult`:
- worker responds -> the task is still running, leave it alone;
- worker is gone (and the scan started before a short grace window) -> it is a
- worker is gone (and the task started before a short grace window) -> it is a
real orphan: the stale task is revoked and marked terminal (clearing the
pending/started alert), and the scan is re-enqueued from scratch.
pending/started alert), and the task is re-enqueued from its stored name and
kwargs.
The re-run is safe because only tasks with proven idempotency are allowlisted.
Scan persistence, for example, clears the scan's prior findings and materialized
summary/compliance rows before re-writing them. Jira sends are allowlisted too:
each finding is reserved in a dispatch table before the external call, so a re-run
skips already-ticketed findings (the worst case is one finding missed if a worker
is hard-killed mid-send, never a duplicate issue). Other external side effects stay
terminal: the S3 upload rebuilds from worker-local files that do not survive a
crash, and report/Security Hub recovery is out of scope.
The re-run is safe because only tasks with proven idempotency are allowlisted: the
summary/aggregation tasks clear and re-write their own rows, and deletions are
idempotent. Scan tasks and external side effects are excluded: re-running a scan is
not safe to do automatically, Jira sends would create duplicate issues, the S3
upload rebuilds from worker-local files that do not survive a crash, and
report/Security Hub recovery is out of scope.
3. **Recovery cap.** Each automatic re-enqueue increments `Scan.recovery_count`.
After `--max-attempts` recoveries (default 3) the scan is marked `FAILED` instead
of re-enqueued, so a task that repeatedly kills its worker cannot loop forever.
3. **Recovery cap.** A per-task Valkey counter limits how often the same task is
re-enqueued. After `--max-attempts` recoveries (default 3) the orphan is marked
terminal instead of re-enqueued, so a task that repeatedly kills its worker cannot
loop forever.
A Postgres advisory lock ensures that, even with multiple API/worker replicas, only
one reconciliation runs at a time; the others no-op.
@@ -63,6 +70,18 @@ All settings have safe defaults; override via environment variables.
| `DJANGO_CELERY_TASK_SOFT_TIME_LIMIT` | hard - 600 | Soft limit; raises `SoftTimeLimitExceeded` for cleanup. |
| `DJANGO_CELERY_LONG_TASK_TIME_LIMIT` | `172800` (48h) | Hard limit for scans and provider/tenant deletions, which can legitimately run for more than a day. |
| `DJANGO_CELERY_LONG_TASK_SOFT_TIME_LIMIT` | long hard - 600 | Soft limit for the long-running tasks above. |
| `DJANGO_TASK_RECOVERY_ENABLED` | `false` | Master switch for orphan-task recovery, disabled by default (opt-in); set to `true` to enable. When off, no orphan is detected, marked terminal, or re-enqueued (attack-paths stale cleanup still runs). |
| `DJANGO_TASK_RECOVERY_SUMMARIES_ENABLED` | `true` | Auto re-enqueue orphaned scan summary/aggregation tasks. |
| `DJANGO_TASK_RECOVERY_DELETIONS_ENABLED` | `true` | Auto re-enqueue orphaned provider/tenant deletion tasks. |
Recovery is opt-in: with the master flag off (the default) the sweep does nothing.
Once enabled, the per-group flags default to on, so every group recovers unless you
turn one off; a task whose group flag is off is marked terminal instead of
re-enqueued.
Turning recovery off only disables this watchdog sweep; it does not change Celery's
broker-level redelivery (`task_acks_late`/`task_reject_on_worker_lost`), which still
re-delivers tasks that keep `acks_late=True` on worker loss, independently of this flag.
`task_acks_late` and `task_reject_on_worker_lost` are enabled in `config/celery.py`.
+14 -4
View File
@@ -68,7 +68,7 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.31.0"
version = "1.32.0"
[tool.uv]
# Transitive pins matching master to avoid silent drift; bump deliberately.
@@ -226,7 +226,7 @@ constraint-dependencies = [
"drf-simple-apikey==2.2.1",
"drf-spectacular==0.27.2",
"drf-spectacular-jsonapi==0.5.1",
"dulwich==0.23.0",
"dulwich==1.2.5",
"duo-client==5.5.0",
"durationpy==0.10",
"email-validator==2.2.0",
@@ -354,7 +354,7 @@ constraint-dependencies = [
"pydantic-core==2.41.5",
"pygithub==2.8.0",
"pygments==2.20.0",
"pyjwt==2.12.1",
"pyjwt==2.13.0",
"pylint==3.2.5",
"pymsalruntime==0.18.1",
"pynacl==1.6.2",
@@ -443,7 +443,17 @@ constraint-dependencies = [
# The microsoft-kiota-http security bump to 1.9.9 (GHSA-7j59-v9qr-6fq9) requires
# microsoft-kiota-abstractions>=1.9.9, which a constraint cannot satisfy against the
# SDK's hard pin; override it to the patched, kiota-aligned version.
#
# prowler@master hard-pins dulwich==0.23.0 and pyjwt==2.12.1 in [project.dependencies].
# dulwich 1.2.5 patches GHSA-897w-fcg9-f6xj (arbitrary file write) and pyjwt 2.13.0
# patches PYSEC-2026-179 (HMAC/JWK key-confusion); a constraint cannot satisfy these
# against the SDK's hard pins, so override them to the patched versions until the SDK
# bump propagates to the pinned master rev. pyjwt keeps the [crypto] extra because an
# override replaces the whole requirement; bare pyjwt would drop it from the consumers
# that request pyjwt[crypto] and leave cryptography (needed for RS256) only transitive.
override-dependencies = [
"okta==3.4.2",
"microsoft-kiota-abstractions==1.9.9"
"microsoft-kiota-abstractions==1.9.9",
"dulwich==1.2.5",
"pyjwt[crypto]==2.13.0"
]
+6 -34
View File
@@ -1,12 +1,14 @@
import logging
import os
import sys
from pathlib import Path
from django.apps import AppConfig
from django.conf import settings
from config.custom_logging import BackendLogger
from config.env import env
from django.apps import AppConfig
from django.conf import settings
logger = logging.getLogger(BackendLogger.API)
@@ -30,7 +32,6 @@ class ApiConfig(AppConfig):
def ready(self):
from api import schema_extensions # noqa: F401
from api import signals # noqa: F401
from api.attack_paths import database as graph_database
# Generate required cryptographic keys if not present, but only if:
# `"manage.py" not in sys.argv[0]`: If an external server (e.g., Gunicorn) is running the app
@@ -41,37 +42,8 @@ class ApiConfig(AppConfig):
):
self._ensure_crypto_keys()
# Commands that don't need Neo4j
SKIP_NEO4J_DJANGO_COMMANDS = [
"makemigrations",
"migrate",
"pgpartition",
"check",
"help",
"showmigrations",
"check_and_fix_socialaccount_sites_migration",
]
# Skip eager Neo4j init for tests, some Django commands, and Celery (prefork pool: driver must stay lazy, no post_fork hook)
if getattr(settings, "TESTING", False) or (
len(sys.argv) > 1
and (
(
"manage.py" in sys.argv[0]
and sys.argv[1] in SKIP_NEO4J_DJANGO_COMMANDS
)
or "celery" in sys.argv[0]
)
):
logger.info(
"Skipping eager Neo4j init: tests, some Django commands, or Celery prefork pool (driver stays lazy)"
)
else:
graph_database.init_driver()
# Neo4j driver is initialized at API startup (see api.attack_paths.database)
# It remains lazy for Celery workers and selected Django commands
# Neo4j driver is created lazily on first use (see api.attack_paths.database).
# App init never contacts Neo4j, so a Neo4j outage cannot block API startup.
def _ensure_crypto_keys(self):
"""
+36 -6
View File
@@ -1,22 +1,24 @@
import atexit
import logging
import threading
from contextlib import contextmanager
from typing import Any, Iterator
from uuid import UUID
import neo4j
import neo4j.exceptions
from config.env import env
from django.conf import settings
from api.attack_paths.retryable_session import RetryableSession
from tasks.jobs.attack_paths.config import (
BATCH_SIZE,
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
from api.attack_paths.retryable_session import RetryableSession
# Without this Celery goes crazy with Neo4j logging
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
@@ -28,6 +30,9 @@ READ_QUERY_TIMEOUT_SECONDS = env.int(
"ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30
)
MAX_CUSTOM_QUERY_NODES = env.int("ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES", default=250)
# Shorter than CONN_ACQUISITION_TIMEOUT — the driver requires acquisition to be
# the longer of the two (it may include opening a new connection).
CONNECTION_TIMEOUT = env.int("NEO4J_CONNECTION_TIMEOUT", default=5)
CONN_ACQUISITION_TIMEOUT = env.int("NEO4J_CONN_ACQUISITION_TIMEOUT", default=15)
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
@@ -58,15 +63,24 @@ def init_driver() -> neo4j.Driver:
uri = get_uri()
config = settings.DATABASES["neo4j"]
_driver = neo4j.GraphDatabase.driver(
driver = neo4j.GraphDatabase.driver(
uri,
auth=(config["USER"], config["PASSWORD"]),
keep_alive=True,
max_connection_lifetime=7200,
connection_timeout=CONNECTION_TIMEOUT,
connection_acquisition_timeout=CONN_ACQUISITION_TIMEOUT,
max_connection_pool_size=50,
)
_driver.verify_connectivity()
# Publish the singleton only after connectivity is verified so a
# failed probe does not leave an unverified driver behind. Close the
# driver on failure so a repeatedly-probed outage cannot leak pools.
try:
driver.verify_connectivity()
except Exception:
driver.close()
raise
_driver = driver
# Register cleanup handler (only runs once since we're inside the _driver is None block)
atexit.register(close_driver)
@@ -161,7 +175,8 @@ def drop_subgraph(database: str, provider_id: str) -> int:
"""
Delete all nodes for a provider from the tenant database.
Uses batched deletion to avoid memory issues with large graphs.
Deletes relationships then nodes in batches (not `DETACH DELETE`) so a dense
provider's graph cannot exceed Neo4j's transaction memory limit.
Silently returns 0 if the database doesn't exist.
"""
provider_label = get_provider_label(provider_id)
@@ -169,13 +184,28 @@ def drop_subgraph(database: str, provider_id: str) -> int:
try:
with get_session(database) as session:
# Phase 1: delete relationships incident to provider nodes in batches.
deleted_count = 1
while deleted_count > 0:
result = session.run(
f"""
MATCH (:`{provider_label}`)-[r]-()
WITH DISTINCT r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
{"batch_size": BATCH_SIZE},
)
deleted_count = result.single().get("deleted_rels_count", 0)
# Phase 2: delete the now relationship-free nodes in batches.
deleted_count = 1
while deleted_count > 0:
result = session.run(
f"""
MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`)
WITH n LIMIT $batch_size
DETACH DELETE n
DELETE n
RETURN COUNT(n) AS deleted_nodes_count
""",
{"batch_size": BATCH_SIZE},
+63
View File
@@ -1,3 +1,5 @@
import logging
import threading
from collections.abc import Iterable, Mapping
from api.models import Provider
@@ -6,8 +8,19 @@ from prowler.lib.check.compliance_models import (
)
from prowler.lib.check.models import CheckMetadata
logger = logging.getLogger(__name__)
AVAILABLE_COMPLIANCE_FRAMEWORKS = {}
# Per-process readiness flags for the background compliance warm-up.
# `STARTED` is set as soon as warming begins (only happens under Gunicorn via
# the post_fork hook); `WARMED` is set when it finishes. The attributes
# endpoint checks both: it returns 503 only while warming is in progress.
# Under `runserver` warming never runs, so `STARTED` stays clear and the
# endpoint keeps lazy-loading as before.
COMPLIANCE_WARMING_STARTED = threading.Event()
COMPLIANCE_WARMED = threading.Event()
class LazyComplianceTemplate(Mapping):
"""Lazy-load compliance templates per provider on first access."""
@@ -174,6 +187,56 @@ def _ensure_provider_loaded(provider_type: Provider.ProviderChoices) -> None:
PROWLER_CHECKS._cache[provider_type] = checks
def warm_compliance_caches(
provider_types: Iterable[str] | None = None,
) -> list[str]:
"""
Eagerly populate the per-process compliance caches at server startup.
Moves the cold-cache catalog load off the request thread so the first
request does not trip the Gunicorn worker timeout. Reads only on-disk
metadata (no database access). Each provider is warmed in isolation;
failures are logged and fall back to lazy loading.
Args:
provider_types (Iterable[str] | None): Subset to warm. Defaults to all.
Returns:
list[str]: Provider types that could not be warmed.
"""
if provider_types is None:
provider_types = Provider.ProviderChoices.values
provider_types = list(provider_types)
COMPLIANCE_WARMING_STARTED.set()
logger.info("Compliance cache warm-up started for providers: %s", provider_types)
failed = []
for provider_type in provider_types:
try:
get_compliance_frameworks(provider_type)
_ensure_provider_loaded(provider_type)
# Prowler check loading may sys.exit (SystemExit, not Exception).
except (Exception, SystemExit):
logger.warning(
"Failed to warm compliance caches for provider '%s'; "
"loading lazily on first request",
provider_type,
exc_info=True,
)
failed.append(provider_type)
# Mark as warmed even when some providers failed: a failed provider falls
# back to a single-provider lazy load, which stays under the worker timeout.
COMPLIANCE_WARMED.set()
logger.info(
"Compliance cache warm-up finished (providers warmed: %d, failed: %s)",
len(provider_types) - len(failed),
failed,
)
return failed
def load_prowler_checks(
prowler_compliance, provider_types: Iterable[str] | None = None
):
+26
View File
@@ -187,6 +187,32 @@ class UpstreamServiceUnavailableError(APIException):
)
class ComplianceWarmingError(APIException):
"""Compliance catalog is still warming (503 Service Unavailable).
Returned by the compliance attributes endpoint while the per-process
catalog warm-up is in progress, so the request thread never triggers the
slow cold load that would trip the Gunicorn worker timeout.
"""
status_code = status.HTTP_503_SERVICE_UNAVAILABLE
default_detail = (
"Compliance data is still loading. Please try again in a few seconds."
)
default_code = "compliance_warming"
def __init__(self, detail=None):
super().__init__(
detail=[
{
"detail": detail or self.default_detail,
"status": str(self.status_code),
"code": self.default_code,
}
]
)
class UpstreamInternalError(APIException):
"""Unexpected error communicating with provider (500 Internal Server Error).
+136 -25
View File
@@ -19,6 +19,7 @@ from api.constants import SEVERITY_ORDER
from api.db_utils import (
FindingDeltaEnumField,
InvitationStateEnumField,
ProviderEnumField,
SeverityEnumField,
StatusEnumField,
)
@@ -66,7 +67,6 @@ from api.uuid_utils import (
uuid7_range,
uuid7_start,
)
from api.provider_types import get_provider_type_choices
from api.v1.serializers import TaskBase
@@ -102,20 +102,30 @@ class BaseProviderFilter(FilterSet):
"""
Abstract base filter for models with direct FK to Provider.
Provides standard provider_id and provider_type filters.
Provides standard provider_id, provider_type, and provider_groups filters.
Subclasses must define Meta.model.
"""
provider_id = UUIDFilter(field_name="provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="provider__provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
class Meta:
abstract = True
@@ -126,20 +136,30 @@ class BaseScanProviderFilter(FilterSet):
"""
Abstract base filter for models with FK to Scan (and Scan has FK to Provider).
Provides standard provider_id and provider_type filters via scan relationship.
Provides standard provider_id, provider_type, and provider_groups filters via scan relationship.
Subclasses must define Meta.model.
"""
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="scan__provider__provider", choices=get_provider_type_choices()
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="scan__provider__provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_groups = UUIDFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
class Meta:
abstract = True
@@ -155,10 +175,20 @@ class CommonFindingFilters(FilterSet):
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
choices=get_provider_type_choices(), field_name="scan__provider__provider"
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_type__in = ChoiceInFilter(
choices=get_provider_type_choices(), field_name="scan__provider__provider"
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_groups = UUIDFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
provider_uid = CharFilter(field_name="scan__provider__uid", lookup_expr="exact")
provider_uid__in = CharInFilter(field_name="scan__provider__uid", lookup_expr="in")
@@ -356,20 +386,26 @@ class ProviderFilter(FilterSet):
included. Providers with no connection attempt (status is null) are
excluded from this filter."""
)
provider = ChoiceFilter(choices=get_provider_type_choices())
provider = ChoiceFilter(choices=Provider.ProviderChoices.choices)
provider__in = ChoiceInFilter(
field_name="provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_type = ChoiceFilter(
choices=get_provider_type_choices(), field_name="provider"
choices=Provider.ProviderChoices.choices, field_name="provider"
)
provider_type__in = ChoiceInFilter(
field_name="provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_groups = UUIDFilter(
field_name="provider_groups__id", lookup_expr="exact", distinct=True
)
provider_groups__in = UUIDInFilter(
field_name="provider_groups__id", lookup_expr="in", distinct=True
)
class Meta:
model = Provider
@@ -381,14 +417,29 @@ class ProviderFilter(FilterSet):
"inserted_at": ["gte", "lte"],
"updated_at": ["gte", "lte"],
}
filter_overrides = {
ProviderEnumField: {
"filter_class": CharFilter,
},
}
class ProviderRelationshipFilterSet(FilterSet):
provider_type = ChoiceFilter(
choices=get_provider_type_choices(), field_name="provider__provider"
choices=Provider.ProviderChoices.choices, field_name="provider__provider"
)
provider_type__in = ChoiceInFilter(
choices=get_provider_type_choices(), field_name="provider__provider"
choices=Provider.ProviderChoices.choices, field_name="provider__provider"
)
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
provider_uid = CharFilter(field_name="provider__uid", lookup_expr="exact")
provider_uid__in = CharInFilter(field_name="provider__uid", lookup_expr="in")
@@ -993,9 +1044,19 @@ class FindingGroupSummaryFilter(_CheckTitleToCheckIdMixin, FilterSet):
provider_id = UUIDFilter(field_name="provider_id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider_id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = CharInFilter(field_name="provider__provider", lookup_expr="in")
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
class Meta:
model = FindingGroupDailySummary
@@ -1093,9 +1154,19 @@ class LatestFindingGroupSummaryFilter(_CheckTitleToCheckIdMixin, FilterSet):
provider_id = UUIDFilter(field_name="provider_id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider_id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = CharInFilter(field_name="provider__provider", lookup_expr="in")
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
class Meta:
model = FindingGroupDailySummary
@@ -1296,10 +1367,20 @@ class ScanSummaryFilter(FilterSet):
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="scan__provider__provider", choices=get_provider_type_choices()
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="scan__provider__provider", choices=get_provider_type_choices()
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
provider_groups = UUIDFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
region = CharFilter(field_name="region")
@@ -1319,10 +1400,20 @@ class DailySeveritySummaryFilter(FilterSet):
provider_id = UUIDFilter(field_name="provider_id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider_id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
date_from = DateFilter(method="filter_noop")
date_to = DateFilter(method="filter_noop")
@@ -1573,13 +1664,23 @@ class ThreatScoreSnapshotFilter(FilterSet):
provider_id = UUIDFilter(field_name="provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=get_provider_type_choices()
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="provider__provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_groups = UUIDFilter(
field_name="provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
compliance_id = CharFilter(field_name="compliance_id", lookup_expr="exact")
compliance_id__in = CharInFilter(field_name="compliance_id", lookup_expr="in")
@@ -1616,13 +1717,23 @@ class ResourceGroupOverviewFilter(FilterSet):
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="scan__provider__provider", choices=get_provider_type_choices()
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="scan__provider__provider",
choices=get_provider_type_choices(),
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_groups = UUIDFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="exact",
distinct=True,
)
provider_groups__in = UUIDInFilter(
field_name="scan__provider__provider_groups__id",
lookup_expr="in",
distinct=True,
)
resource_group = CharFilter(field_name="resource_group", lookup_expr="exact")
resource_group__in = CharInFilter(field_name="resource_group", lookup_expr="in")
@@ -20,7 +20,7 @@ class Command(BaseCommand):
"--max-attempts",
type=int,
default=3,
help="Give up re-running a task after this many recovery attempts (scans are marked FAILED).",
help="Give up re-running a task after this many recovery attempts; it is then left terminal instead of re-enqueued.",
)
parser.add_argument(
"--dry-run",
@@ -39,6 +39,16 @@ class Command(BaseCommand):
self.stdout.write("Reconcile skipped: another run holds the lock.")
return
if result.get("enabled") is False:
message = (
"Task recovery is disabled (DJANGO_TASK_RECOVERY_ENABLED is off); "
"no orphans were recovered."
)
if result.get("attack_paths") is not None:
message += " Attack-paths stale cleanup still ran."
self.stdout.write(message)
return
self.stdout.write(
self.style.SUCCESS(
"Orphan reconcile complete: "
@@ -1,17 +0,0 @@
# Generated by Django 5.1.15 on 2026-05-30 17:38
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0093_okta_provider"),
]
operations = [
migrations.AddField(
model_name="scan",
name="recovery_count",
field=models.IntegerField(default=0),
),
]
@@ -40,7 +40,7 @@ def delete_periodic_task(apps, schema_editor):
class Migration(migrations.Migration):
dependencies = [
("api", "0094_scan_recovery_count"),
("api", "0093_okta_provider"),
("django_celery_beat", "0019_alter_periodictasks_options"),
]
@@ -1,64 +0,0 @@
import uuid
import django.db.models.deletion
from django.db import migrations, models
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0095_reconcile_orphan_tasks_periodic_task"),
]
operations = [
migrations.CreateModel(
name="JiraIssueDispatch",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("finding_id", models.UUIDField()),
(
"integration",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="jira_dispatches",
to="api.integration",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "jira_issue_dispatches",
"abstract": False,
},
),
migrations.AddConstraint(
model_name="jiraissuedispatch",
constraint=models.UniqueConstraint(
fields=("tenant_id", "integration_id", "finding_id"),
name="unique_jira_issue_dispatch",
),
),
migrations.AddConstraint(
model_name="jiraissuedispatch",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_jiraissuedispatch",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
@@ -1,43 +0,0 @@
from django.db import migrations, models
class Migration(migrations.Migration):
"""Expand step of the zero-downtime migration of Provider.provider from a
native PostgreSQL enum to varchar.
Adds a transitional varchar shadow column `provider_str` and a trigger that
keeps it in sync with the `provider` enum column on every INSERT/UPDATE.
Adding a nullable column is metadata-only (no table rewrite, no long lock).
The trigger covers writes from now on; existing rows are populated by a
later backfill. A subsequent migration drops the enum column and renames
`provider_str` to take its place.
"""
dependencies = [
("api", "0096_jiraissuedispatch"),
]
operations = [
migrations.AddField(
model_name="provider",
name="provider_str",
field=models.CharField(blank=True, max_length=50, null=True),
),
migrations.RunSQL(
sql=(
"CREATE OR REPLACE FUNCTION sync_provider_str() RETURNS trigger AS $$\n"
"BEGIN\n"
" NEW.provider_str := NEW.provider::text;\n"
" RETURN NEW;\n"
"END;\n"
"$$ LANGUAGE plpgsql;\n"
"CREATE TRIGGER providers_sync_provider_str\n"
" BEFORE INSERT OR UPDATE ON providers\n"
" FOR EACH ROW EXECUTE FUNCTION sync_provider_str();"
),
reverse_sql=(
"DROP TRIGGER IF EXISTS providers_sync_provider_str ON providers;\n"
"DROP FUNCTION IF EXISTS sync_provider_str();"
),
),
]
@@ -1,25 +0,0 @@
from django.db import migrations
class Migration(migrations.Migration):
"""Synchronous backfill of the `provider_str` shadow column.
A single UPDATE fills rows that predate the 0097 trigger. The providers
table is small, so this is safe inline and guarantees the column is fully
populated before 0099 sets it NOT NULL (no race with an async backfill).
Runs on the migration connection, which is exempt from RLS.
"""
dependencies = [
("api", "0097_provider_str_shadow_column"),
]
operations = [
migrations.RunSQL(
sql=(
"UPDATE providers SET provider_str = provider::text "
"WHERE provider_str IS NULL;"
),
reverse_sql=migrations.RunSQL.noop,
),
]
@@ -1,51 +0,0 @@
from django.db import migrations, models
class Migration(migrations.Migration):
"""Contract step: promote `provider_str` into `provider`.
Drops the trigger and enum column, renames the shadow column, sets it NOT
NULL, and drops the enum type. The unique index is dropped and recreated in
the same transaction, so there is no window for duplicate active providers;
recreated non-concurrently since the table is small, with a short
lock_timeout so the migration fails fast instead of queueing behind a
long-running transaction.
"""
dependencies = [
("api", "0098_backfill_provider_str"),
]
operations = [
migrations.SeparateDatabaseAndState(
state_operations=[
migrations.RemoveField(
model_name="provider",
name="provider_str",
),
migrations.AlterField(
model_name="provider",
name="provider",
field=models.CharField(default="aws", max_length=50),
),
],
database_operations=[
migrations.RunSQL(
sql=(
"SET LOCAL lock_timeout = '10s';\n"
"DROP TRIGGER IF EXISTS providers_sync_provider_str ON providers;\n"
"DROP FUNCTION IF EXISTS sync_provider_str();\n"
"DROP INDEX IF EXISTS unique_provider_uids;\n"
"ALTER TABLE providers DROP COLUMN provider;\n"
"ALTER TABLE providers RENAME COLUMN provider_str TO provider;\n"
"ALTER TABLE providers ALTER COLUMN provider SET DEFAULT 'aws';\n"
"ALTER TABLE providers ALTER COLUMN provider SET NOT NULL;\n"
"DROP TYPE provider;\n"
"CREATE UNIQUE INDEX unique_provider_uids ON providers "
"(tenant_id, provider, uid) WHERE NOT is_deleted;"
),
reverse_sql=migrations.RunSQL.noop,
),
],
),
]
+5 -49
View File
@@ -40,6 +40,7 @@ from api.db_utils import (
InvitationStateEnumField,
MemberRoleEnumField,
ProcessorTypeEnumField,
ProviderEnumField,
ProviderSecretTypeEnumField,
ScanTriggerEnumField,
SeverityEnumField,
@@ -58,7 +59,6 @@ from api.rls import (
Tenant,
)
from prowler.lib.check.models import Severity
from prowler.providers.common.provider import Provider as SDKProvider
fernet = Fernet(settings.SECRETS_ENCRYPTION_KEY.encode())
@@ -482,10 +482,9 @@ class Provider(RowLevelSecurityProtectedModel):
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
is_deleted = models.BooleanField(default=False)
# Stored as a plain varchar: the SDK is the source of truth for which
# providers are valid, so the column accepts any provider name without a
# database-level enum to keep in sync.
provider = models.CharField(max_length=50, default=ProviderChoices.AWS)
provider = ProviderEnumField(
choices=ProviderChoices.choices, default=ProviderChoices.AWS
)
uid = models.CharField(
"Unique identifier for the provider, set by the provider",
max_length=250,
@@ -502,24 +501,13 @@ class Provider(RowLevelSecurityProtectedModel):
def clean(self):
super().clean()
if self.provider not in SDKProvider.get_available_providers():
raise ModelValidationError(
detail=f"{self.provider} is not a supported provider.",
code="invalid",
pointer="/data/attributes/provider",
)
if self.provider == self.ProviderChoices.OKTA and self.uid:
# Mirror the SDK, which lowercases the org domain before connecting.
# Without this the API would reject Acme.okta.com even though the
# SDK would accept it, and stored uids could disagree with the
# authenticated org domain.
self.uid = self.uid.strip().lower()
# Providers the SDK exposes but the API has no specific uid rule for
# (e.g. external providers) fall back to the field-level min-length
# check only, instead of failing on a missing validator.
uid_validator = getattr(self, f"validate_{self.provider}_uid", None)
if uid_validator is not None:
uid_validator(self.uid)
getattr(self, f"validate_{self.provider}_uid")(self.uid)
def save(self, *args, **kwargs):
self.full_clean()
@@ -678,9 +666,6 @@ class Scan(RowLevelSecurityProtectedModel):
state = StateEnumField(choices=StateChoices.choices, default=StateChoices.AVAILABLE)
unique_resource_count = models.IntegerField(default=0)
progress = models.IntegerField(default=0)
# Incremented by the scan-specific orphan-recovery path each time this scan is
# re-pointed to a fresh task; for observability (the retry cap is a Valkey counter).
recovery_count = models.IntegerField(default=0)
scanner_args = models.JSONField(default=dict)
duration = models.IntegerField(null=True, blank=True)
scheduled_at = models.DateTimeField(null=True, blank=True)
@@ -2013,35 +1998,6 @@ class IntegrationProviderRelationship(RowLevelSecurityProtectedModel):
]
class JiraIssueDispatch(RowLevelSecurityProtectedModel):
"""Tracks findings already sent to a Jira integration.
Lets the Jira task be re-run safely (e.g. by orphan recovery): findings with
an existing dispatch row are skipped, so no duplicate issues are created.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
integration = models.ForeignKey(
Integration, on_delete=models.CASCADE, related_name="jira_dispatches"
)
finding_id = models.UUIDField()
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "jira_issue_dispatches"
constraints = [
models.UniqueConstraint(
fields=["tenant_id", "integration_id", "finding_id"],
name="unique_jira_issue_dispatch",
),
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
class SAMLToken(models.Model):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
-15
View File
@@ -1,15 +0,0 @@
from functools import lru_cache
from prowler.providers.common.provider import Provider as SDKProvider
@lru_cache(maxsize=1)
def get_provider_type_choices():
"""Provider-type choices from the SDK's available providers, so they cover
external providers and not just a static enum.
Cached for the process lifetime; hot-installing a provider needs
coordinated cache invalidation (tracked separately) to show up here without
a restart. Shared by the filters and the provider serializer.
"""
return [(name, name) for name in SDKProvider.get_available_providers()]
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.0.3
info:
title: Prowler API
version: 1.31.0
version: 1.32.0
description: |-
Prowler API specification.
+12 -44
View File
@@ -182,23 +182,19 @@ def _make_app():
return ApiConfig("api", api)
def test_ready_initializes_driver_for_api_process(monkeypatch):
@pytest.mark.parametrize(
"argv",
[
["gunicorn"],
["celery", "-A", "api"],
["manage.py", "migrate"],
],
ids=["api", "celery", "manage_py"],
)
def test_ready_never_eagerly_initializes_neo4j_driver(monkeypatch, argv):
"""ready() must never contact Neo4j; the driver is created lazily on first use."""
config = _make_app()
_set_argv(monkeypatch, ["gunicorn"])
_set_testing(monkeypatch, False)
with (
patch.object(ApiConfig, "_ensure_crypto_keys", return_value=None),
patch("api.attack_paths.database.init_driver") as init_driver,
):
config.ready()
init_driver.assert_called_once()
def test_ready_skips_driver_for_celery(monkeypatch):
config = _make_app()
_set_argv(monkeypatch, ["celery", "-A", "api"])
_set_argv(monkeypatch, argv)
_set_testing(monkeypatch, False)
with (
@@ -208,31 +204,3 @@ def test_ready_skips_driver_for_celery(monkeypatch):
config.ready()
init_driver.assert_not_called()
def test_ready_skips_driver_for_manage_py_skip_command(monkeypatch):
config = _make_app()
_set_argv(monkeypatch, ["manage.py", "migrate"])
_set_testing(monkeypatch, False)
with (
patch.object(ApiConfig, "_ensure_crypto_keys", return_value=None),
patch("api.attack_paths.database.init_driver") as init_driver,
):
config.ready()
init_driver.assert_not_called()
def test_ready_skips_driver_when_testing(monkeypatch):
config = _make_app()
_set_argv(monkeypatch, ["gunicorn"])
_set_testing(monkeypatch, True)
with (
patch.object(ApiConfig, "_ensure_crypto_keys", return_value=None),
patch("api.attack_paths.database.init_driver") as init_driver,
):
config.ready()
init_driver.assert_not_called()
@@ -1,15 +1,16 @@
"""
Tests for Neo4j database lazy initialization.
The Neo4j driver connects on first use by default. API processes may
eagerly initialize the driver during app startup, while Celery workers
remain lazy. These tests validate the database module behavior itself.
The Neo4j driver is created on first use for every process type; app startup
never contacts Neo4j. These tests validate the database module behavior itself.
"""
import threading
from unittest.mock import MagicMock, patch
import neo4j
import neo4j.exceptions
import pytest
import api.attack_paths.database as db_module
@@ -59,6 +60,32 @@ class TestLazyInitialization:
assert result is mock_driver
assert db_module._driver is mock_driver
@patch("api.attack_paths.database.settings")
@patch("api.attack_paths.database.neo4j.GraphDatabase.driver")
def test_init_driver_leaves_driver_none_when_verify_fails(
self, mock_driver_factory, mock_settings
):
"""A failed verify_connectivity() must not publish or leak the driver."""
mock_driver = MagicMock()
mock_driver.verify_connectivity.side_effect = (
neo4j.exceptions.ServiceUnavailable("down")
)
mock_driver_factory.return_value = mock_driver
mock_settings.DATABASES = {
"neo4j": {
"HOST": "localhost",
"PORT": 7687,
"USER": "neo4j",
"PASSWORD": "password",
}
}
with pytest.raises(neo4j.exceptions.ServiceUnavailable):
db_module.init_driver()
assert db_module._driver is None
mock_driver.close.assert_called_once()
@patch("api.attack_paths.database.settings")
@patch("api.attack_paths.database.neo4j.GraphDatabase.driver")
def test_init_driver_returns_cached_driver_on_subsequent_calls(
@@ -116,21 +143,23 @@ class TestConnectionAcquisitionTimeout:
@pytest.fixture(autouse=True)
def reset_module_state(self):
original_driver = db_module._driver
original_timeout = db_module.CONN_ACQUISITION_TIMEOUT
original_acq_timeout = db_module.CONN_ACQUISITION_TIMEOUT
original_conn_timeout = db_module.CONNECTION_TIMEOUT
db_module._driver = None
yield
db_module._driver = original_driver
db_module.CONN_ACQUISITION_TIMEOUT = original_timeout
db_module.CONN_ACQUISITION_TIMEOUT = original_acq_timeout
db_module.CONNECTION_TIMEOUT = original_conn_timeout
@patch("api.attack_paths.database.settings")
@patch("api.attack_paths.database.neo4j.GraphDatabase.driver")
def test_driver_receives_configured_timeout(
self, mock_driver_factory, mock_settings
):
"""init_driver() should pass CONN_ACQUISITION_TIMEOUT to the neo4j driver."""
"""init_driver() should pass the configured timeouts to the neo4j driver."""
mock_driver_factory.return_value = MagicMock()
mock_settings.DATABASES = {
"neo4j": {
@@ -141,11 +170,13 @@ class TestConnectionAcquisitionTimeout:
}
}
db_module.CONN_ACQUISITION_TIMEOUT = 42
db_module.CONNECTION_TIMEOUT = 7
db_module.init_driver()
_, kwargs = mock_driver_factory.call_args
assert kwargs["connection_acquisition_timeout"] == 42
assert kwargs["connection_timeout"] == 7
class TestAtexitRegistration:
@@ -511,3 +542,84 @@ class TestHasProviderData:
):
with pytest.raises(db_module.GraphDatabaseQueryException):
db_module.has_provider_data("db-tenant-abc", "provider-123")
class TestDropSubgraph:
"""Test drop_subgraph two-phase batched deletion of a provider's graph."""
@staticmethod
def _result(count):
result = MagicMock()
result.single.return_value.get.return_value = count
return result
@staticmethod
def _session_ctx(session):
ctx = MagicMock()
ctx.__enter__.return_value = session
ctx.__exit__.return_value = False
return ctx
def test_deletes_relationships_then_nodes_in_batches(self):
session = MagicMock()
# Phase 1 (relationships): one full batch then empty.
# Phase 2 (nodes): one full batch then empty.
session.run.side_effect = [
self._result(1000),
self._result(0),
self._result(1000),
self._result(0),
]
with patch(
"api.attack_paths.database.get_session",
return_value=self._session_ctx(session),
):
deleted = db_module.drop_subgraph("db-tenant-abc", "provider-123")
# Only phase-2 node counts contribute to the return value.
assert deleted == 1000
assert session.run.call_count == 4
queries = [call.args[0] for call in session.run.call_args_list]
# Regression guard: the memory blow-up was caused by DETACH DELETE.
assert all("DETACH DELETE" not in query for query in queries)
rel_queries = [query for query in queries if "DELETE r" in query]
node_queries = [query for query in queries if "DELETE n" in query]
assert rel_queries and node_queries
# DISTINCT avoids double-counting relationships matched from both ends.
assert all("DISTINCT r" in query for query in rel_queries)
# Relationships must be fully drained before nodes are deleted.
first_node = next(i for i, q in enumerate(queries) if "DELETE n" in q)
last_rel = max(i for i, q in enumerate(queries) if "DELETE r" in q)
assert last_rel < first_node
def test_returns_zero_when_database_not_found(self):
session_ctx = MagicMock()
session_ctx.__enter__.side_effect = db_module.GraphDatabaseQueryException(
message="Database does not exist",
code="Neo.ClientError.Database.DatabaseNotFound",
)
with patch(
"api.attack_paths.database.get_session",
return_value=session_ctx,
):
assert db_module.drop_subgraph("db-tenant-gone", "provider-123") == 0
def test_raises_on_other_errors(self):
session_ctx = MagicMock()
session_ctx.__enter__.side_effect = db_module.GraphDatabaseQueryException(
message="Connection refused",
code="Neo.TransientError.General.UnknownError",
)
with patch(
"api.attack_paths.database.get_session",
return_value=session_ctx,
):
with pytest.raises(db_module.GraphDatabaseQueryException):
db_module.drop_subgraph("db-tenant-abc", "provider-123")
@@ -10,6 +10,7 @@ from api.compliance import (
get_prowler_provider_checks,
get_prowler_provider_compliance,
load_prowler_checks,
warm_compliance_caches,
)
from api.models import Provider
from prowler.lib.check.compliance_models import (
@@ -267,11 +268,17 @@ def reset_compliance_cache():
"""Reset the module-level cache so each test starts cold."""
previous = dict(compliance_module.AVAILABLE_COMPLIANCE_FRAMEWORKS)
compliance_module.AVAILABLE_COMPLIANCE_FRAMEWORKS.clear()
# The warming flags are module-global; clear them so they do not leak
# between tests that call warm_compliance_caches.
compliance_module.COMPLIANCE_WARMING_STARTED.clear()
compliance_module.COMPLIANCE_WARMED.clear()
try:
yield
finally:
compliance_module.AVAILABLE_COMPLIANCE_FRAMEWORKS.clear()
compliance_module.AVAILABLE_COMPLIANCE_FRAMEWORKS.update(previous)
compliance_module.COMPLIANCE_WARMING_STARTED.clear()
compliance_module.COMPLIANCE_WARMED.clear()
class TestGetComplianceFrameworks:
@@ -321,3 +328,89 @@ class TestGetComplianceFrameworks:
f"loadable by get_bulk_compliance_frameworks_universal: "
f"{sorted(missing)}"
)
class TestWarmComplianceCaches:
def test_warms_all_provider_types_by_default(self, reset_compliance_cache):
provider_types = list(Provider.ProviderChoices.values)
with (
patch("api.compliance.get_compliance_frameworks") as mock_frameworks,
patch("api.compliance._ensure_provider_loaded") as mock_ensure,
):
warm_compliance_caches()
warmed = {call.args[0] for call in mock_frameworks.call_args_list}
assert warmed == set(provider_types)
assert mock_frameworks.call_count == len(provider_types)
assert mock_ensure.call_count == len(provider_types)
def test_warms_only_requested_provider_types(self, reset_compliance_cache):
with (
patch("api.compliance.get_compliance_frameworks") as mock_frameworks,
patch("api.compliance._ensure_provider_loaded") as mock_ensure,
):
warm_compliance_caches([Provider.ProviderChoices.AWS])
mock_frameworks.assert_called_once_with(Provider.ProviderChoices.AWS)
mock_ensure.assert_called_once_with(Provider.ProviderChoices.AWS)
def test_populates_module_cache(self, reset_compliance_cache):
with (
patch(
"api.compliance.get_bulk_compliance_frameworks_universal"
) as mock_get_bulk,
patch("api.compliance._ensure_provider_loaded"),
):
mock_get_bulk.return_value = {"cis_1.4_aws": MagicMock()}
warm_compliance_caches([Provider.ProviderChoices.AWS])
assert (
Provider.ProviderChoices.AWS
in compliance_module.AVAILABLE_COMPLIANCE_FRAMEWORKS
)
def test_failing_provider_does_not_abort_the_rest(self, reset_compliance_cache):
"""A failing provider (even on SystemExit) is isolated; others warm."""
providers = [Provider.ProviderChoices.AWS, Provider.ProviderChoices.OKTA]
def fake_frameworks(provider_type):
if provider_type == Provider.ProviderChoices.OKTA:
raise SystemExit(1)
return []
with (
patch(
"api.compliance.get_compliance_frameworks", side_effect=fake_frameworks
),
patch("api.compliance._ensure_provider_loaded") as mock_ensure,
):
failed = warm_compliance_caches(providers)
assert failed == [Provider.ProviderChoices.OKTA]
mock_ensure.assert_called_once_with(Provider.ProviderChoices.AWS)
def test_sets_readiness_flags(self, reset_compliance_cache):
assert not compliance_module.COMPLIANCE_WARMING_STARTED.is_set()
assert not compliance_module.COMPLIANCE_WARMED.is_set()
with (
patch("api.compliance.get_compliance_frameworks"),
patch("api.compliance._ensure_provider_loaded"),
):
warm_compliance_caches([Provider.ProviderChoices.AWS])
assert compliance_module.COMPLIANCE_WARMING_STARTED.is_set()
assert compliance_module.COMPLIANCE_WARMED.is_set()
def test_marks_warmed_even_when_a_provider_fails(self, reset_compliance_cache):
"""A failed provider still leaves the caches flagged as warmed."""
with (
patch(
"api.compliance.get_compliance_frameworks",
side_effect=SystemExit(1),
),
patch("api.compliance._ensure_provider_loaded"),
):
warm_compliance_caches([Provider.ProviderChoices.AWS])
assert compliance_module.COMPLIANCE_WARMED.is_set()
@@ -0,0 +1,55 @@
from config.django.base import label_postgres_connections
class TestLabelPostgresConnections:
def test_labels_postgres_and_skips_neo4j(self, monkeypatch):
monkeypatch.setenv("DJANGO_APP_COMPONENT", "scan")
databases = {
"default": {"ENGINE": "psqlextra.backend"},
"neo4j": {"HOST": "neo4j", "PORT": "7687"},
}
label_postgres_connections(databases)
assert databases["default"]["OPTIONS"]["application_name"] == "scan:default"
assert "OPTIONS" not in databases["neo4j"]
def test_labels_plain_postgresql_backend(self, monkeypatch):
monkeypatch.setenv("DJANGO_APP_COMPONENT", "api")
databases = {"saas": {"ENGINE": "django.db.backends.postgresql"}}
label_postgres_connections(databases)
assert databases["saas"]["OPTIONS"]["application_name"] == "api:saas"
def test_defaults_component_to_api_when_unset(self, monkeypatch):
monkeypatch.delenv("DJANGO_APP_COMPONENT", raising=False)
databases = {"default": {"ENGINE": "psqlextra.backend"}}
label_postgres_connections(databases)
assert databases["default"]["OPTIONS"]["application_name"] == "api:default"
def test_preserves_existing_options(self, monkeypatch):
monkeypatch.setenv("DJANGO_APP_COMPONENT", "worker")
databases = {
"replica": {
"ENGINE": "psqlextra.backend",
"OPTIONS": {"sslmode": "require"},
}
}
label_postgres_connections(databases)
assert databases["replica"]["OPTIONS"] == {
"sslmode": "require",
"application_name": "worker:replica",
}
def test_truncates_application_name_to_63_bytes(self, monkeypatch):
monkeypatch.setenv("DJANGO_APP_COMPONENT", "c" * 80)
databases = {"default": {"ENGINE": "psqlextra.backend"}}
label_postgres_connections(databases)
assert len(databases["default"]["OPTIONS"]["application_name"]) == 63
-23
View File
@@ -1,23 +0,0 @@
from api.filters import get_provider_type_choices
from prowler.providers.common.provider import Provider as SDKProvider
class TestProviderTypeChoices:
"""Provider-type filter choices are driven by the SDK's available providers
so filtering covers external providers, not just a static enum."""
def test_choices_track_sdk_available_providers(self):
available = set(SDKProvider.get_available_providers())
choices = get_provider_type_choices()
assert {value for value, _ in choices} == available
def test_choices_include_provider_absent_from_legacy_enum(self):
from api.models import Provider
legacy = {value for value, _ in Provider.ProviderChoices.choices}
choice_values = {value for value, _ in get_provider_type_choices()}
# `llm` is exposed by the SDK but is not part of the legacy static enum.
assert "llm" in choice_values
assert "llm" not in legacy
-28
View File
@@ -6,9 +6,7 @@ from django.core.exceptions import ValidationError
from django.db import IntegrityError
from api.db_router import MainRouter
from api.exceptions import ModelValidationError
from api.models import (
Provider,
ProviderComplianceScore,
Resource,
ResourceTag,
@@ -527,29 +525,3 @@ class TestTenantComplianceSummaryModel:
assert summary1.id != summary2.id
assert summary1.requirements_passed != summary2.requirements_passed
@pytest.mark.django_db
class TestProviderDynamicValidation:
"""Provider validity is driven by the SDK's available providers, not a
static enum. Providers the SDK exposes are accepted; for those without a
`validate_<provider>_uid` method only the uid min-length floor applies."""
def test_accepts_provider_without_uid_validator(self, tenants_fixture):
tenant = tenants_fixture[0]
provider = Provider.objects.create(
tenant_id=tenant.id, provider="llm", uid="my-llm-account"
)
assert provider.provider == "llm"
def test_rejects_provider_not_available_in_sdk(self, tenants_fixture):
tenant = tenants_fixture[0]
with pytest.raises(ModelValidationError):
Provider.objects.create(
tenant_id=tenant.id, provider="does-not-exist", uid="whatever"
)
def test_uid_floor_still_enforced_for_external_provider(self, tenants_fixture):
tenant = tenants_fixture[0]
with pytest.raises(ValidationError):
Provider.objects.create(tenant_id=tenant.id, provider="llm", uid="ab")
+1 -96
View File
@@ -1,103 +1,8 @@
from unittest.mock import MagicMock, patch
import pytest
from pydantic import BaseModel
from rest_framework.exceptions import ValidationError
from api.v1.serializer_utils.integrations import S3ConfigSerializer
from api.v1.serializers import (
BaseWriteProviderSecretSerializer,
ImageProviderSecret,
ProviderEnumSerializerField,
)
class TestExternalProviderSecretValidation:
"""A non-built-in provider's secret is validated against the schema it
declares for the chosen secret type through the SDK contract, or accepted as
an object when it declares none (then validated by test_connection)."""
class _StaticCredentials(BaseModel):
api_url: str
api_key: str
class _RoleCredentials(BaseModel):
role_arn: str
def _patch(self, schemas):
provider_class = MagicMock()
provider_class.get_credentials_schema.return_value = schemas
return patch(
"api.v1.serializers.SDKProvider.get_class", return_value=provider_class
)
def test_secret_validated_against_its_type_schema(self):
with self._patch({"static": self._StaticCredentials}):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "static", {"api_url": "u", "api_key": "k"}
)
def test_secret_rejected_when_schema_violated(self):
with self._patch({"static": self._StaticCredentials}):
with pytest.raises(ValidationError):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "static", {"api_url": "u"}
)
def test_secret_must_match_its_type_not_another(self):
"""A secret is validated against the schema for its declared secret_type,
not "any declared schema": a role-shaped secret under secret_type=static
is rejected. See PR #11402 review (josema-xyz / Alan-TheGentleman)."""
schemas = {"static": self._StaticCredentials, "role": self._RoleCredentials}
with self._patch(schemas):
with pytest.raises(ValidationError):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "static", {"role_arn": "arn:aws:iam::x"}
)
def test_rejects_secret_type_not_declared_by_provider(self):
with self._patch({"static": self._StaticCredentials}):
with pytest.raises(ValidationError):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "role", {"role_arn": "arn"}
)
def test_secret_accepted_when_no_schema_declared(self):
with self._patch({}):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "static", {"anything": "goes"}
)
@pytest.mark.parametrize("bad_secret", [["a", "b"], "a-string", None, 42])
def test_secret_rejected_when_not_a_json_object(self, bad_secret):
"""Even with no declared schema, a non-object secret must be rejected so
a list/string/null cannot be persisted and blow up later at
``{**secret}``. See PR #11402 review (Alan-TheGentleman)."""
with self._patch({}):
with pytest.raises(ValidationError):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
"external-template", "static", bad_secret
)
class TestProviderEnumSerializerField:
"""The provider field accepts whatever the SDK exposes (built-in or
external) and rejects anything else with `invalid_choice`."""
def test_accepts_sdk_available_provider(self):
field = ProviderEnumSerializerField()
assert field.run_validation("aws") == "aws"
def test_accepts_external_provider_absent_from_static_enum(self):
field = ProviderEnumSerializerField()
# `llm` is exposed by the SDK but is not part of the legacy static enum.
assert field.run_validation("llm") == "llm"
def test_rejects_unknown_provider(self):
field = ProviderEnumSerializerField()
with pytest.raises(ValidationError) as exc:
field.run_validation("does-not-exist")
assert exc.value.detail[0].code == "invalid_choice"
from api.v1.serializers import ImageProviderSecret
class TestS3ConfigSerializer:
+27 -63
View File
@@ -146,25 +146,11 @@ class TestReturnProwlerProvider:
with pytest.raises(ValueError):
return return_prowler_provider(provider)
def test_return_prowler_provider_external_resolves_via_get_class(self):
"""An external provider name resolves through the SDK resolver, so any
entry-point provider works without a hardcoded branch in the API."""
external_class = type("ExternalProvider", (), {})
provider = MagicMock()
provider.provider = "external-template"
with patch(
"api.utils.SDKProvider.get_class", return_value=external_class
) as mock_get_class:
resolved = return_prowler_provider(provider)
mock_get_class.assert_called_once_with("external-template")
assert resolved is external_class
class TestInitializeProwlerProvider:
@patch("api.utils.return_prowler_provider")
def test_initialize_prowler_provider(self, mock_return_prowler_provider):
provider = MagicMock()
provider.provider = "aws"
provider.secret.secret = {"key": "value"}
mock_return_prowler_provider.return_value = MagicMock()
@@ -176,7 +162,6 @@ class TestInitializeProwlerProvider:
self, mock_return_prowler_provider
):
provider = MagicMock()
provider.provider = "aws"
provider.secret.secret = {"key": "value"}
mutelist_processor = MagicMock()
mutelist_processor.configuration = {"Mutelist": {"key": "value"}}
@@ -192,7 +177,6 @@ class TestProwlerProviderConnectionTest:
@patch("api.utils.return_prowler_provider")
def test_prowler_provider_connection_test(self, mock_return_prowler_provider):
provider = MagicMock()
provider.provider = "aws"
provider.uid = "1234567890"
provider.secret.secret = {"key": "value"}
mock_return_prowler_provider.return_value = MagicMock()
@@ -202,29 +186,6 @@ class TestProwlerProviderConnectionTest:
key="value", provider_id="1234567890", raise_on_exception=False
)
@patch("api.utils.return_prowler_provider")
def test_prowler_provider_connection_test_external_uses_contract(
self, mock_return_prowler_provider
):
"""External providers build connection kwargs through the SDK contract
(get_connection_arguments), with no provider_id forced by the API."""
provider = MagicMock()
provider.provider = "external-template"
provider.uid = "acme-1"
provider.secret.secret = {"api_key": "k"}
mock_return_prowler_provider.return_value.get_connection_arguments.return_value = {
"api_key": "k"
}
prowler_provider_connection_test(provider)
mock_return_prowler_provider.return_value.get_connection_arguments.assert_called_once_with(
"acme-1", {"api_key": "k"}
)
mock_return_prowler_provider.return_value.test_connection.assert_called_once_with(
api_key="k", raise_on_exception=False
)
@pytest.mark.django_db
@patch("api.utils.return_prowler_provider")
def test_prowler_provider_connection_test_without_secret(
@@ -396,6 +357,30 @@ class TestGetProwlerProviderKwargs:
expected_result = {**secret_dict, **expected_extra_kwargs}
assert result == expected_result
def test_get_prowler_provider_kwargs_oraclecloud_converts_region_string_to_set(
self,
):
secret_dict = {
"user": "ocid1.user.oc1..fake",
"fingerprint": "00:11:22:33:44:55:66:77",
"key_content": "-----BEGIN PRIVATE KEY-----\nfake\n-----END PRIVATE KEY-----",
"tenancy": "ocid1.tenancy.oc1..fake",
"region": "us-ashburn-1",
"pass_phrase": "fake-passphrase",
}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.ORACLECLOUD.value
provider.secret = secret_mock
provider.uid = "ocid1.tenancy.oc1..fake"
result = get_prowler_provider_kwargs(provider)
expected_result = {**secret_dict, "region": {"us-ashburn-1"}}
assert result == expected_result
def test_get_prowler_provider_kwargs_with_mutelist(self):
provider_uid = "provider_uid"
secret_dict = {"key": "value"}
@@ -599,8 +584,7 @@ class TestGetProwlerProviderKwargs:
assert result == expected_result
def test_get_prowler_provider_kwargs_unsupported_provider(self):
# A non-built-in provider is resolved dynamically; one that is neither a
# built-in nor an installed entry-point provider cannot be resolved.
# Setup
provider_uid = "provider_uid"
secret_dict = {"key": "value"}
secret_mock = MagicMock()
@@ -611,30 +595,10 @@ class TestGetProwlerProviderKwargs:
provider.secret = secret_mock
provider.uid = provider_uid
with pytest.raises(ValueError):
get_prowler_provider_kwargs(provider)
@patch("api.utils.return_prowler_provider")
def test_get_prowler_provider_kwargs_external_uses_contract(
self, mock_return_prowler_provider
):
"""External providers build constructor kwargs through the SDK contract
(get_scan_arguments), not a hardcoded branch in the API."""
provider = MagicMock()
provider.provider = "external-template"
provider.uid = "acme-1"
provider.secret.secret = {"api_key": "k"}
mock_return_prowler_provider.return_value.get_scan_arguments.return_value = {
"api_key": "k",
"custom": "acme-1",
}
result = get_prowler_provider_kwargs(provider)
mock_return_prowler_provider.return_value.get_scan_arguments.assert_called_once_with(
"acme-1", {"api_key": "k"}, None
)
assert result == {"api_key": "k", "custom": "acme-1"}
expected_result = secret_dict.copy()
assert result == expected_result
def test_get_prowler_provider_kwargs_no_secret(self):
# Setup
+673 -67
View File
@@ -1411,6 +1411,42 @@ class TestProviderViewSet:
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_providers_filter_provider_groups(
self,
authenticated_client,
tenants_fixture,
providers_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider2, provider_group=group2
)
response = authenticated_client.get(
reverse("provider-list"), {"filter[provider_groups]": str(group1.id)}
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert [item["id"] for item in data] == [str(provider1.id)]
response = authenticated_client.get(
reverse("provider-list"),
{"filter[provider_groups__in]": f"{group1.id},{group2.id}"},
)
assert response.status_code == status.HTTP_200_OK
provider_ids = {item["id"] for item in response.json()["data"]}
assert provider_ids == {str(provider1.id), str(provider2.id)}
assert len(response.json()["data"]) == 2
def test_providers_disable_pagination(
self, authenticated_client, providers_fixture, tenants_fixture
):
@@ -1472,9 +1508,9 @@ class TestProviderViewSet:
included_data = response.json()["included"]
for expected_type in expected_resources:
assert any(
d.get("type") == expected_type for d in included_data
), f"Expected type '{expected_type}' not found in included data"
assert any(d.get("type") == expected_type for d in included_data), (
f"Expected type '{expected_type}' not found in included data"
)
def test_providers_retrieve(self, authenticated_client, providers_fixture):
provider1, *_ = providers_fixture
@@ -3715,6 +3751,41 @@ class TestScanViewSet:
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == expected_count
def test_scans_filter_provider_groups(
self,
authenticated_client,
tenants_fixture,
scans_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
scan1, scan2, *_ = scans_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=scan1.provider, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=scan1.provider, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=scan2.provider, provider_group=group2
)
response = authenticated_client.get(
reverse("scan-list"), {"filter[provider_groups]": str(group1.id)}
)
assert response.status_code == status.HTTP_200_OK
assert {item["id"] for item in response.json()["data"]} == {str(scan1.id)}
response = authenticated_client.get(
reverse("scan-list"),
{"filter[provider_groups__in]": f"{group1.id},{group2.id}"},
)
assert response.status_code == status.HTTP_200_OK
scan_ids = {item["id"] for item in response.json()["data"]}
assert scan_ids == {str(scan1.id), str(scan2.id), str(scans_fixture[2].id)}
assert len(response.json()["data"]) == 3
@pytest.mark.parametrize(
"filter_name",
[
@@ -5714,13 +5785,13 @@ class TestAttackPathsScanViewSet:
content_type=API_JSON_CONTENT_TYPE,
)
if i < 10:
assert (
response.status_code == status.HTTP_200_OK
), f"Request {i + 1} should succeed with 200 OK, got {response.status_code}"
assert response.status_code == status.HTTP_200_OK, (
f"Request {i + 1} should succeed with 200 OK, got {response.status_code}"
)
else:
assert (
response.status_code == status.HTTP_429_TOO_MANY_REQUESTS
), f"Request {i + 1} should be throttled"
assert response.status_code == status.HTTP_429_TOO_MANY_REQUESTS, (
f"Request {i + 1} should be throttled"
)
# -- Timeout simulation -------------------------------------------------------
@@ -5923,9 +5994,9 @@ class TestResourceViewSet:
included_data = response.json()["included"]
for expected_type in expected_resources:
assert any(
d.get("type") == expected_type for d in included_data
), f"Expected type '{expected_type}' not found in included data"
assert any(d.get("type") == expected_type for d in included_data), (
f"Expected type '{expected_type}' not found in included data"
)
@pytest.mark.parametrize(
"filter_name, filter_value, expected_count",
@@ -5996,6 +6067,49 @@ class TestResourceViewSet:
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == expected_count
def test_resource_filter_provider_groups(
self,
authenticated_client,
tenants_fixture,
resources_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
resource1, resource2, resource3, *_ = resources_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=resource1.provider, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=resource1.provider, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=resource3.provider, provider_group=group2
)
response = authenticated_client.get(
reverse("resource-list"),
{"filter[updated_at]": TODAY, "filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 2
assert {item["id"] for item in response.json()["data"]} == {
str(resource1.id),
str(resource2.id),
}
response = authenticated_client.get(
reverse("resource-list"),
{
"filter[updated_at]": TODAY,
"filter[provider_groups__in]": f"{group1.id},{group2.id}",
},
)
assert response.status_code == status.HTTP_200_OK
resource_ids = {item["id"] for item in response.json()["data"]}
assert resource_ids == {str(resource1.id), str(resource2.id), str(resource3.id)}
assert len(response.json()["data"]) == 3
def test_resource_filter_by_scan_id(
self, authenticated_client, resources_fixture, scans_fixture
):
@@ -6474,9 +6588,9 @@ class TestResourceViewSet:
(e for e in errors if e["source"]["parameter"] == expected_invalid_param),
None,
)
assert (
error is not None
), f"Expected error for parameter '{expected_invalid_param}'"
assert error is not None, (
f"Expected error for parameter '{expected_invalid_param}'"
)
assert error["code"] == "invalid"
assert error["status"] == "400" # Must be string per JSON:API spec
assert expected_invalid_param in error["detail"]
@@ -7008,16 +7122,16 @@ class TestResourceViewSet:
# Test with completely malformed token
client.credentials(HTTP_AUTHORIZATION="Bearer not.a.valid.jwt.token")
response = client.get(reverse("resource-events", kwargs={"pk": resource.id}))
assert (
response.status_code == status.HTTP_401_UNAUTHORIZED
), f"Expected 401 for malformed token but got {response.status_code}"
assert response.status_code == status.HTTP_401_UNAUTHORIZED, (
f"Expected 401 for malformed token but got {response.status_code}"
)
# Test with empty bearer token
client.credentials(HTTP_AUTHORIZATION="Bearer ")
response = client.get(reverse("resource-events", kwargs={"pk": resource.id}))
assert (
response.status_code == status.HTTP_401_UNAUTHORIZED
), f"Expected 401 for empty bearer token but got {response.status_code}"
assert response.status_code == status.HTTP_401_UNAUTHORIZED, (
f"Expected 401 for empty bearer token but got {response.status_code}"
)
@pytest.mark.django_db
@@ -7152,9 +7266,9 @@ class TestFindingViewSet:
included_data = response.json()["included"]
for expected_type in expected_resources:
assert any(
d.get("type") == expected_type for d in included_data
), f"Expected type '{expected_type}' not found in included data"
assert any(d.get("type") == expected_type for d in included_data), (
f"Expected type '{expected_type}' not found in included data"
)
@pytest.mark.parametrize(
"filter_name, filter_value, expected_count",
@@ -7308,6 +7422,40 @@ class TestFindingViewSet:
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 2
def test_finding_filter_provider_groups(
self,
authenticated_client,
tenants_fixture,
findings_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
finding1, finding2, *_ = findings_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=finding1.scan.provider, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=finding1.scan.provider, provider_group=group2
)
response = authenticated_client.get(
reverse("finding-list"),
{"filter[inserted_at]": TODAY, "filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 2
response = authenticated_client.get(
reverse("finding-list"),
{
"filter[inserted_at]": TODAY,
"filter[provider_groups__in]": f"{group1.id},{group2.id}",
},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 2
@pytest.mark.parametrize(
"filter_name",
(
@@ -7719,9 +7867,9 @@ class TestJWTFields:
reverse("token-obtain"), data, format="json"
)
assert (
response.status_code == status.HTTP_200_OK
), f"Unexpected status code: {response.status_code}"
assert response.status_code == status.HTTP_200_OK, (
f"Unexpected status code: {response.status_code}"
)
access_token = response.data["attributes"]["access"]
payload = jwt.decode(access_token, options={"verify_signature": False})
@@ -7735,23 +7883,23 @@ class TestJWTFields:
# Verify expected fields
for field in expected_fields:
assert field in payload, f"The field '{field}' is not in the JWT"
assert (
payload[field] == expected_fields[field]
), f"The value of '{field}' does not match"
assert payload[field] == expected_fields[field], (
f"The value of '{field}' does not match"
)
# Verify time fields are integers
for time_field in ["exp", "iat", "nbf"]:
assert time_field in payload, f"The field '{time_field}' is not in the JWT"
assert isinstance(
payload[time_field], int
), f"The field '{time_field}' is not an integer"
assert isinstance(payload[time_field], int), (
f"The field '{time_field}' is not an integer"
)
# Verify identification fields are non-empty strings
for id_field in ["jti", "sub", "tenant_id"]:
assert id_field in payload, f"The field '{id_field}' is not in the JWT"
assert (
isinstance(payload[id_field], str) and payload[id_field]
), f"The field '{id_field}' is not a valid string"
assert isinstance(payload[id_field], str) and payload[id_field], (
f"The field '{id_field}' is not a valid string"
)
@pytest.mark.django_db
@@ -9570,6 +9718,188 @@ class TestComplianceOverviewViewSet:
assert "Category" in first_attr
assert "AWSService" in first_attr
def test_compliance_overview_attributes_resolves_provider_from_scan(
self, authenticated_client, tenants_fixture, providers_fixture
):
# csa_ccm_4.0 is a multi-provider universal framework: a single
# compliance_id whose requirements expose different checks per provider.
# Passing a scan must return the check IDs for that scan's provider,
# otherwise the endpoint defaults to the first provider that declares the
# framework and azure/gcp requirements end up with check IDs that match
# no findings.
tenant = tenants_fixture[0]
gcp_provider = providers_fixture[2]
azure_provider = providers_fixture[4]
assert gcp_provider.provider == Provider.ProviderChoices.GCP.value
assert azure_provider.provider == Provider.ProviderChoices.AZURE.value
now = datetime.now(timezone.utc)
gcp_scan = Scan.objects.create(
name="gcp scan",
provider=gcp_provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=tenant.id,
started_at=now,
completed_at=now,
)
azure_scan = Scan.objects.create(
name="azure scan",
provider=azure_provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=tenant.id,
started_at=now,
completed_at=now,
)
def request_attributes(scan_id=None):
params = {"filter[compliance_id]": "csa_ccm_4.0"}
if scan_id is not None:
params["filter[scan_id]"] = str(scan_id)
return authenticated_client.get(
reverse("complianceoverview-attributes"), params
)
def collect_check_ids(scan_id=None):
response = request_attributes(scan_id)
assert response.status_code == status.HTTP_200_OK
check_ids = set()
for item in response.json()["data"]:
check_ids.update(item["attributes"]["attributes"]["check_ids"])
return check_ids
gcp_check_ids = collect_check_ids(gcp_scan.id)
azure_check_ids = collect_check_ids(azure_scan.id)
# Each scan resolves to its own provider's checks, and they differ.
assert gcp_check_ids
assert azure_check_ids
assert gcp_check_ids != azure_check_ids
# The returned check IDs belong to the SDK's per-provider definition.
from api.compliance import get_prowler_provider_compliance
def expected_check_ids(provider_type):
framework = get_prowler_provider_compliance(provider_type)["csa_ccm_4.0"]
expected = set()
for requirement in framework.requirements:
expected.update(requirement.checks.get(provider_type, []))
return expected
assert gcp_check_ids <= expected_check_ids(Provider.ProviderChoices.GCP.value)
assert azure_check_ids <= expected_check_ids(
Provider.ProviderChoices.AZURE.value
)
# An explicit scan_id is authoritative: a non-existent scan must fail
# closed with 404 instead of silently falling back to another provider.
missing_response = request_attributes("00000000-0000-0000-0000-000000000000")
assert missing_response.status_code == status.HTTP_404_NOT_FOUND
# A malformed scan_id is rejected with 404 as well.
malformed_response = request_attributes("not-a-uuid")
assert malformed_response.status_code == status.HTTP_404_NOT_FOUND
# An empty value (filter[scan_id]=) must not fall back to the legacy
# provider picker: the explicit (if blank) selector fails closed.
empty_response = request_attributes("")
assert empty_response.status_code == status.HTTP_404_NOT_FOUND
# A scan belonging to another tenant is not visible (RLS), so it must
# return 404 rather than leaking the fallback provider's check IDs.
other_tenant = Tenant.objects.create(name="Other Compliance Tenant")
foreign_provider = Provider.objects.create(
provider="gcp",
uid="foreign-gcp-test",
alias="foreign_gcp",
tenant_id=other_tenant.id,
)
foreign_scan = Scan.objects.create(
name="foreign scan",
provider=foreign_provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=other_tenant.id,
started_at=now,
completed_at=now,
)
foreign_response = request_attributes(foreign_scan.id)
assert foreign_response.status_code == status.HTTP_404_NOT_FOUND
def test_compliance_overview_attributes_scan_scoped_by_provider_group(
self,
authenticated_client_no_permissions_rbac,
providers_fixture,
):
# A user with limited visibility (no UNLIMITED_VISIBILITY) must only be
# able to resolve scans for providers in its provider groups. Tenant RLS
# alone is not enough here: both scans belong to the same tenant, so the
# endpoint has to scope the scan lookup by provider group, otherwise a
# restricted user could read another provider's compliance metadata.
client = authenticated_client_no_permissions_rbac
limited_user = client.user
membership = Membership.objects.filter(user=limited_user).first()
tenant = membership.tenant
allowed_provider = providers_fixture[2]
denied_provider = providers_fixture[4]
assert allowed_provider.provider == Provider.ProviderChoices.GCP.value
assert denied_provider.provider == Provider.ProviderChoices.AZURE.value
provider_group = ProviderGroup.objects.create(
name="limited-compliance-group",
tenant_id=tenant.id,
)
ProviderGroupMembership.objects.create(
tenant_id=tenant.id,
provider_group=provider_group,
provider=allowed_provider,
)
RoleProviderGroupRelationship.objects.create(
tenant_id=tenant.id,
role=limited_user.roles.first(),
provider_group=provider_group,
)
now = datetime.now(timezone.utc)
allowed_scan = Scan.objects.create(
name="allowed scan",
provider=allowed_provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=tenant.id,
started_at=now,
completed_at=now,
)
denied_scan = Scan.objects.create(
name="denied scan",
provider=denied_provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=tenant.id,
started_at=now,
completed_at=now,
)
def request_attributes(scan_id):
return client.get(
reverse("complianceoverview-attributes"),
{
"filter[compliance_id]": "csa_ccm_4.0",
"filter[scan_id]": str(scan_id),
},
)
# The scan in the user's provider group resolves normally.
assert request_attributes(allowed_scan.id).status_code == status.HTTP_200_OK
# The scan outside the user's provider group is invisible, so it fails
# closed with 404 instead of leaking the other provider's check IDs.
assert (
request_attributes(denied_scan.id).status_code == status.HTTP_404_NOT_FOUND
)
def test_compliance_overview_attributes_missing_compliance_id(
self, authenticated_client
):
@@ -9578,6 +9908,39 @@ class TestComplianceOverviewViewSet:
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_compliance_overview_attributes_503_while_warming(
self, authenticated_client
):
from api.compliance import COMPLIANCE_WARMED, COMPLIANCE_WARMING_STARTED
COMPLIANCE_WARMING_STARTED.set()
COMPLIANCE_WARMED.clear()
try:
response = authenticated_client.get(
reverse("complianceoverview-attributes"),
{"filter[compliance_id]": "aws_account_security_onboarding_aws"},
)
finally:
COMPLIANCE_WARMING_STARTED.clear()
assert response.status_code == status.HTTP_503_SERVICE_UNAVAILABLE
assert response.json()["errors"][0]["code"] == "compliance_warming"
def test_compliance_overview_attributes_serves_when_warming_not_started(
self, authenticated_client
):
# Dev fallback: under runserver warming never runs, so the guard must
# not refuse — the endpoint lazily loads and serves as before.
from api.compliance import COMPLIANCE_WARMED, COMPLIANCE_WARMING_STARTED
COMPLIANCE_WARMING_STARTED.clear()
COMPLIANCE_WARMED.clear()
response = authenticated_client.get(
reverse("complianceoverview-attributes"),
{"filter[compliance_id]": "aws_account_security_onboarding_aws"},
)
assert response.status_code == status.HTTP_200_OK
def test_compliance_overview_task_management_integration(
self, authenticated_client, compliance_requirements_overviews_fixture
):
@@ -10363,6 +10726,87 @@ class TestOverviewViewSet:
assert combined_attributes["muted"] == 3
assert combined_attributes["total"] == 14
def test_overview_findings_provider_groups_filter(
self,
authenticated_client,
tenants_fixture,
providers_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider2, provider_group=group2
)
scan1 = Scan.objects.create(
name="scan-provider-group-one",
provider=provider1,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
scan2 = Scan.objects.create(
name="scan-provider-group-two",
provider=provider2,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="check-provider-group-one",
service="service-a",
severity="high",
region="region-a",
_pass=5,
fail=1,
muted=2,
total=8,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan2,
check_id="check-provider-group-two",
service="service-b",
severity="medium",
region="region-b",
_pass=2,
fail=3,
muted=1,
total=6,
)
response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
attributes = response.json()["data"]["attributes"]
assert attributes["pass"] == 5
assert attributes["fail"] == 1
assert attributes["muted"] == 2
assert attributes["total"] == 8
response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_groups__in]": f"{group1.id},{group2.id}"},
)
assert response.status_code == status.HTTP_200_OK
attributes = response.json()["data"]["attributes"]
assert attributes["pass"] == 7
assert attributes["fail"] == 4
assert attributes["muted"] == 3
assert attributes["total"] == 14
def test_overview_findings_severity_provider_id_in_filter(
self, authenticated_client, tenants_fixture, providers_fixture
):
@@ -11131,9 +11575,21 @@ class TestOverviewViewSet:
@pytest.mark.parametrize(
"filter_key,filter_value_fn,expected_total,expected_failed",
[
("filter[provider_id]", lambda p1, _: str(p1.id), 10, 5),
("filter[provider_id]", lambda p1, *_: str(p1.id), 10, 5),
("filter[provider_type]", lambda *_: "aws", 10, 5),
("filter[provider_type__in]", lambda *_: "aws,gcp", 30, 20),
(
"filter[provider_groups]",
lambda p1, _, group1, __: str(group1.id),
10,
5,
),
(
"filter[provider_groups__in]",
lambda p1, _, group1, group2: f"{group1.id},{group2.id}",
30,
20,
),
],
)
def test_overview_categories_filters(
@@ -11141,6 +11597,7 @@ class TestOverviewViewSet:
authenticated_client,
tenants_fixture,
providers_fixture,
provider_groups_fixture,
create_scan_category_summary,
filter_key,
filter_value_fn,
@@ -11149,6 +11606,16 @@ class TestOverviewViewSet:
):
tenant = tenants_fixture[0]
provider1, _, gcp_provider, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=gcp_provider, provider_group=group2
)
scan1 = Scan.objects.create(
name="categories-scan-1",
@@ -11174,7 +11641,7 @@ class TestOverviewViewSet:
response = authenticated_client.get(
reverse("overview-categories"),
{filter_key: filter_value_fn(provider1, gcp_provider)},
{filter_key: filter_value_fn(provider1, gcp_provider, group1, group2)},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
@@ -11348,10 +11815,22 @@ class TestOverviewViewSet:
@pytest.mark.parametrize(
"filter_key,filter_value_fn,expected_total,expected_failed",
[
("filter[provider_id]", lambda p1, p2: str(p1.id), 10, 5),
("filter[provider_id__in]", lambda p1, p2: f"{p1.id},{p2.id}", 25, 12),
("filter[provider_type]", lambda p1, p2: "aws", 10, 5),
("filter[provider_type__in]", lambda p1, p2: "aws,gcp", 25, 12),
("filter[provider_id]", lambda p1, *_: str(p1.id), 10, 5),
("filter[provider_id__in]", lambda p1, p2, *_: f"{p1.id},{p2.id}", 25, 12),
("filter[provider_type]", lambda *_: "aws", 10, 5),
("filter[provider_type__in]", lambda *_: "aws,gcp", 25, 12),
(
"filter[provider_groups]",
lambda p1, p2, group1, group2: str(group1.id),
10,
5,
),
(
"filter[provider_groups__in]",
lambda p1, p2, group1, group2: f"{group1.id},{group2.id}",
25,
12,
),
],
)
def test_overview_groups_provider_filters(
@@ -11359,6 +11838,7 @@ class TestOverviewViewSet:
authenticated_client,
tenants_fixture,
providers_fixture,
provider_groups_fixture,
create_scan_resource_group_summary,
filter_key,
filter_value_fn,
@@ -11368,6 +11848,16 @@ class TestOverviewViewSet:
tenant = tenants_fixture[0]
provider1 = providers_fixture[0] # AWS
gcp_provider = providers_fixture[2] # GCP
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=gcp_provider, provider_group=group2
)
scan1 = Scan.objects.create(
name="aws-rg-scan",
@@ -11393,7 +11883,7 @@ class TestOverviewViewSet:
response = authenticated_client.get(
reverse("overview-resource-groups"),
{filter_key: filter_value_fn(provider1, gcp_provider)},
{filter_key: filter_value_fn(provider1, gcp_provider, group1, group2)},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
@@ -11568,6 +12058,49 @@ class TestOverviewViewSet:
data = response.json()["data"]
assert len(data) >= 1
def test_compliance_watchlist_provider_groups_filter(
self,
authenticated_client,
provider_compliance_scores_fixture,
providers_fixture,
provider_groups_fixture,
tenants_fixture,
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider2, provider_group=group2
)
response = authenticated_client.get(
reverse("overview-compliance-watchlist"),
{"filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
by_id = {item["id"]: item["attributes"] for item in data}
assert by_id["aws_cis_2.0"]["requirements_passed"] == 1
assert by_id["aws_cis_2.0"]["requirements_failed"] == 1
assert by_id["aws_cis_2.0"]["requirements_manual"] == 1
response = authenticated_client.get(
reverse("overview-compliance-watchlist"),
{"filter[provider_groups__in]": f"{group1.id},{group2.id}"},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
by_id = {item["id"]: item["attributes"] for item in data}
assert by_id["aws_cis_2.0"]["requirements_passed"] == 0
assert by_id["aws_cis_2.0"]["requirements_failed"] == 2
assert by_id["aws_cis_2.0"]["requirements_manual"] == 1
def test_compliance_watchlist_empty_result(self, authenticated_client):
response = authenticated_client.get(reverse("overview-compliance-watchlist"))
assert response.status_code == status.HTTP_200_OK
@@ -11702,9 +12235,9 @@ class TestIntegrationViewSet:
included_data = response.json()["included"]
for expected_type in expected_resources:
assert any(
d.get("type") == expected_type for d in included_data
), f"Expected type '{expected_type}' not found in included data"
assert any(d.get("type") == expected_type for d in included_data), (
f"Expected type '{expected_type}' not found in included data"
)
@pytest.mark.parametrize(
"integration_type, configuration, credentials",
@@ -13141,9 +13674,9 @@ class TestLighthouseConfigViewSet:
)
# Check that API key is masked with asterisks only
masked_api_key = data["attributes"]["api_key"]
assert all(
c == "*" for c in masked_api_key
), "API key should contain only asterisks"
assert all(c == "*" for c in masked_api_key), (
"API key should contain only asterisks"
)
@pytest.mark.parametrize(
"field_name, invalid_value",
@@ -16720,6 +17253,44 @@ class TestFindingGroupViewSet:
# All fixture findings are from AWS provider
assert len(response.json()["data"]) == 5
def test_finding_groups_provider_groups_filter(
self,
authenticated_client,
tenants_fixture,
finding_groups_fixture,
providers_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider2, provider_group=group2
)
response = authenticated_client.get(
reverse("finding-group-list"),
{"filter[inserted_at]": TODAY, "filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 4
response = authenticated_client.get(
reverse("finding-group-list"),
{
"filter[inserted_at]": TODAY,
"filter[provider_groups__in]": f"{group1.id},{group2.id}",
},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 5
def test_finding_groups_check_id_filter(
self, authenticated_client, finding_groups_fixture
):
@@ -16894,9 +17465,9 @@ class TestFindingGroupViewSet:
assert len(data) == 2
for item in data:
resource = item["attributes"]["resource"]
assert (
resource["resource_group"] == "storage"
), "resource_group must be 'storage'"
assert resource["resource_group"] == "storage", (
"resource_group must be 'storage'"
)
def test_resources_name_icontains(
self, authenticated_client, finding_groups_fixture
@@ -17210,12 +17781,12 @@ class TestFindingGroupViewSet:
assert response_p1.status_code == status.HTTP_200_OK
p1_check_ids = {item["id"] for item in response_p1.json()["data"]}
# Provider1 has scan1 with 4 checks
assert (
len(p1_check_ids) == 4
), f"Provider1 should have 4 checks, got {len(p1_check_ids)}"
assert (
"cloudtrail_enabled" not in p1_check_ids
), "cloudtrail_enabled should NOT be in provider1"
assert len(p1_check_ids) == 4, (
f"Provider1 should have 4 checks, got {len(p1_check_ids)}"
)
assert "cloudtrail_enabled" not in p1_check_ids, (
"cloudtrail_enabled should NOT be in provider1"
)
# Get finding groups for provider2 only
response_p2 = authenticated_client.get(
@@ -17225,12 +17796,12 @@ class TestFindingGroupViewSet:
assert response_p2.status_code == status.HTTP_200_OK
p2_check_ids = {item["id"] for item in response_p2.json()["data"]}
# Provider2 has scan2 with 1 check
assert (
len(p2_check_ids) == 1
), f"Provider2 should have 1 check, got {len(p2_check_ids)}"
assert (
"cloudtrail_enabled" in p2_check_ids
), "cloudtrail_enabled should be in provider2"
assert len(p2_check_ids) == 1, (
f"Provider2 should have 1 check, got {len(p2_check_ids)}"
)
assert "cloudtrail_enabled" in p2_check_ids, (
"cloudtrail_enabled should be in provider2"
)
# Test provider_type filter actually filters data
def test_finding_groups_provider_type_filter_actually_filters(
@@ -17253,9 +17824,9 @@ class TestFindingGroupViewSet:
{"filter[inserted_at]": TODAY, "filter[provider_type]": "gcp"},
)
assert response_gcp.status_code == status.HTTP_200_OK
assert (
len(response_gcp.json()["data"]) == 0
), "GCP filter should return 0 results"
assert len(response_gcp.json()["data"]) == 0, (
"GCP filter should return 0 results"
)
def test_finding_groups_pagination(
self, authenticated_client, finding_groups_fixture
@@ -17630,6 +18201,41 @@ class TestFindingGroupViewSet:
# All providers in fixture are AWS
assert len(data) == 5
def test_finding_groups_latest_provider_groups_filter(
self,
authenticated_client,
tenants_fixture,
finding_groups_fixture,
providers_fixture,
provider_groups_fixture,
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
group1, group2, *_ = provider_groups_fixture
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group1
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider1, provider_group=group2
)
ProviderGroupMembership.objects.create(
tenant=tenant, provider=provider2, provider_group=group2
)
response = authenticated_client.get(
reverse("finding-group-latest"),
{"filter[provider_groups]": str(group1.id)},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 4
response = authenticated_client.get(
reverse("finding-group-latest"),
{"filter[provider_groups__in]": f"{group1.id},{group2.id}"},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 5
def test_finding_groups_latest_check_id_filter(
self, authenticated_client, finding_groups_fixture
):
+155 -39
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import TYPE_CHECKING
from allauth.socialaccount.providers.oauth2.client import OAuth2Client
from django.contrib.postgres.aggregates import ArrayAgg
@@ -16,7 +17,30 @@ from prowler.lib.outputs.jira.jira import Jira, JiraBasicAuthError
from prowler.providers.aws.lib.s3.s3 import S3
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHub
from prowler.providers.common.models import Connection
from prowler.providers.common.provider import Provider as SDKProvider
if TYPE_CHECKING:
from prowler.providers.alibabacloud.alibabacloud_provider import (
AlibabacloudProvider,
)
from prowler.providers.aws.aws_provider import AwsProvider
from prowler.providers.azure.azure_provider import AzureProvider
from prowler.providers.cloudflare.cloudflare_provider import CloudflareProvider
from prowler.providers.gcp.gcp_provider import GcpProvider
from prowler.providers.github.github_provider import GithubProvider
from prowler.providers.googleworkspace.googleworkspace_provider import (
GoogleworkspaceProvider,
)
from prowler.providers.iac.iac_provider import IacProvider
from prowler.providers.image.image_provider import ImageProvider
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
from prowler.providers.m365.m365_provider import M365Provider
from prowler.providers.mongodbatlas.mongodbatlas_provider import (
MongodbatlasProvider,
)
from prowler.providers.okta.okta_provider import OktaProvider
from prowler.providers.openstack.openstack_provider import OpenstackProvider
from prowler.providers.oraclecloud.oraclecloud_provider import OraclecloudProvider
from prowler.providers.vercel.vercel_provider import VercelProvider
class CustomOAuth2Client(OAuth2Client):
@@ -55,26 +79,117 @@ def merge_dicts(default_dict: dict, replacement_dict: dict) -> dict:
return result
def return_prowler_provider(provider: Provider) -> type:
"""Resolve the Prowler provider class for the given provider.
The class is resolved dynamically from the SDK, so any provider the SDK
discovers built-in or external entry-point works without a per-provider
branch in the API.
def return_prowler_provider(
provider: Provider,
) -> (
AlibabacloudProvider
| AwsProvider
| AzureProvider
| CloudflareProvider
| GcpProvider
| GithubProvider
| GoogleworkspaceProvider
| IacProvider
| ImageProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
| OktaProvider
| OpenstackProvider
| OraclecloudProvider
| VercelProvider
):
"""Return the Prowler provider class based on the given provider type.
Args:
provider (Provider): The provider whose `provider` type to resolve.
provider (Provider): The provider object containing the provider type and associated secrets.
Returns:
type: The Prowler provider class.
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | GoogleworkspaceProvider | IacProvider | ImageProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: The corresponding provider class.
Raises:
ValueError: If the provider type is not available in the SDK.
ValueError: If the provider type specified in `provider.provider` is not supported.
"""
try:
return SDKProvider.get_class(provider.provider)
except ImportError as error:
raise ValueError(f"Provider type {provider.provider} not supported") from error
match provider.provider:
case Provider.ProviderChoices.AWS.value:
from prowler.providers.aws.aws_provider import AwsProvider
prowler_provider = AwsProvider
case Provider.ProviderChoices.GCP.value:
from prowler.providers.gcp.gcp_provider import GcpProvider
prowler_provider = GcpProvider
case Provider.ProviderChoices.GOOGLEWORKSPACE.value:
from prowler.providers.googleworkspace.googleworkspace_provider import (
GoogleworkspaceProvider,
)
prowler_provider = GoogleworkspaceProvider
case Provider.ProviderChoices.AZURE.value:
from prowler.providers.azure.azure_provider import AzureProvider
prowler_provider = AzureProvider
case Provider.ProviderChoices.KUBERNETES.value:
from prowler.providers.kubernetes.kubernetes_provider import (
KubernetesProvider,
)
prowler_provider = KubernetesProvider
case Provider.ProviderChoices.M365.value:
from prowler.providers.m365.m365_provider import M365Provider
prowler_provider = M365Provider
case Provider.ProviderChoices.GITHUB.value:
from prowler.providers.github.github_provider import GithubProvider
prowler_provider = GithubProvider
case Provider.ProviderChoices.MONGODBATLAS.value:
from prowler.providers.mongodbatlas.mongodbatlas_provider import (
MongodbatlasProvider,
)
prowler_provider = MongodbatlasProvider
case Provider.ProviderChoices.IAC.value:
from prowler.providers.iac.iac_provider import IacProvider
prowler_provider = IacProvider
case Provider.ProviderChoices.ORACLECLOUD.value:
from prowler.providers.oraclecloud.oraclecloud_provider import (
OraclecloudProvider,
)
prowler_provider = OraclecloudProvider
case Provider.ProviderChoices.ALIBABACLOUD.value:
from prowler.providers.alibabacloud.alibabacloud_provider import (
AlibabacloudProvider,
)
prowler_provider = AlibabacloudProvider
case Provider.ProviderChoices.CLOUDFLARE.value:
from prowler.providers.cloudflare.cloudflare_provider import (
CloudflareProvider,
)
prowler_provider = CloudflareProvider
case Provider.ProviderChoices.OPENSTACK.value:
from prowler.providers.openstack.openstack_provider import OpenstackProvider
prowler_provider = OpenstackProvider
case Provider.ProviderChoices.IMAGE.value:
from prowler.providers.image.image_provider import ImageProvider
prowler_provider = ImageProvider
case Provider.ProviderChoices.VERCEL.value:
from prowler.providers.vercel.vercel_provider import VercelProvider
prowler_provider = VercelProvider
case Provider.ProviderChoices.OKTA.value:
from prowler.providers.okta.okta_provider import OktaProvider
prowler_provider = OktaProvider
case _:
raise ValueError(f"Provider type {provider.provider} not supported")
return prowler_provider
def get_prowler_provider_kwargs(
@@ -89,18 +204,6 @@ def get_prowler_provider_kwargs(
Returns:
dict: The provider kwargs for the corresponding provider class.
"""
# External providers declare their own uid/secret -> kwargs mapping through
# the SDK contract; built-in providers keep the explicit mapping below.
if not SDKProvider.is_builtin(provider.provider):
mutelist_content = (
mutelist_processor.configuration.get("Mutelist", {})
if mutelist_processor
else None
)
return return_prowler_provider(provider).get_scan_arguments(
provider.uid, provider.secret.secret, mutelist_content
)
prowler_provider_kwargs = provider.secret.secret
if provider.provider == Provider.ProviderChoices.AZURE.value:
prowler_provider_kwargs = {
@@ -140,6 +243,12 @@ def get_prowler_provider_kwargs(
**prowler_provider_kwargs,
"filter_accounts": [provider.uid],
}
elif provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
if isinstance(prowler_provider_kwargs.get("region"), str):
prowler_provider_kwargs = {
**prowler_provider_kwargs,
"region": {prowler_provider_kwargs["region"]},
}
elif provider.provider == Provider.ProviderChoices.OPENSTACK.value:
# clouds_yaml_content, clouds_yaml_cloud and provider_id are validated
# in the provider itself, so it's not needed here.
@@ -185,7 +294,24 @@ def get_prowler_provider_kwargs(
def initialize_prowler_provider(
provider: Provider,
mutelist_processor: Processor | None = None,
) -> SDKProvider:
) -> (
AlibabacloudProvider
| AwsProvider
| AzureProvider
| CloudflareProvider
| GcpProvider
| GithubProvider
| GoogleworkspaceProvider
| IacProvider
| ImageProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
| OktaProvider
| OpenstackProvider
| OraclecloudProvider
| VercelProvider
):
"""Initialize a Prowler provider instance based on the given provider type.
Args:
@@ -193,8 +319,8 @@ def initialize_prowler_provider(
mutelist_processor (Processor): The mutelist processor object containing the mutelist configuration.
Returns:
SDKProvider: An instance of the corresponding provider class initialized
with the provider's secrets.
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | GoogleworkspaceProvider | IacProvider | ImageProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: An instance of the corresponding provider class
initialized with the provider's secrets.
"""
prowler_provider = return_prowler_provider(provider)
prowler_provider_kwargs = get_prowler_provider_kwargs(provider, mutelist_processor)
@@ -217,16 +343,6 @@ def prowler_provider_connection_test(provider: Provider) -> Connection:
except Provider.secret.RelatedObjectDoesNotExist as secret_error:
return Connection(is_connected=False, error=secret_error)
# External providers declare their own connection kwargs through the SDK
# contract; built-in providers keep the explicit mapping below.
if not SDKProvider.is_builtin(provider.provider):
return prowler_provider.test_connection(
**prowler_provider.get_connection_arguments(
provider.uid, prowler_provider_kwargs
),
raise_on_exception=False,
)
# For IaC provider, construct the kwargs properly for test_connection
if provider.provider == Provider.ProviderChoices.IAC.value:
# Don't pass repository_url from secret, use scan_repository_url with the UID
+1 -61
View File
@@ -10,7 +10,6 @@ from django.core.exceptions import ValidationError as DjangoValidationError
from django.db import IntegrityError
from drf_spectacular.utils import extend_schema_field
from jwt.exceptions import InvalidKeyError
from pydantic import ValidationError as PydanticValidationError
from rest_framework.reverse import reverse
from rest_framework.validators import UniqueTogetherValidator
from rest_framework_json_api import serializers
@@ -72,10 +71,8 @@ from api.v1.serializer_utils.lighthouse import (
OpenAICredentialsSerializer,
)
from api.v1.serializer_utils.processors import ProcessorConfigField
from api.provider_types import get_provider_type_choices
from api.v1.serializer_utils.providers import ProviderSecretField
from prowler.lib.mutelist.mutelist import Mutelist
from prowler.providers.common.provider import Provider as SDKProvider
# Base
@@ -857,9 +854,7 @@ class ProviderGroupMembershipSerializer(RLSSerializer, BaseWriteSerializer):
# Providers
class ProviderEnumSerializerField(serializers.ChoiceField):
def __init__(self, **kwargs):
# Accepted values track the SDK's installed providers (built-in or
# external), shared with the filters via one cached source.
kwargs["choices"] = get_provider_type_choices()
kwargs["choices"] = Provider.ProviderChoices.choices
super().__init__(**kwargs)
@@ -945,12 +940,6 @@ class ProviderIncludeSerializer(RLSSerializer):
class ProviderCreateSerializer(RLSSerializer, BaseWriteSerializer):
# Declared explicitly so provider validation stays at the serializer layer:
# the model column is now a plain varchar with no choices, so without this
# an unknown provider would slip through to Provider.clean() instead of
# being rejected here with `invalid_choice`.
provider = ProviderEnumSerializerField()
class Meta:
model = Provider
fields = [
@@ -1543,59 +1532,10 @@ class FindingMetadataSerializer(BaseSerializerV1):
# Provider secrets
class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
@staticmethod
def _validate_external_provider_secret(
provider_type: str, secret_type: str, secret: dict
):
"""Validate a non-built-in provider's secret against the schema it
declares for the given secret type through the SDK contract.
The provider maps each secret type to one model, so the chosen
secret_type stays bound to the shape it claims. Providers that declare
no schema have their secret accepted as an object and validated by the
provider's ``test_connection``.
"""
if not isinstance(secret, dict):
raise serializers.ValidationError({"secret": ["Must be a JSON object."]})
schemas = SDKProvider.get_class(provider_type).get_credentials_schema()
if not schemas:
return
schema = schemas.get(secret_type)
if schema is None:
raise serializers.ValidationError(
{
"secret_type": [
f"'{secret_type}' is not supported by provider "
f"'{provider_type}'. Supported types: "
f"{', '.join(sorted(schemas))}."
]
}
)
try:
schema.model_validate(secret)
except PydanticValidationError as error:
raise serializers.ValidationError(
{
"secret": [
f"{'/'.join(str(loc) for loc in item['loc']) or 'secret'}: "
f"{item['msg']}"
for item in error.errors()
]
}
)
@staticmethod
def validate_secret_based_on_provider(
provider_type: str, secret_type: ProviderSecret.TypeChoices, secret: dict
):
# External providers validate against the schemas they declare via the
# SDK contract; built-in providers keep their explicit serializers below.
if not SDKProvider.is_builtin(provider_type):
BaseWriteProviderSecretSerializer._validate_external_provider_secret(
provider_type, secret_type, secret
)
return
if secret_type == ProviderSecret.TypeChoices.STATIC:
if provider_type == Provider.ProviderChoices.AWS.value:
serializer = AwsProviderSecret(data=secret)
+92 -2
View File
@@ -30,6 +30,7 @@ from dj_rest_auth.registration.views import SocialLoginView
from django.conf import settings as django_settings
from django.contrib.postgres.aggregates import ArrayAgg, BoolAnd, StringAgg
from django.contrib.postgres.search import SearchQuery
from django.core.exceptions import ValidationError as DjangoValidationError
from django.db import transaction
from django.db.models import (
BooleanField,
@@ -114,6 +115,8 @@ from api.attack_paths import get_queries_for_provider, get_query_by_id
from api.attack_paths import views_helpers as attack_paths_views_helpers
from api.base_views import BaseRLSViewSet, BaseTenantViewset, BaseUserViewset
from api.compliance import (
COMPLIANCE_WARMED,
COMPLIANCE_WARMING_STARTED,
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
get_compliance_frameworks,
get_prowler_provider_compliance,
@@ -122,6 +125,7 @@ from api.constants import SEVERITY_ORDER
from api.db_router import MainRouter
from api.db_utils import rls_transaction
from api.exceptions import (
ComplianceWarmingError,
TaskFailedException,
UpstreamAccessDeniedError,
UpstreamAuthenticationError,
@@ -4641,6 +4645,16 @@ class RoleProviderGroupRelationshipView(RelationshipView, BaseRLSViewSet):
location=OpenApiParameter.QUERY,
description="Compliance framework ID to get attributes for.",
),
OpenApiParameter(
name="filter[scan_id]",
required=False,
type=OpenApiTypes.UUID,
location=OpenApiParameter.QUERY,
description="Scan ID used to resolve the provider for "
"multi-provider universal frameworks (e.g. CSA CCM), so "
"the returned check IDs match the scan's provider. When omitted, "
"the first provider that declares the framework is used.",
),
],
responses={
200: OpenApiResponse(
@@ -5059,6 +5073,13 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
@action(detail=False, methods=["get"], url_name="attributes")
def attributes(self, request):
# While the background warm-up is in progress, refuse immediately
# instead of falling through to the slow cold load on the request
# thread (which would trip the Gunicorn worker timeout). `is_set()` is
# a non-blocking flag read, so this never touches the loader.
if COMPLIANCE_WARMING_STARTED.is_set() and not COMPLIANCE_WARMED.is_set():
raise ComplianceWarmingError()
compliance_id = request.query_params.get("filter[compliance_id]")
if not compliance_id:
raise ValidationError(
@@ -5074,7 +5095,51 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
provider_type = None
# If we couldn't determine from database, try each provider type
# When a scan is provided, resolve the provider from it. Multi-provider
# universal frameworks (e.g. CSA CCM) share a single compliance_id
# across providers but expose different checks per provider, so the
# metadata (and therefore the check IDs the UI uses to fetch findings)
# must be returned for the scan's provider. Without this, the endpoint
# falls back to the first provider that declares the framework and
# returns its check IDs, leaving azure/gcp/... requirements with no
# matching findings.
scan_id = request.query_params.get("filter[scan_id]")
if "filter[scan_id]" in request.query_params:
# An explicit scan_id is authoritative: fail closed instead of
# falling back to another provider. Otherwise an invalid, empty
# (filter[scan_id]=) or inaccessible scan would silently return the
# first provider's check IDs, recreating the multi-provider mismatch
# this endpoint fixes.
if not scan_id:
raise NotFound(detail=f"Scan '{scan_id}' not found.")
# Tenant isolation is already enforced by Postgres RLS on the
# connection (see BaseRLSViewSet). Scope the lookup by provider
# group as well so a user with limited visibility can't resolve
# another provider's scan and read its compliance metadata, mirroring
# the RBAC scoping get_queryset() applies to the rest of the ViewSet.
role = get_role(request.user, request.tenant_id)
if getattr(role, Permissions.UNLIMITED_VISIBILITY.value, False):
scan_queryset = Scan.objects.filter(tenant_id=request.tenant_id)
else:
scan_queryset = Scan.objects.filter(provider__in=get_providers(role))
try:
scan = scan_queryset.select_related("provider").get(id=scan_id)
except (Scan.DoesNotExist, DjangoValidationError, ValueError):
raise NotFound(detail=f"Scan '{scan_id}' not found.")
provider_type = scan.provider.provider
if compliance_id not in get_compliance_frameworks(provider_type):
raise NotFound(
detail=(
f"Compliance framework '{compliance_id}' is not "
f"available for scan '{scan_id}'."
)
)
# Fall back to the first provider that declares the framework. Keeps the
# endpoint working for provider-agnostic callers that omit the scan.
if not provider_type:
for pt in Provider.ProviderChoices.values:
if compliance_id in get_compliance_frameworks(pt):
@@ -5444,6 +5509,14 @@ class OverviewViewSet(BaseRLSViewSet):
)
filters["provider__provider__in"] = types
provider_groups = params.get("filter[provider_groups]")
if provider_groups:
filters["provider__provider_groups__id"] = provider_groups
provider_groups_in = params.get("filter[provider_groups__in]")
if provider_groups_in:
filters["provider__provider_groups__id__in"] = provider_groups_in.split(",")
return filters
@action(detail=False, methods=["get"], url_name="providers")
@@ -5763,6 +5836,18 @@ class OverviewViewSet(BaseRLSViewSet):
location=OpenApiParameter.QUERY,
description="Filter by multiple provider types (comma-separated)",
),
OpenApiParameter(
name="provider_groups",
type=OpenApiTypes.UUID,
location=OpenApiParameter.QUERY,
description="Filter by provider group ID",
),
OpenApiParameter(
name="provider_groups__in",
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
description="Filter by multiple provider group IDs (comma-separated UUIDs)",
),
],
)
@action(detail=False, methods=["get"], url_name="threatscore")
@@ -6104,6 +6189,8 @@ class OverviewViewSet(BaseRLSViewSet):
"provider_id__in",
"provider_type",
"provider_type__in",
"provider_groups",
"provider_groups__in",
}
filtered_queryset = self._apply_filterset(
base_queryset, CategoryOverviewFilter, exclude_keys=provider_filter_keys
@@ -6173,6 +6260,8 @@ class OverviewViewSet(BaseRLSViewSet):
"provider_id__in",
"provider_type",
"provider_type__in",
"provider_groups",
"provider_groups__in",
}
filtered_queryset = self._apply_filterset(
base_queryset,
@@ -8419,9 +8508,10 @@ class FindingGroupViewSet(BaseRLSViewSet):
This endpoint returns finding groups without requiring date filters,
automatically using the latest available data per check_id.
All other filters (provider_id, provider_type, check_id) are still supported.
Provider, provider group, check, and computed filters are still supported.
""",
tags=["Finding Groups"],
filters=True,
)
@action(detail=False, methods=["get"], url_name="latest")
def latest(self, request):
+29
View File
@@ -306,3 +306,32 @@ SESSION_COOKIE_SECURE = True
ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES = env.int(
"ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES", 2880
) # 48h
# Orphan task recovery feature flags. The master switch is OFF by default, so task
# recovery is opt-in; enable it with DJANGO_TASK_RECOVERY_ENABLED=true. The per-group
# toggles default to enabled, so once the master is on every group recovers unless a
# group is explicitly turned off.
TASK_RECOVERY_ENABLED = env.bool("DJANGO_TASK_RECOVERY_ENABLED", False)
TASK_RECOVERY_SUMMARIES_ENABLED = env.bool(
"DJANGO_TASK_RECOVERY_SUMMARIES_ENABLED", True
)
TASK_RECOVERY_DELETIONS_ENABLED = env.bool(
"DJANGO_TASK_RECOVERY_DELETIONS_ENABLED", True
)
def label_postgres_connections(databases):
"""Tag each Postgres connection with ``application_name="<component>:<alias>"``
so connections are attributable by component in ``pg_stat_activity`` (and any
tooling that surfaces ``application_name``). The component (api / worker /
scan / ...) is injected per process by the container entrypoint via
``DJANGO_APP_COMPONENT``; the alias distinguishes which pool inside the
process owns the connection. The neo4j entry is skipped (not a Postgres
backend). Postgres truncates ``application_name`` at 63 bytes.
"""
component = env.str("DJANGO_APP_COMPONENT", default="api")
for alias, config in databases.items():
engine = config.get("ENGINE", "")
if engine.startswith("psqlextra") or "postgresql" in engine:
name = f"{component}:{alias}"[:63]
config.setdefault("OPTIONS", {})["application_name"] = name
+2
View File
@@ -54,6 +54,8 @@ DATABASES = {
DATABASES["default"] = DATABASES["prowler_user"]
label_postgres_connections(DATABASES) # noqa: F405
REST_FRAMEWORK["DEFAULT_RENDERER_CLASSES"] = tuple( # noqa: F405
render_class
for render_class in REST_FRAMEWORK["DEFAULT_RENDERER_CLASSES"] # noqa: F405
@@ -58,3 +58,5 @@ DATABASES = {
}
DATABASES["default"] = DATABASES["prowler_user"]
label_postgres_connections(DATABASES) # noqa: F405
+5
View File
@@ -34,3 +34,8 @@ DRF_API_KEY = {
# JWT
SIMPLE_JWT["ALGORITHM"] = "HS256" # noqa: F405
# pyjwt >= 2.13.0 rejects an empty HMAC signing key, so HS256 tests need a real
# key (>= 32 bytes also avoids the InsecureKeyLengthWarning). Production uses RS256.
SIMPLE_JWT["SIGNING_KEY"] = env.str( # noqa: F405
"DJANGO_TOKEN_SIGNING_KEY", "insecure-testing-jwt-signing-key-do-not-use-in-prod"
)
+25
View File
@@ -1,6 +1,7 @@
import logging
import multiprocessing
import os
import threading
from config.env import env
@@ -11,6 +12,7 @@ os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.django.production")
import django # noqa: E402
django.setup()
from api.compliance import warm_compliance_caches # noqa: E402
from config.django.production import LOGGING as DJANGO_LOGGERS, DEBUG # noqa: E402
from config.custom_logging import BackendLogger # noqa: E402
@@ -41,3 +43,26 @@ def on_reload(_):
def when_ready(_):
gunicorn_logger.info("Gunicorn server is ready")
def _warm_compliance_caches_in_background():
"""Warm compliance caches off the request path and log the outcome."""
failed = warm_compliance_caches()
if failed:
gunicorn_logger.warning("Compliance caches warmed (skipped: %s)", failed)
else:
gunicorn_logger.info("Compliance caches warmed")
def post_fork(_server, worker):
"""Warm compliance caches after each worker fork.
Warm compliance caches in a background thread so the worker becomes ready
immediately. A request for a not-yet-warmed provider lazily loads just that
provider, which stays well under the worker timeout.
"""
threading.Thread(
target=_warm_compliance_caches_in_background,
name="warm-compliance-caches",
daemon=True,
).start()
-9
View File
@@ -11,7 +11,6 @@ from api.db_utils import batch_delete, rls_transaction
from api.models import (
AttackPathsScan,
Finding,
JiraIssueDispatch,
Provider,
ProviderComplianceScore,
Resource,
@@ -81,14 +80,6 @@ def delete_provider(tenant_id: str, pk: str):
deletion_steps = [
("Scan Summaries", ScanSummary.all_objects.filter(scan__provider=instance)),
(
"Jira Issue Dispatches",
JiraIssueDispatch.objects.filter(
finding_id__in=Finding.all_objects.filter(
scan__provider=instance
).values_list("id", flat=True)
),
),
("Findings", Finding.all_objects.filter(scan__provider=instance)),
("Resources", Resource.all_objects.filter(provider=instance)),
("Scans", Scan.all_objects.filter(provider=instance)),
+6
View File
@@ -58,6 +58,9 @@ from prowler.lib.outputs.compliance.mitre_attack.mitre_attack_azure import (
AzureMitreAttack,
)
from prowler.lib.outputs.compliance.mitre_attack.mitre_attack_gcp import GCPMitreAttack
from prowler.lib.outputs.compliance.okta_idaas_stig.okta_idaas_stig_okta import (
OktaIDaaSSTIG,
)
from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore_alibaba import (
ProwlerThreatScoreAlibaba,
)
@@ -152,6 +155,9 @@ COMPLIANCE_CLASS_MAP = {
ProwlerThreatScoreAlibaba,
),
],
"okta": [
(lambda name: name.startswith("okta_idaas_stig"), OktaIDaaSSTIG),
],
}
+51 -100
View File
@@ -9,7 +9,7 @@ from tasks.utils import batched
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.db_utils import REPLICA_MAX_ATTEMPTS, REPLICA_RETRY_BASE_DELAY, rls_transaction
from api.models import Finding, Integration, JiraIssueDispatch, Provider
from api.models import Finding, Integration, Provider
from api.utils import initialize_prowler_integration, initialize_prowler_provider
from prowler.lib.outputs.asff.asff import ASFF
from prowler.lib.outputs.compliance.generic.generic import GenericCompliance
@@ -482,115 +482,66 @@ def send_findings_to_jira(
with rls_transaction(tenant_id):
integration = Integration.objects.get(id=integration_id)
jira_integration = initialize_prowler_integration(integration)
# Idempotency: findings already ticketed for this integration must not be
# sent again on a re-run (e.g. orphan recovery), to avoid duplicate issues
already_sent = {
str(fid)
for fid in JiraIssueDispatch.objects.filter(
integration_id=integration_id, finding_id__in=finding_ids
).values_list("finding_id", flat=True)
}
num_tickets_created = 0
skipped_count = 0
for finding_id in finding_ids:
if str(finding_id) in already_sent:
skipped_count += 1
continue
# Reserve the finding BEFORE the external call. The unique constraint on
# (tenant, integration, finding) makes the dispatch row the single source of
# truth, so a concurrent run or a retry that raced past the bulk pre-check
# cannot create a duplicate issue: created=False means another run already
# claimed it. The reservation is released below if the send does not succeed.
with rls_transaction(tenant_id):
_, created = JiraIssueDispatch.objects.get_or_create(
tenant_id=tenant_id,
integration_id=integration_id,
finding_id=finding_id,
finding_instance = (
Finding.all_objects.select_related("scan__provider")
.prefetch_related("resources")
.get(id=finding_id)
)
if not created:
skipped_count += 1
continue
sent = False
try:
with rls_transaction(tenant_id):
finding_instance = (
Finding.all_objects.select_related("scan__provider")
.prefetch_related("resources")
.get(id=finding_id)
)
# Extract resource information
resource = (
finding_instance.resources.first()
if finding_instance.resources.exists()
else None
)
resource_uid = resource.uid if resource else ""
resource_name = resource.name if resource else ""
resource_tags = {}
if resource and hasattr(resource, "tags"):
resource_tags = resource.get_tags(tenant_id)
# Extract resource information
resource = (
finding_instance.resources.first()
if finding_instance.resources.exists()
else None
)
resource_uid = resource.uid if resource else ""
resource_name = resource.name if resource else ""
resource_tags = {}
if resource and hasattr(resource, "tags"):
resource_tags = resource.get_tags(tenant_id)
# Get region
region = resource.region if resource and resource.region else ""
# Get region
region = resource.region if resource and resource.region else ""
# Extract remediation information from check_metadata
check_metadata = finding_instance.check_metadata
remediation = check_metadata.get("remediation", {})
recommendation = remediation.get("recommendation", {})
remediation_code = remediation.get("code", {})
# Extract remediation information from check_metadata
check_metadata = finding_instance.check_metadata
remediation = check_metadata.get("remediation", {})
recommendation = remediation.get("recommendation", {})
remediation_code = remediation.get("code", {})
# Send the individual finding to Jira
sent = bool(
jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=finding_instance.scan.provider.provider,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get(
"nativeiac", ""
),
remediation_code_terraform=remediation_code.get(
"terraform", ""
),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
)
)
finally:
if not sent:
# Release the reservation so a later run can retry this finding: it
# was not ticketed (send failed or raised), so the row must not block
# a future legitimate send.
with rls_transaction(tenant_id):
JiraIssueDispatch.objects.filter(
tenant_id=tenant_id,
integration_id=integration_id,
finding_id=finding_id,
).delete()
if sent:
num_tickets_created += 1
else:
logger.error(f"Failed to send finding {finding_id} to Jira")
# Send the individual finding to Jira
result = jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=finding_instance.scan.provider.provider,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get("nativeiac", ""),
remediation_code_terraform=remediation_code.get("terraform", ""),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
)
if result:
num_tickets_created += 1
else:
logger.error(f"Failed to send finding {finding_id} to Jira")
return {
"created_count": num_tickets_created,
"failed_count": len(finding_ids) - num_tickets_created - skipped_count,
"skipped_count": skipped_count,
"failed_count": len(finding_ids) - num_tickets_created,
}
+75 -131
View File
@@ -37,35 +37,52 @@ ORPHAN_RECOVERY_LOCK_KEY = 0x70726F77 # "prow"
# Non-terminal states that mean "a worker had this and may have died with it".
IN_FLIGHT_STATES = (states.STARTED, states.RECEIVED)
# Scan tasks are recovered by re-running scan-perform on the EXISTING scan row,
# not by re-enqueuing the original task: re-enqueuing scan-perform-scheduled would
# hit its "a scan is already executing" guard and no-op, leaving the scan stuck.
_SCAN_TASKS = ("scan-perform", "scan-perform-scheduled")
# Tasks with proven idempotency are auto re-enqueued. Scans/summaries clear and
# rewrite their own rows. integration-jira is safe too: each finding is reserved in
# JiraIssueDispatch before the external call, so a re-run skips already-ticketed
# findings (worst case one finding missed on a mid-send crash, never a duplicate).
# Other external side effects stay terminal: integration-s3 rebuilds its upload from
# worker-local files that do not survive a crash, and report/Security Hub recovery is
# out of scope.
REENQUEUEABLE_TASKS = {
*_SCAN_TASKS,
"provider-deletion",
"tenant-deletion",
"scan-summary",
"scan-compliance-overviews",
"scan-provider-compliance-scores",
"scan-daily-severity",
"scan-finding-group-summaries",
"scan-reset-ephemeral-resources",
"integration-jira",
# Tasks with proven idempotency are eligible for auto re-enqueue, grouped so each
# group can be toggled independently by a feature flag (see config.django.base).
# Summaries clear and rewrite their own rows and deletions are idempotent. Tasks with
# external side effects are never eligible: integration-jira would create duplicate
# issues, integration-s3 rebuilds its upload from worker-local files that do not
# survive a crash, and report/Security Hub recovery is out of scope.
RECOVERY_TASK_GROUPS = {
"summaries": {
"scan-summary",
"scan-compliance-overviews",
"scan-provider-compliance-scores",
"scan-daily-severity",
"scan-finding-group-summaries",
"scan-reset-ephemeral-resources",
},
"deletions": {"provider-deletion", "tenant-deletion"},
}
# Tasks excluded from generic recovery: attack-paths scans are handled by their own
# stale-cleanup (which also drops the temp Neo4j db), and the maintenance tasks must
# not self-recover (they run again on their own schedule).
def reenqueueable_tasks() -> set[str]:
"""Task names eligible for auto re-enqueue, honoring the per-group feature flags.
A group whose flag is disabled is dropped, so its orphaned tasks are marked
terminal instead of re-enqueued.
"""
from django.conf import settings
group_enabled = {
"summaries": settings.TASK_RECOVERY_SUMMARIES_ENABLED,
"deletions": settings.TASK_RECOVERY_DELETIONS_ENABLED,
}
return {
task
for group, tasks in RECOVERY_TASK_GROUPS.items()
if group_enabled[group]
for task in tasks
}
# Tasks the watchdog ignores entirely (not even marked terminal): scan tasks are not
# auto-recovered, since re-running a scan is not safe to do automatically; attack-paths
# scans are handled by their own stale-cleanup (which also drops the temp Neo4j db);
# and the maintenance tasks must not self-recover (they run again on their own schedule).
_SKIP_RECOVERY = {
"scan-perform",
"scan-perform-scheduled",
"attack-paths-scan-perform",
"attack-paths-cleanup-stale-scans",
"reconcile-orphan-tasks",
@@ -166,15 +183,22 @@ def reconcile_orphans(
logger.info("Orphan reconcile skipped: another run holds the lock")
return {"acquired": False}
# Populate the task registry so we can re-enqueue any task by name.
import tasks.tasks # noqa: F401
from django.conf import settings
result = _reconcile_task_results(
grace_minutes=grace_minutes,
max_attempts=max_attempts,
window_hours=window_hours,
dry_run=dry_run,
)
if settings.TASK_RECOVERY_ENABLED:
# Populate the task registry so we can re-enqueue any task by name.
import tasks.tasks # noqa: F401
result = _reconcile_task_results(
grace_minutes=grace_minutes,
max_attempts=max_attempts,
window_hours=window_hours,
dry_run=dry_run,
)
result["enabled"] = True
else:
logger.info("Orphan task recovery disabled by feature flag")
result = {"recovered": [], "failed": [], "skipped": [], "enabled": False}
if not dry_run:
from tasks.jobs.attack_paths.cleanup import cleanup_stale_attack_paths_scans
@@ -264,34 +288,27 @@ def _recover_task(task_result, max_attempts: int, window_hours: int) -> str:
task_result.date_done = now
task_result.save(update_fields=["status", "date_done"])
attempt = _recovery_attempt_count(name, kwargs_repr, window_hours)
if name not in REENQUEUEABLE_TASKS or attempt > max_attempts:
reason = (
f"{name} is not allowlisted for auto recovery"
if name not in REENQUEUEABLE_TASKS
else f"recovery cap reached ({attempt}/{max_attempts})"
)
_fail_domain_row(task_result.task_id, name, now)
if name not in reenqueueable_tasks():
logger.warning(
"Orphan %s (%s) not re-enqueued: %s", task_result.task_id, name, reason
"Orphan %s (%s) not re-enqueued: not allowlisted for auto recovery",
task_result.task_id,
name,
)
return "failed"
# Scan tasks: re-run the EXISTING scan row directly via scan-perform, so the
# scheduled-scan "already executing" guard cannot turn recovery into a no-op.
# Falls through to the generic path only if no scan is linked yet (e.g. a
# scheduled task that died before creating one), where re-running it creates one.
if name in _SCAN_TASKS:
scan = _scan_for_task(task_result.task_id)
if scan is not None:
if not _reenqueue_scan(task_result.task_id, scan):
return "failed"
logger.info(
"Re-enqueued orphaned scan %s (was task %s)",
scan.id,
task_result.task_id,
)
return "recovered"
# Count the attempt only once the task is allowlisted, so a task sitting in a
# disabled group does not burn its recovery budget while the flag is off (and is
# not already over the cap the moment the group is re-enabled).
attempt = _recovery_attempt_count(name, kwargs_repr, window_hours)
if attempt > max_attempts:
logger.warning(
"Orphan %s (%s) not re-enqueued: recovery cap reached (%d/%d)",
task_result.task_id,
name,
attempt,
max_attempts,
)
return "failed"
task_obj = current_app.tasks.get(name)
if task_obj is None:
@@ -311,7 +328,6 @@ def _recover_task(task_result, max_attempts: int, window_hours: int) -> str:
task_result.task_id,
name,
)
_fail_domain_row(task_result.task_id, name, now)
return "failed"
new_task_id = str(uuid4())
task_obj.apply_async(
@@ -323,75 +339,3 @@ def _recover_task(task_result, max_attempts: int, window_hours: int) -> str:
"Re-enqueued orphan %s (%s) as %s", task_result.task_id, name, new_task_id
)
return "recovered"
def _scan_for_task(task_id: str):
"""Return the Scan linked to a Celery task id, or None (read across tenants)."""
from api.db_router import MainRouter
from api.models import Scan
return Scan.all_objects.using(MainRouter.admin_db).filter(task_id=task_id).first()
def _reenqueue_scan(old_task_id: str, scan) -> bool:
"""Re-run an orphaned scan via scan-perform on the existing row.
Pre-provisions the new task linkage (TaskResult + api.Task) and relinks the
Scan before enqueuing, so the FK is valid and a worker can never outrun the DB.
The relink is conditional on the scan still pointing at the old task, so a stale
orphan can never clobber a newer linkage.
"""
from django_celery_results.models import TaskResult
from api.db_utils import rls_transaction
from api.models import Scan
from api.models import Task as APITask
from tasks.tasks import perform_scan_task
tenant_id = str(scan.tenant_id)
new_task_id = str(uuid4())
with rls_transaction(tenant_id):
locked_scan = Scan.all_objects.select_for_update().filter(id=scan.id).first()
if locked_scan is None or str(locked_scan.task_id) != old_task_id:
logger.info(
"Scan %s no longer points at task %s; skipping recovery re-enqueue",
scan.id,
old_task_id,
)
return False
task_result_new, _ = TaskResult.objects.get_or_create(
task_id=new_task_id,
defaults={"status": states.PENDING, "task_name": "scan-perform"},
)
APITask.objects.update_or_create(
id=new_task_id,
tenant_id=tenant_id,
defaults={"task_runner_task": task_result_new},
)
locked_scan.task_id = new_task_id
locked_scan.recovery_count = (locked_scan.recovery_count or 0) + 1
locked_scan.save(update_fields=["task_id", "recovery_count", "updated_at"])
perform_scan_task.apply_async(
kwargs={
"tenant_id": tenant_id,
"scan_id": str(scan.id),
"provider_id": str(scan.provider_id),
},
task_id=new_task_id,
)
return True
def _fail_domain_row(old_task_id: str, name: str, now: datetime) -> None:
"""Mark a scan terminal when its task is capped/denylisted instead of re-run."""
from api.db_utils import rls_transaction
from api.models import Scan, StateChoices
if name in _SCAN_TASKS:
scan = _scan_for_task(old_task_id)
if scan is not None:
with rls_transaction(str(scan.tenant_id)):
Scan.all_objects.filter(id=scan.id, task_id=old_task_id).update(
state=StateChoices.FAILED, completed_at=now
)
+12 -18
View File
@@ -118,19 +118,6 @@ ATTACK_SURFACE_PROVIDER_COMPATIBILITY = {
_ATTACK_SURFACE_MAPPING_CACHE: dict[str, dict] = {}
def _clear_scan_rerun_state(tenant_id: str, scan_id: str) -> None:
"""Remove rows derived from a previous execution of this scan."""
with rls_transaction(tenant_id):
Finding.all_objects.filter(scan_id=scan_id).delete()
ResourceScanSummary.objects.filter(scan_id=scan_id).delete()
ScanCategorySummary.objects.filter(scan_id=scan_id).delete()
ScanGroupSummary.objects.filter(scan_id=scan_id).delete()
ScanSummary.objects.filter(scan_id=scan_id).delete()
AttackSurfaceOverview.objects.filter(scan_id=scan_id).delete()
ComplianceRequirementOverview.objects.filter(scan_id=scan_id).delete()
ComplianceOverviewSummary.objects.filter(scan_id=scan_id).delete()
def aggregate_category_counts(
categories: list[str],
severity: str,
@@ -282,6 +269,7 @@ def _store_resources(
provider=provider_instance,
uid=finding.resource_uid,
defaults={
"name": finding.resource_name,
"region": finding.region,
"service": finding.service_name,
"type": finding.resource_type,
@@ -289,6 +277,7 @@ def _store_resources(
)
if not created:
resource_instance.name = finding.resource_name
resource_instance.region = finding.region
resource_instance.service = finding.service_name
resource_instance.type = finding.resource_type
@@ -489,10 +478,9 @@ def _create_compliance_summaries(
)
)
# Idempotent re-run: clear this scan's prior summaries before re-inserting, so
# a recovered scan's summary always reflects its own (re-derived) requirement
# rows rather than keeping a stale row (bulk_create ignore_conflicts alone would
# keep the old one).
# Idempotent re-run: clear this scan's prior summaries before re-inserting, so a
# recovered scan-compliance-overviews run reflects its own re-derived rows instead
# of keeping a stale one (bulk_create ignore_conflicts alone would keep the old).
with rls_transaction(tenant_id):
ComplianceOverviewSummary.objects.filter(scan_id=scan_id).delete()
if summary_objects:
@@ -718,6 +706,12 @@ def _process_finding_micro_batch(
if finding.region and resource_instance.region != finding.region:
resource_instance.region = finding.region
updated = True
if (
finding.resource_name
and resource_instance.name != finding.resource_name
):
resource_instance.name = finding.resource_name
updated = True
if resource_instance.service != finding.service_name:
resource_instance.service = finding.service_name
updated = True
@@ -959,6 +953,7 @@ def _process_finding_micro_batch(
Resource.objects.bulk_update(
resources_to_bulk_update,
[
"name",
"metadata",
"details",
"partition",
@@ -1039,7 +1034,6 @@ def perform_prowler_scan(
scan_instance.state = StateChoices.EXECUTING
scan_instance.started_at = datetime.now(tz=timezone.utc)
scan_instance.save(update_fields=["state", "started_at", "updated_at"])
_clear_scan_rerun_state(tenant_id, scan_id)
# Find the mutelist processor if it exists
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
+14 -2
View File
@@ -260,7 +260,9 @@ def delete_provider_task(provider_id: str, tenant_id: str):
return delete_provider(tenant_id=tenant_id, pk=provider_id)
@shared_task(base=RLSTask, name="scan-perform", queue="scans")
# acks_late=False: a re-run would duplicate findings and the task is not auto-recovered,
# so a crashed scan is dropped rather than redelivered by the broker (as before #11416).
@shared_task(base=RLSTask, name="scan-perform", queue="scans", acks_late=False)
@handle_provider_deletion
def perform_scan_task(
tenant_id: str, scan_id: str, provider_id: str, checks_to_execute: list[str] = None
@@ -304,7 +306,14 @@ def perform_scan_task(
return result
@shared_task(base=RLSTask, bind=True, name="scan-perform-scheduled", queue="scans")
# acks_late=False: like scan-perform; a dropped run is re-fired by Beat on the next tick.
@shared_task(
base=RLSTask,
bind=True,
name="scan-perform-scheduled",
queue="scans",
acks_late=False,
)
@handle_provider_deletion
def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
"""
@@ -1151,10 +1160,13 @@ def security_hub_integration_task(
return upload_security_hub_integration(tenant_id, provider_id, scan_id)
# acks_late=False: Jira sends are not deduplicated and the task is not auto-recovered,
# so a crashed send is dropped rather than redelivered (avoids duplicate Jira issues).
@shared_task(
base=RLSTask,
name="integration-jira",
queue="integrations",
acks_late=False,
)
def jira_integration_task(
tenant_id: str,
+1 -39
View File
@@ -1,12 +1,11 @@
from unittest.mock import call, patch
from uuid import uuid4
import pytest
from django.core.exceptions import ObjectDoesNotExist
from tasks.jobs.deletion import delete_provider, delete_tenant
from api.attack_paths import database as graph_database
from api.models import JiraIssueDispatch, Provider, Tenant, TenantComplianceSummary
from api.models import Provider, Tenant, TenantComplianceSummary
@pytest.mark.django_db
@@ -35,43 +34,6 @@ class TestDeleteProvider:
str(instance.id),
)
def test_delete_provider_removes_jira_dispatches(
self,
providers_fixture,
findings_fixture,
integrations_fixture,
):
"""Deleting a provider removes JiraIssueDispatch rows for its findings only."""
instance = providers_fixture[0]
tenant_id = str(instance.tenant_id)
finding = findings_fixture[0]
integration = integrations_fixture[0]
# Dispatch for one of the provider's findings: must be removed with it.
JiraIssueDispatch.objects.create(
tenant_id=tenant_id,
integration=integration,
finding_id=finding.id,
)
# Dispatch for an unrelated finding: must survive the provider deletion.
unrelated = JiraIssueDispatch.objects.create(
tenant_id=tenant_id,
integration=integration,
finding_id=uuid4(),
)
with (
patch(
"tasks.jobs.deletion.graph_database.get_database_name",
return_value="tenant-db",
),
patch("tasks.jobs.deletion.graph_database.drop_subgraph"),
):
delete_provider(tenant_id, instance.id)
assert not JiraIssueDispatch.objects.filter(finding_id=finding.id).exists()
assert JiraIssueDispatch.objects.filter(pk=unrelated.pk).exists()
def test_delete_provider_does_not_exist(self, tenants_fixture):
with (
patch(
@@ -1640,74 +1640,14 @@ class TestJiraIntegration:
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_skips_already_dispatched(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""A re-run skips findings already ticketed (no duplicate Jira issues)."""
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
mock_integration_model.objects.get.return_value = MagicMock()
# finding-1 was already dispatched in a prior run; finding-2 is new.
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = [
"finding-1"
]
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), True)
mock_jira_integration = MagicMock()
mock_jira_integration.send_finding.return_value = True
mock_initialize_integration.return_value = mock_jira_integration
finding2 = MagicMock()
finding2.id = "finding-2"
finding2.check_id = "check_002"
finding2.severity = "low"
finding2.status = "FAIL"
finding2.status_extended = ""
finding2.compliance = {}
finding2.resources.exists.return_value = False
finding2.resources.first.return_value = None
finding2.scan.provider.provider = "aws"
finding2.check_metadata = {
"checktitle": "C2",
"risk": "",
"remediation": {"recommendation": {}, "code": {}},
}
mock_finding_model.all_objects.select_related.return_value.prefetch_related.return_value.get.return_value = finding2
result = send_findings_to_jira(
"tenant-123", "integration-456", "PROJ", "Task", ["finding-1", "finding-2"]
)
# finding-1 skipped (already sent); only finding-2 sent -> no duplicate.
assert result == {"created_count": 1, "failed_count": 0, "skipped_count": 1}
mock_jira_integration.send_finding.assert_called_once()
assert (
mock_jira_integration.send_finding.call_args.kwargs["check_id"]
== "check_002"
)
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_success(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test successful sending of findings to Jira using send_finding method"""
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), True)
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
@@ -1799,7 +1739,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 2, "failed_count": 0, "skipped_count": 0}
assert result == {"created_count": 2, "failed_count": 0}
# Verify Jira integration was initialized
mock_initialize_integration.assert_called_once_with(integration)
@@ -1831,10 +1771,8 @@ class TestJiraIntegration:
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.logger")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_partial_failure(
self,
mock_jira_dispatch,
mock_logger,
mock_initialize_integration,
mock_integration_model,
@@ -1842,8 +1780,6 @@ class TestJiraIntegration:
mock_rls_transaction,
):
"""Test partial failure when sending findings to Jira"""
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), True)
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
@@ -1897,35 +1833,23 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 2, "failed_count": 1, "skipped_count": 0}
assert result == {"created_count": 2, "failed_count": 1}
# Verify error was logged for the failed finding
mock_logger.error.assert_called_with("Failed to send finding finding-2 to Jira")
# The failed finding's reservation is released so a later run can retry it.
mock_jira_dispatch.objects.filter.assert_any_call(
tenant_id=tenant_id,
integration_id=integration_id,
finding_id="finding-2",
)
mock_jira_dispatch.objects.filter.return_value.delete.assert_called_once()
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_no_resources(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test sending findings to Jira when finding has no resources"""
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), True)
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
@@ -1983,7 +1907,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 1, "failed_count": 0, "skipped_count": 0}
assert result == {"created_count": 1, "failed_count": 0}
# Verify send_finding was called with empty resource fields
call_kwargs = mock_jira_integration.send_finding.call_args.kwargs
@@ -1996,18 +1920,14 @@ class TestJiraIntegration:
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_with_empty_check_metadata(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test sending findings to Jira when check_metadata is empty or missing fields"""
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), True)
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
@@ -2050,7 +1970,7 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 1, "failed_count": 0, "skipped_count": 0}
assert result == {"created_count": 1, "failed_count": 0}
# Verify send_finding was called with default/empty values
call_kwargs = mock_jira_integration.send_finding.call_args.kwargs
@@ -2063,94 +1983,3 @@ class TestJiraIntegration:
assert call_kwargs["remediation_code_cli"] == ""
assert call_kwargs["remediation_code_other"] == ""
assert call_kwargs["compliance"] == {}
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_reserves_before_sending(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""The dispatch row is reserved before the external Jira call (reserve-then-act)."""
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
mock_integration_model.objects.get.return_value = MagicMock()
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
order = []
mock_jira_dispatch.objects.get_or_create.side_effect = lambda **kw: (
order.append(("reserve", kw)) or (MagicMock(), True)
)
mock_jira_integration = MagicMock()
mock_jira_integration.send_finding.side_effect = lambda **kw: (
order.append(("send", kw)) or True
)
mock_initialize_integration.return_value = mock_jira_integration
finding = MagicMock()
finding.id = "finding-1"
finding.check_id = "check_001"
finding.severity = "low"
finding.status = "FAIL"
finding.status_extended = ""
finding.compliance = {}
finding.resources.exists.return_value = False
finding.resources.first.return_value = None
finding.scan.provider.provider = "aws"
finding.check_metadata = {
"checktitle": "C1",
"risk": "",
"remediation": {"recommendation": {}, "code": {}},
}
mock_finding_model.all_objects.select_related.return_value.prefetch_related.return_value.get.return_value = finding
result = send_findings_to_jira(
"tenant-123", "integration-456", "PROJ", "Task", ["finding-1"]
)
assert result == {"created_count": 1, "failed_count": 0, "skipped_count": 0}
# Reservation must precede the external send.
assert [entry[0] for entry in order] == ["reserve", "send"]
# A successful send keeps the reservation (no rollback delete).
mock_jira_dispatch.objects.filter.return_value.delete.assert_not_called()
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.JiraIssueDispatch")
def test_send_findings_to_jira_skips_when_already_reserved(
self,
mock_jira_dispatch,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""A finding that races past the bulk pre-check but loses the reservation
(created=False) is skipped without a second issue, leaving the row intact."""
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
mock_integration_model.objects.get.return_value = MagicMock()
mock_jira_dispatch.objects.filter.return_value.values_list.return_value = []
# Another concurrent run already created the dispatch row.
mock_jira_dispatch.objects.get_or_create.return_value = (MagicMock(), False)
mock_jira_integration = MagicMock()
mock_initialize_integration.return_value = mock_jira_integration
result = send_findings_to_jira(
"tenant-123", "integration-456", "PROJ", "Task", ["finding-1"]
)
assert result == {"created_count": 0, "failed_count": 0, "skipped_count": 1}
mock_jira_integration.send_finding.assert_not_called()
# The reservation belongs to the run that won the race; do not delete it.
mock_jira_dispatch.objects.filter.return_value.delete.assert_not_called()
@@ -4,17 +4,17 @@ from uuid import uuid4
import pytest
from celery import states
from django.test import override_settings
from django_celery_results.models import TaskResult
from api.models import Scan, StateChoices
from api.models import Task as APITask
from tasks.jobs.orphan_recovery import (
_decode_celery_field,
_reconcile_task_results,
_recovery_attempt_count,
_reenqueue_scan,
advisory_lock,
is_worker_alive,
reconcile_orphans,
reenqueueable_tasks,
)
@@ -130,9 +130,83 @@ class TestReconcileTaskResults:
assert tr.task_id in result["failed"]
mock_task.apply_async.assert_not_called()
def test_jira_integration_task_is_reenqueued(self, tenants_fixture):
"""integration-jira is re-enqueued: its JiraIssueDispatch reservation makes a
re-run skip already-ticketed findings, so recovery cannot duplicate issues."""
@override_settings(TASK_RECOVERY_SUMMARIES_ENABLED=False)
def test_disabled_group_task_is_not_reenqueued(self, tenants_fixture):
"""A task whose group feature flag is off stays terminal, not re-enqueued."""
tr = _orphan_result(
name="scan-summary",
kwargs={
"tenant_id": str(tenants_fixture[0].id),
"scan_id": str(uuid4()),
},
worker="dead@gone",
created_minutes_ago=60,
)
p_alive, p_revoke, p_app, mock_task = self._patches(alive=False)
with (
p_alive,
p_revoke,
p_app,
patch("tasks.jobs.orphan_recovery._recovery_attempt_count", return_value=1),
):
result = _reconcile_task_results(
grace_minutes=2, max_attempts=3, window_hours=6, dry_run=False
)
assert tr.task_id in result["failed"]
mock_task.apply_async.assert_not_called()
@override_settings(TASK_RECOVERY_SUMMARIES_ENABLED=False)
def test_disabled_group_task_does_not_consume_recovery_attempt(
self, tenants_fixture
):
"""A disabled-group task is failed without incrementing its Valkey attempt
counter, so re-enabling the group does not start it at the cap."""
tr = _orphan_result(
name="scan-summary",
kwargs={"tenant_id": str(tenants_fixture[0].id), "scan_id": str(uuid4())},
worker="dead@gone",
created_minutes_ago=60,
)
p_alive, p_revoke, p_app, mock_task = self._patches(alive=False)
with (
p_alive,
p_revoke,
p_app,
patch("tasks.jobs.orphan_recovery._recovery_attempt_count") as mock_count,
):
result = _reconcile_task_results(
grace_minutes=2, max_attempts=3, window_hours=6, dry_run=False
)
assert tr.task_id in result["failed"]
mock_count.assert_not_called()
def test_scan_task_is_skipped_entirely(self, tenants_fixture):
"""Scan tasks are excluded from recovery: the watchdog never touches them."""
tr = _orphan_result(
name="scan-perform",
kwargs={
"tenant_id": str(tenants_fixture[0].id),
"scan_id": str(uuid4()),
},
worker="dead@gone",
created_minutes_ago=60,
)
p_alive, p_revoke, p_app, mock_task = self._patches(alive=False)
with p_alive, p_revoke, p_app:
result = _reconcile_task_results(
grace_minutes=2, max_attempts=3, window_hours=6, dry_run=False
)
assert tr.task_id not in result["recovered"]
assert tr.task_id not in result["failed"]
assert tr.task_id not in result["skipped"]
mock_task.apply_async.assert_not_called()
def test_jira_integration_task_is_not_reenqueued(self, tenants_fixture):
"""integration-jira stays terminal: re-running it would create duplicate Jira
issues, so an orphaned send is failed instead of re-enqueued."""
tenant = tenants_fixture[0]
kwargs = {
"tenant_id": str(tenant.id),
@@ -158,13 +232,10 @@ class TestReconcileTaskResults:
grace_minutes=2, max_attempts=3, window_hours=6, dry_run=False
)
assert tr.task_id in result["recovered"]
assert tr.task_id in result["failed"]
tr.refresh_from_db()
assert tr.status == states.REVOKED # stale result cleared (no pending alert)
mock_task.apply_async.assert_called_once()
call = mock_task.apply_async.call_args.kwargs
assert call["kwargs"] == kwargs
assert call["task_id"] != tr.task_id # fresh task id
mock_task.apply_async.assert_not_called()
def test_skips_live_worker(self, tenants_fixture):
tr = _orphan_result(
@@ -246,98 +317,6 @@ class TestReconcileTaskResults:
mock_task.apply_async.assert_not_called()
@pytest.mark.django_db
class TestScanRecovery:
"""Scans are recovered by re-running scan-perform on the EXISTING scan row,
so even a scheduled-scan orphan (whose own task would no-op on its guard) is
actually re-executed."""
def _scan_orphan(self, tenant, provider, name):
old_id = str(uuid4())
tr = TaskResult.objects.create(
task_id=old_id,
status=states.STARTED,
task_name=name,
worker="dead@gone",
task_kwargs=repr(
{"tenant_id": str(tenant.id), "provider_id": str(provider.id)}
),
task_args=repr([]),
)
TaskResult.objects.filter(pk=tr.pk).update(
date_created=datetime.now(tz=timezone.utc) - timedelta(minutes=60)
)
APITask.objects.create(id=old_id, tenant_id=tenant.id, task_runner_task=tr)
scan = Scan.objects.create(
name="scan-orphan",
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.EXECUTING,
tenant_id=tenant.id,
task_id=old_id,
recovery_count=0,
)
return old_id, scan
@pytest.mark.parametrize("name", ["scan-perform", "scan-perform-scheduled"])
def test_scan_recovered_via_scan_perform(
self, tenants_fixture, providers_fixture, name
):
tenant, provider = tenants_fixture[0], providers_fixture[0]
old_id, scan = self._scan_orphan(tenant, provider, name)
with (
patch("tasks.jobs.orphan_recovery.is_worker_alive", return_value=False),
patch("tasks.jobs.orphan_recovery.revoke_task"),
patch("tasks.jobs.orphan_recovery._recovery_attempt_count", return_value=1),
patch("tasks.tasks.perform_scan_task") as mock_scan_task,
):
result = _reconcile_task_results(
grace_minutes=2, max_attempts=3, window_hours=6, dry_run=False
)
assert old_id in result["recovered"]
scan.refresh_from_db()
assert str(scan.task_id) != old_id # relinked to a fresh task
assert scan.recovery_count == 1
assert TaskResult.objects.get(task_id=old_id).status == states.REVOKED
# Recovered by re-running scan-perform on the existing scan row (so the
# scheduled guard cannot no-op it), regardless of the original task name.
mock_scan_task.apply_async.assert_called_once()
assert mock_scan_task.apply_async.call_args.kwargs["kwargs"]["scan_id"] == str(
scan.id
)
def test_reenqueue_skips_when_scan_already_repointed(
self, tenants_fixture, providers_fixture
):
# The scan already points at a newer task, so a stale orphan must not relink
# it or launch a second concurrent run against the same scan row.
tenant, provider = tenants_fixture[0], providers_fixture[0]
newer_id = str(uuid4())
tr = TaskResult.objects.create(
task_id=newer_id, status=states.STARTED, task_name="scan-perform"
)
APITask.objects.create(id=newer_id, tenant_id=tenant.id, task_runner_task=tr)
scan = Scan.objects.create(
name="scan-orphan",
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.EXECUTING,
tenant_id=tenant.id,
task_id=newer_id,
recovery_count=0,
)
with patch("tasks.tasks.perform_scan_task") as mock_scan_task:
recovered = _reenqueue_scan(str(uuid4()), scan)
assert recovered is False
mock_scan_task.apply_async.assert_not_called()
scan.refresh_from_db()
assert scan.recovery_count == 0
@pytest.mark.django_db
class TestOrphanRecoveryHelpers:
def test_advisory_lock_acquires_and_releases(self):
@@ -370,3 +349,60 @@ class TestOrphanRecoveryHelpers:
with patch("redis.from_url", return_value=redis_client):
assert _recovery_attempt_count("probe-task", kwargs_repr, 6) == 1
assert _recovery_attempt_count("probe-task", kwargs_repr, 6) == 2
class TestRecoveryFeatureFlags:
def test_all_groups_enabled_by_default(self):
tasks = reenqueueable_tasks()
assert "scan-summary" in tasks
assert {"provider-deletion", "tenant-deletion"} <= tasks
@override_settings(TASK_RECOVERY_SUMMARIES_ENABLED=False)
def test_summaries_group_flag_excludes_summary_tasks(self):
tasks = reenqueueable_tasks()
assert "scan-summary" not in tasks
assert "scan-compliance-overviews" not in tasks
assert "provider-deletion" in tasks
@override_settings(TASK_RECOVERY_DELETIONS_ENABLED=False)
def test_deletions_group_flag_excludes_deletion_tasks(self):
tasks = reenqueueable_tasks()
assert "provider-deletion" not in tasks
assert "tenant-deletion" not in tasks
assert "scan-summary" in tasks
@pytest.mark.django_db
class TestRecoveryMasterFlag:
@override_settings(TASK_RECOVERY_ENABLED=False)
def test_master_flag_disables_task_recovery(self):
with (
patch(
"tasks.jobs.orphan_recovery._reconcile_task_results"
) as mock_reconcile,
patch(
"tasks.jobs.attack_paths.cleanup.cleanup_stale_attack_paths_scans",
return_value={},
),
):
result = reconcile_orphans(grace_minutes=2, max_attempts=3, dry_run=False)
mock_reconcile.assert_not_called()
assert result["acquired"] is True
assert result["enabled"] is False
@override_settings(TASK_RECOVERY_ENABLED=True)
def test_master_flag_enabled_runs_task_recovery(self):
with (
patch(
"tasks.jobs.orphan_recovery._reconcile_task_results",
return_value={"recovered": [], "failed": [], "skipped": []},
) as mock_reconcile,
patch(
"tasks.jobs.attack_paths.cleanup.cleanup_stale_attack_paths_scans",
return_value={},
),
):
reconcile_orphans(grace_minutes=2, max_attempts=3, dry_run=False)
mock_reconcile.assert_called_once()
+73 -128
View File
@@ -32,15 +32,12 @@ from tasks.utils import CustomEncoder
from api.db_router import MainRouter
from api.exceptions import ProviderConnectionError
from api.models import (
AttackSurfaceOverview,
Finding,
MuteRule,
Provider,
Resource,
ResourceScanSummary,
Scan,
ScanCategorySummary,
ScanGroupSummary,
ScanSummary,
StateChoices,
StatusChoices,
@@ -232,131 +229,6 @@ class TestPerformScan:
# Assert that failed_findings_count is 0 (finding is PASS and muted)
assert scan_resource.failed_findings_count == 0
def test_perform_prowler_scan_idempotent_on_rerun(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
"""Re-running a scan for the same scan_id must not duplicate findings."""
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch("tasks.jobs.scan.ProwlerScan") as mock_prowler_scan_class,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE",
new_callable=dict,
),
patch("api.compliance.PROWLER_CHECKS", new_callable=dict) as mock_checks,
):
mock_checks["aws"] = {"check1": {"compliance1"}}
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
provider_id = str(provider.id)
stale_resource = Resource.objects.create(
tenant_id=tenant.id,
provider=provider,
uid="stale_resource_uid",
name="stale",
region="stale-region",
service="stale-service",
type="stale-type",
)
ResourceScanSummary.objects.create(
tenant_id=tenant.id,
scan_id=scan.id,
resource_id=stale_resource.id,
service="stale-service",
region="stale-region",
resource_type="stale-type",
)
ScanCategorySummary.objects.create(
tenant_id=tenant.id,
scan=scan,
category="stale-category",
severity=Severity.medium,
total_findings=1,
)
ScanGroupSummary.objects.create(
tenant_id=tenant.id,
scan=scan,
resource_group="stale-group",
severity=Severity.medium,
total_findings=1,
)
ScanSummary.objects.create(
tenant_id=tenant.id,
scan=scan,
check_id="stale_check",
service="stale-service",
severity=Severity.medium,
region="stale-region",
total=1,
)
AttackSurfaceOverview.objects.create(
tenant_id=tenant.id,
scan=scan,
attack_surface_type=AttackSurfaceOverview.AttackSurfaceTypeChoices.SECRETS,
total_findings=1,
)
finding = MagicMock()
finding.uid = "dup_probe_finding"
finding.status = StatusChoices.PASS
finding.status_extended = "x"
finding.severity = Severity.medium
finding.check_id = "check1"
finding.get_metadata.return_value = {"key": "value"}
finding.resource_uid = "resource_uid"
finding.resource_name = "resource_name"
finding.region = "region"
finding.service_name = "service_name"
finding.resource_type = "resource_type"
finding.resource_tags = {}
finding.muted = False
finding.raw = {}
finding.resource_metadata = {}
finding.resource_details = {}
finding.partition = "partition"
finding.compliance = {}
mock_scan_instance = MagicMock()
mock_scan_instance.scan.return_value = [(100, [finding])]
mock_prowler_scan_class.return_value = mock_scan_instance
mock_provider_instance = MagicMock()
mock_provider_instance.get_regions.return_value = ["region"]
mock_initialize_prowler_provider.return_value = mock_provider_instance
# Run the same scan twice (simulating an orphan-recovery re-run).
perform_prowler_scan(tenant_id, scan_id, provider_id, ["check1"])
perform_prowler_scan(tenant_id, scan_id, provider_id, ["check1"])
# Neither findings nor resources are duplicated by the re-run: findings are
# scope-deleted before re-insert; resources are upserted by (tenant, provider, uid).
assert Finding.objects.filter(scan=scan).count() == 1
assert Resource.objects.filter(provider=provider).count() == 2
assert ResourceScanSummary.objects.filter(scan_id=scan.id).count() == 1
assert not ResourceScanSummary.objects.filter(
scan_id=scan.id, resource_id=stale_resource.id
).exists()
assert not ScanCategorySummary.objects.filter(scan=scan).exists()
assert not ScanGroupSummary.objects.filter(scan=scan).exists()
assert not ScanSummary.objects.filter(
scan=scan, check_id="stale_check"
).exists()
assert not AttackSurfaceOverview.objects.filter(scan=scan).exists()
@patch("tasks.jobs.scan.ProwlerScan")
@patch(
"tasks.jobs.scan.initialize_prowler_provider",
@@ -443,6 +315,7 @@ class TestPerformScan:
provider=provider_instance,
uid=finding.resource_uid,
defaults={
"name": finding.resource_name,
"region": finding.region,
"service": finding.service_name,
"type": finding.resource_type,
@@ -476,6 +349,7 @@ class TestPerformScan:
resource_instance = MagicMock()
resource_instance.uid = finding.resource_uid
resource_instance.name = "old_name"
resource_instance.region = "us-west-1"
resource_instance.service = "old_service"
resource_instance.type = "old_type"
@@ -494,6 +368,7 @@ class TestPerformScan:
provider=provider_instance,
uid=finding.resource_uid,
defaults={
"name": finding.resource_name,
"region": finding.region,
"service": finding.service_name,
"type": finding.resource_type,
@@ -501,6 +376,7 @@ class TestPerformScan:
)
# Check that resource fields were updated
assert resource_instance.name == finding.resource_name
assert resource_instance.region == finding.region
assert resource_instance.service == finding.service_name
assert resource_instance.type == finding.resource_type
@@ -1693,6 +1569,75 @@ class TestProcessFindingMicroBatch:
assert resource_cache[finding.resource_uid].service == finding.service_name
assert tag_cache.keys() == {("team", "devsec")}
def test_process_finding_micro_batch_refreshes_empty_resource_name(
self, tenants_fixture, scans_fixture
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = scan.provider
# Old resource stored before names were persisted: empty name.
existing_resource = Resource.objects.create(
tenant_id=tenant.id,
provider=provider,
uid="arn:aws:s3:::my-bucket",
name="",
region="us-east-1",
service="s3",
type="bucket",
)
finding = FakeFinding(
uid="finding-empty-name",
status=StatusChoices.PASS,
status_extended="passing",
severity=Severity.low,
check_id="s3_bucket_public_access",
resource_uid=existing_resource.uid,
resource_name="my-bucket",
region="us-east-1",
service_name="s3",
resource_type="bucket",
partition="aws",
raw={"status": "PASS"},
metadata={"source": "prowler"},
)
resource_cache = {existing_resource.uid: existing_resource}
tag_cache = {}
last_status_cache = {}
resource_failed_findings_cache = {existing_resource.uid: 0}
unique_resources: set[tuple[str, str]] = set()
scan_resource_cache: set[tuple[str, str, str, str]] = set()
mute_rules_cache = {}
scan_categories_cache: dict[tuple[str, str], dict[str, int]] = {}
scan_resource_groups_cache: dict[tuple[str, str], dict[str, int]] = {}
group_resources_cache: dict[str, set] = {}
with (
patch("tasks.jobs.scan.rls_transaction", new=noop_rls_transaction),
patch("api.db_utils.rls_transaction", new=noop_rls_transaction),
):
_process_finding_micro_batch(
str(tenant.id),
[finding],
scan,
provider,
resource_cache,
tag_cache,
last_status_cache,
resource_failed_findings_cache,
unique_resources,
scan_resource_cache,
mute_rules_cache,
scan_categories_cache,
scan_resource_groups_cache,
group_resources_cache,
)
existing_resource.refresh_from_db()
assert existing_resource.name == finding.resource_name
def test_process_finding_micro_batch_skips_long_uid(
self, tenants_fixture, scans_fixture
):
Generated
+97 -26
View File
@@ -163,7 +163,7 @@ constraints = [
{ name = "drf-simple-apikey", specifier = "==2.2.1" },
{ name = "drf-spectacular", specifier = "==0.27.2" },
{ name = "drf-spectacular-jsonapi", specifier = "==0.5.1" },
{ name = "dulwich", specifier = "==0.23.0" },
{ name = "dulwich", specifier = "==1.2.5" },
{ name = "duo-client", specifier = "==5.5.0" },
{ name = "durationpy", specifier = "==0.10" },
{ name = "email-validator", specifier = "==2.2.0" },
@@ -291,7 +291,7 @@ constraints = [
{ name = "pydantic-core", specifier = "==2.41.5" },
{ name = "pygithub", specifier = "==2.8.0" },
{ name = "pygments", specifier = "==2.20.0" },
{ name = "pyjwt", specifier = "==2.12.1" },
{ name = "pyjwt", specifier = "==2.13.0" },
{ name = "pylint", specifier = "==3.2.5" },
{ name = "pymsalruntime", specifier = "==0.18.1" },
{ name = "pynacl", specifier = "==1.6.2" },
@@ -374,8 +374,10 @@ constraints = [
{ name = "zstd", specifier = "==1.5.7.3" },
]
overrides = [
{ name = "dulwich", specifier = "==1.2.5" },
{ name = "microsoft-kiota-abstractions", specifier = "==1.9.9" },
{ name = "okta", specifier = "==3.4.2" },
{ name = "pyjwt", extras = ["crypto"], specifier = "==2.13.0" },
]
[[package]]
@@ -393,7 +395,7 @@ version = "1.2.7"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cryptography" },
{ name = "pyjwt" },
{ name = "pyjwt", extra = ["crypto"] },
{ name = "python-dateutil" },
{ name = "requests" },
]
@@ -1074,7 +1076,7 @@ dependencies = [
{ name = "pkginfo" },
{ name = "psutil", marker = "sys_platform != 'cygwin'" },
{ name = "py-deviceid" },
{ name = "pyjwt" },
{ name = "pyjwt", extra = ["crypto"] },
{ name = "pyopenssl" },
{ name = "requests", extra = ["socks"] },
]
@@ -2457,7 +2459,7 @@ source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "django" },
{ name = "djangorestframework" },
{ name = "pyjwt" },
{ name = "pyjwt", extra = ["crypto"] },
]
sdist = { url = "https://files.pythonhosted.org/packages/a8/27/2874a325c11112066139769f7794afae238a07ce6adf96259f08fd37a9d7/djangorestframework_simplejwt-5.5.1.tar.gz", hash = "sha256:e72c5572f51d7803021288e2057afcbd03f17fe11d484096f40a460abc76e87f", size = 101265, upload-time = "2025-07-21T16:52:25.026Z" }
wheels = [
@@ -2576,24 +2578,27 @@ wheels = [
[[package]]
name = "dulwich"
version = "0.23.0"
version = "1.2.5"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.12'" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/4b/ac/ba58cf420640c7bc77ae8e1b31e174d83c9117750c63cf9ea3b5e202e5c4/dulwich-0.23.0.tar.gz", hash = "sha256:0aa6c2489dd5e978b27e9b75983b7331a66c999f0efc54ebe37cab808ed322ae", size = 575116, upload-time = "2025-06-21T17:56:47.494Z" }
sdist = { url = "https://files.pythonhosted.org/packages/7f/85/ceb8ecff5cdeee4ceeebb86b599476dee559041dacc6c2c50cc0d4711549/dulwich-1.2.5.tar.gz", hash = "sha256:0395b2c8924c3424bafe2d9c1edd5348cc4b21ce9c1d6655bf01f9a5c47164c8", size = 1253230, upload-time = "2026-05-28T22:27:55.17Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ae/11/f6bbba8583f69cf19ef4bd7f5fde1a6b5ccaf8b6951781cec8db247116f4/dulwich-0.23.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d68498fdda13ab00791b483daab3bcfe9f9721c037aa458695e6ad81640c57cc", size = 972658, upload-time = "2025-06-21T17:56:13.505Z" },
{ url = "https://files.pythonhosted.org/packages/2b/9d/2720e0ab58666378a33c752a61543f936cd6b06dfe5d84a2215ddc0914b0/dulwich-0.23.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:cb7bb930b12471a1cfcea4b3d25a671dc0ad32573f0ad25684684298959a1527", size = 1049813, upload-time = "2025-06-21T17:56:14.884Z" },
{ url = "https://files.pythonhosted.org/packages/e5/f3/81d8075141dfcc0a0449c2093596e58d3e11444e3af54e819eca63b84dd0/dulwich-0.23.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:a2abbce32fd2bc7902bcc5f69b10bf22576810de21651baaa864b78fd7aec261", size = 1051639, upload-time = "2025-06-21T17:56:16.437Z" },
{ url = "https://files.pythonhosted.org/packages/4f/0d/c06ccb227b096aef5906142fe78b5c79f9070a0ea6152fc219941186d540/dulwich-0.23.0-cp311-cp311-win32.whl", hash = "sha256:9e3151f10ce2a9ff91bca64c74345217f53bdd947dc958032343822009832f7a", size = 642918, upload-time = "2025-06-21T17:56:18.373Z" },
{ url = "https://files.pythonhosted.org/packages/d7/1c/1e99aa34c9aead9e641b2d9934f0a3d00257f75027cf5cdecc8a1a6c18ae/dulwich-0.23.0-cp311-cp311-win_amd64.whl", hash = "sha256:3ae9f1d9dc92d4e9a3f89ba2c55221f7b6442c5dd93b3f6f539a3c9eb3f37bdd", size = 659010, upload-time = "2025-06-21T17:56:19.947Z" },
{ url = "https://files.pythonhosted.org/packages/4a/d7/1e6fba0235babe912e8467b036062e37d11672cbbeb0d8074f9d4559057b/dulwich-0.23.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:52cdef66a7994d29528ca79ca59452518bbba3fd56a9c61c61f6c467c1c7956e", size = 960292, upload-time = "2025-06-21T17:56:21.308Z" },
{ url = "https://files.pythonhosted.org/packages/4b/6a/23f0c487ec03f2752600cab4a8e0dedb38186246c475bf3fa90a8db830d5/dulwich-0.23.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:d473888a6ab9ed5d4a4c3f053cbe5b77f72d54b6efdf5688fed76094316e571e", size = 1047892, upload-time = "2025-06-21T17:56:22.989Z" },
{ url = "https://files.pythonhosted.org/packages/c7/e2/8f3d216be5fd0ee1180d917b59b34b54b9896384cf139f319b5d3a8f16b4/dulwich-0.23.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:19fcf20224c641a61c774da92f098fbaae9938c7e17a52841e64092adf7e78f9", size = 1048699, upload-time = "2025-06-21T17:56:24.602Z" },
{ url = "https://files.pythonhosted.org/packages/8f/c4/18e6223cd4ad1ae9334eb4e6aa5952fd8f5c3d75762918eb90c209fec4ba/dulwich-0.23.0-cp312-cp312-win32.whl", hash = "sha256:7fc8b76b704ef35cd001e993e3aa4e1d666a2064bf467c07c560f12b2959dcaf", size = 641268, upload-time = "2025-06-21T17:56:26.18Z" },
{ url = "https://files.pythonhosted.org/packages/b8/9c/65bfbbac62d8a2967e13f6a1512371c5eb6b906a61fb6dead992669cad0e/dulwich-0.23.0-cp312-cp312-win_amd64.whl", hash = "sha256:cb0566b888b578325350b4d67c61a0de35d417e9877560e3a6df88cae4576a59", size = 657837, upload-time = "2025-06-21T17:56:27.821Z" },
{ url = "https://files.pythonhosted.org/packages/35/31/49318ee9db4b402e6d8b9b01bd4cae9298f59e1bb9bd56cf4a94e48fa069/dulwich-0.23.0-py3-none-any.whl", hash = "sha256:d8da6694ca332bb48775e35ee2215aa4673821164a91b83062f699c69f7cd135", size = 313776, upload-time = "2025-06-21T17:56:46.221Z" },
{ url = "https://files.pythonhosted.org/packages/4a/4a/654ae1671610fdf6b65a64586ad67ddd8550d4d08a632b2a4b9614754b6d/dulwich-1.2.5-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:556593fd11637f80f6018bee1916b1a84f5b420423b470ebb3f1a782ad6ef081", size = 1399277, upload-time = "2026-05-28T22:27:00.801Z" },
{ url = "https://files.pythonhosted.org/packages/85/d8/06ee3bc8eded4bd7adf8adf0c9ea5f19bf96f7e5e626bfaf7311cde4208a/dulwich-1.2.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a70477c991e96cfe8fdd7c866e7251faf71b38bfeb51d6f27554c9cce1caabf3", size = 1382310, upload-time = "2026-05-28T22:27:02.216Z" },
{ url = "https://files.pythonhosted.org/packages/07/17/a03adf50b9095f9f5d863393f21d585dea39bdc4fdf60788ff3a9407a512/dulwich-1.2.5-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:9008ef25cabd379cda4fa86000fc38ca14b72afe17db798a8c85c0b2b7ce4d1e", size = 1470993, upload-time = "2026-05-28T22:27:04.075Z" },
{ url = "https://files.pythonhosted.org/packages/60/58/1dc352d2a5e80befe4338af7208febb44bcfd7496b0dde5ac6dacb07b031/dulwich-1.2.5-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:a5549f4afc973e0a15ea6b0244d57f848d3f3ee13dac557eb311024aebebf128", size = 1497820, upload-time = "2026-05-28T22:27:05.549Z" },
{ url = "https://files.pythonhosted.org/packages/c1/a8/e058959a87e7df7753b112ef66a43ccbc57338c1bbdc23a0edf3833396df/dulwich-1.2.5-cp311-cp311-win32.whl", hash = "sha256:5108acead814d1de8b6262d6d8fb90af7e82f5a4d83788b6b48e39d01800a92f", size = 1066549, upload-time = "2026-05-28T22:27:06.832Z" },
{ url = "https://files.pythonhosted.org/packages/33/91/ff0b444f686718635348986bd73dfce42e947912417893de35de399b878b/dulwich-1.2.5-cp311-cp311-win_amd64.whl", hash = "sha256:5e067b7feceb7034bc99e7c7143a704f1d97d4be7027d9a0aa5a83c0657ff091", size = 1079481, upload-time = "2026-05-28T22:27:08.33Z" },
{ url = "https://files.pythonhosted.org/packages/19/22/4f75770bbe5521cac61c4820ef46d4fbf8c2175d3519ba3d0378d4ba798e/dulwich-1.2.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:701a9ecf7a8a44f5e2459e46befa93530cf36a8b1ae3140aefc007db1d7d0207", size = 1396522, upload-time = "2026-05-28T22:27:09.997Z" },
{ url = "https://files.pythonhosted.org/packages/e5/b1/c07c347681c0cf6acd4b189bf6e8d6207c71a1347b7a1e865eb40faa46b9/dulwich-1.2.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2f90d68bfa97c4ca71de7507984365aefe27b6d248cb28dc99644d0f3ae8c60b", size = 1334826, upload-time = "2026-05-28T22:27:11.582Z" },
{ url = "https://files.pythonhosted.org/packages/13/80/6818eb7ce492e18ab2efa92ab901d173b4b0b159e5681c1424f329600c40/dulwich-1.2.5-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:00b54a1d56ddbacdd8eadd6d4787a51b3a05fefa30eadbf9165fd283a00b90ed", size = 1416616, upload-time = "2026-05-28T22:27:13.195Z" },
{ url = "https://files.pythonhosted.org/packages/14/a7/9790e60d19870f6554f7583722bb324c1355784316f20aeda1c0b5b1491a/dulwich-1.2.5-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:d8f7ea8f47e38e5b0de3fab97e07e9c9161ffddc90b3964512cab2b7749df4e6", size = 1441354, upload-time = "2026-05-28T22:27:14.683Z" },
{ url = "https://files.pythonhosted.org/packages/91/44/0ea8a69c24aa1254ff5996d682eae2eab287d471b937dcdb26d9ea9720b4/dulwich-1.2.5-cp312-cp312-win32.whl", hash = "sha256:8929134acf4ff967203df7600b38535f9b5b590462067a7e30dbce01acb97af9", size = 1017058, upload-time = "2026-05-28T22:27:16.121Z" },
{ url = "https://files.pythonhosted.org/packages/bd/9c/2fcddda7faec3bae52db7c64bfcb5dc756f597f33fae90e8d4e4b4d3b39b/dulwich-1.2.5-cp312-cp312-win_amd64.whl", hash = "sha256:9693d2c9e226b2ea855c1dc3a87e2f4d972f7523fc0f7924e5997e9f4c23d97f", size = 1031731, upload-time = "2026-05-28T22:27:17.633Z" },
{ url = "https://files.pythonhosted.org/packages/07/4b/4a18a59ad230581cd0ef460e96001f90762e566dc2dfdba22aa358eb5a0e/dulwich-1.2.5-py3-none-any.whl", hash = "sha256:1679b376433a0fc7f36586afda1d4ed7427afa7a79d4bf17e5014474eea69fa4", size = 686745, upload-time = "2026-05-28T22:27:53.695Z" },
]
[[package]]
@@ -4031,7 +4036,7 @@ dependencies = [
{ name = "pycryptodomex" },
{ name = "pydantic" },
{ name = "pydash" },
{ name = "pyjwt" },
{ name = "pyjwt", extra = ["crypto"] },
{ name = "python-dateutil" },
{ name = "pyyaml" },
{ name = "requests" },
@@ -4410,8 +4415,8 @@ wheels = [
[[package]]
name = "prowler"
version = "5.27.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=master#0abbb7fc590eaf7de6ed354dd5a217bca261d2b0" }
version = "5.30.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=master#f1d741214a60df17158c3fdc97804fd1fde64f3a" }
dependencies = [
{ name = "alibabacloud-actiontrail20200706" },
{ name = "alibabacloud-credentials" },
@@ -4484,9 +4489,14 @@ dependencies = [
{ name = "pygithub" },
{ name = "python-dateutil" },
{ name = "pytz" },
{ name = "scaleway" },
{ name = "schema" },
{ name = "shodan" },
{ name = "slack-sdk" },
{ name = "stackit-core" },
{ name = "stackit-iaas" },
{ name = "stackit-objectstorage" },
{ name = "stackit-resourcemanager" },
{ name = "tabulate" },
{ name = "tzlocal" },
{ name = "uuid6" },
@@ -4494,7 +4504,7 @@ dependencies = [
[[package]]
name = "prowler-api"
version = "1.31.0"
version = "1.32.0"
source = { virtual = "." }
dependencies = [
{ name = "cartography" },
@@ -4873,11 +4883,11 @@ wheels = [
[[package]]
name = "pyjwt"
version = "2.12.1"
version = "2.13.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c2/27/a3b6e5bf6ff856d2509292e95c8f57f0df7017cf5394921fc4e4ef40308a/pyjwt-2.12.1.tar.gz", hash = "sha256:c74a7a2adf861c04d002db713dd85f84beb242228e671280bf709d765b03672b", size = 102564, upload-time = "2026-03-13T19:27:37.25Z" }
sdist = { url = "https://files.pythonhosted.org/packages/3b/81/58d0ac84e1ef3a3843791d6954d94c0b33d526c75eeb1efbce9d0a4c4077/pyjwt-2.13.0.tar.gz", hash = "sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423", size = 107515, upload-time = "2026-05-21T19:54:36.618Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/e5/7a/8dd906bd22e79e47397a61742927f6747fe93242ef86645ee9092e610244/pyjwt-2.12.1-py3-none-any.whl", hash = "sha256:28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c", size = 29726, upload-time = "2026-03-13T19:27:35.677Z" },
{ url = "https://files.pythonhosted.org/packages/a3/5e/ecf12fdb62546d64385c158514e9b2b671f7832108ef2ecd2020ce0af2d1/pyjwt-2.13.0-py3-none-any.whl", hash = "sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728", size = 31274, upload-time = "2026-05-21T19:54:35.362Z" },
]
[package.optional-dependencies]
@@ -5526,6 +5536,67 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/49/4b/359f28a903c13438ef59ebeee215fb25da53066db67b305c125f1c6d2a25/sqlparse-0.5.5-py3-none-any.whl", hash = "sha256:12a08b3bf3eec877c519589833aed092e2444e68240a3577e8e26148acc7b1ba", size = 46138, upload-time = "2025-12-19T07:17:46.573Z" },
]
[[package]]
name = "stackit-core"
version = "0.2.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cryptography" },
{ name = "pydantic" },
{ name = "pyjwt", extra = ["crypto"] },
{ name = "requests" },
{ name = "urllib3" },
]
sdist = { url = "https://files.pythonhosted.org/packages/24/90/20f9ec7387eec4067cfd3d29055d0e2b5e1e0322c601a7f48125fd8ea35f/stackit_core-0.2.0.tar.gz", hash = "sha256:b8af91877cdb060d6969a303d8cf20bc0b33b345afd91f679c44a987381e2d47", size = 8987, upload-time = "2025-06-12T08:24:45.251Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/ab/b4/7b53187ce68956870d864ccb9ccfb68066c9df9de1c9568fd2feb03c4504/stackit_core-0.2.0-py3-none-any.whl", hash = "sha256:04632fc6742790d08ddfcb7f2313e04d1254827397a80250f838a2f81b92645b", size = 10240, upload-time = "2025-06-12T08:24:44.214Z" },
]
[[package]]
name = "stackit-iaas"
version = "1.4.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pydantic" },
{ name = "python-dateutil" },
{ name = "requests" },
{ name = "stackit-core" },
]
sdist = { url = "https://files.pythonhosted.org/packages/52/07/24e65278300d5c3cb19cb1660bff924c80812cf8aad3e715f826bae5aa80/stackit_iaas-1.4.0.tar.gz", hash = "sha256:93523b23442350c7ebefd9129485c4c2a539f694a9c36a0f8edfaba9862057ea", size = 116236, upload-time = "2026-05-13T09:43:15.996Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/08/51/2201164d7bfacf47539888c735f10f6320c188252384957aa1b23121a210/stackit_iaas-1.4.0-py3-none-any.whl", hash = "sha256:3f4a32321b57ac238f73e5d660c6428186b92cc0425c1f0783ba801e377149d9", size = 316588, upload-time = "2026-05-13T09:43:14.943Z" },
]
[[package]]
name = "stackit-objectstorage"
version = "1.4.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pydantic" },
{ name = "python-dateutil" },
{ name = "requests" },
{ name = "stackit-core" },
]
sdist = { url = "https://files.pythonhosted.org/packages/90/80/b790756af40a5c6d979dd688b2557394ac54b594eb4c08edc33157ba890f/stackit_objectstorage-1.4.0.tar.gz", hash = "sha256:4a3812b4de102b199f061706a802909f9e53ae9b0858769d5bd720f814c8bdbe", size = 31814, upload-time = "2026-05-13T09:43:05.027Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/68/f1/ffa8d5e2ec9f818c72a6f045691364eb4e927ee86641993a70882d00205a/stackit_objectstorage-1.4.0-py3-none-any.whl", hash = "sha256:1a3285c6840d95cff591d84fd21803575cb0d010c398e6575ed92987b9c39866", size = 65061, upload-time = "2026-05-13T09:43:04.13Z" },
]
[[package]]
name = "stackit-resourcemanager"
version = "0.8.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pydantic" },
{ name = "python-dateutil" },
{ name = "requests" },
{ name = "stackit-core" },
]
sdist = { url = "https://files.pythonhosted.org/packages/23/2d/f458f18e48ed2b1c83df52cff7dbdfd5dd904fb2980ffd9385876e47bbd9/stackit_resourcemanager-0.8.0.tar.gz", hash = "sha256:f44542beab4130857f5a7f465cf02defeef657bdf63c1beeb3102f0ba3c003fe", size = 33943, upload-time = "2026-05-13T09:43:08.667Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/c7/9c/38a74d0f7a89b4320f6d2366fb660638bda8860daa08748b12c713d84381/stackit_resourcemanager-0.8.0-py3-none-any.whl", hash = "sha256:dd04bb8353d041a137c4dcba190beabded7acfaff1bc98b218fce20a99389ebc", size = 81288, upload-time = "2026-05-13T09:43:07.81Z" },
]
[[package]]
name = "statsd"
version = "4.0.1"
@@ -5785,7 +5856,7 @@ source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "cryptography" },
{ name = "httpx" },
{ name = "pyjwt" },
{ name = "pyjwt", extra = ["crypto"] },
]
sdist = { url = "https://files.pythonhosted.org/packages/3c/2f/99fb8718274116c5c146c745755620fd5c5943f78ca52ca9b17e94348286/workos-6.0.4.tar.gz", hash = "sha256:b0bfe8fd212b8567422c4ea3732eb33608794033eb3a69900c6b04db183c32d6", size = 172217, upload-time = "2026-04-16T03:09:28.583Z" }
wheels = [
+180
View File
@@ -1538,6 +1538,186 @@ def get_section_container_iso(data, section_1, section_2):
return html.Div(section_containers, className="compliance-data-layout")
def _status_bar(success, failed, classname):
"""Build the stacked PASS/FAIL bar shown next to an accordion title."""
fig = go.Figure(
data=[
go.Bar(
name="Failed",
x=[failed],
y=[""],
orientation="h",
marker=dict(color="#e77676"),
width=[0.8],
),
go.Bar(
name="Success",
x=[success],
y=[""],
orientation="h",
marker=dict(color="#45cc6e"),
width=[0.8],
),
]
)
fig.update_layout(
barmode="stack",
margin=dict(l=10, r=10, t=10, b=10),
paper_bgcolor="rgba(0,0,0,0)",
plot_bgcolor="rgba(0,0,0,0)",
showlegend=False,
width=350,
height=30,
xaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
yaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
annotations=[
dict(
x=success + failed,
y=0,
xref="x",
yref="y",
text=str(success),
showarrow=False,
font=dict(color="#45cc6e", size=14),
xanchor="left",
yanchor="middle",
),
dict(
x=0,
y=0,
xref="x",
yref="y",
text=str(failed),
showarrow=False,
font=dict(color="#e77676", size=14),
xanchor="right",
yanchor="middle",
),
],
)
fig.add_annotation(
x=failed,
y=0.3,
text="|",
showarrow=False,
xanchor="center",
yanchor="middle",
font=dict(size=20),
)
return dcc.Graph(figure=fig, config={"staticPlot": True}, className=classname)
def get_section_containers_generic(data, section_col, id_col):
"""Two-level view: section -> requirement id (+ description) -> checks.
Sorts lexicographically so arbitrary requirement IDs never crash the
version-aware sort used by the CIS renderer.
"""
data["STATUS"] = data["STATUS"].apply(map_status_to_icon)
data[section_col] = data[section_col].astype(str)
data[id_col] = data[id_col].astype(str)
data.sort_values(by=[section_col, id_col], inplace=True)
counts_section = data.groupby([section_col, "STATUS"]).size().unstack(fill_value=0)
counts_id = (
data.groupby([section_col, id_col, "STATUS"]).size().unstack(fill_value=0)
)
def count(counts, key, emoji):
return counts.loc[key, emoji] if emoji in counts.columns else 0
has_description = "REQUIREMENTS_DESCRIPTION" in data.columns
table_cols = ["CHECKID", "STATUS", "REGION", "ACCOUNTID", "RESOURCEID"]
section_containers = []
for section in data[section_col].unique():
graph_div = html.Div(
_status_bar(
count(counts_section, section, pass_emoji),
count(counts_section, section, fail_emoji),
"info-bar",
),
className="graph-section",
)
internal_items = []
for req_id in data[data[section_col] == section][id_col].unique():
specific_data = data[
(data[section_col] == section) & (data[id_col] == req_id)
]
data_table = dash_table.DataTable(
data=specific_data.to_dict("records"),
columns=[
{"name": i, "id": i}
for i in table_cols
if i in specific_data.columns
],
style_table={"overflowX": "auto"},
style_as_list_view=True,
style_cell={"textAlign": "left", "padding": "5px"},
)
graph_div_req = html.Div(
_status_bar(
count(counts_id, (section, req_id), pass_emoji),
count(counts_id, (section, req_id), fail_emoji),
"info-bar-child",
),
className="graph-section-req",
)
title = req_id
if has_description:
title = (
f"{req_id} - {specific_data['REQUIREMENTS_DESCRIPTION'].iloc[0]}"
)
if len(title) > 130:
title = title[:130] + " ..."
internal_items.append(
html.Div(
[
graph_div_req,
dbc.Accordion(
[
dbc.AccordionItem(
title=title,
children=[
html.Div(
[data_table],
className="inner-accordion-content",
)
],
)
],
start_collapsed=True,
flush=True,
),
],
className="accordion-inner--child",
)
)
section_containers.append(
html.Div(
[
graph_div,
dbc.Accordion(
[
dbc.AccordionItem(
title=f"{section}", children=internal_items
)
],
start_collapsed=True,
flush=True,
),
],
className="accordion-inner",
)
)
return html.Div(section_containers, className="compliance-data-layout")
def get_section_containers_format4(data, section_1):
data["STATUS"] = data["STATUS"].apply(map_status_to_icon)
+44
View File
@@ -0,0 +1,44 @@
import warnings
from dashboard.common_methods import (
get_section_containers_format4,
get_section_containers_generic,
)
warnings.filterwarnings("ignore")
def get_table(data):
# Discover REQUIREMENTS_ATTRIBUTES_* columns at runtime.
attr_cols = [c for c in data.columns if c.startswith("REQUIREMENTS_ATTRIBUTES_")]
# Section column (in priority order):
# 1. REQUIREMENTS_ATTRIBUTES_SECTION — most common convention
# 2. First discovered attribute column — covers novel schemas
# 3. None — no section, group flat by requirement id
if "REQUIREMENTS_ATTRIBUTES_SECTION" in attr_cols:
section_col = "REQUIREMENTS_ATTRIBUTES_SECTION"
elif attr_cols:
section_col = attr_cols[0]
else:
section_col = None
base_cols = [
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"STATUS",
"CHECKID",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
# Two levels (section -> requirement id) when a section distinct from the
# id exists; otherwise group flat by requirement id.
if section_col and section_col != "REQUIREMENTS_ID":
needed = [section_col] + base_cols
aux = data[[c for c in needed if c in data.columns]].copy()
return get_section_containers_generic(aux, section_col, "REQUIREMENTS_ID")
aux = data[[c for c in base_cols if c in data.columns]].copy()
return get_section_containers_format4(aux, "REQUIREMENTS_ID")
+1 -1
View File
@@ -156,7 +156,7 @@ def create_layout_compliance(
html.Img(src="assets/favicon.ico", className="w-5 mr-3"),
html.Span("Subscribe to Prowler Cloud"),
],
href="https://prowler.pro/",
href="https://cloud.prowler.com/",
target="_blank",
className="text-prowler-stone-900 inline-flex px-4 py-2 text-xs font-bold uppercase transition-all rounded-lg text-gray-900 hover:bg-prowler-stone-900/10 border-solid border-1 hover:border-prowler-stone-900/10 hover:border-solid hover:border-1 border-prowler-stone-900/10",
),
+57 -31
View File
@@ -215,6 +215,58 @@ else:
)
def _ensure_scope_columns(data):
"""Guarantee ACCOUNTID and REGION exist.
Scope columns always sit between DESCRIPTION and ASSESSMENTDATE, so derive
them positionally for any provider (e.g. Okta's ORGANIZATIONDOMAIN) and
fall back to "-" to avoid a KeyError.
"""
cols = list(data.columns)
scope = []
if "DESCRIPTION" in cols and "ASSESSMENTDATE" in cols:
start, end = cols.index("DESCRIPTION") + 1, cols.index("ASSESSMENTDATE")
scope = [c for c in cols[start:end] if c not in ("ACCOUNTID", "REGION")]
if "ACCOUNTID" not in data.columns:
if scope:
data.rename(columns={scope.pop(0): "ACCOUNTID"}, inplace=True)
else:
data["ACCOUNTID"] = "-"
if "REGION" not in data.columns:
if scope:
data.rename(columns={scope.pop(0): "REGION"}, inplace=True)
else:
data["REGION"] = "-"
return data
def _dispatch_compliance_renderer(data, analytics_input):
"""Resolve the compliance renderer module and return (table, deduped_data).
Tries to import the framework-specific builtin module. On
ModuleNotFoundError (dynamic/external provider with no dedicated module),
falls back to the generic renderer. Any other ImportError is re-raised.
get_table() is called OUTSIDE the try block so errors inside the renderer
surface as real exceptions rather than being swallowed.
"""
current = analytics_input.replace(".", "_")
target = f"dashboard.compliance.{current}"
try:
module = importlib.import_module(target)
except ModuleNotFoundError as exc:
if exc.name != target:
raise
from dashboard.compliance import generic as module
dedup_columns = ["CHECKID", "STATUS", "RESOURCEID", "STATUSEXTENDED"]
if "MUTED" in data.columns:
dedup_columns.insert(2, "MUTED")
data = data.drop_duplicates(subset=dedup_columns)
if "threatscore" in analytics_input:
data = get_threatscore_mean_by_pillar(data)
return module.get_table(data), data
@callback(
[
Output("output", "children"),
@@ -292,7 +344,7 @@ def display_data(
data.rename(columns={"TENANCYID": "ACCOUNTID"}, inplace=True)
# Filter the chosen level of the CIS
if is_level_1:
if is_level_1 and "REQUIREMENTS_ATTRIBUTES_PROFILE" in data.columns:
data = data[data["REQUIREMENTS_ATTRIBUTES_PROFILE"].str.contains("Level 1")]
# Rename the column PROJECTID to ACCOUNTID for GCP
@@ -314,6 +366,9 @@ def display_data(
data.rename(columns={"SUBSCRIPTION": "ACCOUNTID"}, inplace=True)
data["REGION"] = "-"
# Normalize scope columns for any remaining (e.g. dynamic) provider.
data = _ensure_scope_columns(data)
# Filter ACCOUNT
if account_filter == ["All"]:
updated_cloud_account_values = data["ACCOUNTID"].unique()
@@ -409,36 +464,7 @@ def display_data(
# Check cases where the compliance start with AWS_
if "aws_" in analytics_input:
analytics_input = analytics_input + "_aws"
try:
current = analytics_input.replace(".", "_")
compliance_module = importlib.import_module(
f"dashboard.compliance.{current}"
)
# Build subset list based on available columns
dedup_columns = ["CHECKID", "STATUS", "RESOURCEID", "STATUSEXTENDED"]
if "MUTED" in data.columns:
dedup_columns.insert(2, "MUTED")
data = data.drop_duplicates(subset=dedup_columns)
if "threatscore" in analytics_input:
data = get_threatscore_mean_by_pillar(data)
table = compliance_module.get_table(data)
except ModuleNotFoundError:
table = html.Div(
[
html.H5(
"No data found for this compliance",
className="card-title",
style={"text-align": "left", "color": "black"},
)
],
style={
"width": "99%",
"margin-right": "0.8%",
"margin-bottom": "10px",
},
)
table, data = _dispatch_compliance_renderer(data, analytics_input)
df = data.copy()
# Remove Muted rows
+1 -1
View File
@@ -1538,7 +1538,7 @@ def filter_data(
html.Img(src="assets/favicon.ico", className="w-5 mr-3"),
html.Span("Subscribe to Prowler Cloud"),
],
href="https://prowler.pro/",
href="https://cloud.prowler.com/",
target="_blank",
className="text-prowler-stone-900 inline-flex px-4 py-2 text-xs font-bold uppercase transition-all rounded-lg text-gray-900 hover:bg-prowler-stone-900/10 border-solid border-1 hover:border-prowler-stone-900/10 hover:border-solid hover:border-1 border-prowler-stone-900/10",
),
+75 -36
View File
@@ -8,7 +8,77 @@ This guide explains the AI Skills system that provides on-demand context and pat
**What are AI Skills?** Skills are structured instructions that help AI agents (Claude Code, Cursor, Copilot, etc.) understand Prowler's conventions, patterns, and best practices.
</Info>
## Architecture Overview
Skills live in the [`skills/`](https://github.com/prowler-cloud/prowler/tree/master/skills) directory of the Prowler OSS repository. Each skill is a folder containing a `SKILL.md` file with its patterns and metadata.
## Installation
To enable skills for the supported AI coding assistants, run the setup script from the repository root:
```bash
./skills/setup.sh
```
The script creates symlinks so each tool finds the skills in its expected location:
| Tool | Created by setup |
|------|------------------|
| Claude Code | `.claude/skills/` symlink and `CLAUDE.md` |
| Gemini CLI | `.gemini/skills/` symlink and `GEMINI.md` |
| Codex (OpenAI) | `.codex/skills/` symlink (uses `AGENTS.md` natively) |
| GitHub Copilot | `.github/copilot-instructions.md` symlink to `AGENTS.md` |
After running the setup, restart the AI coding assistant to load the skills.
## Using Skills
AI agents discover skills automatically and load them when a request matches a skill trigger. To load a skill manually during a session, point the agent to the skill's `SKILL.md` file:
```text
Read skills/{skill-name}/SKILL.md
```
For the full list of available skills, their triggers, and the Auto-invoke mappings, see the [`skills/README.md`](https://github.com/prowler-cloud/prowler/blob/master/skills/README.md) and [`AGENTS.md`](https://github.com/prowler-cloud/prowler/blob/master/AGENTS.md) in the repository.
## Available Skills
| Type | Skills |
|------|--------|
| **Generic** | typescript, react-19, nextjs-16, tailwind-4, pytest, playwright, django-drf, zod-4, zustand-5, ai-sdk-5, vitest, tdd |
| **Prowler** | prowler, prowler-sdk-check, prowler-api, prowler-ui, prowler-mcp, prowler-provider, prowler-compliance, prowler-compliance-review, prowler-docs, prowler-pr, prowler-ci, prowler-attack-paths-query |
| **Testing** | prowler-test-sdk, prowler-test-api, prowler-test-ui |
| **Meta** | skill-creator, skill-sync |
<Note>
This table is a snapshot. The repository is the source of truth: see [`skills/README.md`](https://github.com/prowler-cloud/prowler/blob/master/skills/README.md) for the current, complete list.
</Note>
## Skill Structure
Each skill follows the [Agent Skills spec](https://agentskills.io):
```text
skills/{skill-name}/
├── SKILL.md # Patterns, rules, decision trees
├── assets/ # Code templates, schemas
└── references/ # Links to local docs (single source of truth)
```
## Key Design Decisions
1. **Self-contained skills** - Critical patterns inline for fast loading
2. **Local doc references** - No web URLs, points to `docs/developer-guide/*.mdx`
3. **Single source of truth** - Skills reference docs, no duplication
4. **On-demand loading** - AI loads only what's needed for the task
## Creating New Skills
Use the `skill-creator` meta-skill to create new skills that follow the Agent Skills spec. See [`AGENTS.md`](https://github.com/prowler-cloud/prowler/blob/master/AGENTS.md) for the full list of available skills and their triggers.
## How Skills Work
The diagrams below explain the internals of the skill system. They are useful for understanding the design, but are not required to install or use skills.
### Architecture Overview
```mermaid
graph LR
@@ -28,7 +98,7 @@ graph LR
style F fill:#1a4d2e,stroke:#66bb6a,color:#fff
```
## How It Works
### Request Lifecycle
```mermaid
sequenceDiagram
@@ -68,7 +138,7 @@ sequenceDiagram
A->>U: Creates check with correct patterns
```
## Before vs After
### With and Without Skills
```mermaid
graph TD
@@ -96,7 +166,7 @@ graph TD
style AFTER fill:#1a4d1a,stroke:#66bb6a,color:#fff
```
## Complete Architecture
### Full Component Map
```mermaid
flowchart TB
@@ -110,7 +180,7 @@ flowchart TB
subgraph GENERIC["Generic Skills"]
G1["typescript"]
G2["react-19"]
G3["nextjs-15"]
G3["nextjs-16"]
G4["tailwind-4"]
G5["pytest"]
G6["playwright"]
@@ -186,34 +256,3 @@ flowchart TB
style STRUCTURE fill:#5c3d1a,stroke:#ffb74d,color:#fff
style DOCS fill:#1a3d4d,stroke:#4dd0e1,color:#fff
```
## Skills Included
| Type | Skills |
|------|--------|
| **Generic** | typescript, react-19, nextjs-15, tailwind-4, pytest, playwright, django-drf, zod-4, zustand-5, ai-sdk-5 |
| **Prowler** | prowler, prowler-sdk-check, prowler-api, prowler-ui, prowler-mcp, prowler-provider, prowler-compliance, prowler-compliance-review, prowler-docs, prowler-pr, prowler-ci |
| **Testing** | prowler-test-sdk, prowler-test-api, prowler-test-ui |
| **Meta** | skill-creator, skill-sync |
## Skill Structure
Each skill follows the [Agent Skills spec](https://agentskills.io):
```
skills/{skill-name}/
├── SKILL.md # Patterns, rules, decision trees
├── assets/ # Code templates, schemas
└── references/ # Links to local docs (single source of truth)
```
## Key Design Decisions
1. **Self-contained skills** - Critical patterns inline for fast loading
2. **Local doc references** - No web URLs, points to `docs/developer-guide/*.mdx`
3. **Single source of truth** - Skills reference docs, no duplication
4. **On-demand loading** - AI loads only what's needed for the task
## Creating New Skills
Use the `skill-creator` meta-skill to create new skills that follow the Agent Skills spec. See `AGENTS.md` for the full list of available skills and their triggers.
@@ -128,8 +128,8 @@ To update the environment file:
Edit the `.env` file and change version values:
```env
PROWLER_UI_VERSION="5.29.0"
PROWLER_API_VERSION="5.29.0"
PROWLER_UI_VERSION="5.30.0"
PROWLER_API_VERSION="5.30.0"
```
<Note>
@@ -35,14 +35,28 @@ The bundled checks require the following read-only scopes:
- `okta.policies.read`
- `okta.brands.read`
- `okta.apps.read`
- `okta.authenticators.read`
- `okta.networkZones.read`
- `okta.apiTokens.read`
- `okta.roles.read`
- `okta.groups.read`
- `okta.logStreams.read`
- `okta.idps.read`
Additional scopes will be needed as more services and checks are added. These are the current ones needed:
| Scope | Used by |
|---|---|
| `okta.policies.read` | Sign-on, password, and authentication policies |
| `okta.policies.read` | Sign-on, password, authentication, and `USER_LIFECYCLE` (Workflow > Automations) policies |
| `okta.brands.read` | Sign-in page customizations (DOD Notice and Consent Banner check) |
| `okta.apps.read` | First-party app settings (Okta Admin Console session), integrated app inventory, and the Authentication Policies bound to Okta applications |
| `okta.authenticators.read` | Okta authenticator configuration, including Okta Verify and Smart Card IdP |
| `okta.networkZones.read` | Network Zone inventory, anonymized-proxy blocklist checks, and API token Network Zone validation |
| `okta.apiTokens.read` | API token metadata and token network conditions |
| `okta.roles.read` | Admin role assignments for API token owners (both direct and group-inherited) |
| `okta.groups.read` | Group memberships of API token owners, used to resolve admin roles inherited via group assignment (e.g. Super Admin granted through the default admin group) |
| `okta.logStreams.read` | Log Stream configuration (`/api/v1/logStreams`) |
| `okta.idps.read` | Identity Providers, including Smart Card (X509) IdPs (`/api/v1/idps`) |
### Required Admin Role
@@ -68,7 +82,9 @@ Okta filters the first-party apps (`saasure`, `okta_enduser`) out of `/api/v1/ap
A fifth check — `application_admin_console_session_idle_timeout_15min` (STIG V-273187) — also requires Super Administrator: it calls `GET /api/v1/first-party-app-settings/admin-console`, which returns `403 E0000006` for every role below Super Administrator.
When the service app runs with Read-Only Administrator, the five checks listed in this section return **MANUAL** instead of PASS/FAIL — the rest of the scan keeps running.
`user_inactivity_automation_35d_enabled` (STIG V-273188) reads `USER_LIFECYCLE` policies (`list_policies(type='USER_LIFECYCLE')`) using the `okta.policies.read` scope. The Read-Only Administrator role is enough to list them; no Super Administrator requirement.
When the service app runs with Read-Only Administrator, the checks listed in this section return **MANUAL** instead of PASS/FAIL — the rest of the scan keeps running.
<Note>
Read-Only Administrator stays the recommended default for the least-privilege framing that aligns with DISA STIG. Assign Super Administrator on a separate run when full coverage of the first-party app checks is needed.
@@ -122,7 +138,7 @@ Okta displays the private key **only once**. If you close the modal without copy
### 5. Grant the required OAuth scopes
On the app, open the **Okta API Scopes** tab and click **Grant** on every scope Prowler needs. The bundled checks require `okta.policies.read`, `okta.brands.read`, and `okta.apps.read`.
On the app, open the **Okta API Scopes** tab and click **Grant** on every scope Prowler needs. The bundled checks require `okta.policies.read`, `okta.brands.read`, `okta.apps.read`, `okta.authenticators.read`, `okta.networkZones.read`, `okta.apiTokens.read`, `okta.roles.read`, `okta.groups.read`, `okta.logStreams.read`, and `okta.idps.read`.
![Okta — grant OAuth scopes](/user-guide/providers/okta/images/grant-permissions.png)
@@ -158,8 +174,8 @@ export OKTA_PRIVATE_KEY_FILE="/secure/path/to/prowler-okta.pem"
# or
export OKTA_PRIVATE_KEY="$(cat /secure/path/to/prowler-okta.pem)"
# Optional — defaults to "okta.policies.read,okta.brands.read,okta.apps.read"
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read"
# Optional — defaults to "okta.policies.read,okta.brands.read,okta.apps.read,okta.authenticators.read,okta.networkZones.read,okta.apiTokens.read,okta.roles.read,okta.groups.read,okta.logStreams.read,okta.idps.read"
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read,okta.authenticators.read,okta.networkZones.read,okta.apiTokens.read,okta.roles.read,okta.groups.read,okta.logStreams.read,okta.idps.read"
uv run python prowler-cli.py okta
```
@@ -200,7 +216,7 @@ Prowler validates credentials at startup by listing one sign-on policy. This err
Raised when the credential probe succeeds at the OAuth layer but the request is rejected because the service app lacks the required scope or admin role:
- **`invalid_scope`** — one of the requested scopes (`okta.policies.read`, `okta.brands.read`, or `okta.apps.read`) is not granted on the service app. Grant the missing scope from **Okta API Scopes**.
- **`invalid_scope`** — one of the requested scopes (`okta.policies.read`, `okta.brands.read`, `okta.apps.read`, `okta.authenticators.read`, `okta.networkZones.read`, `okta.apiTokens.read`, `okta.roles.read`, `okta.groups.read`, `okta.logStreams.read`, and `okta.idps.read`) is not granted on the service app. Grant the missing scope from **Okta API Scopes**.
- **`Forbidden` / `not authorized`** — no admin role is assigned to the service app. Assign **Read-Only Administrator** (or **Super Administrator** for the first-party application checks) from **Admin roles**.
### Application-service checks return MANUAL on first-party apps
@@ -12,7 +12,7 @@ Set up authentication for Okta with the [Okta Authentication](/user-guide/provid
- An Okta organization. The UI examples below use **Identity Engine** terminology such as **Global Session Policy**; Classic Engine exposes the equivalent sign-on policy concepts under older names.
- A **Super Administrator** account on that organization for the one-time service-app setup.
- An **API Services** app integration in the Okta Admin Console with the `okta.policies.read`, `okta.brands.read`, and `okta.apps.read` scopes granted and an admin role assigned. **Read-Only Administrator** covers every `signon` check and runs the per-app network-zone check against the apps the service app can see (under Read-Only Administrator that is typically only the service app's own row — the rest of the org's app inventory stays invisible). **Super Administrator** is required additionally to evaluate the five first-party application checks (Okta Admin Console / Okta Dashboard idle timeout, MFA, phishing-resistant authentication) and to widen the network-zone check to the full app inventory — see [Okta Authentication](/user-guide/providers/okta/authentication#required-admin-role) for the full breakdown.
- An **API Services** app integration in the Okta Admin Console with the `okta.policies.read`, `okta.brands.read`, `okta.apps.read`, `okta.authenticators.read`, `okta.networkZones.read`, `okta.apiTokens.read`, `okta.roles.read`, `okta.groups.read`, `okta.logStreams.read`, and `okta.idps.read` scopes granted and an admin role assigned. **Read-Only Administrator** covers the Sign-On, Network, API Token, User, System Log, and Identity Provider checks, and runs the per-app application network-zone check against the apps the service app can see (under Read-Only Administrator that is typically only the service app's own row — the rest of the org's app inventory stays invisible). **Super Administrator** is required additionally to evaluate the five first-party application checks (Okta Admin Console / Okta Dashboard idle timeout, MFA, phishing-resistant authentication) and to widen the application network-zone check to the full app inventory — see [Okta Authentication](/user-guide/providers/okta/authentication#required-admin-role) for the full breakdown.
- Python 3.10+ and Prowler 5.27.0 or later installed locally.
<CardGroup cols={2}>
@@ -85,8 +85,8 @@ Follow the [Okta Authentication](/user-guide/providers/okta/authentication) guid
export OKTA_ORG_DOMAIN="acme.okta.com"
export OKTA_CLIENT_ID="0oa1234567890abcdef"
export OKTA_PRIVATE_KEY_FILE="/secure/path/to/prowler-okta.pem"
# Optional — defaults to "okta.policies.read,okta.brands.read,okta.apps.read"
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read"
# Optional — defaults to "okta.policies.read,okta.brands.read,okta.apps.read,okta.authenticators.read,okta.networkZones.read,okta.apiTokens.read,okta.roles.read,okta.groups.read,okta.logStreams.read,okta.idps.read"
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read,okta.authenticators.read,okta.networkZones.read,okta.apiTokens.read,okta.roles.read,okta.groups.read,okta.logStreams.read,okta.idps.read"
```
The private key file may contain either a PEM-encoded RSA key or a JWK JSON document.
@@ -143,10 +143,16 @@ prowler okta --config-file /path/to/config.yaml
Prowler for Okta includes security checks across the following services:
| Service | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| **Sign-On** | Global session policy controls (idle timeout, lifetime, rule priority and ordering) |
| **Application** | Okta Admin Console sign-on settings plus Authentication Policy controls for Okta applications (session idle, MFA, phishing resistance, network zones) |
| Service | Description |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Sign-On** | Global session policy controls (idle timeout, lifetime, rule priority and ordering) |
| **Application** | Okta Admin Console sign-on settings plus Authentication Policy controls for Okta applications (session idle, MFA, phishing resistance, network zones) |
| **Authenticator** | Password Policy controls plus Okta Verify FIPS and Smart Card IdP authenticator status |
| **Network** | Network Zone blocklists for anonymized proxy sources |
| **API Token** | API token owner-role validation and Network Zone restrictions |
| **User** | User lifecycle automations (inactivity-based deprovisioning) |
| **System Log** | Log Stream configuration that off-loads audit records to a central SIEM |
| **Identity Provider** | Identity Providers, including Smart Card (X509) IdP status and certificate-chain visibility |
## Troubleshooting
@@ -158,22 +164,29 @@ This is stricter than simply finding the same timeout value somewhere else in th
### Default Scopes
Prowler requests a fixed set of OAuth scopes on every token exchange. The defaults cover every bundled check across the Sign-On and Application services:
Prowler requests a fixed set of OAuth scopes on every token exchange. The defaults cover every bundled check across the Sign-On, Application, Authenticator, Network, API Token, User, System Log, and Identity Provider services:
- `okta.policies.read`
- `okta.brands.read`
- `okta.apps.read`
- `okta.authenticators.read`
- `okta.networkZones.read`
- `okta.apiTokens.read`
- `okta.roles.read`
- `okta.groups.read`
- `okta.logStreams.read`
- `okta.idps.read`
The service app must have these scopes granted in the **Okta API Scopes** tab. When the granted set is narrower than the requested set, the token request fails with an `invalid_scope` error and the scan stops at provider initialization.
The service app must have these scopes granted in the **Okta API Scopes** tab. `okta.groups.read` is required so the API token Super Admin check can resolve admin roles inherited via group membership; without it the check falls back to direct-only role assignments and emits a best-effort caveat. When the granted set is narrower than the requested set, the token request fails with an `invalid_scope` error and the scan stops at provider initialization.
When additional checks are enabled — or when running against a service app that exposes a different scope set — override the default with `OKTA_SCOPES` (comma-separated string for the env var) or `--okta-scopes` (space-separated list for the CLI):
```bash
# Environment variable — comma-separated
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read,okta.users.read"
export OKTA_SCOPES="okta.policies.read,okta.brands.read,okta.apps.read,okta.authenticators.read,okta.networkZones.read,okta.apiTokens.read,okta.roles.read,okta.groups.read,okta.logStreams.read,okta.idps.read,okta.users.read"
# CLI flag — space-separated
prowler okta --okta-scopes okta.policies.read okta.brands.read okta.apps.read okta.users.read
prowler okta --okta-scopes okta.policies.read okta.brands.read okta.apps.read okta.authenticators.read okta.networkZones.read okta.apiTokens.read okta.roles.read okta.groups.read okta.logStreams.read okta.idps.read okta.users.read
```
For the full catalog of OAuth scopes exposed by the Okta Management API, refer to the [Okta OAuth 2.0 scopes documentation](https://developer.okta.com/docs/api/oauth2/).
+3 -2
View File
@@ -12,8 +12,9 @@ reason = """
CVE-2025-45768 is disputed by the pyjwt maintainers. The advisory describes
weak encryption, but the underlying issue is that callers may pick a short
HMAC secret key-length enforcement is the application's responsibility, not
a defect in the library. We are on pyjwt 2.12.1 (latest at pin time) and
enforce key strength in our own auth code, so this advisory does not apply.
a defect in the library. We are on pyjwt 2.13.0 (which now also emits an
InsecureKeyLengthWarning for short HMAC secrets) and enforce key strength in
our own auth code, so this advisory does not apply.
Re-evaluate when a non-disputed advisory or upstream fix lands.
"""
+42 -3
View File
@@ -2,24 +2,63 @@
All notable changes to the **Prowler SDK** are documented in this file.
## [5.30.0] (Prowler UNRELEASED)
## [5.31.0] (Prowler UNRELEASED)
### 🚀 Added
- `securityhub_delegated_admin_enabled_all_regions` check for AWS provider, verifying that Security Hub has a delegated administrator, is active in all opted-in regions, and has organization auto-enable on [(#11259)](https://github.com/prowler-cloud/prowler/pull/11259)
- `config_delegated_admin_and_org_aggregator_all_regions` check for AWS provider, verifying that AWS Config has a delegated administrator and an organization aggregator covering all AWS regions [(#11259)](https://github.com/prowler-cloud/prowler/pull/11259)
- `sagemaker_clarify_exists` check for AWS provider [(#11211)](https://github.com/prowler-cloud/prowler/pull/11211)
- `cloudsql_instance_high_availability_enabled` check for GCP provider, verifying Cloud SQL primary instances use `REGIONAL` availability for automatic zone failover [(#11024)](https://github.com/prowler-cloud/prowler/pull/11024)
- `identity_storage_service_level_admins_scoped` check for OCI provider CIS 3.1 control 1.15, ensuring storage service-level administrators exclude delete permissions [(#11523)](https://github.com/prowler-cloud/prowler/pull/11523)
- `cosmosdb_account_automatic_failover_enabled` check for Azure provider [(#11031)](https://github.com/prowler-cloud/prowler/pull/11031)
---
## [5.30.0] (Prowler v5.30.0)
### 🚀 Added
- DISA Okta IDaaS STIG V1R2 compliance framework for the Okta provider, with a dedicated CSV output formatter and terminal summary table [(#11428)](https://github.com/prowler-cloud/prowler/pull/11428)
- `sagemaker_models_monitor_enabled` check for AWS provider, verifying that each SageMaker monitoring schedule is in the `Scheduled` state so data and model drift is actively detected [(#11278)](https://github.com/prowler-cloud/prowler/pull/11278)
- DORA (Digital Operational Resilience Act, Regulation (EU) 2022/2554) universal compliance framework with AWS provider coverage across the five DORA pillars [(#11131)](https://github.com/prowler-cloud/prowler/pull/11131)
- Okta authenticator and password policy checks for STIG-aligned hardening requirements [(#11465)](https://github.com/prowler-cloud/prowler/pull/11465)
- Okta network zone check to detect whether anonymized proxy traffic is blocked [(#11463)](https://github.com/prowler-cloud/prowler/pull/11463)
- Okta API token checks for super admin ownership and network zone restrictions [(#11464)](https://github.com/prowler-cloud/prowler/pull/11464)
- Support for external/custom providers, checks, and compliance frameworks without modifying core code [(#10700)](https://github.com/prowler-cloud/prowler/pull/10700)
- Public `Provider.get_class()` method that resolves a provider class by name for both built-in and external (entry-point) providers [(#11398)](https://github.com/prowler-cloud/prowler/pull/11398)
- `elbv2_alb_drop_invalid_header_fields_enabled` check for AWS provider, verifying Application Load Balancers have `routing.http.drop_invalid_header_fields.enabled` set to `true` to mitigate HTTP desync attacks (AWS FSBP ELB.4) [(#11471)](https://github.com/prowler-cloud/prowler/pull/11471)
- `user`, `systemlog` and `idp` service for Okta provider with `user_inactivity_automation_35d_enabled`, `systemlog_streaming_enabled` and `idp_smart_card_dod_approved_ca` checks [(#11496)](https://github.com/prowler-cloud/prowler/pull/11496)
- External multi-provider compliance frameworks can be registered via the `prowler.compliance.universal` entry point group [(#11490)](https://github.com/prowler-cloud/prowler/pull/11490)
- AWS AI Security Framework support in the CLI dashboard [(#11475)](https://github.com/prowler-cloud/prowler/pull/11475)
- `entra_service_principal_privileged_role_no_owners` check for M365 provider, failing when a service principal with a permanent Tier 0 directory role has owners on the service principal or its parent app registration [(#11070)](https://github.com/prowler-cloud/prowler/issues/11070)
- `kms_key_rotation_max_90_days` check for GCP provider, verifying KMS customer-managed keys are rotated every 90 days or less in line with the CIS Benchmark [(#11516)](https://github.com/prowler-cloud/prowler/pull/11516)
- `exchange_mailbox_primary_smtp_uses_custom_domain` check for M365 provider [(#11215)](https://github.com/prowler-cloud/prowler/pull/11215)
- `bedrock_agent_role_least_privilege` check for AWS provider, flagging Bedrock Agent execution roles with full-access managed policies, broad `Resource:*` inline statements, or missing permissions boundaries [(#11335)](https://github.com/prowler-cloud/prowler/pull/11335)
- STACKIT ObjectStorage service with Object Lock, default retention policy, and access key expiration checks [(#11397)](https://github.com/prowler-cloud/prowler/pull/11397)
### 🐞 Fixed
- `load_and_validate_config_file` now unwraps namespaced config for every built-in and external provider, and no longer leaks the full file as the provider's config when the file is namespaced [(#10700)](https://github.com/prowler-cloud/prowler/pull/10700)
- `entra_users_mfa_capable` no longer flags pre-provisioned users with future `employeeHireDate`; future-hire date comparisons now tolerate naive datetimes [(#11511)](https://github.com/prowler-cloud/prowler/pull/11511)
- M365 Admin Center group enumeration now follows Microsoft Graph pagination so group-scoped checks include groups beyond the first page [(#11510)](https://github.com/prowler-cloud/prowler/pull/11510)
- GCP `kms_key_rotation_enabled` check now only verifies that automatic key rotation is enabled (any interval) instead of enforcing a 90-day period, resolving the mismatch between the check and its documentation; the CIS, Prowler ThreatScore, and CCC requirements that mandate a 90-day maximum were remapped to the new `kms_key_rotation_max_90_days` check [(#11516)](https://github.com/prowler-cloud/prowler/pull/11516)
- AWS CloudWatch log metric filter checks now validate `filterPattern` clauses regardless of order [(#11345)](https://github.com/prowler-cloud/prowler/pull/11345)
- AWS `bedrock_api_key_no_long_term_credentials` now applies severity per finding (never-expires keys correctly flag as critical, no leak across findings) and aligns title and wording with AWS guidance to prefer short-term Bedrock API keys [(#11526)](https://github.com/prowler-cloud/prowler/pull/11526)
### 🔐 Security
- `dulwich` from 0.23.0 to 1.2.5 and `pyjwt` from 2.12.1 to 2.13.0, patching `GHSA-897w-fcg9-f6xj` (arbitrary file write) and `PYSEC-2026-179` (HMAC/JWK key confusion) [(#11499)](https://github.com/prowler-cloud/prowler/pull/11499)
---
## [5.29.3] (Prowler v5.29.3)
### 🐞 Fixed
- GCP `logging_sink_created` now recognizes organization-level aggregated sinks with `includeChildren=True`, avoiding false failures for covered projects [(#11355)](https://github.com/prowler-cloud/prowler/pull/11355)
- GCP `logging_log_metric_filter_and_alert_*` checks now recognize organization-level aggregated sinks with `includeChildren=True`, no longer false-failing projects covered by a central bucket-scoped metric + alert [(#11488)](https://github.com/prowler-cloud/prowler/pull/11488)
- Jira integration no longer fails with `400 INVALID_INPUT` when a finding has empty fields [(#11474)](https://github.com/prowler-cloud/prowler/pull/11474)
- GCP `iam_service_account_unused` now passes disabled service accounts instead of failing them, since a disabled account cannot authenticate or be used [(#11467)](https://github.com/prowler-cloud/prowler/pull/11467)
- AWS AI Security Framework now renders in the dashboard instead of showing "No data found for this compliance", by adding the missing compliance view module [(#11470)](https://github.com/prowler-cloud/prowler/pull/11470)
---
+36 -2
View File
@@ -19,7 +19,7 @@ from prowler.config.config import (
orange_color,
sarif_file_suffix,
)
from prowler.lib.banner import print_banner
from prowler.lib.banner import print_banner, print_prowler_cloud_banner
from prowler.lib.check.check import (
exclude_checks_to_run,
exclude_services_to_run,
@@ -102,6 +102,9 @@ from prowler.lib.outputs.compliance.mitre_attack.mitre_attack_azure import (
AzureMitreAttack,
)
from prowler.lib.outputs.compliance.mitre_attack.mitre_attack_gcp import GCPMitreAttack
from prowler.lib.outputs.compliance.okta_idaas_stig.okta_idaas_stig_okta import (
OktaIDaaSSTIG,
)
from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore_alibaba import (
ProwlerThreatScoreAlibaba,
)
@@ -199,7 +202,7 @@ def prowler():
if not args.no_banner:
legend = args.verbose or getattr(args, "fixer", None)
print_banner(legend)
print_banner(legend, provider)
# We treat the compliance framework as another output format
if compliance_framework:
@@ -1314,6 +1317,33 @@ def prowler():
)
generated_outputs["compliance"].append(generic_compliance)
generic_compliance.batch_write_data_to_file()
elif provider == "okta":
for compliance_name in input_compliance_frameworks:
if compliance_name.startswith("okta_idaas_stig"):
# Generate Okta IDaaS STIG Finding Object
filename = (
f"{output_options.output_directory}/compliance/"
f"{output_options.output_filename}_{compliance_name}.csv"
)
okta_idaas_stig = OktaIDaaSSTIG(
findings=finding_outputs,
compliance=bulk_compliance_frameworks[compliance_name],
file_path=filename,
)
generated_outputs["compliance"].append(okta_idaas_stig)
okta_idaas_stig.batch_write_data_to_file()
else:
filename = (
f"{output_options.output_directory}/compliance/"
f"{output_options.output_filename}_{compliance_name}.csv"
)
generic_compliance = GenericCompliance(
findings=finding_outputs,
compliance=bulk_compliance_frameworks[compliance_name],
file_path=filename,
)
generated_outputs["compliance"].append(generic_compliance)
generic_compliance.batch_write_data_to_file()
else:
# Dynamic fallback: any external/custom provider
try:
@@ -1446,6 +1476,10 @@ def prowler():
f"\nDetailed compliance results are in {Fore.YELLOW}{output_options.output_directory}/compliance/{Style.RESET_ALL}\n"
)
# Promote Prowler Cloud as the last thing the user sees after the results
if not args.no_banner and not args.only_logs:
print_prowler_cloud_banner(provider)
# If custom checks were passed, remove the modules
if checks_folder:
remove_custom_checks_module(checks_folder, provider)
@@ -1293,7 +1293,8 @@
"storage_ensure_private_endpoints_in_storage_accounts",
"storage_secure_transfer_required_is_enabled",
"vm_ensure_using_managed_disks",
"vm_trusted_launch_enabled"
"vm_trusted_launch_enabled",
"cosmosdb_account_automatic_failover_enabled"
]
},
{
+2 -1
View File
@@ -1087,7 +1087,8 @@
"storage_blob_versioning_is_enabled",
"storage_geo_redundant_enabled",
"vm_scaleset_associated_with_load_balancer",
"vm_scaleset_not_empty"
"vm_scaleset_not_empty",
"cosmosdb_account_automatic_failover_enabled"
],
"gcp": [
"compute_instance_automatic_restart_enabled",
+1 -1
View File
@@ -889,7 +889,7 @@
}
],
"Checks": [
"kms_key_rotation_enabled"
"kms_key_rotation_max_90_days"
]
},
{
+1 -1
View File
@@ -150,7 +150,7 @@
"Id": "1.10",
"Description": "Google Cloud Key Management Service stores cryptographic keys in a hierarchical structure designed for useful and elegant access control management. The format for the rotation schedule depends on the client library that is used. For the gcloud command-line tool, the next rotation time must be in `ISO` or `RFC3339` format, and the rotation period must be in the form `INTEGERUNIT`, where units can be one of seconds (s), minutes (m), hours (h) or days (d).",
"Checks": [
"kms_key_rotation_enabled"
"kms_key_rotation_max_90_days"
],
"Attributes": [
{
+1 -1
View File
@@ -201,7 +201,7 @@
"Id": "1.10",
"Description": "Ensure KMS Encryption Keys Are Rotated Within a Period of 90 Days",
"Checks": [
"kms_key_rotation_enabled"
"kms_key_rotation_max_90_days"
],
"Attributes": [
{
+1 -1
View File
@@ -201,7 +201,7 @@
"Id": "1.10",
"Description": "Ensure KMS Encryption Keys Are Rotated Within a Period of 90 Days",
"Checks": [
"kms_key_rotation_enabled"
"kms_key_rotation_max_90_days"
],
"Attributes": [
{
@@ -117,7 +117,7 @@
"Id": "1.2.4",
"Description": "Ensure KMS Encryption Keys Are Rotated Within a Period of 90 Days",
"Checks": [
"kms_key_rotation_enabled"
"kms_key_rotation_max_90_days"
],
"Attributes": [
{
View File
@@ -0,0 +1,638 @@
{
"Framework": "Okta-IDaaS-STIG",
"Name": "DISA Okta Identity as a Service (IDaaS) STIG V1R2",
"Version": "1R2",
"Provider": "Okta",
"Description": "Defense Information Systems Agency (DISA) Security Technical Implementation Guide (STIG) for Okta Identity as a Service (IDaaS), Version 1 Release 2 (Benchmark Date: 05 Jan 2026).",
"Requirements": [
{
"Id": "OKTA-APP-000020",
"Name": "Okta must log out a session after a 15-minute period of inactivity.",
"Description": "A session timeout lock is a temporary action taken when a user stops work and moves away from the immediate physical vicinity of the information system but does not log out because of the temporary nature of the absence. Rather than relying on the user to manually lock their application session prior to vacating the vicinity, applications must be able to identify when a user's application session has idled and take action to initiate the session lock. The session lock is implemented at the point where session activity can be determined and/or controlled. This is typically at the operating system level and results in a system lock. However, it may be at the application level where the application interface window is secured instead. Satisfies: SRG-APP-000003, SRG-APP-000190",
"Checks": [
"signon_global_session_idle_timeout_15min"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273186r1098825_rule",
"StigID": "OKTA-APP-000020",
"CCI": [
"CCI-000057",
"CCI-001133"
],
"CheckText": "From the Admin Console: 1. Select Security >> Global Session Policy. 2. In the Default Policy, verify a rule is configured at Priority 1 that is not named \"Default Rule\". 3. Click the edit icon next to the Priority 1 rule. 4. Verify the \"Maximum Okta global session idle time\" is set to 15 minutes. If \"Maximum Okta global session idle time\" is not set to 15 minutes, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Global Session Policy. 2. Select the Default Policy. 3. In the Rules table, make these updates: - Click \"Add rule\". - Set \"Maximum Okta global session idle time\" to 15 minutes."
}
]
},
{
"Id": "OKTA-APP-000025",
"Name": "The Okta Admin Console must log out a session after a 15-minute period of inactivity.",
"Description": "A session timeout lock is a temporary action taken when a user stops work and moves away from the immediate physical vicinity of the information system but does not log out because of the temporary nature of the absence. Rather than relying on the user to manually lock their application session prior to vacating the vicinity, applications must be able to identify when a user's application session has idled and take action to initiate the session lock. The session lock is implemented at the point where session activity can be determined and/or controlled. This is typically at the operating system level and results in a system lock. However, it may be at the application level where the application interface window is secured instead.",
"Checks": [
"application_admin_console_session_idle_timeout_15min"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273187r1098828_rule",
"StigID": "OKTA-APP-000025",
"CCI": [
"CCI-000057"
],
"CheckText": "From the Admin Console: 1. Select Applications >> Applications >> Okta Admin Console. 2. In the Sign On tab, under \"Okta Admin Console session\", verify the \"Maximum app session idle time\" is set to 15 minutes. If the \"Maximum app session idle time\" is not set to 15 minutes, this is a finding.",
"FixText": "From the Admin Console: 1. Select Applications >> Applications >> Okta Admin Console. 2. In the Sign On tab, under \"Okta Admin Console session\", set the \"Maximum app session idle time\" to 15 minutes."
}
]
},
{
"Id": "OKTA-APP-000090",
"Name": "Okta must automatically disable accounts after a 35-day period of account inactivity.",
"Description": "Attackers that are able to exploit an inactive account can potentially obtain and maintain undetected access to an application. Owners of inactive accounts will not notice if unauthorized access to their user account has been obtained. Applications must track periods of user inactivity and disable accounts after 35 days of inactivity. Such a process greatly reduces the risk that accounts will be hijacked, leading to a data compromise. To address access requirements, many application developers choose to integrate their applications with enterprise-level authentication/access mechanisms that meet or exceed access control policy requirements. Such integration allows the application developer to off-load those access control functions and focus on core application features and functionality. This policy does not apply to emergency accounts or infrequently used accounts. Infrequently used accounts are local login administrator accounts used by system administrators when network or normal login/access is not available. Emergency accounts are administrator accounts created in response to crisis situations. Satisfies: SRG-APP-000025, SRG-APP-000163, SRG-APP-000700",
"Checks": [
"user_inactivity_automation_35d_enabled"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273188r1098831_rule",
"StigID": "OKTA-APP-000090",
"CCI": [
"CCI-000017",
"CCI-000795",
"CCI-003627"
],
"CheckText": "If Okta Services rely on external directory services for user sourcing, this is not applicable, and the connected directory services must perform this function. Go to Workflows >> Automations and verify that an Automation has been created to disable accounts after 35 days of inactivity. If the Okta configuration does not automatically disable accounts after a 35-day period of account inactivity, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Workflow >> Automations and select \"Add Automation\". 2. Create a name for the Automation (e.g., \"User Inactivity\"). 3. Click \"Add Condition\" and select \"User Inactivity in Okta\". 4. In the duration field, enter 35 days and click \"Save\". 5 Click the edit button next to \"Select Schedule\". 6. Configure the \"Schedule\" field for \"Run Daily\" and set the \"Time\" field to an organizationally defined time to run this automation. Click \"Save\". 7. Click the edit button next to \"Select group membership\". 8. In the \"Applies to\" field, select the group \"Everyone\" by typing it into the field. Click \"Save\". 9. Click \"Add Action\" and select \"Change User lifecycle state in Okta\". 10. In the \"Change user state to\" field, select \"Suspended\" and click \"Save\". 11. Click the \"Inactive\" button near the top of the section screen and select \"Activate\"."
}
]
},
{
"Id": "OKTA-APP-000170",
"Name": "Okta must enforce the limit of three consecutive invalid login attempts by a user during a 15-minute time period.",
"Description": "By limiting the number of failed login attempts, the risk of unauthorized system access via user password guessing, otherwise known as brute forcing, is reduced. Limits are imposed by locking the account. Satisfies: SRG-APP-000065, SRG-APP-000345",
"Checks": [
"authenticator_password_lockout_threshold_3"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273189r1098834_rule",
"StigID": "OKTA-APP-000170",
"CCI": [
"CCI-000044",
"CCI-002238"
],
"CheckText": "If Okta Services rely on external directory services for user sourcing, this check is not applicable, and the connected directory services must perform this function. From the Admin Console: 1. Go to Security >> Authenticators. 2. Click the \"Actions\" button next to \"Password\" and select \"Edit\". 3. For each Password Policy, verify the \"Lock Out\" section has the following values: - \"Lock out after 3 unsuccessful attempts\" is checked. - The value is set to \"3\". If Okta Services are not configured to automatically lock user accounts after three consecutive invalid login attempts, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authenticators. 2. Click the \"Actions\" button next to \"Password\" and select \"Edit\". 3. For each Password Policy, ensure the \"Lock Out\" section has the following values: - \"Lock out after 3 unsuccessful attempts\" is checked. - The value is set to \"3\"."
}
]
},
{
"Id": "OKTA-APP-000180",
"Name": "The Okta Dashboard application must be configured to allow authentication only via non-phishable authenticators.",
"Description": "Requiring the use of non-phishable authenticators protects against brute force/password dictionary attacks. This provides a better level of security while removing the need to lock out accounts after three attempts in 15 minutes.",
"Checks": [
"application_dashboard_phishing_resistant_authentication"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273190r1099763_rule",
"StigID": "OKTA-APP-000180",
"CCI": [
"CCI-000044"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Dashboard\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"Possession factor constraints are\" section, verify the \"Phishing resistant\" box is checked. This will ensure that only phishing-resistant factors are used to access the Okta Dashboard. If in the \"Possession factor constraints are\" section the \"Phishing resistant\" box is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Dashboard\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"Possession factor constraints are\" section, ensure the \"Phishing resistant\" box is checked."
}
]
},
{
"Id": "OKTA-APP-000190",
"Name": "The Okta Admin Console application must be configured to allow authentication only via non-phishable authenticators.",
"Description": "Requiring the use of non-phishable authenticators protects against brute force/password dictionary attacks. This provides a better level of security while removing the need to lock out accounts after three attempts in 15 minutes.",
"Checks": [
"application_admin_console_phishing_resistant_authentication"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273191r1099764_rule",
"StigID": "OKTA-APP-000190",
"CCI": [
"CCI-000044"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Admin Console\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"Possession factor constraints are\" section, verify the \"Phishing resistant\" box is checked. This will ensure that only phishing-resistant factors are used to access the Okta Dashboard. If in the \"Possession factor constraints are\" section the \"Phishing resistant\" box is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Admin Console\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"Possession factor constraints are\" section, ensure the \"Phishing resistant\" box is checked."
}
]
},
{
"Id": "OKTA-APP-000200",
"Name": "Okta must display the Standard Mandatory DOD Notice and Consent Banner before granting access to the application.",
"Description": "Display of the DOD-approved use notification before granting access to the application ensures that privacy and security notification verbiage used is consistent with applicable federal laws, Executive Orders, directives, policies, regulations, standards, and guidance. System use notifications are required only for access via login interfaces with human users and are not required when such human interfaces do not exist. The banner must be formatted in accordance with DTM-08-060. Use the following verbiage for applications that can accommodate banners of 1300 characters: \"You are accessing a U.S. Government (USG) Information System (IS) that is provided for USG-authorized use only. By using this IS (which includes any device attached to this IS), you consent to the following conditions: -The USG routinely intercepts and monitors communications on this IS for purposes including, but not limited to, penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM), law enforcement (LE), and counterintelligence (CI) investigations. -At any time, the USG may inspect and seize data stored on this IS. -Communications using, or data stored on, this IS are not private, are subject to routine monitoring, interception, and search, and may be disclosed or used for any USG-authorized purpose. -This IS includes security measures (e.g., authentication and access controls) to protect USG interests--not for your personal benefit or privacy. -Notwithstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching or monitoring of the content of privileged communications, or work product, related to personal representation or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work product are private and confidential. See User Agreement for details.\" Use the following verbiage for operating systems that have severe limitations on the number of characters that can be displayed in the banner: \"I've read & consent to terms in IS user agreem't.\" Satisfies: SRG-APP-000068, SRG-APP-000069, SRG-APP-000070",
"Checks": [
"signon_dod_warning_banner_configured"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273192r1098843_rule",
"StigID": "OKTA-APP-000200",
"CCI": [
"CCI-000048",
"CCI-000050",
"CCI-001384",
"CCI-001385",
"CCI-001386",
"CCI-001387",
"CCI-001388"
],
"CheckText": "Attempt to log in to the Okta tenant and verify the DOD-approved warning banner is in place. If the required warning banner is not present and complete, this is a finding.",
"FixText": "Follow the supplemental instructions in the \"Okta DOD Warning Banner Configuration Guide\" provided with this STIG package."
}
]
},
{
"Id": "OKTA-APP-000560",
"Name": "The Okta Admin Console application must be configured to use multifactor authentication.",
"Description": "Without the use of multifactor authentication, the ease of access to privileged functions is greatly increased. Multifactor authentication requires using two or more factors to achieve authentication. Factors include: (i) something a user knows (e.g., password/PIN); (ii) something a user has (e.g., cryptographic identification device, token); or (iii) something a user is (e.g., biometric). A privileged account is defined as an information system account with authorizations of a privileged user. Network access is defined as access to an information system by a user (or a process acting on behalf of a user) communicating through a network (e.g., local area network, wide area network, or the internet). Satisfies: SRG-APP-000149, SRG-APP-000154",
"Checks": [
"application_admin_console_mfa_required"
],
"Attributes": [
{
"Section": "CAT I (High)",
"Severity": "high",
"RuleID": "SV-273193r1098846_rule",
"StigID": "OKTA-APP-000560",
"CCI": [
"CCI-000765",
"CCI-004046"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Admin Console\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"User must authenticate with\" field, verify that either \"Password/IdP + Another factor\" or \"Any 2 factor types\" is selected. If either of these settings is incorrect, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Admin Console\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"User must authenticate with\" field, select either \"Password/IdP + Another factor\" or \"Any 2 factor types\"."
}
]
},
{
"Id": "OKTA-APP-000570",
"Name": "The Okta Dashboard application must be configured to use multifactor authentication.",
"Description": "To ensure accountability and prevent unauthenticated access, nonprivileged users must use multifactor authentication to prevent potential misuse and compromise of the system. Multifactor authentication uses two or more factors to achieve authentication. Factors include: (i) Something you know (e.g., password/PIN); (ii) Something you have (e.g., cryptographic identification device, token); or (iii) Something you are (e.g., biometric). A nonprivileged account is any information system account with authorizations of a nonprivileged user. Network access is any access to an application by a user (or process acting on behalf of a user) where the access is obtained through a network connection. Applications integrating with the DOD Active Directory and using the DOD CAC are examples of compliant multifactor authentication solutions. Satisfies: SRG-APP-000150, SRG-APP-000155",
"Checks": [
"application_dashboard_mfa_required"
],
"Attributes": [
{
"Section": "CAT I (High)",
"Severity": "high",
"RuleID": "SV-273194r1098849_rule",
"StigID": "OKTA-APP-000570",
"CCI": [
"CCI-000766",
"CCI-004046"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Dashboard\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"User must authenticate with\" field, verify that either \"Password/IdP + Another factor\" or \"Any 2 factor types\" is selected. If either of these settings is incorrect, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authentication Policies. 2. Click the \"Okta Dashboard\" policy. 3. Click the \"Actions\" button next to the top rule and select \"Edit\". 4. In the \"User must authenticate with\" field, select either \"Password/IdP + Another factor\" or \"Any 2 factor types\"."
}
]
},
{
"Id": "OKTA-APP-000650",
"Name": "Okta must enforce a minimum 15-character password length.",
"Description": "Password complexity, or strength, is a measure of the effectiveness of a password in resisting attempts at guessing and brute-force attacks. Password length is one factor of several that helps to determine strength and how long it takes to crack a password. The shorter the password, the lower the number of possible combinations that need to be tested before the password is compromised. Use of more characters in a password helps to exponentially increase the time and/or resources required to compromise the password.",
"Checks": [
"authenticator_password_minimum_length_15"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273195r1098852_rule",
"StigID": "OKTA-APP-000650",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify the \"Minimum Length\" field is set to at least \"15\" characters. If any policy is not set to at least \"15\", this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set the \"Minimum Length\" field to at least \"15\" characters."
}
]
},
{
"Id": "OKTA-APP-000670",
"Name": "Okta must enforce password complexity by requiring that at least one uppercase character be used.",
"Description": "Use of a complex password helps to increase the time and resources required to compromise the password. Password complexity, or strength, is a measure of the effectiveness of a password in resisting attempts at guessing and brute-force attacks. Password complexity is one factor of several that determine how long it takes to crack a password. The more complex the password is, the greater the number of possible combinations that need to be tested before the password is compromised.",
"Checks": [
"authenticator_password_complexity_uppercase"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273196r1098855_rule",
"StigID": "OKTA-APP-000670",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Upper case letter\" is checked. For each policy, if \"Upper case letter\" is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Upper case letter\" to checked."
}
]
},
{
"Id": "OKTA-APP-000680",
"Name": "Okta must enforce password complexity by requiring that at least one lowercase character be used.",
"Description": "Use of a complex password helps to increase the time and resources required to compromise the password. Password complexity, or strength, is a measure of the effectiveness of a password in resisting attempts at guessing and brute-force attacks. Password complexity is one factor of several that determine how long it takes to crack a password. The more complex the password, the greater the number of possible combinations that need to be tested before the password is compromised.",
"Checks": [
"authenticator_password_complexity_lowercase"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273197r1098858_rule",
"StigID": "OKTA-APP-000680",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Lower case letter\" is checked. For each policy, if \"Lower case letter\" is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Lower case letter\" to checked."
}
]
},
{
"Id": "OKTA-APP-000690",
"Name": "Okta must enforce password complexity by requiring that at least one numeric character be used.",
"Description": "Use of a complex password helps to increase the time and resources required to compromise the password. Password complexity, or strength, is a measure of the effectiveness of a password in resisting attempts at guessing and brute-force attacks. Password complexity is one factor of several that determine how long it takes to crack a password. The more complex the password, the greater the number of possible combinations that need to be tested before the password is compromised.",
"Checks": [
"authenticator_password_complexity_number"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273198r1098861_rule",
"StigID": "OKTA-APP-000690",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Number (0-9)\" is checked. For each policy, if \"Number (0-9)\" is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Number (0-9)\" to checked."
}
]
},
{
"Id": "OKTA-APP-000700",
"Name": "Okta must enforce password complexity by requiring that at least one special character be used.",
"Description": "Use of a complex password helps to increase the time and resources required to compromise the password. Password complexity, or strength, is a measure of the effectiveness of a password in resisting attempts at guessing and brute-force attacks. Password complexity is one factor in determining how long it takes to crack a password. The more complex the password, the greater the number of possible combinations that need to be tested before the password is compromised. Special characters are not alphanumeric. Examples include: ~ ! @ # $ % ^ *.",
"Checks": [
"authenticator_password_complexity_symbol"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273199r1098864_rule",
"StigID": "OKTA-APP-000700",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Symbol (e.g., !@#$%^&*)\" is checked. For each policy, if \"Symbol (e.g., !@#$%^&*)\" is not checked, this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Symbol (e.g., !@#$%^&*)\" to checked."
}
]
},
{
"Id": "OKTA-APP-000740",
"Name": "Okta must enforce 24 hours/one day as the minimum password lifetime.",
"Description": "Enforcing a minimum password lifetime helps prevent repeated password changes to defeat the password reuse or history enforcement requirement. Restricting this setting limits the user's ability to change their password. Passwords must be changed at specific policy-based intervals; however, if the application allows the user to immediately and continually change their password, it could be changed repeatedly in a short period of time to defeat the organization's policy regarding password reuse. Satisfies: SRG-APP-000173, SRG-APP-000870",
"Checks": [
"authenticator_password_minimum_age_24h"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273200r1098867_rule",
"StigID": "OKTA-APP-000740",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Minimum password age is XX hours\" is set to at least \"24\". For each policy, if \"Minimum password age is XX hours\" is not set to at least \"24\", this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Minimum password age is XX hours\" to at least \"24\"."
}
]
},
{
"Id": "OKTA-APP-000745",
"Name": "Okta must enforce a 60-day maximum password lifetime restriction.",
"Description": "Any password, no matter how complex, can eventually be cracked. Therefore, passwords must be changed at specific intervals. One method of minimizing this risk is to use complex passwords and periodically change them. If the application does not limit the lifetime of passwords and force users to change their passwords, there is the risk that the system and/or application passwords could be compromised. This requirement does not include emergency administration accounts, which are meant for access to the application in case of failure. These accounts are not required to have maximum password lifetime restrictions.",
"Checks": [
"authenticator_password_maximum_age_60d"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273201r1098870_rule",
"StigID": "OKTA-APP-000745",
"CCI": [
"CCI-004066"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy, verify \"Password expires after XX days\" is set to \"60\". For each policy, if \"Password expires after XX days\" is not set to \"60\", this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Password expires after XX days\" to \"60\"."
}
]
},
{
"Id": "OKTA-APP-001430",
"Name": "Okta must off-load audit records onto a central log server.",
"Description": "Information stored in one location is vulnerable to accidental or incidental deletion or alteration. Off-loading is a common process in information systems with limited audit storage capacity. Satisfies: SRG-APP-000358, SRG-APP-000080, SRG-APP-000125",
"Checks": [
"systemlog_streaming_enabled"
],
"Attributes": [
{
"Section": "CAT I (High)",
"Severity": "high",
"RuleID": "SV-273202r1099766_rule",
"StigID": "OKTA-APP-001430",
"CCI": [
"CCI-001851",
"CCI-000166",
"CCI-001348"
],
"CheckText": "From the Admin Console: 1. Go to Reports >> Log Streaming. 2. Verify that a Log Stream connection is configured and active. Alternately, interview the information system security manager (ISSM) and verify that an external Security Information and Event Management (SIEM) system is pulling Okta logs via an Application Programming Interface (API). If either of these is not configured, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Reports >> Log Streaming. 2. Select either \"AWS EventBridge\" or \"Splunk Cloud\" and click \"Next\". 3. Complete the necessary fields and click \"Save\". If Log Streaming is not an option because the SIEM required is not an option, customers can use the Okta Log API to export system logs in real time."
}
]
},
{
"Id": "OKTA-APP-001665",
"Name": "Okta must be configured to limit the global session lifetime to 18 hours.",
"Description": "Without reauthentication, users may access resources or perform tasks for which they do not have authorization. When applications provide the capability to change security roles or escalate the functional capability of the application, it is critical the user reauthenticate. In addition to the reauthentication requirements associated with session locks, organizations may require reauthentication of individuals and/or devices in other situations, including (but not limited to) the following circumstances. (i) When authenticators change; (ii) When roles change; (iii) When security categories of information systems change; (iv) When the execution of privileged functions occurs; (v) After a fixed period of time; or (vi) Periodically. Within the DOD, the minimum circumstances requiring reauthentication are privilege escalation and role changes.",
"Checks": [
"signon_global_session_lifetime_18h"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273203r1099958_rule",
"StigID": "OKTA-APP-001665",
"CCI": [
"CCI-002038"
],
"CheckText": "From the Admin Console: 1. Select Security >> Global Session Policy. 2. In the Default Policy, verify a rule is configured at Priority 1 that is not named \"Default Rule\". 3. Click the \"Edit\" icon next to the Priority 1 rule. 4. Verify \"Maximum Okta global session lifetime\" is set to 18 hours. If the above is not set, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Global Session Policy. 2. Select the Default Policy. 3. In the Rules table, make these updates: - Click \"Add rule\". - Set \"Maximum Okta global session lifetime\" to 18 hours."
}
]
},
{
"Id": "OKTA-APP-001670",
"Name": "Okta must be configured to accept Personal Identity Verification (PIV) credentials.",
"Description": "The use of PIV credentials facilitates standardization and reduces the risk of unauthorized access. DOD has mandated the use of the common access card (CAC) to support identity management and personal authentication for systems covered under HSPD 12, as well as a primary component of layered protection for national security systems. Satisfies: SRG-APP-000391, SRG-APP-000402, SRG-APP-000403",
"Checks": [
"authenticator_smart_card_active"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273204r1098879_rule",
"StigID": "OKTA-APP-001670",
"CCI": [
"CCI-001953",
"CCI-002009",
"CCI-002010"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authenticators. 2. Verify that \"Smart Card Authenticator\" is listed and has \"Status\" listed as \"Active\". If \"Smart Card Authenticator\" is not listed or is not listed as \"Active\", this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authenticators. 2. In the \"Setup\" tab, click \"Add authenticator\". 3. Select the configured Smart Card Identity Provider and finish configuration."
}
]
},
{
"Id": "OKTA-APP-001700",
"Name": "The Okta Verify application must be configured to connect only to FIPS-compliant devices.",
"Description": "Without device-to-device authentication, communications with malicious devices may be established. Bidirectional authentication provides stronger safeguards to validate the identity of other devices for connections that are of greater risk. Currently, DOD requires the use of AES for bidirectional authentication because it is the only FIPS-validated AES cipher block algorithm. For distributed architectures (e.g., service-oriented architectures), the decisions regarding the validation of authentication claims may be made by services separate from the services acting on those decisions. In such situations, it is necessary to provide authentication decisions (as opposed to the actual authenticators) to the services that need to act on those decisions. A local connection is any connection with a device communicating without the use of a network. A network connection is any connection with a device that communicates through a network (e.g., local area or wide area network; the internet). A remote connection is any connection with a device communicating through an external network (e.g., the internet). Because of the challenges of applying this requirement on a large scale, organizations are encouraged to apply the requirement only to those limited number (and type) of devices that truly need to support this capability.",
"Checks": [
"authenticator_okta_verify_fips_compliant"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273205r1098882_rule",
"StigID": "OKTA-APP-001700",
"CCI": [
"CCI-001967"
],
"CheckText": "From the Admin Console: 1. Go to Security >> Authenticators. 2. From the \"Setup\" tab, select \"Edit Okta Verify\". 3. Review the \"FIPS Compliance\" field. If FIPS-compliant authentication is not enabled, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Authenticators. 2. From the \"Setup\" tab, select \"Edit Okta Verify\". 3. In the \"FIPS Compliance\" field, choose whether users enrolling in Okta Verify can use FIPS-compliant devices only or any device. 4. Click \"Save\" after making any changes."
}
]
},
{
"Id": "OKTA-APP-001710",
"Name": "Okta must be configured to disable persistent global session cookies.",
"Description": "If cached authentication information is out of date, the validity of the authentication information may be questionable. Satisfies: SRG-APP-000400, SRG-APP-000157",
"Checks": [
"signon_global_session_cookies_not_persistent"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273206r1098885_rule",
"StigID": "OKTA-APP-001710",
"CCI": [
"CCI-002007",
"CCI-001942"
],
"CheckText": "From the Admin Console: 1. Select Security >> Global Session Policy. 2. In the Default Policy, verify a rule is configured at Priority 1 that is not named \"Default Rule\". 3. Click the \"Edit\" icon next to the Priority 1 rule. 4. Verify \"Okta global session cookies persist across browser sessions\" is set to \"Disabled\". If the above it not set, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Global Session Policy. 2. Select the Default Policy. 3. In the \"Rules\" table, make these updates: - Click \"Add rule\". - Set \"Okta global session cookies persist across browser sessions\" to Disable."
}
]
},
{
"Id": "OKTA-APP-001920",
"Name": "Okta must be configured to use only DOD-approved certificate authorities.",
"Description": "Untrusted Certificate Authorities (CA) can issue certificates, but they may be issued by organizations or individuals that seek to compromise DOD systems or by organizations with insufficient security controls. If the CA used for verifying the certificate is not DOD approved, trust of this CA has not been established. The DOD will accept only PKI certificates obtained from a DOD-approved internal or external CA. Reliance on CAs for the establishment of secure sessions includes, for example, the use of Transport Layer Security (TLS) certificates. This requirement focuses on communications protection for the application session rather than for the network packet. This requirement applies to applications that use communications sessions. This includes, but is not limited to, web-based applications and Service-Oriented Architectures (SOA). Satisfies: SRG-APP-000427, SRG-APP-000910",
"Checks": [
"idp_smart_card_dod_approved_ca"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273207r1098888_rule",
"StigID": "OKTA-APP-001920",
"CCI": [
"CCI-002470",
"CCI-004909"
],
"CheckText": "From the Admin Console: 1. Select Security >> Identity Providers (IdPs). 2. Review the list of IdPs with \"Type\" as \"Smart Card\". If the IdP is not listed as \"Active\", this is a finding. 3. Select Actions >> Configure. 4. Under \"Certificate chain\", verify the certificate is from a DOD-approved CA. If the certificate is not from a DOD-approved CA, this is a finding.",
"FixText": "From the Admin Console: 1. Go to Security >> Identity Providers. 2. Click \"Add identity provider.\" 3. Click \"Smart Card IdP\". Click \"Next\". 4. Enter the name of the identity provider. 5. Build a certificate chain: - Click \"Browse\" to open a file explorer. Select the certificate file to add and click \"Open\". - To add another certificate, click \"Add Another\" and repeat step 1. - Click \"Build certificate chain\". On success, the chain and its certificates are shown. If the build failed, correct any issues and try again. - Click \"Reset certificate chain\" if replacing the current chain with a new one. 6. In \"IdP username\", select the \"idpuser.subjectAltNameUpn\" attribute. This is the attribute that stores the Electronic Data Interchange Personnel Identifier (EDIPI) on the CAC. 7. In the \"Match Against\" field, select the Okta Profile Attribute in which the EDIPI is to be stored."
}
]
},
{
"Id": "OKTA-APP-002980",
"Name": "Okta must validate passwords against a list of commonly used, expected, or compromised passwords.",
"Description": "Password-based authentication applies to passwords regardless of whether they are used in single-factor or multifactor authentication. Long passwords or passphrases are preferable over shorter passwords. Enforced composition rules provide marginal security benefits while decreasing usability. However, organizations may choose to establish certain rules for password generation (e.g., minimum character length for long passwords) under certain circumstances and can enforce this requirement in IA-5(1)(h). Account recovery can occur, for example, in situations when a password is forgotten. Cryptographically protected passwords include salted one-way cryptographic hashes of passwords. The list of commonly used, compromised, or expected passwords includes passwords obtained from previous breach corpuses, dictionary words, and repetitive or sequential characters. The list includes context-specific words, such as the name of the service, username, and derivatives thereof.",
"Checks": [
"authenticator_password_common_password_check"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273208r1099769_rule",
"StigID": "OKTA-APP-002980",
"CCI": [
"CCI-004058"
],
"CheckText": "From the Admin Console: 1. Navigate to Security >> Authenticators. 2. Click the \"Actions\" button next to the Password authenticator and select \"Edit\". 3. Under the \"Password Settings\" section, verify the \"Common Password Check\" box is checked. If \"Common Password Check\" is not selected, this is a finding.",
"FixText": "From the Admin Console: 1. Navigate to Security >> Authenticators. 2. Click the \"Actions\" button next to the Password authenticator and select \"Edit\". 3. Under the \"Password Settings\" section, check the \"Common Password Check\" box."
}
]
},
{
"Id": "OKTA-APP-003010",
"Name": "Okta must prohibit password reuse for a minimum of five generations.",
"Description": "Password-based authentication applies to passwords regardless of whether they are used in single-factor or multifactor authentication. Long passwords or passphrases are preferable over shorter passwords. Enforced composition rules provide marginal security benefits while decreasing usability. However, organizations may choose to establish certain rules for password generation (e.g., minimum character length for long passwords) under certain circumstances and can enforce this requirement in IA-5(1)(h). Account recovery can occur, for example, in situations when a password is forgotten. Cryptographically protected passwords include salted one-way cryptographic hashes of passwords. The list of commonly used, compromised, or expected passwords includes passwords obtained from previous breach corpuses, dictionary words, and repetitive or sequential characters. The list includes context-specific words, such as the name of the service, username, and derivatives thereof.",
"Checks": [
"authenticator_password_history_5"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-273209r1098894_rule",
"StigID": "OKTA-APP-003010",
"CCI": [
"CCI-004061"
],
"CheckText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password row\" and select \"Edit\". 3. For each listed policy, verify \"Enforce password history for last XX passwords\" is set to \"5\". If any policy is not set to at least \"5\", this is a finding.",
"FixText": "From the Admin Console: 1. Select Security >> Authenticators. 2. Click the \"Actions\" button next to the \"Password\" row and select \"Edit\". 3. For each listed policy: - Click \"Edit\". - Set \"Enforce password history for last XX passwords\" to \"5\"."
}
]
},
{
"Id": "OKTA-APP-003240",
"Name": "Okta API tokens must be configured with Network Zones to restrict authorization from known networks.",
"Description": "An access token is a piece of data that represents the authorization granted to a user or NPE to access specific systems or information resources. Access tokens enable controlled access to services and resources. Properly managing the lifecycle of access tokens, including their issuance, validation, and revocation, is crucial to maintaining confidentiality of data and systems. Restricting token validity to a specific audience, e.g., an application or security domain, and restricting token validity lifetimes are important practices. Access tokens are revoked or invalidated if they are compromised, lost, or are no longer needed to mitigate the risks associated with stolen or misused tokens. API tokens have the potential to be replicated or stolen (just like a password). Because of this, it is important to only allow API tokens to authenticate from known IP ranges as this limits an adversary's ability to use a token to gain access.",
"Checks": [
"apitoken_restricted_to_network_zone"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-279689r1155066_rule",
"StigID": "OKTA-APP-003240",
"CCI": [
"CCI-005165",
"CCI-000366"
],
"CheckText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"API\" item. 2. Click the \"Tokens\" tab. 3. For each token listed, click the token name link. 4. In the \"Security\" section, verify the \"Token can be used from\" setting is mapped to a known network zone for the application calling the API. If a network zone for each API access token is not defined, this is a finding.",
"FixText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"API\" item. 2. Click the \"Tokens\" tab. 3. For each token listed, click the token name link. 4. In the \"Security\" section, click \"Edit\". 5. Set the \"Token can be used from\" setting to the known network zone for the application calling the API. 6. Click \"Save\"."
}
]
},
{
"Id": "OKTA-APP-003241",
"Name": "Okta API tokens must be created under new dedicated user accounts.",
"Description": "An access token is a piece of data that represents the authorization granted to a user or NPE to access specific systems or information resources. Access tokens enable controlled access to services and resources. Properly managing the lifecycle of access tokens, including their issuance, validation, and revocation, is crucial to maintaining confidentiality of data and systems. Restricting token validity to a specific audience, e.g., an application or security domain, and restricting token validity lifetimes are important practices. Access tokens are revoked or invalidated if they are compromised, lost, or are no longer needed to mitigate the risks associated with stolen or misused tokens. When API tokens are created, they inherit the permissions of the user that created them. Therefore, API tokens should only be created from dedicated accounts and permissions must be constrained to least privilege for that dedicated user account and token. No API tokens should be created using a Super Admin account.",
"Checks": [
"apitoken_not_super_admin"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-279690r1155069_rule",
"StigID": "OKTA-APP-003241",
"CCI": [
"CCI-005165",
"CCI-000366"
],
"CheckText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"API\" item. 2. Click the \"Tokens\" tab. 3. For each token listed, verify that the Role listed is not \"Super Admin\", and that the account has been specifically created for that token. 4. Click the account name to be token to the user profile for that user. 5. Verify the user only has an administrator role (standard or customer) applied that is correctly scoped as required and documented in the Okta Access Control policy. If the token is using a Super Administrator account, or one that is not properly scoped per the Access Control policy, this is a finding. Note: If a Super Admin token is required for system operation, then this permanent finding.",
"FixText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"API\" item. 2. Click the \"Tokens\" tab. 3. For each token listed that has \"Super Admin\" or an improperly scoped Admin account, delete the token and create a new one with the appropriately scoped permissions. 4. Verify the application performing the API calls with the new token has been updated."
}
]
},
{
"Id": "OKTA-APP-003242",
"Name": "The Okta Global Session policy must be configured to allow or deny IP based access in accordance with the Access Control policy for Okta.",
"Description": "To mitigate the risk of unauthorized access to sensitive information by entities that have been issued certificates by DOD-approved PKIs, all DOD systems (e.g., networks, web servers, and web portals) must be properly configured to incorporate access control methods that do not rely solely on the possession of a certificate for access. Successful authentication must not automatically give an entity access to an asset or security boundary. Authorization procedures and controls must be implemented to ensure each authenticated entity also has a validated and current authorization. Authorization is the process of determining whether an entity, once authenticated, is permitted to access a specific asset. Information systems use access control policies and enforcement mechanisms to implement this requirement. Access Control policies include identity-based policies, role-based policies, and attribute-based policies. Access enforcement mechanisms include access control lists, access control matrices, and cryptography. These policies and mechanisms must be employed by the application to control access between users (or processes acting on behalf of users) and objects (e.g., devices, files, records, processes, programs, and domains) in the information system. The Okta Global Session Policy is applied at the organization level and before any application-specific authentication policies are processed. The Okta authorization package should contain an access control policy that defines IP ranges from which to either allow or deny access. This list (either as an explicit allow or explicit deny) can be implemented in the Global Session Policy.",
"Checks": [
"signon_global_session_policy_network_zone_enforced"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-279691r1155072_rule",
"StigID": "OKTA-APP-003242",
"CCI": [
"CCI-000213"
],
"CheckText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"Global Session Policy\" item. 2. In the \"Policy Settings\" section, verify the \"IF User's IP is\" setting is correctly set to either allow or deny based on the organization defined policy. If the Okta Global Session Policy is not configured to restrict access to specific IP ranges, this is a finding.",
"FixText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"Global Session Policy\" item. 2. In the Policy Settings section, configure the \"IF User's IP is\" setting to correctly set the appropriate network to either allow or deny based on the Access Control Policy."
}
]
},
{
"Id": "OKTA-APP-003243",
"Name": "Okta must be configured with Network Zones defined to block anonymized proxies according to organizationally defined policy.",
"Description": "A mechanism to detect and prevent unauthorized communication flow must be configured or provided as part of the system design. If information flow is not enforced based on approved authorizations, the system may become compromised. Information flow control regulates where information is allowed to travel within a system and between interconnected systems. The flow of all application information must be monitored and controlled so it does not introduce any unacceptable risk to the systems or data. Application-specific examples of enforcement occurs in systems that employ rule sets or establish configuration settings that restrict information system services, or provide a message filtering capability based on message content (e.g., implementing key word searches or using document characteristics). Applications providing information flow control must be able to enforce approved authorizations for controlling the flow of information between interconnected systems in accordance with applicable policy. Working with the organizational CSSP, the ISSM should obtain a list of known anonymizer proxies that exist on the commercial internet. If this is not available from the CSSP, then the Okta-provided \"Enhanced dynamic zone blocklist\" should be activated.",
"Checks": [
"network_zone_block_anonymized_proxies"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-279692r1155075_rule",
"StigID": "OKTA-APP-003243",
"CCI": [
"CCI-001414"
],
"CheckText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"Networks' item. 2. If the CSSP has provided a list of anonymizers to block, verify the \"IP Block list\" is configured with them. a. Click the pencil icon next to IP Block list. b. Verify the \"Gateway IPs\" section contains all of the IP ranges in the provided list. 3. If the CSSP is not able to provide a list, then implement the Okta managed list. a. Verify the \"Enhanced dynamic zone blocklist\" is set to \"Active\". If Network Zones are not configured to block anonymous proxies, this is a finding.",
"FixText": "From the Admin Console: 1. Select the \"Security\" menu, and then click the \"Networks\" item. 2. If the CSSP has provided a list of anonymizers to block, add the IP ranges to the \"IP Block list\". a. Click the pencil icon next to IP Block list. b. Add the IP ranges to the \"Gateway IPs\" section and click \"Save\". 3. If the CSSP is not able to provide a list, then implement the Okta managed list. a. Set the \"Enhanced dynamic zone blocklist\" to \"Active\"."
}
]
},
{
"Id": "OKTA-APP-003244",
"Name": "For each application integrated with Okta, network zones must be defined in its authentication policy.",
"Description": "A mechanism to detect and prevent unauthorized communication flow must be configured or provided as part of the system design. If information flow is not enforced based on approved authorizations, the system may become compromised. Information flow control regulates where information is allowed to travel within a system and between interconnected systems. The flow of all application information must be monitored and controlled so it does not introduce any unacceptable risk to the systems or data. Application-specific examples of enforcement occurs in systems that employ rule sets or establish configuration settings that restrict information system services, or provide a message filtering capability based on message content (e.g., implementing key word searches or using document characteristics). Applications providing information flow control must be able to enforce approved authorizations for controlling the flow of information between interconnected systems in accordance with applicable policy. Each application in Okta should have a well defined access control policy that takes into account the end user network. This should be documented in the Access Control policy for each application. As an example, access to an application may be restricted to a specific location by policy. In this case, a network defining that specific location should be created.",
"Checks": [
"application_authentication_policy_network_zone_enforced"
],
"Attributes": [
{
"Section": "CAT II (Medium)",
"Severity": "medium",
"RuleID": "SV-279693r1155078_rule",
"StigID": "OKTA-APP-003244",
"CCI": [
"CCI-001414"
],
"CheckText": "For each application integrated into Okta: 1. From the Admin console, open the \"Security\" menu, and then select \"Networks\". 2. Verify the list of networks includes all necessary allow or block lists. If any application is not configured with network zones, this is a finding.",
"FixText": "For each application, starting at the admin console: 1. Open the \"Applications\" group from the Menu, and then click the \"Applications\" menu item. 2. Click the application name. 3. Click the \"Sign On\" tab. 4. Scroll to the \"User Authentication\" section, and then click \"Edit\". 5. Select the appropriate Authentication policy from the pull down, and then click \"Save\". 6. Click \"View Policy Details\". 7. For each nondefault rule: a. Select \"Edit\" from the Actions menu. b. In the \"IF\" section, verify the \"User is\" setting has the appropriate allow or deny range has been selected based on the Access Control policy for the application. c. Scroll down to the bottom and click \"Save\". 8. For the Catch-All rule: a. Select \"Edit\" from the Actions menu. b. Scroll down to the \"Then\" section. c. For the \"Access is\" setting, select \"Denied\", and then click \"Save\"."
}
]
}
]
}
@@ -302,7 +302,9 @@
{
"Id": "1.15",
"Description": "Ensure storage service-level admins cannot delete resources they manage",
"Checks": [],
"Checks": [
"identity_storage_service_level_admins_scoped"
],
"Attributes": [
{
"Section": "1. Identity and Access Management",
+1 -1
View File
@@ -49,7 +49,7 @@ class _MutableTimestamp:
timestamp = _MutableTimestamp(datetime.today())
timestamp_utc = _MutableTimestamp(datetime.now(timezone.utc))
prowler_version = "5.30.0"
prowler_version = "5.31.0"
html_logo_url = "https://github.com/prowler-cloud/prowler/"
square_logo_img = "https://raw.githubusercontent.com/prowler-cloud/prowler/dc7d2d5aeb92fdf12e8604f42ef6472cd3e8e889/docs/img/prowler-logo-black.png"
aws_logo = "https://user-images.githubusercontent.com/38561120/235953920-3e3fba08-0795-41dc-b480-9bea57db9f2e.png"
+39 -4
View File
@@ -3,12 +3,13 @@ from colorama import Fore, Style
from prowler.config.config import banner_color, orange_color, prowler_version, timestamp
def print_banner(legend: bool = False):
def print_banner(legend: bool = False, provider: str = None):
"""
Prints the banner with optional legend for color codes.
Parameters:
- legend (bool): Flag to indicate whether to print the color legend or not. Default is False.
- provider (str): The provider being scanned, used to tailor the Prowler Cloud banner.
Returns:
- None
@@ -20,13 +21,12 @@ def print_banner(legend: bool = False):
| .__/|_| \___/ \_/\_/ |_|\___|_|v{prowler_version}
|_|{Fore.BLUE} Get the most at https://cloud.prowler.com {Style.RESET_ALL}
{Fore.GREEN}New! Send findings from Prowler CLI to Prowler Cloud{Style.RESET_ALL}
{Fore.GREEN}More details here: goto.prowler.com/import-findings{Style.RESET_ALL}
{Fore.YELLOW}Date: {timestamp.strftime("%Y-%m-%d %H:%M:%S")}{Style.RESET_ALL}
"""
print(banner)
print_prowler_cloud_banner(provider)
if legend:
print(
f"""
@@ -37,3 +37,38 @@ def print_banner(legend: bool = False):
- {Fore.RED}FAIL (Fix required){Style.RESET_ALL}
"""
)
def print_prowler_cloud_banner(provider: str = None):
"""
Prints a promotional banner highlighting what Prowler Cloud adds on top of
the open-source CLI.
Shown at the start and end of a scan to let users know about the managed
platform capabilities they are missing (attack paths, AI, organizations,
continuous scanning, integrations and live compliance dashboards).
Parameters:
- provider (str): The provider that was scanned, used to tailor the message.
Returns:
- None
"""
check = f"{Fore.GREEN}{Style.RESET_ALL}"
bar = f"{banner_color}{Style.RESET_ALL}"
print(
f"""
{bar} {Style.BRIGHT}You're getting a snapshot 📸. Prowler Cloud gives you the full picture:{Style.RESET_ALL}
{bar}
{bar} {check} {Style.BRIGHT}Continuous Security Monitoring{Style.RESET_ALL} - scheduled scans with history, trends and alerts.
{bar} {check} {Style.BRIGHT}Lighthouse AI + MCP{Style.RESET_ALL} - autonomous triage, custom dashboards, prioritization with prevention and remediation.
{bar} {check} {Style.BRIGHT}Alerts{Style.RESET_ALL} - get notified when anything you want is happening.
{bar} {check} {Style.BRIGHT}Live Compliance{Style.RESET_ALL} - dashboards for 50+ frameworks, always up to date.
{bar} {check} {Style.BRIGHT}Remediation{Style.RESET_ALL} - complete guided remediation including Autonomous remediation with Lighthouse AI.
{bar} {check} {Style.BRIGHT}Attack Path Visualization{Style.RESET_ALL} - see how attackers chain risks to reach your crown jewels.
{bar} {check} {Style.BRIGHT}Bulk Provisioning{Style.RESET_ALL} - add your entire AWS Organization in seconds.
{bar} {check} {Style.BRIGHT}Integrations{Style.RESET_ALL} - Anything with our MCP + Jira, Slack, AWS Security Hub, Amazon S3, SSO and RBAC.
{bar}
{bar} {Fore.BLUE}Start free at 👉 cloud.prowler.com{Style.RESET_ALL}
"""
)
+21
View File
@@ -283,6 +283,26 @@ class CSA_CCM_Requirement_Attribute(BaseModel):
ScopeApplicability: list[dict]
class STIG_Requirement_Attribute_Severity(str, Enum):
"""DISA STIG Requirement Attribute Severity (maps to CAT I/II/III)"""
high = "high"
medium = "medium"
low = "low"
class STIG_Requirement_Attribute(BaseModel):
"""DISA STIG Requirement Attribute"""
Section: str
Severity: STIG_Requirement_Attribute_Severity
RuleID: str
StigID: str
CCI: Optional[list[str]] = None
CheckText: Optional[str] = None
FixText: Optional[str] = None
# Base Compliance Model
# TODO: move this to compliance folder
class Compliance_Requirement(BaseModel):
@@ -303,6 +323,7 @@ class Compliance_Requirement(BaseModel):
CCC_Requirement_Attribute,
C5Germany_Requirement_Attribute,
CSA_CCM_Requirement_Attribute,
STIG_Requirement_Attribute,
# Generic_Compliance_Requirement_Attribute must be the last one since it is the fallback for generic compliance framework
Generic_Compliance_Requirement_Attribute,
]
@@ -18,6 +18,9 @@ from prowler.lib.outputs.compliance.kisa_ismsp.kisa_ismsp import get_kisa_ismsp_
from prowler.lib.outputs.compliance.mitre_attack.mitre_attack import (
get_mitre_attack_table,
)
from prowler.lib.outputs.compliance.okta_idaas_stig.okta_idaas_stig import (
get_okta_idaas_stig_table,
)
from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore import (
get_prowler_threatscore_table,
)
@@ -252,6 +255,15 @@ def display_compliance_table(
output_directory,
compliance_overview,
)
elif compliance_framework.startswith("okta_idaas_stig"):
get_okta_idaas_stig_table(
findings,
bulk_checks_metadata,
compliance_framework,
output_filename,
output_directory,
compliance_overview,
)
else:
# Try provider-specific table first, fall back to generic
from prowler.providers.common.provider import Provider
@@ -0,0 +1,32 @@
from typing import Optional
from pydantic.v1 import BaseModel
class OktaIDaaSSTIGModel(BaseModel):
"""
OktaIDaaSSTIGModel generates a finding's output in DISA Okta IDaaS STIG Compliance format.
"""
Provider: str
Description: str
OrganizationDomain: str
AssessmentDate: str
Requirements_Id: str
Requirements_Name: str
Requirements_Description: str
Requirements_Attributes_Section: str
Requirements_Attributes_Severity: str
Requirements_Attributes_RuleID: str
Requirements_Attributes_StigID: str
Requirements_Attributes_CCI: Optional[list[str]] = None
Requirements_Attributes_CheckText: Optional[str] = None
Requirements_Attributes_FixText: Optional[str] = None
Status: str
StatusExtended: str
ResourceId: str
ResourceName: str
CheckId: str
Muted: bool
Framework: str
Name: str
@@ -0,0 +1,98 @@
from colorama import Fore, Style
from tabulate import tabulate
from prowler.config.config import orange_color
def get_okta_idaas_stig_table(
findings: list,
bulk_checks_metadata: dict,
compliance_framework: str,
output_filename: str,
output_directory: str,
compliance_overview: bool,
):
section_table = {
"Provider": [],
"Section": [],
"Status": [],
"Muted": [],
}
pass_count = []
fail_count = []
muted_count = []
sections = {}
for index, finding in enumerate(findings):
check = bulk_checks_metadata[finding.check_metadata.CheckID]
check_compliances = check.Compliance
for compliance in check_compliances:
if compliance.Framework == "Okta-IDaaS-STIG":
for requirement in compliance.Requirements:
for attribute in requirement.Attributes:
section = attribute.Section
if section not in sections:
sections[section] = {"FAIL": 0, "PASS": 0, "Muted": 0}
if finding.muted:
if index not in muted_count:
muted_count.append(index)
sections[section]["Muted"] += 1
else:
if finding.status == "FAIL" and index not in fail_count:
fail_count.append(index)
sections[section]["FAIL"] += 1
elif finding.status == "PASS" and index not in pass_count:
pass_count.append(index)
sections[section]["PASS"] += 1
sections = dict(sorted(sections.items()))
for section in sections:
section_table["Provider"].append(compliance.Provider)
section_table["Section"].append(section)
if sections[section]["FAIL"] > 0:
section_table["Status"].append(
f"{Fore.RED}FAIL({sections[section]['FAIL']}){Style.RESET_ALL}"
)
else:
if sections[section]["PASS"] > 0:
section_table["Status"].append(
f"{Fore.GREEN}PASS({sections[section]['PASS']}){Style.RESET_ALL}"
)
else:
section_table["Status"].append(f"{Fore.GREEN}PASS{Style.RESET_ALL}")
section_table["Muted"].append(
f"{orange_color}{sections[section]['Muted']}{Style.RESET_ALL}"
)
if (
len(fail_count) + len(pass_count) + len(muted_count) > 1
): # If there are no resources, don't print the compliance table
print(
f"\nCompliance Status of {Fore.YELLOW}{compliance_framework.upper()}{Style.RESET_ALL} Framework:"
)
total_findings_count = len(fail_count) + len(pass_count) + len(muted_count)
overview_table = [
[
f"{Fore.RED}{round(len(fail_count) / total_findings_count * 100, 2)}% ({len(fail_count)}) FAIL{Style.RESET_ALL}",
f"{Fore.GREEN}{round(len(pass_count) / total_findings_count * 100, 2)}% ({len(pass_count)}) PASS{Style.RESET_ALL}",
f"{orange_color}{round(len(muted_count) / total_findings_count * 100, 2)}% ({len(muted_count)}) MUTED{Style.RESET_ALL}",
]
]
print(tabulate(overview_table, tablefmt="rounded_grid"))
if not compliance_overview:
if len(fail_count) > 0 and len(section_table["Section"]) > 0:
print(
f"\nFramework {Fore.YELLOW}{compliance_framework.upper()}{Style.RESET_ALL} Results:"
)
print(
tabulate(
section_table,
tablefmt="rounded_grid",
headers="keys",
)
)
print(f"\nDetailed results of {compliance_framework.upper()} are in:")
print(
f" - CSV: {output_directory}/compliance/{output_filename}_{compliance_framework}.csv\n"
)
@@ -0,0 +1,95 @@
from prowler.config.config import timestamp
from prowler.lib.check.compliance_models import Compliance
from prowler.lib.outputs.compliance.compliance_output import ComplianceOutput
from prowler.lib.outputs.compliance.okta_idaas_stig.models import OktaIDaaSSTIGModel
from prowler.lib.outputs.finding import Finding
class OktaIDaaSSTIG(ComplianceOutput):
"""
This class represents the Okta IDaaS STIG compliance output.
Attributes:
- _data (list): A list to store transformed data from findings.
- _file_descriptor (TextIOWrapper): A file descriptor to write data to a file.
Methods:
- transform: Transforms findings into Okta IDaaS STIG compliance format.
"""
def transform(
self,
findings: list[Finding],
compliance: Compliance,
_compliance_name: str,
) -> None:
"""
Transforms a list of findings into Okta IDaaS STIG compliance format.
Parameters:
- findings (list): A list of findings.
- compliance (Compliance): A compliance model.
- _compliance_name (str): The name of the compliance model (unused).
Returns:
- None
"""
for finding in findings:
for requirement in compliance.Requirements:
# Source of truth: framework JSON, not finding.compliance snapshot (avoids CSV/UI count drift).
if finding.check_id in requirement.Checks:
for attribute in requirement.Attributes:
compliance_row = OktaIDaaSSTIGModel(
Provider=finding.provider,
Description=compliance.Description,
OrganizationDomain=finding.account_name,
AssessmentDate=str(timestamp),
Requirements_Id=requirement.Id,
Requirements_Name=requirement.Name,
Requirements_Description=requirement.Description,
Requirements_Attributes_Section=attribute.Section,
Requirements_Attributes_Severity=attribute.Severity.value,
Requirements_Attributes_RuleID=attribute.RuleID,
Requirements_Attributes_StigID=attribute.StigID,
Requirements_Attributes_CCI=attribute.CCI,
Requirements_Attributes_CheckText=attribute.CheckText,
Requirements_Attributes_FixText=attribute.FixText,
Status=finding.status,
StatusExtended=finding.status_extended,
ResourceId=finding.resource_uid,
ResourceName=finding.resource_name,
CheckId=finding.check_id,
Muted=finding.muted,
Framework=compliance.Framework,
Name=compliance.Name,
)
self._data.append(compliance_row)
# Add manual requirements to the compliance output
for requirement in compliance.Requirements:
if not requirement.Checks:
for attribute in requirement.Attributes:
compliance_row = OktaIDaaSSTIGModel(
Provider=compliance.Provider.lower(),
Description=compliance.Description,
OrganizationDomain="",
AssessmentDate=str(timestamp),
Requirements_Id=requirement.Id,
Requirements_Name=requirement.Name,
Requirements_Description=requirement.Description,
Requirements_Attributes_Section=attribute.Section,
Requirements_Attributes_Severity=attribute.Severity.value,
Requirements_Attributes_RuleID=attribute.RuleID,
Requirements_Attributes_StigID=attribute.StigID,
Requirements_Attributes_CCI=attribute.CCI,
Requirements_Attributes_CheckText=attribute.CheckText,
Requirements_Attributes_FixText=attribute.FixText,
Status="MANUAL",
StatusExtended="Manual check",
ResourceId="manual_check",
ResourceName="Manual check",
CheckId="manual",
Muted=False,
Framework=compliance.Framework,
Name=compliance.Name,
)
self._data.append(compliance_row)
@@ -2582,6 +2582,7 @@
"aws": [
"af-south-1",
"ap-east-1",
"ap-east-2",
"ap-northeast-1",
"ap-northeast-2",
"ap-northeast-3",
@@ -2591,6 +2592,9 @@
"ap-southeast-2",
"ap-southeast-3",
"ap-southeast-4",
"ap-southeast-5",
"ap-southeast-6",
"ap-southeast-7",
"ca-central-1",
"ca-west-1",
"eu-central-1",
@@ -2604,6 +2608,7 @@
"il-central-1",
"me-central-1",
"me-south-1",
"mx-central-1",
"sa-east-1",
"us-east-1",
"us-east-2",
@@ -7344,6 +7349,7 @@
"lightsail": {
"regions": {
"aws": [
"ap-east-1",
"ap-northeast-1",
"ap-northeast-2",
"ap-south-1",
@@ -7354,9 +7360,11 @@
"ca-central-1",
"eu-central-1",
"eu-north-1",
"eu-south-2",
"eu-west-1",
"eu-west-2",
"eu-west-3",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-2"
@@ -8269,7 +8277,9 @@
"cn-north-1",
"cn-northwest-1"
],
"aws-eusc": [],
"aws-eusc": [
"eusc-de-east-1"
],
"aws-us-gov": [
"us-gov-east-1",
"us-gov-west-1"
@@ -9220,6 +9230,7 @@
"eu-west-1",
"eu-west-2",
"eu-west-3",
"sa-east-1",
"us-east-1",
"us-east-2",
"us-west-2"
@@ -9986,6 +9997,8 @@
"ap-south-1",
"ap-southeast-1",
"ap-southeast-2",
"ap-southeast-5",
"ap-southeast-7",
"ca-central-1",
"eu-central-1",
"eu-south-2",
@@ -0,0 +1,41 @@
{
"Provider": "aws",
"CheckID": "bedrock_agent_role_least_privilege",
"CheckTitle": "Amazon Bedrock agent execution role follows least privilege",
"CheckType": [
"Software and Configuration Checks/AWS Security Best Practices/Runtime Behavior Analysis",
"TTPs/Privilege Escalation"
],
"ServiceName": "bedrock",
"SubServiceName": "",
"ResourceIdTemplate": "",
"Severity": "high",
"ResourceType": "Other",
"ResourceGroup": "ai_ml",
"Description": "**Bedrock Agent** execution roles (`agentResourceRoleArn`) should grant only the minimum permissions the agent needs. The evaluation FAILs when the role has an AWS-managed `*FullAccess` policy attached, has an inline statement allowing broad actions on `Resource: \"*\"`, or has no permissions boundary configured.",
"Risk": "An overly permissive **Bedrock Agent** execution role turns a successful **prompt injection** into AWS privilege escalation. A model coerced into calling tools can invoke any API the role allows — reading secrets, modifying IAM, exfiltrating data from S3, or pivoting laterally. **Least privilege** plus a **permissions boundary** keeps the blast radius bounded even when guardrails fail.",
"RelatedUrl": "",
"AdditionalURLs": [
"https://docs.aws.amazon.com/bedrock/latest/userguide/agents-permissions.html",
"https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html",
"https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege"
],
"Remediation": {
"Code": {
"CLI": "aws iam put-role-permissions-boundary --role-name <execution_role_name> --permissions-boundary <boundary_policy_arn>",
"NativeIaC": "",
"Other": "1. Identify the Bedrock Agent's execution role (agentResourceRoleArn) in the IAM console\n2. Detach any AWS-managed *FullAccess policies (e.g. AmazonBedrockFullAccess, AdministratorAccess)\n3. Replace inline policies that use Resource: \"*\" with statements scoped to specific resource ARNs and minimal action sets\n4. Attach a permissions boundary that caps what the role can ever do, even if a future policy is added\n5. Re-run Prowler to confirm the check passes",
"Terraform": "```hcl\nresource \"aws_iam_role\" \"bedrock_agent\" {\n name = \"<execution_role_name>\"\n assume_role_policy = data.aws_iam_policy_document.trust.json\n permissions_boundary = aws_iam_policy.bedrock_agent_boundary.arn # CRITICAL: caps maximum privileges\n}\n\nresource \"aws_iam_role_policy\" \"bedrock_agent_inline\" {\n role = aws_iam_role.bedrock_agent.name\n policy = jsonencode({\n Version = \"2012-10-17\",\n Statement = [{\n Effect = \"Allow\",\n Action = [\"s3:GetObject\"], # CRITICAL: narrow action\n Resource = [\"arn:aws:s3:::my-rag-bucket/*\"] # CRITICAL: narrow resource\n }]\n })\n}\n```"
},
"Recommendation": {
"Text": "Apply **least privilege** to every Bedrock Agent execution role: scope `Action` and `Resource` to exactly what the agent needs, avoid AWS-managed `*FullAccess` policies, and always attach a **permissions boundary** so that future policy edits cannot exceed an approved ceiling. Treat agent roles as high-risk because prompt injection can weaponize any granted permission.",
"Url": "https://hub.prowler.com/check/bedrock_agent_role_least_privilege"
}
},
"Categories": [
"gen-ai"
],
"DependsOn": [],
"RelatedTo": [],
"Notes": ""
}
@@ -0,0 +1,101 @@
from prowler.lib.check.models import Check, Check_Report_AWS
from prowler.providers.aws.services.bedrock.bedrock_agent_client import (
bedrock_agent_client,
)
from prowler.providers.aws.services.iam.iam_client import iam_client
from prowler.providers.aws.services.iam.lib.policy import check_admin_access
from prowler.providers.aws.services.iam.lib.privilege_escalation import (
check_privilege_escalation,
)
class bedrock_agent_role_least_privilege(Check):
"""Ensure Bedrock Agent execution roles follow least privilege.
A Bedrock Agent's execution role is evaluated against three criteria:
- No AWS-managed ``*FullAccess`` policy attached.
- No attached or inline policy granting administrative access or known
privilege escalation combinations.
- A permissions boundary is configured on the role.
"""
def execute(self) -> list[Check_Report_AWS]:
"""Run the least-privilege evaluation across all Bedrock Agents.
Returns:
A list of ``Check_Report_AWS`` with one entry per agent. The
status is ``FAIL`` when any of the criteria above is violated,
or when the execution role cannot be resolved in IAM.
"""
findings = []
roles_by_arn = {role.arn: role for role in (iam_client.roles or [])}
for agent in bedrock_agent_client.agents.values():
report = Check_Report_AWS(metadata=self.metadata(), resource=agent)
report.status = "PASS"
report.status_extended = (
f"Bedrock Agent {agent.name} execution role follows least privilege."
)
role = roles_by_arn.get(agent.role_arn) if agent.role_arn else None
if role is None:
report.status = "FAIL"
report.status_extended = (
f"Bedrock Agent {agent.name} execution role could not be "
f"resolved in IAM and cannot be evaluated for least privilege."
)
findings.append(report)
continue
violations = []
for policy in role.attached_policies:
policy_arn = policy.get("PolicyArn", "")
policy_name = policy.get("PolicyName") or policy_arn
if policy_arn.startswith(
"arn:aws:iam::aws:policy/"
) and policy_arn.endswith("FullAccess"):
violations.append(
f"managed policy {policy_name} grants full access"
)
continue
policy_obj = iam_client.policies.get(policy_arn)
if policy_obj is None or not policy_obj.document:
continue
document = policy_obj.document
if check_admin_access(document):
violations.append(
f"managed policy {policy_name} grants administrative access"
)
elif check_privilege_escalation(document):
violations.append(
f"managed policy {policy_name} allows privilege escalation"
)
for inline_name in role.inline_policies:
policy_obj = iam_client.policies.get(f"{role.arn}:policy/{inline_name}")
if policy_obj is None or not policy_obj.document:
continue
document = policy_obj.document
if check_admin_access(document):
violations.append(
f"inline policy {inline_name} grants administrative access"
)
elif check_privilege_escalation(document):
violations.append(
f"inline policy {inline_name} allows privilege escalation"
)
if not role.permissions_boundary:
violations.append("no permissions boundary configured")
if violations:
report.status = "FAIL"
report.status_extended = (
f"Bedrock Agent {agent.name} execution role violates least "
f"privilege: {'; '.join(violations)}."
)
findings.append(report)
return findings
@@ -1,7 +1,7 @@
{
"Provider": "aws",
"CheckID": "bedrock_api_key_no_long_term_credentials",
"CheckTitle": "Amazon Bedrock API key is expired",
"CheckTitle": "Amazon Bedrock long-term API key has expired",
"CheckType": [
"Software and Configuration Checks/AWS Security Best Practices",
"Software and Configuration Checks/Industry and Regulatory Standards/AWS Foundational Security Best Practices",
@@ -14,23 +14,24 @@
"Severity": "high",
"ResourceType": "AwsIamUser",
"ResourceGroup": "IAM",
"Description": "**Bedrock API keys** are evaluated for **lifetime** and **expiration**.\n\nThe finding identifies keys that are long-lived, set to expire far in the future, or configured to `never expire`, and distinguishes them from keys that have already expired.",
"Risk": "Long-lived or non-expiring keys enable persistent access if compromised.\n- Confidentiality: unauthorized inference and exposure of prompts/outputs\n- Availability/Cost: uncontrolled usage and spend spikes\n- Integrity: actions can continue without timely revocation or rotation",
"Description": "AWS recommends Amazon Bedrock **long-term API keys** only for **exploration**; production workloads should use **short-term API keys** (session-scoped, valid up to **12 hours**). This check fails for any active long-term Bedrock API key, escalating to `critical` severity when configured to **never expire**. Already-expired keys pass — they can no longer authenticate.",
"Risk": "Long-term Bedrock API keys persist beyond a session until their stored expiration, and keys set to **never expire** grant indefinite access until manually revoked, enabling unauthorized inference, uncontrolled usage and spend, and activity that continues past timely revocation.",
"RelatedUrl": "",
"AdditionalURLs": [
"https://docs.aws.amazon.com/ja_jp/bedrock/latest/userguide/getting-started-api-keys.html",
"https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#rotate-credentials",
"https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html"
"https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys.html",
"https://docs.aws.amazon.com/bedrock/latest/userguide/api-keys-generate.html",
"https://docs.aws.amazon.com/IAM/latest/UserGuide/security-creds-programmatic-access.html#security-creds-alternatives-to-long-term-access-keys",
"https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#rotate-credentials"
],
"Remediation": {
"Code": {
"CLI": "aws iam delete-service-specific-credential --user-name <username> --service-specific-credential-id <credential-id>",
"NativeIaC": "",
"Other": "1. Sign in to the AWS Management Console and open IAM\n2. Go to Users > select <example_resource_name> > Security credentials\n3. In \"API keys for Amazon Bedrock\", find the non-expired key and click Delete\n4. Confirm deletion to remove the key (removes the long-term credential so the check passes)",
"Other": "1. Sign in to the AWS Management Console and open IAM\n2. Go to Users > select the IAM user backing the Bedrock API key > Security credentials\n3. In \"API keys for Amazon Bedrock\", select the active long-term key and click Delete\n4. For workloads that still need Bedrock access, generate a short-term API key from the Bedrock console (Short-term API keys tab), or call the Bedrock API with short-term credentials issued by AWS STS",
"Terraform": ""
},
"Recommendation": {
"Text": "Prefer **short-term credentials** and **IAM roles**; avoid `never expire`.\n\nEnforce **least privilege**, strict **rotation**, and automatic **expiration** for any long-term key. Store secrets securely, monitor with audit logs, and revoke unused or stale keys quickly.",
"Text": "Use short-term Amazon Bedrock API keys for any non-exploratory workload — they are bound to the IAM principal's session, valid for at most 12 hours, scoped to a single Region, and can be auto-refreshed by the SDK. For existing long-term keys, delete the underlying IAM service-specific credential. If a long-term key must be retained for an exploration scenario, set an explicit short expiration and never select `never expire`.",
"Url": "https://hub.prowler.com/check/bedrock_api_key_no_long_term_credentials"
}
},
@@ -40,5 +41,5 @@
],
"DependsOn": [],
"RelatedTo": [],
"Notes": "This check verifies that Amazon Bedrock API keys have expiration dates set. API keys without expiration dates are considered long-term credentials and pose a security risk. The check follows security best practices for credential management and the principle of least privilege."
"Notes": "AWS recommends against using long-term Amazon Bedrock API keys outside of exploration; production workloads should use short-term API keys (session-scoped, valid up to 12 hours). The IAM `ListServiceSpecificCredentials` API only enumerates long-term keys — short-term keys are session-scoped credentials that never appear here. The check therefore passes only when an existing long-term key has already expired and can no longer authenticate; any active long-term key fails, with critical severity when it is configured to never expire."
}
@@ -1,49 +1,62 @@
from datetime import datetime, timezone
from prowler.lib.check.models import Check, Check_Report_AWS
from prowler.lib.check.models import Check, Check_Report_AWS, Severity
from prowler.providers.aws.services.iam.iam_client import iam_client
# Days threshold above which a Bedrock long-term API key is considered effectively non-expiring.
NEVER_EXPIRES_THRESHOLD_DAYS = 10000
class bedrock_api_key_no_long_term_credentials(Check):
"""
Bedrock API keys should be short-lived to reduce the risk of unauthorized access.
This check verifies if there are any long-term Bedrock API keys.
If there are, it checks if they are expired or will be expired.
If they are expired, it will be marked as PASS.
If they are not expired, it will be marked as FAIL and the severity will be critical if the key will never expire.
"""Amazon Bedrock long-term API keys should not be used outside of exploration.
AWS recommends short-term Bedrock API keys (session-scoped, valid up to 12 hours)
for any non-exploratory workload. ``ListServiceSpecificCredentials`` only enumerates
long-term keys, so every key inspected here is by definition a long-term credential.
PASS when the long-term key has already expired (it can no longer authenticate).
FAIL (critical) when the key is configured to never expire.
FAIL (high) for any other active long-term key.
"""
def execute(self):
"""
Execute the Bedrock API key no long-term credentials check.
Iterate over all the Bedrock API keys and check if they are expired or will be expired.
Returns:
List[Check_Report_AWS]: A list of report objects with the results of the check.
"""
findings = []
for api_key in iam_client.service_specific_credentials:
if api_key.service_name != "bedrock.amazonaws.com":
continue
if api_key.expiration_date:
report = Check_Report_AWS(metadata=self.metadata(), resource=api_key)
# Check if the expiration date is in the future
if api_key.expiration_date > datetime.now(timezone.utc):
report.status = "FAIL"
# Get the days until the expiration date
days_until_expiration = (
api_key.expiration_date - datetime.now(timezone.utc)
).days
if days_until_expiration > 10000:
self.Severity = "critical"
report.status_extended = f"Long-term Bedrock API key {api_key.id} in user {api_key.user.name} exists and never expires."
else:
report.status_extended = f"Long-term Bedrock API key {api_key.id} in user {api_key.user.name} exists and will expire in {days_until_expiration} days."
else:
report.status = "PASS"
report.status_extended = f"Long-term Bedrock API key {api_key.id} in user {api_key.user.name} exists but has expired."
findings.append(report)
if not api_key.expiration_date:
continue
report = Check_Report_AWS(metadata=self.metadata(), resource=api_key)
now = datetime.now(timezone.utc)
if api_key.expiration_date <= now:
report.status = "PASS"
report.status_extended = (
f"Bedrock long-term API key {api_key.id} in user "
f"{api_key.user.name} has already expired and can no longer "
f"authenticate."
)
elif (api_key.expiration_date - now).days > NEVER_EXPIRES_THRESHOLD_DAYS:
report.status = "FAIL"
report.check_metadata.Severity = Severity.critical
report.status_extended = (
f"Bedrock long-term API key {api_key.id} in user "
f"{api_key.user.name} is configured to never expire. Use "
f"short-term Bedrock API keys (session-scoped, valid up to "
f"12 hours) for non-exploratory workloads instead."
)
else:
days_until_expiration = (api_key.expiration_date - now).days
report.status = "FAIL"
report.status_extended = (
f"Bedrock long-term API key {api_key.id} in user "
f"{api_key.user.name} is active and will expire in "
f"{days_until_expiration} days. Use short-term Bedrock API "
f"keys (session-scoped, valid up to 12 hours) for "
f"non-exploratory workloads instead."
)
findings.append(report)
return findings
@@ -146,6 +146,7 @@ class BedrockAgent(AWSService):
self.prompts = {}
self.prompt_scanned_regions: set = set()
self.__threading_call__(self._list_agents)
self.__threading_call__(self._get_agent, self.agents.values())
self.__threading_call__(self._list_prompts)
self.__threading_call__(self._get_prompt, self.prompts.values())
self.__threading_call__(self._list_tags_for_resource, self.agents.values())
@@ -174,6 +175,22 @@ class BedrockAgent(AWSService):
f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
def _get_agent(self, agent):
"""Fetch full agent details to capture the execution role ARN.
list_agents only returns summaries (no agentResourceRoleArn), so we
need a per-agent GetAgent call. Stored on the Agent model for use by
checks like bedrock_agent_role_least_privilege.
"""
logger.info("Bedrock Agent - Getting Agent...")
try:
agent_info = self.regional_clients[agent.region].get_agent(agentId=agent.id)
agent.role_arn = agent_info.get("agent", {}).get("agentResourceRoleArn")
except Exception as error:
logger.error(
f"{agent.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
def _list_prompts(self, regional_client):
"""List all prompts in a region."""
logger.info("Bedrock Agent - Listing Prompts...")
@@ -236,6 +253,7 @@ class Agent(BaseModel):
name: str
arn: str
guardrail_id: Optional[str] = None
role_arn: Optional[str] = None
region: str
tags: Optional[list] = []
@@ -17,9 +17,8 @@
"RelatedUrl": "",
"AdditionalURLs": [
"https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudwatch-alarms-for-cloudtrail.html",
"https://www.clouddefense.ai/compliance-rules/cis-v130/monitoring/cis-v130-4-11",
"https://support.icompaas.com/support/solutions/articles/62000084031-ensure-a-log-metric-filter-and-alarm-exist-for-changes-to-network-access-control-lists-nacl-",
"https://trendmicro.com/trendaivisiononecloudriskmanagement/knowledge-base/aws/CloudWatchLogs/network-acl-changes-alarm.html",
"https://www.trendmicro.com/trendaivisiononecloudriskmanagement/knowledge-base/aws/CloudWatchLogs/network-acl-changes-alarm.html",
"https://support.icompaas.com/support/solutions/articles/62000233134-4-11-ensure-network-access-control-list-nacl-changes-are-monitored-manual-"
],
"Remediation": {
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,16 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_changes_to_network_acls_alarm_configured(Check):
def execute(self):
pattern = r"\$\.eventName\s*=\s*.?CreateNetworkAcl.+\$\.eventName\s*=\s*.?CreateNetworkAclEntry.+\$\.eventName\s*=\s*.?DeleteNetworkAcl.+\$\.eventName\s*=\s*.?DeleteNetworkAclEntry.+\$\.eventName\s*=\s*.?ReplaceNetworkAclEntry.+\$\.eventName\s*=\s*.?ReplaceNetworkAclAssociation.?"
pattern = build_metric_filter_pattern(
event_names=[
"CreateNetworkAcl",
"CreateNetworkAclEntry",
"DeleteNetworkAcl",
"DeleteNetworkAclEntry",
"ReplaceNetworkAclEntry",
"ReplaceNetworkAclAssociation",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,16 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_changes_to_network_gateways_alarm_configured(Check):
def execute(self):
pattern = r"\$\.eventName\s*=\s*.?CreateCustomerGateway.+\$\.eventName\s*=\s*.?DeleteCustomerGateway.+\$\.eventName\s*=\s*.?AttachInternetGateway.+\$\.eventName\s*=\s*.?CreateInternetGateway.+\$\.eventName\s*=\s*.?DeleteInternetGateway.+\$\.eventName\s*=\s*.?DetachInternetGateway.?"
pattern = build_metric_filter_pattern(
event_names=[
"CreateCustomerGateway",
"DeleteCustomerGateway",
"AttachInternetGateway",
"CreateInternetGateway",
"DeleteInternetGateway",
"DetachInternetGateway",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -37,5 +37,5 @@
],
"DependsOn": [],
"RelatedTo": [],
"Notes": ""
"Notes": "Logging and Monitoring"
}
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,18 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_changes_to_network_route_tables_alarm_configured(Check):
def execute(self):
pattern = r"\$\.eventSource\s*=\s*.?ec2.amazonaws.com.+\$\.eventName\s*=\s*.?CreateRoute.+\$\.eventName\s*=\s*.?CreateRouteTable.+\$\.eventName\s*=\s*.?ReplaceRoute.+\$\.eventName\s*=\s*.?ReplaceRouteTableAssociation.+\$\.eventName\s*=\s*.?DeleteRouteTable.+\$\.eventName\s*=\s*.?DeleteRoute.+\$\.eventName\s*=\s*.?DisassociateRouteTable.?"
pattern = build_metric_filter_pattern(
event_source="ec2.amazonaws.com",
event_names=[
"CreateRoute",
"CreateRouteTable",
"ReplaceRoute",
"ReplaceRouteTableAssociation",
"DeleteRouteTable",
"DeleteRoute",
"DisassociateRouteTable",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,21 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_changes_to_vpcs_alarm_configured(Check):
def execute(self):
pattern = r"\$\.eventName\s*=\s*.?CreateVpc.+\$\.eventName\s*=\s*.?DeleteVpc.+\$\.eventName\s*=\s*.?ModifyVpcAttribute.+\$\.eventName\s*=\s*.?AcceptVpcPeeringConnection.+\$\.eventName\s*=\s*.?CreateVpcPeeringConnection.+\$\.eventName\s*=\s*.?DeleteVpcPeeringConnection.+\$\.eventName\s*=\s*.?RejectVpcPeeringConnection.+\$\.eventName\s*=\s*.?AttachClassicLinkVpc.+\$\.eventName\s*=\s*.?DetachClassicLinkVpc.+\$\.eventName\s*=\s*.?DisableVpcClassicLink.+\$\.eventName\s*=\s*.?EnableVpcClassicLink.?"
pattern = build_metric_filter_pattern(
event_names=[
"CreateVpc",
"DeleteVpc",
"ModifyVpcAttribute",
"AcceptVpcPeeringConnection",
"CreateVpcPeeringConnection",
"DeleteVpcPeeringConnection",
"RejectVpcPeeringConnection",
"AttachClassicLinkVpc",
"DetachClassicLinkVpc",
"DisableVpcClassicLink",
"EnableVpcClassicLink",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -15,7 +16,15 @@ class cloudwatch_log_metric_filter_and_alarm_for_aws_config_configuration_change
Check
):
def execute(self):
pattern = r"\$\.eventSource\s*=\s*.?config.amazonaws.com.+\$\.eventName\s*=\s*.?StopConfigurationRecorder.+\$\.eventName\s*=\s*.?DeleteDeliveryChannel.+\$\.eventName\s*=\s*.?PutDeliveryChannel.+\$\.eventName\s*=\s*.?PutConfigurationRecorder.?"
pattern = build_metric_filter_pattern(
event_source="config.amazonaws.com",
event_names=[
"StopConfigurationRecorder",
"DeleteDeliveryChannel",
"PutDeliveryChannel",
"PutConfigurationRecorder",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -15,7 +16,15 @@ class cloudwatch_log_metric_filter_and_alarm_for_cloudtrail_configuration_change
Check
):
def execute(self):
pattern = r"\$\.eventName\s*=\s*.?CreateTrail.+\$\.eventName\s*=\s*.?UpdateTrail.+\$\.eventName\s*=\s*.?DeleteTrail.+\$\.eventName\s*=\s*.?StartLogging.+\$\.eventName\s*=\s*.?StopLogging.?"
pattern = build_metric_filter_pattern(
event_names=[
"CreateTrail",
"UpdateTrail",
"DeleteTrail",
"StartLogging",
"StopLogging",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,10 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_log_metric_filter_authentication_failures(Check):
def execute(self):
pattern = r"\$\.eventName\s*=\s*.?ConsoleLogin.+\$\.errorMessage\s*=\s*.?Failed authentication.?"
pattern = build_metric_filter_pattern(
event_names=["ConsoleLogin"],
extra_clauses=[("errorMessage", "=", "Failed authentication")],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,32 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_log_metric_filter_aws_organizations_changes(Check):
def execute(self):
pattern = r"\$\.eventSource\s*=\s*.?organizations\.amazonaws\.com.+\$\.eventName\s*=\s*.?AcceptHandshake.+\$\.eventName\s*=\s*.?AttachPolicy.+\$\.eventName\s*=\s*.?CancelHandshake.+\$\.eventName\s*=\s*.?CreateAccount.+\$\.eventName\s*=\s*.?CreateOrganization.+\$\.eventName\s*=\s*.?CreateOrganizationalUnit.+\$\.eventName\s*=\s*.?CreatePolicy.+\$\.eventName\s*=\s*.?DeclineHandshake.+\$\.eventName\s*=\s*.?DeleteOrganization.+\$\.eventName\s*=\s*.?DeleteOrganizationalUnit.+\$\.eventName\s*=\s*.?DeletePolicy.+\$\.eventName\s*=\s*.?EnableAllFeatures.+\$\.eventName\s*=\s*.?EnablePolicyType.+\$\.eventName\s*=\s*.?InviteAccountToOrganization.+\$\.eventName\s*=\s*.?LeaveOrganization.+\$\.eventName\s*=\s*.?DetachPolicy.+\$\.eventName\s*=\s*.?DisablePolicyType.+\$\.eventName\s*=\s*.?MoveAccount.+\$\.eventName\s*=\s*.?RemoveAccountFromOrganization.+\$\.eventName\s*=\s*.?UpdateOrganizationalUnit.+\$\.eventName\s*=\s*.?UpdatePolicy.?"
pattern = build_metric_filter_pattern(
event_source="organizations.amazonaws.com",
event_names=[
"AcceptHandshake",
"AttachPolicy",
"CancelHandshake",
"CreateAccount",
"CreateOrganization",
"CreateOrganizationalUnit",
"CreatePolicy",
"DeclineHandshake",
"DeleteOrganization",
"DeleteOrganizationalUnit",
"DeletePolicy",
"EnableAllFeatures",
"EnablePolicyType",
"InviteAccountToOrganization",
"LeaveOrganization",
"DetachPolicy",
"DisablePolicyType",
"MoveAccount",
"RemoveAccountFromOrganization",
"UpdateOrganizationalUnit",
"UpdatePolicy",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,10 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_log_metric_filter_disable_or_scheduled_deletion_of_kms_cmk(Check):
def execute(self):
pattern = r"\$\.eventSource\s*=\s*.?kms.amazonaws.com.+\$\.eventName\s*=\s*.?DisableKey.+\$\.eventName\s*=\s*.?ScheduleKeyDeletion.?"
pattern = build_metric_filter_pattern(
event_source="kms.amazonaws.com",
event_names=["DisableKey", "ScheduleKeyDeletion"],
)
findings = []
report = check_cloudwatch_log_metric_filter(
@@ -17,8 +17,7 @@
"RelatedUrl": "",
"AdditionalURLs": [
"https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudwatch-alarms-for-cloudtrail.html",
"https://support.icompaas.com/support/solutions/articles/62000086674-ensure-a-log-metric-filter-and-alarm-exist-for-s3-bucket-policy-changes",
"https://www.tenable.com/audits/items/CIS_Amazon_Web_Services_Foundations_v5.0.0_L1.audit:8101350d6907e07863ac6748689b3e12"
"https://support.icompaas.com/support/solutions/articles/62000086674-ensure-a-log-metric-filter-and-alarm-exist-for-s3-bucket-policy-changes"
],
"Remediation": {
"Code": {
@@ -6,6 +6,7 @@ from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
cloudwatch_client,
)
from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
build_metric_filter_pattern,
check_cloudwatch_log_metric_filter,
)
from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
@@ -13,7 +14,20 @@ from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
class cloudwatch_log_metric_filter_for_s3_bucket_policy_changes(Check):
def execute(self):
pattern = r"\$\.eventSource\s*=\s*.?s3.amazonaws.com.+\$\.eventName\s*=\s*.?PutBucketAcl.+\$\.eventName\s*=\s*.?PutBucketPolicy.+\$\.eventName\s*=\s*.?PutBucketCors.+\$\.eventName\s*=\s*.?PutBucketLifecycle.+\$\.eventName\s*=\s*.?PutBucketReplication.+\$\.eventName\s*=\s*.?DeleteBucketPolicy.+\$\.eventName\s*=\s*.?DeleteBucketCors.+\$\.eventName\s*=\s*.?DeleteBucketLifecycle.+\$\.eventName\s*=\s*.?DeleteBucketReplication.?"
pattern = build_metric_filter_pattern(
event_source="s3.amazonaws.com",
event_names=[
"PutBucketAcl",
"PutBucketPolicy",
"PutBucketCors",
"PutBucketLifecycle",
"PutBucketReplication",
"DeleteBucketPolicy",
"DeleteBucketCors",
"DeleteBucketLifecycle",
"DeleteBucketReplication",
],
)
findings = []
report = check_cloudwatch_log_metric_filter(

Some files were not shown because too many files have changed in this diff Show More