Compare commits

..
42 Commits
Author SHA1 Message Date
Prowler Botandprowler-bot 7345e051cf chore(changelog): v5.33.1 (#11947)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-10 14:49:59 +02:00
Prowler BotandPepe Fagoaga c2b0135e35 chore(security): allow internal endpoints for Lighthouse AI OpenAI compatible (#11942)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-10 13:34:12 +02:00
1c4d8e3e75 fix(api): harden Lighthouse provider base URLs (#11940)
Co-authored-by: Adrián Peña <adrianjpr@gmail.com>
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-07-10 11:55:02 +02:00
Prowler BotandJosema Camacho b3562a800f fix(api): make AWS Attack Paths query aggregation Neo4j compatible (#11939)
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-07-10 11:28:44 +02:00
Prowler BotandHugo Pereira Brito cea4244db8 fix(jira): surface dispatch failures (#11925)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-07-09 16:43:27 +01:00
Prowler BotandHugo Pereira Brito 52f2da90f6 fix(azure): warn on optional function app permission failures (#11926)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-07-09 16:34:32 +01:00
Prowler BotandPedro Martín d1d6825159 fix(ui): handle level 1 requirements for M365 CIS (#11924)
Co-authored-by: Pedro Martín <pedromarting3@gmail.com>
2026-07-09 17:07:09 +02:00
548dd0f35a fix(aws): check statement Effect instead of policy Statement in SCP a… (#11915)
Co-authored-by: Narahari Raghava <70995755+NarahariRaghava@users.noreply.github.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 11:28:15 +02:00
Prowler BotandAdrián Peña c11fdb9d6d fix(api): invalidate tokens after password updates (#11914)
Co-authored-by: Adrián Peña <adrianjpr@gmail.com>
2026-07-09 11:08:56 +02:00
Prowler BotandHugo Pereira Brito 9f3a0534a8 fix(dms): lazy load ec2 for public access check (#11902)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-07-08 16:55:52 +01:00
Prowler BotandHugo Pereira Brito 44b7afdb25 fix(aws): avoid full ec2 inventory in dlm check (#11900)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-07-08 16:16:32 +01:00
Prowler BotandHugo Pereira Brito fd19a9a048 fix(sdk): limit ECS task definitions by registration date (#11891)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-07-08 13:19:55 +01:00
1da1da54d1 fix(ui): clarify Unlimited Visibility in RBAC forms (#11890)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-08 13:01:02 +01:00
Adrián Peña c123dc3788 chore(changelog): remove backport workflow fragment (#11887) 2026-07-08 11:43:42 +02:00
Prowler BotandAdrián Peña 3517cb331a feat: add changelog fragments workflow (#11886)
Co-authored-by: Adrián Peña <adrianjpr@gmail.com>
2026-07-08 11:22:29 +02:00
Prowler Botandprowler-bot fa365eb106 chore(release): Bump versions to v5.33.1 (#11873)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-07 18:23:06 +02:00
Prowler Botandprowler-bot 4526b91d3b chore(api): Update prowler dependency to v5.33 for release 5.33.0 (#11871)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-07 16:41:18 +02:00
César Arroba 76a2d7bfe6 ci: skip codeql and e2e on changelog-only changes (#11867) 2026-07-07 14:27:41 +02:00
ac3f289de6 feat(ui): gate Sentry, GTM and PostHog behind runtime enable flags (#11682)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:43:34 +02:00
Rubén De la Torre Vico 838f82b255 docs(lighthouse): document Lighthouse AI on Prowler Cloud (#11758) 2026-07-07 13:27:37 +02:00
Josema Camacho 5e00c4bfcf chore(docs): update changelogs for 5.33.0 (#11866) 2026-07-07 13:07:40 +02:00
lydiavilchez f0ae56b8ea feat(docs): auto-generate provider cards in Prowler App tutorial (#11865) 2026-07-07 12:35:52 +02:00
Hugo Pereira BritoandDaniel Barranquero 4484c2f192 fix(azure): refine resource group scoped scan follow-ups (#11796)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 11:20:38 +02:00
Pedro Martín e6bbcb8043 chore(scan-config): improve wording and docs (#11859) 2026-07-07 11:14:19 +02:00
Hugo Pereira Brito ad04e69c35 fix(kubernetes): reject exec auth in cloud kubeconfigs (#11753) 2026-07-07 09:57:46 +01:00
Adrián Peña 6cae37174c fix(api): queue provider scans when one is active (#11848) 2026-07-07 10:28:41 +02:00
Pepe Fagoaga aa6de57430 chore(multi-tenant): delete last tenant from profile page (#11864) 2026-07-07 10:02:36 +02:00
Alejandro Bailo a48aa37f87 fix(ui): Lighthouse chat archive navigation, new chat button, and masked stored credentials (#11860) 2026-07-07 09:57:47 +02:00
Pedro Martín 3cc8f86780 feat(api): remove provider credentials for PDF report (#11845) 2026-07-07 09:45:34 +02:00
AbhinavandDaniel Barranquero 4cb02a0ead fix(azure): read Flexible Server log retention from logfiles.retention_days (fixes #11757) (#11761)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 08:19:44 +02:00
Alejandro Bailo 221c558cee feat(ui): connect Lighthouse v2 to Cloud backend (#11690) 2026-07-06 17:30:31 +02:00
Pedro MartínandCésar Arroba 81c3152ebb chore(changelog): prepare for 5.32.1 (#11856)
Co-authored-by: César Arroba <19954079+cesararroba@users.noreply.github.com>
2026-07-06 16:14:20 +02:00
Pedro Martín 855e9a043e chore(changelog): prepare for 5.32.1 (#11854) 2026-07-06 16:01:31 +02:00
Adrián Peñaandalejandrobailo 7b5d724bb7 fix: handle invitations in social and SAML auth (#11752)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-06 14:49:07 +02:00
Alejandro Bailo 441f2a3c48 fix(ui): enable triage editing in compliance findings table (#11829) 2026-07-06 11:03:37 +02:00
Hugo Pereira Brito efb86bb7ab chore: remove Dependabot config (#11834) 2026-07-03 12:48:17 +01:00
Hugo Pereira Brito 398a0a484f fix(api): restrict user profile updates to self (#11792) 2026-07-03 11:53:01 +01:00
Sanjay Santhanamandpedrooot 55924d8150 fix(compliance): skip MANUAL findings in section tally to avoid KeyError (#11823)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-07-03 12:19:32 +02:00
Josema Camacho 0cf6f2f83e fix(api): add attack paths scan DB defaults (#11826) 2026-07-03 11:30:55 +02:00
Pepe Fagoaga cf18093261 chore(onboarding): rename account -> provider (#11827) 2026-07-03 09:33:37 +02:00
Josema Camacho 1850e209e6 docs(attack-paths): update tutorial for the new UI (#11824) 2026-07-02 18:34:49 +02:00
Prowler Botandprowler-bot 2e37188c9f chore(release): Bump versions to v5.33.0 (#11820)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-02 16:56:03 +02:00
350 changed files with 18662 additions and 2490 deletions
+7 -6
View File
@@ -145,19 +145,20 @@ DJANGO_BROKER_VISIBILITY_TIMEOUT=86400
DJANGO_SENTRY_DSN=
DJANGO_THROTTLE_TOKEN_OBTAIN=50/minute
# Sentry for the web app (server + browser). Empty/unset UI_SENTRY_DSN ⇒
# Sentry disabled, zero egress. SENTRY_RELEASE (unprefixed) feeds the web app's
# server/edge SDKs.
# Sentry for the web app (server + browser). The UI_SENTRY_* values load only
# when UI_SENTRY_ENABLE="true"; without it they are ignored (default off, zero
# egress). The deprecated NEXT_PUBLIC_SENTRY_DSN still activates Sentry without
# the flag. SENTRY_RELEASE (unprefixed) feeds the web app's server/edge SDKs.
UI_SENTRY_DSN=
UI_SENTRY_ENVIRONMENT=local
SENTRY_RELEASE=local
# Reserved runtime public config (registered now; no UI consumer yet)
# POSTHOG_KEY=
# POSTHOG_HOST=
# UI_POSTHOG_KEY=
# UI_POSTHOG_HOST=
# REO_DEV_CLIENT_ID=
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.32.2
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.33.1
# Social login credentials
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
-140
View File
@@ -1,140 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# v5
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "pip"
# directory: "/api"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# - "component/api"
# Dependabot version updates disabled - migrated to Renovate - 2026/07/02
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "github_actions"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "npm"
# directory: "/ui"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "npm"
# - "component/ui"
# Dependabot version updates disabled - migrated to Renovate - 2026/07/02
# - package-ecosystem: "docker"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "docker"
# cooldown:
# default-days: 7
# - package-ecosystem: "pre-commit"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pre-commit"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/04/15
# v4.6
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "pip"
# - "v4"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "github_actions"
# - "v4"
# - package-ecosystem: "docker"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "docker"
# - "v4"
# Dependabot Updates are temporary disabled - 2025/03/19
# v3
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "pip"
# - "v3"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "github_actions"
# - "v3"
+9 -5
View File
@@ -18,8 +18,8 @@ Please add a detailed description of how to review this PR.
<summary><b>Community Checklist</b></summary>
- [ ] This feature/issue is listed in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the issue/feature in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](goto.prowler.com/slack)
- [ ] This feature/issue is listed in the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](https://goto.prowler.com/slack)
</details>
@@ -28,7 +28,7 @@ Please add a detailed description of how to review this PR.
- [ ] Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
- [ ] Review if backport is needed.
- [ ] Review if is needed to change the [Readme.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/prowler/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [prowler/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/prowler/changelog.d), if applicable.
#### SDK/CLI
- Are there new checks included in this PR? Yes / No
@@ -40,7 +40,7 @@ Please add a detailed description of how to review this PR.
- [ ] Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/ui/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [ui/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/ui/changelog.d), if applicable.
#### API
- [ ] All issue/task requirements work as expected on the API
@@ -50,7 +50,11 @@ Please add a detailed description of how to review this PR.
- [ ] Any other relevant evidence of the implementation (if applicable)
- [ ] Verify if API specs need to be regenerated.
- [ ] Check if version updates are required (e.g., specs, uv, etc.).
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/api/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [api/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/api/changelog.d), if applicable.
#### MCP Server
- [ ] All issue/task requirements work as expected on the MCP Server
- [ ] Ensure a changelog fragment is added under [mcp_server/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/mcp_server/changelog.d), if applicable.
### License
+154
View File
@@ -0,0 +1,154 @@
#!/usr/bin/env python3
"""Rename changelog fragments to their PR number before running towncrier.
For every <slug>.<type>.md in <component_dir>/changelog.d/, find the commit that
added it, resolve its PR via the GitHub API (falling back to the squash-commit
subject), and `git mv` it to <PR>.<type>.md so towncrier renders the PR link.
Unresolvable fragments become +<slug>.<type>.md orphans (rendered without link).
"""
import argparse
import json
import os
import re
import subprocess
import sys
import urllib.error
import urllib.request
FRAGMENT_RE = re.compile(
r"^(?P<slug>[A-Za-z0-9][A-Za-z0-9._-]*?)"
r"\.(?P<type>added|changed|deprecated|removed|fixed|security)"
r"(?:\.(?P<counter>[0-9]+))?\.md$"
)
SUBJECT_PR_RE = re.compile(r" \(#([0-9]+)\)$")
IGNORED_FILES = {".gitkeep", "README.md"}
API_TIMEOUT_SECONDS = 10
def git(*args: str) -> str:
result = subprocess.run(["git", *args], check=True, capture_output=True, text=True)
return result.stdout.strip()
def find_adding_commit(path: str) -> str | None:
"""Find the commit that added a file, following renames.
Falls back to a plain (no --follow) lookup: rename detection can lose the
add event for degenerate content (e.g. files identical to many others).
"""
sha = git("log", "--follow", "--diff-filter=A", "--format=%H", "-1", "--", path)
if not sha:
sha = git("log", "--diff-filter=A", "--format=%H", "-1", "--", path)
return sha or None
def pr_from_api(repo: str, sha: str) -> int | None:
"""Resolve the PR associated with a commit via the GitHub API.
Returns None on any network/API failure so the caller can fall back to
parsing the squash-commit subject.
"""
url = f"https://api.github.com/repos/{repo}/commits/{sha}/pulls"
headers = {
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "prowler-changelog-attribution",
}
token = os.environ.get("GITHUB_TOKEN")
if token:
headers["Authorization"] = f"Bearer {token}"
request = urllib.request.Request(url, headers=headers)
try:
with urllib.request.urlopen(request, timeout=API_TIMEOUT_SECONDS) as response:
pulls = json.load(response)
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError):
return None
if isinstance(pulls, list) and pulls:
return pulls[0].get("number")
return None
def pr_from_subject(sha: str) -> int | None:
subject = git("log", "-1", "--format=%s", sha)
match = SUBJECT_PR_RE.search(subject)
return int(match.group(1)) if match else None
def unique_destination(directory: str, base_name: str, fragment_type: str) -> str:
"""Return a non-colliding fragment path, appending a numeric counter if needed."""
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.md")
counter = 0
while os.path.exists(candidate):
counter += 1
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.{counter}.md")
return candidate
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("component_dir", help="Component directory, e.g. prowler")
parser.add_argument("--repo", default="prowler-cloud/prowler")
parser.add_argument(
"--no-api",
action="store_true",
help="Skip the GitHub API and resolve PRs from commit subjects only",
)
args = parser.parse_args()
fragments_dir = os.path.join(args.component_dir, "changelog.d")
if not os.path.isdir(fragments_dir):
print(f"::error::Fragments directory not found: {fragments_dir}")
return 1
malformed = []
to_process = []
for name in sorted(os.listdir(fragments_dir)):
if name in IGNORED_FILES or name.startswith("+"):
continue
match = FRAGMENT_RE.match(name)
if not match:
malformed.append(name)
continue
if match.group("slug").isdigit():
continue
to_process.append((name, match))
if malformed:
for name in malformed:
print(
f"::error::Malformed fragment filename in {fragments_dir}: {name} "
"(expected <slug>.<type>.md with type one of added|changed|"
"deprecated|removed|fixed|security)"
)
return 1
for name, match in to_process:
slug, fragment_type = match.group("slug"), match.group("type")
path = os.path.join(fragments_dir, name)
sha = find_adding_commit(path)
pr_number = None
if sha:
if not args.no_api:
pr_number = pr_from_api(args.repo, sha)
if pr_number is None:
pr_number = pr_from_subject(sha)
if pr_number is not None:
destination = unique_destination(
fragments_dir, str(pr_number), fragment_type
)
else:
destination = unique_destination(fragments_dir, f"+{slug}", fragment_type)
print(
f"::warning::Could not resolve a PR for {path}; renamed to "
f"{os.path.basename(destination)} (entry will render without a PR link)"
)
git("mv", path, destination)
print(f"{path} -> {destination}")
return 0
if __name__ == "__main__":
sys.exit(main())
+13
View File
@@ -0,0 +1,13 @@
{% set category_order = definitions.keys() %}
{% for section, _ in sections.items() %}
{% for category in category_order if category in sections[section] %}
### {{ definitions[category]['name'] }}
{% for text, values in sections[section][category].items() -%}
- {{ text }}{% if values %} {{ values|join(', ') }}{% endif %}{{ "\n" }}
{%- endfor %}
{% endfor %}
{% endfor %}
---
{{ "\n" }}
+1
View File
@@ -62,6 +62,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+2
View File
@@ -9,6 +9,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
pull_request:
branches:
- 'master'
@@ -17,6 +18,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
schedule:
- cron: '00 12 * * *'
@@ -108,6 +108,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Set up Docker Buildx
+1
View File
@@ -77,6 +77,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+1
View File
@@ -111,6 +111,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+478
View File
@@ -0,0 +1,478 @@
name: 'Tools: Compile Changelogs'
run-name: 'Compile changelogs for Prowler ${{ inputs.prowler_version }}'
on:
workflow_dispatch:
inputs:
prowler_version:
description: 'Prowler version being released (e.g., 5.31.0)'
required: true
type: string
target_branch:
description: 'Branch to compile on (master for minor releases, v5.X for patches)'
required: true
type: string
sdk_version:
description: 'SDK version override (empty = auto-derive from prowler/CHANGELOG.md + pending fragment types; "skip" = hold this component back)'
required: false
type: string
api_version:
description: 'API version override (empty = auto-derive; "skip" = hold back)'
required: false
type: string
ui_version:
description: 'UI version override (empty = auto-derive; "skip" = hold back)'
required: false
type: string
mcp_version:
description: 'MCP Server version override (empty = auto-derive; "skip" = hold back)'
required: false
type: string
concurrency:
group: ${{ github.workflow }}-${{ inputs.prowler_version }}
cancel-in-progress: false
env:
PROWLER_VERSION: ${{ inputs.prowler_version }}
TARGET_BRANCH: ${{ inputs.target_branch }}
SDK_VERSION: ${{ inputs.sdk_version }}
API_VERSION: ${{ inputs.api_version }}
UI_VERSION: ${{ inputs.ui_version }}
MCP_VERSION: ${{ inputs.mcp_version }}
permissions: {}
jobs:
compile-changelogs:
if: github.event_name == 'workflow_dispatch' && github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
pull-requests: write
steps:
- name: Harden the runner (Block outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.target_branch }}
fetch-depth: 0 # PR attribution resolves each fragment's adding commit from history
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Install towncrier
run: pip install --no-cache-dir towncrier==25.8.0
- name: Configure Git
run: |
git config --global user.name 'prowler-bot'
git config --global user.email '179230569+prowler-bot@users.noreply.github.com'
- name: Validate version inputs
run: |
if [[ ! "$PROWLER_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid prowler_version syntax: '$PROWLER_VERSION' (must be N.N.N)"
exit 1
fi
if [ "$TARGET_BRANCH" != "master" ] && [[ ! "$TARGET_BRANCH" =~ ^v[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid target_branch syntax: '$TARGET_BRANCH' (must be 'master' or vN.N, e.g. v5.31)"
exit 1
fi
IFS=. read -r prowler_major prowler_minor prowler_patch <<< "$PROWLER_VERSION"
prowler_major=$((10#$prowler_major))
prowler_minor=$((10#$prowler_minor))
prowler_patch=$((10#$prowler_patch))
if [ "$prowler_patch" -eq 0 ]; then
if [ "$TARGET_BRANCH" != "master" ]; then
echo "::error::target_branch must be 'master' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
else
expected_target_branch="v${prowler_major}.${prowler_minor}"
if [ "$TARGET_BRANCH" != "$expected_target_branch" ]; then
echo "::error::target_branch must be '${expected_target_branch}' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
fi
for pair in "sdk_version:$SDK_VERSION" "api_version:$API_VERSION" "ui_version:$UI_VERSION" "mcp_version:$MCP_VERSION"; do
input_name="${pair%%:*}"
input_value="${pair#*:}"
if [ -n "$input_value" ] && [ "$input_value" != "skip" ] && [[ ! "$input_value" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid $input_name syntax: '$input_value' (must be N.N.N, empty for auto-derivation, or 'skip')"
exit 1
fi
done
- name: Compile changelogs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
component_input() {
case "$1" in
prowler) echo "$SDK_VERSION" ;;
api) echo "$API_VERSION" ;;
ui) echo "$UI_VERSION" ;;
mcp_server) echo "$MCP_VERSION" ;;
esac
}
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
pending_fragments() {
find "$1/changelog.d" -maxdepth 1 -type f ! -name '.gitkeep' ! -name 'README.md' | sort
}
# The component's last released version is the first stamped heading
# of its CHANGELOG.md, the same source prepare-release.yml greps.
latest_released_version() {
grep -m1 -E '^## \[v?[0-9]+\.[0-9]+\.[0-9]+\]' "$1/CHANGELOG.md" | sed -E 's/^## \[v?([0-9]+\.[0-9]+\.[0-9]+)\].*/\1/'
}
has_removed_fragments() {
echo "$1" | grep -qE '\.removed(\.[0-9]+)?\.md$'
}
# Resolve every component's effective version before compiling
# anything, so a wrong input cannot leave the tree half-compiled.
# Empty input = auto-derive (latest released version + semver bump
# from the pending fragment types). 'skip' = hold the component back.
errors=0
compiling=""
for component in prowler api ui mcp_server; do
input=$(component_input "$component")
fragments=$(pending_fragments "$component")
if [ "$input" = "skip" ]; then
if [ -n "$fragments" ]; then
echo "::warning::${component}: held back by request; these pending fragments stay for a future release:"
echo "$fragments"
fi
continue
fi
if [ -n "$input" ] && [ -z "$fragments" ]; then
echo "::error::${component}: version input '$input' provided but ${component}/changelog.d/ has no pending fragments (wrong input?)"
errors=1
continue
fi
if [ -z "$fragments" ]; then
continue
fi
removed_fragments=false
if has_removed_fragments "$fragments"; then
removed_fragments=true
fi
current=$(latest_released_version "$component")
if [ -z "$current" ]; then
echo "::error::${component}: could not read the latest released version from ${component}/CHANGELOG.md; restore the released heading before compiling"
errors=1
continue
fi
if [ -n "$input" ]; then
effective="$input"
mode="explicit"
current_key=$(version_key "$current")
effective_key=$(version_key "$effective")
if [[ "$effective_key" < "$current_key" || "$effective_key" == "$current_key" ]]; then
echo "::error::${component}: explicit version '${effective}' must be greater than the latest released version (${current})"
errors=1
continue
fi
else
if [ "$removed_fragments" = "true" ]; then
echo "::error::${component}: pending 'removed' fragments imply a major bump (breaking change); provide its version input explicitly"
errors=1
continue
fi
IFS=. read -r major minor patch <<< "$current"
major=$((10#$major))
minor=$((10#$minor))
patch=$((10#$patch))
if echo "$fragments" | grep -qE '\.(added|changed|deprecated)(\.[0-9]+)?\.md$'; then
effective="${major}.$((minor + 1)).0"
else
effective="${major}.${minor}.$((patch + 1))"
fi
mode="auto"
echo "::notice::${component}: version auto-derived ${current} -> ${effective} from the pending fragment types"
fi
if [ "$removed_fragments" = "true" ]; then
IFS=. read -r current_major _ <<< "$current"
current_major=$((10#$current_major))
IFS=. read -r effective_major effective_minor effective_patch <<< "$effective"
effective_major=$((10#$effective_major))
effective_minor=$((10#$effective_minor))
effective_patch=$((10#$effective_patch))
if [ "$effective_major" -le "$current_major" ] || [ "$effective_minor" -ne 0 ] || [ "$effective_patch" -ne 0 ]; then
echo "::error::${component}: removed fragments require a major component release (${current} -> X.0.0 with X > ${current_major}); got ${effective}"
errors=1
continue
fi
fi
# Without the marker the build would insert the new block above the
# file header instead of below it.
if ! grep -q '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md is missing the '<!-- changelog: release notes start -->' marker; restore it after the intro line before compiling"
errors=1
continue
fi
# A hand-written UNRELEASED block means someone followed the old
# convention; its entries would be left out of the compiled block
# and out of the release notes extraction.
if grep -q '(Prowler UNRELEASED)' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md contains a hand-written '(Prowler UNRELEASED)' block; convert its entries to fragments in ${component}/changelog.d/ and delete the block before compiling"
errors=1
continue
fi
echo "${effective} ${mode}" > "${RUNNER_TEMP}/version-${component}.txt"
compiling="${compiling}${component} "
done
if [ "$errors" -ne 0 ]; then
exit 1
fi
if [ -z "$compiling" ]; then
echo "::error::Nothing to compile: no component has pending fragments to release"
exit 1
fi
body_file="${RUNNER_TEMP}/compile-changelogs-pr-body.md"
{
echo "### Description"
echo ""
echo "Compiles the pending changelog fragments into the per-component \`CHANGELOG.md\` files for Prowler v${PROWLER_VERSION}, replacing the manual stamping PR."
echo ""
echo "| Component | Version | Fragments consumed |"
echo "|---|---|---|"
} > "$body_file"
compiled_components=""
for component in prowler api ui mcp_server; do
if [ ! -f "${RUNNER_TEMP}/version-${component}.txt" ]; then
echo "Skipping ${component} (no pending fragments or held back)"
echo "| \`${component}\` | - | 0 |" >> "$body_file"
continue
fi
read -r version mode < "${RUNNER_TEMP}/version-${component}.txt"
version_label="$version"
if [ "$mode" = "auto" ]; then
version_label="${version} (auto)"
fi
count=$(pending_fragments "$component" | wc -l | tr -d ' ')
echo "Compiling ${component} ${version} (${count} fragments, ${mode} version)..."
# Captured before attribution renames them: these original paths are
# what the forward-sync deletes on master (backports copy fragments
# verbatim, so filenames match across branches).
pending_fragments "$component" > "${RUNNER_TEMP}/consumed-${component}.txt"
pre_lines=$(wc -l < "$component/CHANGELOG.md")
# Attribution must run before the build: towncrier renders the
# first dotted segment of each filename as the PR number.
python .github/scripts/changelog_attribution.py "$component"
towncrier build --config "$component/towncrier.toml" --version "$version" --name "Prowler v${PROWLER_VERSION}" --yes
# The build only inserts lines right after the marker, so the new
# stamped block is exactly the added lines following it. Captured
# for the forward-sync to master.
post_lines=$(wc -l < "$component/CHANGELOG.md")
delta=$((post_lines - pre_lines))
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md" | cut -d: -f1)
sed -n "$((marker_line + 1)),$((marker_line + delta))p" "$component/CHANGELOG.md" > "${RUNNER_TEMP}/block-${component}.md"
compiled_components="${compiled_components}${component} "
echo "| \`${component}\` | ${version_label} | ${count} |" >> "$body_file"
done
echo "COMPILED_COMPONENTS=${compiled_components}" >> "$GITHUB_ENV"
{
echo ""
echo "Review that no pending fragment was dropped (the diff must delete every consumed fragment) and that each new version block is correct, then squash-merge."
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$body_file"
echo "PR_BODY_FILE=${body_file}" >> "$GITHUB_ENV"
- name: Create compile PR
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
branch: compile-changelogs-${{ env.PROWLER_VERSION }}
base: ${{ env.TARGET_BRANCH }}
title: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
body-path: ${{ env.PR_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
# Patch compiles (target_branch = v5.X) leave master holding the consumed
# fragments and missing the new version block. This applies the equivalent
# change to master: insert the same stamped blocks under the marker and
# delete the consumed fragments, so the next minor compile cannot
# re-release entries that already shipped in the patch.
- name: Apply forward-sync to master
if: env.TARGET_BRANCH != 'master'
run: |
set -euo pipefail
git checkout -B master origin/master
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
release_from_heading() {
local heading="$1"
echo "$heading" | sed -E 's/^## \[[^]]+\] \(Prowler v?([0-9]+\.[0-9]+\.[0-9]+)\).*/\1/'
}
insert_changelog_block_ordered() {
local component="$1"
local block_file="$2"
local changelog="${component}/CHANGELOG.md"
local incoming_heading incoming_release incoming_key
local marker_line insertion_line duplicate_line
local line heading existing_release existing_key
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$changelog" | cut -d: -f1)
incoming_heading=$(grep -m1 -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$block_file" || true)
if [ -z "$incoming_heading" ]; then
echo "::error::${block_file} does not contain a stamped Prowler release heading"
exit 1
fi
incoming_release=$(release_from_heading "$incoming_heading")
incoming_key=$(version_key "$incoming_release")
insertion_line=""
duplicate_line=""
while IFS=: read -r line heading; do
existing_release=$(release_from_heading "$heading")
existing_key=$(version_key "$existing_release")
if [[ "$incoming_key" == "$existing_key" ]]; then
duplicate_line="$line"
break
fi
if [[ "$incoming_key" > "$existing_key" ]]; then
insertion_line="$line"
break
fi
done < <(grep -n -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$changelog" || true)
if [ -n "$duplicate_line" ]; then
echo "::error::${changelog} already contains a block for Prowler v${incoming_release} at line ${duplicate_line}; refusing to insert a duplicate"
exit 1
fi
if [ -z "$insertion_line" ]; then
insertion_line=$(($(wc -l < "$changelog") + 1))
fi
if [ "$insertion_line" -le "$marker_line" ]; then
insertion_line=$((marker_line + 1))
fi
{
head -n "$((insertion_line - 1))" "$changelog"
cat "$block_file"
tail -n +"$insertion_line" "$changelog"
} > "${RUNNER_TEMP}/changelog.tmp"
mv "${RUNNER_TEMP}/changelog.tmp" "$changelog"
echo "::notice::Inserted ${component} changelog block for Prowler v${incoming_release} at line ${insertion_line}"
}
sync_body="${RUNNER_TEMP}/forward-sync-pr-body.md"
{
echo "### Description"
echo ""
echo "Forward-syncs the v${PROWLER_VERSION} compiled changelogs from \`${TARGET_BRANCH}\` to \`master\`: inserts the same stamped version blocks under the insertion marker and deletes the consumed fragments, so the next minor compile cannot re-release entries that already shipped in this patch. Opened automatically by the same run that opened the compile PR; review and squash-merge after it."
echo ""
echo "| Component | Fragments deleted on master | Skipped (only on ${TARGET_BRANCH}) |"
echo "|---|---|---|"
} > "$sync_body"
for component in $COMPILED_COMPONENTS; do
block_file="${RUNNER_TEMP}/block-${component}.md"
consumed_file="${RUNNER_TEMP}/consumed-${component}.txt"
if ! grep -qm1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md on master is missing the insertion marker; cannot forward-sync"
exit 1
fi
deleted=0
skipped=0
while IFS= read -r fragment; do
if [ -z "$fragment" ]; then
continue
fi
if [ -f "$fragment" ]; then
git rm -q "$fragment"
deleted=$((deleted + 1))
else
echo "::notice::${fragment} does not exist on master (change landed only on ${TARGET_BRANCH}); skipping its deletion"
skipped=$((skipped + 1))
fi
done < "$consumed_file"
insert_changelog_block_ordered "$component" "$block_file"
echo "| \`${component}\` | ${deleted} | ${skipped} |" >> "$sync_body"
done
{
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$sync_body"
echo "SYNC_BODY_FILE=${sync_body}" >> "$GITHUB_ENV"
- name: Create forward-sync PR
if: env.TARGET_BRANCH != 'master'
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
branch: forward-sync-changelogs-${{ env.PROWLER_VERSION }}
base: master
title: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
body-path: ${{ env.SYNC_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
@@ -102,6 +102,7 @@ jobs:
files_ignore: |
mcp_server/README.md
mcp_server/CHANGELOG.md
mcp_server/changelog.d/**
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
+185 -28
View File
@@ -19,6 +19,60 @@ concurrency:
permissions: {}
jobs:
test-changelog-attribution:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 1
persist-credentials: false
- name: Fetch PR base ref for tj-actions/changed-files
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: git fetch --depth=1 origin "${BASE_REF}"
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
.github/scripts/changelog_attribution.py
.github/workflows/pr-check-changelog.yml
.github/workflows/compile-changelogs.yml
.github/towncrier/template.md.jinja
*/towncrier.toml
tests/github/**
- name: Set up Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Test changelog attribution
if: steps.changed-files.outputs.any_changed == 'true'
run: |
python3 -m pip install --user --disable-pip-version-check pytest==9.0.3 towncrier==25.8.0
python3 -m pytest tests/github
check-changelog:
if: contains(github.event.pull_request.labels.*.name, 'no-changelog') == false
runs-on: ubuntu-latest
@@ -62,53 +116,160 @@ jobs:
uv.lock
pyproject.toml
- name: Check for folder changes and changelog presence
- name: Check for folder changes and changelog fragment presence
id: check-folders
run: |
missing_changelogs=""
fragment_name_re='^[A-Za-z0-9][A-Za-z0-9._-]*\.(added|changed|deprecated|removed|fixed|security)(\.[0-9]+)?\.md$'
manual_pr_link_re='(\[\(#[0-9]+\)\]|\[#[0-9]+\]\(|\(#[0-9]+\)|github\.com/[^[:space:]/]+/[^[:space:]/]+/(pull|issues)/[0-9]+)'
folder_alt=$(echo "$MONITORED_FOLDERS" | tr ' ' '|')
missing_fragments=""
invalid_fragments=""
linked_fragments=""
handwritten_changelogs=""
all_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n')
added=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" | tr ' ' '\n')
added_or_renamed=$(printf '%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
added_modified_or_renamed=$(printf '%s\n%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
# Returns success if the folder has a valid fragment added, modified, or renamed.
has_changelog_update() {
local folder="$1"
if echo "$added_modified_or_renamed" | grep "^${folder}/changelog.d/" | sed "s|^${folder}/changelog.d/||" | grep -qE "$fragment_name_re"; then
return 0
fi
return 1
}
if [[ "${STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED}" == "true" ]]; then
# Check monitored folders
for folder in $MONITORED_FOLDERS; do
# Get files changed in this folder
changed_in_folder=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^${folder}/" || true)
if echo "$all_changed" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "Direct CHANGELOG.md edits are not allowed for ${folder}/"
handwritten_changelogs="${handwritten_changelogs}- \`${folder}/CHANGELOG.md\`"$'\n'
fi
changed_in_folder=$(echo "$all_changed" | grep "^${folder}/" | grep -v "^${folder}/CHANGELOG.md$" || true)
if [ -n "$changed_in_folder" ]; then
echo "Detected changes in ${folder}/"
# Check if CHANGELOG.md was updated
if ! echo "$changed_in_folder" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "No changelog update found for ${folder}/"
missing_changelogs="${missing_changelogs}- \`${folder}\`"$'\n'
if ! has_changelog_update "$folder"; then
echo "No changelog fragment found for ${folder}/"
missing_fragments="${missing_fragments}- \`${folder}\`"$'\n'
fi
fi
done
# Check root-level dependency files (uv.lock, pyproject.toml)
# These are associated with the prowler folder changelog
root_deps_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
root_deps_changed=$(echo "$all_changed" | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
if [ -n "$root_deps_changed" ]; then
echo "Detected changes in root dependency files: $root_deps_changed"
# Check if prowler/CHANGELOG.md was already updated (might have been caught above)
prowler_changelog_updated=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^prowler/CHANGELOG.md$" || true)
if [ -z "$prowler_changelog_updated" ]; then
if ! has_changelog_update "prowler"; then
# Only add if prowler wasn't already flagged
if ! echo "$missing_changelogs" | grep -q "prowler"; then
echo "No changelog update found for root dependency changes"
missing_changelogs="${missing_changelogs}- \`prowler\` (root dependency files changed)"$'\n'
if ! echo "$missing_fragments" | grep -q "prowler"; then
echo "No changelog fragment found for root dependency changes"
missing_fragments="${missing_fragments}- \`prowler\` (root dependency files changed)"$'\n'
fi
fi
fi
# Validate the filename of every fragment added by this PR
added_fragments=$(echo "$added_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $added_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ]; then
continue
fi
if ! echo "$name" | grep -qE "$fragment_name_re"; then
echo "Invalid fragment filename: $fragment"
invalid_fragments="${invalid_fragments}- \`${fragment}\`"$'\n'
fi
done
# Lint fragment content: the PR link is attached automatically at
# compile time, so a hand-written PR or issue link would be wrong
touched_fragments=$(echo "$added_modified_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $touched_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ] || [ ! -f "$fragment" ]; then
continue
fi
if grep -qE "$manual_pr_link_re" "$fragment"; then
echo "Fragment contains a hand-written PR or issue link: $fragment"
linked_fragments="${linked_fragments}- \`${fragment}\`"$'\n'
fi
done
fi
{
echo "missing_changelogs<<EOF"
echo -e "${missing_changelogs}"
echo "EOF"
} >> $GITHUB_OUTPUT
# Suggest a slug derived from the branch name for the bot comment
suggested_slug=$(echo "$HEAD_REF" | tr '[:upper:]' '[:lower:]' | sed 's|.*/||; s/[^a-z0-9._-]/-/g; s/^[^a-z0-9]*//')
if [ -z "$suggested_slug" ]; then
suggested_slug="my-change"
fi
fragment_help="A changelog fragment is a small Markdown file named \`<slug>.<type>.md\` under \`<component>/changelog.d/\`, where \`<type>\` is one of \`added\`, \`changed\`, \`deprecated\`, \`removed\`, \`fixed\` or \`security\`. Its content is the changelog entry text, without the PR link (added automatically at release time) and without a trailing period. For example:
\`\`\`
echo 'Entry text describing the change' > <component>/changelog.d/${suggested_slug}.fixed.md
\`\`\`
If this PR does not need a changelog entry, add the \`no-changelog\` label instead."
if [ -n "$missing_fragments" ] || [ -n "$invalid_fragments" ] || [ -n "$linked_fragments" ] || [ -n "$handwritten_changelogs" ]; then
comment_body=""
if [ -n "$missing_fragments" ]; then
comment_body="⚠️ **Changes detected in the following folders without a changelog fragment:**"$'\n\n'"${missing_fragments}"$'\n'
fi
if [ -n "$invalid_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragment filenames that do not follow the naming convention:**"$'\n\n'"${invalid_fragments}"$'\n'
fi
if [ -n "$linked_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragments containing a hand-written PR or issue link (remove it; the PR link is attached automatically at release time):**"$'\n\n'"${linked_fragments}"$'\n'
fi
if [ -n "$handwritten_changelogs" ]; then
comment_body="${comment_body}⚠️ **Direct \`CHANGELOG.md\` edits are not allowed in regular PRs:**"$'\n\n'"${handwritten_changelogs}"$'\n'
fi
comment_body="${comment_body}${fragment_help}"
else
comment_body="✅ All required changelog fragments are present."
fi
write_multiline_output() {
local name="$1"
local value="$2"
local delimiter
while true; do
delimiter="EOF_$(openssl rand -hex 16)"
if ! grep -qxF "$delimiter" <<< "$value"; then
break
fi
done
{
echo "${name}<<${delimiter}"
if [ -n "$value" ]; then
printf '%s\n' "$value"
fi
echo "${delimiter}"
} >> "$GITHUB_OUTPUT"
}
write_multiline_output "missing_fragments" "$missing_fragments"
write_multiline_output "invalid_fragments" "$invalid_fragments"
write_multiline_output "linked_fragments" "$linked_fragments"
write_multiline_output "handwritten_changelogs" "$handwritten_changelogs"
write_multiline_output "comment_body" "$comment_body"
env:
STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED: ${{ steps.changed-files.outputs.any_changed }}
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES: ${{ steps.changed-files.outputs.added_files }}
STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES: ${{ steps.changed-files.outputs.modified_files }}
STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES: ${{ steps.changed-files.outputs.renamed_files }}
HEAD_REF: ${{ github.event.pull_request.head.ref }}
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
@@ -128,14 +289,10 @@ jobs:
edit-mode: replace
body: |
<!-- changelog-check -->
${{ steps.check-folders.outputs.missing_changelogs != '' && format('⚠️ **Changes detected in the following folders without a corresponding update to the `CHANGELOG.md`:**
${{ steps.check-folders.outputs.comment_body }}
{0}
Please add an entry to the corresponding `CHANGELOG.md` file to maintain a clear history of changes.', steps.check-folders.outputs.missing_changelogs) || '✅ All necessary `CHANGELOG.md` files have been updated.' }}
- name: Fail if changelog is missing
if: steps.check-folders.outputs.missing_changelogs != ''
- name: Fail if changelog fragment is missing or invalid
if: steps.check-folders.outputs.missing_fragments != '' || steps.check-folders.outputs.invalid_fragments != '' || steps.check-folders.outputs.linked_fragments != '' || steps.check-folders.outputs.handwritten_changelogs != ''
run: |
echo "::error::Missing changelog updates in some folders"
echo "::error::Missing, invalid, or disallowed changelog updates"
exit 1
+1
View File
@@ -55,6 +55,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
+2
View File
@@ -12,6 +12,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -23,6 +24,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -113,6 +113,7 @@ jobs:
.github/workflows/sdk-container-checks.yml
files_ignore: |
prowler/CHANGELOG.md
prowler/changelog.d/**
**/AGENTS.md
- name: Set up Docker Buildx
+1
View File
@@ -73,6 +73,7 @@ jobs:
.github/scripts/osv-scan.sh
files_ignore: |
prowler/CHANGELOG.md
prowler/changelog.d/**
**/AGENTS.md
- name: Setup Python with uv
+1
View File
@@ -77,6 +77,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
+2
View File
@@ -10,6 +10,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -19,6 +20,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -102,6 +102,7 @@ jobs:
files: ui/**
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
+2
View File
@@ -14,6 +14,8 @@ on:
- '.github/test-impact.yml'
- 'ui/**'
- 'api/**' # API changes can affect UI E2E
- '!ui/CHANGELOG.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+1
View File
@@ -64,6 +64,7 @@ jobs:
.github/workflows/ui-tests.yml
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
+14 -6
View File
@@ -72,13 +72,13 @@ repos:
exclude: contrib
priority: 30
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/)
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/, docs/scripts/)
- repo: https://github.com/myint/autoflake
rev: v2.3.3
hooks:
- id: autoflake
name: "SDK - autoflake"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--in-place", "--remove-all-unused-imports", "--remove-unused-variable"]
priority: 20
@@ -87,7 +87,7 @@ repos:
hooks:
- id: isort
name: "SDK - isort"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--profile", "black"]
stages: ["pre-commit"]
priority: 20
@@ -97,7 +97,7 @@ repos:
hooks:
- id: black
name: "SDK - black"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 20
- repo: https://github.com/pycqa/flake8
@@ -105,7 +105,7 @@ repos:
hooks:
- id: flake8
name: "SDK - flake8"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--ignore=E266,W503,E203,E501,W605"]
priority: 30
@@ -142,6 +142,14 @@ repos:
files: { glob: ["mcp_server/**/*.py"] }
priority: 20
- id: generate-provider-cards
name: "Docs - regenerate provider cards snippet"
entry: python docs/scripts/generate_provider_cards.py
language: system
files: { glob: ["docs/user-guide/providers/**/getting-started-*.mdx", "docs/scripts/generate_provider_cards.py", "docs/snippets/provider-cards.mdx", "api/src/backend/api/models.py"] }
pass_filenames: false
priority: 20
## PYTHON — uv (API + SDK)
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.11.14
@@ -183,7 +191,7 @@ repos:
entry: pylint --disable=W,C,R,E -j 0 -rn -sn
language: system
types: [python]
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 30
- id: trufflehog
+32
View File
@@ -2,6 +2,35 @@
All notable changes to the **Prowler API** are documented in this file.
<!-- changelog: release notes start -->
## [1.34.1] (Prowler v5.33.1)
### 🐞 Fixed
- Session tokens are rejected after account password updates [(#11914)](https://github.com/prowler-cloud/prowler/pull/11914)
- Jira dispatch task results now surface user-facing Jira failure messages [(#11925)](https://github.com/prowler-cloud/prowler/pull/11925)
- AWS Attack Paths privilege escalation queries no longer fail on Neo4j with `Aggregation column contains implicit grouping expressions` [(#11939)](https://github.com/prowler-cloud/prowler/pull/11939)
### 🔐 Security
- OpenAI-compatible Lighthouse provider base URLs are restricted before connection checks [(#11940)](https://github.com/prowler-cloud/prowler/pull/11940)
- `LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS` environment variable to allow internal hosts as OpenAI-compatible Lighthouse AI base URLs [(#11942)](https://github.com/prowler-cloud/prowler/pull/11942)
---
## [1.34.0] (Prowler v5.33.0)
### 🚀 Added
- Compliance PDF reports no longer require provider credentials: findings are enriched from the provider metadata stored in the database, so reports generate even after the provider secret is deleted or its credentials become invalid [(#11845)](https://github.com/prowler-cloud/prowler/pull/11845)
### 🐞 Fixed
- Provider scans now queue behind active provider scans instead of dispatching concurrently, and resource failed-finding counters retry database conflicts with stable row locking [(#11848)](https://github.com/prowler-cloud/prowler/pull/11848)
---
## [1.33.1] (Prowler v5.32.1)
### 🐞 Fixed
@@ -12,6 +41,7 @@ All notable changes to the **Prowler API** are documented in this file.
### 🔐 Security
- User profile updates now allow users to update their own account while requiring user-management permissions to update other users in the same tenant [(#11792)](https://github.com/prowler-cloud/prowler/pull/11792)
- Kubernetes provider credentials now reject kubeconfigs using `exec` authentication in Prowler Cloud, preventing user-supplied commands from running on Cloud workers [(#11753)](https://github.com/prowler-cloud/prowler/pull/11753)
---
@@ -32,6 +62,8 @@ All notable changes to the **Prowler API** are documented in this file.
- Attack Paths: Provider graph cleanup now deletes Neo4j and Neptune relationships in directed batches before deleting nodes [(#11755)](https://github.com/prowler-cloud/prowler/pull/11755)
- `scan-perform` no longer reports an error when a provider is deleted during a running scan [(#11696)](https://github.com/prowler-cloud/prowler/pull/11696)
---
## [1.32.1] (Prowler v5.31.1)
### 🐞 Fixed
View File
+10
View File
@@ -0,0 +1,10 @@
# Changelog fragments
Each PR adds one small file here instead of editing `CHANGELOG.md` directly, so concurrent PRs never conflict.
- Filename: `<slug>.<type>.md`, e.g. `my-new-check.added.md` (slug is free-form: letters, digits, `.`, `_`, `-`)
- `<type>` is one of: `added`, `changed`, `deprecated`, `removed`, `fixed`, `security`
- Content: one line with the changelog entry text, without the PR link and without a trailing period (the PR link is attached automatically at release time)
- A PR adds as many fragment files as entries it needs, freely mixing types (one file per entry); same-type entries just use different slugs
Fragments are compiled into `CHANGELOG.md` when a release is prepared. Full conventions: `skills/prowler-changelog/SKILL.md`.
+2 -2
View File
@@ -45,7 +45,7 @@ dependencies = [
"gunicorn==26.0.0",
"uvloop==0.22.1",
"lxml==6.1.0",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@v5.32",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@v5.33",
"psycopg2-binary==2.9.9",
"pytest-celery[redis] (==1.3.0)",
"sentry-sdk[django] (==2.56.0)",
@@ -71,7 +71,7 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.33.2"
version = "1.34.1"
# Shared ruff baseline (kept in sync with mcp_server/pyproject.toml).
# target-version tracks this project's lowest supported Python.
+26 -13
View File
@@ -418,7 +418,8 @@ AWS_APPRUNNER_PRIVESC_UPDATE_SERVICE = AttackPathsQueryDefinition(
// Find existing App Runner services with roles attached (potential targets)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'tasks.apprunner.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -523,7 +524,8 @@ AWS_BEDROCK_PRIVESC_INVOKE_CODE_INTERPRETER = AttackPathsQueryDefinition(
// Find roles that trust the Bedrock AgentCore service (already attached to existing code interpreters)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'bedrock-agentcore.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -607,7 +609,8 @@ AWS_CLOUDFORMATION_PRIVESC_UPDATE_STACK = AttackPathsQueryDefinition(
// Find roles that trust CloudFormation service (already attached to existing stacks)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'cloudformation.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -753,7 +756,8 @@ AWS_CLOUDFORMATION_PRIVESC_CHANGESET = AttackPathsQueryDefinition(
// Find roles that trust CloudFormation service (already attached to existing stacks)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'cloudformation.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -844,7 +848,8 @@ AWS_CODEBUILD_PRIVESC_START_BUILD = AttackPathsQueryDefinition(
// Find roles that trust CodeBuild service (already attached to existing projects)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'codebuild.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -880,7 +885,8 @@ AWS_CODEBUILD_PRIVESC_START_BUILD_BATCH = AttackPathsQueryDefinition(
// Find roles that trust CodeBuild service (already attached to existing projects)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'codebuild.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1096,7 +1102,8 @@ AWS_EC2_PRIVESC_MODIFY_INSTANCE_ATTRIBUTE = AttackPathsQueryDefinition(
// Find EC2 instances with instance profiles (potential targets)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1187,7 +1194,8 @@ AWS_EC2_PRIVESC_LAUNCH_TEMPLATE = AttackPathsQueryDefinition(
// Find launch templates in the account (potential targets)
MATCH path_target = (aws)--(template:LaunchTemplate)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1223,7 +1231,8 @@ AWS_EC2INSTANCECONNECT_PRIVESC_SEND_SSH_PUBLIC_KEY = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1539,7 +1548,8 @@ AWS_ECS_PRIVESC_EXECUTE_COMMAND = AttackPathsQueryDefinition(
// Target: roles already attached to running tasks (trust ECS tasks service)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'ecs-tasks.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1622,7 +1632,8 @@ AWS_GLUE_PRIVESC_UPDATE_DEV_ENDPOINT = AttackPathsQueryDefinition(
// Find roles that trust Glue service (already attached to existing dev endpoints)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'glue.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -3337,7 +3348,8 @@ AWS_SSM_PRIVESC_START_SESSION = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -3373,7 +3385,8 @@ AWS_SSM_PRIVESC_SEND_COMMAND = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
+1 -1
View File
@@ -1,7 +1,7 @@
openapi: 3.0.3
info:
title: Prowler API
version: 1.33.2
version: 1.34.1
description: |-
Prowler API specification.
+2 -1
View File
@@ -16,7 +16,7 @@ if TYPE_CHECKING:
class SSEChannelManager(DefaultChannelManager):
"""Connect `django-eventstream` to the platform's SSE viewsets."""
def get_channels_for_request(self, request: Request, view_kwargs: dict) -> set[str]: # noqa: vulture
def get_channels_for_request(self, request: Request, view_kwargs: dict) -> set[str]:
"""Return the request's channels scoped to the active JWT tenant.
Args:
@@ -30,6 +30,7 @@ class SSEChannelManager(DefaultChannelManager):
The subset of `request.sse_channels` whose embedded tenant
matches the active request tenant.
"""
_ = view_kwargs
try:
request_tenant_id = UUID(str(getattr(request, "tenant_id", None)))
except (TypeError, ValueError):
@@ -1,3 +1,4 @@
import json
import time
from datetime import UTC, datetime, timedelta
from uuid import uuid4
@@ -8,6 +9,10 @@ from conftest import TEST_PASSWORD, get_api_tokens, get_authorization_header
from django.urls import reverse
from drf_simple_apikey.crypto import get_crypto
from rest_framework.test import APIClient
from rest_framework_simplejwt.token_blacklist.models import (
BlacklistedToken,
OutstandingToken,
)
@pytest.mark.django_db
@@ -103,6 +108,118 @@ def test_refresh_token(create_test_user, tenants_fixture):
assert new_refresh_response.status_code == 200
@pytest.mark.django_db
def test_password_change_invalidates_existing_tokens(create_test_user, tenants_fixture):
client = APIClient()
new_password = "ChangedSecret123@"
access_token, refresh_token = get_api_tokens(
client, create_test_user.email, TEST_PASSWORD
)
auth_headers = get_authorization_header(access_token)
outstanding_token_ids = list(
OutstandingToken.objects.filter(user=create_test_user).values_list(
"id", flat=True
)
)
assert outstanding_token_ids
assert not BlacklistedToken.objects.filter(
token_id__in=outstanding_token_ids
).exists()
password_change_payload = {
"data": {
"type": "users",
"id": str(create_test_user.id),
"attributes": {"password": new_password},
}
}
password_change_response = client.patch(
reverse("user-detail", kwargs={"pk": create_test_user.id}),
data=json.dumps(password_change_payload),
headers=auth_headers,
content_type="application/vnd.api+json",
)
assert password_change_response.status_code == 200, password_change_response.json()
assert BlacklistedToken.objects.filter(
token_id__in=outstanding_token_ids
).count() == len(outstanding_token_ids)
old_access_response = client.get(reverse("user-me"), headers=auth_headers)
assert old_access_response.status_code == 401
old_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": refresh_token},
}
},
format="vnd.api+json",
)
assert old_refresh_response.status_code == 400
new_access_token, _ = get_api_tokens(client, create_test_user.email, new_password)
new_access_response = client.get(
reverse("user-me"), headers=get_authorization_header(new_access_token)
)
assert new_access_response.status_code == 200
@pytest.mark.django_db
def test_password_change_invalidates_rotated_refresh_token(
create_test_user, tenants_fixture
):
client = APIClient()
new_password = "ChangedSecret123@"
access_token, refresh_token = get_api_tokens(
client, create_test_user.email, TEST_PASSWORD
)
rotated_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": refresh_token},
}
},
format="vnd.api+json",
)
assert rotated_refresh_response.status_code == 200
rotated_refresh_token = rotated_refresh_response.json()["data"]["attributes"][
"refresh"
]
password_change_payload = {
"data": {
"type": "users",
"id": str(create_test_user.id),
"attributes": {"password": new_password},
}
}
password_change_response = client.patch(
reverse("user-detail", kwargs={"pk": create_test_user.id}),
data=json.dumps(password_change_payload),
headers=get_authorization_header(access_token),
content_type="application/vnd.api+json",
)
assert password_change_response.status_code == 200, password_change_response.json()
old_rotated_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": rotated_refresh_token},
}
},
format="vnd.api+json",
)
assert old_rotated_refresh_response.status_code == 400
@pytest.mark.django_db
def test_user_me_when_inviting_users(create_test_user, tenants_fixture, roles_fixture):
client = APIClient()
@@ -189,6 +306,8 @@ def test_user_me_when_inviting_users(create_test_user, tenants_fixture, roles_fi
class TestTokenSwitchTenant:
def test_switch_tenant_with_valid_token(self, tenants_fixture, providers_fixture):
client = APIClient()
aws_provider = providers_fixture[0]
assert aws_provider
test_user = "test_email@prowler.com"
test_password = "Test_password1@"
@@ -1403,6 +1522,8 @@ class TestAPIKeyMultiTenantWorkflows:
Verifies RLS enforcement after authentication ensures tenant isolation.
"""
client = APIClient()
aws_provider = providers_fixture[0]
assert aws_provider
user1 = User.objects.create_user(
name="tenant1_user",
+72 -1
View File
@@ -1,6 +1,6 @@
import pytest
from api.v1.serializer_utils.integrations import S3ConfigSerializer
from api.v1.serializers import ImageProviderSecret
from api.v1.serializers import ImageProviderSecret, KubernetesProviderSecret
from rest_framework.exceptions import ValidationError
@@ -132,3 +132,74 @@ class TestImageProviderSecret:
serializer = ImageProviderSecret(data={"registry_password": "pass"})
assert not serializer.is_valid()
assert "non_field_errors" in serializer.errors
class TestKubernetesProviderSecret:
def test_valid_static_kubeconfig_is_accepted(self):
kubeconfig_content = """
apiVersion: v1
kind: Config
clusters:
- name: test-cluster
cluster:
server: https://kubernetes.example.test
users:
- name: test-user
user:
token: test-token
contexts:
- name: test-context
context:
cluster: test-cluster
user: test-user
current-context: test-context
"""
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": kubeconfig_content}
)
assert serializer.is_valid()
def test_kubeconfig_with_exec_authentication_is_rejected(self):
kubeconfig_content = """
apiVersion: v1
kind: Config
clusters:
- name: test-cluster
cluster:
server: https://kubernetes.example.test
users:
- name: test-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1
command: kubectl
contexts:
- name: test-context
context:
cluster: test-cluster
user: test-user
current-context: test-context
"""
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": kubeconfig_content}
)
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors
def test_malformed_kubeconfig_is_rejected(self):
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": "apiVersion: ["}
)
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors
def test_non_mapping_kubeconfig_is_rejected(self):
serializer = KubernetesProviderSecret(data={"kubeconfig_content": "[]"})
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors
@@ -0,0 +1,246 @@
import socket
import pytest
from api.validators import (
resolve_lighthouse_openai_compatible_host,
validate_lighthouse_openai_compatible_base_url,
)
from django.core.exceptions import ValidationError
from django.test import override_settings
def test_lighthouse_base_url_rejects_http_scheme():
with pytest.raises(ValidationError, match="HTTPS"):
validate_lighthouse_openai_compatible_base_url(
"http://openrouter.ai/api/v1",
resolve_dns=False,
)
@pytest.mark.parametrize(
"base_url",
[
"https://openrouter.ai:0/api/v1",
"https://openrouter.ai:-1/api/v1",
"https://openrouter.ai:65536/api/v1",
"https://openrouter.ai:invalid/api/v1",
],
)
def test_lighthouse_base_url_rejects_invalid_port(base_url):
with pytest.raises(ValidationError, match="port is invalid"):
validate_lighthouse_openai_compatible_base_url(
base_url,
resolve_dns=False,
)
@pytest.mark.parametrize("port", [1, 65535])
def test_lighthouse_base_url_accepts_valid_port_boundaries(port):
assert (
validate_lighthouse_openai_compatible_base_url(
f"https://openrouter.ai:{port}/api/v1",
resolve_dns=False,
)
is None
)
def test_lighthouse_base_url_rejects_localhost():
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
"https://localhost/v1",
resolve_dns=False,
)
@pytest.mark.parametrize("ip_address", ["10.0.0.1", "172.16.0.1", "192.168.1.1"])
def test_lighthouse_base_url_rejects_private_ip_literal(ip_address):
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
f"https://{ip_address}/v1",
resolve_dns=False,
)
def test_lighthouse_base_url_rejects_metadata_ip_literal():
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
"https://169.254.169.254/latest/meta-data",
resolve_dns=False,
)
@pytest.mark.parametrize(
"base_url",
[
"https://[::ffff:169.254.169.254]/v1",
"https://[64:ff9b::a9fe:a9fe]/v1",
"https://[2002:a9fe:a9fe::]/v1",
],
)
def test_lighthouse_base_url_rejects_embedded_non_global_ip(base_url):
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
base_url,
resolve_dns=False,
)
@pytest.mark.parametrize(
"base_url",
[
"https://[::ffff:93.184.216.34]/v1",
"https://[64:ff9b::5db8:d822]/v1",
"https://[2002:5db8:d822::]/v1",
],
)
def test_lighthouse_base_url_accepts_embedded_public_ip(base_url):
assert (
validate_lighthouse_openai_compatible_base_url(
base_url,
resolve_dns=False,
)
is None
)
def test_lighthouse_base_url_accepts_hostname_without_dns_resolution():
assert (
validate_lighthouse_openai_compatible_base_url(
"https://openrouter.ai/api/v1",
resolve_dns=False,
)
is None
)
def test_lighthouse_base_url_rejects_post_dns_internal_address(monkeypatch):
def resolve_to_metadata(*_args, **_kwargs):
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
6,
"",
("169.254.169.254", 443),
)
]
monkeypatch.setattr("api.validators.socket.getaddrinfo", resolve_to_metadata)
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
"https://metadata.example.test/v1"
)
def test_lighthouse_base_url_accepts_public_resolved_address(monkeypatch):
def resolve_to_public(*_args, **_kwargs):
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
6,
"",
("93.184.216.34", 443),
)
]
monkeypatch.setattr("api.validators.socket.getaddrinfo", resolve_to_public)
assert (
validate_lighthouse_openai_compatible_base_url("https://openrouter.ai/api/v1")
is None
)
@override_settings(
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["custom-openai.internal"]
)
def test_lighthouse_base_url_accepts_allowlisted_host_without_resolution(monkeypatch):
def fail_resolution(*_args, **_kwargs):
raise AssertionError("allowlisted hosts must not be resolved")
monkeypatch.setattr("api.validators.socket.getaddrinfo", fail_resolution)
assert (
validate_lighthouse_openai_compatible_base_url(
"https://custom-openai.internal/v1"
)
is None
)
@override_settings(
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["custom-openai.internal"]
)
def test_lighthouse_resolve_returns_allowlisted_hostname_unpinned():
assert resolve_lighthouse_openai_compatible_host(
"Custom-OpenAI.internal.", 443
) == ("custom-openai.internal",)
@override_settings(LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["localhost"])
def test_lighthouse_base_url_accepts_allowlisted_blocked_host():
assert (
validate_lighthouse_openai_compatible_base_url(
"https://localhost/v1",
resolve_dns=False,
)
is None
)
@override_settings(LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["10.0.0.1"])
def test_lighthouse_base_url_accepts_allowlisted_private_ip_literal():
assert (
validate_lighthouse_openai_compatible_base_url(
"https://10.0.0.1/v1",
resolve_dns=False,
)
is None
)
@override_settings(
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=[" Custom-OpenAI.Internal. "]
)
def test_lighthouse_allowlist_entries_are_normalized():
assert (
validate_lighthouse_openai_compatible_base_url(
"https://custom-openai.internal/v1",
resolve_dns=False,
)
is None
)
@override_settings(
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["custom-openai.internal"]
)
def test_lighthouse_base_url_rejects_host_not_in_allowlist():
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
"https://localhost/v1",
resolve_dns=False,
)
@override_settings(LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=[""])
def test_lighthouse_allowlist_ignores_empty_entries():
with pytest.raises(ValidationError, match="external public endpoint"):
validate_lighthouse_openai_compatible_base_url(
"https://localhost/v1",
resolve_dns=False,
)
@override_settings(
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=["custom-openai.internal"]
)
def test_lighthouse_base_url_allowlisted_host_still_requires_https():
with pytest.raises(ValidationError, match="HTTPS"):
validate_lighthouse_openai_compatible_base_url(
"http://custom-openai.internal/v1",
resolve_dns=False,
)
+204 -9
View File
@@ -65,6 +65,7 @@ from api.v1.views import (
TenantFinishACSView,
)
from botocore.exceptions import ClientError, NoCredentialsError
from celery import states
from conftest import (
API_JSON_CONTENT_TYPE,
TEST_PASSWORD,
@@ -2959,7 +2960,24 @@ class TestProviderSecretViewSet:
Provider.ProviderChoices.KUBERNETES.value,
ProviderSecret.TypeChoices.STATIC,
{
"kubeconfig_content": "kubeconfig-content",
"kubeconfig_content": """
apiVersion: v1
kind: Config
clusters:
- name: test-cluster
cluster:
server: https://kubernetes.example.test
users:
- name: test-user
user:
token: test-token
contexts:
- name: test-context
context:
cluster: test-cluster
user: test-user
current-context: test-context
""",
},
),
# M365 client secret credentials
@@ -3644,21 +3662,15 @@ class TestScanViewSet:
),
],
)
@patch("api.v1.views.Task.objects.get")
@patch("api.v1.views.perform_scan_task.apply_async")
@patch("api.v1.views.enqueue_scan_execution_on_commit")
def test_scans_create_valid(
self,
mock_perform_scan_task,
mock_task_get,
mock_enqueue_scan_execution,
authenticated_client,
scan_json_payload,
_expected_scanner_args,
providers_fixture,
tasks_fixture,
):
prowler_task = tasks_fixture[0]
mock_perform_scan_task.return_value.id = prowler_task.id
mock_task_get.return_value = prowler_task
*_, provider5 = providers_fixture
# Provider5 has these scanner_args
# scanner_args={"key1": "value1", "key2": {"key21": "value21"}}
@@ -3683,8 +3695,121 @@ class TestScanViewSet:
assert scan.name == scan_json_payload["data"]["attributes"]["name"]
assert scan.provider == provider5
assert scan.trigger == Scan.TriggerChoices.MANUAL
mock_enqueue_scan_execution.assert_called_once()
# assert scan.scanner_args == expected_scanner_args
@patch("tasks.tasks.perform_scan_task.apply_async")
def test_scans_create_queues_scan_when_provider_has_active_scan(
self,
mock_perform_scan_task,
authenticated_client,
providers_fixture,
tenants_fixture,
django_capture_on_commit_callbacks,
):
tenant, *_ = tenants_fixture
provider, *_ = providers_fixture
task_result = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan-perform",
status=states.PENDING,
)
prowler_task = Task.objects.create(
id=task_result.task_id,
tenant_id=tenant.id,
task_runner_task=task_result,
)
Scan.objects.create(
name="Active scan",
provider=provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
tenant_id=tenant.id,
task=prowler_task,
)
with django_capture_on_commit_callbacks(execute=True):
response = authenticated_client.post(
reverse("scan-list"),
data={
"data": {
"type": "scans",
"attributes": {"name": "Duplicate Scan"},
"relationships": {
"provider": {
"data": {"type": "providers", "id": str(provider.id)}
}
},
}
},
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_202_ACCEPTED
assert response.json()["data"]["id"] != str(prowler_task.id)
assert Scan.objects.count() == 2
queued_scan = Scan.objects.exclude(task=prowler_task).get()
assert queued_scan.trigger == Scan.TriggerChoices.MANUAL
assert queued_scan.state == StateChoices.AVAILABLE
assert queued_scan.task.task_runner_task.status == "QUEUED"
mock_perform_scan_task.assert_not_called()
@patch("tasks.tasks.perform_scan_task.apply_async")
def test_scans_create_queues_scan_when_scheduled_scan_is_claimed(
self,
mock_perform_scan_task,
authenticated_client,
providers_fixture,
tenants_fixture,
django_capture_on_commit_callbacks,
):
tenant, *_ = tenants_fixture
provider, *_ = providers_fixture
task_result = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan-perform-scheduled",
status=states.STARTED,
)
prowler_task = Task.objects.create(
id=task_result.task_id,
tenant_id=tenant.id,
task_runner_task=task_result,
)
Scan.objects.create(
name="Claimed scheduled scan",
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.SCHEDULED,
tenant_id=tenant.id,
task=prowler_task,
)
with django_capture_on_commit_callbacks(execute=True):
response = authenticated_client.post(
reverse("scan-list"),
data={
"data": {
"type": "scans",
"attributes": {"name": "Manual Scan"},
"relationships": {
"provider": {
"data": {"type": "providers", "id": str(provider.id)}
}
},
}
},
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_202_ACCEPTED
assert response.json()["data"]["id"] != str(prowler_task.id)
assert Scan.objects.count() == 2
queued_scan = Scan.objects.exclude(task=prowler_task).get()
assert queued_scan.trigger == Scan.TriggerChoices.MANUAL
assert queued_scan.state == StateChoices.AVAILABLE
assert queued_scan.task.task_runner_task.status == "QUEUED"
mock_perform_scan_task.assert_not_called()
@pytest.mark.parametrize(
"scan_json_payload, error_code",
[
@@ -17005,6 +17130,76 @@ class TestLighthouseProviderConfigViewSet:
error_detail = str(resp.json()).lower()
assert "base_url" in error_detail
@pytest.mark.parametrize(
"base_url",
[
"https://127.0.0.1/v1",
"https://169.254.169.254/latest/meta-data",
],
)
def test_openai_compatible_rejects_internal_base_url_on_create(
self, authenticated_client, base_url
):
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai_compatible",
"base_url": base_url,
"credentials": {"api_key": "compat-key"},
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_400_BAD_REQUEST
assert "base_url" in str(resp.json()).lower()
def test_openai_compatible_rejects_internal_base_url_on_update(
self, authenticated_client
):
create_payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai_compatible",
"base_url": "https://openrouter.ai/api/v1",
"credentials": {"api_key": "compat-key-123"},
},
}
}
create_resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=create_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert create_resp.status_code == status.HTTP_201_CREATED
provider_id = create_resp.json()["data"]["id"]
patch_payload = {
"data": {
"type": "lighthouse-providers",
"id": provider_id,
"attributes": {
"base_url": "https://169.254.169.254/latest/meta-data",
},
}
}
patch_resp = authenticated_client.patch(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
data=patch_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert patch_resp.status_code == status.HTTP_400_BAD_REQUEST
assert "base_url" in str(patch_resp.json()).lower()
def test_openai_compatible_invalid_credentials(self, authenticated_client):
payload = {
"data": {
@@ -0,0 +1,18 @@
from api.db_router import MainRouter
from rest_framework_simplejwt.token_blacklist.models import (
BlacklistedToken,
OutstandingToken,
)
def blacklist_user_refresh_tokens(user_id):
outstanding_token_ids = list(
OutstandingToken.objects.using(MainRouter.admin_db)
.filter(user_id=user_id)
.values_list("id", flat=True)
)
if outstanding_token_ids:
BlacklistedToken.objects.using(MainRouter.admin_db).bulk_create(
[BlacklistedToken(token_id=token_id) for token_id in outstanding_token_ids],
ignore_conflicts=True,
)
@@ -213,7 +213,8 @@ from rest_framework_json_api import serializers
"properties": {
"kubeconfig_content": {
"type": "string",
"description": "The content of the Kubernetes kubeconfig file, encoded as a string.",
"description": "The content of the Kubernetes kubeconfig file, encoded as a string. "
"Kubeconfig exec authentication is not supported in Prowler Cloud for security reasons.",
}
},
"required": ["kubeconfig_content"],
+102 -42
View File
@@ -2,6 +2,7 @@ import base64
import json
from datetime import UTC, datetime, timedelta
import yaml
from api.db_router import MainRouter
from api.exceptions import ConflictException
from api.models import (
@@ -37,6 +38,7 @@ from api.models import (
UserRoleRelationship,
)
from api.rls import Tenant
from api.v1.serializer_utils.authentication import blacklist_user_refresh_tokens
from api.v1.serializer_utils.integrations import (
AWSCredentialSerializer,
IntegrationConfigField,
@@ -55,12 +57,13 @@ from api.v1.serializer_utils.lighthouse import (
)
from api.v1.serializer_utils.processors import ProcessorConfigField
from api.v1.serializer_utils.providers import ProviderSecretField
from api.validators import validate_lighthouse_openai_compatible_base_url
from django.conf import settings
from django.contrib.auth import authenticate
from django.contrib.auth.models import update_last_login
from django.contrib.auth.password_validation import validate_password
from django.core.exceptions import ValidationError as DjangoValidationError
from django.db import IntegrityError
from django.db import IntegrityError, transaction
from drf_spectacular.utils import extend_schema_field
from jwt.exceptions import InvalidKeyError
from prowler.lib.mutelist.mutelist import Mutelist
@@ -71,11 +74,28 @@ from rest_framework_json_api.relations import SerializerMethodResourceRelatedFie
from rest_framework_json_api.serializers import ValidationError
from rest_framework_simplejwt.exceptions import TokenError
from rest_framework_simplejwt.serializers import TokenObtainPairSerializer
from rest_framework_simplejwt.settings import api_settings
from rest_framework_simplejwt.tokens import RefreshToken
from rest_framework_simplejwt.utils import get_md5_hash_password
# Base
def _validate_lighthouse_base_url_without_dns(base_url: str) -> None:
try:
validate_lighthouse_openai_compatible_base_url(base_url, resolve_dns=False)
except DjangoValidationError as error:
raise ValidationError({"base_url": error.messages[0]}) from error
def _reraise_lighthouse_credentials_errors(error: ValidationError) -> None:
details = error.detail.copy()
for key, value in details.items():
error.detail[f"credentials/{key}"] = value
del error.detail[key]
raise error
class BaseModelSerializerV1(serializers.ModelSerializer):
def get_root_meta(self, _resource, _many):
return {"version": "v1"}
@@ -231,6 +251,18 @@ class TokenRefreshSerializer(BaseSerializerV1):
try:
# Validate the refresh token
refresh = RefreshToken(refresh_token)
if api_settings.CHECK_REVOKE_TOKEN:
user_id = refresh.payload.get(api_settings.USER_ID_CLAIM)
try:
user = User.objects.using(MainRouter.admin_db).get(
**{api_settings.USER_ID_FIELD: user_id}
)
except User.DoesNotExist:
raise TokenError("User not found.") from None
if refresh.get(api_settings.REVOKE_TOKEN_CLAIM) != (
get_md5_hash_password(user.password)
):
raise TokenError("The user's password has been changed.")
# Generate new access token
access_token = refresh.access_token
@@ -404,7 +436,13 @@ class UserUpdateSerializer(BaseWriteSerializer):
password = validated_data.pop("password", None)
if password:
validate_password(password, user=instance)
instance.set_password(password)
with transaction.atomic(using=MainRouter.admin_db):
instance.set_password(password)
for attr, value in validated_data.items():
setattr(instance, attr, value)
blacklist_user_refresh_tokens(instance.id)
instance.save(using=MainRouter.admin_db)
return instance
return super().update(instance, validated_data)
@@ -1530,6 +1568,32 @@ class FindingMetadataSerializer(BaseSerializerV1):
# Provider secrets
KUBERNETES_KUBECONFIG_EXEC_ERROR = (
"Kubernetes kubeconfig exec authentication is not supported in Prowler Cloud "
"for security reasons."
)
KUBERNETES_KUBECONFIG_INVALID_ERROR = "Invalid Kubernetes kubeconfig content."
def kubeconfig_contains_exec_auth(kubeconfig: dict) -> bool:
users = kubeconfig.get("users", [])
if not isinstance(users, list):
raise ValidationError(KUBERNETES_KUBECONFIG_INVALID_ERROR)
for user_entry in users:
if not isinstance(user_entry, dict):
raise ValidationError(KUBERNETES_KUBECONFIG_INVALID_ERROR)
user = user_entry.get("user", {})
if not isinstance(user, dict):
raise ValidationError(KUBERNETES_KUBECONFIG_INVALID_ERROR)
if "exec" in user:
return True
return False
class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
@staticmethod
def validate_secret_based_on_provider(
@@ -1711,6 +1775,22 @@ class MongoDBAtlasProviderSecret(serializers.Serializer):
class KubernetesProviderSecret(serializers.Serializer):
kubeconfig_content = serializers.CharField()
def validate_kubeconfig_content(self, kubeconfig_content):
try:
kubeconfig = yaml.safe_load(kubeconfig_content)
except yaml.YAMLError as exc:
raise serializers.ValidationError(
KUBERNETES_KUBECONFIG_INVALID_ERROR
) from exc
if not isinstance(kubeconfig, dict):
raise serializers.ValidationError(KUBERNETES_KUBECONFIG_INVALID_ERROR)
if kubeconfig_contains_exec_auth(kubeconfig):
raise serializers.ValidationError(KUBERNETES_KUBECONFIG_EXEC_ERROR)
return kubeconfig_content
class Meta:
resource_name = "provider-secrets"
@@ -3581,11 +3661,7 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
_reraise_lighthouse_credentials_errors(e)
elif (
provider_type == LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
@@ -3594,27 +3670,20 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
_reraise_lighthouse_credentials_errors(e)
elif (
provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE
):
if not base_url:
raise ValidationError({"base_url": "Base URL is required."})
_validate_lighthouse_base_url_without_dns(base_url)
try:
OpenAICompatibleCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
_reraise_lighthouse_credentials_errors(e)
return super().validate(attrs)
@@ -3677,11 +3746,7 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
_reraise_lighthouse_credentials_errors(e)
elif (
credentials is not None
and provider_type
@@ -3705,11 +3770,7 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
_reraise_lighthouse_credentials_errors(e)
# Then enforce invariants about not changing the auth method
# If the existing config uses an API key, forbid introducing access keys.
@@ -3736,24 +3797,23 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
}
)
elif (
credentials is not None
and provider_type
provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE
):
if base_url is None:
pass
elif not base_url:
effective_base_url = (
base_url if "base_url" in attrs else getattr(self.instance, "base_url")
)
if not effective_base_url:
raise ValidationError({"base_url": "Base URL cannot be empty."})
try:
OpenAICompatibleCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
if "base_url" in attrs:
_validate_lighthouse_base_url_without_dns(effective_base_url)
if credentials is not None:
try:
OpenAICompatibleCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
_reraise_lighthouse_credentials_errors(e)
return super().validate(attrs)
+24 -22
View File
@@ -237,7 +237,7 @@ from api.v1.serializers import (
UserUpdateSerializer,
)
from botocore.exceptions import ClientError, NoCredentialsError, ParamValidationError
from celery import chain, states
from celery import chain
from celery.result import AsyncResult
from config.custom_logging import BackendLogger
from config.env import env
@@ -283,7 +283,6 @@ from django.utils.dateparse import parse_date
from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_control
from django_celery_beat.models import PeriodicTask
from django_celery_results.models import TaskResult
from drf_spectacular.settings import spectacular_settings
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import (
@@ -322,17 +321,20 @@ from tasks.beat import schedule_provider_scan
from tasks.jobs.attack_paths import db_utils as attack_paths_db_utils
from tasks.jobs.export import get_s3_client
from tasks.tasks import (
QUEUED_SCAN_TASK_STATE,
backfill_compliance_summaries_task,
backfill_scan_resource_summaries_task,
check_integration_connection_task,
check_lighthouse_connection_task,
check_lighthouse_provider_connection_task,
check_provider_connection_task,
create_scan_task_record,
delete_provider_task,
delete_tenant_task,
enqueue_scan_execution_on_commit,
get_active_provider_scan,
jira_integration_task,
mute_historical_findings_task,
perform_scan_task,
reaggregate_all_finding_group_summaries_task,
refresh_lighthouse_provider_models_task,
)
@@ -2717,12 +2719,23 @@ class ScanViewSet(BaseRLSViewSet):
def create(self, request, *args, **kwargs):
input_serializer = self.get_serializer(data=request.data)
input_serializer.is_valid(raise_exception=True)
provider = input_serializer.validated_data.get("provider")
active_scan = None
# Broker publish is deferred to on_commit so the worker cannot read
# Scan before BaseRLSViewSet's dispatch-wide atomic commits.
pre_task_id = str(uuid.uuid4())
with transaction.atomic():
if provider:
provider = Provider.objects.select_for_update().get(
id=provider.id,
tenant_id=self.request.tenant_id,
)
active_scan = get_active_provider_scan(
self.request.tenant_id, provider.id
)
scan = input_serializer.save()
scan.task_id = pre_task_id
scan.save(update_fields=["task_id"])
@@ -2733,29 +2746,18 @@ class ScanViewSet(BaseRLSViewSet):
provider_id=str(scan.provider_id),
)
task_result, _ = TaskResult.objects.get_or_create(
task_id=pre_task_id,
defaults={"status": states.PENDING, "task_name": "scan-perform"},
)
prowler_task, _ = Task.objects.update_or_create(
id=pre_task_id,
prowler_task = create_scan_task_record(
tenant_id=self.request.tenant_id,
defaults={"task_runner_task": task_result},
task_id=pre_task_id,
task_status=(QUEUED_SCAN_TASK_STATE if active_scan else None),
)
scan_kwargs = {
"tenant_id": self.request.tenant_id,
"scan_id": str(scan.id),
"provider_id": str(scan.provider_id),
# Disabled for now
# checks_to_execute=scan.scanner_args.get("checks_to_execute")
}
transaction.on_commit(
lambda: perform_scan_task.apply_async(
kwargs=scan_kwargs, task_id=pre_task_id
if not active_scan:
enqueue_scan_execution_on_commit(
tenant_id=self.request.tenant_id,
scan=scan,
task_id=pre_task_id,
)
)
self.response_serializer_class = TaskSerializer
output_serializer = self.get_serializer(prowler_task)
+145
View File
@@ -1,14 +1,155 @@
import ipaddress
import socket
import string
from urllib.parse import urlparse
from django.conf import settings
from django.core.exceptions import ValidationError
from django.utils.translation import gettext as _
LIGHTHOUSE_OPENAI_COMPATIBLE_ALLOWED_SCHEMES = frozenset({"https"})
LIGHTHOUSE_NAT64_WELL_KNOWN_PREFIX = ipaddress.IPv6Network("64:ff9b::/96")
LIGHTHOUSE_BLOCKED_METADATA_HOSTS = frozenset(
{
"169.254.169.254",
"169.254.170.2",
"fd00:ec2::254",
"localhost",
"metadata.google.internal",
}
)
def _normalize_hostname(hostname: str) -> str:
return hostname.rstrip(".").lower()
def _lighthouse_openai_compatible_allowed_hosts() -> frozenset[str]:
return frozenset(
_normalize_hostname(allowed_host.strip())
for allowed_host in settings.LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS
if allowed_host and allowed_host.strip()
)
def _validate_lighthouse_public_ip(address: str) -> None:
ip_address = ipaddress.ip_address(address)
if isinstance(ip_address, ipaddress.IPv6Address):
# Classify transition addresses by their effective IPv4 destination.
embedded_ip_address = ip_address.ipv4_mapped or ip_address.sixtofour
if (
embedded_ip_address is None
and ip_address in LIGHTHOUSE_NAT64_WELL_KNOWN_PREFIX
):
embedded_ip_address = ipaddress.IPv4Address(int(ip_address) & 0xFFFFFFFF)
if embedded_ip_address is not None:
ip_address = embedded_ip_address
if not ip_address.is_global:
raise ValidationError(
_("Base URL must use an external public endpoint."),
code="lighthouse_base_url_not_public",
)
def resolve_lighthouse_openai_compatible_host(
hostname: str,
port: int,
*,
resolve_dns: bool = True,
) -> tuple[str, ...]:
"""Return public IP addresses that are safe for Lighthouse outbound use."""
hostname = _normalize_hostname(hostname)
if hostname in _lighthouse_openai_compatible_allowed_hosts():
# Operator-allowlisted hosts skip the public-endpoint checks; returning
# the hostname makes the network backend connect through regular DNS
# resolution instead of pinned addresses.
return (hostname,)
if hostname in LIGHTHOUSE_BLOCKED_METADATA_HOSTS or hostname.endswith(".localhost"):
raise ValidationError(
_("Base URL must use an external public endpoint."),
code="lighthouse_base_url_blocked_host",
)
try:
_validate_lighthouse_public_ip(hostname)
except ValueError:
if not resolve_dns:
return ()
else:
return (hostname,)
try:
resolved_addresses = socket.getaddrinfo(hostname, port, type=socket.SOCK_STREAM)
except socket.gaierror as error:
raise ValidationError(
_("Base URL host could not be resolved."),
code="lighthouse_base_url_resolution_failed",
) from error
if not resolved_addresses:
raise ValidationError(
_("Base URL host could not be resolved."),
code="lighthouse_base_url_resolution_failed",
)
public_addresses: list[str] = []
for resolved_address in resolved_addresses:
socket_address = resolved_address[4]
resolved_ip_address = socket_address[0]
_validate_lighthouse_public_ip(resolved_ip_address)
if resolved_ip_address not in public_addresses:
public_addresses.append(resolved_ip_address)
return tuple(public_addresses)
def validate_lighthouse_openai_compatible_base_url(
base_url: str,
*,
resolve_dns: bool = True,
) -> None:
"""Validate an OpenAI-compatible Lighthouse base URL before outbound use."""
parsed = urlparse(str(base_url))
if parsed.scheme.lower() not in LIGHTHOUSE_OPENAI_COMPATIBLE_ALLOWED_SCHEMES:
raise ValidationError(
_("Base URL must use HTTPS."),
code="lighthouse_base_url_invalid_scheme",
)
if not parsed.hostname:
raise ValidationError(
_("Base URL must include a host."),
code="lighthouse_base_url_missing_host",
)
try:
port = parsed.port
except ValueError as error:
raise ValidationError(
_("Base URL port is invalid."),
code="lighthouse_base_url_invalid_port",
) from error
if port is not None and not 1 <= port <= 65535:
raise ValidationError(
_("Base URL port is invalid."),
code="lighthouse_base_url_invalid_port",
)
resolve_lighthouse_openai_compatible_host(
parsed.hostname,
port or 443,
resolve_dns=resolve_dns,
)
class MaximumLengthValidator:
def __init__(self, max_length=72):
self.max_length = max_length
def validate(self, password, user=None):
del user
if len(password) > self.max_length:
raise ValidationError(
_(
@@ -31,6 +172,7 @@ class SpecialCharactersValidator:
self.min_special_characters = min_special_characters
def validate(self, password, user=None):
del user
if (
sum(1 for char in password if char in self.special_characters)
< self.min_special_characters
@@ -55,6 +197,7 @@ class UppercaseValidator:
self.min_uppercase = min_uppercase
def validate(self, password, user=None):
del user
if sum(1 for char in password if char.isupper()) < self.min_uppercase:
raise ValidationError(
_(
@@ -75,6 +218,7 @@ class LowercaseValidator:
self.min_lowercase = min_lowercase
def validate(self, password, user=None):
del user
if sum(1 for char in password if char.islower()) < self.min_lowercase:
raise ValidationError(
_(
@@ -95,6 +239,7 @@ class NumericValidator:
self.min_numeric = min_numeric
def validate(self, password, user=None):
del user
if sum(1 for char in password if char.isdigit()) < self.min_numeric:
raise ValidationError(
_(
+10
View File
@@ -230,6 +230,7 @@ SIMPLE_JWT = {
"JTI_CLAIM": "jti",
"USER_ID_FIELD": "id",
"USER_ID_CLAIM": "sub",
"CHECK_REVOKE_TOKEN": True,
# Issuer and Audience claims, for the moment we will keep these values as default values, they may change in the
# future.
"AUDIENCE": env.str("DJANGO_JWT_AUDIENCE", "https://api.prowler.com"),
@@ -316,6 +317,15 @@ ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES = env.int(
# Valid values: "neo4j" (default, OSS and local dev), "neptune" (hosted).
ATTACK_PATHS_SINK_DATABASE = env.str("ATTACK_PATHS_SINK_DATABASE", default="neo4j")
# Lighthouse AI
# Comma-separated hostnames (or IP literals) that bypass the SSRF validation
# applied to OpenAI-compatible provider base URLs, so self-hosted deployments
# can point Lighthouse AI at internal endpoints. Empty by default: every base
# URL must resolve to a public endpoint.
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS = env.list(
"LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS", default=[]
)
# Orphan task recovery feature flags. The master switch is OFF by default, so task
# recovery is opt-in; enable it with DJANGO_TASK_RECOVERY_ENABLED=true. The per-group
# toggles default to enabled, so once the master is on every group recovers unless a
+48 -25
View File
@@ -14,6 +14,7 @@ from prowler.lib.outputs.compliance.generic.generic import GenericCompliance
from prowler.lib.outputs.csv.csv import CSV
from prowler.lib.outputs.finding import Finding as FindingOutput
from prowler.lib.outputs.html.html import HTML
from prowler.lib.outputs.jira.exceptions.exceptions import JiraBaseException
from prowler.lib.outputs.ocsf.ocsf import OCSF
from prowler.providers.aws.aws_provider import AwsProvider
from prowler.providers.aws.lib.s3.s3 import S3
@@ -26,6 +27,8 @@ from tasks.utils import batched
logger = get_task_logger(__name__)
JIRA_GENERIC_SEND_ERROR = "Failed to create Jira issue."
def get_s3_client_from_integration(
integration: Integration,
@@ -483,6 +486,7 @@ def send_findings_to_jira(
jira_integration = initialize_prowler_integration(integration)
num_tickets_created = 0
error_messages = []
for finding_id in finding_ids:
with rls_transaction(tenant_id):
finding_instance = (
@@ -512,35 +516,54 @@ def send_findings_to_jira(
recommendation = remediation.get("recommendation", {})
remediation_code = remediation.get("code", {})
# Send the individual finding to Jira
result = jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=finding_instance.scan.provider.provider,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get("nativeiac", ""),
remediation_code_terraform=remediation_code.get("terraform", ""),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
)
try:
# Send the individual finding to Jira
result = jira_integration.send_finding(
check_id=finding_instance.check_id,
check_title=check_metadata.get("checktitle", ""),
severity=finding_instance.severity,
status=finding_instance.status,
status_extended=finding_instance.status_extended or "",
provider=finding_instance.scan.provider.provider,
region=region,
resource_uid=resource_uid,
resource_name=resource_name,
risk=check_metadata.get("risk", ""),
recommendation_text=recommendation.get("text", ""),
recommendation_url=recommendation.get("url", ""),
remediation_code_native_iac=remediation_code.get("nativeiac", ""),
remediation_code_terraform=remediation_code.get("terraform", ""),
remediation_code_cli=remediation_code.get("cli", ""),
remediation_code_other=remediation_code.get("other", ""),
resource_tags=resource_tags,
compliance=finding_instance.compliance or {},
project_key=project_key,
issue_type=issue_type,
)
except JiraBaseException as error:
error_message = error.message or JIRA_GENERIC_SEND_ERROR
logger.exception(
"Failed to send finding %s to Jira: %s", finding_id, error_message
)
error_messages.append(error_message)
continue
except Exception:
logger.exception("Failed to send finding %s to Jira", finding_id)
error_messages.append(JIRA_GENERIC_SEND_ERROR)
continue
if result:
num_tickets_created += 1
else:
logger.error(f"Failed to send finding {finding_id} to Jira")
error_message = JIRA_GENERIC_SEND_ERROR
logger.error(error_message)
error_messages.append(error_message)
return {
result = {
"created_count": num_tickets_created,
"failed_count": len(finding_ids) - num_tickets_created,
}
if error_messages:
result["error"] = "; ".join(dict.fromkeys(error_messages))
return result
@@ -1,6 +1,15 @@
import ssl
from collections.abc import Iterable
import boto3
import httpcore
import httpx
import openai
from api.models import LighthouseProviderConfiguration, LighthouseProviderModels
from api.validators import (
resolve_lighthouse_openai_compatible_host,
validate_lighthouse_openai_compatible_base_url,
)
from botocore import UNSIGNED
from botocore.config import Config
from botocore.exceptions import BotoCoreError, ClientError
@@ -43,6 +52,90 @@ EXCLUDED_OPENAI_MODEL_SUBSTRINGS = (
"-instruct", # Legacy instruct models (gpt-3.5-turbo-instruct, etc.)
)
OPENAI_COMPATIBLE_AUTHENTICATION_ERROR = "API key is invalid or missing"
OPENAI_COMPATIBLE_CONNECTION_ERROR = "Provider connection failed"
class _OpenAICompatibleProviderError(Exception):
"""Sanitized OpenAI-compatible provider error safe for task results."""
def _sanitize_openai_compatible_error(error: Exception) -> str:
status_code = getattr(error, "status_code", None)
if status_code is None:
response = getattr(error, "response", None)
status_code = getattr(response, "status_code", None)
if status_code == 401:
return OPENAI_COMPATIBLE_AUTHENTICATION_ERROR
return OPENAI_COMPATIBLE_CONNECTION_ERROR
class _LighthouseOpenAICompatibleNetworkBackend(httpcore.SyncBackend):
"""Validate and pin DNS results immediately before TCP connections."""
def connect_tcp(
self,
host: str,
port: int,
timeout: float | None = None,
local_address: str | None = None,
socket_options: Iterable[httpcore.SOCKET_OPTION] | None = None,
) -> httpcore.NetworkStream:
resolved_addresses = resolve_lighthouse_openai_compatible_host(host, port)
last_error: httpcore.ConnectError | httpcore.ConnectTimeout | None = None
for address in resolved_addresses:
try:
return super().connect_tcp(
address,
port,
timeout=timeout,
local_address=local_address,
socket_options=socket_options,
)
except (httpcore.ConnectError, httpcore.ConnectTimeout) as error:
last_error = error
if last_error:
raise last_error
raise httpcore.ConnectError("No resolved addresses are available")
class _LighthouseOpenAICompatibleHTTPTransport(httpx.HTTPTransport):
"""HTTP transport that connects only to validated public IP addresses."""
def __init__(self) -> None:
self._pool = httpcore.ConnectionPool(
ssl_context=ssl.create_default_context(),
network_backend=_LighthouseOpenAICompatibleNetworkBackend(),
)
def _create_openai_compatible_http_client() -> httpx.Client:
"""Create the restricted HTTP client used for OpenAI-compatible providers."""
return httpx.Client(
follow_redirects=False,
trust_env=False,
transport=_LighthouseOpenAICompatibleHTTPTransport(),
)
def _list_openai_compatible_models(base_url: str, api_key: str):
validate_lighthouse_openai_compatible_base_url(base_url)
try:
with _create_openai_compatible_http_client() as http_client:
client = openai.OpenAI(
api_key=api_key,
base_url=base_url,
http_client=http_client,
)
return client.models.list()
except Exception as error:
raise _OpenAICompatibleProviderError(
_sanitize_openai_compatible_error(error)
) from error
def _extract_error_message(e: Exception) -> str:
"""
@@ -114,6 +207,7 @@ def _extract_openai_compatible_params(
return None
if not isinstance(base_url, str) or not base_url:
return None
validate_lighthouse_openai_compatible_base_url(base_url, resolve_dns=False)
return {"base_url": base_url, "api_key": api_key}
@@ -285,13 +379,7 @@ def check_lighthouse_provider_connection(provider_config_id: str) -> dict:
"error": "Base URL or API key is invalid or missing",
}
# Test connection using OpenAI SDK with custom base_url
# Note: base_url should include version (e.g., https://openrouter.ai/api/v1)
client = openai.OpenAI(
api_key=params["api_key"],
base_url=params["base_url"],
)
_ = client.models.list()
_ = _list_openai_compatible_models(params["base_url"], params["api_key"])
else:
return {"connected": False, "error": "Unsupported provider type"}
@@ -361,8 +449,7 @@ def _fetch_openai_compatible_models(base_url: str, api_key: str) -> dict[str, st
Note: base_url should include version (e.g., https://openrouter.ai/api/v1)
"""
client = openai.OpenAI(api_key=api_key, base_url=base_url)
models = client.models.list()
models = _list_openai_compatible_models(base_url, api_key)
available_models: dict[str, str] = {}
for model in models.data:
+7 -19
View File
@@ -11,7 +11,6 @@ from uuid import UUID
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.db_utils import rls_transaction
from api.models import Provider, Scan, ScanSummary, StateChoices, ThreatScoreSnapshot
from api.utils import initialize_prowler_provider
from celery.utils.log import get_task_logger
from config.django.base import DJANGO_TMP_OUTPUT_DIRECTORY
from prowler.lib.check.compliance_models import (
@@ -27,6 +26,7 @@ from tasks.jobs.reports import (
ENSReportGenerator,
NIS2ReportGenerator,
ThreatScoreReportGenerator,
build_provider_metadata,
)
from tasks.jobs.threatscore import compute_threatscore_metrics
from tasks.jobs.threatscore_utils import (
@@ -841,24 +841,12 @@ def generate_compliance_reports(
tenant_id, scan_id
)
# Initialize the Prowler provider once for the whole report batch. Each
# generator used to re-init this in _load_compliance_data, paying the
# boto3/Azure-SDK construction cost 5 times per scan. The instance is
# only used by FindingOutput.transform_api_finding to enrich findings,
# so a single shared instance is correct.
logger.info("Initializing prowler_provider once for all reports (scan %s)", scan_id)
try:
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
prowler_provider = initialize_prowler_provider(provider_obj)
except Exception as init_error:
# If init fails the generators will fall back to lazy init in
# _load_compliance_data; we just log and continue.
logger.warning(
"Could not pre-initialize prowler_provider for scan %s: %s",
scan_id,
init_error,
)
prowler_provider = None
# Build a credential-free provider metadata stub once for the whole
# report batch. FindingOutput.transform_api_finding only reads static
# attributes (type plus a few identity fields), so reports never decrypt
# the ProviderSecret nor construct a cloud SDK session — generation keeps
# working after credentials are deleted or invalidated (PROWLER-2145).
prowler_provider = build_provider_metadata(provider_obj)
# Create shared findings cache up front so the eviction closure below
# can reference it. Defined BEFORE the closure to avoid the UnboundLocalError
@@ -98,6 +98,7 @@ from .config import (
from .csa import CSAReportGenerator
from .ens import ENSReportGenerator
from .nis2 import NIS2ReportGenerator
from .provider_metadata import build_provider_metadata
from .threatscore import ThreatScoreReportGenerator
__all__ = [
@@ -105,6 +106,7 @@ __all__ = [
"BaseComplianceReportGenerator",
"ComplianceData",
"RequirementData",
"build_provider_metadata",
"create_pdf_styles",
"get_requirement_metadata",
# Framework-specific generators
+11 -10
View File
@@ -11,7 +11,6 @@ from typing import Any
from api.db_router import READ_REPLICA_ALIAS
from api.db_utils import rls_transaction
from api.models import Provider, StatusChoices
from api.utils import initialize_prowler_provider
from celery.utils.log import get_task_logger
from prowler.lib.check.compliance_models import (
Compliance,
@@ -52,6 +51,7 @@ from .config import (
PADDING_SMALL,
FrameworkConfig,
)
from .provider_metadata import build_provider_metadata
logger = get_task_logger(__name__)
@@ -178,7 +178,8 @@ class ComplianceData:
attributes_by_requirement_id: Mapping of requirement IDs to their attributes
findings_by_check_id: Mapping of check IDs to their findings
provider_obj: Provider model object
prowler_provider: Initialized Prowler provider
prowler_provider: Credential-free provider metadata stub (see
``build_provider_metadata``)
"""
tenant_id: str
@@ -439,10 +440,10 @@ class BaseComplianceReportGenerator(ABC):
provider_obj: Optional pre-fetched Provider object
requirement_statistics: Optional pre-aggregated statistics
findings_cache: Optional pre-loaded findings cache
prowler_provider: Optional pre-initialized Prowler provider. When
generating multiple reports for the same scan the master
function initializes this once and passes it in to avoid
re-running boto3/Azure-SDK setup per framework.
prowler_provider: Optional provider metadata stub (see
``build_provider_metadata``). When generating multiple
reports for the same scan the master function builds it
once and passes it in.
**kwargs: Additional framework-specific arguments
"""
framework = self.config.display_name
@@ -896,9 +897,9 @@ class BaseComplianceReportGenerator(ABC):
provider_obj: Optional pre-fetched Provider
requirement_statistics: Optional pre-aggregated statistics
findings_cache: Optional pre-loaded findings
prowler_provider: Optional pre-initialized Prowler provider. When
the master function initializes it once and passes it in,
we skip the per-report ``initialize_prowler_provider`` call.
prowler_provider: Optional provider metadata stub. When the
master function builds it once and passes it in, we skip
the per-report ``build_provider_metadata`` call.
Returns:
Aggregated ComplianceData object
@@ -909,7 +910,7 @@ class BaseComplianceReportGenerator(ABC):
provider_obj = Provider.objects.get(id=provider_id)
if prowler_provider is None:
prowler_provider = initialize_prowler_provider(provider_obj)
prowler_provider = build_provider_metadata(provider_obj)
provider_type = provider_obj.provider
# Load compliance framework — fall back to the universal loader
@@ -0,0 +1,124 @@
from types import SimpleNamespace
from prowler.providers.github.models import GithubIdentityInfo
def build_provider_metadata(provider) -> SimpleNamespace:
"""Build a credential-free stand-in for the Prowler SDK provider.
``FindingOutput.transform_api_finding`` only reads static attributes
from the provider (``type`` plus a few identity/metadata fields used to
label accounts), so compliance reports never need the decrypted
``ProviderSecret`` nor a live cloud SDK session. This builds an object
exposing exactly those attributes from the ``Provider`` DB row, which
keeps report generation working when the provider secret has been
deleted or its credentials are no longer valid (PROWLER-2145).
Args:
provider: The API ``Provider`` model instance (only ``provider``,
``uid`` and ``alias`` are read).
Returns:
A ``SimpleNamespace`` mimicking the SDK provider attributes consumed
by ``FindingOutput.transform_api_finding`` / ``generate_output``.
"""
provider_type = provider.provider
uid = provider.uid
display_name = provider.alias or uid
# Defaults cover every attribute read unconditionally in
# FindingOutput.generate_output (``provider.auth_method`` is accessed
# directly for several provider types); identity lookups go through
# get_nested_attribute/getattr, which tolerate missing attributes.
stub = SimpleNamespace(
type=provider_type,
auth_method="",
identity=SimpleNamespace(),
)
if provider_type == "aws":
stub.identity = SimpleNamespace(account=uid)
elif provider_type == "azure":
stub.identity = SimpleNamespace(
identity_type="",
identity_id="",
tenant_ids=[""],
tenant_domain="",
subscriptions={uid: display_name},
)
elif provider_type == "gcp":
stub.identity = SimpleNamespace(profile="")
stub.projects = {
uid: SimpleNamespace(
id=uid,
name=display_name,
labels={},
organization=None,
)
}
elif provider_type == "kubernetes":
stub.identity = SimpleNamespace(context=uid, cluster=uid)
elif provider_type == "m365":
stub.identity = SimpleNamespace(
identity_type="",
identity_id="",
tenant_domain=uid,
tenant_id="",
)
elif provider_type == "github":
# generate_output assigns account fields only inside
# isinstance(identity, Github*IdentityInfo) branches, so the stub
# must carry a real GithubIdentityInfo instance.
stub.identity = GithubIdentityInfo(
account_id=uid,
account_name=display_name,
account_url="",
)
elif provider_type == "mongodbatlas":
stub.identity = SimpleNamespace(
organization_id=uid,
organization_name=display_name,
)
elif provider_type == "iac":
stub.provider_uid = uid
elif provider_type == "oraclecloud":
stub.identity = SimpleNamespace(
tenancy_id=uid,
tenancy_name=display_name,
)
elif provider_type == "alibabacloud":
stub.identity = SimpleNamespace(
identity_arn="",
account_id=uid,
account_name=display_name,
)
elif provider_type == "cloudflare":
stub.identity = SimpleNamespace(
audited_accounts=[uid],
accounts=[],
)
elif provider_type == "openstack":
stub.identity = SimpleNamespace(
username="",
project_id=uid,
project_name=display_name,
)
elif provider_type == "googleworkspace":
stub.identity = SimpleNamespace(
delegated_user="",
customer_id=uid,
domain=display_name,
)
elif provider_type == "vercel":
stub.identity = SimpleNamespace(
team=None,
user_id=uid,
username=display_name,
)
elif provider_type == "okta":
stub.identity = SimpleNamespace(
org_domain=uid,
client_id="",
)
return stub
+55 -10
View File
@@ -1,6 +1,7 @@
import csv
import io
import json
import random
import re
import time
import uuid
@@ -306,6 +307,55 @@ def _store_resources(
return resource_instance, (resource_instance.uid, resource_instance.region)
def _bulk_update_resource_failed_findings_counts(
tenant_id: str,
scan_id: str,
resources_to_update: list[Resource],
) -> None:
"""Persist failed finding counters with stable row locking and retry."""
if not resources_to_update:
return
sorted_resources = sorted(
resources_to_update, key=lambda resource: str(resource.id)
)
for start in range(0, len(sorted_resources), SCAN_DB_BATCH_SIZE):
chunk = sorted_resources[start : start + SCAN_DB_BATCH_SIZE]
chunk_ids = [resource.id for resource in chunk]
for attempt in range(CELERY_DEADLOCK_ATTEMPTS):
try:
with rls_transaction(tenant_id):
list(
Resource.objects.select_for_update()
.filter(id__in=chunk_ids)
.order_by("id")
.values_list("id", flat=True)
)
Resource.objects.bulk_update(
chunk,
["failed_findings_count"],
batch_size=SCAN_DB_BATCH_SIZE,
)
break
except OperationalError:
if attempt < CELERY_DEADLOCK_ATTEMPTS - 1:
logger.warning(
"Resource failed findings count update hit a database "
"conflict on scan %s. Retrying chunk %s/%s "
"(attempt %s/%s).",
scan_id,
start // SCAN_DB_BATCH_SIZE + 1,
(len(sorted_resources) + SCAN_DB_BATCH_SIZE - 1)
// SCAN_DB_BATCH_SIZE,
attempt + 1,
CELERY_DEADLOCK_ATTEMPTS,
)
time.sleep((0.1 * (2**attempt)) + random.uniform(0, 0.1))
continue
raise
def _copy_compliance_requirement_rows(
tenant_id: str, rows: list[dict[str, Any]]
) -> None:
@@ -1182,16 +1232,11 @@ def perform_prowler_scan(
resources_to_update.append(resource_instance)
if resources_to_update:
# Single rls_transaction wrapping the bulk_update (previously
# `update_objects_in_batches` opened one rls_transaction per
# chunk; for tenants with many resources this collapsed N
# BEGINs/COMMITs into 1).
with rls_transaction(tenant_id):
Resource.objects.bulk_update(
resources_to_update,
["failed_findings_count"],
batch_size=SCAN_DB_BATCH_SIZE,
)
_bulk_update_resource_failed_findings_counts(
tenant_id=tenant_id,
scan_id=scan_id,
resources_to_update=resources_to_update,
)
except ProviderDeletedException as e:
logger.warning(str(e))
@@ -178,7 +178,9 @@ def _load_findings_for_requirement_checks(
tenant_id (str): The tenant ID for Row-Level Security context.
scan_id (str): The ID of the scan to retrieve findings for.
check_ids (list[str]): List of check IDs to load findings for.
prowler_provider: The initialized Prowler provider instance.
prowler_provider: Credential-free provider metadata stub (see
``tasks.jobs.reports.build_provider_metadata``) consumed by
``FindingOutput.transform_api_finding``.
findings_cache (dict, optional): Cache of already loaded findings.
If provided, checks are first looked up in cache before querying database.
total_counts_out (dict, optional): If provided, populated with
+269 -46
View File
@@ -2,6 +2,7 @@ import os
from datetime import UTC, datetime, timedelta
from pathlib import Path
from shutil import rmtree
from uuid import uuid4
from api.compliance import (
get_compliance_frameworks,
@@ -10,14 +11,24 @@ from api.compliance import (
from api.db_router import READ_REPLICA_ALIAS
from api.db_utils import delete_related_daily_task, rls_transaction
from api.decorators import handle_provider_deletion, set_tenant
from api.models import Finding, Integration, Provider, Scan, ScanSummary, StateChoices
from api.models import (
Finding,
Integration,
Provider,
Scan,
ScanSummary,
StateChoices,
Task,
)
from api.utils import initialize_prowler_provider
from api.v1.serializers import ScanTaskSerializer
from celery import chain, group, shared_task
from celery import chain, group, shared_task, states
from celery.utils.log import get_task_logger
from config.celery import RLSTask
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE, DJANGO_TMP_OUTPUT_DIRECTORY
from django.db import transaction
from django_celery_beat.models import PeriodicTask
from django_celery_results.models import TaskResult
from prowler.lib.check.compliance_models import Compliance
from prowler.lib.outputs.compliance.compliance import (
process_universal_compliance_frameworks,
@@ -85,6 +96,220 @@ from tasks.utils import (
)
logger = get_task_logger(__name__)
QUEUED_SCAN_TASK_STATE = "QUEUED"
DISPATCHED_SCAN_TASK_STATES = (states.PENDING, states.STARTED, "PROGRESS")
def _get_dispatched_provider_scan(tenant_id: str, provider_id: str):
"""Return a scan that has already been dispatched for a provider."""
executing_scan = (
Scan.objects.select_for_update()
.filter(
tenant_id=tenant_id,
provider_id=provider_id,
state=StateChoices.EXECUTING,
)
.order_by("-inserted_at")
.first()
)
if executing_scan:
return executing_scan
return (
Scan.objects.select_for_update(of=("self",))
.select_related("task__task_runner_task")
.filter(
tenant_id=tenant_id,
provider_id=provider_id,
state__in=(StateChoices.AVAILABLE, StateChoices.SCHEDULED),
task__isnull=False,
task__task_runner_task__status__in=DISPATCHED_SCAN_TASK_STATES,
)
.order_by("-inserted_at")
.first()
)
def _get_queued_provider_scan(tenant_id: str, provider_id: str):
"""Return the next DB-queued scan for a provider."""
return (
Scan.objects.select_for_update(of=("self",))
.select_related("task__task_runner_task")
.filter(
tenant_id=tenant_id,
provider_id=provider_id,
state=StateChoices.AVAILABLE,
task__isnull=False,
task__task_runner_task__status=QUEUED_SCAN_TASK_STATE,
)
.order_by("inserted_at", "id")
.first()
)
def get_active_provider_scan(tenant_id: str, provider_id: str):
"""Return a dispatched or DB-queued scan for a provider."""
return _get_dispatched_provider_scan(
tenant_id, provider_id
) or _get_queued_provider_scan(tenant_id, provider_id)
def create_scan_task_record(
tenant_id: str,
task_id: str,
task_name: str = "scan-perform",
task_status: str | None = states.PENDING,
) -> Task:
if task_status is None:
task_status = states.PENDING
task_result, _ = TaskResult.objects.update_or_create(
task_id=str(task_id),
defaults={"status": task_status, "task_name": task_name},
)
prowler_task, _ = Task.objects.update_or_create(
id=str(task_id),
tenant_id=tenant_id,
defaults={"task_runner_task": task_result},
)
return prowler_task
def enqueue_scan_execution_on_commit(
tenant_id: str,
scan: Scan,
task_id: str,
) -> None:
transaction.on_commit(
lambda: perform_scan_task.apply_async(
kwargs={
"tenant_id": str(tenant_id),
"scan_id": str(scan.id),
"provider_id": str(scan.provider_id),
},
task_id=str(task_id),
)
)
def _get_queued_scheduled_scan(tenant_id: str, provider_id: str):
return (
Scan.objects.select_for_update(of=("self",))
.select_related("task__task_runner_task")
.filter(
tenant_id=tenant_id,
provider_id=provider_id,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
task__isnull=False,
task__task_runner_task__status=QUEUED_SCAN_TASK_STATE,
)
.order_by("inserted_at", "id")
.first()
)
def _get_or_create_queued_scheduled_scan(
tenant_id: str,
provider_id: str,
periodic_task_instance: PeriodicTask,
scheduled_at: datetime,
) -> Scan:
queued_scan = _get_queued_scheduled_scan(tenant_id, provider_id)
if queued_scan:
return queued_scan
task_id = str(uuid4())
queued_task = create_scan_task_record(
tenant_id=tenant_id,
task_id=task_id,
task_status=QUEUED_SCAN_TASK_STATE,
)
return Scan.objects.create(
tenant_id=tenant_id,
name="Daily scheduled scan",
provider_id=provider_id,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
scheduled_at=scheduled_at,
scheduler_task_id=periodic_task_instance.id,
task=queued_task,
)
def _dispatch_next_queued_provider_scan(tenant_id: str, provider_id: str):
with rls_transaction(tenant_id):
if not Provider.objects.select_for_update().filter(pk=provider_id).exists():
return None
if _get_dispatched_provider_scan(tenant_id, provider_id):
return None
queued_scan = _get_queued_provider_scan(tenant_id, provider_id)
if not queued_scan or not queued_scan.task:
return None
task_result = queued_scan.task.task_runner_task
task_result.status = states.PENDING
task_result.task_name = "scan-perform"
task_result.save(update_fields=["status", "task_name"])
enqueue_scan_execution_on_commit(
tenant_id=tenant_id,
scan=queued_scan,
task_id=str(queued_scan.task_id),
)
return queued_scan
def _dispatch_next_queued_provider_scan_best_effort(
tenant_id: str, provider_id: str
) -> None:
try:
_dispatch_next_queued_provider_scan(tenant_id, provider_id)
except Exception:
logger.exception(
"Failed to dispatch next queued scan for provider %s", provider_id
)
def _get_or_create_next_scheduled_scan(
tenant_id: str,
provider_id: str,
periodic_task_instance: PeriodicTask,
next_scan_datetime: datetime,
) -> Scan:
interval = periodic_task_instance.interval
now = datetime.now(UTC)
while next_scan_datetime <= now:
next_scan_datetime += timedelta(**{interval.period: interval.every})
return _get_or_create_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
scheduler_task_id=periodic_task_instance.id,
scheduled_at=next_scan_datetime,
update_state=True,
)
def _ensure_next_scheduled_scan_best_effort(
tenant_id: str,
provider_id: str,
periodic_task_instance: PeriodicTask,
next_scan_datetime: datetime,
) -> None:
try:
with rls_transaction(tenant_id):
_get_or_create_next_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
periodic_task_instance=periodic_task_instance,
next_scan_datetime=next_scan_datetime,
)
except Exception:
logger.exception(
"Failed to ensure next scheduled scan for provider %s", provider_id
)
def _cleanup_orphan_scheduled_scans(
@@ -117,6 +342,7 @@ def _cleanup_orphan_scheduled_scans(
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
scheduler_task_id=scheduler_task_id,
task__isnull=True,
)
scheduled_scan_exists = Scan.objects.filter(
@@ -292,16 +518,17 @@ def perform_scan_task(
)
return None
result = perform_prowler_scan(
tenant_id=tenant_id,
scan_id=scan_id,
provider_id=provider_id,
checks_to_execute=checks_to_execute,
)
_perform_scan_complete_tasks(tenant_id, scan_id, provider_id)
return result
try:
result = perform_prowler_scan(
tenant_id=tenant_id,
scan_id=scan_id,
provider_id=provider_id,
checks_to_execute=checks_to_execute,
)
_perform_scan_complete_tasks(tenant_id, scan_id, provider_id)
return result
finally:
_dispatch_next_queued_provider_scan_best_effort(tenant_id, provider_id)
# acks_late=False: like scan-perform; a dropped run is re-fired by Beat on the next tick.
@@ -335,7 +562,7 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
task_id = self.request.id
with rls_transaction(tenant_id):
if not Provider.objects.filter(pk=provider_id).exists():
if not Provider.objects.select_for_update().filter(pk=provider_id).exists():
logger.warning(
"scheduled scan-perform skipped: provider %s no longer exists "
"(tenant=%s)",
@@ -348,22 +575,6 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
periodic_task_instance = PeriodicTask.objects.get(
name=f"scan-perform-scheduled-{provider_id}"
)
executing_scan = (
Scan.objects.filter(
tenant_id=tenant_id,
provider_id=provider_id,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.EXECUTING,
)
.order_by("-started_at")
.first()
)
if executing_scan:
logger.warning(
f"Scheduled scan already executing for provider {provider_id}. Skipping."
)
return ScanTaskSerializer(instance=executing_scan).data
executed_scan = Scan.objects.filter(
tenant_id=tenant_id,
provider_id=provider_id,
@@ -388,6 +599,26 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
scheduler_task_id=periodic_task_instance.id,
)
active_scan = get_active_provider_scan(tenant_id, provider_id)
if active_scan:
logger.warning(
"Scan already queued or executing for provider %s. Queueing scheduled run.",
provider_id,
)
queued_scheduled_scan = _get_or_create_queued_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
periodic_task_instance=periodic_task_instance,
scheduled_at=current_scan_datetime,
)
_get_or_create_next_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
periodic_task_instance=periodic_task_instance,
next_scan_datetime=next_scan_datetime,
)
return ScanTaskSerializer(instance=queued_scheduled_scan).data
scan_instance = _get_or_create_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
@@ -403,24 +634,16 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
scan_id=str(scan_instance.id),
provider_id=provider_id,
)
_perform_scan_complete_tasks(tenant_id, str(scan_instance.id), provider_id)
return result
finally:
with rls_transaction(tenant_id):
now = datetime.now(UTC)
if next_scan_datetime <= now:
interval_delta = timedelta(**{interval.period: interval.every})
while next_scan_datetime <= now:
next_scan_datetime += interval_delta
_get_or_create_scheduled_scan(
tenant_id=tenant_id,
provider_id=provider_id,
scheduler_task_id=periodic_task_instance.id,
scheduled_at=next_scan_datetime,
update_state=True,
)
_perform_scan_complete_tasks(tenant_id, str(scan_instance.id), provider_id)
return result
_ensure_next_scheduled_scan_best_effort(
tenant_id=tenant_id,
provider_id=provider_id,
periodic_task_instance=periodic_task_instance,
next_scan_datetime=next_scan_datetime,
)
_dispatch_next_queued_provider_scan_best_effort(tenant_id, provider_id)
@shared_task(name="scan-summary", queue="overview")
@@ -5,6 +5,10 @@ from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.models import Integration
from api.utils import prowler_integration_connection_test
from django.db import OperationalError
from prowler.lib.outputs.jira.exceptions.exceptions import (
JiraRefreshTokenError,
JiraRequiredCustomFieldsError,
)
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHubConnection
from prowler.providers.common.models import Connection
from tasks.jobs.integrations import (
@@ -1830,10 +1834,213 @@ class TestJiraIntegration:
)
# Assertions
assert result == {"created_count": 2, "failed_count": 1}
assert result == {
"created_count": 2,
"failed_count": 1,
"error": "Failed to create Jira issue.",
}
# Verify error was logged for the failed finding
mock_logger.error.assert_called_with("Failed to send finding finding-2 to Jira")
mock_logger.error.assert_called_with("Failed to create Jira issue.")
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.logger")
def test_send_findings_to_jira_preserves_exception_message(
self,
mock_logger,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test Jira send exceptions are returned for UI polling."""
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
issue_type = "Task"
finding_ids = ["finding-1"]
error_message = "Jira project requires custom fields: Team is required"
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
integration = MagicMock()
mock_integration_model.objects.get.return_value = integration
mock_jira_integration = MagicMock()
mock_jira_integration.send_finding.side_effect = JiraRequiredCustomFieldsError(
message=error_message
)
mock_initialize_integration.return_value = mock_jira_integration
finding = MagicMock()
finding.id = "finding-1"
finding.check_id = "check_001"
finding.severity = "high"
finding.status = "FAIL"
finding.status_extended = "Resource is not compliant"
finding.compliance = {}
finding.resources.exists.return_value = False
finding.resources.first.return_value = None
finding.scan.provider.provider = "aws"
finding.check_metadata = {
"checktitle": "Check Title",
"risk": "High risk",
"remediation": {"recommendation": {}, "code": {}},
}
mock_select_related = mock_finding_model.all_objects.select_related.return_value
mock_finding_query = mock_select_related.prefetch_related.return_value
mock_finding_query.get.return_value = finding
result = send_findings_to_jira(
tenant_id, integration_id, project_key, issue_type, finding_ids
)
assert result == {
"created_count": 0,
"failed_count": 1,
"error": error_message,
}
mock_logger.exception.assert_called_with(
"Failed to send finding %s to Jira: %s",
"finding-1",
error_message,
)
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.logger")
def test_send_findings_to_jira_preserves_refresh_token_error_message(
self,
mock_logger,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test Jira refresh token exceptions return their UI-friendly message."""
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
issue_type = "Task"
finding_ids = ["finding-1"]
error_message = "Failed to refresh the access token"
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
integration = MagicMock()
mock_integration_model.objects.get.return_value = integration
mock_jira_integration = MagicMock()
mock_jira_integration.send_finding.side_effect = JiraRefreshTokenError(
message=error_message
)
mock_initialize_integration.return_value = mock_jira_integration
finding = MagicMock()
finding.id = "finding-1"
finding.check_id = "check_001"
finding.severity = "high"
finding.status = "FAIL"
finding.status_extended = "Resource is not compliant"
finding.compliance = {}
finding.resources.exists.return_value = False
finding.resources.first.return_value = None
finding.scan.provider.provider = "aws"
finding.check_metadata = {
"checktitle": "Check Title",
"risk": "High risk",
"remediation": {"recommendation": {}, "code": {}},
}
mock_select_related = mock_finding_model.all_objects.select_related.return_value
mock_finding_query = mock_select_related.prefetch_related.return_value
mock_finding_query.get.return_value = finding
result = send_findings_to_jira(
tenant_id, integration_id, project_key, issue_type, finding_ids
)
assert result == {
"created_count": 0,
"failed_count": 1,
"error": error_message,
}
mock_logger.exception.assert_called_with(
"Failed to send finding %s to Jira: %s",
"finding-1",
error_message,
)
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.initialize_prowler_integration")
@patch("tasks.jobs.integrations.logger")
def test_send_findings_to_jira_sanitizes_unexpected_exception_message(
self,
mock_logger,
mock_initialize_integration,
mock_integration_model,
mock_finding_model,
mock_rls_transaction,
):
"""Test unexpected Jira send exceptions do not leak raw details to UI."""
tenant_id = "tenant-123"
integration_id = "integration-456"
project_key = "PROJ"
issue_type = "Task"
finding_ids = ["finding-1"]
mock_rls_transaction.return_value.__enter__ = MagicMock()
mock_rls_transaction.return_value.__exit__ = MagicMock()
integration = MagicMock()
mock_integration_model.objects.get.return_value = integration
mock_jira_integration = MagicMock()
mock_jira_integration.send_finding.side_effect = Exception("token=secret-value")
mock_initialize_integration.return_value = mock_jira_integration
finding = MagicMock()
finding.id = "finding-1"
finding.check_id = "check_001"
finding.severity = "high"
finding.status = "FAIL"
finding.status_extended = "Resource is not compliant"
finding.compliance = {}
finding.resources.exists.return_value = False
finding.resources.first.return_value = None
finding.scan.provider.provider = "aws"
finding.check_metadata = {
"checktitle": "Check Title",
"risk": "High risk",
"remediation": {"recommendation": {}, "code": {}},
}
mock_select_related = mock_finding_model.all_objects.select_related.return_value
mock_finding_query = mock_select_related.prefetch_related.return_value
mock_finding_query.get.return_value = finding
result = send_findings_to_jira(
tenant_id, integration_id, project_key, issue_type, finding_ids
)
assert result == {
"created_count": 0,
"failed_count": 1,
"error": "Failed to create Jira issue.",
}
assert "secret-value" not in result["error"]
mock_logger.exception.assert_called_with(
"Failed to send finding %s to Jira", "finding-1"
)
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Finding")
+14 -17
View File
@@ -1044,10 +1044,10 @@ class TestStaleCleanupProtectionHelpers:
class TestGenerateThreatscoreReportFunction:
"""Test suite for generate_threatscore_report function."""
@patch("tasks.jobs.reports.base.initialize_prowler_provider")
@patch("tasks.jobs.reports.base.build_provider_metadata")
def test_generate_threatscore_report_exception_handling(
self,
mock_initialize_provider,
mock_build_provider_metadata,
tenants_fixture,
scans_fixture,
providers_fixture,
@@ -1057,7 +1057,7 @@ class TestGenerateThreatscoreReportFunction:
scan = scans_fixture[0]
provider = providers_fixture[0]
mock_initialize_provider.side_effect = Exception("Test exception")
mock_build_provider_metadata.side_effect = Exception("Test exception")
with pytest.raises(Exception) as exc_info:
generate_threatscore_report(
@@ -1167,7 +1167,6 @@ class TestGenerateComplianceReportsOptimized:
assert result["cis"] == {"upload": False, "path": ""}
mock_cis.assert_not_called()
@patch("api.utils.initialize_prowler_provider")
@patch("tasks.jobs.report.rmtree")
@patch("tasks.jobs.report._upload_to_s3")
@patch("tasks.jobs.report.generate_cis_report")
@@ -1194,7 +1193,6 @@ class TestGenerateComplianceReportsOptimized:
mock_cis,
mock_upload_to_s3,
mock_rmtree,
mock_init_provider,
):
"""After each framework finishes, exclusive entries are evicted.
@@ -1223,7 +1221,6 @@ class TestGenerateComplianceReportsOptimized:
mock_aggregate_stats.return_value = {}
mock_generate_output_dir.return_value = "/tmp/tenant/scan/x/prowler-out"
mock_upload_to_s3.return_value = "s3://bucket/tenant/scan/x/report.pdf"
mock_init_provider.return_value = Mock(name="prowler_provider")
# Seed the cache as if both frameworks had already loaded their
# findings. We mutate it indirectly: each generator wrapper is a
@@ -1266,7 +1263,7 @@ class TestGenerateComplianceReportsOptimized:
"shared must remain in cache because ENS still needs it"
)
@patch("tasks.jobs.report.initialize_prowler_provider")
@patch("tasks.jobs.report.build_provider_metadata")
@patch("tasks.jobs.report.rmtree")
@patch("tasks.jobs.report._upload_to_s3")
@patch("tasks.jobs.report.generate_cis_report")
@@ -1279,7 +1276,7 @@ class TestGenerateComplianceReportsOptimized:
@patch("tasks.jobs.report.Compliance.get_bulk")
@patch("tasks.jobs.report.Provider.objects.get")
@patch("tasks.jobs.report.ScanSummary.objects.filter")
def test_prowler_provider_initialized_once(
def test_provider_metadata_built_once(
self,
mock_scan_summary_filter,
mock_provider_get,
@@ -1293,11 +1290,11 @@ class TestGenerateComplianceReportsOptimized:
mock_cis,
mock_upload_to_s3,
mock_rmtree,
mock_init_provider,
mock_build_metadata,
):
"""``initialize_prowler_provider`` must be called exactly once for
the whole batch (PROWLER-1733). Previously each generator re-init'd
the SDK provider in ``_load_compliance_data`` 5 inits per scan.
"""``build_provider_metadata`` must be called exactly once for the
whole batch and its result shared across all 5 reports
(PROWLER-1733 / PROWLER-2145).
"""
mock_scan_summary_filter.return_value.exists.return_value = True
mock_provider_get.return_value = Mock(uid="provider-uid", provider="aws")
@@ -1306,7 +1303,7 @@ class TestGenerateComplianceReportsOptimized:
mock_aggregate_stats.return_value = {}
mock_generate_output_dir.return_value = "/tmp/tenant/scan/x/prowler-out"
mock_upload_to_s3.return_value = "s3://bucket/tenant/scan/x/report.pdf"
mock_init_provider.return_value = Mock(name="prowler_provider")
mock_build_metadata.return_value = Mock(name="prowler_provider")
generate_compliance_reports(
tenant_id=str(uuid.uuid4()),
@@ -1325,14 +1322,14 @@ class TestGenerateComplianceReportsOptimized:
mock_nis2.assert_called_once()
mock_csa.assert_called_once()
mock_cis.assert_called_once()
# …but the SDK provider was initialized only once.
assert mock_init_provider.call_count == 1, (
f"expected 1 init, got {mock_init_provider.call_count} "
# …but the provider metadata stub was built only once.
assert mock_build_metadata.call_count == 1, (
f"expected 1 build, got {mock_build_metadata.call_count} "
f"(prowler_provider must be shared across reports)"
)
# The shared instance must reach every wrapper as kwargs.
shared = mock_init_provider.return_value
shared = mock_build_metadata.return_value
for mock_wrapper in (
mock_threatscore,
mock_ens,
@@ -0,0 +1,193 @@
"""Tests for the credential-free provider metadata stub (PROWLER-2145).
Every provider object used here is a plain ``SimpleNamespace`` WITHOUT a
``secret`` attribute: any code path trying to read ``provider.secret`` (the
coupling these tests guard against) would raise ``AttributeError`` and fail
the test. No database is required.
"""
from types import SimpleNamespace
import pytest
from api.models import Provider
from prowler.lib.outputs.finding import Finding as FindingOutput
from prowler.providers.github.models import GithubIdentityInfo
from tasks.jobs.reports import build_provider_metadata
PROVIDER_UID = "provider-uid-123"
PROVIDER_ALIAS = "my-provider-alias"
def _provider_row(provider_type: str, alias: str | None = PROVIDER_ALIAS):
"""Mimic the Provider DB row attributes read by build_provider_metadata."""
return SimpleNamespace(provider=provider_type, uid=PROVIDER_UID, alias=alias)
class TestBuildProviderMetadata:
@pytest.mark.parametrize("provider_type", Provider.ProviderChoices.values)
def test_every_provider_type_gets_safe_defaults(self, provider_type):
stub = build_provider_metadata(_provider_row(provider_type))
assert stub.type == provider_type
assert isinstance(stub.auth_method, str)
assert hasattr(stub, "identity")
def test_aws_identity_account_is_uid(self):
stub = build_provider_metadata(_provider_row("aws"))
assert stub.identity.account == PROVIDER_UID
def test_azure_identity_covers_generate_output_accesses(self):
stub = build_provider_metadata(_provider_row("azure"))
# generate_output indexes tenant_ids[0] and reads these directly.
assert stub.identity.tenant_ids
assert stub.identity.identity_type == ""
assert stub.identity.identity_id == ""
assert stub.identity.subscriptions == {PROVIDER_UID: PROVIDER_ALIAS}
def test_gcp_projects_keyed_by_uid(self):
stub = build_provider_metadata(_provider_row("gcp"))
project = stub.projects[PROVIDER_UID]
assert project.id == PROVIDER_UID
assert project.name == PROVIDER_ALIAS
assert project.labels == {}
# generate_output calls getattr(project, "organization") without a
# default, so the attribute must exist (None skips the org branch).
assert project.organization is None
def test_kubernetes_identity_context_and_cluster(self):
stub = build_provider_metadata(_provider_row("kubernetes"))
assert stub.identity.context == PROVIDER_UID
assert stub.identity.cluster == PROVIDER_UID
def test_github_identity_is_real_identity_info(self):
# generate_output only assigns account fields inside
# isinstance(identity, Github*IdentityInfo) branches.
stub = build_provider_metadata(_provider_row("github"))
assert isinstance(stub.identity, GithubIdentityInfo)
assert stub.identity.account_id == PROVIDER_UID
assert stub.identity.account_name == PROVIDER_ALIAS
def test_iac_provider_uid(self):
stub = build_provider_metadata(_provider_row("iac"))
assert stub.provider_uid == PROVIDER_UID
def test_alias_falls_back_to_uid(self):
stub = build_provider_metadata(_provider_row("azure", alias=None))
assert stub.identity.subscriptions == {PROVIDER_UID: PROVIDER_UID}
def _check_metadata_dict(provider_type: str, check_id: str) -> dict:
return {
"provider": provider_type,
"checkid": check_id,
"checktitle": "Test check title",
"checktype": [],
# CheckMetadata validates ServiceName == check_id.split("_")[0]
"servicename": check_id.split("_")[0],
"subservicename": "",
"severity": "high",
"resourcetype": "resource-type",
"description": "",
"risk": "",
"relatedurl": "",
"remediation": {
"recommendation": {"text": "", "url": ""},
"code": {"nativeiac": "", "terraform": "", "cli": "", "other": ""},
},
"resourceidtemplate": "",
"categories": [],
"dependson": [],
"relatedto": [],
"notes": "",
}
class _FakeFinding:
"""Attribute-faithful Finding stand-in.
A plain object instead of ``Mock``: only the attributes the Django model
exposes exist, so any new provider-attribute read in generate_output
(e.g. cloudflare's ``getattr(finding, "account_id", ...)``) hits the
same missing-attribute path it would hit in production instead of being
masked by Mock auto-created attributes.
"""
def _finding_model(provider_type: str, check_id: str, region: str):
"""Mimic the Django Finding row attributes read by transform_api_finding."""
resource = SimpleNamespace(
uid="resource-uid",
name="resource-name",
metadata="{}",
details="",
region=region,
tags=SimpleNamespace(all=lambda: []),
)
finding = _FakeFinding()
finding.resources = SimpleNamespace(first=lambda: resource)
finding.check_metadata = _check_metadata_dict(provider_type, check_id)
finding.status = "FAIL"
finding.status_extended = "failed for testing"
finding.muted = False
return finding
_FINDING_REGION = "region-x"
# Expected (account_uid, region) of the transformed finding per provider
# type, with resource.region = _FINDING_REGION. Keyed by every
# Provider.ProviderChoices value so that adding a new provider type without
# extending build_provider_metadata (and this table) fails the test below
# instead of breaking PDF generation at runtime.
_EXPECTED_TRANSFORM = {
"aws": (PROVIDER_UID, _FINDING_REGION),
"azure": (PROVIDER_UID, _FINDING_REGION),
"gcp": (PROVIDER_UID, _FINDING_REGION),
# transform_api_finding strips the "namespace: " prefix and
# generate_output re-adds it.
"kubernetes": (PROVIDER_UID, f"namespace: {_FINDING_REGION}"),
"m365": (PROVIDER_UID, _FINDING_REGION),
# For GitHub the owner comes from resource.region.
"github": (_FINDING_REGION, _FINDING_REGION),
"mongodbatlas": (PROVIDER_UID, _FINDING_REGION),
"iac": (PROVIDER_UID, _FINDING_REGION),
"oraclecloud": (PROVIDER_UID, _FINDING_REGION),
"alibabacloud": (PROVIDER_UID, _FINDING_REGION),
# Cloudflare uses the zone name (falls back to resource.name) as region.
"cloudflare": (PROVIDER_UID, "resource-name"),
"openstack": (PROVIDER_UID, _FINDING_REGION),
"image": ("image", _FINDING_REGION),
"googleworkspace": (PROVIDER_UID, _FINDING_REGION),
"vercel": (PROVIDER_UID, "global"),
"okta": (PROVIDER_UID, "global"),
}
class TestTransformApiFindingWithMetadataStub:
"""transform_api_finding must work end-to-end with the stub — i.e.
without a credentialed SDK provider for EVERY API provider type."""
@pytest.mark.parametrize("provider_type", Provider.ProviderChoices.values)
def test_transform_with_stub(self, provider_type):
assert provider_type in _EXPECTED_TRANSFORM, (
f"New provider type {provider_type!r}: add a branch to "
f"build_provider_metadata covering the attributes read by "
f"FindingOutput.generate_output, then add its expected "
f"(account_uid, region) here."
)
expected_account_uid, expected_region = _EXPECTED_TRANSFORM[provider_type]
stub = build_provider_metadata(_provider_row(provider_type))
check_id = f"{provider_type}_test_check"
finding_model = _finding_model(provider_type, check_id, _FINDING_REGION)
output = FindingOutput.transform_api_finding(finding_model, stub)
assert output.check_id == check_id
assert output.status == "FAIL"
assert output.account_uid == expected_account_uid
assert output.region == expected_region
assert output.resource_name
assert output.resource_uid
+94
View File
@@ -21,11 +21,13 @@ from api.models import (
StateChoices,
StatusChoices,
)
from django.db import IntegrityError, OperationalError
from prowler.lib.check.models import Severity
from prowler.lib.outputs.finding import Status
from tasks.jobs.scan import (
_ATTACK_SURFACE_MAPPING_CACHE,
_aggregate_findings_by_region,
_bulk_update_resource_failed_findings_counts,
_copy_compliance_requirement_rows,
_create_compliance_summaries,
_create_finding_delta,
@@ -858,6 +860,98 @@ class TestPerformScan:
# Assert that failed_findings_count was reset to 0 during the scan
assert resource.failed_findings_count == 0
def test_failed_findings_count_update_retries_deadlock_in_stable_order(
self, resources_fixture, monkeypatch
):
resource1, resource2, _ = resources_fixture
tenant_id = str(resource1.tenant_id)
resource1.failed_findings_count = 2
resource2.failed_findings_count = 3
resources_to_update = [resource2, resource1]
expected_order = [
str(resource.id)
for resource in sorted(resources_to_update, key=lambda item: str(item.id))
]
original_bulk_update = Resource.objects.bulk_update
bulk_update_calls = []
def flaky_bulk_update(objects, fields, batch_size=None):
bulk_update_calls.append([str(obj.id) for obj in objects])
if len(bulk_update_calls) == 1:
raise OperationalError("deadlock detected")
return original_bulk_update(objects, fields, batch_size=batch_size)
monkeypatch.setattr("tasks.jobs.scan.SCAN_DB_BATCH_SIZE", 10)
monkeypatch.setattr(Resource.objects, "bulk_update", flaky_bulk_update)
_bulk_update_resource_failed_findings_counts(
tenant_id=tenant_id,
scan_id="scan-id",
resources_to_update=resources_to_update,
)
resource1.refresh_from_db()
resource2.refresh_from_db()
assert resource1.failed_findings_count == 2
assert resource2.failed_findings_count == 3
assert bulk_update_calls == [expected_order, expected_order]
def test_failed_findings_count_update_does_not_retry_integrity_error(
self, resources_fixture, monkeypatch
):
resource, *_ = resources_fixture
resource.failed_findings_count = 2
bulk_update_calls = []
sleep_calls = []
def failing_bulk_update(objects, fields, batch_size=None):
bulk_update_calls.append([str(obj.id) for obj in objects])
raise IntegrityError("constraint violation")
monkeypatch.setattr(Resource.objects, "bulk_update", failing_bulk_update)
monkeypatch.setattr("tasks.jobs.scan.time.sleep", sleep_calls.append)
with pytest.raises(IntegrityError, match="constraint violation"):
_bulk_update_resource_failed_findings_counts(
tenant_id=str(resource.tenant_id),
scan_id="scan-id",
resources_to_update=[resource],
)
assert len(bulk_update_calls) == 1
assert sleep_calls == []
def test_failed_findings_count_update_adds_jitter_to_retry_backoff(
self, resources_fixture, monkeypatch
):
from tasks.jobs import scan as scan_jobs
resource, *_ = resources_fixture
resource.failed_findings_count = 2
bulk_update_calls = []
sleep_calls = []
original_bulk_update = Resource.objects.bulk_update
def flaky_bulk_update(objects, fields, batch_size=None):
bulk_update_calls.append([str(obj.id) for obj in objects])
if len(bulk_update_calls) == 1:
raise OperationalError("deadlock detected")
return original_bulk_update(objects, fields, batch_size=batch_size)
monkeypatch.setattr(Resource.objects, "bulk_update", flaky_bulk_update)
monkeypatch.setattr(scan_jobs, "random", MagicMock())
scan_jobs.random.uniform.return_value = 0.037
monkeypatch.setattr("tasks.jobs.scan.time.sleep", sleep_calls.append)
_bulk_update_resource_failed_findings_counts(
tenant_id=str(resource.tenant_id),
scan_id="scan-id",
resources_to_update=[resource],
)
scan_jobs.random.uniform.assert_called_once_with(0, 0.1)
assert sleep_calls == [0.137]
def test_perform_prowler_scan_with_active_mute_rules(
self,
tenants_fixture,
+583 -8
View File
@@ -3,6 +3,7 @@ from contextlib import contextmanager
from datetime import UTC, datetime, timedelta
from unittest.mock import MagicMock, patch
import httpx
import openai
import pytest
from api.models import (
@@ -14,11 +15,13 @@ from api.models import (
Task,
)
from botocore.exceptions import ClientError
from celery import states
from django_celery_beat.models import IntervalSchedule, PeriodicTask
from django_celery_results.models import TaskResult
from tasks.jobs.lighthouse_providers import (
_create_bedrock_client,
_extract_bedrock_credentials,
_LighthouseOpenAICompatibleNetworkBackend,
)
from tasks.tasks import (
DJANGO_TMP_OUTPUT_DIRECTORY,
@@ -1565,7 +1568,7 @@ class TestCheckLighthouseProviderConnectionTask:
(
LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
{"api_key": "sk-test123"},
"https://openrouter.ai/api/v1",
"https://93.184.216.34/api/v1",
{"connected": True, "error": None},
),
(
@@ -1640,7 +1643,7 @@ class TestCheckLighthouseProviderConnectionTask:
(
LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
{"api_key": "sk-invalid"},
"https://openrouter.ai/api/v1",
"https://93.184.216.34/api/v1",
openai.APIConnectionError(request=MagicMock()),
),
(
@@ -1754,6 +1757,166 @@ class TestCheckLighthouseProviderConnectionTask:
provider_cfg.refresh_from_db()
assert provider_cfg.is_active is False
def test_openai_compatible_connection_rejects_metadata_base_url_without_request(
self, tenants_fixture
):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://169.254.169.254/latest/meta-data",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
eager_result = check_lighthouse_provider_connection_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result["connected"] is False
assert "base url" in result["error"].lower()
mock_openai.assert_not_called()
provider_cfg.refresh_from_db()
assert provider_cfg.is_active is False
def test_openai_compatible_connection_disables_redirects(self, tenants_fixture):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://93.184.216.34/api/v1",
is_active=False,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
mock_client = MagicMock()
mock_client.models.list.return_value = MagicMock()
mock_openai.return_value = mock_client
eager_result = check_lighthouse_provider_connection_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result == {"connected": True, "error": None}
http_client = mock_openai.call_args.kwargs["http_client"]
assert http_client.follow_redirects is False
assert http_client.trust_env is False
def test_openai_compatible_connection_masks_remote_http_error(
self, tenants_fixture
):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://93.184.216.34/api/v1",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
remote_body = "<!DOCTYPE HTML><p>remote 404 body</p>"
response = httpx.Response(
404,
request=httpx.Request("GET", "https://provider.example/v1/models"),
)
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
mock_client = MagicMock()
mock_client.models.list.side_effect = openai.NotFoundError(
remote_body,
response=response,
body=remote_body,
)
mock_openai.return_value = mock_client
eager_result = check_lighthouse_provider_connection_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result == {"connected": False, "error": "Provider connection failed"}
assert remote_body not in result["error"]
provider_cfg.refresh_from_db()
assert provider_cfg.is_active is False
def test_openai_compatible_connection_masks_remote_auth_error(
self, tenants_fixture
):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://93.184.216.34/api/v1",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
remote_body = {"error": {"message": "remote auth detail"}}
response = httpx.Response(
401,
request=httpx.Request("GET", "https://provider.example/v1/models"),
)
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
mock_client = MagicMock()
mock_client.models.list.side_effect = openai.AuthenticationError(
"Unauthorized",
response=response,
body=remote_body,
)
mock_openai.return_value = mock_client
eager_result = check_lighthouse_provider_connection_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result == {"connected": False, "error": "API key is invalid or missing"}
assert "remote auth detail" not in result["error"]
provider_cfg.refresh_from_db()
assert provider_cfg.is_active is False
def test_openai_compatible_network_backend_uses_validated_ip(self, monkeypatch):
backend = _LighthouseOpenAICompatibleNetworkBackend()
stream = MagicMock()
def resolve_to_public_ip(host, port):
del host, port
return ("93.184.216.34",)
monkeypatch.setattr(
"tasks.jobs.lighthouse_providers.resolve_lighthouse_openai_compatible_host",
resolve_to_public_ip,
)
with patch(
"tasks.jobs.lighthouse_providers.httpcore.SyncBackend.connect_tcp",
return_value=stream,
) as mock_connect_tcp:
result = backend.connect_tcp("provider.example", 443, timeout=1.0)
assert result is stream
assert mock_connect_tcp.call_args.args[:2] == ("93.184.216.34", 443)
assert mock_connect_tcp.call_args.kwargs["timeout"] == 1.0
def test_check_connection_provider_does_not_exist(self, tenants_fixture):
"""Test that checking non-existent provider raises DoesNotExist."""
non_existent_id = str(uuid.uuid4())
@@ -1783,7 +1946,7 @@ class TestRefreshLighthouseProviderModelsTask:
(
LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
{"api_key": "sk-test123"},
"https://openrouter.ai/api/v1",
"https://93.184.216.34/api/v1",
{"model-1": "Model One", "model-2": "Model Two"},
2,
),
@@ -1863,6 +2026,106 @@ class TestRefreshLighthouseProviderModelsTask:
== expected_count
)
def test_refresh_models_rejects_metadata_base_url_without_request(
self, tenants_fixture
):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://169.254.169.254/latest/meta-data",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
with patch(
"tasks.jobs.lighthouse_providers._fetch_openai_compatible_models"
) as mock_fetch:
eager_result = refresh_lighthouse_provider_models_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result["created"] == 0
assert result["updated"] == 0
assert result["deleted"] == 0
assert "base url" in result["error"].lower()
mock_fetch.assert_not_called()
def test_refresh_models_disables_redirects(self, tenants_fixture):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://93.184.216.34/api/v1",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
mock_client = MagicMock()
mock_client.models.list.return_value = MagicMock(data=[])
mock_openai.return_value = mock_client
eager_result = refresh_lighthouse_provider_models_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result["created"] == 0
assert result["updated"] == 0
assert result["deleted"] == 0
http_client = mock_openai.call_args.kwargs["http_client"]
assert http_client.follow_redirects is False
assert http_client.trust_env is False
def test_refresh_models_masks_remote_http_error(self, tenants_fixture):
provider_cfg = LighthouseProviderConfiguration(
tenant_id=tenants_fixture[0].id,
provider_type=LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE,
base_url="https://93.184.216.34/api/v1",
is_active=True,
)
provider_cfg.credentials_decoded = {"api_key": "compatible-key"}
provider_cfg.save()
remote_body = "<!DOCTYPE HTML><p>remote 404 body</p>"
response = httpx.Response(
404,
request=httpx.Request("GET", "https://provider.example/v1/models"),
)
with patch("tasks.jobs.lighthouse_providers.openai.OpenAI") as mock_openai:
mock_client = MagicMock()
mock_client.models.list.side_effect = openai.NotFoundError(
remote_body,
response=response,
body=remote_body,
)
mock_openai.return_value = mock_client
eager_result = refresh_lighthouse_provider_models_task.apply(
kwargs={
"provider_config_id": str(provider_cfg.id),
"tenant_id": str(tenants_fixture[0].id),
}
)
assert eager_result.successful()
result = eager_result.result
assert result["created"] == 0
assert result["updated"] == 0
assert result["deleted"] == 0
assert result["error"] == "Provider connection failed"
assert remote_body not in result["error"]
def test_refresh_models_mixed_operations(self, tenants_fixture):
"""Test mixed create, update, and delete operations."""
# Create provider configuration
@@ -2286,6 +2549,51 @@ class TestCleanupOrphanScheduledScans:
assert Scan.objects.filter(id=scheduled_scan.id).exists()
assert Scan.objects.filter(id=available_scan_other_task.id).exists()
def test_cleanup_keeps_db_queued_scheduled_scans(
self, tenants_fixture, providers_fixture
):
"""DB-queued scheduled scans have a task and must not be deleted as orphans."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
periodic_task = self._create_periodic_task(provider.id, tenant.id)
task_result = TaskResult.objects.create(
task_id=str(uuid.uuid4()),
task_name="scan-perform",
status="QUEUED",
)
queued_task = Task.objects.create(
id=task_result.task_id,
task_runner_task=task_result,
tenant_id=tenant.id,
)
queued_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Queued scheduled scan",
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
scheduler_task_id=periodic_task.id,
task=queued_task,
)
scheduled_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Daily scheduled scan",
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.SCHEDULED,
scheduler_task_id=periodic_task.id,
)
deleted_count = _cleanup_orphan_scheduled_scans(
tenant_id=str(tenant.id),
provider_id=str(provider.id),
scheduler_task_id=periodic_task.id,
)
assert deleted_count == 0
assert Scan.objects.filter(id=queued_scan.id).exists()
assert Scan.objects.filter(id=scheduled_scan.id).exists()
@pytest.mark.django_db
class TestPerformScheduledScanTask:
@@ -2334,10 +2642,10 @@ class TestPerformScheduledScanTask:
)
return task_result
def test_skip_when_scheduled_scan_executing(
def test_queues_scheduled_scan_when_scheduled_scan_is_executing(
self, tenants_fixture, providers_fixture
):
"""Skip a scheduled run when another scheduled scan is already executing."""
"""Queue a scheduled run when another scheduled scan is executing."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
periodic_task = self._create_periodic_task(provider.id, tenant.id)
@@ -2364,8 +2672,16 @@ class TestPerformScheduledScanTask:
mock_scan.assert_not_called()
mock_complete_tasks.assert_not_called()
assert result["id"] == str(executing_scan.id)
assert result["state"] == StateChoices.EXECUTING
assert result["id"] != str(executing_scan.id)
assert result["state"] == StateChoices.AVAILABLE
queued_scheduled_scan = Scan.objects.get(
tenant_id=tenant.id,
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
)
assert result["id"] == str(queued_scheduled_scan.id)
assert queued_scheduled_scan.task.task_runner_task.status == "QUEUED"
assert (
Scan.objects.filter(
tenant_id=tenant.id,
@@ -2373,7 +2689,133 @@ class TestPerformScheduledScanTask:
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.SCHEDULED,
).count()
== 0
== 1
)
def test_queues_scheduled_scan_when_manual_scan_is_pending(
self, tenants_fixture, providers_fixture
):
"""Queue one scheduled run when a manual scan is already dispatched."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
self._create_periodic_task(provider.id, tenant.id)
task_id = str(uuid.uuid4())
self._create_task_result(tenant.id, task_id)
manual_task_result = TaskResult.objects.create(
task_id=str(uuid.uuid4()),
task_name="scan-perform",
status=states.PENDING,
)
manual_task = Task.objects.create(
id=manual_task_result.task_id,
task_runner_task=manual_task_result,
tenant_id=tenant.id,
)
manual_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Manual scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
task=manual_task,
)
with (
patch("tasks.tasks.perform_prowler_scan") as mock_scan,
patch("tasks.tasks._perform_scan_complete_tasks") as mock_complete_tasks,
self._override_task_request(perform_scheduled_scan_task, id=task_id),
):
result = perform_scheduled_scan_task.run(
tenant_id=str(tenant.id), provider_id=str(provider.id)
)
mock_scan.assert_not_called()
mock_complete_tasks.assert_not_called()
assert result["id"] != str(manual_scan.id)
assert result["state"] == StateChoices.AVAILABLE
queued_scheduled_scan = Scan.objects.get(
tenant_id=tenant.id,
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
)
assert result["id"] == str(queued_scheduled_scan.id)
assert queued_scheduled_scan.task.task_runner_task.status == "QUEUED"
scheduled_scan = Scan.objects.get(
tenant_id=tenant.id,
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.SCHEDULED,
)
assert scheduled_scan.scheduled_at > datetime.now(UTC)
def test_coalesces_scheduled_scan_when_one_is_already_queued(
self, tenants_fixture, providers_fixture
):
"""Reuse the existing queued scheduled scan instead of adding another."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
periodic_task = self._create_periodic_task(provider.id, tenant.id)
task_id = str(uuid.uuid4())
self._create_task_result(tenant.id, task_id)
manual_task_result = TaskResult.objects.create(
task_id=str(uuid.uuid4()),
task_name="scan-perform",
status=states.PENDING,
)
manual_task = Task.objects.create(
id=manual_task_result.task_id,
task_runner_task=manual_task_result,
tenant_id=tenant.id,
)
Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Manual scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
task=manual_task,
)
queued_task_result = TaskResult.objects.create(
task_id=str(uuid.uuid4()),
task_name="scan-perform",
status="QUEUED",
)
queued_task = Task.objects.create(
id=queued_task_result.task_id,
task_runner_task=queued_task_result,
tenant_id=tenant.id,
)
queued_scheduled_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Daily scheduled scan",
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
scheduler_task_id=periodic_task.id,
task=queued_task,
)
with (
patch("tasks.tasks.perform_prowler_scan") as mock_scan,
patch("tasks.tasks._perform_scan_complete_tasks") as mock_complete_tasks,
self._override_task_request(perform_scheduled_scan_task, id=task_id),
):
result = perform_scheduled_scan_task.run(
tenant_id=str(tenant.id), provider_id=str(provider.id)
)
mock_scan.assert_not_called()
mock_complete_tasks.assert_not_called()
assert result["id"] == str(queued_scheduled_scan.id)
assert (
Scan.objects.filter(
tenant_id=tenant.id,
provider=provider,
trigger=Scan.TriggerChoices.SCHEDULED,
state=StateChoices.AVAILABLE,
).count()
== 1
)
def test_creates_next_scheduled_scan_after_completion(
@@ -2435,6 +2877,41 @@ class TestPerformScheduledScanTask:
== 1
)
def test_next_scheduled_scan_failure_does_not_mask_completed_scan(
self, tenants_fixture, providers_fixture, caplog
):
"""Keep scheduled scan success when next-run creation fails."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
self._create_periodic_task(provider.id, tenant.id)
task_id = str(uuid.uuid4())
self._create_task_result(tenant.id, task_id)
def _complete_scan(tenant_id, scan_id, provider_id):
scan_instance = Scan.objects.get(id=scan_id)
scan_instance.state = StateChoices.COMPLETED
scan_instance.save()
return {"status": "ok"}
with (
patch("tasks.tasks.perform_prowler_scan", side_effect=_complete_scan),
patch("tasks.tasks._perform_scan_complete_tasks"),
patch(
"tasks.tasks._get_or_create_next_scheduled_scan",
side_effect=RuntimeError("scheduler unavailable"),
),
patch("tasks.tasks._dispatch_next_queued_provider_scan") as mock_dispatch,
self._override_task_request(perform_scheduled_scan_task, id=task_id),
caplog.at_level("ERROR"),
):
result = perform_scheduled_scan_task.run(
tenant_id=str(tenant.id), provider_id=str(provider.id)
)
assert result == {"status": "ok"}
mock_dispatch.assert_called_once_with(str(tenant.id), str(provider.id))
assert "Failed to ensure next scheduled scan" in caplog.text
def test_dedupes_multiple_scheduled_scans_before_run(
self, tenants_fixture, providers_fixture
):
@@ -2549,6 +3026,104 @@ class TestPerformScanTask:
mock_scan.assert_not_called()
mock_complete_tasks.assert_not_called()
def test_dispatches_next_queued_scan_after_completion(
self,
tenants_fixture,
providers_fixture,
django_capture_on_commit_callbacks,
):
"""Dispatch the next queued scan for the provider after completion."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
current_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Running scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
)
queued_task_result = TaskResult.objects.create(
task_id=str(uuid.uuid4()),
task_name="scan-perform",
status="QUEUED",
)
queued_task = Task.objects.create(
id=queued_task_result.task_id,
task_runner_task=queued_task_result,
tenant_id=tenant.id,
)
queued_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Queued scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
task=queued_task,
)
def _complete_scan(tenant_id, scan_id, provider_id, checks_to_execute=None):
scan_instance = Scan.objects.get(id=scan_id)
scan_instance.state = StateChoices.COMPLETED
scan_instance.save()
return {"status": "ok"}
with (
patch("tasks.tasks.perform_prowler_scan", side_effect=_complete_scan),
patch("tasks.tasks._perform_scan_complete_tasks"),
patch("tasks.tasks.perform_scan_task.apply_async") as mock_apply_async,
):
with django_capture_on_commit_callbacks(execute=True):
result = perform_scan_task.run(
tenant_id=str(tenant.id),
scan_id=str(current_scan.id),
provider_id=str(provider.id),
)
queued_task_result.refresh_from_db()
assert result == {"status": "ok"}
assert queued_task_result.status == states.PENDING
mock_apply_async.assert_called_once_with(
kwargs={
"tenant_id": str(tenant.id),
"scan_id": str(queued_scan.id),
"provider_id": str(provider.id),
},
task_id=str(queued_task.id),
)
def test_dispatch_failure_does_not_mask_completed_scan(
self, tenants_fixture, providers_fixture, caplog
):
"""Keep scan success when queued dispatch fails after completion."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
current_scan = Scan.objects.create(
tenant_id=tenant.id,
provider=provider,
name="Running scan",
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.AVAILABLE,
)
with (
patch("tasks.tasks.perform_prowler_scan", return_value={"status": "ok"}),
patch("tasks.tasks._perform_scan_complete_tasks"),
patch(
"tasks.tasks._dispatch_next_queued_provider_scan",
side_effect=RuntimeError("dispatch unavailable"),
) as mock_dispatch,
caplog.at_level("ERROR"),
):
result = perform_scan_task.run(
tenant_id=str(tenant.id),
scan_id=str(current_scan.id),
provider_id=str(provider.id),
)
assert result == {"status": "ok"}
mock_dispatch.assert_called_once_with(str(tenant.id), str(provider.id))
assert "Failed to dispatch next queued scan" in caplog.text
@pytest.mark.django_db
class TestReaggregateAllFindingGroupSummaries:
+1
View File
@@ -103,6 +103,7 @@ def _get_or_create_scheduled_scan(
trigger=Scan.TriggerChoices.SCHEDULED,
state__in=(StateChoices.SCHEDULED, StateChoices.AVAILABLE),
scheduler_task_id=scheduler_task_id,
task__isnull=True,
).order_by("scheduled_at", "inserted_at")
)
+39
View File
@@ -0,0 +1,39 @@
[tool.towncrier]
directory = "changelog.d"
filename = "CHANGELOG.md"
start_string = "<!-- changelog: release notes start -->\n"
title_format = "## [{version}] ({name})"
issue_format = "[(#{issue})](https://github.com/prowler-cloud/prowler/pull/{issue})"
template = "../.github/towncrier/template.md.jinja"
underlines = ["", "", ""]
ignore = [".gitkeep", "README.md"]
[[tool.towncrier.type]]
directory = "added"
name = "🚀 Added"
showcontent = true
[[tool.towncrier.type]]
directory = "changed"
name = "🔄 Changed"
showcontent = true
[[tool.towncrier.type]]
directory = "deprecated"
name = "⚠️ Deprecated"
showcontent = true
[[tool.towncrier.type]]
directory = "removed"
name = "❌ Removed"
showcontent = true
[[tool.towncrier.type]]
directory = "fixed"
name = "🐞 Fixed"
showcontent = true
[[tool.towncrier.type]]
directory = "security"
name = "🔐 Security"
showcontent = true
Generated
+4 -4
View File
@@ -4673,8 +4673,8 @@ wheels = [
[[package]]
name = "prowler"
version = "5.32.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=v5.32#4ae7c67d3fc6ccf06296517c9d05824905718ede" }
version = "5.33.0"
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=v5.33#76a2d7bfe61a3ac6c96e497eded30a2978cd3ffe" }
dependencies = [
{ name = "alibabacloud-actiontrail20200706" },
{ name = "alibabacloud-credentials" },
@@ -4762,7 +4762,7 @@ dependencies = [
[[package]]
name = "prowler-api"
version = "1.33.2"
version = "1.34.1"
source = { virtual = "." }
dependencies = [
{ name = "cartography" },
@@ -4862,7 +4862,7 @@ requires-dist = [
{ name = "matplotlib", specifier = "==3.10.8" },
{ name = "neo4j", specifier = "==6.1.0" },
{ name = "openai", specifier = "==1.109.1" },
{ name = "prowler", git = "https://github.com/prowler-cloud/prowler.git?rev=v5.32" },
{ name = "prowler", git = "https://github.com/prowler-cloud/prowler.git?rev=v5.33" },
{ name = "psycopg2-binary", specifier = "==2.9.9" },
{ name = "pytest-celery", extras = ["redis"], specifier = "==1.3.0" },
{ name = "reportlab", specifier = "==4.4.10" },
+19 -1
View File
@@ -39,6 +39,24 @@ The former build-time variables map to the new runtime variables as follows:
The build-time-only Sentry variables used for source-map upload — `SENTRY_ORG`, `SENTRY_PROJECT`, `SENTRY_AUTH_TOKEN`, and `SENTRY_RELEASE` — keep their names, as they are not part of the App's runtime configuration.
## Enabling Third-Party Integrations
Prowler App gates each optional third-party integration behind an explicit enable flag. When an integration is configured through its new `UI_*` variables, it loads only when its flag is set to the exact string `"true"`; any other value, including unset, leaves it off. This default-off behavior keeps a deployment free of third-party egress unless it opts in. Deployments still using the deprecated legacy variable names keep loading without the flag, for backward compatibility (see [Deprecated Names](#deprecated-names)).
| Integration | Enable flag | Required configuration when enabled |
|-------------|-------------|-------------------------------------|
| Sentry (error monitoring) | `UI_SENTRY_ENABLE` | `UI_SENTRY_DSN` |
| Google Tag Manager | `UI_GOOGLE_TAG_MANAGER_ENABLE` | `UI_GOOGLE_TAG_MANAGER_ID` |
| PostHog (product analytics) | `UI_POSTHOG_ENABLE` | `UI_POSTHOG_KEY` and `UI_POSTHOG_HOST` |
When an integration is enabled but its required configuration is missing, Prowler App fails fast at server startup with a clear error, so a misconfigured container never starts silently. A new `UI_*` value set while its enable flag is not `"true"` is ignored, and the server logs a one-time startup warning noting that the integration will not load. Legacy names follow the backward-compatible rule described in [Deprecated Names](#deprecated-names).
PostHog support is currently limited to configuration validation: Prowler App reads and validates the PostHog variables but does not yet load a PostHog client.
<Warning>
Configuring an integration through the new `UI_*` variables now requires its enable flag. A deployment that adopted `UI_SENTRY_DSN` or `UI_GOOGLE_TAG_MANAGER_ID` must also set `UI_SENTRY_ENABLE=true` or `UI_GOOGLE_TAG_MANAGER_ENABLE=true` to keep the integration active. Deployments still using the legacy names (`NEXT_PUBLIC_*`, or `POSTHOG_KEY` and `POSTHOG_HOST`) keep working without the flag.
</Warning>
## Upcoming Breaking Change
<Warning>
@@ -49,5 +67,5 @@ Prowler API, Prowler SDK, and Prowler MCP Server have not yet adopted the conven
## Deprecated Names
- **Prowler App:** The bare server-side `SENTRY_DSN` and `SENTRY_ENVIRONMENT` are no longer read; the server and edge runtimes now read `UI_SENTRY_DSN` and `UI_SENTRY_ENVIRONMENT`. The former `NEXT_PUBLIC_*` build-time variables are deprecated but still read at runtime as a fallback when the matching `UI_*` variable is unset. This fallback will be removed in a future release, so set the `UI_*` runtime variables on the running container.
- **Prowler App:** The bare server-side `SENTRY_DSN` and `SENTRY_ENVIRONMENT` are no longer read; the server and edge runtimes now read `UI_SENTRY_DSN` and `UI_SENTRY_ENVIRONMENT`. The former `NEXT_PUBLIC_*` names — and, for PostHog, the unprefixed `POSTHOG_KEY` and `POSTHOG_HOST` — are deprecated but stay backward compatible: they are read at runtime regardless of the enable flag, so an existing deployment keeps its integration active without opting in. The new `UI_*` names, by contrast, load only when the matching enable flag is set to `"true"`. These legacy names will be removed in a future release, so migrate to the `UI_*` runtime variables — and set the enable flag — on the running container.
- **Prowler API, Prowler SDK, and Prowler MCP Server:** The current, unprefixed variable names are deprecated. They continue to work today and will be removed once the prefixed convention is adopted for each component, as described in [Upcoming Breaking Change](#upcoming-breaking-change).
+1 -1
View File
@@ -3421,7 +3421,7 @@ Use existing providers as templates, this will help you to understand better the
- **Documentation & Maintenance**
- **README Updates**: Update provider-specific documentation
- **Changelog**: Document changes and new features
- **Changelog**: Document changes and new features with a fragment under `prowler/changelog.d/` (see the [Pull Request Template](https://github.com/prowler-cloud/prowler/blob/master/.github/pull_request_template.md))
- **Examples**: Provide usage examples and common scenarios
- **Troubleshooting**: Include common issues and solutions
- **Documentation**: Update the provider documentation to include your new tool provider in the examples and implementation guidance.
@@ -778,7 +778,7 @@ Before opening the pull request:
uv run pre-commit run --all-files
uv run pytest -n auto
```
2. Add a changelog entry under the `### 🚀 Added` section of `prowler/CHANGELOG.md`, describing the new framework and the providers it covers.
2. Add a changelog fragment `prowler/changelog.d/<slug>.added.md`, describing the new framework and the providers it covers (no PR link in the text; it is attached automatically at release time).
3. Follow the [Pull Request Template](https://github.com/prowler-cloud/prowler/blob/master/.github/pull_request_template.md) and set the PR title using Conventional Commits, e.g. `feat(compliance): add My Framework 1.0 for AWS`.
4. Request review from the compliance codeowners listed in `.github/CODEOWNERS`.
+9 -2
View File
@@ -70,7 +70,7 @@
{
"group": "Prowler Lighthouse AI",
"pages": [
"getting-started/products/prowler-lighthouse-ai"
"getting-started/products/prowler-cloud-lighthouse"
]
},
{
@@ -154,8 +154,15 @@
]
},
{
"group": "Lighthouse AI",
"group": "Lighthouse AI (Prowler Cloud)",
"pages": [
"user-guide/tutorials/prowler-cloud-lighthouse-multi-llm"
]
},
{
"group": "Lighthouse AI (Open Source)",
"pages": [
"getting-started/products/prowler-lighthouse-ai",
"user-guide/tutorials/prowler-app-lighthouse",
"user-guide/tutorials/prowler-app-lighthouse-multi-llm"
]
@@ -128,8 +128,8 @@ To update the environment file:
Edit the `.env` file and change version values:
```env
PROWLER_UI_VERSION="5.31.0"
PROWLER_API_VERSION="5.31.0"
PROWLER_UI_VERSION="5.32.0"
PROWLER_API_VERSION="5.32.0"
```
<Note>
@@ -0,0 +1,86 @@
---
title: 'Overview'
---
import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
Prowler Cloud runs an enhanced version of Lighthouse AI in Open Source repository, the Agentic Cloud Defender that helps teams understand, prioritize, and remediate security findings across cloud environments.
<SubscriptionBanner />
<img src="/images/prowler-app/lighthouse/prowler-cloud/chat-animation.gif" alt="Lighthouse AI on Prowler Cloud" />
## What's New
The Agentic Cloud Defender does more than answer questions, it helps teams **find and remediate what actually matters**, cutting through the noise to focus on the risk that counts. Prowler Cloud sharpens Lighthouse AI with the following improvements over the open-source version:
<CardGroup cols={2}>
<Card title="Persistent Chat Sessions" icon="clock-rotate-left">
Conversations are saved and can be revisited or resumed at any time.
</Card>
<Card title="GPT-5.5 by Default" icon="microchip">
An upgraded default model delivers stronger reasoning and tool calling.
</Card>
<Card title="Normal and Agentic Views" icon="table-columns">
Switch between the standard interface and a chat-first agentic view.
</Card>
<Card title="Provider Connection Checks" icon="plug-circle-check">
Credentials are validated automatically when a provider is configured.
</Card>
</CardGroup>
## Chat View
Lighthouse AI is no longer a separate section in the left navigation. Prowler Cloud now offers two application views: a normal view for browsing dashboards, findings, and configuration, and an agentic chat view, powered by Lighthouse AI, for conversational, multi-step security analysis. Conversations are saved automatically, so earlier sessions can be reopened and resumed at any time.
Promoting the chat to a top-level view gives Lighthouse AI the room it needs for a fully agentic workflow and makes the Agentic Cloud Defender a primary way to work in Prowler Cloud.
<img src="/images/prowler-app/lighthouse/prowler-cloud/main-chat-page.png" alt="Lighthouse AI chat view in Prowler Cloud" />
### Tool Usage
Lighthouse AI on Prowler Cloud renders the agent's work as it happens, so responses are easier to follow and to trust. Tool calls and reasoning steps appear in the order they occur within the conversation.
- **Ordered steps:** Tool calls and reasoning are shown in sequence, reflecting how the agent reached its answer.
- **Tool visibility:** The data tools invoked to retrieve findings and other Prowler information are displayed as the agent uses them.
- **Thought process:** The agent's reasoning is presented alongside its actions, and can be expanded to review the full chain of steps.
<img src="/images/prowler-app/lighthouse/prowler-cloud/tool-usage.png" alt="Ordered tool usage and reasoning in the Lighthouse AI chat" />
## Configuration
Configure Lighthouse AI on Prowler Cloud from **Configuration** → **Lighthouse AI**:
1. Click the desired provider (OpenAI, Amazon Bedrock, or OpenAI Compatible).
2. Enter the required credentials.
3. Click **Save**. The connection is validated automatically before the provider becomes available.
<img src="/images/prowler-app/lighthouse/prowler-cloud/config-page.png" alt="Lighthouse AI configuration page in Prowler Cloud" />
### Business Context
At the top of the configuration page, the optional **Business Context** field lets teams add environment priorities, compliance requirements, and ownership details, so responses align with organizational needs.
Lighthouse AI on Prowler Cloud supports OpenAI, Amazon Bedrock, and OpenAI-compatible providers, with GPT-5.5 as the default. For per-provider setup and how to switch the default provider or model, see [Using Multiple LLM Providers](/user-guide/tutorials/prowler-cloud-lighthouse-multi-llm).
## FAQ
**Which LLM providers are supported?**
OpenAI (GPT models, including the default GPT-5.5), Amazon Bedrock (Claude, Llama, Titan, and others), and any OpenAI-compatible service such as OpenRouter.
**Can Lighthouse AI change my cloud environment?**
No. Lighthouse AI has read-only access to security data and no tools to modify resources, even when the connected cloud credentials would allow changes.
## Looking for the Open Source Version?
Lighthouse AI is also available in the self-hosted, open-source Prowler App. For its capabilities, FAQs, and limitations, see the open-source documentation.
<Card title="Lighthouse AI (Open Source)" icon="github" href="/getting-started/products/prowler-lighthouse-ai">
Capabilities, FAQs, and limitations for Lighthouse AI in the open-source Prowler App
</Card>
## Getting Help
For issues or suggestions with Lighthouse AI on Prowler Cloud, request support at [support.prowler.com](https://support.prowler.com) or [reach out through our Slack channel](https://goto.prowler.com/slack).
@@ -8,11 +8,16 @@ import { VersionBadge } from "/snippets/version-badge.mdx"
Prowler Lighthouse AI is a Cloud Security Analyst chatbot that helps you understand, prioritize, and remediate security findings in your cloud environments. It's designed to provide security expertise for teams without dedicated resources, acting as your 24/7 virtual cloud security analyst.
<img src="/images/prowler-app/lighthouse-intro.png" alt="Prowler Lighthouse" />
<img src="/images/prowler-app/lighthouse/oss/intro.png" alt="Prowler Lighthouse" />
<Card title="Set Up Lighthouse AI" icon="rocket" href="/user-guide/tutorials/prowler-app-lighthouse#set-up">
Learn how to configure Lighthouse AI with your preferred LLM provider
</Card>
<CardGroup cols={2}>
<Card title="Lighthouse AI on Prowler Cloud" icon="cloud" href="/getting-started/products/prowler-cloud-lighthouse">
Discover the enhanced Cloud experience: persistent chat sessions, GPT-5.5 by default, a dedicated agentic view, and transparent reasoning
</Card>
<Card title="Set Up Lighthouse AI" icon="rocket" href="/user-guide/tutorials/prowler-app-lighthouse#set-up">
Learn how to configure Lighthouse AI with your preferred LLM provider
</Card>
</CardGroup>
## Capabilities
@@ -26,7 +31,7 @@ Ask questions in plain English about your security findings. Examples:
- "Show me all S3 buckets with public access."
- "What security issues were found in my production accounts?"
<img src="/images/prowler-app/lighthouse-feature1.png" alt="Natural language querying" />
<img src="/images/prowler-app/lighthouse/oss/feature1.png" alt="Natural language querying" />
### Detailed Remediation Guidance
@@ -36,7 +41,7 @@ Get tailored step-by-step instructions for fixing security issues:
- Commands or console steps to implement fixes
- Alternative approaches with different solutions
<img src="/images/prowler-app/lighthouse-feature2.png" alt="Detailed Remediation" />
<img src="/images/prowler-app/lighthouse/oss/feature2.png" alt="Detailed Remediation" />
### Enhanced Context and Analysis
@@ -46,9 +51,9 @@ Lighthouse AI can provide additional context to help you understand the findings
- Provide risk assessments based on your environment and context
- Connect related findings to show broader security patterns
<img src="/images/prowler-app/lighthouse-config.png" alt="Business Context" />
<img src="/images/prowler-app/lighthouse/oss/config.png" alt="Business Context" />
<img src="/images/prowler-app/lighthouse-feature3.png" alt="Contextual Responses" />
<img src="/images/prowler-app/lighthouse/oss/feature3.png" alt="Contextual Responses" />
## Important Notes
Binary file not shown.

Before

Width:  |  Height:  |  Size: 115 KiB

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 316 KiB

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 304 KiB

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 477 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 575 KiB

After

Width:  |  Height:  |  Size: 107 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 324 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 115 KiB

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 423 KiB

After

Width:  |  Height:  |  Size: 120 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 275 KiB

After

Width:  |  Height:  |  Size: 121 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 66 KiB

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 481 KiB

After

Width:  |  Height:  |  Size: 610 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 456 KiB

After

Width:  |  Height:  |  Size: 593 KiB

Before

Width:  |  Height:  |  Size: 96 KiB

After

Width:  |  Height:  |  Size: 96 KiB

Before

Width:  |  Height:  |  Size: 540 KiB

After

Width:  |  Height:  |  Size: 540 KiB

Before

Width:  |  Height:  |  Size: 136 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Before

Width:  |  Height:  |  Size: 147 KiB

After

Width:  |  Height:  |  Size: 147 KiB

Before

Width:  |  Height:  |  Size: 180 KiB

After

Width:  |  Height:  |  Size: 180 KiB

Before

Width:  |  Height:  |  Size: 165 KiB

After

Width:  |  Height:  |  Size: 165 KiB

Before

Width:  |  Height:  |  Size: 347 KiB

After

Width:  |  Height:  |  Size: 347 KiB

Before

Width:  |  Height:  |  Size: 173 KiB

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 236 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 472 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 324 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 505 KiB

+155
View File
@@ -0,0 +1,155 @@
#!/usr/bin/env python3
"""Generate docs/snippets/provider-cards.mdx from provider getting-started pages.
Scans docs/user-guide/providers/<name>/getting-started-*.mdx, keeps only the
providers that Prowler App/Cloud actually supports (source of truth: the
`ProviderChoices` enum in api/src/backend/api/models.py CLI-only providers
such as Linode/LLM/Scaleway/StackIT are excluded), reads the frontmatter
`title`, derives a display name, and emits a snippet exporting a
`ProviderCards` component. Wired into pre-commit so the snippet stays in sync
whenever a provider page or the API enum changes.
"""
from __future__ import annotations
import ast
import re
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[2]
PROVIDERS_DIR = REPO_ROOT / "docs" / "user-guide" / "providers"
SNIPPET_PATH = REPO_ROOT / "docs" / "snippets" / "provider-cards.mdx"
API_MODELS_PATH = REPO_ROOT / "api" / "src" / "backend" / "api" / "models.py"
# Docs folder names that don't match the API enum key. Keep tiny — only rename
# entries when the docs folder disagrees with the API-side identifier.
DOCS_DIR_TO_API_KEY = {
"microsoft365": "m365",
"oci": "oraclecloud",
}
# Folder-name → Mintlify icon override. Providers not listed fall back to
# DEFAULT_ICON. Add an entry only when the default looks wrong for a provider.
ICON_OVERRIDES = {
"alibabacloud": "cloud",
"aws": "aws",
"azure": "microsoft",
"cloudflare": "cloudflare",
"gcp": "google",
"github": "github",
"googleworkspace": "users",
"iac": "code",
"image": "docker",
"kubernetes": "dharmachakra",
"microsoft365": "briefcase",
"mongodbatlas": "leaf",
"oci": "database",
"okta": "key",
"openstack": "cubes",
"vercel": "triangle",
}
DEFAULT_ICON = "cloud"
TITLE_RE = re.compile(r"^\s*title\s*:\s*['\"](?P<title>.+?)['\"]\s*$", re.MULTILINE)
NAME_CLEANUP_RE = re.compile(
r"^Getting Started [Ww]ith (?:the )?(?P<name>.+?)(?: on Prowler)?(?: Provider)?$"
)
def app_supported_provider_keys() -> set[str]:
"""Return the set of provider keys declared in the API's ProviderChoices enum.
Uses ast rather than regex so formatting changes, decorators, comments, or
multi-line values in the enum body don't silently drop or invent providers.
"""
tree = ast.parse(API_MODELS_PATH.read_text(encoding="utf-8"))
for node in ast.walk(tree):
if not (isinstance(node, ast.ClassDef) and node.name == "ProviderChoices"):
continue
keys: set[str] = set()
for item in node.body:
if not isinstance(item, ast.Assign):
continue
value = item.value
# Django TextChoices members look like: NAME = "key", _("Label")
# which parses as an ast.Tuple whose first element is the key.
if isinstance(value, ast.Tuple) and value.elts:
first = value.elts[0]
if isinstance(first, ast.Constant) and isinstance(first.value, str):
keys.add(first.value)
return keys
raise RuntimeError(
f"Could not locate ProviderChoices class in {API_MODELS_PATH.relative_to(REPO_ROOT)}"
)
def extract_title(mdx_path: Path) -> str:
text = mdx_path.read_text(encoding="utf-8")
match = TITLE_RE.search(text)
if not match:
raise ValueError(f"No frontmatter title in {mdx_path}")
return match.group("title")
def display_name(title: str) -> str:
match = NAME_CLEANUP_RE.match(title)
return match.group("name") if match else title
def collect_providers() -> list[dict]:
supported = app_supported_provider_keys()
providers = []
for provider_dir in sorted(PROVIDERS_DIR.iterdir()):
if not provider_dir.is_dir():
continue
api_key = DOCS_DIR_TO_API_KEY.get(provider_dir.name, provider_dir.name)
if api_key not in supported:
continue
pages = sorted(provider_dir.glob("getting-started-*.mdx"))
if not pages:
continue
page = pages[0]
name = display_name(extract_title(page))
href = f"/user-guide/providers/{provider_dir.name}/{page.stem}"
icon = ICON_OVERRIDES.get(provider_dir.name, DEFAULT_ICON)
providers.append({"name": name, "href": href, "icon": icon})
providers.sort(key=lambda p: p["name"].lower())
return providers
def render_snippet(providers: list[dict]) -> str:
cards = "\n".join(
f' <Card title="{p["name"]}" icon="{p["icon"]}" href="{p["href"]}" />'
for p in providers
)
return (
"{/* AUTO-GENERATED by docs/scripts/generate_provider_cards.py — do not edit by hand. */}\n"
"{/* Regenerated on pre-commit whenever any provider getting-started page changes. */}\n"
"\n"
"export const ProviderCards = () => (\n"
" <Columns cols={3}>\n"
f"{cards}\n"
" </Columns>\n"
");\n"
)
def main() -> int:
providers = collect_providers()
if not providers:
print("No provider getting-started pages found", file=sys.stderr)
return 1
new_content = render_snippet(providers)
current = SNIPPET_PATH.read_text(encoding="utf-8") if SNIPPET_PATH.exists() else ""
if new_content == current:
return 0
SNIPPET_PATH.write_text(new_content, encoding="utf-8")
print(
f"Regenerated {SNIPPET_PATH.relative_to(REPO_ROOT)} ({len(providers)} providers)"
)
return 1 # signal pre-commit that the file changed
if __name__ == "__main__":
sys.exit(main())
+23
View File
@@ -0,0 +1,23 @@
{/* AUTO-GENERATED by docs/scripts/generate_provider_cards.py — do not edit by hand. */}
{/* Regenerated on pre-commit whenever any provider getting-started page changes. */}
export const ProviderCards = () => (
<Columns cols={3}>
<Card title="Alibaba Cloud" icon="cloud" href="/user-guide/providers/alibabacloud/getting-started-alibabacloud" />
<Card title="AWS" icon="aws" href="/user-guide/providers/aws/getting-started-aws" />
<Card title="Azure" icon="microsoft" href="/user-guide/providers/azure/getting-started-azure" />
<Card title="Cloudflare" icon="cloudflare" href="/user-guide/providers/cloudflare/getting-started-cloudflare" />
<Card title="GCP" icon="google" href="/user-guide/providers/gcp/getting-started-gcp" />
<Card title="GitHub" icon="github" href="/user-guide/providers/github/getting-started-github" />
<Card title="Google Workspace" icon="users" href="/user-guide/providers/googleworkspace/getting-started-googleworkspace" />
<Card title="IaC" icon="code" href="/user-guide/providers/iac/getting-started-iac" />
<Card title="Image" icon="docker" href="/user-guide/providers/image/getting-started-image" />
<Card title="Kubernetes" icon="dharmachakra" href="/user-guide/providers/kubernetes/getting-started-k8s" />
<Card title="Microsoft 365" icon="briefcase" href="/user-guide/providers/microsoft365/getting-started-m365" />
<Card title="MongoDB Atlas" icon="leaf" href="/user-guide/providers/mongodbatlas/getting-started-mongodbatlas" />
<Card title="Okta" icon="key" href="/user-guide/providers/okta/getting-started-okta" />
<Card title="OpenStack" icon="cubes" href="/user-guide/providers/openstack/getting-started-openstack" />
<Card title="Oracle Cloud Infrastructure (OCI)" icon="database" href="/user-guide/providers/oci/getting-started-oci" />
<Card title="Vercel" icon="triangle" href="/user-guide/providers/vercel/getting-started-vercel" />
</Columns>
);
@@ -22,72 +22,95 @@ Numeric thresholds enforce hard limits. A value outside the accepted range is dr
The following list includes all the AWS checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|---------------------------------------------------------------|--------------------------------------------------|-----------------|
| `acm_certificates_expiration_check` | `days_to_expire_threshold` | Integer |
| `acmpca_certificate_authority_pqc_key_algorithm` | `acmpca_pqc_key_algorithms` | List of Strings |
| `apigateway_restapi_no_secrets_in_stage_variables` | `secrets_ignore_patterns` | List of Strings |
| `appstream_fleet_maximum_session_duration` | `max_session_duration_seconds` | Integer |
| `appstream_fleet_session_disconnect_timeout` | `max_disconnect_timeout_in_seconds` | Integer |
| `appstream_fleet_session_idle_disconnect_timeout` | `max_idle_disconnect_timeout_in_seconds` | Integer |
| `autoscaling_find_secrets_ec2_launch_configuration` | `secrets_ignore_patterns` | List of Strings |
| `awslambda_function_no_secrets_in_code` | `secrets_ignore_patterns` | List of Strings |
| `awslambda_function_no_secrets_in_variables` | `secrets_ignore_patterns` | List of Strings |
| `awslambda_function_using_supported_runtimes` | `obsolete_lambda_runtimes` | Integer |
| `awslambda_function_vpc_is_in_multi_azs` | `lambda_min_azs` | Integer |
| `cloudformation_stack_outputs_find_secrets` | `secrets_ignore_patterns` | List of Strings |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_actions` | List of Strings |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_entropy` | Integer |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_minutes` | Integer |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_actions` | List of Strings |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_entropy` | Integer |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_minutes` | Integer |
| `cloudwatch_log_group_no_secrets_in_logs` | `secrets_ignore_patterns` | List of Strings |
| `cloudwatch_log_group_retention_policy_specific_days_enabled` | `log_group_retention_days` | Integer |
| `codebuild_github_allowed_organizations` | `github_allowed_organizations` | List of Strings |
| `codebuild_project_no_secrets_in_variables` | `excluded_sensitive_environment_variables` | List of Strings |
| `codebuild_project_no_secrets_in_variables` | `secrets_ignore_patterns` | List of Strings |
| `config_recorder_all_regions_enabled` | `mute_non_default_regions` | Boolean |
| `drs_job_exist` | `mute_non_default_regions` | Boolean |
| `ec2_elastic_ip_shodan` | `shodan_api_key` | String |
| `ec2_instance_older_than_specific_days` | `max_ec2_instance_age_in_days` | Integer |
| `ec2_instance_secrets_user_data` | `secrets_ignore_patterns` | List of Strings |
| `ec2_launch_template_no_secrets` | `secrets_ignore_patterns` | List of Strings |
| `ec2_securitygroup_allow_ingress_from_internet_to_any_port` | `ec2_allowed_instance_owners` | List of Strings |
| `ec2_securitygroup_allow_ingress_from_internet_to_any_port` | `ec2_allowed_interface_types` | List of Strings |
| `ec2_securitygroup_allow_ingress_from_internet_to_high_risk_tcp_ports`| `ec2_high_risk_ports` | List of Integer |
| `ec2_securitygroup_with_many_ingress_egress_rules` | `max_security_group_rules` | Integer |
| `ecs_task_definitions_no_environment_secrets` | `secrets_ignore_patterns` | List of Strings |
| `ecr_repositories_scan_vulnerabilities_in_latest_image` | `ecr_repository_vulnerability_minimum_severity` | String |
| `eks_cluster_uses_a_supported_version` | `eks_cluster_oldest_version_supported` | String |
| `eks_control_plane_logging_all_types_enabled` | `eks_required_log_types` | List of Strings |
| `elasticache_redis_cluster_backup_enabled` | `minimum_snapshot_retention_period` | Integer |
| `elb_is_in_multiple_az` | `elb_min_azs` | Integer |
| `elbv2_is_in_multiple_az` | `elbv2_min_azs` | Integer |
| `rolesanywhere_trust_anchor_pqc_pki` | `rolesanywhere_pqc_pca_key_algorithms` | List of Strings |
| `cloudfront_distributions_pqc_tls_enabled` | `cloudfront_pqc_min_protocol_versions` | List of Strings |
| `apigateway_domain_name_pqc_tls_enabled` | `apigateway_pqc_tls_allowed_policies` | List of Strings |
| `guardduty_is_enabled` | `mute_non_default_regions` | Boolean |
| `iam_user_access_not_stale_to_sagemaker` | `max_unused_sagemaker_access_days` | Integer |
| `iam_user_accesskey_unused` | `max_unused_access_keys_days` | Integer |
| `iam_user_console_access_unused` | `max_console_access_days` | Integer |
| `organizations_delegated_administrators` | `organizations_trusted_delegated_administrators` | List of Strings |
| `organizations_scp_check_deny_regions` | `organizations_enabled_regions` | List of Strings |
| `rds_instance_backup_enabled` | `check_rds_instance_replicas` | Boolean |
| `securityhub_enabled` | `mute_non_default_regions` | Boolean |
| `secretsmanager_secret_unused` | `max_days_secret_unused` | Integer |
| `secretsmanager_secret_rotated_periodically` | `max_days_secret_unrotated` | Integer |
| `ssm_document_secrets` | `secrets_ignore_patterns` | List of Strings |
| `trustedadvisor_premium_support_plan_subscribed` | `verify_premium_support_plans` | Boolean |
| `transfer_server_pqc_ssh_kex_enabled` | `transfer_pqc_ssh_allowed_policies` | List of Strings |
| `dynamodb_table_cross_account_access` | `trusted_account_ids` | List of Strings |
| `eventbridge_bus_cross_account_access` | `trusted_account_ids` | List of Strings |
| `eventbridge_schema_registry_cross_account_access` | `trusted_account_ids` | List of Strings |
| `s3_bucket_cross_account_access` | `trusted_account_ids` | List of Strings |
| `ssm_documents_set_as_public` | `trusted_account_ids` | List of Strings |
| `vpc_endpoint_connections_trust_boundaries` | `trusted_account_ids` | List of Strings |
| `vpc_endpoint_services_allowed_principals_trust_boundaries` | `trusted_account_ids` | List of Strings |
| `opensearch_service_domains_not_publicly_accessible` | `trusted_ips` | List of Strings |
| Check Name | Value | Type | Default |
|------------------------------------------------------------------------|---------------------------------------------------|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
| `accessanalyzer_enabled` | `mute_non_default_regions` | Boolean | `False` |
| `acm_certificates_expiration_check` | `days_to_expire_threshold` | Integer | `7` |
| `acm_certificates_with_secure_key_algorithms` | `insecure_key_algorithms` | List of Strings | `["RSA-1024", "P-192"]` |
| `acmpca_certificate_authority_pqc_key_algorithm` | `acmpca_pqc_key_algorithms` | List of Strings | `["ML_DSA_44", "ML_DSA_65", "ML_DSA_87"]` |
| `apigateway_domain_name_pqc_tls_enabled` | `apigateway_pqc_tls_allowed_policies` | List of Strings | `["SecurityPolicy_TLS13_1_2_FIPS_PFS_PQ_2025_09", "SecurityPolicy_TLS13_1_2_PFS_PQ_2025_09", "SecurityPolicy_TLS13_1_2_PQ_2025_09"]` |
| `apigateway_restapi_no_secrets_in_stage_variables` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `appstream_fleet_maximum_session_duration` | `max_session_duration_seconds` | Integer | `36000` |
| `appstream_fleet_session_disconnect_timeout` | `max_disconnect_timeout_in_seconds` | Integer | `300` |
| `appstream_fleet_session_idle_disconnect_timeout` | `max_idle_disconnect_timeout_in_seconds` | Integer | `600` |
| `autoscaling_find_secrets_ec2_launch_configuration` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `awslambda_function_no_secrets_in_code` | `secrets_ignore_files` | List of Strings | `[]` |
| `awslambda_function_no_secrets_in_code` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `awslambda_function_no_secrets_in_variables` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `awslambda_function_using_supported_runtimes` | `obsolete_lambda_runtimes` | List of Strings | See `config.yaml` |
| `awslambda_function_vpc_multi_az` | `lambda_min_azs` | Integer | `2` |
| `cloudformation_stack_cdktoolkit_bootstrap_version` | `recommended_cdk_bootstrap_version` | Integer | `21` |
| `cloudformation_stack_outputs_find_secrets` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `cloudfront_distributions_pqc_tls_enabled` | `cloudfront_pqc_min_protocol_versions` | List of Strings | `["TLSv1.3_2025"]` |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_actions` | List of Strings | See `config.yaml` |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_minutes` | Integer | `1440` |
| `cloudtrail_threat_detection_enumeration` | `threat_detection_enumeration_threshold` | Float | `0.3` |
| `cloudtrail_threat_detection_llm_jacking` | `threat_detection_llm_jacking_actions` | List of Strings | See `config.yaml` |
| `cloudtrail_threat_detection_llm_jacking` | `threat_detection_llm_jacking_minutes` | Integer | `1440` |
| `cloudtrail_threat_detection_llm_jacking` | `threat_detection_llm_jacking_threshold` | Float | `0.4` |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_actions` | List of Strings | See `config.yaml` |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_minutes` | Integer | `1440` |
| `cloudtrail_threat_detection_privilege_escalation` | `threat_detection_privilege_escalation_threshold` | Float | `0.2` |
| `cloudwatch_log_group_no_secrets_in_logs` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `cloudwatch_log_group_retention_policy_specific_days_enabled` | `log_group_retention_days` | Integer | `365` |
| `codebuild_project_no_secrets_in_variables` | `excluded_sensitive_environment_variables` | List of Strings | `[]` |
| `codebuild_project_no_secrets_in_variables` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `codebuild_project_uses_allowed_github_organizations` | `codebuild_github_allowed_organizations` | List of Strings | `[]` |
| `config_delegated_admin_and_org_aggregator_all_regions` | `mute_non_default_regions` | Boolean | `False` |
| `config_recorder_all_regions_enabled` | `mute_non_default_regions` | Boolean | `False` |
| `documentdb_cluster_backup_enabled` | `minimum_backup_retention_period` | Integer | `7` |
| `drs_job_exist` | `mute_non_default_regions` | Boolean | `False` |
| `dynamodb_table_cross_account_access` | `trusted_account_ids` | List of Strings | `[]` |
| `ec2_elastic_ip_shodan` | `shodan_api_key` | String | `null` |
| `ec2_instance_older_than_specific_days` | `max_ec2_instance_age_in_days` | Integer | `180` |
| `ec2_instance_secrets_user_data` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `ec2_launch_template_no_secrets` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `ec2_securitygroup_allow_ingress_from_internet_to_any_port` | `ec2_allowed_instance_owners` | List of Strings | `["amazon-elb"]` |
| `ec2_securitygroup_allow_ingress_from_internet_to_any_port` | `ec2_allowed_interface_types` | List of Strings | `["api_gateway_managed", "vpc_endpoint"]` |
| `ec2_securitygroup_allow_ingress_from_internet_to_high_risk_tcp_ports` | `ec2_high_risk_ports` | List of Integer | `[25, 110, 135, 143, 445, 3000, 4333, 5000, 5500, 8080, 8088]` |
| `ec2_securitygroup_with_many_ingress_egress_rules` | `max_security_group_rules` | Integer | `50` |
| `ecr_repositories_scan_vulnerabilities_in_latest_image` | `ecr_repository_vulnerability_minimum_severity` | String | `"MEDIUM"` |
| `ecs_service_fargate_latest_platform_version` | `fargate_linux_latest_version` | String | `"1.4.0"` |
| `ecs_service_fargate_latest_platform_version` | `fargate_windows_latest_version` | String | `"1.0.0"` |
| `ecs_task_definitions_no_environment_secrets` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `eks_cluster_uses_a_supported_version` | `eks_cluster_oldest_version_supported` | String | `"1.28"` |
| `eks_control_plane_logging_all_types_enabled` | `eks_required_log_types` | List of Strings | `["api", "audit", "authenticator", "controllerManager", "scheduler"]` |
| `elasticache_redis_cluster_backup_enabled` | `minimum_snapshot_retention_period` | Integer | `7` |
| `elb_is_in_multiple_az` | `elb_min_azs` | Integer | `2` |
| `elbv2_is_in_multiple_az` | `elbv2_min_azs` | Integer | `2` |
| `eventbridge_bus_cross_account_access` | `trusted_account_ids` | List of Strings | `[]` |
| `eventbridge_schema_registry_cross_account_access` | `trusted_account_ids` | List of Strings | `[]` |
| `glue_etl_jobs_no_secrets_in_arguments` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `guardduty_delegated_admin_enabled_all_regions` | `mute_non_default_regions` | Boolean | `False` |
| `guardduty_is_enabled` | `mute_non_default_regions` | Boolean | `False` |
| `iam_role_access_not_stale_to_bedrock` | `max_unused_bedrock_access_days` | Integer | `60` |
| `iam_user_access_not_stale_to_bedrock` | `max_unused_bedrock_access_days` | Integer | `60` |
| `iam_user_access_not_stale_to_sagemaker` | `max_unused_sagemaker_access_days` | Integer | `90` |
| `iam_user_accesskey_unused` | `max_unused_access_keys_days` | Integer | `45` |
| `iam_user_console_access_unused` | `max_console_access_days` | Integer | `45` |
| `kinesis_stream_data_retention_period` | `min_kinesis_stream_retention_hours` | Integer | `168` |
| `neptune_cluster_backup_enabled` | `minimum_backup_retention_period` | Integer | `7` |
| `opensearch_service_domains_not_publicly_accessible` | `trusted_ips` | List of Strings | `[]` |
| `organizations_delegated_administrators` | `organizations_trusted_delegated_administrators` | List of Strings | `[]` |
| `organizations_scp_check_deny_regions` | `organizations_enabled_regions` | List of Strings | `[]` |
| `rds_instance_backup_enabled` | `check_rds_instance_replicas` | Boolean | `False` |
| `rolesanywhere_trust_anchor_pqc_pki` | `rolesanywhere_pqc_pca_key_algorithms` | List of Strings | `["ML_DSA_44", "ML_DSA_65", "ML_DSA_87"]` |
| `s3_bucket_cross_account_access` | `trusted_account_ids` | List of Strings | `[]` |
| `s3_bucket_object_public` | `s3_bucket_object_public_enabled` | Boolean | `False` |
| `s3_bucket_object_public` | `s3_bucket_object_public_max_objects` | Integer | `100` |
| `s3_bucket_object_public` | `s3_bucket_object_public_sample_size` | Integer | `3` |
| `secretsmanager_has_restrictive_resource_policy` | `organizations_trusted_ids` | List of Strings | `[]` |
| `secretsmanager_secret_rotated_periodically` | `max_days_secret_unrotated` | Integer | `90` |
| `secretsmanager_secret_unused` | `max_days_secret_unused` | Integer | `90` |
| `securityhub_delegated_admin_enabled_all_regions` | `mute_non_default_regions` | Boolean | `False` |
| `securityhub_enabled` | `mute_non_default_regions` | Boolean | `False` |
| `ssm_document_secrets` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `ssm_documents_set_as_public` | `trusted_account_ids` | List of Strings | `[]` |
| `stepfunctions_statemachine_no_secrets_in_definition` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `transfer_server_pqc_ssh_kex_enabled` | `transfer_pqc_ssh_allowed_policies` | List of Strings | `["TransferSecurityPolicy-2025-03", "TransferSecurityPolicy-FIPS-2025-03", "TransferSecurityPolicy-AS2Restricted-2025-07"]` |
| `trustedadvisor_premium_support_plan_subscribed` | `verify_premium_support_plans` | Boolean | `True` |
| `vpc_endpoint_connections_trust_boundaries` | `trusted_account_ids` | List of Strings | `[]` |
| `vpc_endpoint_services_allowed_principals_trust_boundaries` | `trusted_account_ids` | List of Strings | `[]` |
### Resource Scan Limit
@@ -180,20 +203,20 @@ Secret validation makes outbound network calls that authenticate with each disco
### Configurable Checks
The following list includes all the Azure checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|---------------------------------------------------------------|--------------------------------------------------|-----------------|
| `network_public_ip_shodan` | `shodan_api_key` | String |
| `app_ensure_php_version_is_latest` | `php_latest_version` | String |
| `app_ensure_python_version_is_latest` | `python_latest_version` | String |
| `app_ensure_java_version_is_latest` | `java_latest_version` | String |
| `sqlserver_recommended_minimal_tls_version` | `recommended_minimal_tls_versions` | List of Strings |
| `vm_sufficient_daily_backup_retention_period` | `vm_backup_min_daily_retention_days` | Integer |
| `vm_desired_sku_size` | `desired_vm_sku_sizes` | List of Strings |
| `storage_smb_channel_encryption_with_secure_algorithm` | `recommended_smb_channel_encryption_algorithms` | List of Strings |
| `defender_attack_path_notifications_properly_configured` | `defender_attack_path_minimal_risk_level` | String |
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_threshold` | Float |
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_minutes` | Integer |
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_actions` | List of Strings |
| Check Name | Value | Type | Default |
|----------------------------------------------------------|-------------------------------------------------|-----------------|------------------------------------------------------------|
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_actions` | List of Strings | See `config.yaml` |
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_minutes` | Integer | `1440` |
| `apim_threat_detection_llm_jacking` | `apim_threat_detection_llm_jacking_threshold` | Float | `0.1` |
| `app_ensure_java_version_is_latest` | `java_latest_version` | String | `"17"` |
| `app_ensure_php_version_is_latest` | `php_latest_version` | String | `"8.2"` |
| `app_ensure_python_version_is_latest` | `python_latest_version` | String | `"3.12"` |
| `defender_attack_path_notifications_properly_configured` | `defender_attack_path_minimal_risk_level` | String | `"High"` |
| `network_public_ip_shodan` | `shodan_api_key` | String | `null` |
| `sqlserver_recommended_minimal_tls_version` | `recommended_minimal_tls_versions` | List of Strings | `["1.2", "1.3"]` |
| `storage_smb_channel_encryption_with_secure_algorithm` | `recommended_smb_channel_encryption_algorithms` | List of Strings | `["AES-256-GCM"]` |
| `vm_desired_sku_size` | `desired_vm_sku_sizes` | List of Strings | `["Standard_A8_v2", "Standard_DS3_v2", "Standard_D4s_v3"]` |
| `vm_sufficient_daily_backup_retention_period` | `vm_backup_min_daily_retention_days` | Integer | `7` |
## GCP
@@ -201,23 +224,28 @@ The following list includes all the Azure checks with configurable variables tha
### Configurable Checks
The following list includes all the GCP checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|---------------------------------------------------------------|--------------------------------------------------|-----------------|
| `compute_configuration_changes` | `compute_audit_log_lookback_days` | Integer |
| `compute_instance_group_multiple_zones` | `mig_min_zones` | Integer |
| Check Name | Value | Type | Default |
|---------------------------------------------------|-----------------------------------|---------|---------|
| `cloudstorage_bucket_sufficient_retention_period` | `storage_min_retention_days` | Integer | `90` |
| `compute_instance_group_multiple_zones` | `mig_min_zones` | Integer | `2` |
| `compute_public_address_shodan` | `shodan_api_key` | String | `null` |
| `compute_snapshot_not_outdated` | `max_snapshot_age_days` | Integer | `90` |
| `iam_sa_user_managed_key_unused` | `max_unused_account_days` | Integer | `180` |
| `iam_service_account_unused` | `max_unused_account_days` | Integer | `180` |
| `secretmanager_secret_rotation_enabled` | `secretmanager_max_rotation_days` | Integer | `90` |
## Kubernetes
### Configurable Checks
The following list includes all the Kubernetes checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|---------------------------------------------------------------|--------------------------------------------------|-----------------|
| `audit_log_maxbackup` | `audit_log_maxbackup` | String |
| `audit_log_maxsize` | `audit_log_maxsize` | String |
| `audit_log_maxage` | `audit_log_maxage` | String |
| `apiserver_strong_ciphers` | `apiserver_strong_ciphers` | String |
| `kubelet_strong_ciphers_only` | `kubelet_strong_ciphers` | String |
| Check Name | Value | Type | Default |
|-------------------------------------|----------------------------|-----------------|----------------------------------------------------------------------------------------|
| `apiserver_audit_log_maxage_set` | `audit_log_maxage` | Integer | `30` |
| `apiserver_audit_log_maxbackup_set` | `audit_log_maxbackup` | Integer | `10` |
| `apiserver_audit_log_maxsize_set` | `audit_log_maxsize` | Integer | `100` |
| `apiserver_strong_ciphers_only` | `apiserver_strong_ciphers` | List of Strings | `["TLS_AES_128_GCM_SHA256", "TLS_AES_256_GCM_SHA384", "TLS_CHACHA20_POLY1305_SHA256"]` |
| `kubelet_strong_ciphers_only` | `kubelet_strong_ciphers` | List of Strings | See `config.yaml` |
## M365
@@ -225,11 +253,13 @@ The following list includes all the Kubernetes checks with configurable variable
### Configurable Checks
The following list includes all the Microsoft 365 checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|---------------------------------------------------------------|--------------------------------------------------|-----------------|
| `entra_admin_users_sign_in_frequency_enabled` | `sign_in_frequency` | Integer |
| `teams_external_file_sharing_restricted` | `allowed_cloud_storage_services` | List of Strings |
| `exchange_organization_mailtips_enabled` | `recommended_mailtips_large_audience_threshold` | Integer |
| Check Name | Value | Type | Default |
|--------------------------------------------------------------------|-------------------------------------------------|-----------------|--------------------|
| `defender_malware_policy_comprehensive_attachments_filter_applied` | `recommended_blocked_file_types` | List of Strings | See check defaults |
| `entra_admin_users_sign_in_frequency_enabled` | `sign_in_frequency` | Integer | `4` |
| `exchange_organization_mailtips_enabled` | `recommended_mailtips_large_audience_threshold` | Integer | `25` |
| `exchange_user_mailbox_auditing_enabled` | `audit_log_age` | Integer | `90` |
| `teams_external_file_sharing_restricted` | `allowed_cloud_storage_services` | List of Strings | `[]` |
## GitHub
@@ -237,35 +267,76 @@ The following list includes all the Microsoft 365 checks with configurable varia
### Configurable Checks
The following list includes all the GitHub checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|--------------------------------------------|---------------------------------------------|---------|
| `repository_inactive_not_archived` | `inactive_not_archived_days_threshold` | Integer |
| Check Name | Value | Type | Default |
|------------------------------------|----------------------------------------|---------|---------|
| `repository_inactive_not_archived` | `inactive_not_archived_days_threshold` | Integer | `180` |
## Vercel
### Configurable Checks
The following list includes all the Vercel checks with configurable variables that can be changed in the configuration YAML file:
| Check Name | Value | Type |
|-----------------------------------------------------|------------------------------------|-----------------|
| `authentication_no_stale_tokens` | `stale_token_threshold_days` | Integer |
| `authentication_token_not_expired` | `days_to_expire_threshold` | Integer |
| `deployment_production_uses_stable_target` | `stable_branches` | List of Strings |
| `domain_ssl_certificate_valid` | `days_to_expire_threshold` | Integer |
| `project_environment_no_secrets_in_plain_type` | `secret_suffixes` | List of Strings |
| `team_member_role_least_privilege` | `max_owner_percentage` | Integer |
| `team_member_role_least_privilege` | `max_owners` | Integer |
| `team_no_stale_invitations` | `stale_invitation_threshold_days` | Integer |
| Check Name | Value | Type | Default |
|------------------------------------------------|-----------------------------------|-----------------|--------------------------------------------------------------------------|
| `authentication_no_stale_tokens` | `stale_token_threshold_days` | Integer | `90` |
| `authentication_token_not_expired` | `days_to_expire_threshold` | Integer | `7` |
| `deployment_production_uses_stable_target` | `stable_branches` | List of Strings | `["main", "master"]` |
| `domain_ssl_certificate_valid` | `days_to_expire_threshold` | Integer | `7` |
| `project_environment_no_secrets_in_plain_type` | `secret_suffixes` | List of Strings | `["_KEY", "_SECRET", "_TOKEN", "_PASSWORD", "_API_KEY", "_PRIVATE_KEY"]` |
| `team_member_role_least_privilege` | `max_owner_percentage` | Integer | `20` |
| `team_member_role_least_privilege` | `max_owners` | Integer | `3` |
| `team_no_stale_invitations` | `stale_invitation_threshold_days` | Integer | `30` |
## Okta
### Configurable Checks
The following list includes all the Okta checks with configurable variables that can be changed in the configuration YAML file:
| Check Name | Value | Type |
|---------------------------------------------------------------|------------------------------------|---------|
| `application_admin_console_session_idle_timeout_15min` | `okta_admin_console_idle_timeout_max_minutes` | Integer |
| `signon_global_session_idle_timeout_15min` | `okta_max_session_idle_minutes` | Integer |
| Check Name | Value | Type | Default |
|--------------------------------------------------------|-----------------------------------------------|-----------------|---------|
| `application_admin_console_session_idle_timeout_15min` | `okta_admin_console_idle_timeout_max_minutes` | Integer | `15` |
| `idp_smart_card_dod_approved_ca` | `okta_dod_approved_ca_issuer_patterns` | List of Strings | `[]` |
| `signon_global_session_idle_timeout_15min` | `okta_max_session_idle_minutes` | Integer | `15` |
| `signon_global_session_lifetime_18h` | `okta_max_session_lifetime_minutes` | Integer | `1080` |
| `user_inactivity_automation_35d_enabled` | `okta_user_inactivity_max_days` | Integer | `35` |
## Alibaba Cloud
### Configurable Checks
The following list includes all the Alibaba Cloud checks with configurable variables that can be changed in the configuration YAML file:
| Check Name | Value | Type | Default |
|--------------------------------------|--------------------------------|---------|---------|
| `cs_kubernetes_cluster_check_recent` | `max_cluster_check_days` | Integer | `7` |
| `cs_kubernetes_cluster_check_weekly` | `max_cluster_check_days` | Integer | `7` |
| `ram_user_console_access_unused` | `max_console_access_days` | Integer | `90` |
| `rds_instance_sql_audit_retention` | `min_rds_audit_retention_days` | Integer | `180` |
| `sls_logstore_retention_period` | `min_log_retention_days` | Integer | `365` |
## MongoDB Atlas
### Configurable Checks
The following list includes all the MongoDB Atlas checks with configurable variables that can be changed in the configuration YAML file:
| Check Name | Value | Type | Default |
|----------------------------------------------------|---------------------------------------------|---------|---------|
| `organizations_service_account_secrets_expiration` | `max_service_account_secret_validity_hours` | Integer | `8` |
## OpenStack
### Configurable Checks
The following list includes all the OpenStack checks with configurable variables that can be changed in the configuration YAML file:
| Check Name | Value | Type | Default |
|---------------------------------------------------|---------------------------|-----------------|---------|
| `blockstorage_snapshot_metadata_sensitive_data` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `blockstorage_volume_metadata_sensitive_data` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `compute_instance_metadata_sensitive_data` | `secrets_ignore_patterns` | List of Strings | `[]` |
| `image_not_shared_with_multiple_projects` | `image_sharing_threshold` | Integer | `5` |
| `objectstorage_container_metadata_sensitive_data` | `secrets_ignore_patterns` | List of Strings | `[]` |
## Config YAML File Structure
@@ -210,7 +210,9 @@ For more detailed guidance on subscription management and permissions:
The following security checks require the `ProwlerRole` permissions for execution. Ensure the role is assigned to the identity assumed by Prowler before running these checks:
- `app_function_access_keys_configured`
- `app_function_application_insights_enabled`
- `app_function_ftps_deployment_disabled`
- `app_function_latest_runtime_version`
---
@@ -17,7 +17,7 @@ prowler azure --az-cli-auth --azure-resource-group <resource-group-1> <resource-
For example, to scan only `rg-production` and `rg-staging`:
```console
prowler azure --az-cli-auth --azure-resource-group rg-prod1 rg-prod2
prowler azure --az-cli-auth --azure-resource-group rg-production rg-staging
```
This works with all supported authentication methods:
@@ -23,6 +23,10 @@ title: 'Getting Started with Kubernetes'
For Kubernetes, Prowler App uses a `kubeconfig` file to authenticate. Paste the contents of your `kubeconfig` file into the `Kubeconfig content` field.
<Note>
Kubeconfigs that use `users[].user.exec` authentication are not supported in Prowler Cloud/App. For security reasons, Prowler Cloud does not run commands declared by uploaded kubeconfigs. Use kubeconfig credentials that do not rely on `exec` authentication, such as the ServiceAccount token flow documented below.
</Note>
By default, the `kubeconfig` file is located at `~/.kube/config`.
![Kubernetes Credentials](/images/kubernetes-credentials.png)
@@ -21,13 +21,13 @@ By mapping these relationships as a graph, Attack Paths reveals risks that indiv
The following prerequisites are required for Attack Paths:
- **An AWS provider is configured** with valid credentials in Prowler App. For setup instructions, see [Getting Started with AWS](/user-guide/providers/aws/getting-started-aws).
- **At least one scan has completed** on the configured AWS provider. Attack Paths scans run automatically alongside regular security scans, no separate configuration is required.
- **At least one scan has completed** on the configured AWS provider and produced graph data. Attack Paths scans run automatically alongside regular security scans, no separate configuration is required.
## How Attack Paths Scans Work
Attack Paths scans are generated automatically when a security scan runs on an AWS provider. Each completed scan produces graph data that maps relationships between IAM principals, policies, trust configurations, and other resources.
Once the scan finishes and the graph data is ready, the scan appears in the Attack Paths scan table with a **Completed** status. Scans that are still processing display as **Executing** or **Scheduled**.
Once the scan finishes and graph data is ready, the scan appears in the Attack Paths scan table with a **Completed** status and a check in the **Graph** column. Scans that are still queued or running remain visible, but they cannot be selected until graph data is ready.
<Note>
Since Prowler scans all configured providers every **24 hours** by default,
@@ -41,25 +41,28 @@ To open Attack Paths, click **Attack Paths** in the left navigation menu.
<img
src="/images/prowler-app/attack-paths/navigation.png"
alt="Attack Paths navigation menu entry"
width="700"
width="320"
/>
The main interface is divided into two areas:
The Attack Paths page guides you through the workflow on one page:
- **Left panel:** A table listing all available Attack Paths scans
- **Right panel:** The query selector, parameter form, and execute controls
- Select a scan with graph data.
- Choose a built-in query or a custom openCypher query.
- Add parameters when the selected query requires them.
- Execute the query and explore the resulting graph.
## Selecting a Scan
The scans table displays all Attack Paths scans with the following columns:
- **Provider / Account:** The AWS provider alias and account identifier
- **Last scan date:** When the scan completed
- **Status:** Current state of the scan (Completed, Executing, Scheduled, or Failed)
- **Progress:** Completion percentage for in-progress scans
- **Duration:** Total scan time
- **Select:** A radio button used to choose a scan. The radio button is disabled when graph data is not available.
- **Provider:** The AWS provider alias and account identifier.
- **Last Scan Date:** When the scan completed.
- **Status:** Current state of the scan, such as **Completed**, **Executing**, **Scheduled**, or **Failed**.
- **Graph:** Whether Attack Paths graph data is available for the scan.
- **Duration:** Total scan time.
To select a scan for analysis, click **Select** on any row with a **Completed** status.
To select a scan for analysis, click the radio button on any row with a **Completed** status and available graph data.
<img
src="/images/prowler-app/attack-paths/scan-list-table.png"
@@ -68,19 +71,18 @@ To select a scan for analysis, click **Select** on any row with a **Completed**
/>
<Note>
Only scans with a **Completed** status and ready graph data can be selected.
Scans that are still executing or have failed appear with disabled action
buttons.
Only scans with graph data can be selected. Disabled rows include a tooltip
that explains why the graph is not available yet.
</Note>
## Choosing a Query
After selecting a scan, the right panel activates a query dropdown. Each query targets a specific type of privilege escalation or misconfiguration pattern.
After selecting a scan, the query selector becomes available. Each query targets a specific privilege escalation, exposure, inventory, or misconfiguration pattern.
To choose a query, click the dropdown and select from the available options. Each option displays:
- **Query name:** A descriptive title (e.g., "IAM Privilege Escalation via AssumeRole")
- **Short description:** A brief summary of what the query detects
- **Query name:** A descriptive title, such as **Internet-Exposed EC2 with Sensitive S3 Access**.
- **Short description:** A brief summary of what the query detects.
<img
src="/images/prowler-app/attack-paths/query-selector.png"
@@ -88,16 +90,17 @@ To choose a query, click the dropdown and select from the available options. Eac
width="700"
/>
Once selected, a description card appears below the dropdown with additional context about the query, including attribution links to external references when available.
Once selected, a description panel appears below the dropdown with more context about the query.
## Configuring Query Parameters
Some queries accept optional or required parameters to narrow the scope of the analysis. When a query has parameters, a dynamic form appears below the query description.
Some queries accept optional or required parameters to narrow the scope of the analysis. When a query has parameters, a form appears below the query description.
- **Required fields** are marked with an asterisk (\*) and must be filled before executing
- **Optional fields** refine the query results but are not mandatory
- **Required fields** are marked with an asterisk (\*) and must be filled before executing.
- **Optional fields** refine the query results but are not mandatory.
- Queries without parameters show no parameter form.
If a query requires no parameters, the form displays a message confirming that the query is ready to execute.
For example, **Internet-Exposed EC2 with Sensitive S3 Access** uses **Tag key** and **Tag value** fields to identify sensitive S3 buckets.
<img
src="/images/prowler-app/attack-paths/query-parameters.png"
@@ -117,7 +120,7 @@ Custom queries are sandboxed to keep the graph database safe and responsive:
- **Read-only:** Only read operations are allowed. Statements that mutate the graph (`CREATE`, `MERGE`, `SET`, `DELETE`, `REMOVE`, `DROP`, `LOAD CSV`, `CALL { ... }` writes, etc.) are rejected before execution.
- **Length limit:** Each query is capped at **10,000 characters**.
- **Scoped to the selected scan:** Results are automatically scoped to the provider and scan selected on the left panel. There is no need to filter by tenant or scan identifier in the query body.
- **Scoped to the selected scan:** Results are automatically scoped to the provider and scan selected in the scan table. There is no need to filter by tenant or scan identifier in the query body.
### Example Queries
@@ -284,53 +287,75 @@ In addition to the upstream schema, Prowler enriches the graph with:
## Executing a Query
To run the selected query against the scan data, click **Execute Query**. The button displays a loading state while the query processes.
To run the selected query against the scan data, click **Execute Query**. The button is disabled until a query is selected and all required parameters are valid.
The button displays a loading state while the query runs. After the query completes, the graph appears below the query builder.
If the query returns no results, an informational message appears. Common reasons include:
- **No matching patterns found:** The scanned environment does not contain the privilege escalation chain the query targets
- **Insufficient permissions:** The scan credentials may not have captured all the data the query needs
- **No matching patterns found:** The scanned environment does not contain the pattern the query targets.
- **Not enough permissions:** The scan credentials may not have captured all the data the query needs.
- **Server unavailable:** The graph service may be temporarily unavailable.
<img
src="/images/prowler-app/attack-paths/execute-query.png"
alt="Attack Paths right panel with query selected and execute button"
alt="Attack Paths query builder with query selected and execute button"
width="700"
/>
## Exploring the Graph
After a successful execution, the graph visualization renders below the query builder in a full-width panel. The graph maps relationships between cloud resources, IAM entities, and security findings.
After a successful execution, the graph visualization renders below the query builder. The graph maps relationships between cloud resources, IAM entities, public exposure, and security findings.
### Node Types
- **Resource nodes** (rounded pills): Represent cloud resources such as IAM roles, policies, EC2 instances, and S3 buckets. Each resource type has a distinct color.
- **Finding nodes** (hexagons): Represent Prowler security findings linked to resources in the graph. Colors indicate severity level (critical, high, medium, low).
- **Provider root nodes:** Represent the AWS account or provider root for the selected scan.
- **Resource nodes:** Represent cloud resources such as IAM roles, policies, EC2 instances, security groups, and S3 buckets.
- **Internet nodes:** Represent exposure from the public internet.
- **Finding nodes:** Represent Prowler findings linked to resources. Finding colors indicate risk level, such as critical, high, medium, or low.
### Edge Types
- **Solid lines:** Direct relationships between resources (e.g., a role attached to a policy)
- **Dashed lines:** Connections between resources and their associated findings
- **Normal edges:** Direct relationships between graph nodes, such as role-to-policy or resource-to-security-group relationships.
- **Finding edges:** Dashed relationships between resources and their associated findings.
- **Highlighted paths:** Green edges that show the active path when you hover a node or focus a finding.
A **legend** at the bottom of the graph lists all node types and edge types present in the current view.
The standard graph view includes a minimap and a legend below the canvas. The legend shows the provider roots, visible node types, finding risk levels, node states, and edge types present in the current view.
<img
src="/images/prowler-app/attack-paths/graph-visualization.png"
alt="Attack Paths graph showing nodes, edges, and legend"
alt="Attack Paths graph showing nodes and edges"
width="700"
/>
## Interacting with the Graph
### Filtering by Node
The graph banner describes the main interactions:
Click any node in the graph to filter the view and display only paths that pass through that node. When a filter is active:
- Click a finding to focus its connected path.
- Click a resource with findings to show or hide its related findings.
- Hover a node to highlight its connected path.
- An information banner shows which node is selected
- Click **Back to Full View** to restore the complete graph
### Showing Related Findings
Resource nodes with related findings are clickable. Click one of these resources to show its finding nodes. Click the resource again to hide them.
The graph automatically fits the selected resource and its related findings when the findings are shown.
### Focusing a Finding Path
Click a finding node to focus the graph on the path connected to that finding. When the graph is focused:
- The graph shows **Back to Full View**.
- The status banner shows the selected finding.
- The graph keeps only the connected path in view.
- The finding detail drawer opens.
After you close the drawer, the graph remains focused on the selected path.
<img
src="/images/prowler-app/attack-paths/graph-filtered.png"
alt="Attack Paths graph filtered to show paths through a selected node"
alt="Attack Paths graph focused on a selected finding path"
width="700"
/>
@@ -339,29 +364,29 @@ Click any node in the graph to filter the view and display only paths that pass
The toolbar in the top-right corner of the graph provides:
- **Zoom in / Zoom out:** Adjust the zoom level
- **Fit to screen:** Reset the view to fit all nodes
- **Export:** Download the current graph as an SVG file
- **Fullscreen:** Open the graph in a full-screen modal with a side-by-side node detail panel
- **Fit graph to view:** Reset the view to fit the visible graph
- **Export graph:** Download the current graph as a PNG file
- **Fullscreen:** Open the graph in a full-size modal
<Note>
Use **Ctrl + Scroll** (or **Cmd + Scroll** on macOS) to zoom directly within
the graph area.
</Note>
## Viewing Node Details
## Viewing Finding Details
Click any node to open the **Node Details** panel below the graph. This panel displays:
Click a finding node to open the finding detail drawer. The drawer uses the same finding detail layout as the Findings page and includes:
- **Node type:** The resource category (e.g., "IAM Role," "EC2 Instance")
- **Properties:** All attributes of the selected node, including identifiers, timestamps, and configuration details
- **Related findings** (for resource nodes): A list of Prowler findings linked to the resource, with severity, title, and status
- **Affected resources** (for finding nodes): A list of resources associated with the finding
- The finding title, status, and severity.
- The affected resource summary.
- Overview, remediation, evidence, related findings, scans, and events tabs when data is available.
- A Lighthouse AI action when the account has access to Lighthouse AI.
For finding nodes, a "View Finding" button links directly to the finding detail page for further investigation.
Resource nodes do not open a node detail panel. When a resource has related findings, clicking it expands or collapses those finding nodes in the graph.
<img
src="/images/prowler-app/attack-paths/node-details.png"
alt="Attack Paths node detail panel showing properties and related findings"
alt="Attack Paths finding detail drawer"
width="700"
/>
@@ -369,16 +394,28 @@ For finding nodes, a "View Finding" button links directly to the finding detail
To expand the graph for detailed exploration, click the fullscreen icon in the graph toolbar. The fullscreen modal provides:
- The full graph visualization with all zoom and export controls
- A side panel for node details that appears when a node is selected
- All filtering and interaction capabilities available in the standard view
- The graph in a full-size modal.
- The same zoom, fit, and export controls.
- The same node expansion, finding focus, hover highlight, and minimap interactions available in the standard view.
<img
src="/images/prowler-app/attack-paths/fullscreen-mode.png"
alt="Attack Paths fullscreen mode with graph and node detail side panel"
alt="Attack Paths fullscreen graph mode"
width="700"
/>
## Available Queries
The query selector includes custom openCypher and built-in AWS queries for common security investigation workflows. Available queries are loaded from the selected scan and may change as new query packs are added.
Available queries include:
- **Custom openCypher query:** Write and run a read-only graph query.
- **Exposure queries:** Find internet-exposed EC2 instances, load balancers, open security groups, and resources by public IP.
- **Inventory queries:** List resources such as RDS instances.
- **Misconfiguration queries:** Find unencrypted RDS instances, public S3 buckets, and wildcard IAM statements.
- **Privilege escalation queries:** Detect IAM and AWS service paths based on known attack techniques, including queries based on [pathfinding.cloud](https://pathfinding.cloud) research by Datadog.
## Using Attack Paths with the MCP Server and Lighthouse AI
Attack Paths capabilities are also available through the [Prowler MCP Server](/getting-started/products/prowler-mcp), enabling interaction with Attack Paths data via AI assistants like Claude Desktop, Cursor, and other MCP clients.
@@ -387,10 +424,10 @@ Attack Paths capabilities are also available through the [Prowler MCP Server](/g
The following MCP tools are available for Attack Paths:
- **`prowler_app_list_attack_paths_scans`** - List and filter Attack Paths scans
- **`prowler_app_list_attack_paths_queries`** - Discover available queries for a completed scan
- **`prowler_app_run_attack_paths_query`** - Execute a query and retrieve graph results with nodes and relationships
- **`prowler_app_get_attack_paths_cartography_schema`** - Retrieve the Cartography graph schema for custom openCypher queries
- **`prowler_app_list_attack_paths_scans`** - List and filter Attack Paths scans.
- **`prowler_app_list_attack_paths_queries`** - Discover available queries for a completed scan.
- **`prowler_app_run_attack_paths_query`** - Execute a query and retrieve graph results with nodes and relationships.
- **`prowler_app_get_attack_paths_cartography_schema`** - Retrieve the Cartography graph schema for custom openCypher queries.
### Example Questions
@@ -405,110 +442,6 @@ Ask through the MCP Server or Lighthouse AI:
- "Are there any CloudFormation stacks that could be hijacked for privilege escalation?"
- "Show me all roles that can be assumed for lateral movement"
### Supported Queries
Attack Paths currently supports the following built-in queries for AWS:
#### Custom Attack Path Queries
| Query | Description |
| ------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| **Internet-Exposed EC2 with Sensitive S3 Access** | Find SSH-exposed EC2 instances that can assume roles to read tagged sensitive S3 buckets |
#### Basic Resource Queries
| Query | Description |
| ------------------------------------------- | ------------------------------------------------------------------- |
| **RDS Instances Inventory** | List all provisioned RDS database instances in the account |
| **Unencrypted RDS Instances** | Find RDS instances with storage encryption disabled |
| **S3 Buckets with Anonymous Access** | Find S3 buckets that allow anonymous access |
| **IAM Statements Allowing All Actions** | Find IAM policy statements that allow all actions via wildcard (\*) |
| **IAM Statements Allowing Policy Deletion** | Find IAM policy statements that allow iam:DeletePolicy |
| **IAM Statements Allowing Create Actions** | Find IAM policy statements that allow any create action |
#### Network Exposure Queries
| Query | Description |
| ----------------------------------------------------- | ----------------------------------------------------------------------------------- |
| **Internet-Exposed EC2 Instances** | Find EC2 instances flagged as exposed to the internet |
| **Open Security Groups on Internet-Facing Resources** | Find internet-facing resources with security groups allowing inbound from 0.0.0.0/0 |
| **Internet-Exposed Classic Load Balancers** | Find Classic Load Balancers exposed to the internet with their listeners |
| **Internet-Exposed ALB/NLB Load Balancers** | Find ELBv2 (ALB/NLB) load balancers exposed to the internet with their listeners |
| **Resource Lookup by Public IP** | Find the AWS resource associated with a given public IP address |
#### Privilege Escalation Queries
These queries are based on research from [pathfinding.cloud](https://pathfinding.cloud) by Datadog.
| Query | Description |
| -------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **App Runner Service Creation with Privileged Role (APPRUNNER-001)** | Create an App Runner service with a privileged IAM role to gain its permissions |
| **App Runner Service Update for Role Access (APPRUNNER-002)** | Update an existing App Runner service to leverage its already-attached privileged role |
| **Bedrock Code Interpreter with Privileged Role (BEDROCK-001)** | Create a Bedrock AgentCore Code Interpreter with a privileged role attached |
| **Bedrock Code Interpreter Session Hijacking (BEDROCK-002)** | Start a session on an existing Bedrock code interpreter to exfiltrate its privileged role credentials |
| **CloudFormation Stack Creation with Privileged Role (CLOUDFORMATION-001)** | Create a CloudFormation stack with a privileged role to provision arbitrary AWS resources |
| **CloudFormation Stack Update for Role Access (CLOUDFORMATION-002)** | Update an existing CloudFormation stack to leverage its already-attached privileged service role |
| **CloudFormation StackSet Creation with Privileged Role (CLOUDFORMATION-003)** | Create a CloudFormation StackSet with a privileged execution role to provision arbitrary resources across accounts |
| **CloudFormation StackSet Update with Privileged Role (CLOUDFORMATION-004)** | Update an existing CloudFormation StackSet to inject malicious resources using a privileged execution role |
| **CloudFormation Change Set Privilege Escalation (CLOUDFORMATION-005)** | Create and execute a change set on an existing stack to leverage its privileged service role |
| **CodeBuild Project Creation with Privileged Role (CODEBUILD-001)** | Create a CodeBuild project with a privileged role to execute arbitrary code via a malicious buildspec |
| **CodeBuild Buildspec Override for Role Access (CODEBUILD-002)** | Start a build on an existing CodeBuild project with a buildspec override to execute code with its privileged role |
| **CodeBuild Batch Buildspec Override for Role Access (CODEBUILD-003)** | Start a batch build on an existing CodeBuild project with a buildspec override to execute code with its privileged role |
| **CodeBuild Batch Project Creation with Privileged Role (CODEBUILD-004)** | Create a CodeBuild project configured for batch builds with a privileged role to execute arbitrary code via a malicious buildspec |
| **Data Pipeline Creation with Privileged Role (DATAPIPELINE-001)** | Create a Data Pipeline with a privileged role to execute arbitrary commands on provisioned infrastructure |
| **EC2 Instance Launch with Privileged Role (EC2-001)** | Launch EC2 instances with privileged IAM roles to gain their permissions via IMDS |
| **EC2 Role Hijacking via UserData Injection (EC2-002)** | Inject malicious scripts into EC2 instance userData to gain the attached role's permissions |
| **Spot Instance Launch with Privileged Role (EC2-003)** | Launch EC2 Spot Instances with privileged IAM roles to gain their permissions via IMDS |
| **Launch Template Poisoning for Role Access (EC2-004)** | Inject malicious userData into launch templates that reference privileged roles, no PassRole needed |
| **EC2 Instance Connect SSH Access for Role Credentials (EC2INSTANCECONNECT-003)** | Push a temporary SSH key to an EC2 instance via Instance Connect to access its attached role credentials through IMDS |
| **ECS Service Creation with Privileged Role (ECS-001 - New Cluster)** | Create an ECS cluster and service with a privileged Fargate task role to execute arbitrary code |
| **ECS Task Execution with Privileged Role (ECS-002 - New Cluster)** | Create an ECS cluster and run a one-off Fargate task with a privileged role to execute arbitrary code |
| **ECS Service Creation with Privileged Role (ECS-003 - Existing Cluster)** | Deploy a Fargate service with a privileged role on an existing ECS cluster |
| **ECS Task Execution with Privileged Role (ECS-004 - Existing Cluster)** | Run a one-off Fargate task with a privileged role on an existing ECS cluster |
| **ECS Task Start with Privileged Role on EC2 (ECS-005 - Existing Cluster)** | Register a task definition with a privileged role and start it on an EC2 container instance to execute arbitrary code |
| **ECS Exec Container Hijacking for Role Credentials (ECS-006)** | Shell into a running ECS container via ECS Exec to steal the attached task role's credentials |
| **Glue Dev Endpoint with Privileged Role (GLUE-001)** | Create a Glue development endpoint with a privileged role attached to gain its permissions |
| **Glue Dev Endpoint SSH Hijacking via Update (GLUE-002)** | Update an existing Glue development endpoint to inject an SSH public key and access its attached role credentials |
| **Glue Job Creation with Privileged Role (GLUE-003)** | Create a Glue job with a privileged role and start it to execute arbitrary code with that role's permissions |
| **Glue Job Creation with Scheduled Trigger and Privileged Role (GLUE-004)** | Create a Glue job with a privileged role and a scheduled trigger to persistently execute arbitrary code |
| **Glue Job Hijacking via Update with Privileged Role (GLUE-005)** | Update an existing Glue job to attach a privileged role and inject malicious code, then start it to gain that role's permissions |
| **Glue Job Hijacking with Scheduled Trigger and Privileged Role (GLUE-006)** | Update an existing Glue job to attach a privileged role and inject malicious code, then create a scheduled trigger for persistent automated execution |
| **Policy Version Override for Self-Escalation (IAM-001)** | Create a new version of an attached policy with administrative permissions, instantly escalating the principal's own privileges |
| **Access Key Creation for Lateral Movement (IAM-002)** | Create access keys for other IAM users to gain their permissions and move laterally across the account |
| **Access Key Rotation Attack for Lateral Movement (IAM-003)** | Delete and recreate access keys for other IAM users to bypass the two-key limit and gain their permissions |
| **Console Login Profile Creation for Lateral Movement (IAM-004)** | Create console login profiles for other IAM users to access the AWS Console with their permissions |
| **Inline Policy Injection for Self-Escalation (IAM-005)** | Attach an inline policy with administrative permissions to your own role, instantly escalating privileges |
| **Console Password Override for Lateral Movement (IAM-006)** | Change the console password of other IAM users to log in as them and gain their permissions |
| **Inline Policy Injection on User for Self-Escalation (IAM-007)** | Attach an inline policy with administrative permissions to your own IAM user, instantly escalating privileges |
| **Managed Policy Attachment on User for Self-Escalation (IAM-008)** | Attach existing managed policies with administrative permissions to your own IAM user, instantly escalating privileges |
| **Managed Policy Attachment on Role for Self-Escalation (IAM-009)** | Attach existing managed policies with administrative permissions to your own IAM role, instantly escalating privileges |
| **Managed Policy Attachment on Group for Self-Escalation (IAM-010)** | Attach existing managed policies with administrative permissions to a group you belong to, escalating privileges for all group members |
| **Inline Policy Injection on Group for Self-Escalation (IAM-011)** | Attach an inline policy with administrative permissions to a group you belong to, escalating privileges for all group members |
| **Trust Policy Hijacking for Role Assumption (IAM-012)** | Modify a role's trust policy to allow yourself to assume it, gaining the role's permissions |
| **Group Membership Hijacking for Privilege Escalation (IAM-013)** | Add yourself to a privileged IAM group to inherit its permissions, gaining access to all policies attached to the group |
| **Managed Policy Attachment with Role Assumption for Lateral Movement (IAM-014)** | Attach administrative managed policies to another role you can assume, then assume it to gain elevated privileges |
| **Managed Policy Attachment with Access Key Creation for Lateral Movement (IAM-015)** | Attach administrative managed policies to another IAM user and create access keys for them to gain programmatic access with elevated privileges |
| **Policy Version Override with Role Assumption for Lateral Movement (IAM-016)** | Create a new version of a customer-managed policy attached to another role with administrative permissions, then assume that role to gain elevated access |
| **Inline Policy Injection with Role Assumption for Lateral Movement (IAM-017)** | Attach an inline policy with administrative permissions to another role you can assume, then assume it to gain elevated privileges |
| **Inline Policy Injection with Access Key Creation for Lateral Movement (IAM-018)** | Attach an inline policy with administrative permissions to another IAM user and create access keys for them to gain programmatic access with elevated privileges |
| **Managed Policy Attachment with Trust Policy Hijacking for Privilege Escalation (IAM-019)** | Attach administrative managed policies to a role and modify its trust policy to allow yourself to assume it, gaining elevated privileges without prior assume-role access |
| **Policy Version Override with Trust Policy Hijacking for Privilege Escalation (IAM-020)** | Create a new version of a customer-managed policy attached to a role with administrative permissions and modify its trust policy to assume it, without prior assume-role access |
| **Inline Policy Injection with Trust Policy Hijacking for Privilege Escalation (IAM-021)** | Add an inline policy with administrative permissions to a role and modify its trust policy to allow yourself to assume it, gaining elevated privileges without prior assume-role access |
| **Lambda Function Creation with Privileged Role (LAMBDA-001)** | Create a Lambda function with a privileged IAM role and invoke it to execute code with that role's permissions |
| **Lambda Function Creation with Event Source Trigger (LAMBDA-002)** | Create a Lambda function with a privileged IAM role and an event source mapping to trigger it automatically, executing code with the role's permissions |
| **Lambda Function Code Injection (LAMBDA-003)** | Modify the code of an existing Lambda function to execute arbitrary commands with the function's execution role permissions |
| **Lambda Function Code Injection with Direct Invocation (LAMBDA-004)** | Modify the code of an existing Lambda function and invoke it directly to execute arbitrary commands with the function's execution role permissions |
| **Lambda Function Code Injection with Resource Policy Grant (LAMBDA-005)** | Modify the code of an existing Lambda function and grant yourself invocation permission via its resource-based policy to execute code with the function's execution role |
| **Lambda Function Creation with Resource Policy Invocation (LAMBDA-006)** | Create a Lambda function with a privileged IAM role and grant yourself invocation permission via its resource-based policy to execute code with the role's permissions |
| **SageMaker Notebook Creation with Privileged Role (SAGEMAKER-001)** | Create a SageMaker notebook instance with a privileged IAM role to execute arbitrary code with the role's permissions via the Jupyter environment |
| **SageMaker Training Job Creation with Privileged Role (SAGEMAKER-002)** | Create a SageMaker training job with a privileged IAM role to execute arbitrary container code with the role's permissions |
| **SageMaker Processing Job Creation with Privileged Role (SAGEMAKER-003)** | Create a SageMaker processing job with a privileged IAM role to execute arbitrary container code with the role's permissions |
| **SageMaker Presigned Notebook URL for Privilege Escalation (SAGEMAKER-004)** | Generate a presigned URL to access an existing SageMaker notebook instance and execute code with its execution role's permissions |
| **SageMaker Notebook Lifecycle Config Injection (SAGEMAKER-005)** | Inject a malicious lifecycle configuration into an existing SageMaker notebook to execute code with the notebook's execution role during startup |
| **SSM Session Access for EC2 Role Credentials (SSM-001)** | Start an SSM session on an EC2 instance to access its attached role credentials through IMDS |
| **SSM Send Command for EC2 Role Credentials (SSM-002)** | Execute commands on an EC2 instance via SSM Run Command to access its attached role credentials through IMDS |
| **Role Assumption for Privilege Escalation (STS-001)** | Assume IAM roles with elevated permissions by exploiting bidirectional trust between the starting principal and the target role |
These tools enable workflows such as:
- Asking an AI assistant to identify privilege escalation paths in a specific AWS account
@@ -32,13 +32,13 @@ All three providers can be configured for a tenant, but only one can be set as t
When visiting Lighthouse AI chat, the default provider's default model loads automatically. Users can switch to any available LLM model (including those from non-default providers) using the dropdown in chat.
<img src="/images/prowler-app/lighthouse-switch-models.png" alt="Switch models in Lighthouse AI chat interface" />
<img src="/images/prowler-app/lighthouse/oss/switch-models.png" alt="Switch models in Lighthouse AI chat interface" />
## Configuring Providers
Navigate to **Configuration** → **Lighthouse AI** to see all three provider options with a **Connect** button under each.
<img src="/images/prowler-app/lighthouse-configuration.png" alt="Prowler Lighthouse Configuration" />
<img src="/images/prowler-app/lighthouse/oss/configuration.png" alt="Prowler Lighthouse Configuration" />
### Connecting a Provider
@@ -128,6 +128,25 @@ To connect a provider:
3. Configure in Lighthouse AI:
- **API Key**: OpenRouter API key
- **Base URL**: `https://openrouter.ai/api/v1`
### Base URL Validation
To prevent server-side request forgery (SSRF), Prowler API validates the base URL before connecting to it:
- The URL must use HTTPS.
- The host must resolve to a public IP address. Private, loopback, link-local, and cloud metadata addresses are rejected.
<Warning>
This validation can break configurations that point to internal endpoints, such as a self-hosted Ollama server. This is intentional: it fixes a security issue where the Prowler API could be directed to internal services. Internal endpoints must now be allowed explicitly through `LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS`.
</Warning>
To allow internal endpoints, set a comma-separated list of hostnames or IP addresses in the Prowler API environment (for Docker Compose deployments, the shared `.env` file):
```bash
LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS=custom-openai.internal,10.0.0.20
```
Hosts in this list skip the public-endpoint validation. HTTPS is still required, so the endpoint needs a certificate the Prowler API trusts.
</Tab>
</Tabs>
@@ -139,7 +158,7 @@ To set a different provider as default:
2. Click **Configure** under the desired provider to set as default
3. Click **Set as Default**
<img src="/images/prowler-app/lighthouse-set-default-provider.png" alt="Set default LLM provider" />
<img src="/images/prowler-app/lighthouse/oss/set-default-provider.png" alt="Set default LLM provider" />
## Updating Provider Credentials
@@ -8,6 +8,10 @@ import { VersionBadge } from "/snippets/version-badge.mdx"
Prowler Lighthouse AI integrates Large Language Models (LLMs) with Prowler security findings data.
<Info>
Using Prowler Cloud? Lighthouse AI on Prowler Cloud adds persistent chat sessions, GPT-5.5 as the default model, a dedicated agentic (chat) view, and transparent reasoning. See [Lighthouse AI on Prowler Cloud](/getting-started/products/prowler-cloud-lighthouse).
</Info>
Behind the scenes, Lighthouse AI works as follows:
- Lighthouse AI runs as a [Langchain agent](https://docs.langchain.com/oss/javascript/langchain/agents) in NextJS
@@ -47,7 +51,7 @@ Getting started with Prowler Lighthouse AI is easy:
For detailed configuration instructions for each provider, see [Using Multiple LLM Providers with Lighthouse](/user-guide/tutorials/prowler-app-lighthouse-multi-llm).
</Note>
<img src="/images/prowler-app/lighthouse-configuration.png" alt="Lighthouse AI Configuration" />
<img src="/images/prowler-app/lighthouse/oss/configuration.png" alt="Lighthouse AI Configuration" />
### Adding Business Context
@@ -23,7 +23,9 @@ A Scan Configuration lets you **override specific values, per provider**, on top
- Changes take effect on the provider's **next scan** and do not re-run past scans.
<Note>
The full set of configurable values and their defaults lives in [`prowler/config/config.yaml`](https://github.com/prowler-cloud/prowler/blob/master/prowler/config/config.yaml). For what each key means and which checks read it, see the [Configuration File tutorial](/user-guide/cli/tutorials/configuration_file).
</Note>
## Required Permissions
+4 -26
View File
@@ -2,6 +2,8 @@
title: 'Prowler Cloud'
---
import { ProviderCards } from "/snippets/provider-cards.mdx"
**Prowler Cloud** is a web application that simplifies running Prowler. This tutorial will guide you through setting up and using it.
We refer to **Prowler App** as the self-hosted version of **Prowler Cloud**.
@@ -79,32 +81,8 @@ Select the cloud provider to scan and configure authentication credentials. Each
For detailed instructions on configuring credentials for each provider, refer to the provider-specific getting started guides:
<Columns cols={3}>
<Card title="AWS" icon="aws" href="/user-guide/providers/aws/getting-started-aws">
Configure AWS authentication using IAM Access Keys or Assumed Role credentials.
</Card>
<Card title="Azure" icon="microsoft" href="/user-guide/providers/azure/getting-started-azure">
Set up Azure authentication using Service Principal credentials.
</Card>
<Card title="Google Cloud" icon="google" href="/user-guide/providers/gcp/getting-started-gcp">
Configure GCP authentication with Service Account or Application Default Credentials.
</Card>
<Card title="Oracle Cloud Infrastructure" icon="cloud" href="/user-guide/providers/oci/getting-started-oci">
Connect OCI with API key credentials to scan compartments and regions.
</Card>
<Card title="Kubernetes" icon="cloud" href="/user-guide/providers/kubernetes/getting-started-k8s">
Set up Kubernetes authentication using kubeconfig files for cluster access.
</Card>
<Card title="Microsoft 365" icon="microsoft" href="/user-guide/providers/microsoft365/getting-started-m365">
Configure M365 authentication with Application Certificate or Client Secret.
</Card>
<Card title="GitHub" icon="github" href="/user-guide/providers/github/getting-started-github">
Set up GitHub authentication using Personal Access Token, OAuth App, or GitHub App.
</Card>
<Card title="Infrastructure as Code" icon="code" href="/user-guide/providers/iac/getting-started-iac">
Scan IaC public or private repositories for security issues.
</Card>
</Columns>
<ProviderCards />
## Step 5: Test Connection
After adding your credentials of your cloud account, click the `Launch` button to verify that Prowler App can successfully connect to your provider:
@@ -0,0 +1,81 @@
---
title: 'Using Multiple LLM Providers'
---
import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
Lighthouse AI on Prowler Cloud supports multiple Large Language Model (LLM) providers, so teams can choose the option that best fits their infrastructure, compliance requirements, and cost considerations. This guide explains how to configure and manage providers for Lighthouse AI on Prowler Cloud.
<SubscriptionBanner />
For a feature overview, see [Lighthouse AI on Prowler Cloud](/getting-started/products/prowler-cloud-lighthouse).
## Supported Providers
- **OpenAI:** GPT models, including the default GPT-5.5.
- **Amazon Bedrock:** AWS-hosted access to Claude, Llama, Titan, and other models.
- **OpenAI Compatible:** Any service exposing an OpenAI-compatible API endpoint, such as OpenRouter.
## Model Requirements
Lighthouse AI requires models that support all of the following:
- **Text input:** Ability to receive text prompts.
- **Text output:** Ability to generate text responses.
- **Tool calling:** Ability to invoke tools to retrieve data from Prowler.
Models without these capabilities are not going to work well.
## Configuring Providers
Navigate to **Configuration** → **Lighthouse AI** to see the available providers, each with its own connection option.
To connect a provider:
1. Click the desired provider.
2. Enter the required credentials.
3. Click **Save**. The connection is validated automatically before the provider becomes available.
<img src="/images/prowler-app/lighthouse/prowler-cloud/config-page.png" alt="Lighthouse AI configuration page in Prowler Cloud" />
The required information differs per provider:
<Tabs>
<Tab title="OpenAI">
- **API Key:** OpenAI API key (starts with `sk-` or `sk-proj-`), created from the [OpenAI platform](https://platform.openai.com/api-keys).
Confirm the OpenAI account has sufficient credits and that the default GPT-5.5 model is not blocked in the organization settings.
</Tab>
<Tab title="Amazon Bedrock">
Connect using either an Amazon Bedrock API key or AWS IAM credentials.
**Amazon Bedrock API key**
- **Bedrock API Key:** The key generated from Amazon Bedrock.
- **AWS Region:** Region where Bedrock is available.
**AWS IAM Access Keys**
- **AWS Access Key ID** and **AWS Secret Access Key** for the IAM user.
- **AWS Region:** Region where Bedrock is available.
- The IAM user requires the `AmazonBedrockLimitedAccess` managed policy.
</Tab>
<Tab title="OpenAI Compatible">
Connect to any service exposing an OpenAI-compatible API endpoint that Prowler Cloud can reach over the Internet.
- **API Key:** API key from the compatible service.
- **Base URL:** API endpoint URL including the version (for example, `https://openrouter.ai/api/v1`).
### Example: OpenRouter
1. Create an account at [OpenRouter](https://openrouter.ai/).
2. Generate an API key from the OpenRouter dashboard.
3. Enter the API key and set the base URL to `https://openrouter.ai/api/v1`.
</Tab>
</Tabs>
## Model Recommendations
GPT-5.5 is the default and recommended model for Lighthouse AI on Prowler Cloud. Models from Amazon Bedrock and OpenAI-compatible endpoints can also be used, but performance is not guaranteed. Ensure any selected model supports text input, text output, and tool calling.
## Getting Help
For issues or suggestions with Lighthouse AI on Prowler Cloud, request support at [support.prowler.com](https://support.prowler.com) or [reach out through our Slack channel](https://goto.prowler.com/slack).

Some files were not shown because too many files have changed in this diff Show More