Compare commits

...
Author SHA1 Message Date
HugoPBrito d35e5a4bec fix: enforce cloudflare token/key format validation
- Reject API key-shaped values in Cloudflare api_token

- Validate Cloudflare api_key as 32-character hexadecimal format

- Add UI form validation to prevent token/key cross-input

- Update API and UI changelog entries for the current PR
2026-03-02 10:14:13 +01:00
HugoPBrito 861be13b7d Merge branch 'master' of https://github.com/prowler-cloud/prowler into ensure-key-format-cloudflare 2026-03-02 09:49:06 +01:00
Pepe Fagoaga 8af9b333c9 ci: restore persist credentials when no output is generated (#10211) 2026-03-02 09:14:02 +01:00
Pepe Fagoaga 4e71a9dcf1 ci(security): Add zizmor (#10208) 2026-03-02 08:25:13 +01:00
Pepe Fagoaga 7adcbed727 fix(ci): zizmor security improvements (#10207) 2026-03-02 08:24:51 +01:00
Andoni Alonso 8be218b29f fix(ci): harden GitHub Actions workflows against expression injection (#10200) 2026-03-01 19:58:43 +01:00
Alejandro Bailo 80e84d1da4 fix(ui): stabilize provider wizard modal and DataTable rendering (#10194) 2026-02-27 14:35:13 +01:00
HugoPBrito 62809e523e docs(api): add changelog entry for cloudflare token fix 2026-02-27 14:05:55 +01:00
HugoPBrito 5ff6c3c35f fix(api): reject cloudflare api key in token field
- Add serializer validation for Cloudflare api_token values

- Reject 32-character hexadecimal values in token credentials

- Add serializer tests for valid token and api key-shaped token
2026-02-27 14:02:46 +01:00
mintlify[bot] fff80a920b chore(docs): Add Reo tracking beacon (#10193)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-02-27 13:07:46 +01:00
mintlify[bot]andPepe Fagoaga 90a4579230 docs(install): Add missing notes for Docker Compose installation (#10192)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-02-27 12:53:59 +01:00
Pedro Martín 2f44be8db4 docs(aws): add AWS Organizations (#10183) 2026-02-27 12:28:16 +01:00
Alejandro Bailo 288593d01e fix(ui): patch npm transitive dependency vulnerabilities (#10187) 2026-02-27 10:31:20 +01:00
Alejandro Bailo ddb6c03c0e test(ui): fix provider E2E test selectors and reliability (#10178) 2026-02-27 10:12:54 +01:00
mintlify[bot] 79d4476713 docs(import): Add billing impact section to Findings Import (#10186)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-02-27 10:11:16 +01:00
Anthonyandalejandrobailo 06f6e8b99b fix(ui): apply provider/account filters to Findings Severity Over Time chart (#10103)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-02-27 10:10:47 +01:00
Adrián Peña 8ee4a9e3fc fix(sdk): scope scan_id by provider and account (#10184) 2026-02-26 19:19:29 +01:00
Adrián Peña 336cbe1844 feat(ingestions): allow multiple scan_ids and providers inside the ocsf (#10182) 2026-02-26 17:56:21 +01:00
Andoni AlonsoandHugoPBrito c8ce590039 feat(m365): add entra_default_app_management_policy_enabled security check (#9898)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2026-02-26 16:14:29 +01:00
Josema CamachoandAdrián Jesús Peña Rodríguez b3a67fa1a0 feat(api): add accept header text/plain to attack paths query endpoints for support llm-friendly output (#10162)
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
2026-02-26 12:53:58 +01:00
Adrián Peña 902558f2d4 feat(api): block attack-paths-scans custom queries and schema endpoints (#10177) 2026-02-26 12:27:52 +01:00
Alan Buscaglia 09302f9d7d fix(ci): include E2E test paths in impact analysis module matching (#10176) 2026-02-26 12:10:36 +01:00
Andoni AlonsoandHugoPBrito df09b14c75 feat(m365): add entra_all_apps_conditional_access_coverage security check (#9902)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2026-02-26 11:37:09 +01:00
Adrián Peña eacb3430cb fix(api): recalc tenant compliance summary after provider deletion (#10172) 2026-02-26 11:18:15 +01:00
Alan Buscaglia c151d08712 fix(skills): add Bash 3.2 compatibility to sync.sh (#9841) 2026-02-26 10:26:22 +01:00
Pedro Martín fac089ab78 feat(compliance): add SecNumCloud for AWS (#10117) 2026-02-26 09:31:19 +01:00
Rubén De la Torre Vico d15cabee20 feat(ui): add attack paths tools to Lighthouse allowed list (#10175) 2026-02-25 16:42:13 +01:00
Andoni Alonso ee7ecabe29 docs: add pre-configured GitHub PAT creation links (#10174) 2026-02-25 14:13:53 +01:00
Alejandro Bailo 2a58781e37 test(ui): update E2E page objects and improve test stability (#10158) 2026-02-25 13:30:54 +01:00
Alejandro Bailo f403971885 feat(ui): add AWS Organizations bulk connect flow (#10157) 2026-02-25 13:16:34 +01:00
Alejandro Bailo 7935e926ac feat(ui): replace route-based provider flow with modal wizard (#10156) 2026-02-25 13:08:17 +01:00
Alejandro Bailo 231bfd6f41 feat(ui): add organization server actions and scan launching (#10155) 2026-02-25 12:56:26 +01:00
Alejandro Bailo fe8d5893af feat(ui): add organization and wizard types and stores (#10154) 2026-02-25 12:45:15 +01:00
Hugo Pereira BritoandAndoni Alonso db1db7d366 feat(m365): add entra_require_mfa_for_management_api security check (#10150)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2026-02-25 12:29:23 +01:00
Alejandro Bailo 6d9ef78df1 style(ui): improve shadcn primitives and add shared components (#10153) 2026-02-25 12:19:08 +01:00
lydiavilchez 9ee8072572 feat(googleworkspace): add Google Workspace provider with directory service and super admin check (#10022) 2026-02-25 12:17:13 +01:00
Hugo Pereira Brito 6935c4eb1b feat(m365): add entra_app_enforced_restrictions security check (#10058) 2026-02-25 11:53:35 +01:00
Adrián Peña e47f2b4033 fix(api): harden security hub retries (#10144) 2026-02-25 11:34:41 +01:00
Rubén De la Torre Vico 7077a56331 chore(mcp_server): bump MCP Server package version to 0.4.0 (#10171) 2026-02-25 11:31:35 +01:00
mintlify[bot] 964cc45b14 docs(rbac): add permissions table with scope (#10163)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-02-25 11:17:17 +01:00
Rubén De la Torre Vico a8e504887b feat(mcp_server): add tools related with attack paths (#10145) 2026-02-25 10:56:40 +01:00
2115344de8 docs: add findings ingestion documentation (#10159)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-02-24 19:15:46 +01:00
6962622fd2 fix(aws): filter VPC endpoint services by audited account to prevent AccessDenied errors (#10152)
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: jfagoagas <16007882+jfagoagas@users.noreply.github.com>
2026-02-24 18:30:31 +01:00
Adrián Peña 2a4ee830cc feat(sdk): add --export-ocsf flag for OCSF ingestion to Prowler Cloud (#10095) 2026-02-24 17:47:35 +01:00
Josema Camacho 247bde1ef4 feat(attack-paths): add custom query and cartography schema endpoints (#10149) 2026-02-24 15:49:50 +01:00
Andoni Alonso c159181d27 feat(api): add Image provider support for container image scanning (#10128) 2026-02-24 13:06:34 +01:00
Daniel Barranquero 030d053c84 chore(openstack): support multi-region in the same provider (#10135) 2026-02-24 12:50:52 +01:00
358 changed files with 24082 additions and 2777 deletions

No files matched your search

+1
View File
@@ -70,6 +70,7 @@ NEO4J_DBMS_CONNECTOR_BOLT_LISTEN_ADDRESS=0.0.0.0:7687
ATTACK_PATHS_BATCH_SIZE=1000
ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES=3
ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS=30
ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES=250
# Celery-Prowler task settings
TASK_RETRY_DELAY_SECONDS=0.1
@@ -35,7 +35,9 @@ runs:
shell: bash
run: |
python -m pip install --upgrade pip
pipx install poetry==${{ inputs.poetry-version }}
pipx install poetry==${INPUTS_POETRY_VERSION}
env:
INPUTS_POETRY_VERSION: ${{ inputs.poetry-version }}
- name: Update poetry.lock with latest Prowler commit
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
+10 -5
View File
@@ -26,16 +26,18 @@ runs:
id: status
shell: bash
run: |
if [[ "${{ inputs.step-outcome }}" == "success" ]]; then
if [[ "${INPUTS_STEP_OUTCOME}" == "success" ]]; then
echo "STATUS_TEXT=Completed" >> $GITHUB_ENV
echo "STATUS_COLOR=#6aa84f" >> $GITHUB_ENV
elif [[ "${{ inputs.step-outcome }}" == "failure" ]]; then
elif [[ "${INPUTS_STEP_OUTCOME}" == "failure" ]]; then
echo "STATUS_TEXT=Failed" >> $GITHUB_ENV
echo "STATUS_COLOR=#fc3434" >> $GITHUB_ENV
else
# No outcome provided - pending/in progress state
echo "STATUS_COLOR=#dbab09" >> $GITHUB_ENV
fi
env:
INPUTS_STEP_OUTCOME: ${{ inputs.step-outcome }}
- name: Send Slack notification (new message)
if: inputs.update-ts == ''
@@ -67,8 +69,11 @@ runs:
id: slack-notification
shell: bash
run: |
if [[ "${{ inputs.update-ts }}" == "" ]]; then
echo "ts=${{ steps.slack-notification-post.outputs.ts }}" >> $GITHUB_OUTPUT
if [[ "${INPUTS_UPDATE_TS}" == "" ]]; then
echo "ts=${STEPS_SLACK_NOTIFICATION_POST_OUTPUTS_TS}" >> $GITHUB_OUTPUT
else
echo "ts=${{ inputs.update-ts }}" >> $GITHUB_OUTPUT
echo "ts=${INPUTS_UPDATE_TS}" >> $GITHUB_OUTPUT
fi
env:
INPUTS_UPDATE_TS: ${{ inputs.update-ts }}
STEPS_SLACK_NOTIFICATION_POST_OUTPUTS_TS: ${{ steps.slack-notification-post.outputs.ts }}
+11 -5
View File
@@ -54,7 +54,7 @@ runs:
trivy-db-${{ runner.os }}-
- name: Run Trivy vulnerability scan (JSON)
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
uses: aquasecurity/trivy-action@e368e328979b113139d6f9068e03accaed98a518 # 0.34.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'json'
@@ -66,7 +66,7 @@ runs:
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
uses: aquasecurity/trivy-action@e368e328979b113139d6f9068e03accaed98a518 # 0.34.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'sarif'
@@ -105,11 +105,14 @@ runs:
echo "### 🔒 Container Security Scan" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${{ inputs.image-name }}:${{ inputs.image-tag }}\`" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${INPUTS_IMAGE_NAME}:${INPUTS_IMAGE_TAG}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- 🔴 Critical: $CRITICAL" >> $GITHUB_STEP_SUMMARY
echo "- 🟠 High: $HIGH" >> $GITHUB_STEP_SUMMARY
echo "- **Total**: $TOTAL" >> $GITHUB_STEP_SUMMARY
env:
INPUTS_IMAGE_NAME: ${{ inputs.image-name }}
INPUTS_IMAGE_TAG: ${{ inputs.image-tag }}
- name: Comment scan results on PR
if: inputs.create-pr-comment == 'true' && github.event_name == 'pull_request'
@@ -123,7 +126,7 @@ runs:
const comment = require('./.github/scripts/trivy-pr-comment.js');
// Unique identifier to find our comment
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
const marker = `<!-- trivy-scan-comment:${process.env.IMAGE_NAME} -->`;
const body = marker + '\n' + comment;
// Find existing comment
@@ -159,6 +162,9 @@ runs:
if: inputs.fail-on-critical == 'true' && steps.security-check.outputs.critical != '0'
shell: bash
run: |
echo "::error::Found ${{ steps.security-check.outputs.critical }} critical vulnerabilities"
echo "::error::Found ${STEPS_SECURITY_CHECK_OUTPUTS_CRITICAL} critical vulnerabilities"
echo "::warning::Please update packages or use a different base image"
exit 1
env:
STEPS_SECURITY_CHECK_OUTPUTS_CRITICAL: ${{ steps.security-check.outputs.critical }}
+6
View File
@@ -15,6 +15,8 @@ updates:
labels:
- "dependencies"
- "pip"
cooldown:
default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "pip"
@@ -37,6 +39,8 @@ updates:
labels:
- "dependencies"
- "github_actions"
cooldown:
default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "npm"
@@ -59,6 +63,8 @@ updates:
labels:
- "dependencies"
- "docker"
cooldown:
default-days: 7
# Dependabot Updates are temporary disabled - 2025/04/15
# v4.6
+8
View File
@@ -62,6 +62,11 @@ provider/openstack:
- any-glob-to-any-file: "prowler/providers/openstack/**"
- any-glob-to-any-file: "tests/providers/openstack/**"
provider/googleworkspace:
- changed-files:
- any-glob-to-any-file: "prowler/providers/googleworkspace/**"
- any-glob-to-any-file: "tests/providers/googleworkspace/**"
github_actions:
- changed-files:
- any-glob-to-any-file: ".github/workflows/*"
@@ -83,6 +88,7 @@ mutelist:
- any-glob-to-any-file: "prowler/providers/alibabacloud/lib/mutelist/**"
- any-glob-to-any-file: "prowler/providers/cloudflare/lib/mutelist/**"
- any-glob-to-any-file: "prowler/providers/openstack/lib/mutelist/**"
- any-glob-to-any-file: "prowler/providers/googleworkspace/lib/mutelist/**"
- any-glob-to-any-file: "tests/lib/mutelist/**"
- any-glob-to-any-file: "tests/providers/aws/lib/mutelist/**"
- any-glob-to-any-file: "tests/providers/azure/lib/mutelist/**"
@@ -94,6 +100,8 @@ mutelist:
- any-glob-to-any-file: "tests/providers/alibabacloud/lib/mutelist/**"
- any-glob-to-any-file: "tests/providers/cloudflare/lib/mutelist/**"
- any-glob-to-any-file: "tests/providers/openstack/lib/mutelist/**"
- any-glob-to-any-file: "prowler/providers/googleworkspace/lib/mutelist/**"
- any-glob-to-any-file: "tests/providers/googleworkspace/lib/mutelist/**"
integration/s3:
- changed-files:
+20 -1
View File
@@ -27,7 +27,7 @@ ignored:
# IDE/Editor configs
- .vscode/**
- .idea/**
# Examples and contrib (not production code)
- examples/**
- contrib/**
@@ -61,6 +61,8 @@ critical:
- ui/types/**
- ui/config/**
- ui/middleware.ts
- ui/tsconfig.json
- ui/playwright.config.ts
# CI/CD changes
- .github/workflows/**
@@ -272,6 +274,7 @@ modules:
- ui/components/providers/**
- ui/actions/providers/**
- ui/app/**/providers/**
- ui/tests/providers/**
tests: []
e2e:
- ui/tests/providers/**
@@ -281,6 +284,7 @@ modules:
- ui/components/findings/**
- ui/actions/findings/**
- ui/app/**/findings/**
- ui/tests/findings/**
tests: []
e2e:
- ui/tests/findings/**
@@ -290,6 +294,7 @@ modules:
- ui/components/scans/**
- ui/actions/scans/**
- ui/app/**/scans/**
- ui/tests/scans/**
tests: []
e2e:
- ui/tests/scans/**
@@ -299,6 +304,7 @@ modules:
- ui/components/compliance/**
- ui/actions/compliances/**
- ui/app/**/compliance/**
- ui/tests/compliance/**
tests: []
e2e:
- ui/tests/compliance/**
@@ -308,8 +314,12 @@ modules:
- ui/components/auth/**
- ui/actions/auth/**
- ui/app/(auth)/**
- ui/tests/auth/**
- ui/tests/sign-in/**
- ui/tests/sign-up/**
tests: []
e2e:
- ui/tests/auth/**
- ui/tests/sign-in/**
- ui/tests/sign-up/**
@@ -318,6 +328,7 @@ modules:
- ui/components/invitations/**
- ui/actions/invitations/**
- ui/app/**/invitations/**
- ui/tests/invitations/**
tests: []
e2e:
- ui/tests/invitations/**
@@ -327,6 +338,7 @@ modules:
- ui/components/roles/**
- ui/actions/roles/**
- ui/app/**/roles/**
- ui/tests/roles/**
tests: []
e2e:
- ui/tests/roles/**
@@ -336,6 +348,7 @@ modules:
- ui/components/users/**
- ui/actions/users/**
- ui/app/**/users/**
- ui/tests/users/**
tests: []
e2e:
- ui/tests/users/**
@@ -345,6 +358,7 @@ modules:
- ui/components/integrations/**
- ui/actions/integrations/**
- ui/app/**/integrations/**
- ui/tests/integrations/**
tests: []
e2e:
- ui/tests/integrations/**
@@ -354,6 +368,7 @@ modules:
- ui/components/resources/**
- ui/actions/resources/**
- ui/app/**/resources/**
- ui/tests/resources/**
tests: []
e2e:
- ui/tests/resources/**
@@ -361,6 +376,7 @@ modules:
- name: ui-profile
match:
- ui/app/**/profile/**
- ui/tests/profile/**
tests: []
e2e:
- ui/tests/profile/**
@@ -371,6 +387,7 @@ modules:
- ui/actions/lighthouse/**
- ui/app/**/lighthouse/**
- ui/lib/lighthouse/**
- ui/tests/lighthouse/**
tests: []
e2e:
- ui/tests/lighthouse/**
@@ -379,6 +396,7 @@ modules:
match:
- ui/components/overview/**
- ui/actions/overview/**
- ui/tests/home/**
tests: []
e2e:
- ui/tests/home/**
@@ -397,6 +415,7 @@ modules:
- ui/components/attack-paths/**
- ui/actions/attack-paths/**
- ui/app/**/attack-paths/**
- ui/tests/attack-paths/**
tests: []
e2e:
- ui/tests/attack-paths/**
+30 -10
View File
@@ -29,6 +29,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Get current API version
id: get_api_version
@@ -79,12 +81,14 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next API minor version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
CURRENT_API_VERSION="${{ needs.detect-release-type.outputs.current_api_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
CURRENT_API_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION}"
# API version follows Prowler minor + 1
# For Prowler 5.17.0 -> API 1.18.0
@@ -97,6 +101,10 @@ jobs:
echo "Prowler release version: ${MAJOR_VERSION}.${MINOR_VERSION}.0"
echo "Current API version: $CURRENT_API_VERSION"
echo "Next API minor version (for master): $NEXT_API_VERSION"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION: ${{ needs.detect-release-type.outputs.current_api_version }}
- name: Bump API versions in files for master
run: |
@@ -132,12 +140,13 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
persist-credentials: false
- name: Calculate first API patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
CURRENT_API_VERSION="${{ needs.detect-release-type.outputs.current_api_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
CURRENT_API_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION}"
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
# API version follows Prowler minor + 1
@@ -151,6 +160,10 @@ jobs:
echo "Prowler release version: ${MAJOR_VERSION}.${MINOR_VERSION}.0"
echo "First API patch version (for ${VERSION_BRANCH}): $FIRST_API_PATCH_VERSION"
echo "Version branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION: ${{ needs.detect-release-type.outputs.current_api_version }}
- name: Bump API versions in files for version branch
run: |
@@ -193,13 +206,15 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next API patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
CURRENT_API_VERSION="${{ needs.detect-release-type.outputs.current_api_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
PATCH_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION}
CURRENT_API_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION}"
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
# Extract current API patch to increment it
@@ -222,6 +237,11 @@ jobs:
echo "::error::Invalid API version format: $CURRENT_API_VERSION"
exit 1
fi
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION: ${{ needs.detect-release-type.outputs.patch_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_API_VERSION: ${{ needs.detect-release-type.outputs.current_api_version }}
- name: Bump API versions in files for version branch
run: |
+3
View File
@@ -34,6 +34,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for API changes
id: check-changes
+2
View File
@@ -43,6 +43,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
+24 -9
View File
@@ -58,6 +58,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Notify container push started
id: slack-notification
@@ -94,6 +96,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
@@ -138,18 +142,22 @@ jobs:
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${RELEASE_TAG} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Install regctl
if: always()
@@ -159,9 +167,11 @@ jobs:
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64" || true
echo "Cleanup completed"
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
notify-release-completed:
if: always() && needs.notify-release-started.result == 'success' && (github.event_name == 'release' || github.event_name == 'workflow_dispatch')
@@ -171,15 +181,20 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Determine overall outcome
id: outcome
run: |
if [[ "${{ needs.container-build-push.result }}" == "success" && "${{ needs.create-manifest.result }}" == "success" ]]; then
if [[ "${NEEDS_CONTAINER_BUILD_PUSH_RESULT}" == "success" && "${NEEDS_CREATE_MANIFEST_RESULT}" == "success" ]]; then
echo "outcome=success" >> $GITHUB_OUTPUT
else
echo "outcome=failure" >> $GITHUB_OUTPUT
fi
env:
NEEDS_CONTAINER_BUILD_PUSH_RESULT: ${{ needs.container-build-push.result }}
NEEDS_CREATE_MANIFEST_RESULT: ${{ needs.create-manifest.result }}
- name: Notify container push completed
uses: ./.github/actions/slack-notification
@@ -29,6 +29,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check if Dockerfile changed
id: dockerfile-changed
@@ -64,6 +67,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for API changes
id: check-changes
+3
View File
@@ -34,6 +34,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for API changes
id: check-changes
+4 -1
View File
@@ -43,7 +43,7 @@ jobs:
services:
postgres:
image: postgres
image: postgres:17@sha256:2cd82735a36356842d5eb1ef80db3ae8f1154172f0f653db48fde079b2a0b7f7
env:
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
@@ -74,6 +74,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for API changes
id: check-changes
+1
View File
@@ -1,6 +1,7 @@
name: 'Tools: Backport'
on:
# zizmor: ignore[dangerous-triggers] - intentional: needs write access for backport PRs, no PR code checkout
pull_request_target:
branches:
- 'master'
+44
View File
@@ -0,0 +1,44 @@
name: 'CI: Zizmor'
on:
push:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/**'
pull_request:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/**'
schedule:
- cron: '30 06 * * *'
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
zizmor:
if: github.repository == 'prowler-cloud/prowler'
name: GitHub Actions Security Audit
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
security-events: write
contents: read
actions: read
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Run zizmor
uses: zizmorcore/zizmor-action@0dce2577a4760a2749d8cfb7a84b7d5585ebcb7d # v0.5.0
with:
token: ${{ github.token }}
+2 -1
View File
@@ -25,8 +25,9 @@ jobs:
- name: Create backport label for minor releases
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_EVENT_RELEASE_TAG_NAME: ${{ github.event.release.tag_name }}
run: |
RELEASE_TAG="${{ github.event.release.tag_name }}"
RELEASE_TAG="${GITHUB_EVENT_RELEASE_TAG_NAME}"
if [ -z "$RELEASE_TAG" ]; then
echo "Error: No release tag provided"
+30 -10
View File
@@ -29,6 +29,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Get current documentation version
id: get_docs_version
@@ -79,12 +81,14 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next minor version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
CURRENT_DOCS_VERSION="${{ needs.detect-release-type.outputs.current_docs_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
CURRENT_DOCS_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION}"
NEXT_MINOR_VERSION=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).0
echo "CURRENT_DOCS_VERSION=${CURRENT_DOCS_VERSION}" >> "${GITHUB_ENV}"
@@ -93,6 +97,10 @@ jobs:
echo "Current documentation version: $CURRENT_DOCS_VERSION"
echo "Current release version: $PROWLER_VERSION"
echo "Next minor version: $NEXT_MINOR_VERSION"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION: ${{ needs.detect-release-type.outputs.current_docs_version }}
- name: Bump versions in documentation for master
run: |
@@ -132,12 +140,13 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
persist-credentials: false
- name: Calculate first patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
CURRENT_DOCS_VERSION="${{ needs.detect-release-type.outputs.current_docs_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
CURRENT_DOCS_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION}"
FIRST_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.1
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -148,6 +157,10 @@ jobs:
echo "First patch version: $FIRST_PATCH_VERSION"
echo "Version branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION: ${{ needs.detect-release-type.outputs.current_docs_version }}
- name: Bump versions in documentation for version branch
run: |
@@ -193,13 +206,15 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
CURRENT_DOCS_VERSION="${{ needs.detect-release-type.outputs.current_docs_version }}"
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
PATCH_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION}
CURRENT_DOCS_VERSION="${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION}"
NEXT_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.$((PATCH_VERSION + 1))
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -212,6 +227,11 @@ jobs:
echo "Current release version: $PROWLER_VERSION"
echo "Next patch version: $NEXT_PATCH_VERSION"
echo "Target branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION: ${{ needs.detect-release-type.outputs.patch_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_CURRENT_DOCS_VERSION: ${{ needs.detect-release-type.outputs.current_docs_version }}
- name: Bump versions in documentation for patch version
run: |
+1
View File
@@ -26,6 +26,7 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Scan for secrets with TruffleHog
uses: trufflesecurity/trufflehog@ef6e76c3c4023279497fab4721ffa071a722fd05 # v3.92.4
+1
View File
@@ -1,6 +1,7 @@
name: 'Tools: PR Labeler'
on:
# zizmor: ignore[dangerous-triggers] - intentional: needs write access to apply labels, no PR code checkout
pull_request_target:
branches:
- 'master'
+25 -10
View File
@@ -57,6 +57,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Notify container push started
id: slack-notification
@@ -92,6 +94,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
@@ -144,30 +148,36 @@ jobs:
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${RELEASE_TAG} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64" || true
echo "Cleanup completed"
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
notify-release-completed:
if: always() && needs.notify-release-started.result == 'success' && (github.event_name == 'release' || github.event_name == 'workflow_dispatch')
@@ -177,15 +187,20 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Determine overall outcome
id: outcome
run: |
if [[ "${{ needs.container-build-push.result }}" == "success" && "${{ needs.create-manifest.result }}" == "success" ]]; then
if [[ "${NEEDS_CONTAINER_BUILD_PUSH_RESULT}" == "success" && "${NEEDS_CREATE_MANIFEST_RESULT}" == "success" ]]; then
echo "outcome=success" >> $GITHUB_OUTPUT
else
echo "outcome=failure" >> $GITHUB_OUTPUT
fi
env:
NEEDS_CONTAINER_BUILD_PUSH_RESULT: ${{ needs.container-build-push.result }}
NEEDS_CREATE_MANIFEST_RESULT: ${{ needs.create-manifest.result }}
- name: Notify container push completed
uses: ./.github/actions/slack-notification
@@ -29,6 +29,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check if Dockerfile changed
id: dockerfile-changed
@@ -63,6 +66,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for MCP changes
id: check-changes
+6 -2
View File
@@ -29,7 +29,7 @@ jobs:
- name: Parse and validate version
id: parse-version
run: |
PROWLER_VERSION="${{ env.RELEASE_TAG }}"
PROWLER_VERSION="${RELEASE_TAG}"
echo "version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
# Extract major version
@@ -61,9 +61,13 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@5a095e7a2014a4212f075830d4f7277575a9d098 # v7
with:
enable-cache: false
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
+9 -4
View File
@@ -32,6 +32,8 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
fetch-depth: 0
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Get changed files
id: changed-files
@@ -50,11 +52,11 @@ jobs:
run: |
missing_changelogs=""
if [[ "${{ steps.changed-files.outputs.any_changed }}" == "true" ]]; then
if [[ "${STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED}" == "true" ]]; then
# Check monitored folders
for folder in $MONITORED_FOLDERS; do
# Get files changed in this folder
changed_in_folder=$(echo "${{ steps.changed-files.outputs.all_changed_files }}" | tr ' ' '\n' | grep "^${folder}/" || true)
changed_in_folder=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^${folder}/" || true)
if [ -n "$changed_in_folder" ]; then
echo "Detected changes in ${folder}/"
@@ -69,11 +71,11 @@ jobs:
# Check root-level dependency files (poetry.lock, pyproject.toml)
# These are associated with the prowler folder changelog
root_deps_changed=$(echo "${{ steps.changed-files.outputs.all_changed_files }}" | tr ' ' '\n' | grep -E "^(poetry\.lock|pyproject\.toml)$" || true)
root_deps_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep -E "^(poetry\.lock|pyproject\.toml)$" || true)
if [ -n "$root_deps_changed" ]; then
echo "Detected changes in root dependency files: $root_deps_changed"
# Check if prowler/CHANGELOG.md was already updated (might have been caught above)
prowler_changelog_updated=$(echo "${{ steps.changed-files.outputs.all_changed_files }}" | tr ' ' '\n' | grep "^prowler/CHANGELOG.md$" || true)
prowler_changelog_updated=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^prowler/CHANGELOG.md$" || true)
if [ -z "$prowler_changelog_updated" ]; then
# Only add if prowler wasn't already flagged
if ! echo "$missing_changelogs" | grep -q "prowler"; then
@@ -89,6 +91,9 @@ jobs:
echo -e "${missing_changelogs}"
echo "EOF"
} >> $GITHUB_OUTPUT
env:
STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED: ${{ steps.changed-files.outputs.any_changed }}
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
+5 -1
View File
@@ -1,6 +1,7 @@
name: 'Tools: PR Conflict Checker'
on:
# zizmor: ignore[dangerous-triggers] - intentional: needs write access for conflict labels/comments, checkout uses PR head SHA for read-only grep
pull_request_target:
types:
- 'opened'
@@ -29,6 +30,7 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
persist-credentials: false
- name: Get changed files
id: changed-files
@@ -45,7 +47,7 @@ jobs:
HAS_CONFLICTS=false
# Check each changed file for conflict markers
for file in ${{ steps.changed-files.outputs.all_changed_files }}; do
for file in ${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}; do
if [ -f "$file" ]; then
echo "Checking file: $file"
@@ -70,6 +72,8 @@ jobs:
echo "has_conflicts=false" >> $GITHUB_OUTPUT
echo "No conflict markers found in changed files"
fi
env:
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
- name: Manage conflict label
env:
+6 -3
View File
@@ -1,6 +1,7 @@
name: 'Tools: PR Merged'
on:
# zizmor: ignore[dangerous-triggers] - intentional: needs read access to merged PR metadata, no PR code checkout
pull_request_target:
branches:
- 'master'
@@ -25,8 +26,10 @@ jobs:
- name: Calculate short commit SHA
id: vars
run: |
SHORT_SHA="${{ github.event.pull_request.merge_commit_sha }}"
echo "SHORT_SHA=${SHORT_SHA::7}" >> $GITHUB_ENV
SHORT_SHA="${GITHUB_EVENT_PULL_REQUEST_MERGE_COMMIT_SHA}"
echo "short_sha=${SHORT_SHA::7}" >> $GITHUB_OUTPUT
env:
GITHUB_EVENT_PULL_REQUEST_MERGE_COMMIT_SHA: ${{ github.event.pull_request.merge_commit_sha }}
- name: Trigger Cloud repository pull request
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
@@ -37,7 +40,7 @@ jobs:
client-payload: |
{
"PROWLER_COMMIT_SHA": "${{ github.event.pull_request.merge_commit_sha }}",
"PROWLER_COMMIT_SHORT_SHA": "${{ env.SHORT_SHA }}",
"PROWLER_COMMIT_SHORT_SHA": "${{ steps.vars.outputs.short_sha }}",
"PROWLER_PR_NUMBER": "${{ github.event.pull_request.number }}",
"PROWLER_PR_TITLE": ${{ toJson(github.event.pull_request.title) }},
"PROWLER_PR_LABELS": ${{ toJson(github.event.pull_request.labels.*.name) }},
+1
View File
@@ -31,6 +31,7 @@ jobs:
with:
fetch-depth: 0
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
+22 -7
View File
@@ -68,17 +68,22 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next minor version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
NEXT_MINOR_VERSION=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).0
echo "NEXT_MINOR_VERSION=${NEXT_MINOR_VERSION}" >> "${GITHUB_ENV}"
echo "Current version: $PROWLER_VERSION"
echo "Next minor version: $NEXT_MINOR_VERSION"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
- name: Bump versions in files for master
run: |
@@ -113,11 +118,12 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
persist-credentials: false
- name: Calculate first patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
FIRST_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.1
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -127,6 +133,9 @@ jobs:
echo "First patch version: $FIRST_PATCH_VERSION"
echo "Version branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
- name: Bump versions in files for version branch
run: |
@@ -168,12 +177,14 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
PATCH_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION}
NEXT_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.$((PATCH_VERSION + 1))
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -184,6 +195,10 @@ jobs:
echo "Current version: $PROWLER_VERSION"
echo "Next patch version: $NEXT_PATCH_VERSION"
echo "Target branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION: ${{ needs.detect-release-type.outputs.patch_version }}
- name: Bump versions in files for version branch
run: |
@@ -21,6 +21,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Check for duplicate test names across providers
run: |
+3
View File
@@ -32,6 +32,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for SDK changes
id: check-changes
+2
View File
@@ -50,6 +50,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
+36 -17
View File
@@ -62,6 +62,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
@@ -116,6 +118,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Notify container push started
id: slack-notification
@@ -152,6 +156,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
@@ -214,36 +220,44 @@ jobs:
if: github.event_name == 'push'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-arm64
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-arm64
env:
NEEDS_SETUP_OUTPUTS_LATEST_TAG: ${{ needs.setup.outputs.latest_tag }}
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.setup.outputs.prowler_version }} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.setup.outputs.stable_tag }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.setup.outputs.prowler_version }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.setup.outputs.stable_tag }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.prowler_version }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.stable_tag }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-arm64
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${NEEDS_SETUP_OUTPUTS_PROWLER_VERSION} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${NEEDS_SETUP_OUTPUTS_STABLE_TAG} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${NEEDS_SETUP_OUTPUTS_PROWLER_VERSION} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${NEEDS_SETUP_OUTPUTS_STABLE_TAG} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_PROWLER_VERSION} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_STABLE_TAG} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-arm64
env:
NEEDS_SETUP_OUTPUTS_PROWLER_VERSION: ${{ needs.setup.outputs.prowler_version }}
NEEDS_SETUP_OUTPUTS_STABLE_TAG: ${{ needs.setup.outputs.stable_tag }}
NEEDS_SETUP_OUTPUTS_LATEST_TAG: ${{ needs.setup.outputs.latest_tag }}
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.latest_tag }}-arm64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_LATEST_TAG}-arm64" || true
echo "Cleanup completed"
env:
NEEDS_SETUP_OUTPUTS_LATEST_TAG: ${{ needs.setup.outputs.latest_tag }}
notify-release-completed:
if: always() && needs.notify-release-started.result == 'success' && (github.event_name == 'release' || github.event_name == 'workflow_dispatch')
@@ -253,15 +267,20 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Determine overall outcome
id: outcome
run: |
if [[ "${{ needs.container-build-push.result }}" == "success" && "${{ needs.create-manifest.result }}" == "success" ]]; then
if [[ "${NEEDS_CONTAINER_BUILD_PUSH_RESULT}" == "success" && "${NEEDS_CREATE_MANIFEST_RESULT}" == "success" ]]; then
echo "outcome=success" >> $GITHUB_OUTPUT
else
echo "outcome=failure" >> $GITHUB_OUTPUT
fi
env:
NEEDS_CONTAINER_BUILD_PUSH_RESULT: ${{ needs.container-build-push.result }}
NEEDS_CREATE_MANIFEST_RESULT: ${{ needs.create-manifest.result }}
- name: Notify container push completed
uses: ./.github/actions/slack-notification
@@ -28,6 +28,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check if Dockerfile changed
id: dockerfile-changed
@@ -63,6 +66,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for SDK changes
id: check-changes
+5 -3
View File
@@ -28,7 +28,7 @@ jobs:
- name: Parse and validate version
id: parse-version
run: |
PROWLER_VERSION="${{ env.RELEASE_TAG }}"
PROWLER_VERSION="${RELEASE_TAG}"
echo "version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
# Extract major version
@@ -60,6 +60,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Install Poetry
run: pipx install poetry==2.1.1
@@ -68,7 +70,6 @@ jobs:
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'poetry'
- name: Build Prowler package
run: poetry build
@@ -92,6 +93,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Install Poetry
run: pipx install poetry==2.1.1
@@ -100,7 +103,6 @@ jobs:
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'poetry'
- name: Install toml package
run: pip install toml
@@ -28,6 +28,7 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: 'master'
persist-credentials: false
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
@@ -82,9 +83,14 @@ jobs:
- name: PR creation result
run: |
if [[ "${{ steps.create-pr.outputs.pull-request-number }}" ]]; then
echo "✓ Pull request #${{ steps.create-pr.outputs.pull-request-number }} created successfully"
echo "URL: ${{ steps.create-pr.outputs.pull-request-url }}"
if [[ "${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER}" ]]; then
echo "✓ Pull request #${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER} created successfully"
echo "URL: ${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_URL}"
else
echo "✓ No changes detected - AWS regions are up to date"
fi
env:
STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER: ${{ steps.create-pr.outputs.pull-request-number }}
STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_URL: ${{ steps.create-pr.outputs.pull-request-url }}
@@ -26,6 +26,7 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: 'master'
persist-credentials: false
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
@@ -85,9 +86,14 @@ jobs:
- name: PR creation result
run: |
if [[ "${{ steps.create-pr.outputs.pull-request-number }}" ]]; then
echo "✓ Pull request #${{ steps.create-pr.outputs.pull-request-number }} created successfully"
echo "URL: ${{ steps.create-pr.outputs.pull-request-url }}"
if [[ "${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER}" ]]; then
echo "✓ Pull request #${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER} created successfully"
echo "URL: ${STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_URL}"
else
echo "✓ No changes detected - OCI regions are up to date"
fi
env:
STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_NUMBER: ${{ steps.create-pr.outputs.pull-request-number }}
STEPS_CREATE_PR_OUTPUTS_PULL_REQUEST_URL: ${{ steps.create-pr.outputs.pull-request-url }}
+4 -1
View File
@@ -25,12 +25,15 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for SDK changes
id: check-changes
uses: tj-actions/changed-files@e0021407031f5be11a464abee9a0776171c79891 # v47.0.1
with:
files:
files:
./**
.github/workflows/sdk-security.yml
files_ignore: |
+38 -6
View File
@@ -32,6 +32,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for SDK changes
id: check-changes
@@ -119,7 +122,7 @@ jobs:
"wafv2": ["cognito", "elbv2"],
}
changed_raw = """${{ steps.changed-aws.outputs.all_changed_files }}"""
changed_raw = """${STEPS_CHANGED_AWS_OUTPUTS_ALL_CHANGED_FILES}"""
# all_changed_files is space-separated, not newline-separated
# Strip leading "./" if present for consistent path handling
changed_files = [Path(f.lstrip("./")) for f in changed_raw.split() if f]
@@ -174,20 +177,25 @@ jobs:
else:
print("AWS service test paths: none detected")
PY
env:
STEPS_CHANGED_AWS_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-aws.outputs.all_changed_files }}
- name: Run AWS tests
if: steps.changed-aws.outputs.any_changed == 'true'
run: |
echo "AWS run_all=${{ steps.aws-services.outputs.run_all }}"
echo "AWS service_paths='${{ steps.aws-services.outputs.service_paths }}'"
echo "AWS run_all=${STEPS_AWS_SERVICES_OUTPUTS_RUN_ALL}"
echo "AWS service_paths='${STEPS_AWS_SERVICES_OUTPUTS_SERVICE_PATHS}'"
if [ "${{ steps.aws-services.outputs.run_all }}" = "true" ]; then
if [ "${STEPS_AWS_SERVICES_OUTPUTS_RUN_ALL}" = "true" ]; then
poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml tests/providers/aws
elif [ -z "${{ steps.aws-services.outputs.service_paths }}" ]; then
elif [ -z "${STEPS_AWS_SERVICES_OUTPUTS_SERVICE_PATHS}" ]; then
echo "No AWS service paths detected; skipping AWS tests."
else
poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml ${{ steps.aws-services.outputs.service_paths }}
poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml ${STEPS_AWS_SERVICES_OUTPUTS_SERVICE_PATHS}
fi
env:
STEPS_AWS_SERVICES_OUTPUTS_RUN_ALL: ${{ steps.aws-services.outputs.run_all }}
STEPS_AWS_SERVICES_OUTPUTS_SERVICE_PATHS: ${{ steps.aws-services.outputs.service_paths }}
- name: Upload AWS coverage to Codecov
if: steps.changed-aws.outputs.any_changed == 'true'
@@ -438,6 +446,30 @@ jobs:
flags: prowler-py${{ matrix.python-version }}-openstack
files: ./openstack_coverage.xml
# Google Workspace Provider
- name: Check if Google Workspace files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-googleworkspace
uses: tj-actions/changed-files@e0021407031f5be11a464abee9a0776171c79891 # v47.0.1
with:
files: |
./prowler/**/googleworkspace/**
./tests/**/googleworkspace/**
./poetry.lock
- name: Run Google Workspace tests
if: steps.changed-googleworkspace.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/googleworkspace --cov-report=xml:googleworkspace_coverage.xml tests/providers/googleworkspace
- name: Upload Google Workspace coverage to Codecov
if: steps.changed-googleworkspace.outputs.any_changed == 'true'
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-googleworkspace
files: ./googleworkspace_coverage.xml
# Lib
- name: Check if Lib files changed
if: steps.check-changes.outputs.any_changed == 'true'
+30 -14
View File
@@ -49,6 +49,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Get changed files
id: changed-files
@@ -66,47 +69,60 @@ jobs:
id: impact
run: |
echo "Changed files:"
echo "${{ steps.changed-files.outputs.all_changed_files }}" | tr ' ' '\n'
echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n'
echo ""
python .github/scripts/test-impact.py ${{ steps.changed-files.outputs.all_changed_files }}
python .github/scripts/test-impact.py ${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}
env:
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
- name: Set convenience flags
id: set-flags
run: |
if [[ -n "${{ steps.impact.outputs.sdk-tests }}" ]]; then
if [[ -n "${STEPS_IMPACT_OUTPUTS_SDK_TESTS}" ]]; then
echo "has-sdk-tests=true" >> $GITHUB_OUTPUT
else
echo "has-sdk-tests=false" >> $GITHUB_OUTPUT
fi
if [[ -n "${{ steps.impact.outputs.api-tests }}" ]]; then
if [[ -n "${STEPS_IMPACT_OUTPUTS_API_TESTS}" ]]; then
echo "has-api-tests=true" >> $GITHUB_OUTPUT
else
echo "has-api-tests=false" >> $GITHUB_OUTPUT
fi
if [[ -n "${{ steps.impact.outputs.ui-e2e }}" ]]; then
if [[ -n "${STEPS_IMPACT_OUTPUTS_UI_E2E}" ]]; then
echo "has-ui-e2e=true" >> $GITHUB_OUTPUT
else
echo "has-ui-e2e=false" >> $GITHUB_OUTPUT
fi
env:
STEPS_IMPACT_OUTPUTS_SDK_TESTS: ${{ steps.impact.outputs.sdk-tests }}
STEPS_IMPACT_OUTPUTS_API_TESTS: ${{ steps.impact.outputs.api-tests }}
STEPS_IMPACT_OUTPUTS_UI_E2E: ${{ steps.impact.outputs.ui-e2e }}
- name: Summary
run: |
echo "## Test Impact Analysis" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [[ "${{ steps.impact.outputs.run-all }}" == "true" ]]; then
if [[ "${STEPS_IMPACT_OUTPUTS_RUN_ALL}" == "true" ]]; then
echo "🚨 **Critical path changed - running ALL tests**" >> $GITHUB_STEP_SUMMARY
else
echo "### Affected Modules" >> $GITHUB_STEP_SUMMARY
echo "\`${{ steps.impact.outputs.modules }}\`" >> $GITHUB_STEP_SUMMARY
echo "\`${STEPS_IMPACT_OUTPUTS_MODULES}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### Tests to Run" >> $GITHUB_STEP_SUMMARY
echo "| Category | Paths |" >> $GITHUB_STEP_SUMMARY
echo "|----------|-------|" >> $GITHUB_STEP_SUMMARY
echo "| SDK Tests | \`${{ steps.impact.outputs.sdk-tests || 'none' }}\` |" >> $GITHUB_STEP_SUMMARY
echo "| API Tests | \`${{ steps.impact.outputs.api-tests || 'none' }}\` |" >> $GITHUB_STEP_SUMMARY
echo "| UI E2E | \`${{ steps.impact.outputs.ui-e2e || 'none' }}\` |" >> $GITHUB_STEP_SUMMARY
echo "| SDK Tests | \`${STEPS_IMPACT_OUTPUTS_SDK_TESTS:-none}\` |" >> $GITHUB_STEP_SUMMARY
echo "| API Tests | \`${STEPS_IMPACT_OUTPUTS_API_TESTS:-none}\` |" >> $GITHUB_STEP_SUMMARY
echo "| UI E2E | \`${STEPS_IMPACT_OUTPUTS_UI_E2E:-none}\` |" >> $GITHUB_STEP_SUMMARY
fi
env:
STEPS_IMPACT_OUTPUTS_RUN_ALL: ${{ steps.impact.outputs.run-all }}
STEPS_IMPACT_OUTPUTS_SDK_TESTS: ${{ steps.impact.outputs.sdk-tests }}
STEPS_IMPACT_OUTPUTS_API_TESTS: ${{ steps.impact.outputs.api-tests }}
STEPS_IMPACT_OUTPUTS_UI_E2E: ${{ steps.impact.outputs.ui-e2e }}
STEPS_IMPACT_OUTPUTS_MODULES: ${{ steps.impact.outputs.modules }}
+22 -7
View File
@@ -68,17 +68,22 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next minor version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
NEXT_MINOR_VERSION=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).0
echo "NEXT_MINOR_VERSION=${NEXT_MINOR_VERSION}" >> "${GITHUB_ENV}"
echo "Current version: $PROWLER_VERSION"
echo "Next minor version: $NEXT_MINOR_VERSION"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
- name: Bump UI version in .env for master
run: |
@@ -115,11 +120,12 @@ jobs:
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
persist-credentials: false
- name: Calculate first patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
FIRST_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.1
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -129,6 +135,9 @@ jobs:
echo "First patch version: $FIRST_PATCH_VERSION"
echo "Version branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
- name: Bump UI version in .env for version branch
run: |
@@ -172,12 +181,14 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Calculate next patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
MAJOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION}
MINOR_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION}
PATCH_VERSION=${NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION}
NEXT_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.$((PATCH_VERSION + 1))
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
@@ -188,6 +199,10 @@ jobs:
echo "Current version: $PROWLER_VERSION"
echo "Next patch version: $NEXT_PATCH_VERSION"
echo "Target branch: $VERSION_BRANCH"
env:
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MAJOR_VERSION: ${{ needs.detect-release-type.outputs.major_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_MINOR_VERSION: ${{ needs.detect-release-type.outputs.minor_version }}
NEEDS_DETECT_RELEASE_TYPE_OUTPUTS_PATCH_VERSION: ${{ needs.detect-release-type.outputs.patch_version }}
- name: Bump UI version in .env for version branch
run: |
+2
View File
@@ -46,6 +46,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Initialize CodeQL
uses: github/codeql-action/init@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
+25 -10
View File
@@ -60,6 +60,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Notify container push started
id: slack-notification
@@ -96,6 +98,8 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
@@ -143,30 +147,36 @@ jobs:
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${RELEASE_TAG} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${NEEDS_SETUP_OUTPUTS_SHORT_SHA}-arm64" || true
echo "Cleanup completed"
env:
NEEDS_SETUP_OUTPUTS_SHORT_SHA: ${{ needs.setup.outputs.short-sha }}
notify-release-completed:
if: always() && needs.notify-release-started.result == 'success' && (github.event_name == 'release' || github.event_name == 'workflow_dispatch')
@@ -176,15 +186,20 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Determine overall outcome
id: outcome
run: |
if [[ "${{ needs.container-build-push.result }}" == "success" && "${{ needs.create-manifest.result }}" == "success" ]]; then
if [[ "${NEEDS_CONTAINER_BUILD_PUSH_RESULT}" == "success" && "${NEEDS_CREATE_MANIFEST_RESULT}" == "success" ]]; then
echo "outcome=success" >> $GITHUB_OUTPUT
else
echo "outcome=failure" >> $GITHUB_OUTPUT
fi
env:
NEEDS_CONTAINER_BUILD_PUSH_RESULT: ${{ needs.container-build-push.result }}
NEEDS_CREATE_MANIFEST_RESULT: ${{ needs.create-manifest.result }}
- name: Notify container push completed
uses: ./.github/actions/slack-notification
@@ -29,6 +29,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check if Dockerfile changed
id: dockerfile-changed
@@ -64,6 +67,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for UI changes
id: check-changes
+18 -9
View File
@@ -15,6 +15,9 @@ on:
- 'ui/**'
- 'api/**' # API changes can affect UI E2E
permissions:
contents: read
jobs:
# First, analyze which tests need to run
impact-analysis:
@@ -76,20 +79,24 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
persist-credentials: false
- name: Show test scope
run: |
echo "## E2E Test Scope" >> $GITHUB_STEP_SUMMARY
if [[ "${{ env.RUN_ALL_TESTS }}" == "true" ]]; then
if [[ "${RUN_ALL_TESTS}" == "true" ]]; then
echo "Running **ALL** E2E tests (critical path changed)" >> $GITHUB_STEP_SUMMARY
else
echo "Running tests matching: \`${{ env.E2E_TEST_PATHS }}\`" >> $GITHUB_STEP_SUMMARY
echo "Running tests matching: \`${E2E_TEST_PATHS}\`" >> $GITHUB_STEP_SUMMARY
fi
echo ""
echo "Affected modules: \`${{ needs.impact-analysis.outputs.modules }}\`" >> $GITHUB_STEP_SUMMARY
echo "Affected modules: \`${NEEDS_IMPACT_ANALYSIS_OUTPUTS_MODULES}\`" >> $GITHUB_STEP_SUMMARY
env:
NEEDS_IMPACT_ANALYSIS_OUTPUTS_MODULES: ${{ needs.impact-analysis.outputs.modules }}
- name: Create k8s Kind Cluster
uses: helm/kind-action@v1
uses: helm/kind-action@ef37e7f390d99f746eb8b610417061a60e82a6cc # v1
with:
cluster_name: kind
@@ -150,7 +157,7 @@ jobs:
node-version: '24.13.0'
- name: Setup pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@41ff72655975bd51cab0327fa583b6e92b6d3061 # v4
with:
version: 10
run_install: false
@@ -195,14 +202,14 @@ jobs:
- name: Run E2E tests
working-directory: ./ui
run: |
if [[ "${{ env.RUN_ALL_TESTS }}" == "true" ]]; then
if [[ "${RUN_ALL_TESTS}" == "true" ]]; then
echo "Running ALL E2E tests..."
pnpm run test:e2e
else
echo "Running targeted E2E tests: ${{ env.E2E_TEST_PATHS }}"
echo "Running targeted E2E tests: ${E2E_TEST_PATHS}"
# Convert glob patterns to playwright test paths
# e.g., "ui/tests/providers/**" -> "tests/providers"
TEST_PATHS="${{ env.E2E_TEST_PATHS }}"
TEST_PATHS="${E2E_TEST_PATHS}"
# Remove ui/ prefix and convert ** to empty (playwright handles recursion)
TEST_PATHS=$(echo "$TEST_PATHS" | sed 's|ui/||g' | sed 's|\*\*||g' | tr ' ' '\n' | sort -u)
# Drop auth setup helpers (not runnable test suites)
@@ -244,6 +251,8 @@ jobs:
echo "" >> $GITHUB_STEP_SUMMARY
echo "No UI E2E tests needed for this change." >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Affected modules: \`${{ needs.impact-analysis.outputs.modules }}\`" >> $GITHUB_STEP_SUMMARY
echo "Affected modules: \`${NEEDS_IMPACT_ANALYSIS_OUTPUTS_MODULES}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "To run all tests, modify a file in a critical path (e.g., \`ui/lib/**\`)." >> $GITHUB_STEP_SUMMARY
env:
NEEDS_IMPACT_ANALYSIS_OUTPUTS_MODULES: ${{ needs.impact-analysis.outputs.modules }}
+8 -3
View File
@@ -31,6 +31,9 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
- name: Check for UI changes
id: check-changes
@@ -81,7 +84,7 @@ jobs:
- name: Setup pnpm
if: steps.check-changes.outputs.any_changed == 'true'
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@41ff72655975bd51cab0327fa583b6e92b6d3061 # v4
with:
version: 10
run_install: false
@@ -122,10 +125,12 @@ jobs:
if: steps.check-changes.outputs.any_changed == 'true' && steps.critical-changes.outputs.any_changed != 'true' && steps.changed-source.outputs.all_changed_files != ''
run: |
echo "Running tests related to changed files:"
echo "${{ steps.changed-source.outputs.all_changed_files }}"
echo "${STEPS_CHANGED_SOURCE_OUTPUTS_ALL_CHANGED_FILES}"
# Convert space-separated to vitest related format (remove ui/ prefix for relative paths)
CHANGED_FILES=$(echo "${{ steps.changed-source.outputs.all_changed_files }}" | tr ' ' '\n' | sed 's|^ui/||' | tr '\n' ' ')
CHANGED_FILES=$(echo "${STEPS_CHANGED_SOURCE_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | sed 's|^ui/||' | tr '\n' ' ')
pnpm exec vitest related $CHANGED_FILES --run
env:
STEPS_CHANGED_SOURCE_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-source.outputs.all_changed_files }}
- name: Run unit tests (test files only changed)
if: steps.check-changes.outputs.any_changed == 'true' && steps.critical-changes.outputs.any_changed != 'true' && steps.changed-source.outputs.all_changed_files == ''
+7 -11
View File
@@ -148,21 +148,17 @@ Prowler App offers flexible installation methods tailored to various environment
**Commands**
``` console
curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/docker-compose.yml
curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/.env
VERSION=$(curl -s https://api.github.com/repos/prowler-cloud/prowler/releases/latest | jq -r .tag_name)
curl -sLO "https://raw.githubusercontent.com/prowler-cloud/prowler/refs/tags/${VERSION}/docker-compose.yml"
# Environment variables can be customized in the .env file. Using default values in production environments is not recommended.
curl -sLO "https://raw.githubusercontent.com/prowler-cloud/prowler/refs/tags/${VERSION}/.env"
docker compose up -d
```
> Containers are built for `linux/amd64`.
> [!WARNING]
> 🔒 For a secure setup, the API auto-generates a unique key pair, `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY`, and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate keys, delete the stored key files and restart the API.
### Configuring Your Workstation for Prowler App
If your workstation's architecture is incompatible, you can resolve this by:
- **Setting the environment variable**: `DOCKER_DEFAULT_PLATFORM=linux/amd64`
- **Using the following flag in your Docker command**: `--platform linux/amd64`
> Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
### Common Issues with Docker Pull Installation
+6
View File
@@ -9,6 +9,8 @@ All notable changes to the **Prowler API** are documented in this file.
- Finding group summaries and resources endpoints for hierarchical findings views [(#9961)](https://github.com/prowler-cloud/prowler/pull/9961)
- OpenStack provider support [(#10003)](https://github.com/prowler-cloud/prowler/pull/10003)
- PDF report for the CSA CCM compliance framework [(#10088)](https://github.com/prowler-cloud/prowler/pull/10088)
- `image` provider support for container image scanning [(#10128)](https://github.com/prowler-cloud/prowler/pull/10128)
- Attack Paths: Custom query and Cartography schema endpoints (temporarily blocked) [(#10149)](https://github.com/prowler-cloud/prowler/pull/10149)
### 🔄 Changed
@@ -28,11 +30,15 @@ All notable changes to the **Prowler API** are documented in this file.
- Attack Paths: Query results now filtered by provider, preventing future cross-tenant and cross-provider data leakage [(#10118)](https://github.com/prowler-cloud/prowler/pull/10118)
- Attack Paths: Add private labels and properties in Attack Paths graphs for avoiding future overlapping with Cartography's ones [(#10124)](https://github.com/prowler-cloud/prowler/pull/10124)
- Attack Paths: Query endpoint executes them in read only mode [(#10140)](https://github.com/prowler-cloud/prowler/pull/10140)
- Attack Paths: `Accept` header query endpoints also accepts `text/plain`, supporting compact plain-text format for LLM consumption [(#10162)](https://github.com/prowler-cloud/prowler/pull/10162)
### 🐞 Fixed
- Attack Paths: Orphaned temporary Neo4j databases are now cleaned up on scan failure and provider deletion [(#10101)](https://github.com/prowler-cloud/prowler/pull/10101)
- Attack Paths: scan no longer raises `DatabaseError` when provider is deleted mid-scan [(#10116)](https://github.com/prowler-cloud/prowler/pull/10116)
- Tenant compliance summaries recalculated after provider deletion [(#10172)](https://github.com/prowler-cloud/prowler/pull/10172)
- Security Hub export retries transient replica conflicts without failing integrations [(#10144)](https://github.com/prowler-cloud/prowler/pull/10144)
- Cloudflare provider secrets now reject API key format in `api_token` and non-key values in `api_key` credentials [(#10195)](https://github.com/prowler-cloud/prowler/pull/10195)
### 🔐 Security
+1 -1
View File
@@ -5,7 +5,7 @@ LABEL maintainer="https://github.com/prowler-cloud/api"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
ARG TRIVY_VERSION=0.66.0
ARG TRIVY_VERSION=0.69.1
ENV TRIVY_VERSION=${TRIVY_VERSION}
# hadolint ignore=DL3008
+1 -1
View File
@@ -6745,7 +6745,7 @@ tzlocal = "5.3.1"
type = "git"
url = "https://github.com/prowler-cloud/prowler.git"
reference = "master"
resolved_reference = "ceb4691c3657e7db3d178896bfc241d14f194295"
resolved_reference = "6962622fd21401886371add25463f77228cd9c1f"
[[package]]
name = "psutil"
@@ -30,6 +30,7 @@ SERVICE_UNAVAILABLE_MAX_RETRIES = env.int(
READ_QUERY_TIMEOUT_SECONDS = env.int(
"ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30
)
MAX_CUSTOM_QUERY_NODES = env.int("ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES", default=250)
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
@@ -0,0 +1,19 @@
from tasks.jobs.attack_paths.config import DEPRECATED_PROVIDER_RESOURCE_LABEL
CARTOGRAPHY_SCHEMA_METADATA = f"""
MATCH (n:{DEPRECATED_PROVIDER_RESOURCE_LABEL} {{provider_id: $provider_id}})
WHERE n._module_name STARTS WITH 'cartography:'
AND NOT n._module_name IN ['cartography:ontology', 'cartography:prowler']
AND n._module_version IS NOT NULL
RETURN n._module_name AS module_name, n._module_version AS module_version
LIMIT 1
"""
GITHUB_SCHEMA_URL = (
"https://github.com/cartography-cncf/cartography/blob/"
"{version}/docs/root/modules/{provider}/schema.md"
)
RAW_SCHEMA_URL = (
"https://raw.githubusercontent.com/cartography-cncf/cartography/"
"refs/tags/{version}/docs/root/modules/{provider}/schema.md"
)
@@ -2,16 +2,25 @@ import logging
from typing import Any, Iterable
import neo4j
from rest_framework.exceptions import APIException, PermissionDenied, ValidationError
from api.attack_paths import database as graph_database, AttackPathsQueryDefinition
from api.attack_paths.queries.schema import (
CARTOGRAPHY_SCHEMA_METADATA,
GITHUB_SCHEMA_URL,
RAW_SCHEMA_URL,
)
from config.custom_logging import BackendLogger
from tasks.jobs.attack_paths.config import INTERNAL_LABELS
from tasks.jobs.attack_paths.config import INTERNAL_LABELS, INTERNAL_PROPERTIES
logger = logging.getLogger(BackendLogger.API)
def normalize_run_payload(raw_data):
# Predefined query helpers
def normalize_query_payload(raw_data):
if not isinstance(raw_data, dict): # Let the serializer handle this
return raw_data
@@ -31,7 +40,7 @@ def normalize_run_payload(raw_data):
return raw_data
def prepare_query_parameters(
def prepare_parameters(
definition: AttackPathsQueryDefinition,
provided_parameters: dict[str, Any],
provider_uid: str,
@@ -80,7 +89,7 @@ def prepare_query_parameters(
return clean_parameters
def execute_attack_paths_query(
def execute_query(
database_name: str,
definition: AttackPathsQueryDefinition,
parameters: dict[str, Any],
@@ -106,7 +115,103 @@ def execute_attack_paths_query(
)
def _serialize_graph(graph, provider_id: str):
# Custom query helpers
def normalize_custom_query_payload(raw_data):
if not isinstance(raw_data, dict):
return raw_data
if "data" in raw_data and isinstance(raw_data.get("data"), dict):
data_section = raw_data.get("data") or {}
attributes = data_section.get("attributes") or {}
return {"query": attributes.get("query")}
return raw_data
def execute_custom_query(
database_name: str,
cypher: str,
provider_id: str,
) -> dict[str, Any]:
try:
graph = graph_database.execute_read_query(
database=database_name,
cypher=cypher,
)
serialized = _serialize_graph(graph, provider_id)
return _truncate_graph(serialized)
except graph_database.WriteQueryNotAllowedException:
raise PermissionDenied(
"Attack Paths query execution failed: read-only queries are enforced"
)
except graph_database.GraphDatabaseQueryException as exc:
logger.error(f"Custom cypher query failed: {exc}")
raise APIException(
"Attack Paths query execution failed due to a database error"
)
# Cartography schema helpers
def get_cartography_schema(
database_name: str, provider_id: str
) -> dict[str, str] | None:
try:
with graph_database.get_session(
database_name, default_access_mode=neo4j.READ_ACCESS
) as session:
result = session.run(
CARTOGRAPHY_SCHEMA_METADATA,
{"provider_id": provider_id},
)
record = result.single()
except graph_database.GraphDatabaseQueryException as exc:
logger.error(f"Cartography schema query failed: {exc}")
raise APIException(
"Unable to retrieve cartography schema due to a database error"
)
if not record:
return None
module_name = record["module_name"]
version = record["module_version"]
provider = module_name.split(":")[1]
return {
"id": f"{provider}-{version}",
"provider": provider,
"cartography_version": version,
"schema_url": GITHUB_SCHEMA_URL.format(version=version, provider=provider),
"raw_schema_url": RAW_SCHEMA_URL.format(version=version, provider=provider),
}
# Private helpers
def _truncate_graph(graph: dict[str, Any]) -> dict[str, Any]:
if graph["total_nodes"] > graph_database.MAX_CUSTOM_QUERY_NODES:
graph["truncated"] = True
graph["nodes"] = graph["nodes"][: graph_database.MAX_CUSTOM_QUERY_NODES]
kept_node_ids = {node["id"] for node in graph["nodes"]}
graph["relationships"] = [
rel
for rel in graph["relationships"]
if rel["source"] in kept_node_ids and rel["target"] in kept_node_ids
]
return graph
def _serialize_graph(graph, provider_id: str) -> dict[str, Any]:
nodes = []
kept_node_ids = set()
for node in graph.nodes:
@@ -146,6 +251,8 @@ def _serialize_graph(graph, provider_id: str):
return {
"nodes": nodes,
"relationships": relationships,
"total_nodes": len(nodes),
"truncated": False,
}
@@ -154,7 +261,11 @@ def _filter_labels(labels: Iterable[str]) -> list[str]:
def _serialize_properties(properties: dict[str, Any]) -> dict[str, Any]:
"""Convert Neo4j property values into JSON-serializable primitives."""
"""Convert Neo4j property values into JSON-serializable primitives.
Filters out internal properties (Cartography metadata and provider
isolation fields) defined in INTERNAL_PROPERTIES.
"""
def _serialize_value(value: Any) -> Any:
# Neo4j temporal and spatial values expose `to_native` returning Python primitives
@@ -169,4 +280,176 @@ def _serialize_properties(properties: dict[str, Any]) -> dict[str, Any]:
return value
return {key: _serialize_value(val) for key, val in properties.items()}
return {
key: _serialize_value(val)
for key, val in properties.items()
if key not in INTERNAL_PROPERTIES
}
# Text serialization
def serialize_graph_as_text(graph: dict[str, Any]) -> str:
"""
Convert a serialized graph dict into a compact text format for LLM consumption.
Follows the incident-encoding pattern (nodes with context + sequential edges)
which research shows is optimal for LLM path-reasoning tasks.
Example::
>>> serialize_graph_as_text({
... "nodes": [
... {"id": "n1", "labels": ["AWSAccount"], "properties": {"name": "prod"}},
... {"id": "n2", "labels": ["EC2Instance"], "properties": {}},
... ],
... "relationships": [
... {"id": "r1", "label": "RESOURCE", "source": "n1", "target": "n2", "properties": {}},
... ],
... "total_nodes": 2, "truncated": False,
... })
## Nodes (2)
- AWSAccount "n1" (name: "prod")
- EC2Instance "n2"
## Relationships (1)
- AWSAccount "n1" -[RESOURCE]-> EC2Instance "n2"
## Summary
- Total nodes: 2
- Truncated: false
"""
nodes = graph.get("nodes", [])
relationships = graph.get("relationships", [])
node_lookup = {node["id"]: node for node in nodes}
lines = [f"## Nodes ({len(nodes)})"]
for node in nodes:
lines.append(f"- {_format_node_signature(node)}")
lines.append("")
lines.append(f"## Relationships ({len(relationships)})")
for rel in relationships:
lines.append(f"- {_format_relationship(rel, node_lookup)}")
lines.append("")
lines.append("## Summary")
lines.append(f"- Total nodes: {graph.get('total_nodes', len(nodes))}")
lines.append(f"- Truncated: {str(graph.get('truncated', False)).lower()}")
return "\n".join(lines)
def _format_node_signature(node: dict[str, Any]) -> str:
"""
Format a node as its reference followed by its properties.
Example::
>>> _format_node_signature({"id": "n1", "labels": ["AWSRole"], "properties": {"name": "admin"}})
'AWSRole "n1" (name: "admin")'
>>> _format_node_signature({"id": "n2", "labels": ["AWSAccount"], "properties": {}})
'AWSAccount "n2"'
"""
reference = _format_node_reference(node)
properties = _format_properties(node.get("properties", {}))
if properties:
return f"{reference} {properties}"
return reference
def _format_node_reference(node: dict[str, Any]) -> str:
"""
Format a node as labels + quoted id (no properties).
Example::
>>> _format_node_reference({"id": "n1", "labels": ["EC2Instance", "NetworkExposed"]})
'EC2Instance, NetworkExposed "n1"'
"""
labels = ", ".join(node.get("labels", []))
return f'{labels} "{node["id"]}"'
def _format_relationship(rel: dict[str, Any], node_lookup: dict[str, dict]) -> str:
"""
Format a relationship as source -[LABEL (props)]-> target.
Example::
>>> _format_relationship(
... {"id": "r1", "label": "STS_ASSUMEROLE_ALLOW", "source": "n1", "target": "n2",
... "properties": {"weight": 1}},
... {"n1": {"id": "n1", "labels": ["AWSRole"]},
... "n2": {"id": "n2", "labels": ["AWSRole"]}},
... )
'AWSRole "n1" -[STS_ASSUMEROLE_ALLOW (weight: 1)]-> AWSRole "n2"'
"""
source = _format_node_reference(node_lookup[rel["source"]])
target = _format_node_reference(node_lookup[rel["target"]])
props = _format_properties(rel.get("properties", {}))
label = f"{rel['label']} {props}" if props else rel["label"]
return f"{source} -[{label}]-> {target}"
def _format_properties(properties: dict[str, Any]) -> str:
"""
Format properties as a parenthesized key-value list.
Returns an empty string when no properties are present.
Example::
>>> _format_properties({"name": "prod", "account_id": "123456789012"})
'(name: "prod", account_id: "123456789012")'
>>> _format_properties({})
''
"""
if not properties:
return ""
parts = [f"{k}: {_format_value(v)}" for k, v in properties.items()]
return f"({', '.join(parts)})"
def _format_value(value: Any) -> str:
"""
Format a value using Cypher-style syntax (unquoted dict keys, lowercase bools).
Example::
>>> _format_value("prod")
'"prod"'
>>> _format_value(True)
'true'
>>> _format_value([80, 443])
'[80, 443]'
>>> _format_value({"env": "prod"})
'{env: "prod"}'
>>> _format_value(None)
'null'
"""
if isinstance(value, str):
return f'"{value}"'
if isinstance(value, bool):
return str(value).lower()
if isinstance(value, (list, tuple)):
inner = ", ".join(_format_value(v) for v in value)
return f"[{inner}]"
if isinstance(value, dict):
inner = ", ".join(f"{k}: {_format_value(v)}" for k, v in value.items())
return f"{{{inner}}}"
if value is None:
return "null"
return str(value)
+6 -1
View File
@@ -74,6 +74,7 @@ def rls_transaction(
value: str,
parameter: str = POSTGRES_TENANT_VAR,
using: str | None = None,
retry_on_replica: bool = True,
):
"""
Creates a new database transaction setting the given configuration value for Postgres RLS. It validates the
@@ -92,10 +93,11 @@ def rls_transaction(
alias = db_alias
is_replica = READ_REPLICA_ALIAS and alias == READ_REPLICA_ALIAS
max_attempts = REPLICA_MAX_ATTEMPTS if is_replica else 1
max_attempts = REPLICA_MAX_ATTEMPTS if is_replica and retry_on_replica else 1
for attempt in range(1, max_attempts + 1):
router_token = None
yielded_cursor = False
# On final attempt, fallback to primary
if attempt == max_attempts and is_replica:
@@ -118,9 +120,12 @@ def rls_transaction(
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yielded_cursor = True
yield cursor
return
except OperationalError as e:
if yielded_cursor:
raise
# If on primary or max attempts reached, raise
if not is_replica or attempt == max_attempts:
raise
@@ -0,0 +1,38 @@
from django.db import migrations
import api.db_utils
class Migration(migrations.Migration):
dependencies = [
("api", "0082_backfill_finding_group_summaries"),
]
operations = [
migrations.AlterField(
model_name="provider",
name="provider",
field=api.db_utils.ProviderEnumField(
choices=[
("aws", "AWS"),
("azure", "Azure"),
("gcp", "GCP"),
("kubernetes", "Kubernetes"),
("m365", "M365"),
("github", "GitHub"),
("mongodbatlas", "MongoDB Atlas"),
("iac", "IaC"),
("oraclecloud", "Oracle Cloud Infrastructure"),
("alibabacloud", "Alibaba Cloud"),
("cloudflare", "Cloudflare"),
("openstack", "OpenStack"),
("image", "Image"),
],
default="aws",
),
),
migrations.RunSQL(
"ALTER TYPE provider ADD VALUE IF NOT EXISTS 'image';",
reverse_sql=migrations.RunSQL.noop,
),
]
+10
View File
@@ -292,6 +292,7 @@ class Provider(RowLevelSecurityProtectedModel):
ALIBABACLOUD = "alibabacloud", _("Alibaba Cloud")
CLOUDFLARE = "cloudflare", _("Cloudflare")
OPENSTACK = "openstack", _("OpenStack")
IMAGE = "image", _("Image")
@staticmethod
def validate_aws_uid(value):
@@ -426,6 +427,15 @@ class Provider(RowLevelSecurityProtectedModel):
pointer="/data/attributes/uid",
)
@staticmethod
def validate_image_uid(value):
if not re.match(r"^[a-zA-Z0-9][a-zA-Z0-9._/:@-]{2,249}$", value):
raise ModelValidationError(
detail="Image provider ID must be a valid container image reference.",
code="image-uid",
pointer="/data/attributes/uid",
)
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
+15 -1
View File
@@ -1,15 +1,29 @@
from contextlib import nullcontext
from rest_framework.renderers import BaseRenderer
from rest_framework_json_api.renderers import JSONRenderer
from api.db_utils import rls_transaction
class PlainTextRenderer(BaseRenderer):
media_type = "text/plain"
format = "text"
def render(self, data, accepted_media_type=None, renderer_context=None):
encoding = self.charset or "utf-8"
if isinstance(data, str):
return data.encode(encoding)
if data is None:
return b""
return str(data).encode(encoding)
class APIJSONRenderer(JSONRenderer):
"""JSONRenderer override to apply tenant RLS when there are included resources in the request."""
def render(self, data, accepted_media_type=None, renderer_context=None):
request = renderer_context.get("request")
request = renderer_context.get("request") if renderer_context else None
tenant_id = getattr(request, "tenant_id", None) if request else None
db_alias = getattr(request, "db_alias", None) if request else None
include_param_present = "include" in request.query_params if request else False
File diff suppressed because it is too large. Load diff
+412 -16
View File
@@ -16,7 +16,7 @@ def _make_neo4j_error(message, code):
return neo4j.exceptions.Neo4jError._hydrate_neo4j(code=code, message=message)
def test_normalize_run_payload_extracts_attributes_section():
def test_normalize_query_payload_extracts_attributes_section():
payload = {
"data": {
"id": "ignored",
@@ -27,21 +27,21 @@ def test_normalize_run_payload_extracts_attributes_section():
}
}
result = views_helpers.normalize_run_payload(payload)
result = views_helpers.normalize_query_payload(payload)
assert result == {"id": "aws-rds", "parameters": {"ip": "192.0.2.0"}}
def test_normalize_run_payload_passthrough_for_non_dict():
def test_normalize_query_payload_passthrough_for_non_dict():
sentinel = "not-a-dict"
assert views_helpers.normalize_run_payload(sentinel) is sentinel
assert views_helpers.normalize_query_payload(sentinel) is sentinel
def test_prepare_query_parameters_includes_provider_and_casts(
def test_prepare_parameters_includes_provider_and_casts(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(cast_type=int)
result = views_helpers.prepare_query_parameters(
result = views_helpers.prepare_parameters(
definition,
{"limit": "5"},
provider_uid="123456789012",
@@ -60,26 +60,26 @@ def test_prepare_query_parameters_includes_provider_and_casts(
({"limit": 10, "extra": True}, "Unknown parameter"),
],
)
def test_prepare_query_parameters_validates_names(
def test_prepare_parameters_validates_names(
attack_paths_query_definition_factory, provided, expected_message
):
definition = attack_paths_query_definition_factory()
with pytest.raises(ValidationError) as exc:
views_helpers.prepare_query_parameters(
views_helpers.prepare_parameters(
definition, provided, provider_uid="1", provider_id="p1"
)
assert expected_message in str(exc.value)
def test_prepare_query_parameters_validates_cast(
def test_prepare_parameters_validates_cast(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(cast_type=int)
with pytest.raises(ValidationError) as exc:
views_helpers.prepare_query_parameters(
views_helpers.prepare_parameters(
definition,
{"limit": "not-an-int"},
provider_uid="1",
@@ -89,7 +89,7 @@ def test_prepare_query_parameters_validates_cast(
assert "Invalid value" in str(exc.value)
def test_execute_attack_paths_query_serializes_graph(
def test_execute_query_serializes_graph(
attack_paths_query_definition_factory, attack_paths_graph_stub_classes
):
definition = attack_paths_query_definition_factory(
@@ -139,7 +139,7 @@ def test_execute_attack_paths_query_serializes_graph(
"api.attack_paths.views_helpers.graph_database.execute_read_query",
return_value=graph_result,
) as mock_execute_read_query:
result = views_helpers.execute_attack_paths_query(
result = views_helpers.execute_query(
database_name, definition, parameters, provider_id=provider_id
)
@@ -153,7 +153,7 @@ def test_execute_attack_paths_query_serializes_graph(
assert result["relationships"][0]["label"] == "OWNS"
def test_execute_attack_paths_query_wraps_graph_errors(
def test_execute_query_wraps_graph_errors(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(
@@ -175,14 +175,14 @@ def test_execute_attack_paths_query_wraps_graph_errors(
patch("api.attack_paths.views_helpers.logger") as mock_logger,
):
with pytest.raises(APIException):
views_helpers.execute_attack_paths_query(
views_helpers.execute_query(
database_name, definition, parameters, provider_id="test-provider-123"
)
mock_logger.error.assert_called_once()
def test_execute_attack_paths_query_raises_permission_denied_on_read_only(
def test_execute_query_raises_permission_denied_on_read_only(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(
@@ -204,7 +204,7 @@ def test_execute_attack_paths_query_raises_permission_denied_on_read_only(
),
):
with pytest.raises(PermissionDenied):
views_helpers.execute_attack_paths_query(
views_helpers.execute_query(
database_name, definition, parameters, provider_id="test-provider-123"
)
@@ -242,6 +242,319 @@ def test_serialize_graph_filters_by_provider_id(attack_paths_graph_stub_classes)
assert result["relationships"][0]["id"] == "r1"
# -- serialize_graph_as_text -------------------------------------------------------
def test_serialize_graph_as_text_renders_nodes_and_relationships():
graph = {
"nodes": [
{
"id": "n1",
"labels": ["AWSAccount"],
"properties": {"account_id": "123456789012", "name": "prod"},
},
{
"id": "n2",
"labels": ["EC2Instance", "NetworkExposed"],
"properties": {"name": "web-server-1", "exposed_internet": True},
},
],
"relationships": [
{
"id": "r1",
"label": "RESOURCE",
"source": "n1",
"target": "n2",
"properties": {},
},
],
"total_nodes": 2,
"truncated": False,
}
result = views_helpers.serialize_graph_as_text(graph)
assert result.startswith("## Nodes (2)")
assert '- AWSAccount "n1" (account_id: "123456789012", name: "prod")' in result
assert (
'- EC2Instance, NetworkExposed "n2" (name: "web-server-1", exposed_internet: true)'
in result
)
assert "## Relationships (1)" in result
assert '- AWSAccount "n1" -[RESOURCE]-> EC2Instance, NetworkExposed "n2"' in result
assert "## Summary" in result
assert "- Total nodes: 2" in result
assert "- Truncated: false" in result
def test_serialize_graph_as_text_empty_graph():
graph = {
"nodes": [],
"relationships": [],
"total_nodes": 0,
"truncated": False,
}
result = views_helpers.serialize_graph_as_text(graph)
assert "## Nodes (0)" in result
assert "## Relationships (0)" in result
assert "- Total nodes: 0" in result
assert "- Truncated: false" in result
def test_serialize_graph_as_text_truncated_flag():
graph = {
"nodes": [{"id": "n1", "labels": ["Node"], "properties": {}}],
"relationships": [],
"total_nodes": 500,
"truncated": True,
}
result = views_helpers.serialize_graph_as_text(graph)
assert "- Total nodes: 500" in result
assert "- Truncated: true" in result
def test_serialize_graph_as_text_relationship_with_properties():
graph = {
"nodes": [
{"id": "n1", "labels": ["AWSRole"], "properties": {"name": "role-a"}},
{"id": "n2", "labels": ["AWSRole"], "properties": {"name": "role-b"}},
],
"relationships": [
{
"id": "r1",
"label": "STS_ASSUMEROLE_ALLOW",
"source": "n1",
"target": "n2",
"properties": {"weight": 1, "reason": "trust-policy"},
},
],
"total_nodes": 2,
"truncated": False,
}
result = views_helpers.serialize_graph_as_text(graph)
assert '-[STS_ASSUMEROLE_ALLOW (weight: 1, reason: "trust-policy")]->' in result
def test_serialize_properties_filters_internal_fields():
properties = {
"name": "prod",
# Cartography metadata
"lastupdated": 1234567890,
"firstseen": 1234567800,
"_module_name": "cartography:aws",
"_module_version": "0.98.0",
# Provider isolation
"_provider_id": "42",
"_provider_element_id": "42:abc123",
"provider_id": "42",
"provider_element_id": "42:abc123",
}
result = views_helpers._serialize_properties(properties)
assert result == {"name": "prod"}
def test_serialize_graph_as_text_node_without_properties():
graph = {
"nodes": [{"id": "n1", "labels": ["AWSAccount"], "properties": {}}],
"relationships": [],
"total_nodes": 1,
"truncated": False,
}
result = views_helpers.serialize_graph_as_text(graph)
assert '- AWSAccount "n1"' in result
# No trailing parentheses when no properties
assert '- AWSAccount "n1" (' not in result
def test_serialize_graph_as_text_complex_property_values():
graph = {
"nodes": [
{
"id": "n1",
"labels": ["SecurityGroup"],
"properties": {
"ports": [80, 443],
"tags": {"env": "prod"},
"enabled": None,
},
},
],
"relationships": [],
"total_nodes": 1,
"truncated": False,
}
result = views_helpers.serialize_graph_as_text(graph)
assert "ports: [80, 443]" in result
assert 'tags: {env: "prod"}' in result
assert "enabled: null" in result
# -- normalize_custom_query_payload ------------------------------------------------
def test_normalize_custom_query_payload_extracts_query():
payload = {
"data": {
"type": "attack-paths-custom-query-run-requests",
"attributes": {
"query": "MATCH (n) RETURN n",
},
}
}
result = views_helpers.normalize_custom_query_payload(payload)
assert result == {"query": "MATCH (n) RETURN n"}
def test_normalize_custom_query_payload_passthrough_for_non_dict():
sentinel = "not-a-dict"
assert views_helpers.normalize_custom_query_payload(sentinel) is sentinel
def test_normalize_custom_query_payload_passthrough_for_flat_dict():
payload = {"query": "MATCH (n) RETURN n"}
result = views_helpers.normalize_custom_query_payload(payload)
assert result == {"query": "MATCH (n) RETURN n"}
# -- execute_custom_query ----------------------------------------------
def test_execute_custom_query_serializes_graph(
attack_paths_graph_stub_classes,
):
provider_id = "test-provider-123"
node_1 = attack_paths_graph_stub_classes.Node(
"node-1", ["AWSAccount"], {"provider_id": provider_id}
)
node_2 = attack_paths_graph_stub_classes.Node(
"node-2", ["RDSInstance"], {"provider_id": provider_id}
)
relationship = attack_paths_graph_stub_classes.Relationship(
"rel-1", "OWNS", node_1, node_2, {"provider_id": provider_id}
)
graph_result = MagicMock()
graph_result.nodes = [node_1, node_2]
graph_result.relationships = [relationship]
with patch(
"api.attack_paths.views_helpers.graph_database.execute_read_query",
return_value=graph_result,
) as mock_execute:
result = views_helpers.execute_custom_query(
"db-tenant-test", "MATCH (n) RETURN n", provider_id
)
mock_execute.assert_called_once_with(
database="db-tenant-test",
cypher="MATCH (n) RETURN n",
)
assert len(result["nodes"]) == 2
assert result["relationships"][0]["label"] == "OWNS"
assert result["truncated"] is False
assert result["total_nodes"] == 2
def test_execute_custom_query_raises_permission_denied_on_write():
with patch(
"api.attack_paths.views_helpers.graph_database.execute_read_query",
side_effect=graph_database.WriteQueryNotAllowedException(
message="Read query not allowed",
code="Neo.ClientError.Statement.AccessMode",
),
):
with pytest.raises(PermissionDenied):
views_helpers.execute_custom_query(
"db-tenant-test", "CREATE (n) RETURN n", "provider-1"
)
def test_execute_custom_query_wraps_graph_errors():
with (
patch(
"api.attack_paths.views_helpers.graph_database.execute_read_query",
side_effect=graph_database.GraphDatabaseQueryException("boom"),
),
patch("api.attack_paths.views_helpers.logger") as mock_logger,
):
with pytest.raises(APIException):
views_helpers.execute_custom_query(
"db-tenant-test", "MATCH (n) RETURN n", "provider-1"
)
mock_logger.error.assert_called_once()
# -- _truncate_graph ----------------------------------------------------------
def test_truncate_graph_no_truncation_needed():
graph = {
"nodes": [{"id": f"n{i}"} for i in range(5)],
"relationships": [{"id": "r1", "source": "n0", "target": "n1"}],
"total_nodes": 5,
"truncated": False,
}
result = views_helpers._truncate_graph(graph)
assert result["truncated"] is False
assert result["total_nodes"] == 5
assert len(result["nodes"]) == 5
assert len(result["relationships"]) == 1
def test_truncate_graph_truncates_nodes_and_removes_orphan_relationships():
with patch.object(graph_database, "MAX_CUSTOM_QUERY_NODES", 3):
graph = {
"nodes": [{"id": f"n{i}"} for i in range(5)],
"relationships": [
{"id": "r1", "source": "n0", "target": "n1"},
{"id": "r2", "source": "n0", "target": "n4"},
{"id": "r3", "source": "n3", "target": "n4"},
],
"total_nodes": 5,
"truncated": False,
}
result = views_helpers._truncate_graph(graph)
assert result["truncated"] is True
assert result["total_nodes"] == 5
assert len(result["nodes"]) == 3
assert {n["id"] for n in result["nodes"]} == {"n0", "n1", "n2"}
# r1 kept (both endpoints in n0-n2), r2 and r3 dropped (n4 not in kept set)
assert len(result["relationships"]) == 1
assert result["relationships"][0]["id"] == "r1"
def test_truncate_graph_empty_graph():
graph = {"nodes": [], "relationships": [], "total_nodes": 0, "truncated": False}
result = views_helpers._truncate_graph(graph)
assert result["truncated"] is False
assert result["total_nodes"] == 0
assert result["nodes"] == []
assert result["relationships"] == []
# -- execute_read_query read-only enforcement ---------------------------------
@@ -342,3 +655,86 @@ def test_execute_read_query_rejects_apoc_real_create(mock_neo4j_session, cypher)
with pytest.raises(graph_database.WriteQueryNotAllowedException):
graph_database.execute_read_query(database="test-db", cypher=cypher)
# -- get_cartography_schema ---------------------------------------------------
@pytest.fixture
def mock_schema_session():
"""Mock get_session for cartography schema tests."""
mock_result = MagicMock()
mock_session = MagicMock()
mock_session.run.return_value = mock_result
with patch(
"api.attack_paths.views_helpers.graph_database.get_session"
) as mock_get_session:
mock_get_session.return_value.__enter__ = MagicMock(return_value=mock_session)
mock_get_session.return_value.__exit__ = MagicMock(return_value=False)
yield mock_session, mock_result
def test_get_cartography_schema_returns_urls(mock_schema_session):
mock_session, mock_result = mock_schema_session
mock_result.single.return_value = {
"module_name": "cartography:aws",
"module_version": "0.129.0",
}
result = views_helpers.get_cartography_schema("db-tenant-test", "provider-123")
mock_session.run.assert_called_once()
assert result["id"] == "aws-0.129.0"
assert result["provider"] == "aws"
assert result["cartography_version"] == "0.129.0"
assert "0.129.0" in result["schema_url"]
assert "/aws/" in result["schema_url"]
assert "raw.githubusercontent.com" in result["raw_schema_url"]
assert "/aws/" in result["raw_schema_url"]
def test_get_cartography_schema_returns_none_when_no_data(mock_schema_session):
_, mock_result = mock_schema_session
mock_result.single.return_value = None
result = views_helpers.get_cartography_schema("db-tenant-test", "provider-123")
assert result is None
@pytest.mark.parametrize(
"module_name,expected_provider",
[
("cartography:aws", "aws"),
("cartography:azure", "azure"),
("cartography:gcp", "gcp"),
],
)
def test_get_cartography_schema_extracts_provider(
mock_schema_session, module_name, expected_provider
):
_, mock_result = mock_schema_session
mock_result.single.return_value = {
"module_name": module_name,
"module_version": "1.0.0",
}
result = views_helpers.get_cartography_schema("db-tenant-test", "provider-123")
assert result["id"] == f"{expected_provider}-1.0.0"
assert result["provider"] == expected_provider
def test_get_cartography_schema_wraps_database_error():
with (
patch(
"api.attack_paths.views_helpers.graph_database.get_session",
side_effect=graph_database.GraphDatabaseQueryException("boom"),
),
patch("api.attack_paths.views_helpers.logger") as mock_logger,
):
with pytest.raises(APIException):
views_helpers.get_cartography_schema("db-tenant-test", "provider-123")
mock_logger.error.assert_called_once()
@@ -550,6 +550,36 @@ class TestRlsTransaction:
mock_sleep.assert_any_call(1.0)
assert mock_logger.info.call_count == 2
def test_rls_transaction_operational_error_inside_context_no_retry(
self, tenants_fixture, enable_read_replica
):
"""Test OperationalError raised inside context does not retry."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.return_value.__enter__.return_value = None
mock_atomic.return_value.__exit__.return_value = False
with patch("api.db_utils.time.sleep") as mock_sleep:
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(OperationalError):
with rls_transaction(tenant_id):
raise OperationalError("Conflict with recovery")
mock_sleep.assert_not_called()
def test_rls_transaction_max_three_attempts_for_replica(
self, tenants_fixture, enable_read_replica
):
@@ -579,6 +609,38 @@ class TestRlsTransaction:
assert mock_atomic.call_count == 3
def test_rls_transaction_replica_no_retry_when_disabled(
self, tenants_fixture, enable_read_replica
):
"""Test replica retry is disabled when retry_on_replica=False."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = OperationalError("Replica error")
with patch("api.db_utils.time.sleep") as mock_sleep:
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(OperationalError):
with rls_transaction(
tenant_id, retry_on_replica=False
):
pass
assert mock_atomic.call_count == 1
mock_sleep.assert_not_called()
def test_rls_transaction_only_one_attempt_for_primary(self, tenants_fixture):
"""Test only 1 attempt for primary database."""
tenant = tenants_fixture[0]
@@ -2,6 +2,11 @@ import pytest
from rest_framework.exceptions import ValidationError
from api.v1.serializer_utils.integrations import S3ConfigSerializer
from api.v1.serializers import (
CloudflareApiKeyProviderSecret,
CloudflareTokenProviderSecret,
ImageProviderSecret,
)
class TestS3ConfigSerializer:
@@ -98,3 +103,73 @@ class TestS3ConfigSerializer:
serializer = S3ConfigSerializer(data=data)
assert not serializer.is_valid()
assert "output_directory" in serializer.errors
class TestImageProviderSecret:
"""Test cases for ImageProviderSecret validation."""
def test_valid_no_credentials(self):
serializer = ImageProviderSecret(data={})
assert serializer.is_valid()
def test_valid_token_only(self):
serializer = ImageProviderSecret(data={"registry_token": "tok"})
assert serializer.is_valid()
def test_valid_username_and_password(self):
serializer = ImageProviderSecret(
data={"registry_username": "user", "registry_password": "pass"}
)
assert serializer.is_valid()
def test_valid_token_with_username_only(self):
serializer = ImageProviderSecret(
data={"registry_token": "tok", "registry_username": "user"}
)
assert serializer.is_valid()
def test_invalid_username_without_password(self):
serializer = ImageProviderSecret(data={"registry_username": "user"})
assert not serializer.is_valid()
assert "non_field_errors" in serializer.errors
def test_invalid_password_without_username(self):
serializer = ImageProviderSecret(data={"registry_password": "pass"})
assert not serializer.is_valid()
assert "non_field_errors" in serializer.errors
class TestCloudflareProviderSecret:
"""Test cases for Cloudflare provider credential formats."""
def test_valid_api_token(self):
serializer = CloudflareTokenProviderSecret(
data={"api_token": "Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY"}
)
assert serializer.is_valid(), serializer.errors
def test_invalid_api_token_with_api_key_format(self):
serializer = CloudflareTokenProviderSecret(
data={"api_token": "144c9defac04969c7bfad8efaa8ea194"}
)
assert not serializer.is_valid()
assert "api_token" in serializer.errors
def test_valid_api_key_and_email(self):
serializer = CloudflareApiKeyProviderSecret(
data={
"api_key": "144c9defac04969c7bfad8efaa8ea194",
"api_email": "user@example.com",
}
)
assert serializer.is_valid(), serializer.errors
def test_invalid_api_key_with_token_format(self):
serializer = CloudflareApiKeyProviderSecret(
data={
"api_key": "Sn3lZJTBX6kkg7OdcBUAxOO963GEIyGQqnFTOFYY",
"api_email": "user@example.com",
}
)
assert not serializer.is_valid()
assert "api_key" in serializer.errors
+160
View File
@@ -24,6 +24,7 @@ from prowler.providers.cloudflare.cloudflare_provider import CloudflareProvider
from prowler.providers.gcp.gcp_provider import GcpProvider
from prowler.providers.github.github_provider import GithubProvider
from prowler.providers.iac.iac_provider import IacProvider
from prowler.providers.image.image_provider import ImageProvider
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
from prowler.providers.m365.m365_provider import M365Provider
from prowler.providers.mongodbatlas.mongodbatlas_provider import MongodbatlasProvider
@@ -122,6 +123,7 @@ class TestReturnProwlerProvider:
(Provider.ProviderChoices.ALIBABACLOUD.value, AlibabacloudProvider),
(Provider.ProviderChoices.CLOUDFLARE.value, CloudflareProvider),
(Provider.ProviderChoices.OPENSTACK.value, OpenstackProvider),
(Provider.ProviderChoices.IMAGE.value, ImageProvider),
],
)
def test_return_prowler_provider(self, provider_type, expected_provider):
@@ -188,6 +190,47 @@ class TestProwlerProviderConnectionTest:
assert isinstance(connection.error, Provider.secret.RelatedObjectDoesNotExist)
assert str(connection.error) == "Provider has no secret."
@patch("api.utils.return_prowler_provider")
def test_prowler_provider_connection_test_image_provider(
self, mock_return_prowler_provider
):
"""Test connection test for Image provider with credentials."""
provider = MagicMock()
provider.uid = "docker.io/myns/myimage:latest"
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret.secret = {
"registry_username": "user",
"registry_password": "pass",
"registry_token": "tok123",
}
mock_return_prowler_provider.return_value = MagicMock()
prowler_provider_connection_test(provider)
mock_return_prowler_provider.return_value.test_connection.assert_called_once_with(
image="docker.io/myns/myimage:latest",
raise_on_exception=False,
registry_username="user",
registry_password="pass",
registry_token="tok123",
)
@patch("api.utils.return_prowler_provider")
def test_prowler_provider_connection_test_image_provider_no_creds(
self, mock_return_prowler_provider
):
"""Test connection test for Image provider without credentials."""
provider = MagicMock()
provider.uid = "alpine:3.18"
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret.secret = {}
mock_return_prowler_provider.return_value = MagicMock()
prowler_provider_connection_test(provider)
mock_return_prowler_provider.return_value.test_connection.assert_called_once_with(
image="alpine:3.18",
raise_on_exception=False,
)
class TestGetProwlerProviderKwargs:
@pytest.mark.parametrize(
@@ -336,6 +379,123 @@ class TestGetProwlerProviderKwargs:
}
assert result == expected_result
def test_get_prowler_provider_kwargs_image_provider_registry_url(self):
"""Test that Image provider with a registry URL gets 'registry' kwarg."""
provider_uid = "docker.io/myns"
secret_dict = {
"registry_username": "user",
"registry_password": "pass",
}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {
"registry": provider_uid,
"registry_username": "user",
"registry_password": "pass",
}
assert result == expected_result
def test_get_prowler_provider_kwargs_image_provider_image_ref(self):
"""Test that Image provider with a full image reference gets 'images' kwarg."""
provider_uid = "docker.io/myns/myimage:latest"
secret_dict = {
"registry_username": "user",
"registry_password": "pass",
}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {
"images": [provider_uid],
"registry_username": "user",
"registry_password": "pass",
}
assert result == expected_result
def test_get_prowler_provider_kwargs_image_provider_dockerhub_image(self):
"""Test that Image provider with a short DockerHub image gets 'images' kwarg."""
provider_uid = "alpine:3.18"
secret_dict = {}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {"images": [provider_uid]}
assert result == expected_result
def test_get_prowler_provider_kwargs_image_provider_filters_falsy_secrets(self):
"""Test that falsy secret values are filtered out for Image provider."""
provider_uid = "docker.io/myns/myimage:latest"
secret_dict = {
"registry_username": "",
"registry_password": "",
}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {"images": [provider_uid]}
assert result == expected_result
def test_get_prowler_provider_kwargs_image_provider_ignores_mutelist(self):
"""Test that Image provider does NOT receive mutelist_content.
Image provider uses Trivy's built-in mutelist logic, so it should not
receive mutelist_content even when a mutelist processor is configured.
"""
provider_uid = "docker.io/myns/myimage:latest"
secret_dict = {
"registry_username": "user",
"registry_password": "pass",
}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
mutelist_processor = MagicMock()
mutelist_processor.configuration = {"Mutelist": {"key": "value"}}
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IMAGE.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider, mutelist_processor)
assert "mutelist_content" not in result
expected_result = {
"images": [provider_uid],
"registry_username": "user",
"registry_password": "pass",
}
assert result == expected_result
def test_get_prowler_provider_kwargs_unsupported_provider(self):
# Setup
provider_uid = "provider_uid"
+432 -9
View File
@@ -30,6 +30,7 @@ from django.test import RequestFactory
from django.urls import reverse
from django_celery_results.models import TaskResult
from rest_framework import status
from rest_framework.exceptions import PermissionDenied
from rest_framework.response import Response
from api.attack_paths import (
@@ -3993,6 +3994,8 @@ class TestAttackPathsScanViewSet:
"properties": {},
}
],
"total_nodes": 1,
"truncated": False,
}
expected_db_name = f"db-tenant-{attack_paths_scan.provider.tenant_id}"
@@ -4006,11 +4009,11 @@ class TestAttackPathsScanViewSet:
return_value=expected_db_name,
) as mock_get_db_name,
patch(
"api.v1.views.attack_paths_views_helpers.prepare_query_parameters",
"api.v1.views.attack_paths_views_helpers.prepare_parameters",
return_value=prepared_parameters,
) as mock_prepare,
patch(
"api.v1.views.attack_paths_views_helpers.execute_attack_paths_query",
"api.v1.views.attack_paths_views_helpers.execute_query",
return_value=graph_payload,
) as mock_execute,
patch("api.v1.views.graph_database.clear_cache") as mock_clear_cache,
@@ -4046,6 +4049,74 @@ class TestAttackPathsScanViewSet:
assert attributes["nodes"] == graph_payload["nodes"]
assert attributes["relationships"] == graph_payload["relationships"]
def test_run_attack_paths_query_returns_text_when_accept_text_plain(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
query_definition = AttackPathsQueryDefinition(
id="aws-rds",
name="RDS inventory",
short_description="List account RDS assets.",
description="List account RDS assets",
provider=provider.provider,
cypher="MATCH (n) RETURN n",
parameters=[],
)
graph_payload = {
"nodes": [
{
"id": "node-1",
"labels": ["AWSAccount"],
"properties": {"name": "root"},
}
],
"relationships": [],
"total_nodes": 1,
"truncated": False,
}
with (
patch("api.v1.views.get_query_by_id", return_value=query_definition),
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
patch(
"api.v1.views.attack_paths_views_helpers.prepare_parameters",
return_value={"provider_uid": provider.uid},
),
patch(
"api.v1.views.attack_paths_views_helpers.execute_query",
return_value=graph_payload,
),
patch("api.v1.views.graph_database.clear_cache"),
):
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-run",
kwargs={"pk": attack_paths_scan.id},
),
data=self._run_payload("aws-rds"),
content_type=API_JSON_CONTENT_TYPE,
HTTP_ACCEPT="text/plain",
)
assert response.status_code == status.HTTP_200_OK
assert response["Content-Type"] == "text/plain"
body = response.content.decode()
assert "## Nodes (1)" in body
assert "## Relationships (0)" in body
assert "## Summary" in body
def test_run_attack_paths_query_blocks_when_graph_data_not_ready(
self,
authenticated_client,
@@ -4099,14 +4170,16 @@ class TestAttackPathsScanViewSet:
with (
patch("api.v1.views.get_query_by_id", return_value=query_definition),
patch(
"api.v1.views.attack_paths_views_helpers.prepare_query_parameters",
"api.v1.views.attack_paths_views_helpers.prepare_parameters",
return_value={"provider_uid": provider.uid},
),
patch(
"api.v1.views.attack_paths_views_helpers.execute_attack_paths_query",
"api.v1.views.attack_paths_views_helpers.execute_query",
return_value={
"nodes": [{"id": "n1", "labels": ["AWSAccount"], "properties": {}}],
"relationships": [],
"total_nodes": 1,
"truncated": False,
},
),
patch("api.v1.views.graph_database.clear_cache"),
@@ -4152,14 +4225,16 @@ class TestAttackPathsScanViewSet:
with (
patch("api.v1.views.get_query_by_id", return_value=query_definition),
patch(
"api.v1.views.attack_paths_views_helpers.prepare_query_parameters",
"api.v1.views.attack_paths_views_helpers.prepare_parameters",
return_value={"provider_uid": provider.uid},
),
patch(
"api.v1.views.attack_paths_views_helpers.execute_attack_paths_query",
"api.v1.views.attack_paths_views_helpers.execute_query",
return_value={
"nodes": [{"id": "n1", "labels": ["AWSAccount"], "properties": {}}],
"relationships": [],
"total_nodes": 1,
"truncated": False,
},
),
patch("api.v1.views.graph_database.clear_cache"),
@@ -4230,12 +4305,17 @@ class TestAttackPathsScanViewSet:
with (
patch("api.v1.views.get_query_by_id", return_value=query_definition),
patch(
"api.v1.views.attack_paths_views_helpers.prepare_query_parameters",
"api.v1.views.attack_paths_views_helpers.prepare_parameters",
return_value={"provider_uid": provider.uid},
),
patch(
"api.v1.views.attack_paths_views_helpers.execute_attack_paths_query",
return_value={"nodes": [], "relationships": []},
"api.v1.views.attack_paths_views_helpers.execute_query",
return_value={
"nodes": [],
"relationships": [],
"total_nodes": 0,
"truncated": False,
},
),
patch("api.v1.views.graph_database.clear_cache"),
):
@@ -4257,6 +4337,349 @@ class TestAttackPathsScanViewSet:
else:
assert "errors" in payload
# -- run_custom_attack_paths_query action ------------------------------------
@staticmethod
def _custom_query_payload(query="MATCH (n) RETURN n"):
return {
"data": {
"type": "attack-paths-custom-query-run-requests",
"attributes": {"query": query},
}
}
# TODO: Remove skip once queries/custom and schema endpoints are unblocked
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_run_custom_query_returns_graph(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
graph_payload = {
"nodes": [
{
"id": "node-1",
"labels": ["AWSAccount"],
"properties": {"name": "root"},
}
],
"relationships": [],
"total_nodes": 1,
"truncated": False,
}
with (
patch(
"api.v1.views.attack_paths_views_helpers.execute_custom_query",
return_value=graph_payload,
) as mock_execute,
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
patch("api.v1.views.graph_database.clear_cache"),
):
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-custom",
kwargs={"pk": attack_paths_scan.id},
),
data=self._custom_query_payload(),
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_200_OK
mock_execute.assert_called_once_with(
"db-test",
"MATCH (n) RETURN n",
str(attack_paths_scan.provider_id),
)
attributes = response.json()["data"]["attributes"]
assert len(attributes["nodes"]) == 1
assert attributes["total_nodes"] == 1
assert attributes["truncated"] is False
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_run_custom_query_returns_text_when_accept_text_plain(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
graph_payload = {
"nodes": [
{
"id": "node-1",
"labels": ["AWSAccount"],
"properties": {"name": "root"},
}
],
"relationships": [],
"total_nodes": 1,
"truncated": False,
}
with (
patch(
"api.v1.views.attack_paths_views_helpers.execute_custom_query",
return_value=graph_payload,
),
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
patch("api.v1.views.graph_database.clear_cache"),
):
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-custom",
kwargs={"pk": attack_paths_scan.id},
),
data=self._custom_query_payload(),
content_type=API_JSON_CONTENT_TYPE,
HTTP_ACCEPT="text/plain",
)
assert response.status_code == status.HTTP_200_OK
assert response["Content-Type"] == "text/plain"
body = response.content.decode()
assert "## Nodes (1)" in body
assert "## Relationships (0)" in body
assert "## Summary" in body
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_run_custom_query_returns_404_when_no_nodes(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
with (
patch(
"api.v1.views.attack_paths_views_helpers.execute_custom_query",
return_value={
"nodes": [],
"relationships": [],
"total_nodes": 0,
"truncated": False,
},
),
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
patch("api.v1.views.graph_database.clear_cache"),
):
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-custom",
kwargs={"pk": attack_paths_scan.id},
),
data=self._custom_query_payload(),
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_404_NOT_FOUND
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_run_custom_query_returns_400_when_graph_not_ready(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=False,
)
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-custom",
kwargs={"pk": attack_paths_scan.id},
),
data=self._custom_query_payload(),
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert "not available" in response.json()["errors"][0]["detail"]
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_run_custom_query_returns_403_for_write_query(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
with (
patch(
"api.v1.views.attack_paths_views_helpers.execute_custom_query",
side_effect=PermissionDenied(
"Attack Paths query execution failed: read-only queries are enforced"
),
),
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
):
response = authenticated_client.post(
reverse(
"attack-paths-scans-queries-custom",
kwargs={"pk": attack_paths_scan.id},
),
data=self._custom_query_payload("CREATE (n) RETURN n"),
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_403_FORBIDDEN
# -- cartography_schema action ------------------------------------------------
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_cartography_schema_returns_urls(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
schema_data = {
"id": "aws-0.129.0",
"provider": "aws",
"cartography_version": "0.129.0",
"schema_url": "https://github.com/cartography-cncf/cartography/blob/0.129.0/docs/root/modules/aws/schema.md",
"raw_schema_url": "https://raw.githubusercontent.com/cartography-cncf/cartography/refs/tags/0.129.0/docs/root/modules/aws/schema.md",
}
with (
patch(
"api.v1.views.attack_paths_views_helpers.get_cartography_schema",
return_value=schema_data,
) as mock_get_schema,
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
):
response = authenticated_client.get(
reverse(
"attack-paths-scans-schema",
kwargs={"pk": attack_paths_scan.id},
)
)
assert response.status_code == status.HTTP_200_OK
mock_get_schema.assert_called_once_with(
"db-test", str(attack_paths_scan.provider_id)
)
attributes = response.json()["data"]["attributes"]
assert attributes["provider"] == "aws"
assert attributes["cartography_version"] == "0.129.0"
assert "schema.md" in attributes["schema_url"]
assert "raw.githubusercontent.com" in attributes["raw_schema_url"]
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_cartography_schema_returns_404_when_no_metadata(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=True,
)
with (
patch(
"api.v1.views.attack_paths_views_helpers.get_cartography_schema",
return_value=None,
),
patch(
"api.v1.views.graph_database.get_database_name",
return_value="db-test",
),
):
response = authenticated_client.get(
reverse(
"attack-paths-scans-schema",
kwargs={"pk": attack_paths_scan.id},
)
)
assert response.status_code == status.HTTP_404_NOT_FOUND
assert "No cartography schema metadata" in str(response.json())
@pytest.mark.skip(reason="Endpoint temporarily blocked")
def test_cartography_schema_returns_400_when_graph_not_ready(
self,
authenticated_client,
providers_fixture,
scans_fixture,
create_attack_paths_scan,
):
provider = providers_fixture[0]
attack_paths_scan = create_attack_paths_scan(
provider,
scan=scans_fixture[0],
graph_data_ready=False,
)
response = authenticated_client.get(
reverse(
"attack-paths-scans-schema",
kwargs={"pk": attack_paths_scan.id},
)
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
@pytest.mark.django_db
class TestResourceViewSet:
+48 -8
View File
@@ -28,6 +28,7 @@ if TYPE_CHECKING:
from prowler.providers.gcp.gcp_provider import GcpProvider
from prowler.providers.github.github_provider import GithubProvider
from prowler.providers.iac.iac_provider import IacProvider
from prowler.providers.image.image_provider import ImageProvider
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
from prowler.providers.m365.m365_provider import M365Provider
from prowler.providers.mongodbatlas.mongodbatlas_provider import (
@@ -83,6 +84,7 @@ def return_prowler_provider(
| GcpProvider
| GithubProvider
| IacProvider
| ImageProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
@@ -95,7 +97,7 @@ def return_prowler_provider(
provider (Provider): The provider object containing the provider type and associated secrets.
Returns:
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | IacProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: The corresponding provider class.
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | IacProvider | ImageProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: The corresponding provider class.
Raises:
ValueError: If the provider type specified in `provider.provider` is not supported.
@@ -159,6 +161,10 @@ def return_prowler_provider(
from prowler.providers.openstack.openstack_provider import OpenstackProvider
prowler_provider = OpenstackProvider
case Provider.ProviderChoices.IMAGE.value:
from prowler.providers.image.image_provider import ImageProvider
prowler_provider = ImageProvider
case _:
raise ValueError(f"Provider type {provider.provider} not supported")
return prowler_provider
@@ -216,16 +222,32 @@ def get_prowler_provider_kwargs(
"filter_accounts": [provider.uid],
}
elif provider.provider == Provider.ProviderChoices.OPENSTACK.value:
# No extra kwargs needed: clouds_yaml_content and clouds_yaml_cloud from the
# secret are sufficient. Validating project_id (provider.uid) against the
# clouds.yaml is not feasible because not all auth methods include it and the
# Keystone API is unavailable on public clouds.
# clouds_yaml_content, clouds_yaml_cloud and provider_id are validated
# in the provider itself, so it's not needed here.
pass
elif provider.provider == Provider.ProviderChoices.IMAGE.value:
# Detect whether uid is a registry URL (e.g. "docker.io/andoniaf") or
# a concrete image reference (e.g. "docker.io/andoniaf/myimage:latest").
from prowler.providers.image.image_provider import ImageProvider
if ImageProvider._is_registry_url(provider.uid):
prowler_provider_kwargs = {
"registry": provider.uid,
**{k: v for k, v in prowler_provider_kwargs.items() if v},
}
else:
prowler_provider_kwargs = {
"images": [provider.uid],
**{k: v for k, v in prowler_provider_kwargs.items() if v},
}
if mutelist_processor:
mutelist_content = mutelist_processor.configuration.get("Mutelist", {})
# IaC provider doesn't support mutelist (uses Trivy's built-in logic)
if mutelist_content and provider.provider != Provider.ProviderChoices.IAC.value:
# IaC and Image providers don't support mutelist (both use Trivy's built-in logic)
if mutelist_content and provider.provider not in (
Provider.ProviderChoices.IAC.value,
Provider.ProviderChoices.IMAGE.value,
):
prowler_provider_kwargs["mutelist_content"] = mutelist_content
return prowler_provider_kwargs
@@ -242,6 +264,7 @@ def initialize_prowler_provider(
| GcpProvider
| GithubProvider
| IacProvider
| ImageProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
@@ -255,7 +278,7 @@ def initialize_prowler_provider(
mutelist_processor (Processor): The mutelist processor object containing the mutelist configuration.
Returns:
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | IacProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: An instance of the corresponding provider class
AlibabacloudProvider | AwsProvider | AzureProvider | CloudflareProvider | GcpProvider | GithubProvider | IacProvider | ImageProvider | KubernetesProvider | M365Provider | MongodbatlasProvider | OpenstackProvider | OraclecloudProvider: An instance of the corresponding provider class
initialized with the provider's secrets.
"""
prowler_provider = return_prowler_provider(provider)
@@ -294,9 +317,26 @@ def prowler_provider_connection_test(provider: Provider) -> Connection:
openstack_kwargs = {
"clouds_yaml_content": prowler_provider_kwargs["clouds_yaml_content"],
"clouds_yaml_cloud": prowler_provider_kwargs["clouds_yaml_cloud"],
"provider_id": provider.uid,
"raise_on_exception": False,
}
return prowler_provider.test_connection(**openstack_kwargs)
elif provider.provider == Provider.ProviderChoices.IMAGE.value:
image_kwargs = {
"image": provider.uid,
"raise_on_exception": False,
}
if prowler_provider_kwargs.get("registry_username"):
image_kwargs["registry_username"] = prowler_provider_kwargs[
"registry_username"
]
if prowler_provider_kwargs.get("registry_password"):
image_kwargs["registry_password"] = prowler_provider_kwargs[
"registry_password"
]
if prowler_provider_kwargs.get("registry_token"):
image_kwargs["registry_token"] = prowler_provider_kwargs["registry_token"]
return prowler_provider.test_connection(**image_kwargs)
else:
return prowler_provider.test_connection(
**prowler_provider_kwargs,
+64
View File
@@ -1,5 +1,6 @@
import base64
import json
import re
from datetime import datetime, timedelta, timezone
from django.conf import settings
@@ -1219,6 +1220,13 @@ class AttackPathsQueryRunRequestSerializer(BaseSerializerV1):
resource_name = "attack-paths-query-run-requests"
class AttackPathsCustomQueryRunRequestSerializer(BaseSerializerV1):
query = serializers.CharField()
class JSONAPIMeta:
resource_name = "attack-paths-custom-query-run-requests"
class AttackPathsNodeSerializer(BaseSerializerV1):
id = serializers.CharField()
labels = serializers.ListField(child=serializers.CharField())
@@ -1242,11 +1250,24 @@ class AttackPathsRelationshipSerializer(BaseSerializerV1):
class AttackPathsQueryResultSerializer(BaseSerializerV1):
nodes = AttackPathsNodeSerializer(many=True)
relationships = AttackPathsRelationshipSerializer(many=True)
total_nodes = serializers.IntegerField()
truncated = serializers.BooleanField()
class JSONAPIMeta:
resource_name = "attack-paths-query-results"
class AttackPathsCartographySchemaSerializer(BaseSerializerV1):
id = serializers.CharField()
provider = serializers.CharField()
cartography_version = serializers.CharField()
schema_url = serializers.URLField()
raw_schema_url = serializers.URLField()
class JSONAPIMeta:
resource_name = "attack-paths-cartography-schemas"
class ResourceTagSerializer(RLSSerializer):
"""
Serializer for the ResourceTag model
@@ -1528,6 +1549,8 @@ class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
)
elif provider_type == Provider.ProviderChoices.OPENSTACK.value:
serializer = OpenStackCloudsYamlProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.IMAGE.value:
serializer = ImageProviderSecret(data=secret)
else:
raise serializers.ValidationError(
{"provider": f"Provider type not supported {provider_type}"}
@@ -1682,6 +1705,15 @@ class OracleCloudProviderSecret(serializers.Serializer):
class CloudflareTokenProviderSecret(serializers.Serializer):
api_token = serializers.CharField()
def validate_api_token(self, value: str) -> str:
# Cloudflare Global API Key is 32 hex chars; reject it in token field.
if re.fullmatch(r"[a-fA-F0-9]{32}", (value or "").strip()):
raise serializers.ValidationError(
"This value matches Cloudflare API Key format. "
"Use 'api_key' and 'api_email' instead."
)
return value
class Meta:
resource_name = "provider-secrets"
@@ -1690,6 +1722,14 @@ class CloudflareApiKeyProviderSecret(serializers.Serializer):
api_key = serializers.CharField()
api_email = serializers.EmailField()
def validate_api_key(self, value: str) -> str:
if not re.fullmatch(r"[a-fA-F0-9]{32}", (value or "").strip()):
raise serializers.ValidationError(
"Invalid Cloudflare API Key format. "
"Use a 32-character hexadecimal Global API Key."
)
return value
class Meta:
resource_name = "provider-secrets"
@@ -1702,6 +1742,30 @@ class OpenStackCloudsYamlProviderSecret(serializers.Serializer):
resource_name = "provider-secrets"
class ImageProviderSecret(serializers.Serializer):
registry_username = serializers.CharField(required=False)
registry_password = serializers.CharField(required=False)
registry_token = serializers.CharField(required=False)
class Meta:
resource_name = "provider-secrets"
def validate(self, attrs):
token = attrs.get("registry_token")
username = attrs.get("registry_username")
password = attrs.get("registry_password")
if not token:
if username and not password:
raise serializers.ValidationError(
"registry_password is required when registry_username is provided."
)
if password and not username:
raise serializers.ValidationError(
"registry_username is required when registry_password is provided."
)
return attrs
class AlibabaCloudProviderSecret(serializers.Serializer):
access_key_id = serializers.CharField()
access_key_secret = serializers.CharField()
+23
View File
@@ -1,5 +1,7 @@
from allauth.socialaccount.providers.saml.views import ACSView, MetadataView, SLSView
from django.http import JsonResponse
from django.urls import include, path
from django.views.decorators.csrf import csrf_exempt
from drf_spectacular.views import SpectacularRedocView
from rest_framework_nested import routers
@@ -48,6 +50,16 @@ from api.v1.views import (
UserViewSet,
)
@csrf_exempt
def _blocked_endpoint(request, *args, **kwargs):
return JsonResponse(
{"errors": [{"detail": "This endpoint is not available."}]},
status=405,
content_type="application/vnd.api+json",
)
router = routers.DefaultRouter(trailing_slash=False)
router.register(r"users", UserViewSet, basename="user")
@@ -197,6 +209,17 @@ urlpatterns = [
path("tokens/saml", SAMLTokenValidateView.as_view(), name="token-saml"),
path("tokens/google", GoogleSocialLoginView.as_view(), name="token-google"),
path("tokens/github", GithubSocialLoginView.as_view(), name="token-github"),
# TODO: Remove these blocked endpoints once they are properly tested
path(
"attack-paths-scans/<uuid:pk>/queries/custom",
_blocked_endpoint,
name="attack-paths-scans-queries-custom-blocked",
),
path(
"attack-paths-scans/<uuid:pk>/schema",
_blocked_endpoint,
name="attack-paths-scans-schema-blocked",
),
path("", include(router.urls)),
path("", include(tenants_router.urls)),
path("", include(users_router.urls)),
+134 -3
View File
@@ -3,6 +3,7 @@ import glob
import json
import logging
import os
from collections import defaultdict
from copy import deepcopy
from datetime import datetime, timedelta, timezone
@@ -10,6 +11,7 @@ from decimal import ROUND_HALF_UP, Decimal, InvalidOperation
from urllib.parse import urljoin
import sentry_sdk
from allauth.socialaccount.models import SocialAccount, SocialApp
from allauth.socialaccount.providers.github.views import GitHubOAuth2Adapter
from allauth.socialaccount.providers.google.views import GoogleOAuth2Adapter
@@ -97,6 +99,7 @@ from api.attack_paths import database as graph_database
from api.attack_paths import get_queries_for_provider, get_query_by_id
from api.attack_paths import views_helpers as attack_paths_views_helpers
from api.base_views import BaseRLSViewSet, BaseTenantViewset, BaseUserViewset
from api.renderers import APIJSONRenderer, PlainTextRenderer
from api.compliance import (
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
get_compliance_frameworks,
@@ -205,6 +208,8 @@ from api.utils import (
from api.uuid_utils import datetime_to_uuid7, uuid7_start
from api.v1.mixins import DisablePaginationMixin, PaginateByPkMixin, TaskManagementMixin
from api.v1.serializers import (
AttackPathsCartographySchemaSerializer,
AttackPathsCustomQueryRunRequestSerializer,
AttackPathsQueryResultSerializer,
AttackPathsQueryRunRequestSerializer,
AttackPathsQuerySerializer,
@@ -2398,6 +2403,40 @@ class TaskViewSet(BaseRLSViewSet):
),
},
),
run_custom_attack_paths_query=extend_schema(
tags=["Attack Paths"],
summary="Execute a custom openCypher query",
description="Execute a raw openCypher query against the Attack Paths graph. "
"Results are filtered to the scan's provider and truncated to a maximum node count.",
request=AttackPathsCustomQueryRunRequestSerializer,
responses={
200: OpenApiResponse(AttackPathsQueryResultSerializer),
403: OpenApiResponse(description="Read-only queries are enforced"),
404: OpenApiResponse(description="No results found for the given query"),
500: OpenApiResponse(
description="Query execution failed due to a database error"
),
},
),
cartography_schema=extend_schema(
tags=["Attack Paths"],
summary="Retrieve cartography schema metadata",
description="Return the cartography provider, version, and links to the schema documentation "
"for the cloud provider associated with this Attack Paths scan.",
request=None,
responses={
200: OpenApiResponse(AttackPathsCartographySchemaSerializer),
400: OpenApiResponse(
description="Attack Paths data is not yet available (graph_data_ready is false)"
),
404: OpenApiResponse(
description="No cartography schema metadata found for this provider"
),
500: OpenApiResponse(
description="Unable to retrieve cartography schema due to a database error"
),
},
),
)
class AttackPathsScanViewSet(BaseRLSViewSet):
queryset = AttackPathsScan.objects.all()
@@ -2423,6 +2462,12 @@ class AttackPathsScanViewSet(BaseRLSViewSet):
if self.action == "run_attack_paths_query":
return AttackPathsQueryRunRequestSerializer
if self.action == "run_custom_attack_paths_query":
return AttackPathsCustomQueryRunRequestSerializer
if self.action == "cartography_schema":
return AttackPathsCartographySchemaSerializer
return super().get_serializer_class()
def get_queryset(self):
@@ -2483,11 +2528,13 @@ class AttackPathsScanViewSet(BaseRLSViewSet):
serializer = AttackPathsQuerySerializer(queries, many=True)
return Response(serializer.data, status=status.HTTP_200_OK)
@extend_schema(parameters=[OpenApiParameter("format", exclude=True)])
@action(
detail=True,
methods=["post"],
url_path="queries/run",
url_name="queries-run",
renderer_classes=[APIJSONRenderer, PlainTextRenderer],
)
def run_attack_paths_query(self, request, pk=None):
attack_paths_scan = self.get_object()
@@ -2499,7 +2546,7 @@ class AttackPathsScanViewSet(BaseRLSViewSet):
}
)
payload = attack_paths_views_helpers.normalize_run_payload(request.data)
payload = attack_paths_views_helpers.normalize_query_payload(request.data)
serializer = AttackPathsQueryRunRequestSerializer(data=payload)
serializer.is_valid(raise_exception=True)
@@ -2516,14 +2563,14 @@ class AttackPathsScanViewSet(BaseRLSViewSet):
attack_paths_scan.provider.tenant_id
)
provider_id = str(attack_paths_scan.provider_id)
parameters = attack_paths_views_helpers.prepare_query_parameters(
parameters = attack_paths_views_helpers.prepare_parameters(
query_definition,
serializer.validated_data.get("parameters", {}),
attack_paths_scan.provider.uid,
provider_id,
)
graph = attack_paths_views_helpers.execute_attack_paths_query(
graph = attack_paths_views_helpers.execute_query(
database_name,
query_definition,
parameters,
@@ -2535,9 +2582,93 @@ class AttackPathsScanViewSet(BaseRLSViewSet):
if not graph.get("nodes"):
status_code = status.HTTP_404_NOT_FOUND
if isinstance(request.accepted_renderer, PlainTextRenderer):
text = attack_paths_views_helpers.serialize_graph_as_text(graph)
return Response(text, status=status_code, content_type="text/plain")
response_serializer = AttackPathsQueryResultSerializer(graph)
return Response(response_serializer.data, status=status_code)
@extend_schema(parameters=[OpenApiParameter("format", exclude=True)])
@action(
detail=True,
methods=["post"],
url_path="queries/custom",
url_name="queries-custom",
renderer_classes=[APIJSONRenderer, PlainTextRenderer],
)
def run_custom_attack_paths_query(self, request, pk=None):
attack_paths_scan = self.get_object()
if not attack_paths_scan.graph_data_ready:
raise ValidationError(
{
"detail": "Attack Paths data is not available for querying - a scan must complete at least once before queries can be run"
}
)
payload = attack_paths_views_helpers.normalize_custom_query_payload(
request.data
)
serializer = AttackPathsCustomQueryRunRequestSerializer(data=payload)
serializer.is_valid(raise_exception=True)
database_name = graph_database.get_database_name(
attack_paths_scan.provider.tenant_id
)
provider_id = str(attack_paths_scan.provider_id)
graph = attack_paths_views_helpers.execute_custom_query(
database_name,
serializer.validated_data["query"],
provider_id,
)
graph_database.clear_cache(database_name)
status_code = status.HTTP_200_OK
if not graph.get("nodes"):
status_code = status.HTTP_404_NOT_FOUND
if isinstance(request.accepted_renderer, PlainTextRenderer):
text = attack_paths_views_helpers.serialize_graph_as_text(graph)
return Response(text, status=status_code, content_type="text/plain")
response_serializer = AttackPathsQueryResultSerializer(graph)
return Response(response_serializer.data, status=status_code)
@action(
detail=True,
methods=["get"],
url_path="schema",
url_name="schema",
)
def cartography_schema(self, request, pk=None):
attack_paths_scan = self.get_object()
if not attack_paths_scan.graph_data_ready:
raise ValidationError(
{
"detail": "Attack Paths data is not available for querying - a scan must complete at least once before the schema can be retrieved"
}
)
database_name = graph_database.get_database_name(
attack_paths_scan.provider.tenant_id
)
provider_id = str(attack_paths_scan.provider_id)
schema = attack_paths_views_helpers.get_cartography_schema(
database_name, provider_id
)
if not schema:
return Response(
{"detail": "No cartography schema metadata found for this provider"},
status=status.HTTP_404_NOT_FOUND,
)
serializer = AttackPathsCartographySchemaSerializer(schema)
return Response(serializer.data, status=status.HTTP_200_OK)
@extend_schema_view(
list=extend_schema(
@@ -62,6 +62,27 @@ INTERNAL_LABELS: list[str] = [
*[config.deprecated_resource_label for config in PROVIDER_CONFIGS.values()],
]
# Provider isolation properties
PROVIDER_ISOLATION_PROPERTIES: list[str] = [
"_provider_id",
"_provider_element_id",
"provider_id",
"provider_element_id",
]
# Cartography bookkeeping metadata
CARTOGRAPHY_METADATA_PROPERTIES: list[str] = [
"lastupdated",
"firstseen",
"_module_name",
"_module_version",
]
INTERNAL_PROPERTIES: list[str] = [
*PROVIDER_ISOLATION_PROPERTIES,
*CARTOGRAPHY_METADATA_PROPERTIES,
]
# Provider Config Accessors
# -------------------------
@@ -14,6 +14,7 @@ from api.attack_paths import database as graph_database
from tasks.jobs.attack_paths.config import (
BATCH_SIZE,
DEPRECATED_PROVIDER_RESOURCE_LABEL,
PROVIDER_ISOLATION_PROPERTIES,
PROVIDER_RESOURCE_LABEL,
)
from tasks.jobs.attack_paths.indexes import IndexType, create_indexes
@@ -199,11 +200,6 @@ def sync_relationships(
def _strip_internal_properties(props: dict[str, Any]) -> None:
"""Remove internal properties that shouldn't be copied during sync."""
for key in [
"_provider_element_id",
"_provider_id",
"provider_element_id", # Deprecated
"provider_id", # Deprecated
]:
"""Remove provider isolation properties before the += spread in sync templates."""
for key in PROVIDER_ISOLATION_PROPERTIES:
props.pop(key, None)
+42
View File
@@ -1,5 +1,9 @@
from celery.utils.log import get_task_logger
from django.db import DatabaseError
from tasks.jobs.queries import (
COMPLIANCE_DELETE_EMPTY_TENANT_SUMMARY_SQL,
COMPLIANCE_UPSERT_TENANT_SUMMARY_SQL,
)
from api.attack_paths import database as graph_database
from api.db_router import MainRouter
@@ -8,6 +12,7 @@ from api.models import (
AttackPathsScan,
Finding,
Provider,
ProviderComplianceScore,
Resource,
Scan,
ScanSummary,
@@ -17,6 +22,28 @@ from api.models import (
logger = get_task_logger(__name__)
def _recalculate_tenant_compliance_summary(tenant_id: str, compliance_ids: list[str]):
if not compliance_ids:
return
compliance_ids = sorted(set(compliance_ids))
with rls_transaction(tenant_id, using=MainRouter.default_db) as cursor:
# Serialize tenant-level summary updates to avoid concurrent recomputes
cursor.execute(
"SELECT pg_advisory_xact_lock(hashtext(%s))",
[tenant_id],
)
cursor.execute(
COMPLIANCE_UPSERT_TENANT_SUMMARY_SQL,
[tenant_id, tenant_id, compliance_ids],
)
cursor.execute(
COMPLIANCE_DELETE_EMPTY_TENANT_SUMMARY_SQL,
[tenant_id, compliance_ids],
)
def delete_provider(tenant_id: str, pk: str):
"""
Gracefully deletes an instance of a provider along with its related data.
@@ -39,6 +66,12 @@ def delete_provider(tenant_id: str, pk: str):
logger.info(f"Provider `{pk}` already deleted, skipping")
return {}
compliance_ids = list(
ProviderComplianceScore.objects.filter(provider=instance)
.values_list("compliance_id", flat=True)
.distinct()
)
attack_paths_scan_ids = list(
AttackPathsScan.all_objects.filter(provider=instance).values_list(
"id", flat=True
@@ -90,6 +123,15 @@ def delete_provider(tenant_id: str, pk: str):
logger.error(f"Error deleting Provider: {db_error}")
raise
try:
_recalculate_tenant_compliance_summary(tenant_id, compliance_ids)
except Exception as db_error:
logger.error(
"Error recalculating tenant compliance summary after provider delete: %s",
db_error,
)
raise
return deletion_summary
+1
View File
@@ -137,6 +137,7 @@ COMPLIANCE_CLASS_MAP = {
# IaC provider doesn't have specific compliance frameworks yet
# Trivy handles its own compliance checks
],
"image": [],
"oraclecloud": [
(lambda name: name.startswith("cis_"), OracleCloudCIS),
(lambda name: name.startswith("csa_"), OracleCloudCSA),
+115 -79
View File
@@ -1,12 +1,14 @@
import os
import time
from glob import glob
from celery.utils.log import get_task_logger
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
from django.db import OperationalError
from tasks.utils import batched
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.db_utils import rls_transaction
from api.db_utils import REPLICA_MAX_ATTEMPTS, REPLICA_RETRY_BASE_DELAY, rls_transaction
from api.models import Finding, Integration, Provider
from api.utils import initialize_prowler_integration, initialize_prowler_provider
from prowler.lib.outputs.asff.asff import ASFF
@@ -17,11 +19,11 @@ from prowler.lib.outputs.html.html import HTML
from prowler.lib.outputs.ocsf.ocsf import OCSF
from prowler.providers.aws.aws_provider import AwsProvider
from prowler.providers.aws.lib.s3.s3 import S3
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHub
from prowler.providers.common.models import Connection
from prowler.providers.aws.lib.security_hub.exceptions.exceptions import (
SecurityHubNoEnabledRegionsError,
)
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHub
from prowler.providers.common.models import Connection
logger = get_task_logger(__name__)
@@ -291,96 +293,130 @@ def upload_security_hub_integration(
total_findings_sent[integration.id] = 0
# Process findings in batches to avoid memory issues
max_attempts = REPLICA_MAX_ATTEMPTS if READ_REPLICA_ALIAS else 1
has_findings = False
batch_number = 0
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
qs = (
Finding.all_objects.filter(tenant_id=tenant_id, scan_id=scan_id)
.order_by("uid")
.iterator()
)
for batch, _ in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
batch_number += 1
has_findings = True
# Transform findings for this batch
transformed_findings = [
FindingOutput.transform_api_finding(
finding, prowler_provider
)
for finding in batch
]
# Convert to ASFF format
asff_transformer = ASFF(
findings=transformed_findings,
file_path="",
file_extension="json",
for attempt in range(1, max_attempts + 1):
read_alias = None
if READ_REPLICA_ALIAS:
read_alias = (
READ_REPLICA_ALIAS
if attempt < max_attempts
else MainRouter.default_db
)
asff_transformer.transform(transformed_findings)
# Get the batch of ASFF findings
batch_asff_findings = asff_transformer.data
if batch_asff_findings:
# Create Security Hub client for first batch or reuse existing
if not security_hub_client:
connected, security_hub = (
get_security_hub_client_from_integration(
integration, tenant_id, batch_asff_findings
)
try:
batch_number = 0
has_findings = False
with rls_transaction(
tenant_id,
using=read_alias,
retry_on_replica=False,
):
qs = (
Finding.all_objects.filter(
tenant_id=tenant_id, scan_id=scan_id
)
.order_by("uid")
.iterator()
)
if not connected:
if isinstance(
security_hub.error,
SecurityHubNoEnabledRegionsError,
):
logger.warning(
f"Security Hub integration {integration.id} has no enabled regions"
for batch, _ in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
batch_number += 1
has_findings = True
# Transform findings for this batch
transformed_findings = [
FindingOutput.transform_api_finding(
finding, prowler_provider
)
for finding in batch
]
# Convert to ASFF format
asff_transformer = ASFF(
findings=transformed_findings,
file_path="",
file_extension="json",
)
asff_transformer.transform(transformed_findings)
# Get the batch of ASFF findings
batch_asff_findings = asff_transformer.data
if batch_asff_findings:
# Create Security Hub client for first batch or reuse existing
if not security_hub_client:
connected, security_hub = (
get_security_hub_client_from_integration(
integration,
tenant_id,
batch_asff_findings,
)
)
if not connected:
if isinstance(
security_hub.error,
SecurityHubNoEnabledRegionsError,
):
logger.warning(
f"Security Hub integration {integration.id} has no enabled regions"
)
else:
logger.error(
f"Security Hub connection failed for integration {integration.id}: "
f"{security_hub.error}"
)
break # Skip this integration
security_hub_client = security_hub
logger.info(
f"Sending {'fail' if send_only_fails else 'all'} findings to Security Hub via "
f"integration {integration.id}"
)
else:
logger.error(
f"Security Hub connection failed for integration {integration.id}: "
f"{security_hub.error}"
# Update findings in existing client for this batch
security_hub_client._findings_per_region = (
security_hub_client.filter(
batch_asff_findings,
send_only_fails,
)
)
break # Skip this integration
security_hub_client = security_hub
logger.info(
f"Sending {'fail' if send_only_fails else 'all'} findings to Security Hub via "
f"integration {integration.id}"
)
else:
# Update findings in existing client for this batch
security_hub_client._findings_per_region = (
security_hub_client.filter(
batch_asff_findings, send_only_fails
)
)
# Send this batch to Security Hub
try:
findings_sent = security_hub_client.batch_send_to_security_hub()
total_findings_sent[integration.id] += (
findings_sent
)
# Send this batch to Security Hub
try:
findings_sent = (
security_hub_client.batch_send_to_security_hub()
)
total_findings_sent[integration.id] += findings_sent
if findings_sent > 0:
logger.debug(
f"Sent batch {batch_number} with {findings_sent} findings to Security Hub"
)
except Exception as batch_error:
logger.error(
f"Failed to send batch {batch_number} to Security Hub: {str(batch_error)}"
)
if findings_sent > 0:
logger.debug(
f"Sent batch {batch_number} with {findings_sent} findings to Security Hub"
)
except Exception as batch_error:
logger.error(
f"Failed to send batch {batch_number} to Security Hub: {str(batch_error)}"
)
# Clear memory after processing each batch
asff_transformer._data.clear()
del batch_asff_findings
del transformed_findings
# Clear memory after processing each batch
asff_transformer._data.clear()
del batch_asff_findings
del transformed_findings
break
except OperationalError as e:
if attempt == max_attempts:
raise
delay = REPLICA_RETRY_BASE_DELAY * (2 ** (attempt - 1))
logger.info(
"RLS query failed during Security Hub integration "
f"(attempt {attempt}/{max_attempts}), retrying in {delay}s. Error: {e}"
)
time.sleep(delay)
if not has_findings:
logger.info(
+14
View File
@@ -93,6 +93,20 @@ COMPLIANCE_UPSERT_TENANT_SUMMARY_SQL = """
updated_at = NOW()
"""
# Delete tenant compliance summaries with no remaining provider scores.
# Parameters: [tenant_id, compliance_ids_array]
COMPLIANCE_DELETE_EMPTY_TENANT_SUMMARY_SQL = """
DELETE FROM tenant_compliance_summaries tcs
WHERE tcs.tenant_id = %s
AND tcs.compliance_id = ANY(%s)
AND NOT EXISTS (
SELECT 1
FROM provider_compliance_scores pcs
WHERE pcs.tenant_id = tcs.tenant_id
AND pcs.compliance_id = tcs.compliance_id
)
"""
# Upsert tenant compliance summary for ALL compliance IDs in tenant.
# Used by backfill when recalculating entire tenant summary.
# Parameters: [tenant_id, tenant_id]
+49 -3
View File
@@ -1,12 +1,11 @@
from unittest.mock import call, patch
import pytest
from django.core.exceptions import ObjectDoesNotExist
from tasks.jobs.deletion import delete_provider, delete_tenant
from api.attack_paths import database as graph_database
from api.models import Provider, Tenant
from tasks.jobs.deletion import delete_provider, delete_tenant
from api.models import Provider, Tenant, TenantComplianceSummary
@pytest.mark.django_db
@@ -104,6 +103,53 @@ class TestDeleteProvider:
assert result
assert not Provider.all_objects.filter(pk=instance.id).exists()
def test_delete_provider_recalculates_tenant_compliance_summary(
self,
providers_fixture,
provider_compliance_scores_fixture,
):
instance = providers_fixture[0]
tenant_id = instance.tenant_id
TenantComplianceSummary.objects.create(
tenant_id=tenant_id,
compliance_id="aws_cis_2.0",
requirements_passed=99,
requirements_failed=99,
requirements_manual=99,
total_requirements=99,
)
TenantComplianceSummary.objects.create(
tenant_id=tenant_id,
compliance_id="gdpr_aws",
requirements_passed=99,
requirements_failed=99,
requirements_manual=99,
total_requirements=99,
)
with (
patch(
"tasks.jobs.deletion.graph_database.get_database_name",
return_value="tenant-db",
),
patch("tasks.jobs.deletion.graph_database.drop_subgraph"),
):
delete_provider(str(tenant_id), instance.id)
updated_summary = TenantComplianceSummary.objects.get(
tenant_id=tenant_id,
compliance_id="aws_cis_2.0",
)
assert updated_summary.requirements_passed == 1
assert updated_summary.requirements_failed == 1
assert updated_summary.requirements_manual == 0
assert updated_summary.total_requirements == 2
assert not TenantComplianceSummary.objects.filter(
tenant_id=tenant_id,
compliance_id="gdpr_aws",
).exists()
@pytest.mark.django_db
class TestDeleteTenant:
@@ -1,6 +1,7 @@
from unittest.mock import MagicMock, patch
import pytest
from django.db import OperationalError
from tasks.jobs.integrations import (
get_s3_client_from_integration,
get_security_hub_client_from_integration,
@@ -1056,6 +1057,84 @@ class TestSecurityHubIntegrationUploads:
mock_security_hub.batch_send_to_security_hub.assert_called_once()
mock_security_hub.archive_previous_findings.assert_called_once()
@patch("tasks.jobs.integrations.time.sleep")
@patch("tasks.jobs.integrations.batched")
@patch("tasks.jobs.integrations.get_security_hub_client_from_integration")
@patch("tasks.jobs.integrations.initialize_prowler_provider")
@patch("tasks.jobs.integrations.rls_transaction")
@patch("tasks.jobs.integrations.Integration")
@patch("tasks.jobs.integrations.Provider")
@patch("tasks.jobs.integrations.Finding")
def test_upload_security_hub_integration_retries_on_operational_error(
self,
mock_finding_model,
mock_provider_model,
mock_integration_model,
mock_rls,
mock_initialize_provider,
mock_get_security_hub,
mock_batched,
mock_sleep,
):
"""Test SecurityHub upload retries on transient OperationalError."""
tenant_id = "tenant-id"
provider_id = "provider-id"
scan_id = "scan-123"
integration = MagicMock()
integration.id = "integration-1"
integration.configuration = {
"send_only_fails": True,
"archive_previous_findings": False,
}
mock_integration_model.objects.filter.return_value = [integration]
provider = MagicMock()
mock_provider_model.objects.get.return_value = provider
mock_prowler_provider = MagicMock()
mock_initialize_provider.return_value = mock_prowler_provider
mock_findings = [MagicMock(), MagicMock()]
mock_finding_model.all_objects.filter.return_value.order_by.return_value.iterator.return_value = iter(
mock_findings
)
transformed_findings = [MagicMock(), MagicMock()]
with patch("tasks.jobs.integrations.FindingOutput") as mock_finding_output:
mock_finding_output.transform_api_finding.side_effect = transformed_findings
with patch("tasks.jobs.integrations.ASFF") as mock_asff:
mock_asff_instance = MagicMock()
finding1 = MagicMock()
finding1.Compliance.Status = "FAILED"
finding2 = MagicMock()
finding2.Compliance.Status = "FAILED"
mock_asff_instance.data = [finding1, finding2]
mock_asff_instance._data = MagicMock()
mock_asff.return_value = mock_asff_instance
mock_security_hub = MagicMock()
mock_security_hub.batch_send_to_security_hub.return_value = 2
mock_get_security_hub.return_value = (True, mock_security_hub)
mock_rls.return_value.__enter__.return_value = None
mock_rls.return_value.__exit__.return_value = False
mock_batched.side_effect = [
OperationalError("Conflict with recovery"),
[(mock_findings, None)],
]
with patch("tasks.jobs.integrations.REPLICA_MAX_ATTEMPTS", 2):
with patch("tasks.jobs.integrations.READ_REPLICA_ALIAS", "replica"):
result = upload_security_hub_integration(
tenant_id, provider_id, scan_id
)
assert result is True
mock_sleep.assert_called_once()
@patch("tasks.jobs.integrations.get_security_hub_client_from_integration")
@patch("tasks.jobs.integrations.initialize_prowler_provider")
@patch("tasks.jobs.integrations.rls_transaction")
@@ -0,0 +1,24 @@
import warnings
from dashboard.common_methods import get_section_containers_format3
warnings.filterwarnings("ignore")
def get_table(data):
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
].copy()
return get_section_containers_format3(
aux, "REQUIREMENTS_ATTRIBUTES_SECTION", "REQUIREMENTS_ID"
)
+15
View File
@@ -99,6 +99,7 @@
},
"user-guide/tutorials/prowler-app-rbac",
"user-guide/tutorials/prowler-app-api-keys",
"user-guide/tutorials/prowler-app-findings-ingestion",
{
"group": "Mutelist",
"expanded": true,
@@ -116,6 +117,13 @@
"user-guide/tutorials/prowler-app-jira-integration"
]
},
{
"group": "AWS Organizations",
"expanded": true,
"pages": [
"user-guide/tutorials/prowler-cloud-aws-organizations"
]
},
{
"group": "Lighthouse AI",
"pages": [
@@ -227,6 +235,13 @@
"user-guide/providers/microsoft365/use-of-powershell"
]
},
{
"group": "Google Workspace",
"pages": [
"user-guide/providers/googleworkspace/getting-started-googleworkspace",
"user-guide/providers/googleworkspace/authentication"
]
},
{
"group": "GitHub",
"pages": [
@@ -23,9 +23,15 @@ Refer to the [Prowler App Tutorial](/user-guide/tutorials/prowler-app) for detai
```bash
VERSION=$(curl -s https://api.github.com/repos/prowler-cloud/prowler/releases/latest | jq -r .tag_name)
curl -sLO "https://raw.githubusercontent.com/prowler-cloud/prowler/refs/tags/${VERSION}/docker-compose.yml"
# Environment variables can be customized in the .env file. Using default values in production environments is not recommended.
curl -sLO "https://raw.githubusercontent.com/prowler-cloud/prowler/refs/tags/${VERSION}/.env"
docker compose up -d
```
<Callout icon="lock" iconType="regular" color="#e74c3c">
For a secure setup, the API auto-generates a unique key pair, `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY`, and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate keys, delete the stored key files and restart the API.
</Callout>
</Tab>
<Tab title="GitHub">
_Requirements_:
Binary file not shown.

After

Width:  |  Height:  |  Size: 262 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 90 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 145 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 171 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

@@ -0,0 +1,71 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1100 420" font-family="'Inter', 'Segoe UI', system-ui, -apple-system, sans-serif">
<defs>
<filter id="shadow" x="-4%" y="-4%" width="108%" height="108%">
<feDropShadow dx="0" dy="2" stdDeviation="3" flood-opacity="0.08"/>
</filter>
</defs>
<!-- Title -->
<text x="550" y="40" text-anchor="middle" font-size="22" font-weight="700" fill="#4285F4">Onboarding Flow</text>
<!-- Step 1 -->
<rect x="30" y="70" width="220" height="220" rx="12" fill="#fff" stroke="#4285F4" stroke-width="2.5" stroke-dasharray="8 4" filter="url(#shadow)"/>
<circle cx="140" cy="100" r="22" fill="#4285F4"/>
<text x="140" y="107" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">1</text>
<text x="140" y="145" text-anchor="middle" font-size="15" font-weight="700" fill="#1a1a2e">Create Management</text>
<text x="140" y="165" text-anchor="middle" font-size="15" font-weight="700" fill="#1a1a2e">Account Role</text>
<rect x="80" y="185" width="120" height="24" rx="12" fill="#E8F0FE"/>
<text x="140" y="201" text-anchor="middle" font-size="11" font-weight="600" fill="#4285F4">Manually in IAM</text>
<text x="140" y="232" text-anchor="middle" font-size="12" fill="#5f6368">Allows Prowler to</text>
<text x="140" y="248" text-anchor="middle" font-size="12" fill="#5f6368">discover your org</text>
<text x="140" y="264" text-anchor="middle" font-size="12" fill="#5f6368">structure</text>
<!-- Arrow 1→2 -->
<path d="M260 180 L290 180" stroke="#9aa0a6" stroke-width="2" fill="none" marker-end="url(#arrowhead)"/>
<defs>
<marker id="arrowhead" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
<polygon points="0 0, 10 3.5, 0 7" fill="#9aa0a6"/>
</marker>
</defs>
<!-- Step 2 -->
<rect x="300" y="70" width="220" height="220" rx="12" fill="#fff" stroke="#7B61FF" stroke-width="2.5" filter="url(#shadow)"/>
<circle cx="410" cy="100" r="22" fill="#7B61FF"/>
<text x="410" y="107" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">2</text>
<text x="410" y="145" text-anchor="middle" font-size="15" font-weight="700" fill="#1a1a2e">Deploy StackSet</text>
<rect x="340" y="165" width="140" height="24" rx="12" fill="#F3F0FF"/>
<text x="410" y="181" text-anchor="middle" font-size="11" font-weight="600" fill="#7B61FF">In AWS Console</text>
<text x="410" y="212" text-anchor="middle" font-size="12" fill="#5f6368">Creates ProwlerScan</text>
<text x="410" y="228" text-anchor="middle" font-size="12" fill="#5f6368">role in every</text>
<text x="410" y="244" text-anchor="middle" font-size="12" fill="#5f6368">member account</text>
<!-- Arrow 2→3 -->
<path d="M530 180 L560 180" stroke="#9aa0a6" stroke-width="2" fill="none" marker-end="url(#arrowhead)"/>
<!-- Step 3 -->
<rect x="570" y="70" width="220" height="220" rx="12" fill="#fff" stroke="#00BFA5" stroke-width="2.5" filter="url(#shadow)"/>
<circle cx="680" cy="100" r="22" fill="#00BFA5"/>
<text x="680" y="107" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">3</text>
<text x="680" y="145" text-anchor="middle" font-size="15" font-weight="700" fill="#1a1a2e">Run the Wizard</text>
<rect x="615" y="165" width="130" height="24" rx="12" fill="#E0F7F4"/>
<text x="680" y="181" text-anchor="middle" font-size="11" font-weight="600" fill="#00BFA5">In Prowler Cloud</text>
<text x="680" y="212" text-anchor="middle" font-size="12" fill="#5f6368">Discovers accounts,</text>
<text x="680" y="228" text-anchor="middle" font-size="12" fill="#5f6368">tests connections</text>
<!-- Arrow 3→4 -->
<path d="M800 180 L830 180" stroke="#9aa0a6" stroke-width="2" fill="none" marker-end="url(#arrowhead)"/>
<!-- Step 4 -->
<rect x="840" y="70" width="220" height="220" rx="12" fill="#fff" stroke="#F9AB00" stroke-width="2.5" filter="url(#shadow)"/>
<circle cx="950" cy="100" r="22" fill="#F9AB00"/>
<text x="950" y="107" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">4</text>
<text x="950" y="145" text-anchor="middle" font-size="15" font-weight="700" fill="#1a1a2e">Launch Scans</text>
<rect x="898" y="165" width="104" height="24" rx="12" fill="#FEF7E0"/>
<text x="950" y="181" text-anchor="middle" font-size="11" font-weight="600" fill="#F9AB00">Automatic</text>
<text x="950" y="212" text-anchor="middle" font-size="12" fill="#5f6368">Scans run on all</text>
<text x="950" y="228" text-anchor="middle" font-size="12" fill="#5f6368">connected accounts</text>
<text x="950" y="244" text-anchor="middle" font-size="12" fill="#5f6368">on your schedule</text>
<!-- Footer -->
<text x="550" y="340" text-anchor="middle" font-size="13" fill="#9aa0a6">Steps 1 and 2 are done once in AWS | Steps 3 and 4 are done in Prowler Cloud</text>
</svg>

After

Width:  |  Height:  |  Size: 4.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 229 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 164 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 227 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 220 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 221 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

@@ -0,0 +1,96 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1100 520" font-family="'Inter', 'Segoe UI', system-ui, -apple-system, sans-serif">
<defs>
<filter id="shadow" x="-4%" y="-4%" width="108%" height="108%">
<feDropShadow dx="0" dy="2" stdDeviation="4" flood-opacity="0.08"/>
</filter>
<linearGradient id="mgmtHeader" x1="0" y1="0" x2="1" y2="0">
<stop offset="0%" stop-color="#4285F4"/>
<stop offset="100%" stop-color="#5E97F6"/>
</linearGradient>
<linearGradient id="memberHeader" x1="0" y1="0" x2="1" y2="0">
<stop offset="0%" stop-color="#00BFA5"/>
<stop offset="100%" stop-color="#1DE9B6"/>
</linearGradient>
</defs>
<!-- Title -->
<text x="550" y="36" text-anchor="middle" font-size="22" font-weight="700" fill="#4285F4">Two Roles Architecture</text>
<!-- ===== Management Account Card ===== -->
<rect x="30" y="60" width="440" height="380" rx="14" fill="#fff" stroke="#4285F4" stroke-width="2" filter="url(#shadow)"/>
<!-- Header bar -->
<rect x="30" y="60" width="440" height="44" rx="14" fill="url(#mgmtHeader)"/>
<rect x="30" y="90" width="440" height="14" fill="url(#mgmtHeader)"/>
<text x="250" y="88" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">Management Account</text>
<!-- Inner card -->
<rect x="50" y="118" width="400" height="300" rx="10" fill="#F0F4FF" stroke="#C5D6F7" stroke-width="1"/>
<text x="250" y="148" text-anchor="middle" font-size="15" font-weight="700" fill="#4285F4">Management Role</text>
<!-- Purpose -->
<text x="70" y="178" font-size="13" font-weight="700" fill="#1a1a2e">Purpose:</text>
<text x="70" y="196" font-size="12" fill="#5f6368">Discover Organization structure + scan management account</text>
<!-- Permissions -->
<text x="70" y="222" font-size="13" font-weight="700" fill="#1a1a2e">Permissions:</text>
<text x="82" y="240" font-size="11" fill="#5f6368">SecurityAudit (AWS managed policy)</text>
<text x="82" y="256" font-size="11" fill="#5f6368">ViewOnlyAccess (AWS managed policy)</text>
<text x="82" y="272" font-size="11" fill="#5f6368">Additional read-only (inline policy)</text>
<text x="82" y="288" font-size="11" fill="#4285F4" font-weight="600">organizations:DescribeAccount</text>
<text x="82" y="304" font-size="11" fill="#4285F4" font-weight="600">organizations:DescribeOrganization</text>
<text x="82" y="320" font-size="11" fill="#4285F4" font-weight="600">organizations:ListAccounts</text>
<text x="82" y="336" font-size="11" fill="#4285F4" font-weight="600">organizations:ListAccountsForParent</text>
<text x="82" y="352" font-size="11" fill="#4285F4" font-weight="600">organizations:ListOrganizationalUnitsForParent</text>
<text x="82" y="368" font-size="11" fill="#4285F4" font-weight="600">organizations:ListRoots</text>
<text x="82" y="384" font-size="11" fill="#4285F4" font-weight="600">organizations:ListTagsForResource</text>
<!-- Deploy badge -->
<rect x="145" y="400" width="210" height="28" rx="14" fill="#FFF3E0" stroke="#F9AB00" stroke-width="1.5"/>
<text x="250" y="419" text-anchor="middle" font-size="12" font-weight="700" fill="#E65100">Deploy: MANUALLY in IAM Console</text>
<!-- ===== Prowler Cloud connector ===== -->
<rect x="490" y="195" width="120" height="36" rx="8" fill="#F5F5F5" stroke="#E0E0E0" stroke-width="1"/>
<text x="550" y="218" text-anchor="middle" font-size="12" font-weight="600" fill="#5f6368">Prowler Cloud</text>
<!-- Connector lines -->
<line x1="490" y1="213" x2="470" y2="213" stroke="#9aa0a6" stroke-width="1.5" stroke-dasharray="4 3"/>
<line x1="610" y1="213" x2="630" y2="213" stroke="#9aa0a6" stroke-width="1.5" stroke-dasharray="4 3"/>
<!-- ===== Member Accounts Card ===== -->
<rect x="630" y="60" width="440" height="380" rx="14" fill="#fff" stroke="#00BFA5" stroke-width="2" filter="url(#shadow)"/>
<!-- Header bar -->
<rect x="630" y="60" width="440" height="44" rx="14" fill="url(#memberHeader)"/>
<rect x="630" y="90" width="440" height="14" fill="url(#memberHeader)"/>
<text x="850" y="88" text-anchor="middle" font-size="16" font-weight="700" fill="#fff">Member Accounts</text>
<!-- Inner card -->
<rect x="650" y="118" width="400" height="300" rx="10" fill="#E8F8F5" stroke="#B2DFDB" stroke-width="1"/>
<text x="850" y="148" text-anchor="middle" font-size="15" font-weight="700" fill="#00897B">ProwlerScan Role (per account)</text>
<!-- Purpose -->
<text x="670" y="178" font-size="13" font-weight="700" fill="#1a1a2e">Purpose:</text>
<text x="670" y="196" font-size="12" fill="#5f6368">Security scanning of each account</text>
<!-- Permissions -->
<text x="670" y="222" font-size="13" font-weight="700" fill="#1a1a2e">Permissions:</text>
<text x="682" y="240" font-size="11" fill="#5f6368">SecurityAudit (AWS managed policy)</text>
<text x="682" y="256" font-size="11" fill="#5f6368">ViewOnlyAccess (AWS managed policy)</text>
<text x="682" y="272" font-size="11" fill="#5f6368">Additional read-only (inline policy)</text>
<!-- Scope -->
<text x="670" y="302" font-size="13" font-weight="700" fill="#1a1a2e">Scope:</text>
<text x="682" y="320" font-size="12" fill="#5f6368">Read-only access across all AWS services</text>
<text x="682" y="338" font-size="12" fill="#5f6368">No write or modify permissions</text>
<!-- Deploy badge -->
<rect x="735" y="400" width="230" height="28" rx="14" fill="#E8F5E9" stroke="#66BB6A" stroke-width="1.5"/>
<text x="850" y="419" text-anchor="middle" font-size="12" font-weight="700" fill="#2E7D32">Deploy: via CloudFormation StackSet</text>
<!-- Footer labels -->
<text x="250" y="478" text-anchor="middle" font-size="14" font-weight="700" fill="#4285F4">Prowler discovers</text>
<text x="250" y="496" text-anchor="middle" font-size="12" fill="#5f6368">your org structure</text>
<text x="850" y="478" text-anchor="middle" font-size="14" font-weight="700" fill="#00BFA5">Prowler scans each</text>
<text x="850" y="496" text-anchor="middle" font-size="12" fill="#5f6368">account for findings</text>
</svg>

After

Width:  |  Height:  |  Size: 5.9 KiB

+2
View File
@@ -0,0 +1,2 @@
// Reo tracking beacon
!function(){var e,t,n;e="1fca1c3c1571b22",t=function(){Reo.init({clientID:"1fca1c3c1571b22"})},(n=document.createElement("script")).src="https://static.reo.dev/"+e+"/reo.js",n.defer=!0,n.onload=t,document.head.appendChild(n)}();
@@ -2,6 +2,12 @@
title: 'AWS Organizations in Prowler'
---
<Info>
**Using Prowler Cloud?** You can onboard your entire AWS Organization through the UI with automatic account discovery, OU-aware tree selection, and bulk connection testing — no scripts or YAML files required.
See [AWS Organizations in Prowler Cloud](/user-guide/tutorials/prowler-cloud-aws-organizations) for the full walkthrough.
</Info>
Prowler can integrate with AWS Organizations to manage the visibility and onboarding of accounts centrally.
When trusted access is enabled with the Organization, Prowler can discover accounts as they are created and even automate deployment of the Prowler Scan IAM Role.
@@ -118,6 +118,15 @@ Fine-Grained Personal Access Tokens are ideal for:
**Create a Fine-Grained Token:**
<Tip>
**Quick Setup:** Use these pre-configured links to create a token with the required permissions already selected:
- [Create token for user repositories](https://github.com/settings/personal-access-tokens/new?name=Prowler+Security+Scanner&description=Fine-grained+PAT+for+Prowler+security+scanning&expires_in=90&administration=read&contents=read&vulnerability_alerts=read&emails=read) — scans personal repositories
- [Create token for organization scanning](https://github.com/settings/personal-access-tokens/new?name=Prowler+Security+Scanner&description=Fine-grained+PAT+for+Prowler+organization+security+scanning&expires_in=90&administration=read&contents=read&vulnerability_alerts=read&emails=read&organization_administration=read&members=read) — scans organization repositories and settings
For organization scanning, change the **Resource Owner** to the target organization after the page loads. Organization permissions only appear when an organization is selected.
</Tip>
1. Navigate to **GitHub Settings** > **Developer settings**.
2. Click **Personal access tokens** > **Fine-grained tokens** > **Generate new token**.
@@ -98,7 +98,7 @@ Before adding GitHub to Prowler Cloud/App, ensure you have:
2. Click **Verify** to test the connection
3. Click **Save**
**Don't have a token yet?** See [How to create a Personal Access Token](/user-guide/providers/github/authentication#create-a-fine-grained-personal-access-token)
**Don't have a token yet?** [Create a pre-configured token on GitHub](https://github.com/settings/personal-access-tokens/new?name=Prowler+Security+Scanner&description=Fine-grained+PAT+for+Prowler+security+scanning&expires_in=90&administration=read&contents=read&vulnerability_alerts=read&emails=read) or see [How to create a Personal Access Token](/user-guide/providers/github/authentication#create-a-fine-grained-personal-access-token) for detailed instructions.
</Tab>
<Tab title="OAuth App Token">
@@ -0,0 +1,156 @@
---
title: 'Google Workspace Authentication in Prowler'
---
Prowler for Google Workspace uses a **Service Account with Domain-Wide Delegation** to authenticate to the Google Workspace Admin SDK. This allows Prowler to read directory data on behalf of a super administrator without requiring an interactive login.
## Required OAuth Scopes
Prowler requests the following read-only OAuth 2.0 scopes from the Google Workspace Admin SDK:
| Scope | Description |
|-------|-------------|
| `https://www.googleapis.com/auth/admin.directory.user.readonly` | Read access to user accounts and their admin status |
| `https://www.googleapis.com/auth/admin.directory.domain.readonly` | Read access to domain information |
| `https://www.googleapis.com/auth/admin.directory.customer.readonly` | Read access to customer information (Customer ID) |
<Warning>
The delegated user must be a **super administrator** in your Google Workspace organization. Using a non-admin account will result in permission errors when accessing the Admin SDK.
</Warning>
## Setup Steps
### Step 1: Create a GCP Project (if needed)
If you don't have a GCP project, create one at [https://console.cloud.google.com](https://console.cloud.google.com).
The project is only used to host the Service Account — it does not need to have any Google Workspace data in it.
### Step 2: Enable the Admin SDK API
1. Go to the [Google Cloud Console](https://console.cloud.google.com)
2. Select your project
3. Navigate to **APIs & Services → Library**
4. Search for **Admin SDK API**
5. Click **Enable**
### Step 3: Create a Service Account
1. In the Google Cloud Console, navigate to **IAM & Admin → Service Accounts**
2. Click **Create Service Account**
3. Give it a descriptive name (e.g., `prowler-googleworkspace-reader`)
4. Click **Create and Continue**
5. Skip the optional role and user access steps — click **Done**
<Note>
The Service Account does not need any GCP IAM roles. Its access to Google Workspace is granted entirely through Domain-Wide Delegation in the next steps.
</Note>
### Step 4: Generate a JSON Key
1. Click on the Service Account you just created
2. Go to the **Keys** tab
3. Click **Add Key → Create new key**
4. Select **JSON** format
5. Click **Create** — the key file will download automatically
6. Store it securely (e.g., `~/.config/prowler/googleworkspace-sa.json`)
<Warning>
This JSON key grants access to your Google Workspace organization. Never commit it to version control, share it in plain text, or store it in an insecure location.
</Warning>
### Step 5: Configure Domain-Wide Delegation in Google Workspace
1. Go to the [Google Workspace Admin Console](https://admin.google.com)
2. Navigate to **Security → Access and data control → API controls**
3. Click **Manage Domain Wide Delegation**
4. Click **Add new**
5. Enter the **Client ID** of the Service Account (found in the JSON key as `client_id`, or on the Service Account details page)
6. In the **OAuth scopes** field, enter the following scopes as a comma-separated list:
```
https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.domain.readonly,https://www.googleapis.com/auth/admin.directory.customer.readonly
```
7. Click **Authorize**
<Note>
Domain-Wide Delegation must be configured by a Google Workspace **super administrator**. It may take a few minutes to propagate after saving.
</Note>
### Step 6: Store Credentials Securely
Set your credentials as environment variables:
```bash
export GOOGLEWORKSPACE_CREDENTIALS_FILE="/path/to/googleworkspace-sa.json"
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
```
Alternatively, if you need to pass credentials as a string (e.g., in CI/CD pipelines):
```bash
export GOOGLEWORKSPACE_CREDENTIALS_CONTENT=$(cat /path/to/googleworkspace-sa.json)
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
```
## Credential Lookup Order
Prowler resolves credentials in the following order:
1. `GOOGLEWORKSPACE_CREDENTIALS_FILE` environment variable
2. `GOOGLEWORKSPACE_CREDENTIALS_CONTENT` environment variable
The delegated user must be provided via the `GOOGLEWORKSPACE_DELEGATED_USER` environment variable.
## Best Practices
- **Use environment variables** — Never hardcode credentials in scripts or commands
- **Use a dedicated Service Account** — Create one specifically for Prowler, separate from other integrations
- **Use read-only scopes** — Prowler only requires the three read-only scopes listed above
- **Restrict key access** — Set file permissions to `600` on the JSON key file
- **Rotate keys regularly** — Delete and regenerate the JSON key periodically
- **Use a least-privilege super admin** — Consider using a dedicated super admin account for Prowler's delegated user rather than a personal admin account
```bash
# Secure the key file
chmod 600 /path/to/googleworkspace-sa.json
```
## Troubleshooting
### `GoogleWorkspaceMissingDelegatedUserError`
The delegated user email was not provided. Set it via environment variable:
```bash
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
```
### `GoogleWorkspaceNoCredentialsError`
No credentials were found. Ensure either `GOOGLEWORKSPACE_CREDENTIALS_FILE` or `GOOGLEWORKSPACE_CREDENTIALS_CONTENT` is set.
### `GoogleWorkspaceInvalidCredentialsError`
The JSON key file is malformed or cannot be parsed. Verify the file was downloaded correctly and is valid JSON:
```bash
python3 -c "import json; json.load(open('/path/to/key.json'))" && echo "Valid JSON"
```
### `GoogleWorkspaceImpersonationError`
The Service Account cannot impersonate the delegated user. This usually means Domain-Wide Delegation has not been configured, or the OAuth scopes are incorrect. Verify:
- The Service Account Client ID is correctly entered in the Admin Console
- All three required OAuth scopes are included
- The delegated user is a super administrator
### Permission Denied on Admin SDK calls
If Prowler connects but returns empty results or permission errors for specific API calls:
- Confirm Domain-Wide Delegation is fully propagated (wait a few minutes after setup)
- Verify all three scopes are authorized in the Admin Console
- Ensure the delegated user is an active super administrator
@@ -0,0 +1,100 @@
---
title: 'Getting Started with Google Workspace'
---
import { VersionBadge } from "/snippets/version-badge.mdx";
<VersionBadge version="5.19.0" />
Prowler for Google Workspace allows you to audit your organization's Google Workspace environment for security misconfigurations, including super administrator account hygiene, domain settings, and more.
## Prerequisites
Before running Prowler with the Google Workspace provider, ensure you have:
1. A Google Workspace account with super administrator privileges
2. A Google Cloud Platform (GCP) project to host the Service Account
3. Authentication configured (see [Authentication](/user-guide/providers/googleworkspace/authentication)):
- A **Service Account JSON key** from a GCP project with Domain-Wide Delegation enabled
## Quick Start
### Step 1: Set Up Authentication
Set your Service Account credentials file path and delegated user email as environment variables:
```bash
export GOOGLEWORKSPACE_CREDENTIALS_FILE="/path/to/service-account-key.json"
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
```
### Step 2: Run Prowler
```bash
prowler googleworkspace
```
Prowler will authenticate as the delegated user and run all available security checks against your Google Workspace organization.
## Authentication
Prowler uses a **Service Account with Domain-Wide Delegation** to authenticate to Google Workspace. This requires:
- A Service Account created in a GCP project
- The Admin SDK API enabled in that project
- Domain-Wide Delegation configured in the Google Workspace Admin Console
- A super admin user email to impersonate
### Using Environment Variables (Recommended)
```bash
export GOOGLEWORKSPACE_CREDENTIALS_FILE="/path/to/service-account-key.json"
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
prowler googleworkspace
```
Alternatively, pass the credentials content directly as a JSON string:
```bash
export GOOGLEWORKSPACE_CREDENTIALS_CONTENT='{"type": "service_account", ...}'
export GOOGLEWORKSPACE_DELEGATED_USER="admin@yourdomain.com"
prowler googleworkspace
```
<Note>
The delegated user must be a super admin email in your Google Workspace organization. The service account credentials must be provided via environment variables (`GOOGLEWORKSPACE_CREDENTIALS_FILE` or `GOOGLEWORKSPACE_CREDENTIALS_CONTENT`).
</Note>
## Understanding the Output
When Prowler runs successfully, it will display the credentials being used:
```
Using the Google Workspace credentials below:
┌─────────────────────────────────────────────────────────┐
│ Google Workspace Domain: yourdomain.com │
│ Customer ID: C0xxxxxxx │
│ Delegated User: admin@yourdomain.com │
│ Authentication Method: Service Account with Domain-Wide │
│ Delegation │
└─────────────────────────────────────────────────────────┘
```
Findings are reported per check. For example, the `directory_super_admin_count` check verifies the number of super administrators is within a recommended range (2–4):
- **PASS** — 2 to 4 super administrators found
- **FAIL** — 0 or 1 (single point of failure) or 5+ (excessive privilege exposure)
Output files are saved in the configured output directory (default: `output/`) in CSV, JSON-OCSF, and HTML formats.
## Configuration
Prowler uses a configuration file to customize provider behavior. To use a custom configuration:
```bash
prowler googleworkspace --config-file /path/to/config.yaml
```
## Next Steps
- [Authentication](/user-guide/providers/googleworkspace/authentication) — Detailed guide on setting up a Service Account and Domain-Wide Delegation
@@ -337,6 +337,99 @@ prowler openstack --clouds-yaml-cloud ovh-staging --output-directory ./reports/o
prowler openstack --clouds-yaml-cloud infomaniak-production --output-directory ./reports/infomaniak/
```
## Multi-Region Scanning
Many OpenStack providers (OVH, Infomaniak, etc.) offer resources across multiple regions within the same project. By default, the `clouds.yaml` file downloaded from Horizon uses `region_name` which targets a **single region**. Prowler supports scanning **all regions** in a single run by using the `regions` key instead.
### Configuring Multi-Region
Replace the `region_name` key with a `regions` list in your `clouds.yaml`:
```yaml
clouds:
ovh-multiregion:
auth:
auth_url: https://auth.cloud.ovh.net/v3
username: user-xxxxxxxxxx
password: your-password-here
project_id: your-project-id
user_domain_name: Default
project_domain_name: Default
regions:
- UK1
- DE1
identity_api_version: "3"
```
Then run Prowler as usual:
```bash
prowler openstack --clouds-yaml-cloud ovh-multiregion
```
Prowler will create a separate connection to each region and scan all resources across them. Findings in the output will include the region where each resource was found.
<Warning>
You must use **either** `region_name` (single region) **or** `regions` (multi-region), not both. Prowler will raise an error if both keys are present in the same cloud configuration.
</Warning>
### How It Works
The `region_name` and `regions` keys are part of the [OpenStack SDK configuration format](https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html#site-specific-file-locations). When `regions` is set, the SDK can produce a separate cloud config object for each region — but it does not iterate over them automatically. Prowler uses this to create one authenticated connection per region and iterates over all of them when listing resources. This means:
- **Authentication** is tested against every configured region during connection setup
- **Resources** from all regions are collected in a single scan
- **Findings** include the specific region for each resource
- If a single region fails to connect, the entire scan fails (fail-fast)
### Finding Your Available Regions
To discover which regions are available for your project, use the OpenStack CLI:
```bash
openstack --os-cloud your-cloud region list
```
Or check your provider's control panel for a list of available regions.
### Single-Region vs Multi-Region
| Configuration | Key | Behavior |
|--------------|-----|----------|
| Single region | `region_name: UK1` | Scans resources in UK1 only |
| Multi-region | `regions: [UK1, DE1]` | Scans resources in both UK1 and DE1 |
You can keep both configurations as separate cloud entries in the same `clouds.yaml` file:
```yaml
clouds:
# Single region entry — targets UK1 only
ovh:
auth:
auth_url: https://auth.cloud.ovh.net/v3
username: user-xxxxxxxxxx
password: your-password-here
project_id: your-project-id
user_domain_name: Default
project_domain_name: Default
region_name: UK1
identity_api_version: "3"
# Multi-region entry — targets UK1 and DE1
ovh-multiregion:
auth:
auth_url: https://auth.cloud.ovh.net/v3
username: user-xxxxxxxxxx
password: your-password-here
project_id: your-project-id
user_domain_name: Default
project_domain_name: Default
regions:
- UK1
- DE1
identity_api_version: "3"
```
## Creating a User With Reader Role
For security auditing, Prowler only needs **read-only access** to your OpenStack resources.
@@ -534,3 +627,4 @@ Using Public Cloud credentials can limit Keystone API access, so the command abo
- [OpenStack Documentation](https://docs.openstack.org/)
- [OpenStack Security Guide](https://docs.openstack.org/security-guide/)
- [clouds.yaml Format](https://docs.openstack.org/python-openstackclient/latest/configuration/index.html)
- [OpenStack SDK Configuration (`region_name` / `regions`)](https://docs.openstack.org/openstacksdk/latest/user/config/configuration.html#site-specific-file-locations)
Loaded 100 of 358 files, more files were not shown because too many files have changed in this diff. Show more