Compare commits

..

94 Commits

Author SHA1 Message Date
Andoni A. 0cb4784187 docs: add AWS Orgs tip in bulk provisioning tutorial 2025-10-22 12:46:55 +02:00
Andoni A. 124676e893 docs: remove how to configure aws creds -- too much detailed 2025-10-22 12:40:27 +02:00
Andoni A. e08c2f2605 docs: review with docs styleguide 2025-10-22 12:28:13 +02:00
Andoni A. 9758fc36df chore: use Prowler API key instead of temporal token 2025-10-22 12:22:28 +02:00
Andoni A. 56bb5e92cc docs: include AWS Orgs bulk importer tutorial 2025-10-22 12:12:36 +02:00
Andoni A. 2ef750d133 Merge branch 'master' into DEVREL-99-provision-all-aws-accounts-in-an-organization 2025-10-22 11:49:05 +02:00
César Arroba 18f3bc098c chore(github): trigger only if repository is prowler (#8974) 2025-10-22 09:27:33 +02:00
César Arroba 67b1983d85 chore(github): fix action (#8973) 2025-10-22 09:10:47 +02:00
César Arroba a3db23af7d chore(github): improve conventional commits action (#8969) 2025-10-21 17:57:29 +02:00
César Arroba 3eaa21f06f chore(github): improve backport label action (#8970) 2025-10-21 17:57:04 +02:00
Rubén De la Torre Vico 5d5c109067 chore(aws): enhance metadata for dlm service (#8860)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-21 17:40:19 +02:00
César Arroba c6cb4e4814 chore(github): improve backport action (#8968) 2025-10-21 17:14:40 +02:00
César Arroba ab06a09173 chore(api): improve pull request action (#8963) 2025-10-21 17:10:48 +02:00
Rubén De la Torre Vico 9c6c007f73 fix(mcp): add missing argument to health check (#8967) 2025-10-21 16:45:05 +02:00
Rubén De la Torre Vico 206f23b5a5 chore(aws): enhance metadata for dms service (#8861)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-21 16:31:18 +02:00
Andoni Alonso 5c9e9bc86a docs: fix security heading (#8965) 2025-10-21 16:13:55 +02:00
Rubén De la Torre Vico 34554d6123 feat(mcp): add support for production deployment with uvicorn (#8958) 2025-10-21 16:03:24 +02:00
Pepe Fagoaga 000cb93157 chore: remove security template as it's already there (#8964) 2025-10-21 19:34:42 +05:45
Adrián Jesús Peña Rodríguez 524209bdf2 feat(api): add provider_id__in filter for ScanSummary queries (#8951) 2025-10-21 15:24:09 +02:00
César Arroba c4a0da8204 chore(github): review and update issue templates (#8961) 2025-10-21 13:40:25 +02:00
César Arroba f0cba0321c chore(codeql): improve API CodeQL action and settings (#8962) 2025-10-21 13:40:07 +02:00
dependabot[bot] 79888c9312 chore(deps): bump playwright and @playwright/test in /ui (#8956)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-21 13:22:21 +02:00
Rubén De la Torre Vico a79910a694 chore(aws): enhance metadata for cloudtrail service (#8831)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-10-21 12:45:31 +02:00
César Arroba 4cadee7bb1 chore(github): update codeowners file (#8960) 2025-10-21 11:48:21 +02:00
Pedro Martín 756d436a2f feat(compliance): improve CCC catalogs (#8944) 2025-10-21 03:16:05 +02:00
Alejandro Bailo 5e85ef5835 feat(ui): new card components and derivates for overview (#8921)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-10-20 16:49:09 +02:00
Prowler Bot 0fa9e2da6c chore(regions_update): Changes in regions for AWS services (#8946)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-10-20 09:20:29 -04:00
Andoni Alonso ce7510db28 docs: remove anchors from redirects (#8953) 2025-10-20 14:58:53 +02:00
Pepe Fagoaga 8e3d50c807 fix(docs): redirect user-guide-tutorials (#8945) 2025-10-20 14:51:15 +02:00
Pepe Fagoaga d8908d2ccc docs(fix): space in providers table (#8938) 2025-10-20 14:39:03 +02:00
Alejandro Bailo 0b9969a723 feat: update M365 credentials form (#8929)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-10-20 13:51:11 +02:00
StylusFrost 985d73f44f test(ui): enhance Playwright test setups for user authentication (#8881)
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
2025-10-20 13:45:20 +02:00
Pedro Martín 1d705e22da feat(util): add from_yaml_to_json.py (#8943) 2025-10-20 12:29:29 +02:00
Rubén De la Torre Vico ca55d4ce86 chore(aws): enhance metadata for directoryservice service (#8859)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-20 12:20:16 +02:00
Hugo Pereira Brito 0201073fcb fix(docs): small enhancement in warning (#8950) 2025-10-20 12:19:49 +02:00
Alejandro Bailo 928c556721 fix: Mutelist view blinks at opening (#8932) 2025-10-17 19:26:57 +02:00
Rubén De la Torre Vico a653ad7852 chore(deps): remove docs group dependency (#8937) 2025-10-17 16:37:32 +02:00
Sergio Garcia a3c811f801 docs(github): clarify GitHub App configuration requirements (#8930) 2025-10-17 09:30:54 -04:00
Hugo Pereira Brito c85d3e9188 feat(docs): add M365 certificate and azure cli authentication methods (#8939) 2025-10-17 13:42:48 +02:00
Rubén De la Torre Vico 6f394cf9de docs(mcp): add comprehensive MCP Server documentation (#8931) 2025-10-17 11:48:48 +02:00
Rubén De la Torre Vico ba765fa07d chore(aws): enhance metadata for efs service (#8889)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-16 17:05:26 +02:00
Daniel Barranquero d928ee442f fix(gcp): no resource_name errors (#8928) 2025-10-16 14:58:45 +02:00
Alejandro Bailo 30ab5f52b9 feat(ui): add comprehensive agentic files (#8885)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-10-16 11:37:58 +02:00
Sergio Garcia c424707e32 feat(oci): Add Oracle Cloud Infrastructure provider with CIS 3.0 (#8893)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-15 13:05:51 -04:00
Pedro Martín 92efbe3926 chore(readme): update compliance numbers (#8926) 2025-10-15 18:17:15 +02:00
Pedro Martín 4a61578dd8 feat(compliance): add CCC catalogs for AWS, Azure and GCP (#8000)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-10-15 21:48:20 +05:45
Rubén De la Torre Vico ec75b5d0a3 feat(mcp): migrate documentation search from ReadTheDocs to Mintlify API (#8916) 2025-10-15 17:40:18 +02:00
Pepe Fagoaga db5bab51ae chore: delete mkdocs.yml (#8924) 2025-10-15 11:13:39 -04:00
Pepe Fagoaga be476b732a chore: delete readthedocs preview environment (#8923) 2025-10-15 20:54:40 +05:45
Andoni Alonso 434b37f758 docs: add prowler old root path redirect (#8922) 2025-10-15 20:41:46 +05:45
Andoni Alonso c08c27e5c6 docs: migrate to Mintlify (#8894) 2025-10-15 16:38:56 +02:00
Hugo Pereira Brito 8773751779 chore(api): enhance m365 user auth deprecation (#8913)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-10-15 15:41:40 +02:00
Víctor Fernández Poyatos f70a959a49 docs: API keys support (#8918) 2025-10-15 12:37:34 +02:00
Rubén De la Torre Vico 20314cad8c chore(mcp): add changelog with first version (#8884) 2025-10-15 12:04:48 +02:00
Pedro Martín 564ad56d2f feat(compliance): add C5 Germany for aws (#8830)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-10-15 11:47:23 +02:00
César Arroba b2d91c97d8 chore(mcp): modify MCP container action (#8902) 2025-10-14 18:18:27 +02:00
César Arroba c232195df4 chore(mcp): check for MCP changes on release preparation action (#8904) 2025-10-14 18:06:15 +02:00
Alan Buscaglia b4b9d800a8 style(ui): Migrate from Work Sans to Inter font (#8914) 2025-10-14 17:33:26 +02:00
dependabot[bot] fc1d3d4a47 chore(deps-dev): bump authlib from 1.6.4 to 1.6.5 in /api (#8910)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-14 09:49:52 -04:00
Pedro Martín d4be0f4d7a fix(compliance): add missing attributes for Mitre-Attack (#8907)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-14 15:48:02 +02:00
dependabot[bot] 305339ffb4 chore(deps-dev): bump authlib from 1.6.4 to 1.6.5 (#8900)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-10-14 09:31:42 -04:00
Daniel Barranquero 272e4547b2 fix(gcp): keyerrors in services cloudsql and monitoring (#8909) 2025-10-14 09:30:00 -04:00
Prowler Bot 8c3e1b96f9 chore(regions_update): Changes in regions for AWS services (#8901)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-10-14 09:27:32 -04:00
Rubén De la Torre Vico d496f5a58e fix(mcp): change int and float types to str (#8896) 2025-10-14 13:41:02 +02:00
Víctor Fernández Poyatos 5789e87f4f fix(api-keys): update created field to never update (#8908) 2025-10-14 13:30:41 +02:00
Alan Buscaglia 1994750151 fix(ui): Api Key Implementation Retouches (#8906) 2025-10-14 12:27:59 +02:00
Rubén De la Torre Vico 27304a8007 feat(mcp): add health check endpoint (#8905) 2025-10-14 12:16:51 +02:00
Rubén De la Torre Vico 9761651f8d chore(aws): enhance metadata for cloudfront service (#8829)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-14 09:26:33 +02:00
Rubén De la Torre Vico 406aace585 chore(aws): enhance metadata for autoscaling service (#8824)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-10-13 16:52:29 +02:00
Andoni A. f734b249a4 chore: create script to generate AWS accounts list from AWS Org for bulk provisioning 2025-10-13 16:07:33 +02:00
Rubén De la Torre Vico ebd5814112 chore(aws): enhance metadata for backup service (#8826)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-10-13 14:22:49 +02:00
Alan Buscaglia 42e816081e feat: reusable graph components (#8873)
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
2025-10-13 13:53:28 +02:00
Alan Buscaglia 741217ce80 feat(ui): API keys implementation (#8874) 2025-10-13 13:48:00 +02:00
Rubén De la Torre Vico 5f9ab68bd9 feat(mcp): add GitHub Action to publish MCP Server container to DockerHub (#8875)
Co-authored-by: César Arroba <19954079+cesararroba@users.noreply.github.com>
2025-10-13 10:31:02 +02:00
Alejandro Bailo fba2854f65 fix(ui): minor bugs (#8898) 2025-10-10 14:56:34 +02:00
Víctor Fernández Poyatos 8794515318 fix(api-keys): make name required and unique (#8891) 2025-10-10 12:35:27 +02:00
Víctor Fernández Poyatos 335db928dc feat(database): add db read replica support (#8869) 2025-10-10 12:27:43 +02:00
Alejandro Bailo 046baa8eb9 feat(ui): refreshToken implementation (#8864) 2025-10-10 11:02:10 +02:00
Alan Buscaglia ef60ea99c3 fix(api): throw errors for all non-ok responses (#8880) 2025-10-10 10:47:04 +02:00
Hugo Pereira Brito 1483efa18e feat(m365): add M365 certificate auth to API (#8538) 2025-10-10 10:43:11 +02:00
Hugo Pereira Brito b74744b135 feat(m365): add M365 certificate auth to API (#8538) 2025-10-09 16:50:28 +02:00
Pepe Fagoaga e80eed6baf chore(ui): remove .env.template (#8887) 2025-10-09 19:06:12 +05:45
Adrián Jesús Peña Rodríguez 1ba22f6f45 feat(api): update role mapping logic in TenantFinishACSView to handle single/manage account users (#8882) 2025-10-09 14:30:26 +02:00
Hugo Pereira Brito da6b7b89cb fix(tests): jira test double lines (#8886) 2025-10-09 13:44:01 +02:00
Hugo Pereira Brito cc9aa7f7ee feat(jira): support of ADF for MarkDown metadata fields (#8878) 2025-10-09 12:31:31 +02:00
Hugo Pereira Brito ecf749fce8 chore(m365): deprecate user auth (#8865) 2025-10-09 12:24:24 +02:00
Pedro Martín 1a7f52fc9c fix(threatscore): improve the way ThreatScore is calculated (#8582)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-10-09 11:50:10 +02:00
Víctor Fernández Poyatos b630234cdf fix(api-key): use admin connector to validate authentication (#8883) 2025-10-09 11:26:21 +02:00
Víctor Fernández Poyatos d6685eec1f feat(api-keys): support include parameter for entity details (#8876) 2025-10-09 11:14:13 +02:00
Pepe Fagoaga 86cff92d1f fix: conventional commit checker (#8879) 2025-10-08 13:19:43 -05:00
Rubén De la Torre Vico 28e81783ef feat(mcp): add API key support for STDIO mode and enhance HTTP mode authentication (#8823) 2025-10-08 15:52:26 +02:00
Rubén De la Torre Vico 13266b8743 feat(mcp): add Prowler Documentation MCP server (#8795) 2025-10-08 12:22:42 +02:00
Rubén De la Torre Vico 4e143cf013 feat(mcp): add HTTP transport support (#8784) 2025-10-08 11:32:39 +02:00
Rubén De la Torre Vico 5cfe140b7b fix(mcp): accept string type for all parameter types in MCP server (#8866) 2025-10-08 10:31:57 +02:00
1538 changed files with 75481 additions and 6794 deletions
+6
View File
@@ -29,6 +29,12 @@ POSTGRES_ADMIN_PASSWORD=postgres
POSTGRES_USER=prowler
POSTGRES_PASSWORD=postgres
POSTGRES_DB=prowler_db
# Read replica settings (optional)
# POSTGRES_REPLICA_HOST=postgres-db
# POSTGRES_REPLICA_PORT=5432
# POSTGRES_REPLICA_USER=prowler
# POSTGRES_REPLICA_PASSWORD=postgres
# POSTGRES_REPLICA_DB=prowler_db
# Celery-Prowler task settings
TASK_RETRY_DELAY_SECONDS=0.1
+27 -5
View File
@@ -1,6 +1,28 @@
# SDK
/* @prowler-cloud/sdk
/.github/ @prowler-cloud/sdk
prowler @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
tests @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
api @prowler-cloud/api
ui @prowler-cloud/ui
/prowler/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
/tests/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
/dashboard/ @prowler-cloud/sdk
/docs/ @prowler-cloud/sdk
/examples/ @prowler-cloud/sdk
/util/ @prowler-cloud/sdk
/contrib/ @prowler-cloud/sdk
/permissions/ @prowler-cloud/sdk
/codecov.yml @prowler-cloud/sdk @prowler-cloud/api
# API
/api/ @prowler-cloud/api
# UI
/ui/ @prowler-cloud/ui
# AI
/mcp_server/ @prowler-cloud/ai
# Platform
/.github/ @prowler-cloud/platform
/Makefile @prowler-cloud/platform
/kubernetes/ @prowler-cloud/platform
**/Dockerfile* @prowler-cloud/platform
**/docker-compose*.yml @prowler-cloud/platform
**/docker-compose*.yaml @prowler-cloud/platform
+44
View File
@@ -3,6 +3,41 @@ description: Create a report to help us improve
labels: ["bug", "status/needs-triage"]
body:
- type: checkboxes
id: search
attributes:
label: Issue search
options:
- label: I have searched the existing issues and this bug has not been reported yet
required: true
- type: dropdown
id: component
attributes:
label: Which component is affected?
multiple: true
options:
- Prowler CLI/SDK
- Prowler API
- Prowler UI
- Prowler Dashboard
- Prowler MCP Server
- Documentation
- Other
validations:
required: true
- type: dropdown
id: provider
attributes:
label: Cloud Provider (if applicable)
multiple: true
options:
- AWS
- Azure
- GCP
- Kubernetes
- GitHub
- Microsoft 365
- Not applicable
- type: textarea
id: reproduce
attributes:
@@ -78,6 +113,15 @@ body:
prowler --version
validations:
required: true
- type: input
id: python-version
attributes:
label: Python version
description: Which Python version are you using?
placeholder: |-
python --version
validations:
required: true
- type: input
id: pip-version
attributes:
+10
View File
@@ -1 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: 📖 Documentation
url: https://docs.prowler.com
about: Check our comprehensive documentation for guides and tutorials
- name: 💬 GitHub Discussions
url: https://github.com/prowler-cloud/prowler/discussions
about: Ask questions and discuss with the community
- name: 🌟 Prowler Community
url: https://goto.prowler.com/slack
about: Join our community for support and updates
@@ -3,6 +3,42 @@ description: Suggest an idea for this project
labels: ["feature-request", "status/needs-triage"]
body:
- type: checkboxes
id: search
attributes:
label: Feature search
options:
- label: I have searched the existing issues and this feature has not been requested yet
required: true
- type: dropdown
id: component
attributes:
label: Which component would this feature affect?
multiple: true
options:
- Prowler CLI/SDK
- Prowler API
- Prowler UI
- Prowler Dashboard
- Prowler MCP Server
- Documentation
- New component/Integration
validations:
required: true
- type: dropdown
id: provider
attributes:
label: Related to specific cloud provider?
multiple: true
options:
- AWS
- Azure
- GCP
- Kubernetes
- GitHub
- Microsoft 365
- All providers
- Not provider-specific
- type: textarea
id: Problem
attributes:
@@ -19,6 +55,14 @@ body:
description: A clear and concise description of what you want to happen.
validations:
required: true
- type: textarea
id: use-case
attributes:
label: Use case and benefits
description: Who would benefit from this feature and how?
placeholder: This would help security teams by...
validations:
required: true
- type: textarea
id: Alternatives
attributes:
@@ -0,0 +1,71 @@
name: 'Setup Python with Poetry'
description: 'Setup Python environment with Poetry and install dependencies'
author: 'Prowler'
inputs:
python-version:
description: 'Python version to use'
required: true
working-directory:
description: 'Working directory for Poetry'
required: false
default: '.'
poetry-version:
description: 'Poetry version to install'
required: false
default: '2.1.1'
install-dependencies:
description: 'Install Python dependencies with Poetry'
required: false
default: 'true'
runs:
using: 'composite'
steps:
- name: Replace @master with current branch in pyproject.toml
if: github.event_name == 'pull_request' && github.base_ref == 'master'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
echo "Using branch: $BRANCH_NAME"
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
- name: Install poetry
shell: bash
run: |
python -m pip install --upgrade pip
pipx install poetry==${{ inputs.poetry-version }}
- name: Update SDK resolved_reference to latest commit
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
echo "Latest commit hash: $LATEST_COMMIT"
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
}' poetry.lock
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock
shell: bash
working-directory: ${{ inputs.working-directory }}
run: poetry lock
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ inputs.python-version }}
cache: 'poetry'
cache-dependency-path: ${{ inputs.working-directory }}/poetry.lock
- name: Install Python dependencies
if: inputs.install-dependencies == 'true'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
poetry install --no-root
poetry run pip list
+152
View File
@@ -0,0 +1,152 @@
name: 'Container Security Scan with Trivy'
description: 'Scans container images for vulnerabilities using Trivy and reports results'
author: 'Prowler'
inputs:
image-name:
description: 'Container image name to scan'
required: true
image-tag:
description: 'Container image tag to scan'
required: true
default: ${{ github.sha }}
severity:
description: 'Severities to scan for (comma-separated)'
required: false
default: 'CRITICAL,HIGH,MEDIUM,LOW'
fail-on-critical:
description: 'Fail the build if critical vulnerabilities are found'
required: false
default: 'false'
upload-sarif:
description: 'Upload results to GitHub Security tab'
required: false
default: 'true'
create-pr-comment:
description: 'Create a comment on the PR with scan results'
required: false
default: 'true'
artifact-retention-days:
description: 'Days to retain the Trivy report artifact'
required: false
default: '2'
outputs:
critical-count:
description: 'Number of critical vulnerabilities found'
value: ${{ steps.security-check.outputs.critical }}
high-count:
description: 'Number of high vulnerabilities found'
value: ${{ steps.security-check.outputs.high }}
total-count:
description: 'Total number of vulnerabilities found'
value: ${{ steps.security-check.outputs.total }}
runs:
using: 'composite'
steps:
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true'
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '0'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true'
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
sarif_file: 'trivy-results.sarif'
category: 'trivy-container'
- name: Run Trivy vulnerability scan (JSON)
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'json'
output: 'trivy-report.json'
severity: ${{ inputs.severity }}
exit-code: '0'
- name: Upload Trivy report artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: always()
with:
name: trivy-scan-report-${{ inputs.image-name }}
path: trivy-report.json
retention-days: ${{ inputs.artifact-retention-days }}
- name: Generate security summary
id: security-check
shell: bash
run: |
CRITICAL=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="CRITICAL")] | length' trivy-report.json)
HIGH=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="HIGH")] | length' trivy-report.json)
TOTAL=$(jq '[.Results[]?.Vulnerabilities[]?] | length' trivy-report.json)
echo "critical=$CRITICAL" >> $GITHUB_OUTPUT
echo "high=$HIGH" >> $GITHUB_OUTPUT
echo "total=$TOTAL" >> $GITHUB_OUTPUT
echo "### 🔒 Container Security Scan" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${{ inputs.image-name }}:${{ inputs.image-tag }}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- 🔴 Critical: $CRITICAL" >> $GITHUB_STEP_SUMMARY
echo "- 🟠 High: $HIGH" >> $GITHUB_STEP_SUMMARY
echo "- **Total**: $TOTAL" >> $GITHUB_STEP_SUMMARY
- name: Comment scan results on PR
if: inputs.create-pr-comment == 'true' && github.event_name == 'pull_request'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
IMAGE_NAME: ${{ inputs.image-name }}
GITHUB_SHA: ${{ inputs.image-tag }}
SEVERITY: ${{ inputs.severity }}
with:
script: |
const comment = require('./.github/scripts/trivy-pr-comment.js');
// Unique identifier to find our comment
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
const body = marker + '\n' + comment;
// Find existing comment
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existingComment = comments.find(c => c.body?.includes(marker));
if (existingComment) {
// Update existing comment
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existingComment.id,
body: body
});
console.log('✅ Updated existing Trivy scan comment');
} else {
// Create new comment
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: body
});
console.log('✅ Created new Trivy scan comment');
}
- name: Check for critical vulnerabilities
if: inputs.fail-on-critical == 'true' && steps.security-check.outputs.critical != '0'
shell: bash
run: |
echo "::error::Found ${{ steps.security-check.outputs.critical }} critical vulnerabilities"
echo "::warning::Please update packages or use a different base image"
exit 1
+11 -2
View File
@@ -1,3 +1,12 @@
name: "API - CodeQL Config"
name: 'API: CodeQL Config'
paths:
- "api/"
- 'api/'
paths-ignore:
- 'api/tests/**'
- 'api/**/__pycache__/**'
- 'api/**/migrations/**'
- 'api/**/*.md'
queries:
- uses: security-and-quality
+5
View File
@@ -42,6 +42,11 @@ provider/mongodbatlas:
- any-glob-to-any-file: "prowler/providers/mongodbatlas/**"
- any-glob-to-any-file: "tests/providers/mongodbatlas/**"
provider/oci:
- changed-files:
- any-glob-to-any-file: "prowler/providers/oraclecloud/**"
- any-glob-to-any-file: "tests/providers/oraclecloud/**"
github_actions:
- changed-files:
- any-glob-to-any-file: ".github/workflows/*"
+102
View File
@@ -0,0 +1,102 @@
const fs = require('fs');
// Configuration from environment variables
const REPORT_FILE = process.env.TRIVY_REPORT_FILE || 'trivy-report.json';
const IMAGE_NAME = process.env.IMAGE_NAME || 'container-image';
const GITHUB_SHA = process.env.GITHUB_SHA || 'unknown';
const GITHUB_REPOSITORY = process.env.GITHUB_REPOSITORY || '';
const GITHUB_RUN_ID = process.env.GITHUB_RUN_ID || '';
const SEVERITY = process.env.SEVERITY || 'CRITICAL,HIGH,MEDIUM,LOW';
// Parse severities to scan
const scannedSeverities = SEVERITY.split(',').map(s => s.trim());
// Read and parse the Trivy report
const report = JSON.parse(fs.readFileSync(REPORT_FILE, 'utf-8'));
let vulnCount = 0;
let vulnsByType = { CRITICAL: 0, HIGH: 0, MEDIUM: 0, LOW: 0 };
let affectedPackages = new Set();
if (report.Results && Array.isArray(report.Results)) {
for (const result of report.Results) {
if (result.Vulnerabilities && Array.isArray(result.Vulnerabilities)) {
for (const vuln of result.Vulnerabilities) {
vulnCount++;
if (vulnsByType[vuln.Severity] !== undefined) {
vulnsByType[vuln.Severity]++;
}
if (vuln.PkgName) {
affectedPackages.add(vuln.PkgName);
}
}
}
}
}
const shortSha = GITHUB_SHA.substring(0, 7);
const timestamp = new Date().toISOString().replace('T', ' ').substring(0, 19) + ' UTC';
// Severity icons and labels
const severityConfig = {
CRITICAL: { icon: '🔴', label: 'Critical' },
HIGH: { icon: '🟠', label: 'High' },
MEDIUM: { icon: '🟡', label: 'Medium' },
LOW: { icon: '🔵', label: 'Low' }
};
let comment = '## 🔒 Container Security Scan\n\n';
comment += `**Image:** \`${IMAGE_NAME}:${shortSha}\`\n`;
comment += `**Last scan:** ${timestamp}\n\n`;
if (vulnCount === 0) {
comment += '### ✅ No Vulnerabilities Detected\n\n';
comment += 'The container image passed all security checks. No known CVEs were found.\n';
} else {
comment += '### 📊 Vulnerability Summary\n\n';
comment += '| Severity | Count |\n';
comment += '|----------|-------|\n';
// Only show severities that were scanned
for (const severity of scannedSeverities) {
const config = severityConfig[severity];
const count = vulnsByType[severity] || 0;
const isBold = (severity === 'CRITICAL' || severity === 'HIGH') && count > 0;
const countDisplay = isBold ? `**${count}**` : count;
comment += `| ${config.icon} ${config.label} | ${countDisplay} |\n`;
}
comment += `| **Total** | **${vulnCount}** |\n\n`;
if (affectedPackages.size > 0) {
comment += `**${affectedPackages.size}** package(s) affected\n\n`;
}
if (vulnsByType.CRITICAL > 0) {
comment += '### ⚠️ Action Required\n\n';
comment += '**Critical severity vulnerabilities detected.** These should be addressed before merging:\n';
comment += '- Review the detailed scan results\n';
comment += '- Update affected packages to patched versions\n';
comment += '- Consider using a different base image if updates are unavailable\n\n';
} else if (vulnsByType.HIGH > 0) {
comment += '### ⚠️ Attention Needed\n\n';
comment += '**High severity vulnerabilities found.** Please review and plan remediation:\n';
comment += '- Assess the risk and exploitability\n';
comment += '- Prioritize updates in the next maintenance cycle\n\n';
} else {
comment += '### ️ Review Recommended\n\n';
comment += 'Medium/Low severity vulnerabilities found. Consider addressing during regular maintenance.\n\n';
}
}
comment += '---\n';
comment += '📋 **Resources:**\n';
if (GITHUB_REPOSITORY && GITHUB_RUN_ID) {
comment += `- [Download full report](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}) (see artifacts)\n`;
}
comment += '- [View in Security tab](https://github.com/' + (GITHUB_REPOSITORY || 'repository') + '/security/code-scanning)\n';
comment += '- Scanned with [Trivy](https://github.com/aquasecurity/trivy)\n';
module.exports = comment;
+30 -33
View File
@@ -1,36 +1,34 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: API - CodeQL
name: 'API: CodeQL'
on:
push:
branches:
- "master"
- "v5.*"
- 'master'
- 'v5.*'
paths:
- "api/**"
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
pull_request:
branches:
- "master"
- "v5.*"
- 'master'
- 'v5.*'
paths:
- "api/**"
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
schedule:
- cron: '00 12 * * *'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
analyze:
name: Analyze
name: CodeQL Security Analysis
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
actions: read
contents: read
@@ -39,21 +37,20 @@ jobs:
strategy:
fail-fast: false
matrix:
language: [ 'python' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
language:
- 'python'
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
category: "/language:${{matrix.language}}"
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
category: '/language:${{ matrix.language }}'
+148 -153
View File
@@ -1,20 +1,30 @@
name: API - Pull Request
name: 'API: Pull Request'
on:
push:
branches:
- "master"
- "v5.*"
- 'master'
- 'v5.*'
paths:
- ".github/workflows/api-pull-request.yml"
- "api/**"
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
pull_request:
branches:
- "master"
- "v5.*"
- 'master'
- 'v5.*'
paths:
- ".github/workflows/api-pull-request.yml"
- "api/**"
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
POSTGRES_HOST: localhost
@@ -29,21 +39,94 @@ env:
VALKEY_DB: 0
API_WORKING_DIR: ./api
IMAGE_NAME: prowler-api
IGNORE_FILES: |
api/docs/**
api/README.md
api/CHANGELOG.md
jobs:
test:
code-quality:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version: ["3.12"]
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Poetry check
run: poetry check --lock
- name: Ruff lint
run: poetry run ruff check . --exclude contrib
- name: Ruff format
run: poetry run ruff format --check . --exclude contrib
- name: Pylint
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Bandit
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
tests:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
# Service containers to run with `test`
services:
# Label used to access the service container
postgres:
image: postgres
env:
@@ -52,7 +135,6 @@ jobs:
POSTGRES_USER: ${{ env.POSTGRES_USER }}
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_DB: ${{ env.POSTGRES_DB }}
# Set health checks to wait until postgres has started
ports:
- 5432:5432
options: >-
@@ -66,7 +148,6 @@ jobs:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
VALKEY_DB: ${{ env.VALKEY_DB }}
# Set health checks to wait until postgres has started
ports:
- 6379:6379
options: >-
@@ -76,158 +157,72 @@ jobs:
--health-retries 5
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Test if changes are in not ignored paths
id: are-non-ignored-files-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
.github/workflows/api-pull-request.yml
files_ignore: ${{ env.IGNORE_FILES }}
- name: Replace @master with current branch in pyproject.toml - Only for pull requests to `master`
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'pull_request' && github.base_ref == 'master'
run: |
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
echo "Using branch: $BRANCH_NAME"
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
- name: Install poetry
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
python -m pip install --upgrade pip
pipx install poetry==2.1.1
- name: Update SDK's poetry.lock resolved_reference to latest commit - Only for push events to `master`
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/master'
run: |
# Get the latest commit hash from the prowler-cloud/prowler repository
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
echo "Latest commit hash: $LATEST_COMMIT"
# Update the resolved_reference specifically for prowler-cloud/prowler repository
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
}' poetry.lock
# Verify the change was made
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry lock
- name: Set up Python ${{ matrix.python-version }}
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"
working-directory: ./api
- name: Install dependencies
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry install --no-root
poetry run pip list
VERSION=$(curl --silent "https://api.github.com/repos/hadolint/hadolint/releases/latest" | \
grep '"tag_name":' | \
sed -E 's/.*"v([^"]+)".*/\1/' \
) && curl -L -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/v${VERSION}/hadolint-Linux-x86_64" \
&& chmod +x /tmp/hadolint
- name: Poetry check
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry check --lock
- name: Lint with ruff
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run ruff check . --exclude contrib
- name: Check Format with ruff
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run ruff format --check . --exclude contrib
- name: Lint with pylint
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
- name: Bandit
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: |
poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
- name: Hadolint
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
/tmp/hadolint Dockerfile --ignore=DL3013
- name: Test with pytest
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Run tests with pytest
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Upload coverage reports to Codecov
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: api
test-container-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Test if changes are in not ignored paths
id: are-non-ignored-files-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
dockerfile-lint:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Lint Dockerfile with Hadolint
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
files: api/**
files_ignore: ${{ env.IGNORE_FILES }}
dockerfile: api/Dockerfile
ignore: DL3013
container-build-and-scan:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up Docker Buildx
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build Container
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
- name: Build container
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.API_WORKING_DIR }}
push: false
tags: ${{ env.IMAGE_NAME }}:latest
outputs: type=docker
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan container with Trivy
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
+22 -15
View File
@@ -1,28 +1,35 @@
name: Prowler - Automatic Backport
name: 'Tools: Backport'
on:
pull_request_target:
branches: ['master']
types: ['labeled', 'closed']
branches:
- 'master'
types:
- 'labeled'
- 'closed'
paths:
- '.github/workflows/backport.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: false
env:
# The prefix of the label that triggers the backport must not contain the branch name
# so, for example, if the branch is 'master', the label should be 'backport-to-<branch>'
BACKPORT_LABEL_PREFIX: backport-to-
BACKPORT_LABEL_IGNORE: was-backported
jobs:
backport:
name: Backport PR
if: github.event.pull_request.merged == true && !(contains(github.event.pull_request.labels.*.name, 'backport')) && !(contains(github.event.pull_request.labels.*.name, 'was-backported'))
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
id-token: write
pull-requests: write
contents: write
pull-requests: write
steps:
- name: Check labels
id: preview_label_check
id: label_check
uses: agilepathway/label-checker@c3d16ad512e7cea5961df85ff2486bb774caf3c5 # v1.6.65
with:
allow_failure: true
@@ -31,17 +38,17 @@ jobs:
none_of: ${{ env.BACKPORT_LABEL_IGNORE }}
repo_token: ${{ secrets.GITHUB_TOKEN }}
- name: Backport Action
if: steps.preview_label_check.outputs.label_check == 'success'
- name: Backport PR
if: steps.label_check.outputs.label_check == 'success'
uses: sorenlouv/backport-github-action@ad888e978060bc1b2798690dd9d03c4036560947 # v9.5.1
with:
github_token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
auto_backport_label_prefix: ${{ env.BACKPORT_LABEL_PREFIX }}
- name: Info log
if: ${{ success() && steps.preview_label_check.outputs.label_check == 'success' }}
- name: Display backport info log
if: success() && steps.label_check.outputs.label_check == 'success'
run: cat ~/.backport/backport.info.log
- name: Debug log
if: ${{ failure() && steps.preview_label_check.outputs.label_check == 'success' }}
- name: Display backport debug log
if: failure() && steps.label_check.outputs.label_check == 'success'
run: cat ~/.backport/backport.debug.log
@@ -1,36 +0,0 @@
name: Prowler - Pull Request Documentation Link
on:
pull_request:
branches:
- 'master'
- 'v3'
paths:
- 'docs/**'
- '.github/workflows/build-documentation-on-pr.yml'
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
jobs:
documentation-link:
name: Documentation Link
runs-on: ubuntu-latest
steps:
- name: Find existing documentation comment
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-comment
with:
issue-number: ${{ env.PR_NUMBER }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- prowler-docs-link -->'
- name: Create or update PR comment with the Prowler Documentation URI
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ env.PR_NUMBER }}
body: |
<!-- prowler-docs-link -->
You can check the documentation for this PR here -> [Prowler Documentation](https://prowler-prowler-docs--${{ env.PR_NUMBER }}.com.readthedocs.build/projects/prowler-open-source/en/${{ env.PR_NUMBER }}/)
edit-mode: replace
+20 -12
View File
@@ -1,23 +1,31 @@
name: Prowler - Conventional Commit
name: 'Tools: Conventional Commit'
on:
pull_request:
types:
- "opened"
- "edited"
- "synchronize"
branches:
- "master"
- "v3"
- "v4.*"
- "v5.*"
- 'master'
- 'v3'
- 'v4.*'
- 'v5.*'
types:
- 'opened'
- 'edited'
- 'synchronize'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
conventional-commit-check:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: read
steps:
- name: conventional-commit-check
id: conventional-commit-check
- name: Check PR title format
uses: agenthunt/conventional-commit-checker-action@9e552d650d0e205553ec7792d447929fc78e012b # v2.0.0
with:
pr-title-regex: '^([^\s(]+)(?:\(([^)]+)\))?: (.+)'
pr-title-regex: '^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\([^)]+\))?!?: .+'
+49 -46
View File
@@ -1,67 +1,70 @@
name: Prowler - Create Backport Label
name: 'Tools: Backport Label'
on:
release:
types: [published]
types:
- 'published'
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
cancel-in-progress: false
env:
BACKPORT_LABEL_PREFIX: backport-to-
BACKPORT_LABEL_COLOR: B60205
jobs:
create_label:
create-label:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
contents: read
issues: write
steps:
- name: Create backport label
- name: Create backport label for minor releases
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
OWNER_REPO: ${{ github.repository }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
VERSION_ONLY=${RELEASE_TAG#v} # Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
RELEASE_TAG="${{ github.event.release.tag_name }}"
if [ -z "$RELEASE_TAG" ]; then
echo "Error: No release tag provided"
exit 1
fi
echo "Processing release tag: $RELEASE_TAG"
# Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
VERSION_ONLY="${RELEASE_TAG#v}"
# Check if it's a minor version (X.Y.0)
if [[ "$VERSION_ONLY" =~ ^[0-9]+\.[0-9]+\.0$ ]]; then
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is a minor version. Proceeding to create backport label."
if [[ "$VERSION_ONLY" =~ ^([0-9]+)\.([0-9]+)\.0$ ]]; then
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is a minor version. Proceeding to create backport label."
TWO_DIGIT_VERSION=${VERSION_ONLY%.0} # Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
# Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
MAJOR="${BASH_REMATCH[1]}"
MINOR="${BASH_REMATCH[2]}"
TWO_DIGIT_VERSION="${MAJOR}.${MINOR}"
FINAL_LABEL_NAME="backport-to-v${TWO_DIGIT_VERSION}"
FINAL_DESCRIPTION="Backport PR to the v${TWO_DIGIT_VERSION} branch"
LABEL_NAME="${BACKPORT_LABEL_PREFIX}v${TWO_DIGIT_VERSION}"
LABEL_DESC="Backport PR to the v${TWO_DIGIT_VERSION} branch"
LABEL_COLOR="$BACKPORT_LABEL_COLOR"
echo "Effective label name will be: ${FINAL_LABEL_NAME}"
echo "Effective description will be: ${FINAL_DESCRIPTION}"
echo "Label name: $LABEL_NAME"
echo "Label description: $LABEL_DESC"
# Check if the label already exists
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER_REPO}/labels/${FINAL_LABEL_NAME}")
if [ "${STATUS_CODE}" -eq 200 ]; then
echo "Label '${FINAL_LABEL_NAME}' already exists."
elif [ "${STATUS_CODE}" -eq 404 ]; then
echo "Label '${FINAL_LABEL_NAME}' does not exist. Creating it..."
# Prepare JSON data payload
JSON_DATA=$(printf '{"name":"%s","description":"%s","color":"B60205"}' "${FINAL_LABEL_NAME}" "${FINAL_DESCRIPTION}")
CREATE_STATUS_CODE=$(curl -s -o /tmp/curl_create_response.json -w "%{http_code}" -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token ${GITHUB_TOKEN}" \
--data "${JSON_DATA}" \
"https://api.github.com/repos/${OWNER_REPO}/labels")
CREATE_RESPONSE_BODY=$(cat /tmp/curl_create_response.json)
rm -f /tmp/curl_create_response.json
if [ "$CREATE_STATUS_CODE" -eq 201 ]; then
echo "Label '${FINAL_LABEL_NAME}' created successfully."
else
echo "Error creating label '${FINAL_LABEL_NAME}'. Status: $CREATE_STATUS_CODE"
echo "Response: $CREATE_RESPONSE_BODY"
exit 1
fi
# Check if label already exists
if gh label list --repo ${{ github.repository }} --limit 1000 | grep -q "^${LABEL_NAME}[[:space:]]"; then
echo "Label '$LABEL_NAME' already exists."
else
echo "Error checking for label '${FINAL_LABEL_NAME}'. HTTP Status: ${STATUS_CODE}"
exit 1
echo "Label '$LABEL_NAME' does not exist. Creating it..."
gh label create "$LABEL_NAME" \
--description "$LABEL_DESC" \
--color "$LABEL_COLOR" \
--repo ${{ github.repository }}
echo "Label '$LABEL_NAME' created successfully."
fi
else
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is not a minor version. Skipping backport label creation."
exit 0
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is not a minor version. Skipping backport label creation."
fi
+1 -1
View File
@@ -1,4 +1,4 @@
name: Prowler - Find secrets
name: 'Tools: TruffleHog'
on: pull_request
@@ -0,0 +1,113 @@
name: 'MCP: Container Build and Push'
on:
push:
branches:
- "master"
paths:
- "mcp_server/**"
- ".github/workflows/mcp-container-build-push.yml"
# Uncomment to test this workflow on PRs
# pull_request:
# branches:
# - "master"
# paths:
# - "mcp_server/**"
# - ".github/workflows/mcp-container-build-push.yml"
release:
types: [published]
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./mcp_server
# Container registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-mcp
jobs:
setup:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
outputs:
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
steps:
- name: Calculate short SHA
id: set-short-sha
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
container-build-push:
needs: setup
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push container (latest)
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.head_commit.timestamp }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push container (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.release.published_at }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Trigger deployment
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: mcp-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
@@ -76,10 +76,12 @@ jobs:
UI_VERSION=$(extract_latest_version "ui/CHANGELOG.md")
API_VERSION=$(extract_latest_version "api/CHANGELOG.md")
SDK_VERSION=$(extract_latest_version "prowler/CHANGELOG.md")
MCP_VERSION=$(extract_latest_version "mcp_server/CHANGELOG.md")
echo "UI_VERSION=${UI_VERSION}" >> "${GITHUB_ENV}"
echo "API_VERSION=${API_VERSION}" >> "${GITHUB_ENV}"
echo "SDK_VERSION=${SDK_VERSION}" >> "${GITHUB_ENV}"
echo "MCP_VERSION=${MCP_VERSION}" >> "${GITHUB_ENV}"
if [ -n "$UI_VERSION" ]; then
echo "Read UI version from changelog: $UI_VERSION"
@@ -99,11 +101,18 @@ jobs:
echo "Warning: No SDK version found in prowler/CHANGELOG.md"
fi
if [ -n "$MCP_VERSION" ]; then
echo "Read MCP version from changelog: $MCP_VERSION"
else
echo "Warning: No MCP version found in mcp_server/CHANGELOG.md"
fi
echo "Prowler version: $PROWLER_VERSION"
echo "Branch name: $BRANCH_NAME"
echo "UI version: $UI_VERSION"
echo "API version: $API_VERSION"
echo "SDK version: $SDK_VERSION"
echo "MCP version: $MCP_VERSION"
echo "Is minor release: $([ $PATCH_VERSION -eq 0 ] && echo 'true' || echo 'false')"
else
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
@@ -183,7 +192,26 @@ jobs:
touch "prowler_changelog.md"
fi
# Combine changelogs in order: UI, API, SDK
# MCP has changes if the changelog references this Prowler version
# Check if the changelog contains "(Prowler X.Y.Z)" or "(Prowler UNRELEASED)"
if [ -f "mcp_server/CHANGELOG.md" ]; then
MCP_PROWLER_REF=$(grep -m 1 "^## \[.*\] (Prowler" mcp_server/CHANGELOG.md | sed -E 's/.*\(Prowler ([^)]+)\).*/\1/' | tr -d '[:space:]')
if [ "$MCP_PROWLER_REF" = "$PROWLER_VERSION" ] || [ "$MCP_PROWLER_REF" = "UNRELEASED" ]; then
echo "HAS_MCP_CHANGES=true" >> $GITHUB_ENV
echo "✓ MCP changes detected - Prowler reference: $MCP_PROWLER_REF (version: $MCP_VERSION)"
extract_changelog "mcp_server/CHANGELOG.md" "$MCP_VERSION" "mcp_changelog.md"
else
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
echo " No MCP changes for this release (Prowler reference: $MCP_PROWLER_REF, input: $PROWLER_VERSION)"
touch "mcp_changelog.md"
fi
else
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
echo " No MCP changelog found"
touch "mcp_changelog.md"
fi
# Combine changelogs in order: UI, API, SDK, MCP
> combined_changelog.md
if [ "$HAS_UI_CHANGES" = "true" ] && [ -s "ui_changelog.md" ]; then
@@ -207,6 +235,13 @@ jobs:
echo "" >> combined_changelog.md
fi
if [ "$HAS_MCP_CHANGES" = "true" ] && [ -s "mcp_changelog.md" ]; then
echo "## MCP" >> combined_changelog.md
echo "" >> combined_changelog.md
cat mcp_changelog.md >> combined_changelog.md
echo "" >> combined_changelog.md
fi
echo "Combined changelog preview:"
cat combined_changelog.md
@@ -367,4 +402,4 @@ jobs:
- name: Clean up temporary files
run: |
rm -f prowler_changelog.md api_changelog.md ui_changelog.md combined_changelog.md
rm -f prowler_changelog.md api_changelog.md ui_changelog.md mcp_changelog.md combined_changelog.md
@@ -13,7 +13,7 @@ jobs:
contents: read
pull-requests: write
env:
MONITORED_FOLDERS: "api ui prowler dashboard"
MONITORED_FOLDERS: "api ui prowler mcp_server"
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+16 -1
View File
@@ -249,6 +249,21 @@ jobs:
run: |
poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodb_atlas_coverage.xml tests/providers/mongodbatlas
# Test OCI
- name: OCI - Check if any file has changed
id: oci-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/oraclecloud/**
./tests/providers/oraclecloud/**
./poetry.lock
- name: OCI - Test
if: steps.oci-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oci_coverage.xml tests/providers/oraclecloud
# Common Tests
- name: Lib - Test
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
@@ -268,4 +283,4 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./lib_coverage.xml,./config_coverage.xml
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./oci_coverage.xml,./lib_coverage.xml,./config_coverage.xml
+4 -3
View File
@@ -82,12 +82,13 @@ prowler dashboard
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Stage | Interface |
|---|---|---|---|---|---|---|---|
| AWS | 576 | 82 | 36 | 10 | Official | Stable | UI, API, CLI |
| GCP | 79 | 13 | 10 | 3 | Official | Stable | UI, API, CLI |
| Azure | 162 | 19 | 11 | 4 | Official | Stable | UI, API, CLI |
| AWS | 576 | 82 | 38 | 10 | Official | Stable | UI, API, CLI |
| GCP | 79 | 13 | 11 | 3 | Official | Stable | UI, API, CLI |
| Azure | 162 | 19 | 12 | 4 | Official | Stable | UI, API, CLI |
| Kubernetes | 83 | 7 | 5 | 7 | Official | Stable | UI, API, CLI |
| GitHub | 17 | 2 | 1 | 0 | Official | Stable | UI, API, CLI |
| M365 | 70 | 7 | 3 | 2 | Official | Stable | UI, API, CLI |
| OCI | 51 | 13 | 1 | 10 | Official | Stable | CLI |
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | Beta | CLI |
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | Beta | CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | Beta | CLI |
+7
View File
@@ -7,7 +7,14 @@ All notable changes to the **Prowler API** are documented in this file.
### Added
- Default JWT keys are generated and stored if they are missing from configuration [(#8655)](https://github.com/prowler-cloud/prowler/pull/8655)
- `compliance_name` for each compliance [(#7920)](https://github.com/prowler-cloud/prowler/pull/7920)
- Support C5 compliance framework for the AWS provider [(#8830)](https://github.com/prowler-cloud/prowler/pull/8830)
- Support for M365 Certificate authentication [(#8538)](https://github.com/prowler-cloud/prowler/pull/8538)
- API Key support [(#8805)](https://github.com/prowler-cloud/prowler/pull/8805)
- SAML role mapping protection for single-admin tenants to prevent accidental lockout [(#8882)](https://github.com/prowler-cloud/prowler/pull/8882)
- Support for `passed_findings` and `total_findings` fields in compliance requirement overview for accurate Prowler ThreatScore calculation [(#8582)](https://github.com/prowler-cloud/prowler/pull/8582)
- Database read replica support [(#8869)](https://github.com/prowler-cloud/prowler/pull/8869)
- Support Common Cloud Controls for AWS, Azure and GCP [(#8000)](https://github.com/prowler-cloud/prowler/pull/8000)
- Add `provider_id__in` filter support to findings and findings severity overview endpoints [(#8951)](https://github.com/prowler-cloud/prowler/pull/8951)
### Changed
- Now the MANAGE_ACCOUNT permission is required to modify or read user permissions instead of MANAGE_USERS [(#8281)](https://github.com/prowler-cloud/prowler/pull/8281)
+4 -4
View File
@@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 2.1.4 and should not be changed by hand.
# This file is automatically @generated by Poetry 2.2.0 and should not be changed by hand.
[[package]]
name = "about-time"
@@ -273,14 +273,14 @@ tests-mypy = ["mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" a
[[package]]
name = "authlib"
version = "1.6.4"
version = "1.6.5"
description = "The ultimate Python library in building OAuth and OpenID Connect servers and clients."
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "authlib-1.6.4-py2.py3-none-any.whl", hash = "sha256:39313d2a2caac3ecf6d8f95fbebdfd30ae6ea6ae6a6db794d976405fdd9aa796"},
{file = "authlib-1.6.4.tar.gz", hash = "sha256:104b0442a43061dc8bc23b133d1d06a2b0a9c2e3e33f34c4338929e816287649"},
{file = "authlib-1.6.5-py2.py3-none-any.whl", hash = "sha256:3e0e0507807f842b02175507bdee8957a1d5707fd4afb17c32fb43fee90b6e3a"},
{file = "authlib-1.6.5.tar.gz", hash = "sha256:6aaf9c79b7cc96c900f0b284061691c5d4e61221640a948fe690b556a6d6d10b"},
]
[package.dependencies]
+21 -2
View File
@@ -10,6 +10,7 @@ from rest_framework.exceptions import AuthenticationFailed
from rest_framework.request import Request
from rest_framework_simplejwt.authentication import JWTAuthentication
from api.db_router import MainRouter
from api.models import TenantAPIKey, TenantAPIKeyManager
@@ -19,6 +20,22 @@ class TenantAPIKeyAuthentication(BaseAPIKeyAuth):
def __init__(self):
self.key_crypto = get_crypto()
def _authenticate_credentials(self, request, key):
"""
Override to use admin connection, bypassing RLS during authentication.
Delegates to parent after temporarily routing model queries to admin DB.
"""
# Temporarily point the model's manager to admin database
original_objects = self.model.objects
self.model.objects = self.model.objects.using(MainRouter.admin_db)
try:
# Call parent method which will now use admin database
return super()._authenticate_credentials(request, key)
finally:
# Restore original manager
self.model.objects = original_objects
def authenticate(self, request: Request):
prefixed_key = self.get_key(request)
@@ -43,13 +60,15 @@ class TenantAPIKeyAuthentication(BaseAPIKeyAuth):
api_key_pk = UUID(api_key_pk)
try:
api_key_instance = TenantAPIKey.objects.get(id=api_key_pk, prefix=prefix)
api_key_instance = TenantAPIKey.objects.using(MainRouter.admin_db).get(
id=api_key_pk, prefix=prefix
)
except TenantAPIKey.DoesNotExist:
raise AuthenticationFailed("Invalid API Key.")
# Update last_used_at
api_key_instance.last_used_at = timezone.now()
api_key_instance.save(update_fields=["last_used_at"])
api_key_instance.save(update_fields=["last_used_at"], using=MainRouter.admin_db)
return entity, {
"tenant_id": str(api_key_instance.tenant_id),
+83 -18
View File
@@ -1,13 +1,15 @@
from django.conf import settings
from django.core.exceptions import ObjectDoesNotExist
from django.db import transaction
from rest_framework import permissions
from rest_framework.exceptions import NotAuthenticated
from rest_framework.filters import SearchFilter
from rest_framework.permissions import SAFE_METHODS
from rest_framework_json_api import filters
from rest_framework_json_api.views import ModelViewSet
from api.authentication import CombinedJWTOrAPIKeyAuthentication
from api.db_router import MainRouter
from api.db_router import MainRouter, reset_read_db_alias, set_read_db_alias
from api.db_utils import POSTGRES_USER_VAR, rls_transaction
from api.filters import CustomDjangoFilterBackend
from api.models import Role, Tenant
@@ -31,6 +33,20 @@ class BaseViewSet(ModelViewSet):
ordering_fields = "__all__"
ordering = ["id"]
def _get_request_db_alias(self, request):
if request is None:
return MainRouter.default_db
read_alias = (
MainRouter.replica_db
if request.method in SAFE_METHODS
and MainRouter.replica_db in settings.DATABASES
else None
)
if read_alias:
return read_alias
return MainRouter.default_db
def initial(self, request, *args, **kwargs):
"""
Sets required_permissions before permissions are checked.
@@ -48,8 +64,21 @@ class BaseViewSet(ModelViewSet):
class BaseRLSViewSet(BaseViewSet):
def dispatch(self, request, *args, **kwargs):
with transaction.atomic():
return super().dispatch(request, *args, **kwargs)
self.db_alias = self._get_request_db_alias(request)
alias_token = None
try:
if self.db_alias != MainRouter.default_db:
alias_token = set_read_db_alias(self.db_alias)
if request is not None:
request.db_alias = self.db_alias
with transaction.atomic(using=self.db_alias):
return super().dispatch(request, *args, **kwargs)
finally:
if alias_token is not None:
reset_read_db_alias(alias_token)
self.db_alias = MainRouter.default_db
def initial(self, request, *args, **kwargs):
# Ideally, this logic would be in the `.setup()` method but DRF view sets don't call it
@@ -61,7 +90,9 @@ class BaseRLSViewSet(BaseViewSet):
if tenant_id is None:
raise NotAuthenticated("Tenant ID is not present in token")
with rls_transaction(tenant_id):
with rls_transaction(
tenant_id, using=getattr(self, "db_alias", MainRouter.default_db)
):
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
@@ -73,18 +104,33 @@ class BaseRLSViewSet(BaseViewSet):
class BaseTenantViewset(BaseViewSet):
def dispatch(self, request, *args, **kwargs):
with transaction.atomic():
tenant = super().dispatch(request, *args, **kwargs)
self.db_alias = self._get_request_db_alias(request)
alias_token = None
try:
# If the request is a POST, create the admin role
if request.method == "POST":
isinstance(tenant, dict) and self._create_admin_role(tenant.data["id"])
except Exception as e:
self._handle_creation_error(e, tenant)
raise
if self.db_alias != MainRouter.default_db:
alias_token = set_read_db_alias(self.db_alias)
return tenant
if request is not None:
request.db_alias = self.db_alias
with transaction.atomic(using=self.db_alias):
tenant = super().dispatch(request, *args, **kwargs)
try:
# If the request is a POST, create the admin role
if request.method == "POST":
isinstance(tenant, dict) and self._create_admin_role(
tenant.data["id"]
)
except Exception as e:
self._handle_creation_error(e, tenant)
raise
return tenant
finally:
if alias_token is not None:
reset_read_db_alias(alias_token)
self.db_alias = MainRouter.default_db
def _create_admin_role(self, tenant_id):
Role.objects.using(MainRouter.admin_db).create(
@@ -117,14 +163,31 @@ class BaseTenantViewset(BaseViewSet):
raise NotAuthenticated("Tenant ID is not present in token")
user_id = str(request.user.id)
with rls_transaction(value=user_id, parameter=POSTGRES_USER_VAR):
with rls_transaction(
value=user_id,
parameter=POSTGRES_USER_VAR,
using=getattr(self, "db_alias", MainRouter.default_db),
):
return super().initial(request, *args, **kwargs)
class BaseUserViewset(BaseViewSet):
def dispatch(self, request, *args, **kwargs):
with transaction.atomic():
return super().dispatch(request, *args, **kwargs)
self.db_alias = self._get_request_db_alias(request)
alias_token = None
try:
if self.db_alias != MainRouter.default_db:
alias_token = set_read_db_alias(self.db_alias)
if request is not None:
request.db_alias = self.db_alias
with transaction.atomic(using=self.db_alias):
return super().dispatch(request, *args, **kwargs)
finally:
if alias_token is not None:
reset_read_db_alias(alias_token)
self.db_alias = MainRouter.default_db
def initial(self, request, *args, **kwargs):
# TODO refactor after improving RLS on users
@@ -137,6 +200,8 @@ class BaseUserViewset(BaseViewSet):
if tenant_id is None:
raise NotAuthenticated("Tenant ID is not present in token")
with rls_transaction(tenant_id):
with rls_transaction(
tenant_id, using=getattr(self, "db_alias", MainRouter.default_db)
):
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
+10 -6
View File
@@ -150,12 +150,16 @@ def generate_scan_compliance(
requirement["checks"][check_id] = status
requirement["checks_status"][status.lower()] += 1
if requirement["status"] != "FAIL" and any(
value == "FAIL" for value in requirement["checks"].values()
):
requirement["status"] = "FAIL"
compliance_overview[compliance_id]["requirements_status"]["passed"] -= 1
compliance_overview[compliance_id]["requirements_status"]["failed"] += 1
if requirement["status"] != "FAIL" and any(
value == "FAIL" for value in requirement["checks"].values()
):
requirement["status"] = "FAIL"
compliance_overview[compliance_id]["requirements_status"][
"passed"
] -= 1
compliance_overview[compliance_id]["requirements_status"][
"failed"
] += 1
def generate_compliance_overview_template(prowler_compliance: dict):
+30
View File
@@ -1,9 +1,31 @@
from contextvars import ContextVar
from django.conf import settings
ALLOWED_APPS = ("django", "socialaccount", "account", "authtoken", "silk")
_read_db_alias = ContextVar("read_db_alias", default=None)
def set_read_db_alias(alias: str | None):
if not alias:
return None
return _read_db_alias.set(alias)
def get_read_db_alias() -> str | None:
return _read_db_alias.get()
def reset_read_db_alias(token) -> None:
if token is not None:
_read_db_alias.reset(token)
class MainRouter:
default_db = "default"
admin_db = "admin"
replica_db = "replica"
def db_for_read(self, model, **hints): # noqa: F841
model_table_name = model._meta.db_table
@@ -11,6 +33,9 @@ class MainRouter:
model_table_name.startswith(f"{app}_") for app in ALLOWED_APPS
):
return self.admin_db
read_alias = get_read_db_alias()
if read_alias:
return read_alias
return None
def db_for_write(self, model, **hints): # noqa: F841
@@ -27,3 +52,8 @@ class MainRouter:
if {obj1._state.db, obj2._state.db} <= {self.default_db, self.admin_db}:
return True
return None
READ_REPLICA_ALIAS = (
MainRouter.replica_db if MainRouter.replica_db in settings.DATABASES else None
)
+33 -11
View File
@@ -6,12 +6,14 @@ from datetime import datetime, timedelta, timezone
from django.conf import settings
from django.contrib.auth.models import BaseUserManager
from django.db import connection, models, transaction
from django.db import DEFAULT_DB_ALIAS, connection, connections, models, transaction
from django_celery_beat.models import PeriodicTask
from psycopg2 import connect as psycopg2_connect
from psycopg2.extensions import AsIs, new_type, register_adapter, register_type
from rest_framework_json_api.serializers import ValidationError
from api.db_router import get_read_db_alias, reset_read_db_alias, set_read_db_alias
DB_USER = settings.DATABASES["default"]["USER"] if not settings.TESTING else "test"
DB_PASSWORD = (
settings.DATABASES["default"]["PASSWORD"] if not settings.TESTING else "test"
@@ -49,7 +51,11 @@ def psycopg_connection(database_alias: str):
@contextmanager
def rls_transaction(value: str, parameter: str = POSTGRES_TENANT_VAR):
def rls_transaction(
value: str,
parameter: str = POSTGRES_TENANT_VAR,
using: str | None = None,
):
"""
Creates a new database transaction setting the given configuration value for Postgres RLS. It validates the
if the value is a valid UUID.
@@ -57,16 +63,32 @@ def rls_transaction(value: str, parameter: str = POSTGRES_TENANT_VAR):
Args:
value (str): Database configuration parameter value.
parameter (str): Database configuration parameter name, by default is 'api.tenant_id'.
using (str | None): Optional database alias to run the transaction against. Defaults to the
active read alias (if any) or Django's default connection.
"""
with transaction.atomic():
with connection.cursor() as cursor:
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yield cursor
requested_alias = using or get_read_db_alias()
db_alias = requested_alias or DEFAULT_DB_ALIAS
if db_alias not in connections:
db_alias = DEFAULT_DB_ALIAS
router_token = None
try:
if db_alias != DEFAULT_DB_ALIAS:
router_token = set_read_db_alias(db_alias)
with transaction.atomic(using=db_alias):
conn = connections[db_alias]
with conn.cursor() as cursor:
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yield cursor
finally:
if router_token is not None:
reset_read_db_alias(router_token)
class CustomUserManager(BaseUserManager):
+1
View File
@@ -765,6 +765,7 @@ class ComplianceOverviewFilter(FilterSet):
class ScanSummaryFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
+14 -3
View File
@@ -2,6 +2,7 @@
import uuid
import django.core.validators
import django.db.models.deletion
import drf_simple_apikey.models
from django.conf import settings
@@ -20,7 +21,13 @@ class Migration(migrations.Migration):
migrations.CreateModel(
name="TenantAPIKey",
fields=[
("name", models.CharField(blank=True, max_length=255, null=True)),
(
"name",
models.CharField(
max_length=255,
validators=[django.core.validators.MinLengthValidator(3)],
),
),
(
"expiry_date",
models.DateTimeField(
@@ -37,7 +44,7 @@ class Migration(migrations.Migration):
help_text="If the API key is revoked, entities cannot use it anymore. (This cannot be undone.)",
),
),
("created", models.DateTimeField(auto_now=True)),
("created", models.DateTimeField(auto_now_add=True, editable=False)),
(
"whitelisted_ips",
models.JSONField(
@@ -110,7 +117,11 @@ class Migration(migrations.Migration):
"constraints": [
models.UniqueConstraint(
fields=("tenant_id", "prefix"), name="unique_api_key_prefixes"
)
),
models.UniqueConstraint(
fields=("tenant_id", "name"),
name="unique_api_key_name_per_tenant",
),
],
},
),
@@ -0,0 +1,21 @@
# Generated by Django 5.1.12 on 2025-10-07 10:41
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0048_api_key"),
]
operations = [
migrations.AddField(
model_name="compliancerequirementoverview",
name="passed_findings",
field=models.IntegerField(default=0),
),
migrations.AddField(
model_name="compliancerequirementoverview",
name="total_findings",
field=models.IntegerField(default=0),
),
]
+8
View File
@@ -220,6 +220,8 @@ class Membership(models.Model):
class TenantAPIKey(AbstractAPIKey, RowLevelSecurityProtectedModel):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
name = models.CharField(max_length=100, validators=[MinLengthValidator(3)])
created = models.DateTimeField(auto_now_add=True, editable=False)
prefix = models.CharField(
max_length=11,
unique=True,
@@ -255,6 +257,10 @@ class TenantAPIKey(AbstractAPIKey, RowLevelSecurityProtectedModel):
fields=("tenant_id", "prefix"),
name="unique_api_key_prefixes",
),
models.UniqueConstraint(
fields=("tenant_id", "name"),
name="unique_api_key_name_per_tenant",
),
]
indexes = [
@@ -1293,6 +1299,8 @@ class ComplianceRequirementOverview(RowLevelSecurityProtectedModel):
passed_checks = models.IntegerField(default=0)
failed_checks = models.IntegerField(default=0)
total_checks = models.IntegerField(default=0)
passed_findings = models.IntegerField(default=0)
total_findings = models.IntegerField(default=0)
scan = models.ForeignKey(
Scan,
+2 -1
View File
@@ -11,11 +11,12 @@ class APIJSONRenderer(JSONRenderer):
def render(self, data, accepted_media_type=None, renderer_context=None):
request = renderer_context.get("request")
tenant_id = getattr(request, "tenant_id", None) if request else None
db_alias = getattr(request, "db_alias", None) if request else None
include_param_present = "include" in request.query_params if request else False
# Use rls_transaction if needed for included resources, otherwise do nothing
context_manager = (
rls_transaction(tenant_id)
rls_transaction(tenant_id, using=db_alias)
if tenant_id and include_param_present
else nullcontext()
)
+173 -26
View File
@@ -86,6 +86,17 @@ paths:
description: A search term.
schema:
type: string
- in: query
name: include
schema:
type: array
items:
type: string
enum:
- entity
description: include query parameter to allow the client to customize which
related resources should be returned.
explode: false
- name: page[number]
required: false
in: query
@@ -186,6 +197,17 @@ paths:
format: uuid
description: A UUID string identifying this tenant api key.
required: true
- in: query
name: include
schema:
type: array
items:
type: string
enum:
- entity
description: include query parameter to allow the client to customize which
related resources should be returned.
explode: false
tags:
- API Keys
security:
@@ -3589,6 +3611,16 @@ paths:
schema:
type: string
format: uuid
- in: query
name: filter[provider_id__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_type]
schema:
@@ -3756,6 +3788,16 @@ paths:
schema:
type: string
format: uuid
- in: query
name: filter[provider_id__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_type]
schema:
@@ -3958,6 +4000,16 @@ paths:
schema:
type: string
format: uuid
- in: query
name: filter[provider_id__in]
schema:
type: array
items:
type: string
format: uuid
description: Multiple values may be separated by commas.
explode: false
style: form
- in: query
name: filter[provider_type]
schema:
@@ -4706,6 +4758,7 @@ paths:
type: array
items:
type: string
x-spec-enum-id: 4c1e219dad1cc0e7
enum:
- aws
- azure
@@ -11987,26 +12040,47 @@ components:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory
where the application is registered.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
user:
type: email
description: User microsoft email address.
deprecated: true
password:
type: string
description: User password.
deprecated: true
required:
- client_id
- client_secret
- tenant_id
- user
- password
- type: object
title: M365 Certificate Credentials
properties:
client_id:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory
where the application is registered.
certificate_content:
type: string
description: The certificate content in base64 format for
certificate-based authentication.
required:
- client_id
- tenant_id
- certificate_content
- type: object
title: GCP Static Credentials
properties:
@@ -12410,8 +12484,8 @@ components:
properties:
name:
type: string
nullable: true
maxLength: 255
minLength: 3
maxLength: 100
prefix:
type: string
readOnly: true
@@ -12431,6 +12505,8 @@ components:
readOnly: true
nullable: true
description: Last time this API key was used for authentication
required:
- name
relationships:
type: object
properties:
@@ -13851,26 +13927,47 @@ components:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory where
the application is registered.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
user:
type: email
description: User microsoft email address.
deprecated: true
password:
type: string
description: User password.
deprecated: true
required:
- client_id
- client_secret
- tenant_id
- user
- password
- type: object
title: M365 Certificate Credentials
properties:
client_id:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory where
the application is registered.
certificate_content:
type: string
description: The certificate content in base64 format for certificate-based
authentication.
required:
- client_id
- tenant_id
- certificate_content
- type: object
title: GCP Static Credentials
properties:
@@ -14099,26 +14196,47 @@ components:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory
where the application is registered.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
user:
type: email
description: User microsoft email address.
deprecated: true
password:
type: string
description: User password.
deprecated: true
required:
- client_id
- client_secret
- tenant_id
- user
- password
- type: object
title: M365 Certificate Credentials
properties:
client_id:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory
where the application is registered.
certificate_content:
type: string
description: The certificate content in base64 format for
certificate-based authentication.
required:
- client_id
- tenant_id
- certificate_content
- type: object
title: GCP Static Credentials
properties:
@@ -14363,26 +14481,47 @@ components:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory where
the application is registered.
client_secret:
type: string
description: The client secret associated with the application
(client) ID, providing secure access.
user:
type: email
description: User microsoft email address.
deprecated: true
password:
type: string
description: User password.
deprecated: true
required:
- client_id
- client_secret
- tenant_id
- user
- password
- type: object
title: M365 Certificate Credentials
properties:
client_id:
type: string
description: The Azure application (client) ID for authentication
in Azure AD.
tenant_id:
type: string
description: The Azure tenant ID, representing the directory where
the application is registered.
certificate_content:
type: string
description: The certificate content in base64 format for certificate-based
authentication.
required:
- client_id
- tenant_id
- certificate_content
- type: object
title: GCP Static Credentials
properties:
@@ -15636,8 +15775,8 @@ components:
properties:
name:
type: string
nullable: true
maxLength: 255
maxLength: 100
minLength: 3
prefix:
type: string
readOnly: true
@@ -15659,6 +15798,8 @@ components:
format: date-time
nullable: true
description: Last time this API key was used for authentication
required:
- name
relationships:
type: object
properties:
@@ -15705,8 +15846,8 @@ components:
properties:
name:
type: string
nullable: true
maxLength: 255
maxLength: 100
minLength: 3
prefix:
type: string
readOnly: true
@@ -15732,6 +15873,8 @@ components:
api_key:
type: string
readOnly: true
required:
- name
relationships:
type: object
properties:
@@ -15782,8 +15925,8 @@ components:
properties:
name:
type: string
nullable: true
maxLength: 255
minLength: 3
maxLength: 100
prefix:
type: string
readOnly: true
@@ -15810,6 +15953,8 @@ components:
api_key:
type: string
readOnly: true
required:
- name
relationships:
type: object
properties:
@@ -15877,8 +16022,8 @@ components:
properties:
name:
type: string
nullable: true
maxLength: 255
maxLength: 100
minLength: 3
prefix:
type: string
readOnly: true
@@ -15897,6 +16042,8 @@ components:
readOnly: true
nullable: true
description: Last time this API key was used for authentication
required:
- name
relationships:
type: object
properties:
@@ -1003,6 +1003,504 @@ class TestCombinedAuthentication:
assert api_key.last_used_at is not None
@pytest.mark.django_db
class TestAPIKeyRLSBypass:
"""Test RLS bypass fix for API key authentication.
These tests verify that API key authentication works correctly even when
RLS context is not set, which is critical since we don't know the tenant_id
until we look up the API key (which itself is protected by RLS).
The fix ensures all database operations during authentication use the admin
database, bypassing RLS constraints.
"""
def test_api_key_authentication_without_rls_context(
self, create_test_user, tenants_fixture, api_keys_fixture
):
"""Verify API key authentication works without pre-existing RLS context.
This is the core fix: authentication must succeed even when prowler.tenant_id
is not set, since we need to look up the API key to discover the tenant.
"""
client = APIClient()
api_key = api_keys_fixture[0]
api_key_headers = get_api_key_header(api_key._raw_key)
response = client.get(reverse("provider-list"), headers=api_key_headers)
assert response.status_code == 200
assert "data" in response.json()
def test_api_key_lookup_uses_admin_database(
self, create_test_user, tenants_fixture
):
"""Verify API key lookup uses admin database during authentication.
The TenantAPIKey model is RLS-protected, so queries against it would
normally fail without prowler.tenant_id set. The fix routes lookups
to the admin database which bypasses RLS.
"""
client = APIClient()
tenant = tenants_fixture[0]
role = Role.objects.create(
tenant_id=tenant.id,
name="Admin DB Test Role",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=create_test_user,
role=role,
tenant_id=tenant.id,
)
api_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Admin DB Test Key",
tenant_id=tenant.id,
entity=create_test_user,
)
api_key_headers = get_api_key_header(raw_key)
response = client.get(reverse("provider-list"), headers=api_key_headers)
assert response.status_code == 200
api_key.refresh_from_db()
assert api_key.last_used_at is not None
def test_tenant_context_established_after_authentication(
self, create_test_user, tenants_fixture, api_keys_fixture
):
"""Verify correct tenant context is established after API key auth.
After authentication, the tenant_id from the API key should be used
to set up the proper RLS context for subsequent queries.
"""
client = APIClient()
api_key = api_keys_fixture[0]
api_key_headers = get_api_key_header(api_key._raw_key)
# Use tenant-list endpoint to get actual tenant IDs
tenant_response = client.get(reverse("tenant-list"), headers=api_key_headers)
assert tenant_response.status_code == 200
tenant_data = tenant_response.json()["data"]
tenant_ids = [t["id"] for t in tenant_data]
# Verify the API key's tenant is in the list of accessible tenants
assert str(api_key.tenant_id) in tenant_ids
def test_concurrent_authentication_different_tenants(self, tenants_fixture):
"""Verify multiple API keys from different tenants can authenticate simultaneously.
This tests that the admin database routing works correctly in concurrent
scenarios and doesn't cause tenant isolation issues.
"""
client = APIClient()
user1 = User.objects.create_user(
name="concurrent_user1",
email="concurrent1@test.com",
password=TEST_PASSWORD,
)
user2 = User.objects.create_user(
name="concurrent_user2",
email="concurrent2@test.com",
password=TEST_PASSWORD,
)
tenant1 = tenants_fixture[0]
tenant2 = tenants_fixture[1]
Membership.objects.create(user=user1, tenant=tenant1)
Membership.objects.create(user=user2, tenant=tenant2)
role1 = Role.objects.create(
tenant_id=tenant1.id,
name="Concurrent Role 1",
unlimited_visibility=True,
manage_account=True,
)
role2 = Role.objects.create(
tenant_id=tenant2.id,
name="Concurrent Role 2",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=user1,
role=role1,
tenant_id=tenant1.id,
)
UserRoleRelationship.objects.create(
user=user2,
role=role2,
tenant_id=tenant2.id,
)
api_key1, raw_key1 = TenantAPIKey.objects.create_api_key(
name="Concurrent Key 1",
tenant_id=tenant1.id,
entity=user1,
)
api_key2, raw_key2 = TenantAPIKey.objects.create_api_key(
name="Concurrent Key 2",
tenant_id=tenant2.id,
entity=user2,
)
headers1 = get_api_key_header(raw_key1)
headers2 = get_api_key_header(raw_key2)
response1 = client.get(reverse("provider-list"), headers=headers1)
response2 = client.get(reverse("provider-list"), headers=headers2)
assert response1.status_code == 200
assert response2.status_code == 200
api_key1.refresh_from_db()
api_key2.refresh_from_db()
assert api_key1.last_used_at is not None
assert api_key2.last_used_at is not None
assert api_key1.tenant_id == tenant1.id
assert api_key2.tenant_id == tenant2.id
def test_api_key_update_last_used_uses_admin_db(
self, create_test_user, tenants_fixture, api_keys_fixture
):
"""Verify last_used_at update uses admin database.
The update to last_used_at during authentication must also use the
admin database since it occurs before RLS context is established.
"""
client = APIClient()
api_key = api_keys_fixture[0]
assert api_key.last_used_at is None
api_key_headers = get_api_key_header(api_key._raw_key)
first_response = client.get(reverse("provider-list"), headers=api_key_headers)
assert first_response.status_code == 200
api_key.refresh_from_db()
first_timestamp = api_key.last_used_at
assert first_timestamp is not None
time.sleep(0.1)
second_response = client.get(reverse("provider-list"), headers=api_key_headers)
assert second_response.status_code == 200
api_key.refresh_from_db()
second_timestamp = api_key.last_used_at
assert second_timestamp > first_timestamp
def test_api_key_prefix_lookup_bypasses_rls(
self, create_test_user, tenants_fixture
):
"""Verify prefix-based API key lookup works without RLS context.
The authentication process splits the key into prefix and encrypted parts,
then looks up by prefix. This lookup must work via admin database.
"""
client = APIClient()
tenant = tenants_fixture[0]
role = Role.objects.create(
tenant_id=tenant.id,
name="Prefix Test Role",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=create_test_user,
role=role,
tenant_id=tenant.id,
)
api_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Prefix Test Key",
tenant_id=tenant.id,
entity=create_test_user,
)
prefix = raw_key.split(".")[0]
assert prefix == api_key.prefix
api_key_headers = get_api_key_header(raw_key)
response = client.get(reverse("provider-list"), headers=api_key_headers)
assert response.status_code == 200
def test_expired_api_key_check_uses_admin_db(
self, create_test_user, tenants_fixture
):
"""Verify expired API key validation works via admin database.
Checking if a key is expired requires reading from TenantAPIKey,
which must use admin database during authentication.
"""
client = APIClient()
tenant = tenants_fixture[0]
expired_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Expired Test Key",
tenant_id=tenant.id,
entity=create_test_user,
expiry_date=datetime.now(timezone.utc) - timedelta(days=1),
)
api_key_headers = get_api_key_header(raw_key)
response = client.get(reverse("provider-list"), headers=api_key_headers)
assert response.status_code == 401
assert "expired" in response.json()["errors"][0]["detail"].lower()
def test_revoked_api_key_check_uses_admin_db(
self, create_test_user, tenants_fixture
):
"""Verify revoked API key validation works via admin database.
Checking if a key is revoked requires reading from TenantAPIKey,
which must use admin database during authentication.
"""
client = APIClient()
tenant = tenants_fixture[0]
role = Role.objects.create(
tenant_id=tenant.id,
name="Revoked Test Role",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=create_test_user,
role=role,
tenant_id=tenant.id,
)
api_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Revoked Test Key",
tenant_id=tenant.id,
entity=create_test_user,
)
api_key.revoked = True
api_key.save()
api_key_headers = get_api_key_header(raw_key)
response = client.get(reverse("provider-list"), headers=api_key_headers)
assert response.status_code == 401
assert "revoked" in response.json()["errors"][0]["detail"].lower()
@pytest.mark.django_db
class TestAPIKeyMultiTenantWorkflows:
"""Test complete multi-tenant workflows using API keys.
These integration tests verify end-to-end scenarios where API keys
are used across different tenants and ensure proper isolation.
"""
def test_user_with_multiple_tenant_memberships_api_keys(self, tenants_fixture):
"""User with memberships in multiple tenants can use different API keys.
Tests that a user can have separate API keys for different tenants
and each key only accesses resources in its tenant.
"""
client = APIClient()
user = User.objects.create_user(
name="multi_tenant_user",
email="multitenant@test.com",
password=TEST_PASSWORD,
)
tenant1 = tenants_fixture[0]
tenant2 = tenants_fixture[1]
Membership.objects.create(user=user, tenant=tenant1)
Membership.objects.create(user=user, tenant=tenant2)
role1 = Role.objects.create(
tenant_id=tenant1.id,
name="Multi Tenant Role 1",
unlimited_visibility=True,
manage_account=True,
)
role2 = Role.objects.create(
tenant_id=tenant2.id,
name="Multi Tenant Role 2",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=user,
role=role1,
tenant_id=tenant1.id,
)
UserRoleRelationship.objects.create(
user=user,
role=role2,
tenant_id=tenant2.id,
)
key1, raw_key1 = TenantAPIKey.objects.create_api_key(
name="Tenant 1 Key",
tenant_id=tenant1.id,
entity=user,
)
key2, raw_key2 = TenantAPIKey.objects.create_api_key(
name="Tenant 2 Key",
tenant_id=tenant2.id,
entity=user,
)
headers1 = get_api_key_header(raw_key1)
headers2 = get_api_key_header(raw_key2)
response1 = client.get(reverse("provider-list"), headers=headers1)
response2 = client.get(reverse("provider-list"), headers=headers2)
assert response1.status_code == 200
assert response2.status_code == 200
me_response1 = client.get(reverse("user-me"), headers=headers1)
me_response2 = client.get(reverse("user-me"), headers=headers2)
assert me_response1.status_code == 200
assert me_response2.status_code == 200
assert me_response1.json()["data"]["id"] == str(user.id)
assert me_response2.json()["data"]["id"] == str(user.id)
def test_api_key_cannot_access_different_tenant_resources(
self, tenants_fixture, providers_fixture
):
"""API key from one tenant cannot access resources from another tenant.
Verifies RLS enforcement after authentication ensures tenant isolation.
"""
client = APIClient()
user1 = User.objects.create_user(
name="tenant1_user",
email="tenant1user@test.com",
password=TEST_PASSWORD,
)
user2 = User.objects.create_user(
name="tenant2_user",
email="tenant2user@test.com",
password=TEST_PASSWORD,
)
tenant1 = tenants_fixture[0]
tenant2 = tenants_fixture[1]
Membership.objects.create(user=user1, tenant=tenant1)
Membership.objects.create(user=user2, tenant=tenant2)
role1 = Role.objects.create(
tenant_id=tenant1.id,
name="Isolation Test Role 1",
unlimited_visibility=True,
manage_account=True,
)
role2 = Role.objects.create(
tenant_id=tenant2.id,
name="Isolation Test Role 2",
unlimited_visibility=True,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=user1,
role=role1,
tenant_id=tenant1.id,
)
UserRoleRelationship.objects.create(
user=user2,
role=role2,
tenant_id=tenant2.id,
)
key1, raw_key1 = TenantAPIKey.objects.create_api_key(
name="Isolation Key 1",
tenant_id=tenant1.id,
entity=user1,
)
headers1 = get_api_key_header(raw_key1)
provider_response = client.get(reverse("provider-list"), headers=headers1)
assert provider_response.status_code == 200
providers_data = provider_response.json()["data"]
if providers_data:
for provider in providers_data:
provider_tenant_id = str(tenants_fixture[0].id)
assert str(tenant2.id) != provider_tenant_id
def test_api_key_workflow_create_authenticate_revoke(
self, create_test_user_rbac, tenants_fixture
):
"""Complete workflow: create API key via JWT, use it, then revoke via JWT.
Tests the full lifecycle using both JWT and API key authentication.
"""
client = APIClient()
tenants_fixture[0]
jwt_access_token, _ = get_api_tokens(
client, create_test_user_rbac.email, TEST_PASSWORD
)
jwt_headers = get_authorization_header(jwt_access_token)
create_response = client.post(
reverse("api-key-list"),
data={
"data": {
"type": "api-keys",
"attributes": {
"name": "Workflow Test Key",
},
}
},
format="vnd.api+json",
headers=jwt_headers,
)
assert create_response.status_code == 201
api_key_data = create_response.json()["data"]
api_key_id = api_key_data["id"]
raw_api_key = api_key_data["attributes"]["api_key"]
api_key_headers = get_api_key_header(raw_api_key)
auth_response = client.get(reverse("provider-list"), headers=api_key_headers)
assert auth_response.status_code == 200
revoke_response = client.delete(
reverse("api-key-revoke", kwargs={"pk": api_key_id}),
headers=jwt_headers,
)
assert revoke_response.status_code == 200
revoked_auth_response = client.get(
reverse("provider-list"), headers=api_key_headers
)
assert revoked_auth_response.status_code == 401
assert "revoked" in revoked_auth_response.json()["errors"][0]["detail"].lower()
def get_api_key_header(api_key: str) -> dict:
"""Helper to create API key authorization header."""
return {"Authorization": f"Api-Key {api_key}"}
@@ -0,0 +1,384 @@
import time
from datetime import datetime, timedelta, timezone
from unittest.mock import patch
from uuid import uuid4
import pytest
from django.test import RequestFactory
from rest_framework.exceptions import AuthenticationFailed
from api.authentication import TenantAPIKeyAuthentication
from api.db_router import MainRouter
from api.models import TenantAPIKey
@pytest.mark.django_db
class TestTenantAPIKeyAuthentication:
@pytest.fixture
def auth_backend(self):
"""Create an instance of TenantAPIKeyAuthentication."""
return TenantAPIKeyAuthentication()
@pytest.fixture
def request_factory(self):
"""Create a Django request factory."""
return RequestFactory()
def test_authenticate_credentials_uses_admin_database(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that _authenticate_credentials routes queries to admin database."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
# Extract the encrypted key part (after the prefix and separator)
_, encrypted_key = raw_key.split(TenantAPIKey.objects.separator, 1)
# Create a mock request
request = request_factory.get("/")
# Call the method
entity, auth_dict = auth_backend._authenticate_credentials(
request, encrypted_key
)
# Verify that the entity is the user associated with the API key
assert entity == api_key.entity
assert entity.id == api_key.entity.id
def test_authenticate_credentials_restores_manager_on_success(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that the manager is restored after successful authentication."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
_, encrypted_key = raw_key.split(TenantAPIKey.objects.separator, 1)
# Store the original manager
original_manager = TenantAPIKey.objects
request = request_factory.get("/")
# Call the method
auth_backend._authenticate_credentials(request, encrypted_key)
# Verify the manager was restored
assert TenantAPIKey.objects == original_manager
def test_authenticate_credentials_restores_manager_on_exception(
self, auth_backend, request_factory
):
"""Test that the manager is restored even when an exception occurs."""
# Store the original manager
original_manager = TenantAPIKey.objects
request = request_factory.get("/")
# Try to authenticate with an invalid key that will raise an exception
with pytest.raises(Exception):
auth_backend._authenticate_credentials(request, "invalid_encrypted_key")
# Verify the manager was restored despite the exception
assert TenantAPIKey.objects == original_manager
def test_authenticate_valid_api_key(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test successful authentication with a valid API key."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
# Create a request with the API key in the Authorization header
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# Authenticate
entity, auth_dict = auth_backend.authenticate(request)
# Verify the entity and auth dict
assert entity == api_key.entity
assert auth_dict["tenant_id"] == str(api_key.tenant_id)
assert auth_dict["sub"] == str(api_key.entity.id)
assert auth_dict["api_key_prefix"] == api_key.prefix
# Verify that last_used_at was updated
api_key.refresh_from_db()
assert api_key.last_used_at is not None
assert (datetime.now(timezone.utc) - api_key.last_used_at).seconds < 5
def test_authenticate_valid_api_key_uses_admin_database(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that authenticate uses admin database for API key lookup."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# Mock the manager's using method to verify it's called with admin_db
with patch.object(
TenantAPIKey.objects, "using", wraps=TenantAPIKey.objects.using
) as mock_using:
auth_backend.authenticate(request)
# Verify that .using('admin') was called
mock_using.assert_called_with(MainRouter.admin_db)
def test_authenticate_invalid_key_format_missing_separator(
self, auth_backend, request_factory
):
"""Test authentication fails with invalid API key format (no separator)."""
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = "Api-Key invalid_key_no_separator"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "Invalid API Key."
def test_authenticate_invalid_key_format_empty_prefix(
self, auth_backend, request_factory
):
"""Test authentication fails with empty prefix."""
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = "Api-Key .encrypted_part"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "Invalid API Key."
def test_authenticate_invalid_encrypted_key(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test authentication fails with invalid encrypted key."""
api_key = api_keys_fixture[0]
# Create an invalid key with valid prefix but invalid encryption
invalid_key = f"{api_key.prefix}.invalid_encrypted_data"
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {invalid_key}"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "Invalid API Key."
def test_authenticate_revoked_api_key(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test authentication fails with a revoked API key."""
# Use the revoked API key (index 2 from fixture)
api_key = api_keys_fixture[2]
raw_key = api_key._raw_key
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# The revoked key should fail during credential validation
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "This API Key has been revoked."
def test_authenticate_expired_api_key(
self, auth_backend, create_test_user, tenants_fixture, request_factory
):
"""Test authentication fails with an expired API key."""
tenant = tenants_fixture[0]
user = create_test_user
# Create an expired API key
api_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Expired API Key",
tenant_id=tenant.id,
entity=user,
expiry_date=datetime.now(timezone.utc) - timedelta(days=1),
)
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "API Key has already expired."
def test_authenticate_nonexistent_api_key(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test authentication fails when API key doesn't exist in database."""
# Create a valid-looking encrypted key with a non-existent UUID
api_key = api_keys_fixture[0]
non_existent_uuid = str(uuid4())
# Manually create an encrypted key with a non-existent ID
payload = {
"_pk": non_existent_uuid,
"_exp": (datetime.now(timezone.utc) + timedelta(days=30)).timestamp(),
}
encrypted_key = auth_backend.key_crypto.generate(payload)
fake_key = f"{api_key.prefix}.{encrypted_key}"
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {fake_key}"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "No entity matching this api key."
def test_authenticate_updates_last_used_at(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that last_used_at is updated on successful authentication."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
# Store the original last_used_at
original_last_used = api_key.last_used_at
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# Authenticate
auth_backend.authenticate(request)
# Refresh from database
api_key.refresh_from_db()
# Verify last_used_at was updated
assert api_key.last_used_at is not None
if original_last_used:
assert api_key.last_used_at > original_last_used
def test_authenticate_saves_to_admin_database(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that the API key save operation uses admin database."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# Mock the save method to verify it's called with using='admin'
with patch.object(TenantAPIKey, "save") as mock_save:
auth_backend.authenticate(request)
# Verify save was called with using=admin_db
mock_save.assert_called_once_with(
update_fields=["last_used_at"], using=MainRouter.admin_db
)
def test_authenticate_returns_correct_auth_dict(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that the auth dict contains all required fields."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
entity, auth_dict = auth_backend.authenticate(request)
# Verify all required fields are present
assert "tenant_id" in auth_dict
assert "sub" in auth_dict
assert "api_key_prefix" in auth_dict
# Verify values are correct
assert auth_dict["tenant_id"] == str(api_key.tenant_id)
assert auth_dict["sub"] == str(api_key.entity.id)
assert auth_dict["api_key_prefix"] == api_key.prefix
def test_authenticate_with_multiple_api_keys_same_tenant(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test that authentication works correctly with multiple API keys for the same tenant."""
# Test with first API key
api_key1 = api_keys_fixture[0]
raw_key1 = api_key1._raw_key
request1 = request_factory.get("/")
request1.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key1}"
entity1, auth_dict1 = auth_backend.authenticate(request1)
assert entity1 == api_key1.entity
assert auth_dict1["api_key_prefix"] == api_key1.prefix
# Test with second API key
api_key2 = api_keys_fixture[1]
raw_key2 = api_key2._raw_key
request2 = request_factory.get("/")
request2.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key2}"
entity2, auth_dict2 = auth_backend.authenticate(request2)
assert entity2 == api_key2.entity
assert auth_dict2["api_key_prefix"] == api_key2.prefix
# Verify they're different keys but same tenant
assert auth_dict1["api_key_prefix"] != auth_dict2["api_key_prefix"]
assert auth_dict1["tenant_id"] == auth_dict2["tenant_id"]
def test_authenticate_with_wrong_prefix_in_db(
self, auth_backend, api_keys_fixture, request_factory
):
"""Test authentication fails when prefix doesn't match database."""
api_key = api_keys_fixture[0]
raw_key = api_key._raw_key
# Extract the encrypted part and combine with wrong prefix
_, encrypted_part = raw_key.split(TenantAPIKey.objects.separator, 1)
wrong_key = f"pk_wrong123.{encrypted_part}"
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {wrong_key}"
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "Invalid API Key."
def test_authenticate_credentials_exception_handling(
self, auth_backend, request_factory
):
"""Test that exceptions in _authenticate_credentials are properly handled."""
request = request_factory.get("/")
# Test with completely invalid data that will cause InvalidToken
with pytest.raises(Exception):
auth_backend._authenticate_credentials(request, "completely_invalid")
def test_authenticate_with_expired_timestamp(
self, auth_backend, create_test_user, tenants_fixture, request_factory
):
"""Test that expired timestamp in encrypted key causes authentication failure."""
tenant = tenants_fixture[0]
user = create_test_user
# Create an API key with a very short expiry
api_key, raw_key = TenantAPIKey.objects.create_api_key(
name="Short-lived API Key",
tenant_id=tenant.id,
entity=user,
expiry_date=datetime.now(timezone.utc) + timedelta(seconds=1),
)
# Wait for the key to expire
time.sleep(2)
request = request_factory.get("/")
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
# Should fail with expired key
with pytest.raises(AuthenticationFailed) as exc_info:
auth_backend.authenticate(request)
assert str(exc_info.value.detail) == "API Key has already expired."
+530 -25
View File
@@ -46,6 +46,7 @@ from api.models import (
SAMLConfiguration,
SAMLToken,
Scan,
ScanSummary,
StateChoices,
Task,
TenantAPIKey,
@@ -1870,17 +1871,7 @@ class TestProviderSecretViewSet:
"kubeconfig_content": "kubeconfig-content",
},
),
# M365 with STATIC secret - no user or password
(
Provider.ProviderChoices.M365.value,
ProviderSecret.TypeChoices.STATIC,
{
"client_id": "client-id",
"client_secret": "client-secret",
"tenant_id": "tenant-id",
},
),
# M365 with user only
# M365 client secret credentials
(
Provider.ProviderChoices.M365.value,
ProviderSecret.TypeChoices.STATIC,
@@ -1889,27 +1880,17 @@ class TestProviderSecretViewSet:
"client_secret": "client-secret",
"tenant_id": "tenant-id",
"user": "test@domain.com",
},
),
# M365 with password only
(
Provider.ProviderChoices.M365.value,
ProviderSecret.TypeChoices.STATIC,
{
"client_id": "client-id",
"client_secret": "client-secret",
"tenant_id": "tenant-id",
"password": "supersecret",
},
),
# M365 with user and password
# M365 certificate credentials (valid base64)
(
Provider.ProviderChoices.M365.value,
ProviderSecret.TypeChoices.STATIC,
{
"client_id": "client-id",
"client_secret": "client-secret",
"tenant_id": "tenant-id",
"certificate_content": "VGVzdCBjZXJ0aWZpY2F0ZSBjb250ZW50",
"user": "test@domain.com",
"password": "supersecret",
},
@@ -2279,6 +2260,50 @@ class TestProviderSecretViewSet:
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_m365_provider_secrets_invalid_certificate_base64(
self, authenticated_client, providers_fixture
):
"""Test M365 provider secret creation with invalid base64 certificate content"""
# Find M365 provider from fixture
m365_provider = None
for provider in providers_fixture:
if provider.provider == Provider.ProviderChoices.M365.value:
m365_provider = provider
break
assert m365_provider is not None, "M365 provider not found in fixture"
data = {
"data": {
"type": "provider-secrets",
"attributes": {
"name": "M365 Certificate Invalid Base64",
"secret_type": "static",
"secret": {
"client_id": "client-id",
"tenant_id": "tenant-id",
"certificate_content": "invalid-base64-content!@#$%",
"user": "test@domain.com",
"password": "supersecret",
},
},
"relationships": {
"provider": {
"data": {"type": "providers", "id": str(m365_provider.id)}
}
},
}
}
response = authenticated_client.post(
reverse("providersecret-list"),
data=json.dumps(data),
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert "certificate content is not valid base64 encoded data" in str(
response.json()
)
@pytest.mark.django_db
class TestScanViewSet:
@@ -5742,6 +5767,171 @@ class TestOverviewViewSet:
assert service1_data["attributes"]["muted"] == 1
assert service2_data["attributes"]["muted"] == 0
def test_overview_findings_provider_id_in_filter(
self, authenticated_client, tenants_fixture, providers_fixture
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
scan1 = Scan.objects.create(
name="scan-one",
provider=provider1,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
scan2 = Scan.objects.create(
name="scan-two",
provider=provider2,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="check-provider-one",
service="service-a",
severity="high",
region="region-a",
_pass=5,
fail=1,
muted=2,
total=8,
new=5,
changed=2,
unchanged=1,
fail_new=1,
fail_changed=0,
pass_new=3,
pass_changed=2,
muted_new=1,
muted_changed=1,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan2,
check_id="check-provider-two",
service="service-b",
severity="medium",
region="region-b",
_pass=2,
fail=3,
muted=1,
total=6,
new=3,
changed=2,
unchanged=1,
fail_new=2,
fail_changed=1,
pass_new=1,
pass_changed=1,
muted_new=1,
muted_changed=0,
)
single_response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_id__in]": str(provider1.id)},
)
assert single_response.status_code == status.HTTP_200_OK
single_attributes = single_response.json()["data"]["attributes"]
assert single_attributes["pass"] == 5
assert single_attributes["fail"] == 1
assert single_attributes["muted"] == 2
assert single_attributes["total"] == 8
combined_response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
)
assert combined_response.status_code == status.HTTP_200_OK
combined_attributes = combined_response.json()["data"]["attributes"]
assert combined_attributes["pass"] == 7
assert combined_attributes["fail"] == 4
assert combined_attributes["muted"] == 3
assert combined_attributes["total"] == 14
def test_overview_findings_severity_provider_id_in_filter(
self, authenticated_client, tenants_fixture, providers_fixture
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
scan1 = Scan.objects.create(
name="severity-scan-one",
provider=provider1,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
scan2 = Scan.objects.create(
name="severity-scan-two",
provider=provider2,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="severity-check-one",
service="service-a",
severity="high",
region="region-a",
_pass=4,
fail=4,
muted=0,
total=8,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="severity-check-two",
service="service-a",
severity="medium",
region="region-b",
_pass=2,
fail=2,
muted=0,
total=4,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan2,
check_id="severity-check-three",
service="service-b",
severity="critical",
region="region-c",
_pass=1,
fail=2,
muted=0,
total=3,
)
single_response = authenticated_client.get(
reverse("overview-findings_severity"),
{"filter[provider_id__in]": str(provider1.id)},
)
assert single_response.status_code == status.HTTP_200_OK
single_attributes = single_response.json()["data"]["attributes"]
assert single_attributes["high"] == 8
assert single_attributes["medium"] == 4
assert single_attributes["critical"] == 0
combined_response = authenticated_client.get(
reverse("overview-findings_severity"),
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
)
assert combined_response.status_code == status.HTTP_200_OK
combined_attributes = combined_response.json()["data"]["attributes"]
assert combined_attributes["high"] == 8
assert combined_attributes["medium"] == 4
assert combined_attributes["critical"] == 3
@pytest.mark.django_db
class TestScheduleViewSet:
@@ -5781,10 +5971,12 @@ class TestScheduleViewSet:
)
assert response.status_code == status.HTTP_404_NOT_FOUND
@patch("tasks.beat.perform_scheduled_scan_task.apply_async")
@patch("api.v1.views.Task.objects.get")
def test_schedule_daily_already_scheduled(
self,
mock_task_get,
mock_apply_async,
authenticated_client,
providers_fixture,
tasks_fixture,
@@ -5792,6 +5984,7 @@ class TestScheduleViewSet:
provider, *_ = providers_fixture
prowler_task = tasks_fixture[0]
mock_task_get.return_value = prowler_task
mock_apply_async.return_value.id = prowler_task.id
json_payload = {
"provider_id": str(provider.id),
}
@@ -6892,6 +7085,188 @@ class TestTenantFinishACSView:
assert response.status_code == 302
assert "sso_saml_failed=true" in response.url
def test_dispatch_skips_role_mapping_when_single_manage_account_user(
self, create_test_user, tenants_fixture, saml_setup, settings, monkeypatch
):
"""Test that role mapping is skipped when tenant has only one user with MANAGE_ACCOUNT role"""
monkeypatch.setenv("SAML_SSO_CALLBACK_URL", "http://localhost/sso-complete")
user = create_test_user
tenant = tenants_fixture[0]
# Create a single role with manage_account=True for the user
admin_role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant=tenant,
manage_account=True,
manage_users=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user, role=admin_role, tenant_id=tenant.id
)
social_account = SocialAccount(
user=user,
provider="saml",
extra_data={
"firstName": ["John"],
"lastName": ["Doe"],
"organization": ["testing_company"],
"userType": ["no_permissions"], # This should be ignored
},
)
request = RequestFactory().get(
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
)
request.user = user
request.session = {}
with (
patch(
"allauth.socialaccount.providers.saml.views.get_app_or_404"
) as mock_get_app_or_404,
patch(
"allauth.socialaccount.models.SocialApp.objects.get"
) as mock_socialapp_get,
patch(
"allauth.socialaccount.models.SocialAccount.objects.get"
) as mock_sa_get,
patch("api.models.SAMLDomainIndex.objects.get") as mock_saml_domain_get,
patch("api.models.SAMLConfiguration.objects.get") as mock_saml_config_get,
patch("api.models.User.objects.get") as mock_user_get,
):
mock_get_app_or_404.return_value = MagicMock(
provider="saml", client_id="testtenant", name="Test App", settings={}
)
mock_sa_get.return_value = social_account
mock_socialapp_get.return_value = MagicMock(provider_id="saml")
mock_saml_domain_get.return_value = SimpleNamespace(tenant_id=tenant.id)
mock_saml_config_get.return_value = MagicMock()
mock_user_get.return_value = user
view = TenantFinishACSView.as_view()
response = view(request, organization_slug="testtenant")
assert response.status_code == 302
# Verify the admin role is still assigned (not changed to no_permissions)
assert (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(user=user, role=admin_role, tenant_id=tenant.id)
.exists()
)
# Verify no_permissions role was NOT created in the database
assert (
not Role.objects.using(MainRouter.admin_db)
.filter(name="no_permissions", tenant=tenant)
.exists()
)
# Verify no_permissions role was NOT assigned to the user
assert not (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(user=user, role__name="no_permissions", tenant_id=tenant.id)
.exists()
)
def test_dispatch_applies_role_mapping_when_multiple_manage_account_users(
self, create_test_user, tenants_fixture, saml_setup, settings, monkeypatch
):
"""Test that role mapping is applied when tenant has multiple users with MANAGE_ACCOUNT role"""
monkeypatch.setenv("SAML_SSO_CALLBACK_URL", "http://localhost/sso-complete")
user = create_test_user
tenant = tenants_fixture[0]
# Create a second user with manage_account=True
second_admin = User.objects.using(MainRouter.admin_db).create(
email="admin2@prowler.com", name="Second Admin"
)
admin_role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant=tenant,
manage_account=True,
manage_users=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user, role=admin_role, tenant_id=tenant.id
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=second_admin, role=admin_role, tenant_id=tenant.id
)
social_account = SocialAccount(
user=user,
provider="saml",
extra_data={
"firstName": ["John"],
"lastName": ["Doe"],
"organization": ["testing_company"],
"userType": ["viewer"], # This SHOULD be applied
},
)
request = RequestFactory().get(
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
)
request.user = user
request.session = {}
with (
patch(
"allauth.socialaccount.providers.saml.views.get_app_or_404"
) as mock_get_app_or_404,
patch(
"allauth.socialaccount.models.SocialApp.objects.get"
) as mock_socialapp_get,
patch(
"allauth.socialaccount.models.SocialAccount.objects.get"
) as mock_sa_get,
patch("api.models.SAMLDomainIndex.objects.get") as mock_saml_domain_get,
patch("api.models.SAMLConfiguration.objects.get") as mock_saml_config_get,
patch("api.models.User.objects.get") as mock_user_get,
):
mock_get_app_or_404.return_value = MagicMock(
provider="saml", client_id="testtenant", name="Test App", settings={}
)
mock_sa_get.return_value = social_account
mock_socialapp_get.return_value = MagicMock(provider_id="saml")
mock_saml_domain_get.return_value = SimpleNamespace(tenant_id=tenant.id)
mock_saml_config_get.return_value = MagicMock()
mock_user_get.return_value = user
view = TenantFinishACSView.as_view()
response = view(request, organization_slug="testtenant")
assert response.status_code == 302
# Verify the viewer role was created and assigned (role mapping was applied)
viewer_role = Role.objects.using(MainRouter.admin_db).get(
name="viewer", tenant=tenant
)
assert (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(user=user, role=viewer_role, tenant_id=tenant.id)
.exists()
)
# Verify the admin role was removed (replaced by viewer)
assert not (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(user=user, role=admin_role, tenant_id=tenant.id)
.exists()
)
@pytest.mark.django_db
class TestLighthouseConfigViewSet:
@@ -7530,8 +7905,6 @@ class TestTenantApiKeyViewSet:
(
[
{"name": "New API Key"},
{"name": ""},
{},
]
),
)
@@ -7572,6 +7945,18 @@ class TestTenantApiKeyViewSet:
{"name": "Invalid Expiry", "expires_at": "not-a-date"},
"expires_at",
),
(
{"name": ""},
"name",
),
(
{},
"name",
),
(
{"name": "AB"}, # Too short (min length is 3)
"name",
),
]
),
)
@@ -7600,6 +7985,58 @@ class TestTenantApiKeyViewSet:
== f"/data/attributes/{error_pointer}"
)
def test_api_keys_create_duplicate_name(
self, authenticated_client, api_keys_fixture
):
"""Test creating an API key with a duplicate name fails."""
# Use the name of an existing API key
existing_name = api_keys_fixture[0].name
data = {
"data": {
"type": "api-keys",
"attributes": {
"name": existing_name,
},
}
}
response = authenticated_client.post(
reverse("api-key-list"),
data=json.dumps(data),
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert "errors" in response.json()
error_detail = response.json()["errors"][0]["detail"]
assert "already exists" in error_detail.lower()
def test_api_keys_update_duplicate_name(
self, authenticated_client, api_keys_fixture
):
"""Test updating an API key with a duplicate name fails."""
# Get two different API keys
first_api_key = api_keys_fixture[0]
second_api_key = api_keys_fixture[1]
# Try to update the second API key to have the same name as the first one
data = {
"data": {
"type": "api-keys",
"id": str(second_api_key.id),
"attributes": {
"name": first_api_key.name,
},
}
}
response = authenticated_client.patch(
reverse("api-key-detail", kwargs={"pk": second_api_key.id}),
data=json.dumps(data),
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert "errors" in response.json()
error_detail = response.json()["errors"][0]["detail"]
assert "already exists" in error_detail.lower()
def test_api_keys_create_multiple_unique_prefixes(
self, authenticated_client, api_keys_fixture
):
@@ -7677,6 +8114,27 @@ class TestTenantApiKeyViewSet:
api_key.refresh_from_db()
assert api_key.revoked is True
def test_api_keys_revoke_preserves_created_field(
self, authenticated_client, api_keys_fixture
):
"""Test that revoking an API key preserves the created timestamp."""
api_key = api_keys_fixture[0] # Not revoked
assert api_key.revoked is False
# Record the original created timestamp
original_created = api_key.created
response = authenticated_client.delete(
reverse("api-key-revoke", kwargs={"pk": api_key.id})
)
assert response.status_code == status.HTTP_200_OK
# Verify in database
api_key.refresh_from_db()
assert api_key.revoked is True
# Verify created field has not changed
assert api_key.created == original_created
def test_api_keys_revoke_already_revoked(
self, authenticated_client, api_keys_fixture
):
@@ -8024,6 +8482,53 @@ class TestTenantApiKeyViewSet:
assert data["relationships"]["entity"]["data"]["type"] == "users"
assert data["relationships"]["entity"]["data"]["id"] == str(api_key.entity.id)
def test_api_keys_retrieve_with_entity_include(
self, authenticated_client, api_keys_fixture
):
"""Test retrieving API key with ?include=entity returns user data without memberships."""
api_key = api_keys_fixture[0]
response = authenticated_client.get(
reverse("api-key-detail", kwargs={"pk": api_key.id}),
{"include": "entity"},
)
assert response.status_code == status.HTTP_200_OK
response_data = response.json()
# Verify the main data contains the entity relationship
data = response_data["data"]
assert "entity" in data["relationships"]
assert data["relationships"]["entity"]["data"]["type"] == "users"
assert data["relationships"]["entity"]["data"]["id"] == str(api_key.entity.id)
# Verify included section exists
assert "included" in response_data
assert len(response_data["included"]) == 1
# Verify included user data
included_user = response_data["included"][0]
assert included_user["type"] == "users"
assert included_user["id"] == str(api_key.entity.id)
# Refresh entity from database to get current state
# (in case other tests modified the shared session-scoped user fixture)
api_key.entity.refresh_from_db()
# Verify UserIncludeSerializer fields are present
user_attrs = included_user["attributes"]
assert "name" in user_attrs
assert "email" in user_attrs
assert "company_name" in user_attrs
assert "date_joined" in user_attrs
assert user_attrs["name"] == api_key.entity.name
assert user_attrs["email"] == api_key.entity.email
# Verify memberships field is NOT included (excluded by UserIncludeSerializer)
assert "memberships" not in user_attrs
# Verify roles relationship is present
assert "relationships" in included_user
assert "roles" in included_user["relationships"]
def test_api_keys_entity_auto_assigned_on_create(
self, authenticated_client, create_test_user
):
@@ -105,23 +105,25 @@ from rest_framework_json_api import serializers
"type": "string",
"description": "The Azure application (client) ID for authentication in Azure AD.",
},
"client_secret": {
"type": "string",
"description": "The client secret associated with the application (client) ID, providing "
"secure access.",
},
"tenant_id": {
"type": "string",
"description": "The Azure tenant ID, representing the directory where the application is "
"registered.",
},
"client_secret": {
"type": "string",
"description": "The client secret associated with the application (client) ID, providing "
"secure access.",
},
"user": {
"type": "email",
"description": "User microsoft email address.",
"deprecated": True,
},
"password": {
"type": "string",
"description": "User password.",
"deprecated": True,
},
},
"required": [
@@ -132,6 +134,30 @@ from rest_framework_json_api import serializers
"password",
],
},
{
"type": "object",
"title": "M365 Certificate Credentials",
"properties": {
"client_id": {
"type": "string",
"description": "The Azure application (client) ID for authentication in Azure AD.",
},
"tenant_id": {
"type": "string",
"description": "The Azure tenant ID, representing the directory where the application is "
"registered.",
},
"certificate_content": {
"type": "string",
"description": "The certificate content in base64 format for certificate-based authentication.",
},
},
"required": [
"client_id",
"tenant_id",
"certificate_content",
],
},
{
"type": "object",
"title": "GCP Static Credentials",
+80 -1
View File
@@ -1,3 +1,4 @@
import base64
import json
from datetime import datetime, timedelta, timezone
@@ -317,6 +318,23 @@ class UserSerializer(BaseSerializerV1):
)
class UserIncludeSerializer(UserSerializer):
class Meta:
model = User
fields = [
"id",
"name",
"email",
"company_name",
"date_joined",
"roles",
]
included_serializers = {
"roles": "api.v1.serializers.RoleIncludeSerializer",
}
class UserCreateSerializer(BaseWriteSerializer):
password = serializers.CharField(write_only=True)
company_name = serializers.CharField(required=False)
@@ -1378,10 +1396,38 @@ class AzureProviderSecret(serializers.Serializer):
class M365ProviderSecret(serializers.Serializer):
client_id = serializers.CharField()
client_secret = serializers.CharField()
client_secret = serializers.CharField(required=False)
tenant_id = serializers.CharField()
user = serializers.EmailField(required=False)
password = serializers.CharField(required=False)
certificate_content = serializers.CharField(required=False)
def validate(self, attrs):
if attrs.get("client_secret") and attrs.get("certificate_content"):
raise serializers.ValidationError(
"You cannot provide both client_secret and certificate_content."
)
if not attrs.get("client_secret") and not attrs.get("certificate_content"):
raise serializers.ValidationError(
"You must provide either client_secret or certificate_content."
)
return super().validate(attrs)
def validate_certificate_content(self, certificate_content):
"""Validate that M365 certificate content is valid base64 encoded data."""
if certificate_content:
try:
base64.b64decode(certificate_content, validate=True)
except Exception as e:
raise ValidationError(
{
"certificate_content": [
f"The provided certificate content is not valid base64 encoded data: {str(e)}"
]
},
code="m365-certificate-content",
)
return certificate_content
class Meta:
resource_name = "provider-secrets"
@@ -1974,6 +2020,17 @@ class ComplianceOverviewDetailSerializer(serializers.Serializer):
resource_name = "compliance-requirements-details"
class ComplianceOverviewDetailThreatscoreSerializer(ComplianceOverviewDetailSerializer):
"""
Serializer for detailed compliance requirement information for Threatscore.
Includes additional fields specific to the Threatscore framework.
"""
passed_findings = serializers.IntegerField()
total_findings = serializers.IntegerField()
class ComplianceOverviewAttributesSerializer(serializers.Serializer):
id = serializers.CharField()
compliance_name = serializers.CharField()
@@ -2779,6 +2836,10 @@ class TenantApiKeySerializer(RLSSerializer):
"entity",
]
included_serializers = {
"entity": "api.v1.serializers.UserIncludeSerializer",
}
class TenantApiKeyCreateSerializer(RLSSerializer, BaseWriteSerializer):
"""Serializer for creating new API keys."""
@@ -2811,6 +2872,13 @@ class TenantApiKeyCreateSerializer(RLSSerializer, BaseWriteSerializer):
"api_key": {"read_only": True},
}
def validate_name(self, value):
"""Validate that the name is unique within the tenant."""
tenant_id = self.context.get("tenant_id")
if TenantAPIKey.objects.filter(tenant_id=tenant_id, name=value).exists():
raise ValidationError("An API key with this name already exists.")
return value
def get_api_key(self, obj):
"""Return the raw API key if it was stored during creation."""
return getattr(obj, "_raw_api_key", None)
@@ -2852,3 +2920,14 @@ class TenantApiKeyUpdateSerializer(RLSSerializer, BaseWriteSerializer):
"inserted_at": {"read_only": True},
"last_used_at": {"read_only": True},
}
def validate_name(self, value):
"""Validate that the name is unique within the tenant, excluding current instance."""
tenant_id = self.context.get("tenant_id")
if (
TenantAPIKey.objects.filter(tenant_id=tenant_id, name=value)
.exclude(id=self.instance.id)
.exists()
):
raise ValidationError("An API key with this name already exists.")
return value
+62 -37
View File
@@ -142,6 +142,7 @@ from api.v1.mixins import PaginateByPkMixin, TaskManagementMixin
from api.v1.serializers import (
ComplianceOverviewAttributesSerializer,
ComplianceOverviewDetailSerializer,
ComplianceOverviewDetailThreatscoreSerializer,
ComplianceOverviewMetadataSerializer,
ComplianceOverviewSerializer,
FindingDynamicFilterSerializer,
@@ -665,36 +666,48 @@ class TenantFinishACSView(FinishACSView):
.get(email_domain=email_domain)
.tenant
)
role_name = (
extra.get("userType", ["no_permissions"])[0].strip()
if extra.get("userType")
else "no_permissions"
# Check if tenant has only one user with MANAGE_ACCOUNT role
users_with_manage_account = (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(role__manage_account=True, tenant_id=tenant.id)
.values("user")
.distinct()
.count()
)
try:
role = Role.objects.using(MainRouter.admin_db).get(
name=role_name, tenant=tenant
# Only apply role mapping from userType if tenant does NOT have exactly one user with MANAGE_ACCOUNT
if users_with_manage_account != 1:
role_name = (
extra.get("userType", ["no_permissions"])[0].strip()
if extra.get("userType")
else "no_permissions"
)
except Role.DoesNotExist:
role = Role.objects.using(MainRouter.admin_db).create(
name=role_name,
tenant=tenant,
manage_users=False,
manage_account=False,
manage_billing=False,
manage_providers=False,
manage_integrations=False,
manage_scans=False,
unlimited_visibility=False,
try:
role = Role.objects.using(MainRouter.admin_db).get(
name=role_name, tenant=tenant
)
except Role.DoesNotExist:
role = Role.objects.using(MainRouter.admin_db).create(
name=role_name,
tenant=tenant,
manage_users=False,
manage_account=False,
manage_billing=False,
manage_providers=False,
manage_integrations=False,
manage_scans=False,
unlimited_visibility=False,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).filter(
user=user,
tenant_id=tenant.id,
).delete()
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).filter(
user=user,
tenant_id=tenant.id,
).delete()
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
membership, _ = Membership.objects.using(MainRouter.admin_db).get_or_create(
user=user,
tenant=tenant,
@@ -3436,15 +3449,16 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
)
filtered_queryset = self.filter_queryset(self.get_queryset())
all_requirements = (
filtered_queryset.values(
"requirement_id", "framework", "version", "description"
)
.distinct()
.annotate(
total_instances=Count("id"),
manual_count=Count("id", filter=Q(requirement_status="MANUAL")),
)
all_requirements = filtered_queryset.values(
"requirement_id",
"framework",
"version",
"description",
).annotate(
total_instances=Count("id"),
manual_count=Count("id", filter=Q(requirement_status="MANUAL")),
passed_findings_sum=Sum("passed_findings"),
total_findings_sum=Sum("total_findings"),
)
passed_instances = (
@@ -3463,6 +3477,8 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
total_instances = requirement["total_instances"]
passed_count = passed_counts.get(requirement_id, 0)
is_manual = requirement["manual_count"] == total_instances
passed_findings = requirement["passed_findings_sum"] or 0
total_findings = requirement["total_findings_sum"] or 0
if is_manual:
requirement_status = "MANUAL"
elif passed_count == total_instances:
@@ -3477,10 +3493,19 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
"version": requirement["version"],
"description": requirement["description"],
"status": requirement_status,
"passed_findings": passed_findings,
"total_findings": total_findings,
}
)
serializer = self.get_serializer(requirements_summary, many=True)
# Use different serializer for threatscore framework
if "threatscore" not in compliance_id:
serializer = self.get_serializer(requirements_summary, many=True)
else:
serializer = ComplianceOverviewDetailThreatscoreSerializer(
requirements_summary, many=True
)
return Response(serializer.data, status=status.HTTP_200_OK)
@action(detail=False, methods=["get"], url_name="attributes")
+23 -8
View File
@@ -5,24 +5,39 @@ DEBUG = env.bool("DJANGO_DEBUG", default=True)
ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["*"])
# Database
default_db_name = env("POSTGRES_DB", default="prowler_db")
default_db_user = env("POSTGRES_USER", default="prowler_user")
default_db_password = env("POSTGRES_PASSWORD", default="prowler")
default_db_host = env("POSTGRES_HOST", default="postgres-db")
default_db_port = env("POSTGRES_PORT", default="5432")
DATABASES = {
"prowler_user": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB", default="prowler_db"),
"USER": env("POSTGRES_USER", default="prowler_user"),
"PASSWORD": env("POSTGRES_PASSWORD", default="prowler"),
"HOST": env("POSTGRES_HOST", default="postgres-db"),
"PORT": env("POSTGRES_PORT", default="5432"),
"NAME": default_db_name,
"USER": default_db_user,
"PASSWORD": default_db_password,
"HOST": default_db_host,
"PORT": default_db_port,
},
"admin": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB", default="prowler_db"),
"NAME": default_db_name,
"USER": env("POSTGRES_ADMIN_USER", default="prowler"),
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD", default="S3cret"),
"HOST": env("POSTGRES_HOST", default="postgres-db"),
"PORT": env("POSTGRES_PORT", default="5432"),
"HOST": default_db_host,
"PORT": default_db_port,
},
"replica": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_REPLICA_DB", default=default_db_name),
"USER": env("POSTGRES_REPLICA_USER", default=default_db_user),
"PASSWORD": env("POSTGRES_REPLICA_PASSWORD", default=default_db_password),
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
REST_FRAMEWORK["DEFAULT_RENDERER_CLASSES"] = tuple( # noqa: F405
+24 -9
View File
@@ -6,22 +6,37 @@ ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["localhost", "127.0.0.
# Database
# TODO Use Django database routers https://docs.djangoproject.com/en/5.0/topics/db/multi-db/#automatic-database-routing
default_db_name = env("POSTGRES_DB")
default_db_user = env("POSTGRES_USER")
default_db_password = env("POSTGRES_PASSWORD")
default_db_host = env("POSTGRES_HOST")
default_db_port = env("POSTGRES_PORT")
DATABASES = {
"prowler_user": {
"ENGINE": "django.db.backends.postgresql",
"NAME": env("POSTGRES_DB"),
"USER": env("POSTGRES_USER"),
"PASSWORD": env("POSTGRES_PASSWORD"),
"HOST": env("POSTGRES_HOST"),
"PORT": env("POSTGRES_PORT"),
"ENGINE": "psqlextra.backend",
"NAME": default_db_name,
"USER": default_db_user,
"PASSWORD": default_db_password,
"HOST": default_db_host,
"PORT": default_db_port,
},
"admin": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB"),
"NAME": default_db_name,
"USER": env("POSTGRES_ADMIN_USER"),
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD"),
"HOST": env("POSTGRES_HOST"),
"PORT": env("POSTGRES_PORT"),
"HOST": default_db_host,
"PORT": default_db_port,
},
"replica": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_REPLICA_DB", default=default_db_name),
"USER": env("POSTGRES_REPLICA_USER", default=default_db_user),
"PASSWORD": env("POSTGRES_REPLICA_PASSWORD", default=default_db_password),
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
+8
View File
@@ -20,6 +20,10 @@ from prowler.lib.outputs.asff.asff import ASFF
from prowler.lib.outputs.compliance.aws_well_architected.aws_well_architected import (
AWSWellArchitected,
)
from prowler.lib.outputs.compliance.ccc.ccc_aws import CCC_AWS
from prowler.lib.outputs.compliance.ccc.ccc_azure import CCC_Azure
from prowler.lib.outputs.compliance.ccc.ccc_gcp import CCC_GCP
from prowler.lib.outputs.compliance.c5.c5_aws import AWSC5
from prowler.lib.outputs.compliance.cis.cis_aws import AWSCIS
from prowler.lib.outputs.compliance.cis.cis_azure import AzureCIS
from prowler.lib.outputs.compliance.cis.cis_gcp import GCPCIS
@@ -73,12 +77,15 @@ COMPLIANCE_CLASS_MAP = {
(lambda name: name.startswith("iso27001_"), AWSISO27001),
(lambda name: name.startswith("kisa"), AWSKISAISMSP),
(lambda name: name == "prowler_threatscore_aws", ProwlerThreatScoreAWS),
(lambda name: name == "ccc_aws", CCC_AWS),
(lambda name: name.startswith("c5_"), AWSC5),
],
"azure": [
(lambda name: name.startswith("cis_"), AzureCIS),
(lambda name: name == "mitre_attack_azure", AzureMitreAttack),
(lambda name: name.startswith("ens_"), AzureENS),
(lambda name: name.startswith("iso27001_"), AzureISO27001),
(lambda name: name == "ccc_azure", CCC_Azure),
(lambda name: name == "prowler_threatscore_azure", ProwlerThreatScoreAzure),
],
"gcp": [
@@ -87,6 +94,7 @@ COMPLIANCE_CLASS_MAP = {
(lambda name: name.startswith("ens_"), GCPENS),
(lambda name: name.startswith("iso27001_"), GCPISO27001),
(lambda name: name == "prowler_threatscore_gcp", ProwlerThreatScoreGCP),
(lambda name: name == "ccc_gcp", CCC_GCP),
],
"kubernetes": [
(lambda name: name.startswith("cis_"), KubernetesCIS),
+2 -1
View File
@@ -5,6 +5,7 @@ from celery.utils.log import get_task_logger
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
from tasks.utils import batched
from api.db_router import READ_REPLICA_ALIAS
from api.db_utils import rls_transaction
from api.models import Finding, Integration, Provider
from api.utils import initialize_prowler_integration, initialize_prowler_provider
@@ -289,7 +290,7 @@ def upload_security_hub_integration(
has_findings = False
batch_number = 0
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
qs = (
Finding.all_objects.filter(tenant_id=tenant_id, scan_id=scan_id)
.order_by("uid")
+238 -39
View File
@@ -1,8 +1,12 @@
import csv
import io
import json
import time
import uuid
from collections import defaultdict
from copy import deepcopy
from datetime import datetime, timezone
from typing import Any
from celery.utils.log import get_task_logger
from config.settings.celery import CELERY_DEADLOCK_ATTEMPTS
@@ -14,8 +18,11 @@ from api.compliance import (
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
generate_scan_compliance,
)
from api.db_router import READ_REPLICA_ALIAS, MainRouter
from api.db_utils import (
create_objects_in_batches,
POSTGRES_TENANT_VAR,
SET_CONFIG_QUERY,
psycopg_connection,
rls_transaction,
update_objects_in_batches,
)
@@ -40,6 +47,28 @@ from prowler.lib.scan.scan import Scan as ProwlerScan
logger = get_task_logger(__name__)
# Column order must match `ComplianceRequirementOverview` schema in
# `api/models.py`. Keep this list minimal but sufficient to populate all
# non-nullable fields plus the counters we care about.
COMPLIANCE_REQUIREMENT_COPY_COLUMNS = (
"id",
"tenant_id",
"inserted_at",
"compliance_id",
"framework",
"version",
"description",
"region",
"requirement_id",
"requirement_status",
"passed_checks",
"failed_checks",
"total_checks",
"passed_findings",
"total_findings",
"scan_id",
)
def _create_finding_delta(
last_status: FindingStatus | None | str, new_status: FindingStatus | None
@@ -107,6 +136,124 @@ def _store_resources(
return resource_instance, (resource_instance.uid, resource_instance.region)
def _copy_compliance_requirement_rows(
tenant_id: str, rows: list[dict[str, Any]]
) -> None:
"""Stream compliance requirement rows into Postgres using COPY.
We leverage the admin connection (when available) to bypass the COPY + RLS
restriction, writing only the fields required by
``ComplianceRequirementOverview``.
Args:
tenant_id: Target tenant UUID.
rows: List of row dictionaries prepared by
:func:`create_compliance_requirements`.
"""
csv_buffer = io.StringIO()
writer = csv.writer(csv_buffer)
datetime_now = datetime.now(tz=timezone.utc)
for row in rows:
writer.writerow(
[
str(row.get("id")),
str(row.get("tenant_id")),
(row.get("inserted_at") or datetime_now).isoformat(),
row.get("compliance_id") or "",
row.get("framework") or "",
row.get("version") or "",
row.get("description") or "",
row.get("region") or "",
row.get("requirement_id") or "",
row.get("requirement_status") or "",
row.get("passed_checks", 0),
row.get("failed_checks", 0),
row.get("total_checks", 0),
row.get("passed_findings", 0),
row.get("total_findings", 0),
str(row.get("scan_id")),
]
)
csv_buffer.seek(0)
copy_sql = (
"COPY compliance_requirements_overviews ("
+ ", ".join(COMPLIANCE_REQUIREMENT_COPY_COLUMNS)
+ ") FROM STDIN WITH (FORMAT CSV, DELIMITER ',', QUOTE '\"', ESCAPE '\"', NULL '\\N')"
)
try:
with psycopg_connection(MainRouter.admin_db) as connection:
connection.autocommit = False
try:
with connection.cursor() as cursor:
cursor.execute(SET_CONFIG_QUERY, [POSTGRES_TENANT_VAR, tenant_id])
cursor.copy_expert(copy_sql, csv_buffer)
connection.commit()
except Exception:
connection.rollback()
raise
finally:
csv_buffer.close()
def _persist_compliance_requirement_rows(
tenant_id: str, rows: list[dict[str, Any]]
) -> None:
"""Persist compliance requirement rows using COPY with ORM fallback.
Args:
tenant_id: Target tenant UUID.
rows: Precomputed row dictionaries that reflect the compliance
overview state for a scan.
"""
if not rows:
return
try:
_copy_compliance_requirement_rows(tenant_id, rows)
except Exception as error:
logger.exception(
"COPY bulk insert for compliance requirements failed; falling back to ORM bulk_create",
exc_info=error,
)
fallback_objects = [
ComplianceRequirementOverview(
id=row["id"],
tenant_id=row["tenant_id"],
inserted_at=row["inserted_at"],
compliance_id=row["compliance_id"],
framework=row["framework"],
version=row["version"],
description=row["description"],
region=row["region"],
requirement_id=row["requirement_id"],
requirement_status=row["requirement_status"],
passed_checks=row["passed_checks"],
failed_checks=row["failed_checks"],
total_checks=row["total_checks"],
passed_findings=row.get("passed_findings", 0),
total_findings=row.get("total_findings", 0),
scan_id=row["scan_id"],
)
for row in rows
]
with rls_transaction(tenant_id):
ComplianceRequirementOverview.objects.bulk_create(
fallback_objects, batch_size=500
)
def _normalized_compliance_key(framework: str | None, version: str | None) -> str:
"""Return normalized identifier used to group compliance totals."""
normalized_framework = (framework or "").lower().replace("-", "").replace("_", "")
normalized_version = (version or "").lower().replace("-", "").replace("_", "")
return f"{normalized_framework}{normalized_version}"
def perform_prowler_scan(
tenant_id: str,
scan_id: str,
@@ -143,7 +290,7 @@ def perform_prowler_scan(
scan_instance.save()
# Find the mutelist processor if it exists
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
try:
mutelist_processor = Processor.objects.get(
tenant_id=tenant_id, processor_type=Processor.ProcessorChoices.MUTELIST
@@ -272,7 +419,7 @@ def perform_prowler_scan(
unique_resources.add((resource_instance.uid, resource_instance.region))
# Process finding
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
finding_uid = finding.uid
last_first_seen_at = None
if finding_uid not in last_status_cache:
@@ -305,6 +452,12 @@ def perform_prowler_scan(
# If the finding is muted at this time the reason must be the configured Mutelist
muted_reason = "Muted by mutelist" if finding.muted else None
# Increment failed_findings_count cache if the finding status is FAIL and not muted
if status == FindingStatus.FAIL and not finding.muted:
resource_uid = finding.resource_uid
resource_failed_findings_cache[resource_uid] += 1
with rls_transaction(tenant_id):
# Create the finding
finding_instance = Finding.objects.create(
tenant_id=tenant_id,
@@ -325,11 +478,6 @@ def perform_prowler_scan(
)
finding_instance.add_resources([resource_instance])
# Increment failed_findings_count cache if the finding status is FAIL and not muted
if status == FindingStatus.FAIL and not finding.muted:
resource_uid = finding.resource_uid
resource_failed_findings_cache[resource_uid] += 1
# Update scan resource summaries
scan_resource_cache.add(
(
@@ -439,7 +587,7 @@ def aggregate_findings(tenant_id: str, scan_id: str):
- muted_new: Muted findings with a delta of 'new'.
- muted_changed: Muted findings with a delta of 'changed'.
"""
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
findings = Finding.objects.filter(tenant_id=tenant_id, scan_id=scan_id)
aggregation = findings.values(
@@ -582,15 +730,32 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
ValidationError: If tenant_id is not a valid UUID.
"""
try:
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
scan_instance = Scan.objects.get(pk=scan_id)
provider_instance = scan_instance.provider
prowler_provider = return_prowler_provider(provider_instance)
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
provider_instance.provider
]
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
threatscore_requirements_by_check: dict[str, set[str]] = {}
threatscore_framework = compliance_template.get(
modeled_threatscore_compliance_id
)
if threatscore_framework:
for requirement_id, requirement in threatscore_framework[
"requirements"
].items():
for check_id in requirement["checks"]:
threatscore_requirements_by_check.setdefault(check_id, set()).add(
requirement_id
)
# Get check status data by region from findings
findings = (
Finding.all_objects.filter(scan_id=scan_id, muted=False)
.only("id", "check_id", "status")
.only("id", "check_id", "status", "compliance")
.prefetch_related(
Prefetch(
"resources",
@@ -601,14 +766,36 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
.iterator(chunk_size=1000)
)
findings_count_by_compliance = {}
check_status_by_region = {}
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
for finding in findings:
for resource in finding.small_resources:
region = resource.region
current_status = check_status_by_region.setdefault(region, {})
if current_status.get(finding.check_id) != "FAIL":
current_status[finding.check_id] = finding.status
if modeled_threatscore_compliance_id in finding.compliance:
for requirement_id in finding.compliance[
modeled_threatscore_compliance_id
]:
compliance_key = findings_count_by_compliance.setdefault(
region, {}
).setdefault(
modeled_threatscore_compliance_id.lower().replace(
"-", ""
),
{},
)
if requirement_id not in compliance_key:
compliance_key[requirement_id] = {
"total": 0,
"pass": 0,
}
compliance_key[requirement_id]["total"] += 1
if finding.status == "PASS":
compliance_key[requirement_id]["pass"] += 1
try:
# Try to get regions from provider
@@ -617,11 +804,6 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
# If not available, use regions from findings
regions = set(check_status_by_region.keys())
# Get compliance template for the provider
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
provider_instance.provider
]
# Create compliance data by region
compliance_overview_by_region = {
region: deepcopy(compliance_template) for region in regions
@@ -640,36 +822,53 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
status,
)
# Prepare compliance requirement objects
compliance_requirement_objects = []
# Prepare compliance requirement rows
compliance_requirement_rows: list[dict[str, Any]] = []
utc_datetime_now = datetime.now(tz=timezone.utc)
for region, compliance_data in compliance_overview_by_region.items():
for compliance_id, compliance in compliance_data.items():
modeled_compliance_id = _normalized_compliance_key(
compliance["framework"], compliance["version"]
)
# Create an overview record for each requirement within each compliance framework
for requirement_id, requirement in compliance["requirements"].items():
compliance_requirement_objects.append(
ComplianceRequirementOverview(
tenant_id=tenant_id,
scan=scan_instance,
region=region,
compliance_id=compliance_id,
framework=compliance["framework"],
version=compliance["version"],
requirement_id=requirement_id,
description=requirement["description"],
passed_checks=requirement["checks_status"]["pass"],
failed_checks=requirement["checks_status"]["fail"],
total_checks=requirement["checks_status"]["total"],
requirement_status=requirement["status"],
)
checks_status = requirement["checks_status"]
compliance_requirement_rows.append(
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": utc_datetime_now,
"compliance_id": compliance_id,
"framework": compliance["framework"],
"version": compliance["version"] or "",
"description": requirement.get("description") or "",
"region": region,
"requirement_id": requirement_id,
"requirement_status": requirement["status"],
"passed_checks": checks_status["pass"],
"failed_checks": checks_status["fail"],
"total_checks": checks_status["total"],
"scan_id": scan_instance.id,
"passed_findings": findings_count_by_compliance.get(
region, {}
)
.get(modeled_compliance_id, {})
.get(requirement_id, {})
.get("pass", 0),
"total_findings": findings_count_by_compliance.get(
region, {}
)
.get(modeled_compliance_id, {})
.get(requirement_id, {})
.get("total", 0),
}
)
# Bulk create requirement records
create_objects_in_batches(
tenant_id, ComplianceRequirementOverview, compliance_requirement_objects
)
# Bulk create requirement records using PostgreSQL COPY
_persist_compliance_requirement_rows(tenant_id, compliance_requirement_rows)
return {
"requirements_created": len(compliance_requirement_objects),
"requirements_created": len(compliance_requirement_rows),
"regions_processed": list(regions),
"compliance_frameworks": (
list(compliance_overview_by_region.get(list(regions)[0], {}).keys())
+56 -52
View File
@@ -34,6 +34,7 @@ from tasks.jobs.scan import (
from tasks.utils import batched, get_next_execution_datetime
from api.compliance import get_compliance_frameworks
from api.db_router import READ_REPLICA_ALIAS
from api.db_utils import rls_transaction
from api.decorators import set_tenant
from api.models import Finding, Integration, Provider, Scan, ScanSummary, StateChoices
@@ -343,70 +344,73 @@ def generate_outputs_task(scan_id: str, provider_id: str, tenant_id: str):
.order_by("uid")
.iterator()
)
for batch, is_last in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
fos = [FindingOutput.transform_api_finding(f, prowler_provider) for f in batch]
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
for batch, is_last in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
fos = [
FindingOutput.transform_api_finding(f, prowler_provider) for f in batch
]
# Outputs
for mode, cfg in OUTPUT_FORMATS_MAPPING.items():
# Skip ASFF generation if not needed
if mode == "json-asff" and not generate_asff:
continue
# Outputs
for mode, cfg in OUTPUT_FORMATS_MAPPING.items():
# Skip ASFF generation if not needed
if mode == "json-asff" and not generate_asff:
continue
cls = cfg["class"]
suffix = cfg["suffix"]
extra = cfg.get("kwargs", {}).copy()
if mode == "html":
extra.update(provider=prowler_provider, stats=scan_summary)
cls = cfg["class"]
suffix = cfg["suffix"]
extra = cfg.get("kwargs", {}).copy()
if mode == "html":
extra.update(provider=prowler_provider, stats=scan_summary)
writer, initialization = get_writer(
output_writers,
cls,
lambda cls=cls, fos=fos, suffix=suffix: cls(
findings=fos,
file_path=out_dir,
file_extension=suffix,
from_cli=False,
),
is_last,
)
if not initialization:
writer.transform(fos)
writer.batch_write_data_to_file(**extra)
writer._data.clear()
writer, initialization = get_writer(
output_writers,
cls,
lambda cls=cls, fos=fos, suffix=suffix: cls(
findings=fos,
file_path=out_dir,
file_extension=suffix,
from_cli=False,
),
is_last,
)
if not initialization:
writer.transform(fos)
writer.batch_write_data_to_file(**extra)
writer._data.clear()
# Compliance CSVs
for name in frameworks_avail:
compliance_obj = frameworks_bulk[name]
# Compliance CSVs
for name in frameworks_avail:
compliance_obj = frameworks_bulk[name]
klass = GenericCompliance
for condition, cls in COMPLIANCE_CLASS_MAP.get(provider_type, []):
if condition(name):
klass = cls
break
klass = GenericCompliance
for condition, cls in COMPLIANCE_CLASS_MAP.get(provider_type, []):
if condition(name):
klass = cls
break
filename = f"{comp_dir}_{name}.csv"
filename = f"{comp_dir}_{name}.csv"
writer, initialization = get_writer(
compliance_writers,
name,
lambda klass=klass, fos=fos: klass(
findings=fos,
compliance=compliance_obj,
file_path=filename,
from_cli=False,
),
is_last,
)
if not initialization:
writer.transform(fos, compliance_obj, name)
writer.batch_write_data_to_file()
writer._data.clear()
writer, initialization = get_writer(
compliance_writers,
name,
lambda klass=klass, fos=fos: klass(
findings=fos,
compliance=compliance_obj,
file_path=filename,
from_cli=False,
),
is_last,
)
if not initialization:
writer.transform(fos, compliance_obj, name)
writer.batch_write_data_to_file()
writer._data.clear()
compressed = _compress_output_files(out_dir)
upload_uri = _upload_to_s3(tenant_id, compressed, scan_id)
# S3 integrations (need output_directory)
with rls_transaction(tenant_id):
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
s3_integrations = Integration.objects.filter(
integrationproviderrelationship__provider_id=provider_id,
integration_type=Integration.IntegrationChoices.AMAZON_S3,
+776 -1
View File
@@ -1,17 +1,22 @@
import csv
import json
import uuid
from datetime import datetime
from datetime import datetime, timezone
from io import StringIO
from unittest.mock import MagicMock, patch
import pytest
from tasks.jobs.scan import (
_copy_compliance_requirement_rows,
_create_finding_delta,
_persist_compliance_requirement_rows,
_store_resources,
create_compliance_requirements,
perform_prowler_scan,
)
from tasks.utils import CustomEncoder
from api.db_router import MainRouter
from api.exceptions import ProviderConnectionError
from api.models import Finding, Provider, Resource, Scan, StateChoices, StatusChoices
from prowler.lib.check.models import Severity
@@ -1045,3 +1050,773 @@ class TestCreateComplianceRequirements:
assert "requirements_created" in result
assert result["requirements_created"] >= 0
class TestComplianceRequirementCopy:
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_streams_csv(
self, mock_psycopg_connection, settings
):
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
captured = {}
def copy_side_effect(sql, file_obj):
captured["sql"] = sql
captured["data"] = file_obj.read()
cursor.copy_expert.side_effect = copy_side_effect
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": None,
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
mock_psycopg_connection.assert_called_once_with("admin")
connection.cursor.assert_called_once()
cursor.execute.assert_called_once()
cursor.copy_expert.assert_called_once()
csv_rows = list(csv.reader(StringIO(captured["data"])))
assert csv_rows[0][0] == str(row["id"])
assert csv_rows[0][5] == ""
assert csv_rows[0][-1] == str(row["scan_id"])
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
@patch("tasks.jobs.scan.rls_transaction")
@patch(
"tasks.jobs.scan._copy_compliance_requirement_rows",
side_effect=Exception("copy failed"),
)
def test_persist_compliance_requirement_rows_fallback(
self, mock_copy, mock_rls_transaction, mock_bulk_create
):
inserted_at = datetime.now(timezone.utc)
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"inserted_at": inserted_at,
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
tenant_id = row["tenant_id"]
ctx = MagicMock()
ctx.__enter__.return_value = None
ctx.__exit__.return_value = False
mock_rls_transaction.return_value = ctx
_persist_compliance_requirement_rows(tenant_id, [row])
mock_copy.assert_called_once_with(tenant_id, [row])
mock_rls_transaction.assert_called_once_with(tenant_id)
mock_bulk_create.assert_called_once()
args, kwargs = mock_bulk_create.call_args
objects = args[0]
assert len(objects) == 1
fallback = objects[0]
assert fallback.version == row["version"]
assert fallback.compliance_id == row["compliance_id"]
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
@patch("tasks.jobs.scan.rls_transaction")
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
def test_persist_compliance_requirement_rows_no_rows(
self, mock_copy, mock_rls_transaction, mock_bulk_create
):
_persist_compliance_requirement_rows(str(uuid.uuid4()), [])
mock_copy.assert_not_called()
mock_rls_transaction.assert_not_called()
mock_bulk_create.assert_not_called()
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_multiple_rows(
self, mock_psycopg_connection, settings
):
"""Test COPY with multiple rows to ensure batch processing works correctly."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
captured = {}
def copy_side_effect(sql, file_obj):
captured["sql"] = sql
captured["data"] = file_obj.read()
cursor.copy_expert.side_effect = copy_side_effect
tenant_id = str(uuid.uuid4())
scan_id = uuid.uuid4()
inserted_at = datetime.now(timezone.utc)
rows = [
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": "1.0",
"description": "First requirement",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 5,
"failed_checks": 0,
"total_checks": 5,
"scan_id": scan_id,
},
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": "1.0",
"description": "Second requirement",
"region": "us-west-2",
"requirement_id": "req-2",
"requirement_status": "FAIL",
"passed_checks": 3,
"failed_checks": 2,
"total_checks": 5,
"scan_id": scan_id,
},
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "aws_foundational_security_aws",
"framework": "AWS-Foundational-Security-Best-Practices",
"version": "2.0",
"description": "Third requirement",
"region": "eu-west-1",
"requirement_id": "req-3",
"requirement_status": "MANUAL",
"passed_checks": 0,
"failed_checks": 0,
"total_checks": 3,
"scan_id": scan_id,
},
]
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(tenant_id, rows)
mock_psycopg_connection.assert_called_once_with("admin")
connection.cursor.assert_called_once()
cursor.execute.assert_called_once()
cursor.copy_expert.assert_called_once()
csv_rows = list(csv.reader(StringIO(captured["data"])))
assert len(csv_rows) == 3
# Validate first row
assert csv_rows[0][0] == str(rows[0]["id"])
assert csv_rows[0][1] == tenant_id
assert csv_rows[0][3] == "cisa_aws"
assert csv_rows[0][4] == "CISA"
assert csv_rows[0][6] == "First requirement"
assert csv_rows[0][7] == "us-east-1"
assert csv_rows[0][10] == "5"
assert csv_rows[0][11] == "0"
assert csv_rows[0][12] == "5"
# Validate second row
assert csv_rows[1][0] == str(rows[1]["id"])
assert csv_rows[1][7] == "us-west-2"
assert csv_rows[1][9] == "FAIL"
assert csv_rows[1][10] == "3"
assert csv_rows[1][11] == "2"
# Validate third row
assert csv_rows[2][0] == str(rows[2]["id"])
assert csv_rows[2][3] == "aws_foundational_security_aws"
assert csv_rows[2][5] == "2.0"
assert csv_rows[2][9] == "MANUAL"
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_null_values(
self, mock_psycopg_connection, settings
):
"""Test COPY handles NULL/None values correctly in nullable fields."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
captured = {}
def copy_side_effect(sql, file_obj):
captured["sql"] = sql
captured["data"] = file_obj.read()
cursor.copy_expert.side_effect = copy_side_effect
# Row with all nullable fields set to None/empty
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "test_framework",
"framework": "Test",
"version": None, # nullable
"description": None, # nullable
"region": "",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 0,
"failed_checks": 0,
"total_checks": 0,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
csv_rows = list(csv.reader(StringIO(captured["data"])))
assert len(csv_rows) == 1
# Validate that None values are converted to empty strings in CSV
assert csv_rows[0][5] == "" # version
assert csv_rows[0][6] == "" # description
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_special_characters(
self, mock_psycopg_connection, settings
):
"""Test COPY correctly escapes special characters in CSV."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
captured = {}
def copy_side_effect(sql, file_obj):
captured["sql"] = sql
captured["data"] = file_obj.read()
cursor.copy_expert.side_effect = copy_side_effect
# Row with special characters that need escaping
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": 'framework"with"quotes',
"framework": "Framework,with,commas",
"version": "1.0",
"description": 'Description with "quotes", commas, and\nnewlines',
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
# Verify CSV was generated (csv module handles escaping automatically)
csv_rows = list(csv.reader(StringIO(captured["data"])))
assert len(csv_rows) == 1
# Verify special characters are preserved after CSV parsing
assert csv_rows[0][3] == 'framework"with"quotes'
assert csv_rows[0][4] == "Framework,with,commas"
assert "quotes" in csv_rows[0][6]
assert "commas" in csv_rows[0][6]
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_missing_inserted_at(
self, mock_psycopg_connection, settings
):
"""Test COPY uses current datetime when inserted_at is missing."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
captured = {}
def copy_side_effect(sql, file_obj):
captured["sql"] = sql
captured["data"] = file_obj.read()
cursor.copy_expert.side_effect = copy_side_effect
# Row without inserted_at field
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "test_framework",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
# Note: inserted_at is intentionally missing
}
before_call = datetime.now(timezone.utc)
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
after_call = datetime.now(timezone.utc)
csv_rows = list(csv.reader(StringIO(captured["data"])))
assert len(csv_rows) == 1
# Verify inserted_at was auto-generated and is a valid ISO datetime
inserted_at_str = csv_rows[0][2]
inserted_at = datetime.fromisoformat(inserted_at_str)
assert before_call <= inserted_at <= after_call
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_transaction_rollback_on_copy_error(
self, mock_psycopg_connection, settings
):
"""Test transaction is rolled back when copy_expert fails."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
# Simulate copy_expert failure
cursor.copy_expert.side_effect = Exception("COPY command failed")
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "test",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
with pytest.raises(Exception, match="COPY command failed"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
# Verify rollback was called
connection.rollback.assert_called_once()
connection.commit.assert_not_called()
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_transaction_rollback_on_set_config_error(
self, mock_psycopg_connection, settings
):
"""Test transaction is rolled back when SET_CONFIG fails."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
# Simulate cursor.execute failure
cursor.execute.side_effect = Exception("SET prowler.tenant_id failed")
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "test",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
with pytest.raises(Exception, match="SET prowler.tenant_id failed"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
# Verify rollback was called
connection.rollback.assert_called_once()
connection.commit.assert_not_called()
@patch("tasks.jobs.scan.psycopg_connection")
def test_copy_compliance_requirement_rows_commit_on_success(
self, mock_psycopg_connection, settings
):
"""Test transaction is committed on successful COPY."""
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
connection = MagicMock()
cursor = MagicMock()
cursor_context = MagicMock()
cursor_context.__enter__.return_value = cursor
cursor_context.__exit__.return_value = False
connection.cursor.return_value = cursor_context
connection.__enter__.return_value = connection
connection.__exit__.return_value = False
context_manager = MagicMock()
context_manager.__enter__.return_value = connection
context_manager.__exit__.return_value = False
mock_psycopg_connection.return_value = context_manager
cursor.copy_expert.return_value = None # Success
row = {
"id": uuid.uuid4(),
"tenant_id": str(uuid.uuid4()),
"compliance_id": "test",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
with patch.object(MainRouter, "admin_db", "admin"):
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
# Verify commit was called and rollback was not
connection.commit.assert_called_once()
connection.rollback.assert_not_called()
# Verify autocommit was disabled
assert connection.autocommit is False
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
def test_persist_compliance_requirement_rows_success(self, mock_copy):
"""Test successful COPY path without fallback to ORM."""
mock_copy.return_value = None # Success, no exception
tenant_id = str(uuid.uuid4())
rows = [
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": datetime.now(timezone.utc),
"compliance_id": "test",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
]
_persist_compliance_requirement_rows(tenant_id, rows)
# Verify COPY was called
mock_copy.assert_called_once_with(tenant_id, rows)
@patch("tasks.jobs.scan.logger")
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
@patch("tasks.jobs.scan.rls_transaction")
@patch(
"tasks.jobs.scan._copy_compliance_requirement_rows",
side_effect=Exception("COPY failed"),
)
def test_persist_compliance_requirement_rows_fallback_logging(
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_logger
):
"""Test logger.exception is called when COPY fails and fallback occurs."""
tenant_id = str(uuid.uuid4())
row = {
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": datetime.now(timezone.utc),
"compliance_id": "test",
"framework": "Test",
"version": "1.0",
"description": "desc",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 1,
"failed_checks": 0,
"total_checks": 1,
"scan_id": uuid.uuid4(),
}
ctx = MagicMock()
ctx.__enter__.return_value = None
ctx.__exit__.return_value = False
mock_rls_transaction.return_value = ctx
_persist_compliance_requirement_rows(tenant_id, [row])
# Verify logger.exception was called
mock_logger.exception.assert_called_once()
args, kwargs = mock_logger.exception.call_args
assert "COPY bulk insert" in args[0]
assert "falling back to ORM" in args[0]
assert kwargs.get("exc_info") is not None
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
@patch("tasks.jobs.scan.rls_transaction")
@patch(
"tasks.jobs.scan._copy_compliance_requirement_rows",
side_effect=Exception("copy failed"),
)
def test_persist_compliance_requirement_rows_fallback_multiple_rows(
self, mock_copy, mock_rls_transaction, mock_bulk_create
):
"""Test ORM fallback with multiple rows."""
tenant_id = str(uuid.uuid4())
scan_id = uuid.uuid4()
inserted_at = datetime.now(timezone.utc)
rows = [
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": "1.0",
"description": "First requirement",
"region": "us-east-1",
"requirement_id": "req-1",
"requirement_status": "PASS",
"passed_checks": 5,
"failed_checks": 0,
"total_checks": 5,
"scan_id": scan_id,
},
{
"id": uuid.uuid4(),
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "cisa_aws",
"framework": "CISA",
"version": "1.0",
"description": "Second requirement",
"region": "us-west-2",
"requirement_id": "req-2",
"requirement_status": "FAIL",
"passed_checks": 2,
"failed_checks": 3,
"total_checks": 5,
"scan_id": scan_id,
},
]
ctx = MagicMock()
ctx.__enter__.return_value = None
ctx.__exit__.return_value = False
mock_rls_transaction.return_value = ctx
_persist_compliance_requirement_rows(tenant_id, rows)
mock_copy.assert_called_once_with(tenant_id, rows)
mock_rls_transaction.assert_called_once_with(tenant_id)
mock_bulk_create.assert_called_once()
args, kwargs = mock_bulk_create.call_args
objects = args[0]
assert len(objects) == 2
assert kwargs["batch_size"] == 500
# Validate first object
assert objects[0].id == rows[0]["id"]
assert objects[0].tenant_id == rows[0]["tenant_id"]
assert objects[0].compliance_id == rows[0]["compliance_id"]
assert objects[0].framework == rows[0]["framework"]
assert objects[0].region == rows[0]["region"]
assert objects[0].passed_checks == 5
assert objects[0].failed_checks == 0
# Validate second object
assert objects[1].id == rows[1]["id"]
assert objects[1].requirement_id == rows[1]["requirement_id"]
assert objects[1].requirement_status == rows[1]["requirement_status"]
assert objects[1].passed_checks == 2
assert objects[1].failed_checks == 3
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
@patch("tasks.jobs.scan.rls_transaction")
@patch(
"tasks.jobs.scan._copy_compliance_requirement_rows",
side_effect=Exception("copy failed"),
)
def test_persist_compliance_requirement_rows_fallback_all_fields(
self, mock_copy, mock_rls_transaction, mock_bulk_create
):
"""Test ORM fallback correctly maps all fields from row dict to model."""
tenant_id = str(uuid.uuid4())
row_id = uuid.uuid4()
scan_id = uuid.uuid4()
inserted_at = datetime.now(timezone.utc)
row = {
"id": row_id,
"tenant_id": tenant_id,
"inserted_at": inserted_at,
"compliance_id": "aws_foundational_security_aws",
"framework": "AWS-Foundational-Security-Best-Practices",
"version": "2.0",
"description": "Ensure MFA is enabled",
"region": "eu-west-1",
"requirement_id": "iam.1",
"requirement_status": "FAIL",
"passed_checks": 10,
"failed_checks": 5,
"total_checks": 15,
"scan_id": scan_id,
}
ctx = MagicMock()
ctx.__enter__.return_value = None
ctx.__exit__.return_value = False
mock_rls_transaction.return_value = ctx
_persist_compliance_requirement_rows(tenant_id, [row])
args, kwargs = mock_bulk_create.call_args
objects = args[0]
assert len(objects) == 1
obj = objects[0]
# Validate ALL fields are correctly mapped
assert obj.id == row_id
assert obj.tenant_id == tenant_id
assert obj.inserted_at == inserted_at
assert obj.compliance_id == "aws_foundational_security_aws"
assert obj.framework == "AWS-Foundational-Security-Best-Practices"
assert obj.version == "2.0"
assert obj.description == "Ensure MFA is enabled"
assert obj.region == "eu-west-1"
assert obj.requirement_id == "iam.1"
assert obj.requirement_status == "FAIL"
assert obj.passed_checks == 10
assert obj.failed_checks == 5
assert obj.total_checks == 15
assert obj.scan_id == scan_id
+43
View File
@@ -0,0 +1,43 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_DESCRIPTION"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SECTION"
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
aux = data[
[
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_DESCRIPTION",
)
+36
View File
@@ -0,0 +1,36 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
+36
View File
@@ -0,0 +1,36 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
+36
View File
@@ -0,0 +1,36 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
+41
View File
@@ -0,0 +1,41 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
warnings.filterwarnings("ignore")
def get_table(data):
"""
Generate CIS OCI Foundations Benchmark v3.0 compliance table.
Args:
data: DataFrame containing compliance check results with columns:
- REQUIREMENTS_ID: CIS requirement ID (e.g., "1.1", "2.1")
- REQUIREMENTS_DESCRIPTION: Description of the requirement
- REQUIREMENTS_ATTRIBUTES_SECTION: CIS section name
- CHECKID: Prowler check identifier
- STATUS: Check status (PASS/FAIL)
- REGION: OCI region
- TENANCYID: OCI tenancy OCID
- RESOURCEID: Resource OCID or identifier
Returns:
Section containers organized by CIS sections for dashboard display
"""
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"TENANCYID",
"RESOURCEID",
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
)
+4
View File
@@ -0,0 +1,4 @@
.idea/
.git/
.claude/
AGENTS.md
+45
View File
@@ -0,0 +1,45 @@
# Prowler Documentation
This repository contains the Prowler Open Source documentation powered by [Mintlify](https://mintlify.com).
## Documentation Structure
- **Getting Started**: Overview, installation, and basic usage guides
- **User Guide**: Comprehensive guides for Prowler App, CLI, providers, and compliance
- **Developer Guide**: Technical documentation for developers contributing to Prowler
## Local Development
Install the [Mintlify CLI](https://www.npmjs.com/package/mint) to preview documentation changes locally:
```bash
npm i -g mint
```
Run the following command at the root of your documentation (where `mint.json` is located):
```bash
mint dev
```
View your local preview at `http://localhost:3000`.
## Publishing Changes
Changes pushed to the main branch are automatically deployed to production through Mintlify's GitHub integration.
## Documentation Guidelines
When contributing to the documentation, please follow the Prowler documentation style guide located in the `.claude` directory.
## Troubleshooting
- If your dev environment isn't running: Run `mint update` to ensure you have the most recent version of the CLI.
- If a page loads as a 404: Make sure you are running in a folder with a valid `mint.json` file and that the page path is correctly listed in the navigation.
## Resources
- [Prowler GitHub Repository](https://github.com/prowler-cloud/prowler)
- [Prowler Documentation](https://docs.prowler.com/)
- [Mintlify Documentation](https://mintlify.com/docs)
- [Mintlify Community](https://mintlify.com/community)
-61
View File
@@ -1,61 +0,0 @@
## Access Prowler App
After [installation](../installation/prowler-app.md), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
<img src="../../img/sign-up-button.png" alt="Sign Up Button" width="320"/>
<img src="../../img/sign-up.png" alt="Sign Up" width="285"/>
???+ note "User creation and default tenant behavior"
When creating a new user, the behavior depends on whether an invitation is provided:
- **Without an invitation**:
- A new tenant is automatically created.
- The new user is assigned to this tenant.
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
## Log In
Access Prowler App by logging in with **email and password**.
<img src="../../img/log-in.png" alt="Log In" width="285"/>
## Add Cloud Provider
Configure a cloud provider for scanning:
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
2. Select the cloud provider.
3. Enter the provider's identifier (Optional: Add an alias):
- **AWS**: Account ID
- **GCP**: Project ID
- **Azure**: Subscription ID
- **Kubernetes**: Cluster ID
- **M365**: Domain ID
4. Follow the guided instructions to add and authenticate your credentials.
## Start a Scan
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
Click `Go to Scans` to monitor progress.
## View Results
Review findings during scan execution in the following sections:
- **Overview** Provides a high-level summary of your scans.
<img src="../../products/img/overview.png" alt="Overview" width="700"/>
- **Compliance** Displays compliance insights based on security frameworks.
<img src="../../img/compliance.png" alt="Compliance" width="700"/>
> For detailed usage instructions, refer to the [Prowler App Guide](../tutorials/prowler-app.md).
???+ note
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
+3 -1
View File
@@ -1,4 +1,6 @@
# Contact Us
---
title: 'Contact Us'
---
For technical support or any type of inquiries, you are very welcome to:
@@ -1,12 +1,14 @@
# AWS Provider
---
title: 'AWS Provider'
---
In this page you can find all the details about [Amazon Web Services (AWS)](https://aws.amazon.com/) provider implementation in Prowler.
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](../tutorials/aws/getting-started-aws.md).
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](/user-guide/providers/aws/getting-started-aws).
## AWS Provider Classes Architecture
The AWS provider implementation follows the general [Provider structure](./provider.md). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the AWS provider.
The AWS provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the AWS provider.
### `AwsProvider` (Main Class)
@@ -33,7 +35,7 @@ The AWS provider implementation follows the general [Provider structure](./provi
### `AWSService` (Service Base Class)
- **Location:** [`prowler/providers/aws/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/lib/service/service.py)
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for AWS.
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for AWS.
- **Key AWS Responsibilities:**
- Receives an `AwsProvider` instance to access session, identity, and configuration.
- Manages clients for all services by regions.
@@ -52,12 +54,12 @@ The AWS provider implementation follows the general [Provider structure](./provi
## Specific Patterns in AWS Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/aws/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services)
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
### AWS Service Common Patterns
@@ -74,12 +76,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in AWS Checks
The AWS checks pattern is described in [checks page](./checks.md). You can find all the right now implemented checks:
The AWS checks pattern is described in [checks page](/developer-guide/checks). You can find all the right now implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/aws/services/s3/s3_bucket_acl_prohibited/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services/s3/s3_bucket_acl_prohibited))
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new check is following the [check creation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is following the [check creation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,12 +1,14 @@
# Azure Provider
---
title: 'Azure Provider'
---
In this page you can find all the details about [Microsoft Azure](https://azure.microsoft.com/) provider implementation in Prowler.
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](../tutorials/azure/getting-started-azure.md).
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](/user-guide/providers/azure/getting-started-azure).
## Azure Provider Classes Architecture
The Azure provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the Azure provider.
The Azure provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the Azure provider.
### `AzureProvider` (Main Class)
@@ -32,7 +34,7 @@ The Azure provider implementation follows the general [Provider structure](./pro
### `AzureService` (Service Base Class)
- **Location:** [`prowler/providers/azure/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/lib/service/service.py)
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Azure.
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Azure.
- **Key Azure Responsibilities:**
- Receives an `AzureProvider` instance to access session, identity, and configuration.
- Manages clients for all services by subscription.
@@ -50,12 +52,12 @@ The Azure provider implementation follows the general [Provider structure](./pro
## Specific Patterns in Azure Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/azure/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
### Azure Service Common Patterns
@@ -68,12 +70,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in Azure Checks
The Azure checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
The Azure checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is the [Azure check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is the [Azure check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,8 +1,10 @@
# Check Metadata Guidelines
---
title: 'Check Metadata Guidelines'
---
## Introduction
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) section.
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) section.
## Check Title Guidelines
@@ -1,4 +1,6 @@
# Prowler Checks
---
title: 'Prowler Checks'
---
This guide explains how to create new checks in Prowler.
@@ -12,7 +14,7 @@ The most common high level steps to create a new check are:
1. Prerequisites:
- Verify the check does not already exist by searching [Prowler Hub](https://hub.prowler.com) or checking `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>/`.
- Ensure required provider and service exist. If not, follow the [Provider](./provider.md) and [Service](./services.md) documentation to create them.
- Ensure required provider and service exist. If not, follow the [Provider](/developer-guide/provider) and [Service](/developer-guide/services) documentation to create them.
- Confirm the service has implemented all required methods and attributes for the check (in most cases, you will need to add or modify some methods in the service to get the data you need for the check).
2. Navigate to the service directory. The path should be as follows: `prowler/providers/<provider>/services/<service>`.
3. Create a check-specific folder. The path should follow this pattern: `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>`. Adhere to the [Naming Format for Checks](#naming-format-for-checks).
@@ -20,7 +22,7 @@ The most common high level steps to create a new check are:
5. Run the check locally to ensure it works as expected. For checking you can use the CLI in the next way:
- To ensure the check has been detected by Prowler: `poetry run python prowler-cli.py <provider> --list-checks | grep <check_name>`.
- To run the check, to find possible issues: `poetry run python prowler-cli.py <provider> --log-level ERROR --verbose --check <check_name>`.
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](./unit-testing.md) documentation.
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](/developer-guide/unit-testing) documentation.
7. If the check and its corresponding tests are working as expected, you can submit a PR to Prowler.
### Naming Format for Checks
@@ -39,8 +41,8 @@ The name components are:
Each check in Prowler follows a straightforward structure. Within the newly created folder, three files must be added to implement the check logic:
- `__init__.py` (empty file) Ensures Python treats the check folder as a package.
- `<check_name>.py` (code file) Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](./checks.md#prowlers-check-code-structure) for more information.
- `<check_name>.metadata.json` (metadata file) Defines the check's metadata for contextual information. Please refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) for more information.
- `<check_name>.py` (code file) Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](/developer-guide/checks#prowlers-check-code-structure) for more information.
- `<check_name>.metadata.json` (metadata file) Defines the check's metadata for contextual information. Please refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) for more information.
## Prowler's Check Code Structure
@@ -50,9 +52,10 @@ Below the code for a generic check is presented. It is strongly recommended to c
Report fields are the most dependent on the provider, consult the `CheckReport<Provider>` class for more information on what can be included in the report [here](https://github.com/prowler-cloud/prowler/blob/master/prowler/lib/check/models.py).
???+ note
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
<Note>
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
</Note>
```python title="Generic Check Class"
# Required Imports
# Import the base Check class and the provider-specific CheckReport class
@@ -213,7 +216,7 @@ Each check **must** populate the report with an unique identifier for the audite
### Configurable Checks in Prowler
See [Configurable Checks](./configurable-checks.md) for detailed information on making checks configurable using the `audit_config` object and configuration file.
See [Configurable Checks](/developer-guide/configurable-checks) for detailed information on making checks configurable using the `audit_config` object and configuration file.
## Metadata Structure for Prowler Checks
@@ -273,16 +276,17 @@ The `CheckTitle` field must be plain text, clearly and succinctly define **the b
**Always write the `CheckTitle` to describe the *PASS* case**, the desired secure or compliant state of the resource(s). This helps ensure that findings are easy to interpret and that the title always reflects the best practice being met.
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](./check-metadata-guidelines.md#check-title-guidelines).
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](/developer-guide/check-metadata-guidelines#check-title-guidelines).
#### CheckType
???+ warning
This field is only applicable to the AWS provider.
<Warning>
This field is only applicable to the AWS provider.
</Warning>
It follows the [AWS Security Hub Types](https://docs.aws.amazon.com/securityhub/latest/userguide/asff-required-attributes.html#Types) format using the pattern `namespace/category/classifier`.
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](./check-metadata-guidelines.md#check-type-guidelines-aws-only).
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](/developer-guide/check-metadata-guidelines#check-type-guidelines-aws-only).
#### ServiceName
@@ -314,13 +318,13 @@ The type of resource being audited. This field helps categorize and organize fin
A concise, natural language explanation that **clearly describes what the finding means**, focusing on clarity and context rather than technical implementation details. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. This field is limited to maximum 400 characters.
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](./check-metadata-guidelines.md#description-guidelines).
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](/developer-guide/check-metadata-guidelines#description-guidelines).
#### Risk
A clear, natural language explanation of **why this finding poses a cybersecurity risk**. Focus on how it may impact confidentiality, integrity, or availability. If those do not apply, describe any relevant operational or financial risks. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. Limit your explanation to 400 characters.
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](./check-metadata-guidelines.md#risk-guidelines).
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](/developer-guide/check-metadata-guidelines#risk-guidelines).
#### RelatedUrl
@@ -328,9 +332,10 @@ For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelin
#### AdditionalURLs
???+ warning
URLs must be valid and not repeated.
<Warning>
URLs must be valid and not repeated.
</Warning>
A list of official documentation URLs for further reading. These should be authoritative sources that provide additional context, best practices, or detailed information about the security control being checked. Prefer official provider documentation, security standards, or well-established security resources. Avoid third-party blogs or unofficial sources unless they are highly reputable and directly relevant.
#### Remediation
@@ -345,17 +350,17 @@ Provides both code examples and best practice recommendations for addressing the
- **Terraform**: HashiCorp Configuration Language (HCL) code with an example of a compliant configuration.
- **Other**: Manual steps through web interfaces or other tools to make the finding compliant.
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](./check-metadata-guidelines.md#remediation-code-guidelines).
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](/developer-guide/check-metadata-guidelines#remediation-code-guidelines).
- **Recommendation**
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](./check-metadata-guidelines.md#recommendation-guidelines).
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](/developer-guide/check-metadata-guidelines#recommendation-guidelines).
- **Url**: [Prowler Hub URL](https://hub.prowler.com/) of the check. This URL is always composed by `https://hub.prowler.com/check/<check_id>`.
#### Categories
One or more functional groupings used for execution filtering (e.g., `internet-exposed`). You can define new categories just by adding to this field.
For the complete list of available categories, see [Categories Guidelines](./check-metadata-guidelines.md#categories-guidelines).
For the complete list of available categories, see [Categories Guidelines](/developer-guide/check-metadata-guidelines#categories-guidelines).
#### DependsOn
@@ -1,4 +1,6 @@
# Configurable Checks in Prowler
---
title: 'Configurable Checks in Prowler'
---
Prowler empowers users to extend and adapt cloud security coverage by making checks configurable through the use of the `audit_config` object. This approach enables customization of checks to meet specific requirements through a configuration file.
@@ -41,6 +43,6 @@ When adding a new configurable check to Prowler, update the following files:
- **Test Fixtures:** If tests depend on this configuration, add the variable to `tests/config/fixtures/config.yaml`.
- **Documentation:** Document the new variable in the list of configurable checks in `docs/tutorials/configuration_file.md`.
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](../tutorials/configuration_file.md).
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](/user-guide/cli/tutorials/configuration_file).
This approach ensures that checks are easily configurable, making Prowler highly adaptable to different environments and requirements.
@@ -1,4 +1,6 @@
# Debugging in Prowler
---
title: 'Debugging in Prowler'
---
Debugging in Prowler simplifies the development process, allowing developers to efficiently inspect and resolve unexpected issues during execution.
-28
View File
@@ -1,28 +0,0 @@
## Contributing to Documentation
Prowler documentation is built using `mkdocs`, allowing contributors to easily add or enhance documentation.
### Installation and Setup
Install all necessary dependencies using: `poetry install --with docs`.
1. Install `mkdocs` using your preferred package manager.
2. Running the Documentation Locally
Navigate to the `prowler` repository folder.
Start the local documentation server by running: `mkdocs serve`.
Open `http://localhost:8000` in your browser to view live updates.
3. Making Documentation Changes
Make all needed changes to docs or add new documents. Edit existing Markdown (.md) files inside `prowler/docs`.
To add new sections or files, update the `mkdocs.yaml` file located in the root directory of Prowlers repository.
4. Submitting Changes
Once documentation updates are complete:
Submit a pull request for review.
The Prowler team will assess and merge contributions.
Your efforts help improve Prowler documentation—thank you for contributing!
+42
View File
@@ -0,0 +1,42 @@
---
title: 'Contributing to Documentation'
---
Prowler documentation is built using [Mintlify](https://www.mintlify.com/docs), allowing contributors to easily add or enhance documentation.
## Installation and Setup
<Steps>
<Step title="Install Mintlify CLI">
```bash
npm i -g mint
```
For detailed instructions, check the [Mintlify documentation](https://www.mintlify.com/docs/installation).
</Step>
<Step title="Preview Documentation Locally">
Start the local development server to preview changes in real-time.
```bash
mint dev
```
A local preview of your documentation will be available at http://localhost:3000
</Step>
<Step title="Make Documentation Changes">
Edit existing Markdown (.mdx) files inside the `docs` directory or add new documents.
For reference about formatting, check the [Mintlify documentation](https://www.mintlify.com/docs/create/text).
To add new sections or files, update the [`docs/docs.json`](https://github.com/prowler-cloud/prowler/blob/master/docs/docs.json) file to include them in the navigation.
</Step>
<Step title="Submit Changes">
Once documentation updates are complete, submit a pull request for review.
The Prowler team will assess and merge contributions.
</Step>
</Steps>
Your efforts help improve Prowler documentation—thank you for contributing!
@@ -1,12 +1,14 @@
# Google Cloud Provider
---
title: 'Google Cloud Provider'
---
This page details the [Google Cloud Platform (GCP)](https://cloud.google.com/) provider implementation in Prowler.
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](../tutorials/gcp/getting-started-gcp.md).
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](/user-guide/providers/gcp/getting-started-gcp).
## GCP Provider Classes Architecture
The GCP provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
The GCP provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
### Main Class
@@ -32,7 +34,7 @@ The GCP provider implementation follows the general [Provider structure](./provi
### `GCPService` (Service Base Class)
- **Location:** [`prowler/providers/gcp/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/lib/service/service.py)
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for GCP.
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for GCP.
- **Key GCP Responsibilities:**
- Receives a `GcpProvider` instance to access session, identity, and configuration.
- Manages clients for all services by project.
@@ -95,12 +97,12 @@ def _get_instances(self):
## Specific Patterns in GCP Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/gcp/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
### GCP Service Common Patterns
@@ -117,12 +119,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in GCP Checks
The GCP checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
The GCP checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,12 +1,14 @@
# GitHub Provider
---
title: 'GitHub Provider'
---
This page details the [GitHub](https://github.com/) provider implementation in Prowler.
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](../tutorials/github/getting-started-github.md).
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](/user-guide/providers/github/getting-started-github).
## GitHub Provider Classes Architecture
The GitHub provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
The GitHub provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
### `GithubProvider` (Main Class)
@@ -48,12 +50,12 @@ The GitHub provider implementation follows the general [Provider structure](./pr
## Specific Patterns in GitHub Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/github/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
### GitHub Service Common Patterns
@@ -66,12 +68,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in GitHub Checks
The GitHub checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
The GitHub checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/github/services/repository/repository_secret_scanning_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services/repository/repository_secret_scanning_enabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
### Check Report Class
@@ -1,3 +0,0 @@
# Integration Tests
Coming soon ...
@@ -0,0 +1,5 @@
---
title: 'Integration Tests'
---
Coming soon ...
@@ -1,4 +1,6 @@
# Creating a New Integration
---
title: 'Creating a New Integration'
---
## Introduction
@@ -151,7 +153,7 @@ Refer to the [Prowler Developer Guide](https://docs.prowler.com/projects/prowler
# More properties and methods
```
* Test Connection Method:
* Validating Credentials or Tokens
@@ -1,4 +1,6 @@
# Introduction to developing in Prowler
---
title: 'Introduction to developing in Prowler'
---
Extending Prowler
@@ -48,11 +50,12 @@ poetry install --with dev
eval $(poetry env activate)
```
???+ important
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
<Warning>
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
</Warning>
## Contributing to Prowler
### Ways to Contribute
@@ -64,24 +67,24 @@ Here are some ideas for collaborating with Prowler:
2. **Expand Prowler's Capabilities**: Prowler is constantly evolving, and you can be a part of its growth. Whether you are adding checks, supporting new services, or introducing integrations, your contributions help improve the tool for everyone. Here is how you can get involved:
- **Adding New Checks**
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](./checks.md).
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](/developer-guide/checks).
- **Adding New Services**
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](./services.md). Do not forget to include relevant checks to validate functionality.
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](/developer-guide/services). Do not forget to include relevant checks to validate functionality.
- **Adding New Providers**
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](./provider.md). This typically involves setting up new services and checks to ensure compatibility.
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](/developer-guide/provider). This typically involves setting up new services and checks to ensure compatibility.
- **Adding New Output Formats**
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](./outputs.md).
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](/developer-guide/outputs).
- **Adding New Integrations**
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](./integrations.md).
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](/developer-guide/integrations).
- **Proposing or Implementing Features**
Got an idea to make Prowler better? Whether it is a brand-new feature or an enhancement to an existing one, you are welcome to propose it or help implement community-requested improvements.
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](./documentation.md).
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](/developer-guide/documentation).
4. **Bug Fixes**: If you find any issues or bugs, you can report them in the [GitHub Issues](https://github.com/prowler-cloud/prowler/issues) page and if you want you can also fix them.
@@ -105,9 +108,10 @@ pre-commit installed at .git/hooks/pre-commit
Before merging pull requests, several automated checks and utilities ensure code security and updated dependencies:
???+ note
These should have been already installed if `poetry install --with dev` was already run.
<Note>
These should have been already installed if `poetry install --with dev` was already run.
</Note>
- [`bandit`](https://pypi.org/project/bandit/) for code security review.
- [`safety`](https://pypi.org/project/safety/) and [`dependabot`](https://github.com/features/security) for dependencies.
- [`hadolint`](https://github.com/hadolint/hadolint) and [`dockle`](https://github.com/goodwithtech/dockle) for container security.
@@ -123,9 +127,10 @@ All dependencies are listed in the `pyproject.toml` file.
For proper code documentation, refer to the following and follow the code documentation practices presented there: [Google Python Style Guide - Comments and Docstrings](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings).
???+ note
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
<Note>
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
</Note>
### Repository Folder Structure
Understanding the layout of the Prowler codebase will help you quickly find where to add new features, checks, or integrations. The following is a high-level overview from the root of the repository:
@@ -173,4 +178,4 @@ To test Prowler from a specific branch (for example, to try out changes from a p
pipx install "git+https://github.com/prowler-cloud/prowler.git@branch-name"
```
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
@@ -1,12 +1,14 @@
# Kubernetes Provider
---
title: 'Kubernetes Provider'
---
This page details the [Kubernetes](https://kubernetes.io/) provider implementation in Prowler.
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](../tutorials/kubernetes/in-cluster.md) or [Non In-Cluster Execution](../tutorials/kubernetes/outside-cluster.md) guides.
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](/user-guide/providers/kubernetes/in-cluster) or [Non In-Cluster Execution](/user-guide/providers/kubernetes/outside-cluster) guides.
## Kubernetes Provider Classes Architecture
The Kubernetes provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
The Kubernetes provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
### `KubernetesProvider` (Main Class)
@@ -31,7 +33,7 @@ The Kubernetes provider implementation follows the general [Provider structure](
### `KubernetesService` (Service Base Class)
- **Location:** [`prowler/providers/kubernetes/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/lib/service/service.py)
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Kubernetes.
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Kubernetes.
- **Key Kubernetes Responsibilities:**
- Receives a `KubernetesProvider` instance to access session, identity, and configuration.
- Manages the Kubernetes API client and context.
@@ -50,12 +52,12 @@ The Kubernetes provider implementation follows the general [Provider structure](
## Specific Patterns in Kubernetes Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/kubernetes/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other already implemented services as reference.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other already implemented services as reference.
### Kubernetes Service Common Patterns
@@ -69,12 +71,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in Kubernetes Checks
The Kubernetes checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
The Kubernetes checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](./checks.md#creating-a-check) and taking other checks as reference.
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](/developer-guide/checks#creating-a-check) and taking other checks as reference.
### Check Report Class
@@ -1,4 +1,6 @@
# Extending Prowler Lighthouse AI
---
title: 'Extending Prowler Lighthouse AI'
---
This guide helps developers customize and extend Prowler Lighthouse AI by adding or modifying AI agents.
@@ -15,9 +17,10 @@ AI agents fall into two main categories:
Prowler Lighthouse AI is an autonomous agent - selecting the right tool(s) based on the users query.
???+ note
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
<Note>
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
</Note>
### LLM Dependency
The autonomous nature of agents depends on the underlying LLM. Autonomous agents using identical system prompts and tools but powered by different LLM providers might approach user queries differently. Agent with one LLM might solve a problem efficiently, while with another it might take a different route or fail entirely.
@@ -30,7 +33,7 @@ Prowler Lighthouse AI uses a multi-agent architecture orchestrated by the [Langg
### Architecture Components
<img src="../../tutorials/img/lighthouse-architecture.png" alt="Prowler Lighthouse architecture">
<img src="/images/prowler-app/lighthouse-architecture.png" alt="Prowler Lighthouse architecture" />
Prowler Lighthouse AI integrates with the NextJS application:
@@ -67,9 +70,10 @@ Modifying the supervisor prompt allows you to:
- Modify task delegation to specialized agents
- Set up guardrails (query types to answer or decline)
???+ note
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
<Note>
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
</Note>
### How to Create New Specialized Agents
The supervisor agent and all specialized agents are defined in the `route.ts` file. The supervisor agent uses [langgraph-supervisor](https://www.npmjs.com/package/@langchain/langgraph-supervisor), while other agents use the prebuilt [create-react-agent](https://langchain-ai.github.io/langgraphjs/how-tos/create-react-agent/).
@@ -77,14 +81,16 @@ The supervisor agent and all specialized agents are defined in the `route.ts` fi
To add new capabilities or all Lighthouse AI to interact with other APIs, create additional specialized agents:
1. First determine what the new agent would do. Create a detailed prompt defining the agent's purpose and capabilities. You can see an example from [here](https://github.com/prowler-cloud/prowler/blob/master/ui/lib/lighthouse/prompts.ts#L359-L385).
???+ note
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
<Note>
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
</Note>
2. Create necessary tools for the agents to access specific data or perform actions. A tool is a specialized function that extends the capabilities of LLM by allowing it to access external data or APIs. A tool is triggered by LLM based on the description of the tool and the user's query.
For example, the description of `getScanTool` is "Fetches detailed information about a specific scan by its ID." If the description doesn't convey what the tool is capable of doing, LLM will not invoke the function. If the description of `getScanTool` was set to something random or not set at all, LLM will not answer queries like "Give me the critical issues from the scan ID xxxxxxxxxxxxxxx"
???+ note
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
<Note>
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
</Note>
3. Use the `createReactAgent` function to define a new agent. For example, the rolesAgent name is "roles_agent" and has access to call tools "*getRolesTool*" and "*getRoleTool*"
```js
const rolesAgent = createReactAgent({
@@ -1,12 +1,14 @@
# LLM Provider
---
title: 'LLM Provider'
---
This page details the [Large Language Model (LLM)](https://en.wikipedia.org/wiki/Large_language_model) provider implementation in Prowler.
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](../tutorials/llm/getting-started-llm.md).
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](/user-guide/providers/llm/getting-started-llm).
## LLM Provider Classes Architecture
The LLM provider implementation follows the general [Provider structure](./provider.md). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
The LLM provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
### Main Class
@@ -1,8 +1,10 @@
# Microsoft 365 (M365) Provider
---
title: 'Microsoft 365 (M365) Provider'
---
This page details the [Microsoft 365 (M365)](https://www.microsoft.com/en-us/microsoft-365) provider implementation in Prowler.
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](../tutorials/microsoft365/getting-started-m365.md).
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](/user-guide/providers/microsoft365/getting-started-m365).
---
@@ -15,14 +17,14 @@ By default, Prowler will audit the Microsoft Entra ID tenant and its supported s
- **Required modules:**
- [ExchangeOnlineManagement](https://www.powershellgallery.com/packages/ExchangeOnlineManagement/3.6.0) (≥ 3.6.0)
- [MicrosoftTeams](https://www.powershellgallery.com/packages/MicrosoftTeams/6.6.0) (≥ 6.6.0)
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](../tutorials/microsoft365/authentication.md#supported-powershell-versions) and [Needed PowerShell Modules](../tutorials/microsoft365/authentication.md#required-powershell-modules).
- For more details and troubleshooting, see [Use of PowerShell in M365](../tutorials/microsoft365/use-of-powershell.md).
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](/user-guide/providers/microsoft365/authentication#supported-powershell-versions) and [Needed PowerShell Modules](/user-guide/providers/microsoft365/authentication#required-powershell-modules).
- For more details and troubleshooting, see [Use of PowerShell in M365](/user-guide/providers/microsoft365/use-of-powershell).
---
## M365 Provider Classes Architecture
The M365 provider implementation follows the general [Provider structure](./provider.md). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
The M365 provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
### `M365Provider` (Main Class)
@@ -73,12 +75,12 @@ The M365 provider implementation follows the general [Provider structure](./prov
## Specific Patterns in M365 Services
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/m365/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is by following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
The best reference to understand how to implement a new service is by following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
### M365 Service Common Patterns
@@ -92,12 +94,12 @@ The best reference to understand how to implement a new service is by following
## Specific Patterns in M365 Checks
The M365 checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
The M365 checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/m365/services/entra/entra_users_mfa_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services/entra/entra_users_mfa_enabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
### Check Report Class
@@ -1,4 +1,6 @@
# Create a Custom Output Format
---
title: 'Create a Custom Output Format'
---
## Introduction
@@ -1,4 +1,6 @@
# Prowler Providers
---
title: 'Prowler Providers'
---
## Introduction
@@ -14,9 +16,10 @@ A provider is any platform or service that offers resources, data, or functional
For providers supported by Prowler, refer to [Prowler Hub](https://hub.prowler.com/).
???+ important
There are some custom providers added by the community, like [NHN Cloud](https://www.nhncloud.com/), that are not maintained by the Prowler team, but can be used in the Prowler CLI. They can be checked directly at the [Prowler GitHub repository](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
<Warning>
There are some custom providers added by the community, like [NHN Cloud](https://www.nhncloud.com/), that are not maintained by the Prowler team, but can be used in the Prowler CLI. They can be checked directly at the [Prowler GitHub repository](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
</Warning>
## Adding a New Provider
To integrate an unsupported Prowler provider and implement its security checks, create a dedicated folder for all related files (e.g., services, checks)."
@@ -31,7 +34,7 @@ Within this folder the following folders are also to be created:
- `arguments/arguments.py` Handles provider-specific argument parsing.
- `mutelist/mutelist.py` Manages the mutelist functionality for the provider.
- `services` Stores all [services](./services.md) that the provider offers and want to be audited by [Prowler checks](./checks.md).
- `services` Stores all [services](/developer-guide/services) that the provider offers and want to be audited by [Prowler checks](/developer-guide/checks).
- `__init__.py` (empty) Ensures Python recognizes this folder as a package.
@@ -41,9 +44,10 @@ Within this folder the following folders are also to be created:
By adhering to this structure, Prowler can effectively support services and security checks for additional providers.
???+ important
If your new provider requires a Python library (such as an official SDK or API client) to connect to its services, make sure to add it as a dependency in the `pyproject.toml` file. This ensures that all contributors and users have the necessary packages installed when working with your provider.
<Warning>
If your new provider requires a Python library (such as an official SDK or API client) to connect to its services, make sure to add it as a dependency in the `pyproject.toml` file. This ensures that all contributors and users have the necessary packages installed when working with your provider.
</Warning>
## Provider Structure in Prowler
Prowler's provider architecture is designed to facilitate security audits through a generic service tailored to each provider. This is accomplished by passing the necessary parameters to the constructor, which initializes all required session values.
@@ -1,4 +1,6 @@
# Renaming Checks in Prowler
---
title: 'Renaming Checks in Prowler'
---
To rename a check in Prowler, follow these steps when aligning with Check ID structure, fixing typos, or updating check logic that requires a new name.
@@ -1,4 +1,6 @@
# Creating a New Security Compliance Framework in Prowler
---
title: 'Creating a New Security Compliance Framework in Prowler'
---
## Introduction
@@ -1,15 +1,18 @@
# Prowler Services
---
title: 'Prowler Services'
---
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](./provider.md).
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](/developer-guide/provider).
???+note
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](./provider.md) documentation to create it from scratch.
<Note>
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](/developer-guide/provider) documentation to create it from scratch.
</Note>
## Introduction
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](./provider.md), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](/developer-guide/provider), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](./checks.md) to generate the security findings.
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](/developer-guide/checks) to generate the security findings.
## Adding a New Service
@@ -159,9 +162,10 @@ class <Service>(ServiceParentClass):
)
```
???+note
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
<Note>
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
</Note>
#### Resource Models
Resource models define structured classes used within services to store and process data extracted from API calls. They are defined in the same file as the service class, but outside of the class, usually at the bottom of the file.
@@ -231,11 +235,11 @@ Before implementing a new service, verify that Prowler's existing permissions fo
Provider-Specific Permissions Documentation:
- [AWS](../tutorials/aws/authentication.md#required-permissions)
- [Azure](../tutorials/azure/authentication.md#required-permissions)
- [GCP](../tutorials/gcp/authentication.md#required-permissions)
- [M365](../tutorials/microsoft365/authentication.md#required-permissions)
- [GitHub](../tutorials/github/authentication.md)
- [AWS](/user-guide/providers/aws/authentication#required-permissions)
- [Azure](/user-guide/providers/azure/authentication#required-permissions)
- [GCP](/user-guide/providers/gcp/authentication#required-permissions)
- [M365](/user-guide/providers/microsoft365/authentication#required-permissions)
- [GitHub](/user-guide/providers/github/authentication)
## Best Practices
@@ -1,4 +1,6 @@
# Unit Tests for Prowler Checks
---
title: 'Unit Tests for Prowler Checks'
---
Unit tests for Prowler checks vary based on the provider being evaluated.
@@ -39,7 +41,7 @@ To execute the Prowler test suite, install the necessary dependencies listed in
### Prerequisites
If you have not installed Prowler yet, refer to the [developer guide introduction](./introduction.md#getting-the-code-and-installing-all-dependencies).
If you have not installed Prowler yet, refer to the [developer guide introduction](/developer-guide/introduction#getting-the-code-and-installing-all-dependencies).
### Executing Tests
@@ -57,16 +59,18 @@ Other Commands for Running Tests
- Running tests for a provider check:
`pytest -n auto -vvv -s -x tests/providers/<provider>/services/<service>/<check>`
???+ note
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
<Note>
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
</Note>
## AWS Testing Approaches
For AWS provider, different testing approaches apply based on API coverage based on several criteria.
???+ note
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
<Note>
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
</Note>
- AWS API Calls Covered by [Moto](https://github.com/getmoto/moto):
- Service Tests: `@mock_aws`
- Checks Tests: `@mock_aws`
@@ -205,12 +209,14 @@ class Test_iam_password_policy_uppercase:
If the IAM service required for testing is not supported by the Moto library, use [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock) to inject objects into the service client.
???+ warning
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
<Warning>
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
???+ note
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
</Warning>
<Note>
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
</Note>
#### Mocking Service Objects Using MagicMock
The following code demonstrates how to use MagicMock to create service objects.
@@ -377,13 +383,15 @@ class Test_iam_password_policy_uppercase:
# Refer to the previous section for the check test, as the implementation remains unchanged.
```
???+ note
This example does not use Moto to simplify the setup.
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
<Note>
This example does not use Moto to simplify the setup.
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
???+ note
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
</Note>
<Note>
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
</Note>
#### Mocking Several Services
Since the provider is being mocked, multiple attributes can be configured to customize its behavior:
@@ -488,7 +496,7 @@ will cause that the service will be initialised twice:
Later, when importing `<service>_client.py` at `<check>.py`, Python uses the mocked instance since the patch was applied at the correct reference point.
In the [next section](./unit-testing.md#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
In the [next section](/developer-guide/unit-testing#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
##### Mocking the Service and the Service Client at the Service Client Level
@@ -642,9 +650,10 @@ class Test_compute_project_os_login_enabled:
The testing of Google Cloud Services follows the same principles as the one of Google Cloud checks. While all API calls must be mocked, attribute setup for API calls in this scenario is defined in the fixtures file, specifically within the [fixtures file](https://github.com/prowler-cloud/prowler/blob/master/tests/providers/gcp/gcp_fixtures.py) in the `mock_api_client` function.
???+ important
Every method within a service must be tested to ensure full coverage and accurate validation.
<Warning>
Every method within a service must be tested to ensure full coverage and accurate validation.
</Warning>
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
```python title="BigQuery Service Test"
@@ -907,9 +916,12 @@ class Test_app_ensure_http_is_redirected_to_https:
The testing of Azure Services follows the same principles as the one of Google Cloud checks. All API calls are still mocked, but for methods that initialize attributes via an API call, use the [patch](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch) decorator at the beginning of the class to ensure proper mocking.
???+ important "Remember"
Every method within a service must be tested to ensure full coverage and accurate validation.
<Warning>
**Remember**
Every method within a service must be tested to ensure full coverage and accurate validation.
</Warning>
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
```python title="AppInsights Service Test"
+433
View File
@@ -0,0 +1,433 @@
{
"$schema": "https://mintlify.com/docs.json",
"theme": "mint",
"name": "Prowler Documentation",
"colors": {
"primary": "#000000",
"light": "#10B981",
"dark": "#10B981"
},
"favicon": "/favicon.ico",
"logo": {
"dark": "/images/prowler-logo-white.png",
"light": "/images/prowler-logo-black.png"
},
"navigation": {
"tabs": [
{
"tab": "Getting Started",
"groups": [
{
"group": "Welcome",
"pages": [
"introduction"
]
},
{
"group": "Prowler Cloud",
"pages": [
"getting-started/products/prowler-cloud",
"getting-started/products/prowler-cloud-pricing",
"getting-started/products/prowler-cloud-aws-marketplace",
"getting-started/goto/prowler-cloud",
"getting-started/goto/prowler-api-reference"
]
},
{
"group": "Prowler CLI",
"pages": [
"getting-started/products/prowler-cli",
"getting-started/installation/prowler-cli",
"getting-started/basic-usage/prowler-cli"
]
},
{
"group": "Prowler App",
"pages": [
"getting-started/products/prowler-app",
"getting-started/installation/prowler-app",
"getting-started/basic-usage/prowler-app"
]
},
{
"group": "Prowler Lighthouse AI",
"pages": [
"user-guide/tutorials/prowler-app-lighthouse"
]
},
{
"group": "Prowler MCP Server",
"pages": [
"getting-started/products/prowler-mcp",
"getting-started/installation/prowler-mcp",
"getting-started/basic-usage/prowler-mcp",
"getting-started/basic-usage/prowler-mcp-tools"
]
},
{
"group": "Prowler Hub",
"pages": [
"getting-started/products/prowler-hub",
"getting-started/goto/prowler-hub"
]
},
{
"group": "Prowler vs. Others",
"pages": [
"getting-started/comparison/index",
"getting-started/comparison/awssecurityhub",
"getting-started/comparison/gcp",
"getting-started/comparison/microsoftdefender",
"getting-started/comparison/microsoftsentinel"
]
}
]
},
{
"tab": "Guides",
"groups": [
{
"group": "Prowler Cloud/App",
"pages": [
"user-guide/tutorials/prowler-app",
{
"group": "Authentication",
"pages": [
"user-guide/tutorials/prowler-app-social-login",
"user-guide/tutorials/prowler-app-sso"
]
},
"user-guide/tutorials/prowler-app-rbac",
"user-guide/providers/prowler-app-api-keys",
"user-guide/tutorials/prowler-app-mute-findings",
{
"group": "Integrations",
"expanded": true,
"pages": [
"user-guide/tutorials/prowler-app-s3-integration",
"user-guide/tutorials/prowler-app-security-hub-integration",
"user-guide/tutorials/prowler-app-jira-integration"
]
},
"user-guide/tutorials/prowler-app-lighthouse",
{
"group": "Tutorials",
"pages": [
"user-guide/tutorials/prowler-app-sso-entra",
"user-guide/tutorials/bulk-provider-provisioning",
"user-guide/tutorials/aws-organizations-bulk-provisioning"
]
}
]
},
{
"group": "CLI",
"pages": [
"user-guide/cli/tutorials/misc",
"user-guide/cli/tutorials/reporting",
"user-guide/cli/tutorials/compliance",
"user-guide/cli/tutorials/dashboard",
"user-guide/cli/tutorials/configuration_file",
"user-guide/cli/tutorials/logging",
"user-guide/cli/tutorials/mutelist",
{
"group": "Integrations",
"pages": [
"user-guide/providers/aws/securityhub",
"user-guide/cli/tutorials/integrations",
"user-guide/providers/aws/s3"
]
},
"user-guide/cli/tutorials/fixer",
"user-guide/cli/tutorials/check-aliases",
"user-guide/cli/tutorials/custom-checks-metadata",
"user-guide/cli/tutorials/pentesting",
"user-guide/cli/tutorials/scan-unused-services",
"user-guide/cli/tutorials/quick-inventory",
{
"group": "Tutorials",
"pages": [
"user-guide/cli/tutorials/parallel-execution"
]
}
]
},
{
"group": "Providers",
"pages": [
{
"group": "AWS",
"pages": [
"user-guide/providers/aws/getting-started-aws",
"user-guide/providers/aws/authentication",
"user-guide/providers/aws/role-assumption",
"user-guide/providers/aws/organizations",
"user-guide/providers/aws/regions-and-partitions",
"user-guide/providers/aws/tag-based-scan",
"user-guide/providers/aws/resource-arn-based-scan",
"user-guide/providers/aws/boto3-configuration",
"user-guide/providers/aws/threat-detection",
"user-guide/providers/aws/cloudshell",
"user-guide/providers/aws/multiaccount"
]
},
{
"group": "Azure",
"pages": [
"user-guide/providers/azure/getting-started-azure",
"user-guide/providers/azure/authentication",
"user-guide/providers/azure/use-non-default-cloud",
"user-guide/providers/azure/subscriptions",
"user-guide/providers/azure/create-prowler-service-principal"
]
},
{
"group": "Google Cloud",
"pages": [
"user-guide/providers/gcp/getting-started-gcp",
"user-guide/providers/gcp/authentication",
"user-guide/providers/gcp/projects",
"user-guide/providers/gcp/organization",
"user-guide/providers/gcp/retry-configuration"
]
},
{
"group": "Kubernetes",
"pages": [
"user-guide/providers/kubernetes/in-cluster",
"user-guide/providers/kubernetes/outside-cluster",
"user-guide/providers/kubernetes/misc"
]
},
{
"group": "Microsoft 365",
"pages": [
"user-guide/providers/microsoft365/getting-started-m365",
"user-guide/providers/microsoft365/authentication",
"user-guide/providers/microsoft365/use-of-powershell"
]
},
{
"group": "GitHub",
"pages": [
"user-guide/providers/github/getting-started-github",
"user-guide/providers/github/authentication"
]
},
{
"group": "IaC",
"pages": [
"user-guide/providers/iac/getting-started-iac",
"user-guide/providers/iac/authentication"
]
},
{
"group": "MongoDB Atlas",
"pages": [
"user-guide/providers/mongodbatlas/getting-started-mongodbatlas",
"user-guide/providers/mongodbatlas/authentication"
]
},
{
"group": "LLM",
"pages": [
"user-guide/providers/llm/getting-started-llm"
]
},
{
"group": "Oracle Cloud Infrastructure",
"pages": [
"user-guide/providers/oci/getting-started-oci",
"user-guide/providers/oci/authentication"
]
}
]
},
{
"group": "Compliance",
"pages": [
"user-guide/compliance/tutorials/threatscore"
]
}
]
},
{
"tab": "Developer Guide",
"groups": [
{
"group": "Concepts",
"pages": [
"developer-guide/introduction",
"developer-guide/provider",
"developer-guide/services",
"developer-guide/checks",
"developer-guide/outputs",
"developer-guide/integrations",
"developer-guide/security-compliance-framework",
"developer-guide/lighthouse"
]
},
{
"group": "Providers",
"pages": [
"developer-guide/aws-details",
"developer-guide/azure-details",
"developer-guide/gcp-details",
"developer-guide/kubernetes-details",
"developer-guide/m365-details",
"developer-guide/github-details",
"developer-guide/llm-details"
]
},
{
"group": "Miscellaneous",
"pages": [
"developer-guide/documentation",
{
"group": "Testing",
"pages": [
"developer-guide/unit-testing",
"developer-guide/integration-testing"
]
},
"developer-guide/debugging",
"developer-guide/configurable-checks",
"developer-guide/renaming-checks",
"developer-guide/check-metadata-guidelines"
]
}
]
},
{
"tab": "Security",
"pages": [
"security"
]
},
{
"tab": "Contact Us",
"pages": [
"contact"
]
},
{
"tab": "Troubleshooting",
"pages": [
"troubleshooting"
]
},
{
"tab": "About Us",
"icon": "/favicon.ico",
"href": "https://prowler.com/about#team"
},
{
"tab": "Changelog",
"icon": "github",
"href": "https://github.com/prowler-cloud/prowler/releases"
},
{
"tab": "Public Roadmap",
"href": "https://roadmap.prowler.com/"
}
],
"global": {
"anchors": [
{
"anchor": "GitHub",
"href": "https://github.com/prowler-cloud/prowler",
"icon": "github"
},
{
"anchor": "Slack",
"href": "https://goto.prowler.com/slack",
"icon": "slack"
},
{
"anchor": "YouTube",
"href": "https://www.youtube.com/@prowlercloud",
"icon": "youtube"
}
]
}
},
"navbar": {
"links": [
{
"label": "Prowler Hub",
"href": "https://hub.prowler.com"
},
{
"label": "Prowler Cloud",
"href": "https://cloud.prowler.com",
"style": "primary"
}
]
},
"analytics": {
"ga4": {
"measurementId": "G-KBKV70W5Y2"
}
},
"feedback": {
"thumbsRating": true,
"suggestEdit": true,
"raiseIssue": true
},
"footer": {
"socials": {
"x-twitter": "https://x.com/prowlercloud",
"github": "https://github.com/prowler-cloud/prowler",
"linkedin": "https://www.linkedin.com/company/prowler-security",
"youtube": "https://www.youtube.com/@prowlercloud",
"slack": "https://goto.prowler.com/slack",
"website": "https://prowler.com"
}
},
"redirects": [
{
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-lighthouse",
"destination": "/user-guide/tutorials/prowler-app-lighthouse"
},
{
"source": "/projects/prowler-open-source/en/latest/developer-guide/introduction",
"destination": "/developer-guide/introduction"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/aws/getting-started-aws",
"destination": "/user-guide/providers/aws/getting-started-aws"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/azure/getting-started-azure",
"destination": "/user-guide/providers/azure/getting-started-azure"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/gcp/getting-started-gcp",
"destination": "/user-guide/providers/gcp/getting-started-gcp"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/microsoft365/getting-started-m365",
"destination": "/user-guide/providers/microsoft365/getting-started-m365"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/github/getting-started-github",
"destination": "/user-guide/providers/github/getting-started-github"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-sso",
"destination": "/user-guide/tutorials/prowler-app-sso"
},
{
"source": "/projects/prowler-open-source/en/latest",
"destination": "/introduction"
},
{
"source": "/projects/prowler-saas/en/latest/:slug*",
"destination": "https://docs.prowler.pro/en/latest/:slug*"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/:slug*",
"destination": "/user-guide/tutorials/:slug*"
}
]
}
@@ -0,0 +1,70 @@
---
title: 'Basic Usage'
---
## Access Prowler App
After [installation](/getting-started/installation/prowler-app), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
<img src="/images/sign-up-button.png" alt="Sign Up Button" width="320" />
<img src="/images/sign-up.png" alt="Sign Up" width="285" />
<Note>
**User creation and default tenant behavior**
When creating a new user, the behavior depends on whether an invitation is provided:
- **Without an invitation**:
- A new tenant is automatically created.
- The new user is assigned to this tenant.
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
</Note>
## Log In
Access Prowler App by logging in with **email and password**.
<img src="/images/log-in.png" alt="Log In" width="285" />
## Add Cloud Provider
Configure a cloud provider for scanning:
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
2. Select the cloud provider.
3. Enter the provider's identifier (Optional: Add an alias):
- **AWS**: Account ID
- **GCP**: Project ID
- **Azure**: Subscription ID
- **Kubernetes**: Cluster ID
- **M365**: Domain ID
4. Follow the guided instructions to add and authenticate your credentials.
## Start a Scan
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
Click `Go to Scans` to monitor progress.
## View Results
Review findings during scan execution in the following sections:
- **Overview** Provides a high-level summary of your scans.
<img src="/images/products/overview.png" alt="Overview" width="700" />
- **Compliance** Displays compliance insights based on security frameworks.
<img src="/images/compliance.png" alt="Compliance" width="700" />
> For detailed usage instructions, refer to the [Prowler App Guide](/user-guide/tutorials/prowler-app).
<Note>
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
</Note>
@@ -1,18 +1,24 @@
---
title: 'Basic Usage'
---
## Running Prowler
Running Prowler requires specifying the provider (e.g `aws`, `gcp`, `azure`, `kubernetes`, `m365`, `github`, `iac` or `mongodbatlas`):
???+ note
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
<Note>
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
</Note>
```console
prowler <provider>
```
![Prowler Execution](../img/short-display.png)
![Prowler Execution](/images/short-display.png)
???+ note
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
<Note>
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
</Note>
## Verbose Output
If you prefer the former verbose output, use: `--verbose`. This allows seeing more info while Prowler is running, minimal output is displayed unless verbosity is enabled.
@@ -26,7 +32,7 @@ prowler <provider> -M csv json-asff json-ocsf html
```
The HTML report is saved in the output directory, alongside other reports. It will look like this:
![Prowler Execution](../img/html-output.png)
![Prowler Execution](/images/html-output.png)
## Listing Available Checks and Services
@@ -58,7 +64,7 @@ prowler kubernetes --excluded-services controllermanager
```
## Additional Options
Explore more advanced time-saving execution methods in the [Miscellaneous](../tutorials/misc.md) section.
Explore more advanced time-saving execution methods in the [Miscellaneous](/user-guide/cli/tutorials/misc) section.
Access the help menu and view all available options with `-h`/`--help`:
@@ -74,10 +80,11 @@ Use a custom AWS profile with `-p`/`--profile` and/or specific AWS regions with
prowler aws --profile custom-profile -f us-east-1 eu-south-2
```
???+ note
By default, `prowler` will scan all AWS regions.
<Note>
By default, `prowler` will scan all AWS regions.
See more details about AWS Authentication in the [Authentication Section](../tutorials/aws/authentication.md) section.
</Note>
See more details about AWS Authentication in the [Authentication Section](/user-guide/providers/aws/authentication) section.
## Azure
@@ -97,7 +104,7 @@ prowler azure --browser-auth --tenant-id "XXXXXXXX"
prowler azure --managed-identity-auth
```
See more details about Azure Authentication in the [Authentication Section](../tutorials/azure/authentication.md)
See more details about Azure Authentication in the [Authentication Section](/user-guide/providers/azure/authentication)
By default, Prowler scans all accessible subscriptions. Scan specific subscriptions using the following flag (using az cli auth as example):
@@ -154,9 +161,10 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
```console
prowler kubernetes --kubeconfig-file path
```
???+ note
<Note>
If no `--kubeconfig-file` is provided, Prowler will use the default KubeConfig file location (`~/.kube/config`).
</Note>
- **In-Cluster Execution**
To run Prowler inside the cluster, apply the provided YAML configuration to deploy a job in a new namespace:
@@ -170,9 +178,10 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
kubectl logs prowler-XXXXX --namespace prowler-ns
```
???+ note
<Note>
By default, Prowler scans all namespaces in the active Kubernetes context. Use the `--context`flag to specify the context to be scanned and `--namespaces` to restrict scanning to specific namespaces.
</Note>
## Microsoft 365
Microsoft 365 requires specifying the auth method:
@@ -182,9 +191,6 @@ Microsoft 365 requires specifying the auth method:
# To use service principal authentication for MSGraph and PowerShell modules
prowler m365 --sp-env-auth
# To use both service principal (for MSGraph) and user credentials (for PowerShell modules)
prowler m365 --env-auth
# To use az cli authentication
prowler m365 --az-cli-auth
@@ -193,7 +199,7 @@ prowler m365 --browser-auth --tenant-id "XXXXXXXX"
```
See more details about M365 Authentication in the [Authentication Section](../tutorials/microsoft365/authentication.md) section.
See more details about M365 Authentication in the [Authentication Section](/user-guide/providers/microsoft365/authentication) section.
## GitHub
@@ -214,13 +220,14 @@ Prowler enables security scanning of your **GitHub account**, including **Reposi
prowler github --github-app-id app_id --github-app-key app_key
```
???+ note
<Note>
If no login method is explicitly provided, Prowler will automatically attempt to authenticate using environment variables in the following order of precedence:
1. `GITHUB_PERSONAL_ACCESS_TOKEN`
2. `OAUTH_APP_TOKEN`
3. `GITHUB_APP_ID` and `GITHUB_APP_KEY`
</Note>
## Infrastructure as Code (IaC)
Prowler's Infrastructure as Code (IaC) provider enables you to scan local or remote infrastructure code for security and compliance issues using [Trivy](https://trivy.dev/). This provider supports a wide range of IaC frameworks, allowing you to assess your code before deployment.
@@ -247,14 +254,15 @@ prowler iac --scan-path ./my-iac-directory --frameworks terraform kubernetes
prowler iac --scan-path ./my-iac-directory --exclude-path ./my-iac-directory/test,./my-iac-directory/examples
```
???+ note
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
- The IaC provider does not require cloud authentication for local scans.
- It is ideal for CI/CD pipelines and local development environments.
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
<Note>
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
- The IaC provider does not require cloud authentication for local scans.
- It is ideal for CI/CD pipelines and local development environments.
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
See more details about IaC scanning in the [IaC Tutorial](../tutorials/iac/getting-started-iac.md) section.
</Note>
See more details about IaC scanning in the [IaC Tutorial](/user-guide/providers/iac/getting-started-iac) section.
## MongoDB Atlas
@@ -279,4 +287,30 @@ You can filter scans to specific organizations or projects:
prowler mongodbatlas --atlas-project-id <project_id>
```
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](../tutorials/mongodbatlas/authentication.md)
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](/user-guide/providers/mongodbatlas/authentication)
## Oracle Cloud
Prowler allows you to scan your Oracle Cloud deployments for security and compliance issues.
You have two options to authenticate:
1. OCI Config File Authentication: this config file can be generated using the OCI CLI with the `oci session authenticate` command or created manually using the OCI Console. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#oci-session-authentication).
```console
prowler oci
```
You can add different profiles to the config file to scan different tenancies or regions. In order to scan a specific profile, use the `--profile` flag:
```console
prowler oci --profile <profile_name>
```
2. Instance Principal Authentication: when running Prowler on an OCI Compute instance, you can use Instance Principal authentication. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#instance-principal-authentication).
```console
prowler oci --use-instance-principal
```
See more details about Oracle Cloud Authentication in [Oracle Cloud Authentication](/user-guide/providers/oci/authentication)
@@ -0,0 +1,122 @@
---
title: "Tools Reference"
---
Complete reference guide for all tools available in the Prowler MCP Server. Tools are organized by namespace.
## Tool Categories Summary
| Category | Tool Count | Authentication Required |
|----------|------------|------------------------|
| Prowler Hub | 10 tools | No |
| Prowler Documentation | 2 tools | No |
| Prowler Cloud/App | 28 tools | Yes |
## Tool Naming Convention
All tools follow a consistent naming pattern with prefixes:
- `prowler_hub_*` - Prowler Hub catalog and compliance tools
- `prowler_docs_*` - Prowler documentation search and retrieval
- `prowler_app_*` - Prowler Cloud and App (Self-Managed) management tools
## Prowler Hub Tools
Access Prowler's security check catalog and compliance frameworks. **No authentication required.**
### Check Discovery
- **`prowler_hub_get_checks`** - List security checks with advanced filtering options
- **`prowler_hub_get_check_filters`** - Return available filter values for checks (providers, services, severities, categories, compliances)
- **`prowler_hub_search_checks`** - Full-text search across check metadata
- **`prowler_hub_get_check_raw_metadata`** - Fetch raw check metadata in JSON format
### Check Code
- **`prowler_hub_get_check_code`** - Fetch the Python implementation code for a security check
- **`prowler_hub_get_check_fixer`** - Fetch the automated fixer code for a check (if available)
### Compliance Frameworks
- **`prowler_hub_get_compliance_frameworks`** - List and filter compliance frameworks
- **`prowler_hub_search_compliance_frameworks`** - Full-text search across compliance frameworks
### Provider Information
- **`prowler_hub_list_providers`** - List Prowler official providers and their services
- **`prowler_hub_get_artifacts_count`** - Get total count of checks and frameworks in Prowler Hub
## Prowler Documentation Tools
Search and access official Prowler documentation. **No authentication required.**
- **`prowler_docs_search`** - Search the official Prowler documentation using full-text search
- **`prowler_docs_get_document`** - Retrieve the full markdown content of a specific documentation file
## Prowler Cloud/App Tools
Manage Prowler Cloud or Prowler App (Self-Managed) features. **Requires authentication.**
<Note>
These tools require a valid API key. See the [Configuration Guide](/getting-started/basic-usage/prowler-mcp) for authentication setup.
</Note>
### Findings Management
- **`prowler_app_list_findings`** - List security findings with advanced filtering
- **`prowler_app_get_finding`** - Get detailed information about a specific finding
- **`prowler_app_get_latest_findings`** - Retrieve latest findings from the most recent scans
- **`prowler_app_get_findings_metadata`** - Get unique metadata values from filtered findings
- **`prowler_app_get_latest_findings_metadata`** - Get metadata from latest findings across all providers
### Provider Management
- **`prowler_app_list_providers`** - List all providers with filtering options
- **`prowler_app_create_provider`** - Create a new provider in the current tenant
- **`prowler_app_get_provider`** - Get detailed information about a specific provider
- **`prowler_app_update_provider`** - Update provider details (alias, etc.)
- **`prowler_app_delete_provider`** - Delete a specific provider
- **`prowler_app_test_provider_connection`** - Test provider connection status
### Provider Secrets Management
- **`prowler_app_list_provider_secrets`** - List all provider secrets with filtering
- **`prowler_app_add_provider_secret`** - Add or update credentials for a provider
- **`prowler_app_get_provider_secret`** - Get detailed information about a provider secret
- **`prowler_app_update_provider_secret`** - Update provider secret details
- **`prowler_app_delete_provider_secret`** - Delete a provider secret
### Scan Management
- **`prowler_app_list_scans`** - List all scans with filtering options
- **`prowler_app_create_scan`** - Trigger a manual scan for a specific provider
- **`prowler_app_get_scan`** - Get detailed information about a specific scan
- **`prowler_app_update_scan`** - Update scan details
- **`prowler_app_get_scan_compliance_report`** - Download compliance report as CSV
- **`prowler_app_get_scan_report`** - Download ZIP file containing complete scan report
### Schedule Management
- **`prowler_app_schedules_daily_scan`** - Create a daily scheduled scan for a provider
### Processor Management
- **`prowler_app_processors_list`** - List all processors with filtering
- **`prowler_app_processors_create`** - Create a new processor (currently only mute lists supported)
- **`prowler_app_processors_retrieve`** - Get processor details by ID
- **`prowler_app_processors_partial_update`** - Update processor configuration
- **`prowler_app_processors_destroy`** - Delete a processor
## Usage Tips
- Use natural language to interact with the tools through your AI assistant
- Tools can be combined for complex workflows
- Filter options are available on most list tools
- Authentication is only required for Prowler Cloud/App tools
## Additional Resources
- [MCP Protocol Specification](https://modelcontextprotocol.io)
- [Prowler API Documentation](https://api.prowler.com/api/v1/docs)
- [Prowler Hub API](https://hub.prowler.com/api/docs)
- [GitHub Repository](https://github.com/prowler-cloud/prowler)
@@ -0,0 +1,247 @@
---
title: "Configuration"
---
Configure your MCP client to connect to Prowler MCP Server.
## Step 1: Get Your API Key (Optional)
<Note>
**Authentication is optional**: Prowler Hub and Prowler Documentation features work without authentication. An API key is only required for Prowler Cloud and Prowler App (Self-Managed) features.
</Note>
To use Prowler Cloud or Prowler App (Self-Managed) features. To get the API key, please refer to the [API Keys](/user-guide/providers/prowler-app-api-keys) guide.
<Warning>
Keep the API key secure. Never share it publicly or commit it to version control.
</Warning>
## Step 2: Configure Your MCP Client
Choose the configuration based on your deployment:
- **STDIO Mode**: Local installation only (runs as subprocess).
- **HTTP Mode**: Prowler Cloud MCP Server or self-hosted Prowler MCP Server.
### HTTP Mode (Prowler Cloud MCP Server or self-hosted Prowler MCP Server)
<Tabs>
<Tab title="Native HTTP Support (Cursor, VSCode)">
**Clients that support HTTP with custom headers natively**
For example: Cursor, VSCode, LobeChat, etc.
**Configuration:**
```json
{
"mcpServers": {
"prowler": {
"url": "https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
"headers": {
"Authorization": "Bearer pk_your_api_key_here"
}
}
}
}
```
</Tab>
<Tab title="Using mcp-remote (Claude Desktop)">
**For clients without native HTTP support (like Claude Desktop)**
For example: Claude Desktop.
**Configuration:**
```json
{
"mcpServers": {
"prowler": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
"--header",
"Authorization: Bearer ${PROWLER_APP_API_KEY}"
],
"env": {
"PROWLER_APP_API_KEY": "pk_your_api_key_here"
}
}
}
}
```
<Info>
The `mcp-remote` tool acts as a bridge for clients that don't support HTTP natively. Learn more at [mcp-remote on npm](https://www.npmjs.com/package/mcp-remote).
</Info>
</Tab>
</Tabs>
### STDIO Mode (Local Installation Only)
STDIO mode is only available when running the MCP server locally.
<Tabs>
<Tab title="Using uvx">
**Run from source or local installation**
```json
{
"mcpServers": {
"prowler": {
"command": "uvx",
"args": ["/absolute/path/to/prowler/mcp_server/"],
"env": {
"PROWLER_APP_API_KEY": "pk_your_api_key_here",
"PROWLER_API_BASE_URL": "https://api.prowler.com"
}
}
}
}
```
<Note>
Replace `/absolute/path/to/prowler/mcp_server/` with the actual path. The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
</Note>
</Tab>
<Tab title="Using Docker">
**Run with Docker image**
```json
{
"mcpServers": {
"prowler": {
"command": "docker",
"args": [
"run",
"--rm",
"-i",
"--env",
"PROWLER_APP_API_KEY=pk_your_api_key_here",
"--env",
"PROWLER_API_BASE_URL=https://api.prowler.com",
"prowlercloud/prowler-mcp"
]
}
}
}
```
<Note>
The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
</Note>
</Tab>
</Tabs>
## Step 3: Start Using Prowler MCP
Restart your MCP client and start asking questions:
- *"Show me all critical findings from my AWS accounts"*
- *"What does the S3 bucket public access check do?"*
- *"Onboard this new AWS account in my Prowler Organization"*
## Authentication Methods
Prowler MCP Server supports two authentication methods to connect to Prowler Cloud or Prowler App (Self-Managed):
### API Key (Recommended)
Use your Prowler API key directly in the Bearer token:
```
Authorization: Bearer pk_your_api_key_here
```
This is the recommended method for most users.
### JWT Token
Alternatively, obtain a JWT token from Prowler:
```bash
curl -X POST https://api.prowler.com/api/v1/tokens \
-H "Content-Type: application/vnd.api+json" \
-H "Accept: application/vnd.api+json" \
-d '{
"data": {
"type": "tokens",
"attributes": {
"email": "your-email@example.com",
"password": "your-password"
}
}
}'
```
Use the returned JWT token in place of the API key:
```
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
```
<Warning>
JWT tokens are only valid for 30 minutes. You need to generate a new token if you want to continue using the MCP server.
</Warning>
## Troubleshooting
### Server Not Detected
- Restart your MCP client after configuration changes
- Check the configuration file syntax (valid JSON)
- Review client logs for specific error messages
- Verify the server URL is correct
### Authentication Failures
**Error: Unauthorized (401)**
- Verify your API key is correct
- Ensure the key hasn't expired
- Check you're using the right API endpoint
### Connection Issues
**Cannot Reach Server:**
- Verify the server URL is correct
- Check network connectivity
- For local servers, ensure the server is running
- Check firewall settings
## Security Best Practices
1. **Protect Your API Key**
- Never commit API keys to version control.
- Use environment variables or secure vaults.
- Rotate keys regularly.
2. **Network Security**
- Use HTTPS for production deployments.
- Restrict network access to the MCP server.
- Consider VPN for remote access.
3. **Least Privilege**
- API key gives the permission of the user who created the key, make sure to use the key with the minimal required permissions.
- Review the tools that are gonna be used and how they are gonna be used to avoid prompt injections or unintended behavior.
## Next Steps
Now that your MCP server is configured:
<CardGroup cols={1}>
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
Explore all available tools
</Card>
</CardGroup>
## Getting Help
Need assistance with configuration?
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
@@ -0,0 +1,94 @@
---
title: 'AWS Security Hub'
---
AWS Security Hub remains a managed service designed for centralizing security alerts and compliance status within AWS environments. It integrates with various AWS security services and provides a consolidated view of security findings.
## Key Features and Strengths
- **Centralized Dashboard for AWS:** Provides a single pane of glass to monitor and manage security findings from multiple AWS services like GuardDuty, Inspector, and Config.
- **Compliance Checks:** Automatically checks for compliance against standards like CIS and PCI DSS within AWS environments.
- **AWS Native Automation:** Offers seamless automation for incident response using AWS Lambda and CloudWatch Events, reducing the time to react to security issues.
- **User-Friendly Interface:** Accessible via the AWS Management Console, offering a streamlined experience for managing security across AWS accounts.
## Limitations
- **AWS-Centric:** Limited to AWS environments, with no direct support for multi-cloud or hybrid environments.
- **Dependency on AWS Config:** Some of its checks depend on AWS Config, which may not be enabled in all regions or accounts.
- **Vendor Lock-In:** Tightly coupled with AWS, making it less suitable for organizations with a cloud-agnostic strategy.
## Prowler
Prowler is an open-source, multi-cloud security tool that offers extensive customization and flexibility, making it ideal for organizations with complex or multi-cloud environments. Here are the updated features and advantages:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler is inherently multi-region and can scan multiple AWS accounts without requiring additional configuration or enabling specific services like AWS Config.
- **Minimal Setup Requirements:**
- All Prowler needs is a role with appropriate permissions to start scanning. Theres no need to enable specific services or configure complex setups.
- **Versatile Execution Environment:**
- Prowler can be run from various environments, including a local workstation, container, AWS CloudShell, or even from another AWS account or cloud provider by assuming a role. This flexibility makes it easy to integrate into different operational workflows.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis, or locally for easy sharing and discussion. This flexibility is particularly useful for collaborative security assessments.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. It also supports integration with Amazon QuickSight for in-depth analysis and offers a SaaS model with resource-based pricing, making it adaptable to different organizational needs.
- **Security Hub Integration for Cost-Effective Operations:**
- Prowler can send results directly into Security Hub in any AWS account, including only failed findings. This selective reporting can make Security Hub more cost-effective by reducing the volume of data processed.
- **Custom Checks and Compliance Frameworks:**
- Users can write custom checks, remediations, and compliance frameworks in minutes, tailoring the tool to their specific security policies and operational needs.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box for AWS, providing comprehensive coverage across various regulatory requirements and best practices.
- **Kubernetes and Multi-Cloud Support:**
- Prowler extends its scanning capabilities beyond AWS, offering support for Kubernetes clusters (including EKS), as well as environments in Google Cloud Platform (GCP) and Azure. This multi-cloud capability is essential for organizations with diverse cloud footprints.
- **All-Region Checks:**
- Prowler runs all checks in all regions, regardless of AWS Config resource type support, ensuring comprehensive coverage across your entire AWS environment.
## Comparison Summary
### Scope and Environment
- **Security Hub** is ideal for AWS-centric environments needing a managed service for monitoring and automating security across AWS resources.
- **Prowler** is better suited for organizations operating in multi-cloud or hybrid environments, offering flexibility, customization, and support for multiple cloud providers including AWS, Azure, GCP, and Kubernetes.
### Setup and Maintenance
- **Security Hub** requires enabling and configuring AWS services by region, per account, and can become more than one person's full-time role including Config. Security Hub operates only within the AWS ecosystem.
- **Prowler** requires minimal setup, only needing appropriate permissions, and can be executed from various environments, making it more versatile in different operational contexts.
### Customization and Flexibility
- **Security Hub** offers predefined compliance checks and automation within AWS but is less flexible in terms of customization.
- **Prowler** allows for highly customizable checks, remediation actions, and compliance frameworks, with the ability to adapt quickly to organizational needs and regulatory changes.
### Cost Efficiency
- **Security Hub** may involve additional costs for processing and storing findings.
- **Prowler** can optimize costs by selectively sending failed findings to Security Hub and storing results locally or in S3, which can be more cost-effective.
### Multi-Cloud and Multi-Region Support
- **Security Hub** is confined to AWS, with region-specific checks depending on AWS Config.
- **Prowler** is inherently multi-region and multi-cloud, offering consistent and comprehensive checks across different cloud environments and regions.
## Conclusion
For a CISO or security professional evaluating these tools, the decision between AWS Security Hub and Prowler will depend on the organizations cloud strategy, compliance needs, and the level of flexibility required:
- If the organization is heavily invested in AWS and prefers a managed, integrated security service that offers ease of use and automation within the AWS ecosystem, **AWS Security Hub** is the more appropriate choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool that can run comprehensive, multi-region scans across AWS, Azure, GCP, and Kubernetes, **Prowler** provides a more powerful and flexible solution, especially for those needing to adapt quickly to evolving security and compliance requirements.
+97
View File
@@ -0,0 +1,97 @@
---
title: 'GCP Cloud Security Command Center (Cloud SCC)'
---
Google Cloud Security Command Center (Cloud SCC) is a centralized security and risk management platform for Google Cloud Platform (GCP). It provides visibility into assets, vulnerabilities, and threats across GCP environments, helping organizations to manage and improve their security posture.
## Key Features and Strengths
- **Centralized Security Visibility:** Cloud SCC provides a single pane of glass to monitor the security and risk status across your GCP resources. It aggregates findings from various GCP security services, such as Security Health Analytics, Web Security Scanner, and Event Threat Detection.
- **Asset Inventory and Classification:** Cloud SCC offers comprehensive asset discovery and classification across GCP, giving security teams a detailed inventory of their cloud resources, including their configurations and security states.
- **Threat Detection and Monitoring:** The platform integrates with GCPs threat detection tools, such as Googles Event Threat Detection, which analyzes logs for suspicious activities and potential threats.
- **Compliance Monitoring:** Cloud SCC helps monitor compliance with various regulatory standards by continuously assessing your GCP resources against best practices and security benchmarks.
- **Automated Remediation:** Cloud SCC can trigger automated responses to security findings through integrations with Google Cloud Functions or other orchestration tools, helping to mitigate risks quickly.
- **Native GCP Integration:** Cloud SCC is deeply integrated with the GCP ecosystem, offering seamless operation within Google Cloud environments and leveraging Google's extensive security expertise.
## Limitations
- **GCP-Centric:** While Cloud SCC is powerful within the GCP ecosystem, it is primarily focused on GCP and does not natively extend to multi-cloud environments without additional tools or connectors.
- **Cost Considerations:** As a managed service within GCP, costs can scale with the amount of data ingested and the complexity of the environment, especially as additional features or higher volumes of data are utilized.
- **Dependency on GCP Services:** Cloud SCC's capabilities depend on other GCP services being enabled, such as Security Health Analytics and Web Security Scanner, which may increase overall complexity and cost.
## Prowler
Prowler is an open-source, multi-cloud security tool designed to perform detailed security assessments and compliance checks across diverse cloud environments, including AWS, Azure, GCP, and Kubernetes. Here are the key advantages of Prowler when compared to GCP Cloud SCC:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler inherently supports multi-region and multi-account scanning across multiple cloud providers, including GCP, AWS, Azure, and Kubernetes. It does not require additional configuration to perform these scans, making it immediately useful for organizations operating in multiple cloud environments.
- **Minimal Setup Requirements:**
- Prowler requires only appropriate roles and permissions to start scanning. It doesnt necessitate enabling specific services within GCP, which can simplify the setup process and reduce dependencies.
- **Versatile Execution Environment:**
- Prowler can be run from various environments, such as a local workstation, container, Google Cloud Shell, or even other cloud providers by assuming a role. This versatility allows for flexible deployment and integration into existing security operations.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored in an S3 bucket for AWS, Google Cloud Storage (GCS) for GCP, or locally, allowing for quick analysis and easy sharing. This flexibility is particularly advantageous for multi-cloud security assessments and collaborative security processes.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. These reports can be tailored to specific needs and easily integrated with other security tools or dashboards, providing comprehensive insights across all cloud environments.
- **SIEM Integration and Cost Efficiency:**
- Prowler can be configured to send findings directly into SIEM systems, including those integrated with GCP or other platforms. By sending only failed findings or selected results, Prowler helps manage costs associated with data ingestion and analysis in SIEM platforms.
- **Custom Checks and Compliance Frameworks:**
- Prowler allows for the creation of custom security checks, remediation actions, and compliance frameworks, providing flexibility that can be adapted to the unique security policies and regulatory requirements of an organization.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box, with capabilities to extend these frameworks to GCP environments as well as other cloud platforms. This broad compliance coverage ensures that organizations can maintain adherence to various regulatory requirements.
- **Kubernetes and Multi-Cloud Support:**
- Prowler is designed to support security assessments across cloud environments, including Kubernetes clusters and GCP. This multi-cloud capability is essential for organizations that operate across diverse cloud platforms and require consistent security posture management.
- **All-Region Checks:**
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage across an organizations cloud resources, regardless of the region or cloud provider.
## Comparison Summary
### Scope and Environment
- **GCP Cloud SCC** is ideal for organizations primarily using GCP, offering a centralized platform for managing security and compliance within the GCP ecosystem.
- **Prowler** excels in multi-cloud environments, offering flexibility and comprehensive security checks across AWS, Azure, GCP, and Kubernetes without being confined to a single cloud provider.
### Setup and Flexibility
- **GCP Cloud SCC** requires enabling various GCP services and may involve more complex setup, especially for multi-region or multi-account scenarios within GCP.
- **Prowler** requires minimal setup and can be deployed quickly across different cloud environments, offering a more straightforward approach to multi-cloud security management.
### Customization and Compliance
- **GCP Cloud SCC** provides predefined compliance checks within the GCP environment but may require additional tools or customization for broader or more specific requirements.
- **Prowler** allows for extensive customization of security checks, compliance frameworks, and reporting, providing a flexible solution that can be tailored to an organizations specific needs across various cloud platforms.
### Cost Efficiency
- **GCP Cloud SCC** costs can scale with the volume of data processed and the number of enabled services, which may be significant in large or complex environments. SCC pricing is confusing to understand, and starts at $0.071 per vCPU hour for some tiers and depending on the scan service. Take a look at the pricing model [here](https://cloud.google.com/security-command-center/pricing), godspeed.
- **Prowler** helps manage costs by allowing selective reporting, such as sending only failed findings to SIEMs, and storing results in cost-effective ways, such as local storage or cloud buckets. Prowler is always $0.001 per resource per day - no per account charge.
### Multi-Cloud and Multi-Region Support
- **GCP Cloud SCC** is focused on GCP and may require additional tools for comprehensive multi-cloud support.
- **Prowler** is inherently multi-cloud, supporting AWS, Azure, GCP, and Kubernetes out of the box, making it an ideal choice for organizations with diverse cloud footprints.
## Conclusion
For a CISO evaluating these tools, the decision between GCP Cloud Security Command Center (Cloud SCC) and Prowler hinges on the organizations cloud strategy, security management needs, and the level of flexibility and multi-cloud support required:
- If the organization is heavily invested in GCP and needs a centralized platform that integrates seamlessly with GCP services for asset management, threat detection, and compliance monitoring, **GCP Cloud SCC** is likely the better choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
+10
View File
@@ -0,0 +1,10 @@
---
title: 'Comparison'
---
Click to learn more about each cloud security provider and learn how Prowler is differentiated.
- [AWS Security Hub](/getting-started/comparison/awssecurityhub)
- [Microsoft Sentinel](/getting-started/comparison/microsoftsentinel)
- [Microsoft Defender](/getting-started/comparison/microsoftdefender)
- [Google Cloud Security Command Center](/getting-started/comparison/gcp)
@@ -0,0 +1,101 @@
---
title: 'Microsoft Defender for Cloud'
---
**Use open-source scanning to validate and extend Microsoft Defender for Cloud**
---
## **Overview**
If you're using Microsoft Defender for Cloud to monitor your Azure infrastructure, Prowler can complement it with fully transparent, customizable scans across Azure, AWS, GCP, and Kubernetes. Prowler helps you validate policies, automate compliance, and gain deeper visibility—all from the CLI, API or our Prowler UI.
You can run Prowler alongside Defender for Cloud to:
* Double-check security posture with open-source checks.
* Customize rules for your organizations policies.
* Bring your own, or community contributed policies.
* Automate multi-cloud scans in CI/CD or scheduled jobs.
---
## **Why use Prowler with Defender for Cloud**
Microsoft Defender for Cloud offers centralized dashboards, alerting, and some cross-cloud coverage. Prowler provides full transparency and control over whats being checked and how those checks work—no vendor lock-in, no surprises.
Use them together to get:
* More confidence in your security posture
* Checks you can inspect, modify, and version
* CLI-first, portable scanning across clouds
* Open-source tooling that integrates easily into pipelines and audits
---
## **Quickstart**
Heres how to install Prowler and run a scan in your Azure account.
### **1\. Install Prowler**
```
git clone https://github.com/prowler-cloud/prowler
cd prowler
./install.sh
```
### **2\. Authenticate with Azure**
Make sure you're signed in and select your subscription:
```
az login
export AZURE_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
```
### **3\. Run a scan**
```
./prowler -p Azure -f az-aks -f az-general
```
This will run checks focused on Azure Kubernetes Service (AKS) and general Azure best practices.
### **4\. Review results**
```
cat output/prowler-output-*.json
open output/prowler-output-*.html
```
You can export findings in JSON, CSV, JUnit, HTML, or AWS Security Hubcompatible formats.
---
## **Compare capabilities**
| Feature | Microsoft Defender for Cloud | Prowler |
| ----- | ----- | ----- |
| Azure-native posture management | ✅ | ✅ |
| AWS, GCP, and Kubernetes support | ⚠️ (limited) | ✅ |
| Custom policy creation | ❌ | ✅ |
| CLI-first, scriptable | ❌ | ✅ |
| Open source | ❌ | ✅ |
| Compliance mappings (CIS, NIST, etc.) | ✅ (limited control) | ✅ (customizable) |
| Exportable detections | ❌ | ✅ |
---
## **Common use cases**
**✅ Validate policies**
Run Prowler to confirm your Azure policies are configured as expected and compliant with frameworks like CIS or NIST.
**✅ Automate compliance scans**
Schedule regular Prowler scans in your CI/CD pipeline or infrastructure monitoring workflows. Generate reports for auditors or internal reviews.
**✅ Extend detection coverage**
If Defender for Cloud doesnt cover all the services or resources in your environment, Prowlers checks fill in the gaps.
**✅ Build custom checks**
Security is never one-size-fits-all. Prowler lets you write your own checks for organization-specific policies.
@@ -0,0 +1,93 @@
---
title: 'Microsoft Sentinel'
---
Microsoft Sentinel is a scalable, cloud-native security information and event management (SIEM) and security orchestration automated response (SOAR) solution. It's designed to collect, detect, investigate, and respond to threats across the enterprise, primarily within the Azure cloud environment but also extending to on-premises and other cloud environments through various connectors.
## Key Features and Strengths
- **SIEM and SOAR Capabilities:** Microsoft Sentinel combines SIEM and SOAR functionalities, allowing it to collect and analyze large volumes of data from various sources and automate responses to detected threats.
- **Native Azure Integration:** As part of the Azure ecosystem, Sentinel integrates seamlessly with Azure services, providing deep visibility and analytics for Azure resources.
- **Advanced Threat Detection:** Sentinel uses AI and machine learning to detect potential threats and anomalous activities, leveraging Microsoft's extensive threat intelligence network.
- **Scalability and Flexibility:** Being cloud-native, Sentinel scales automatically to handle increasing data volumes and complexity without requiring extensive infrastructure management.
- **Customizable Dashboards and Analytics:** Sentinel offers customizable dashboards and analytics, allowing security teams to tailor their views and queries to specific needs.
- **Multi-Source Data Ingestion:** While focused on Azure, Sentinel can ingest data from multiple sources, including AWS, GCP, on-premises environments, and third-party security products.
## Limitations
- **Azure-Centric:** While it supports multi-cloud environments, its primary focus and strengths are within the Azure ecosystem. Integration with other cloud platforms and on-premises environments may require additional connectors and configurations.
- **Cost Considerations:** As a SIEM tool, Sentinel can become expensive, particularly as data ingestion and analysis volumes grow. The cost model is based on data volume, which can add up quickly in large environments.
- **Complexity in Customization:** Although Sentinel offers advanced customization, setting up and fine-tuning these customizations can require significant expertise and effort, particularly in multi-cloud environments.
## Prowler
Prowler is an open-source, multi-cloud security tool that offers extensive flexibility and customization, making it ideal for organizations that need to maintain a strong security posture across diverse cloud environments. Here are the key advantages of Prowler, particularly when compared to Microsoft Sentinel:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler is inherently multi-region and multi-account, requiring no additional configuration to scan across these environments. This capability is available out of the box without needing to enable specific services or create complex setups.
- **Minimal Setup Requirements:**
- Prowler requires only a role with appropriate permissions to begin scanning. Theres no need for extensive setup, making it easier and quicker to deploy across various environments.
- **Versatile Execution Environment:**
- Prowler can be run from a local workstation, container, AWS CloudShell, or even from other cloud providers like Azure or GCP by assuming a role. This versatility allows security teams to integrate Prowler into a wide range of operational workflows without being tied to a single cloud environment.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis or locally for easy sharing and collaboration. This flexibility is particularly useful for multi-cloud security assessments and incident response.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. Additionally, it can integrate with Amazon QuickSight for advanced analytics, and offers a SaaS model with resource-based pricing, making it adaptable to various organizational needs.
- **SIEM Integration and Cost Efficiency:**
- While Microsoft Sentinel has a built-in SIEM functionality, Prowler can send results directly into SIEM systems, including Microsoft Sentinel. By sending only failed findings, Prowler can help optimize costs associated with data ingestion and storage in SIEM platforms.
- **Custom Checks and Compliance Frameworks:**
- Prowler enables users to write custom checks, remediations, and compliance frameworks quickly, allowing organizations to adapt the tool to their specific security policies and regulatory requirements.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box, providing comprehensive coverage for AWS environments, which can be extended to multi-cloud scenarios.
- **Kubernetes and Multi-Cloud Support:**
- Prowler is designed to support security assessments beyond AWS, including Kubernetes clusters (including EKS) and environments in Azure and GCP. This capability is critical for organizations that operate across multiple cloud platforms and require consistent security posture management.
- **All-Region Checks:**
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage without the limitations that may be imposed by region-specific configurations or services.
## Comparison Summary
### Scope and Environment
- **Microsoft Sentinel** is an advanced SIEM/SOAR tool optimized for Azure environments, with support for multi-cloud and on-premises systems through connectors.
- **Prowler** is a flexible, multi-cloud security tool that excels in environments where organizations need to manage security across AWS, Azure, GCP, and Kubernetes with minimal setup and high customizability.
### Setup and Flexibility
- **Microsoft Sentinel** requires more setup, especially when integrating with non-Azure environments, and its cost scales with data ingestion.
- **Prowler** requires minimal setup and can be easily deployed in any cloud or on-premises environment. Its ability to run from various environments and store results in flexible formats makes it particularly adaptable.
### Customization and Compliance
- **Microsoft Sentinel** offers powerful but complex customization options, primarily within the Azure ecosystem.
- **Prowler** provides straightforward customization of security checks, remediation actions, and compliance frameworks, with broad support for multiple compliance standards out of the box.
### Cost Efficiency
- **Microsoft Sentinel** can become costly as data volumes grow, particularly in large or multi-cloud environments.
- **Prowler** helps control costs by enabling selective reporting (e.g., sending only failed findings to SIEMs like Sentinel) and storing results in cost-effective ways, such as S3 or locally.
### Multi-Cloud and Multi-Region Support
- **Microsoft Sentinel** supports multi-cloud environments but may require additional configuration and connectors.
- **Prowler** is designed for multi-cloud environments from the ground up, with inherent support for AWS, Azure, GCP, Kubernetes, and all regions, making it an ideal tool for organizations with diverse cloud footprints.
## Conclusion
For a CISO or security professional evaluating these tools, the decision between Microsoft Sentinel and Prowler hinges on the organization's cloud strategy, SIEM needs, and the level of customization and flexibility required:
- If the organization is heavily invested in Azure and needs an integrated SIEM/SOAR solution with advanced threat detection, analytics, and automation capabilities, **Microsoft Sentinel** is likely the better choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
@@ -0,0 +1,4 @@
---
title: "Prowler API Reference"
url: "https://api.prowler.com/api/v1/docs"
---
@@ -0,0 +1,4 @@
---
title: "Go to Cloud"
url: "https://cloud.prowler.com"
---
@@ -0,0 +1,4 @@
---
title: "Go to Hub"
url: "https://hub.prowler.com"
---
@@ -0,0 +1,5 @@
---
title: "Prowler MCP"
url: "https://github.com/prowler-cloud/prowler/tree/master/mcp_server"
tag: "new!"
---
@@ -1,14 +1,19 @@
---
title: 'Installation'
---
### Installation
Prowler App supports multiple installation methods based on your environment.
Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed usage instructions.
Refer to the [Prowler App Tutorial](/user-guide/tutorials/prowler-app) for detailed usage instructions.
???+ warning
Prowler configuration is based in `.env` files. Every version of Prowler can have differences on that file, so, please, use the file that corresponds with that version or repository branch or tag.
=== "Docker Compose"
<Warning>
Prowler configuration is based in `.env` files. Every version of Prowler can have differences on that file, so, please, use the file that corresponds with that version or repository branch or tag.
</Warning>
<Tabs>
<Tab title="Docker Compose">
_Requirements_:
* `Docker Compose` installed: https://docs.docker.com/compose/install/.
@@ -20,25 +25,8 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/.env
docker compose up -d
```
> Containers are built for `linux/amd64`. If your workstation's architecture is different, please set `DOCKER_DEFAULT_PLATFORM=linux/amd64` in your environment or use the `--platform linux/amd64` flag in the docker command.
> Enjoy Prowler App at http://localhost:3000 by signing up with your email and password.
???+ note
You can change the environment variables in the `.env` file. Note that it is not recommended to use the default values in production environments.
???+ note
For a secure setup, leave empty or remove `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY` in `.env` before first start. When absent, the API autogenerates a unique key pair and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate, delete the stored key files and restart the API.
???+ note
There is a development mode available, you can use the file https://github.com/prowler-cloud/prowler/blob/master/docker-compose-dev.yml to run the app in development mode.
???+ warning
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
=== "GitHub"
</Tab>
<Tab title="GitHub">
_Requirements_:
* `git` installed.
@@ -46,8 +34,9 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
* `npm` installed: [npm installation](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).
* `Docker Compose` installed: https://docs.docker.com/compose/install/.
???+ warning
Make sure to have `api/.env` and `ui/.env.local` files with the required environment variables. You can find the required environment variables in the [`api/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/api/.env.example) and [`ui/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/ui/.env.template) files.
<Warning>
Make sure to have `api/.env` and `ui/.env.local` files with the required environment variables. You can find the required environment variables in the [`api/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/api/.env.example) and [`ui/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/ui/.env.template) files.
</Warning>
_Commands to run the API_:
@@ -64,11 +53,12 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
gunicorn -c config/guniconf.py config.wsgi:application
```
???+ important
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
<Warning>
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the Poetry environment activation guide: https://python-poetry.org/docs/managing-environments/#activating-the-environment
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the Poetry environment activation guide: https://python-poetry.org/docs/managing-environments/#activating-the-environment
</Warning>
> Now, you can access the API documentation at http://localhost:8080/api/v1/docs.
@@ -110,10 +100,11 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
> Enjoy Prowler App at http://localhost:3000 by signing up with your email and password.
???+ warning
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
<Warning>
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
</Warning>
</Tab>
</Tabs>
### Update Prowler App
Upgrade Prowler App installation using one of two options:
@@ -136,9 +127,12 @@ docker compose pull --policy always
The `--policy always` flag ensures that Docker pulls the latest images even if they already exist locally.
???+ note "What Gets Preserved During Upgrade"
Everything is preserved, nothing will be deleted after the update.
<Note>
**What Gets Preserved During Upgrade**
Everything is preserved, nothing will be deleted after the update.
</Note>
### Troubleshooting
If containers don't start, check logs for errors:
@@ -1,10 +1,13 @@
---
title: 'Installation'
---
## Installation
Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/). Install it as a Python package with `Python >= 3.9, <= 3.12`:
=== "pipx"
<Tabs>
<Tab title="pipx">
[pipx](https://pipx.pypa.io/stable/) installs Python applications in isolated environments. Use `pipx` for global installation.
_Requirements_:
@@ -19,17 +22,11 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
pipx install prowler
prowler -v
```
Upgrade Prowler to the latest version:
``` bash
pipx upgrade prowler
```
=== "pip"
???+ warning
This method modifies the chosen installation environment. Consider using [pipx](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) for global installation.
</Tab>
<Tab title="pip">
<Warning>
This method modifies the chosen installation environment. Consider using [pipx](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) for global installation.
</Warning>
_Requirements_:
@@ -49,9 +46,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
``` bash
pip install --upgrade prowler
```
=== "Docker"
</Tab>
<Tab title="Docker">
_Requirements_:
* Have `docker` installed: https://docs.docker.com/get-docker/.
@@ -69,9 +65,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
--env AWS_SECRET_ACCESS_KEY \
--env AWS_SESSION_TOKEN toniblyx/prowler:latest
```
=== "GitHub"
</Tab>
<Tab title="GitHub">
_Requirements for Developers_:
* `git`
@@ -86,11 +81,12 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
poetry install
poetry run python prowler-cli.py -v
```
???+ note
If you want to clone Prowler from Windows, use `git config core.longpaths true` to allow long file paths.
=== "Amazon Linux 2"
<Note>
If you want to clone Prowler from Windows, use `git config core.longpaths true` to allow long file paths.
</Note>
</Tab>
<Tab title="Amazon Linux 2">
_Requirements_:
* `Python >= 3.9, <= 3.12`
@@ -104,9 +100,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
pipx install prowler
prowler -v
```
=== "Ubuntu"
</Tab>
<Tab title="Ubuntu">
_Requirements_:
* `Ubuntu 23.04` or above, if you are using an older version of Ubuntu check [pipx installation](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) and ensure you have `Python >= 3.9, <= 3.12`.
@@ -122,9 +117,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
pipx install prowler
prowler -v
```
=== "Brew"
</Tab>
<Tab title="Brew">
_Requirements_:
* `Brew` installed in your Mac or Linux
@@ -136,9 +130,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
brew install prowler
prowler -v
```
=== "AWS CloudShell"
</Tab>
<Tab title="AWS CloudShell">
After the migration of AWS CloudShell from Amazon Linux 2 to Amazon Linux 2023 [[1]](https://aws.amazon.com/about-aws/whats-new/2023/12/aws-cloudshell-migrated-al2023/) [[2]](https://docs.aws.amazon.com/cloudshell/latest/userguide/cloudshell-AL2023-migration.html), there is no longer a need to manually compile Python 3.9 as it is already included in AL2023. Prowler can thus be easily installed following the generic method of installation via pip. Follow the steps below to successfully execute Prowler v4 in AWS CloudShell:
_Requirements_:
@@ -158,11 +151,11 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
prowler aws
```
???+ note
To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/tmp/output/prowler-output-123456789012-20221220191331.csv`
=== "Azure CloudShell"
<Note>
To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/tmp/output/prowler-output-123456789012-20221220191331.csv`
</Note>
</Tab>
<Tab title="Azure CloudShell">
_Requirements_:
* Open Azure CloudShell `bash`.
@@ -176,18 +169,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
cd /tmp
prowler azure --az-cli-auth
```
</Tab>
</Tabs>
## Container versions
The available versions of Prowler CLI are the following:
@@ -0,0 +1,276 @@
---
title: "Installation"
---
There are **two ways** to use Prowler MCP Server:
<CardGroup cols={2}>
<Card title="Option 1: Managed by Prowler" icon="cloud" color="#10B981">
**No installation required** - Just configuration
Use `https://mcp.prowler.com/mcp`
</Card>
<Card title="Option 2: Run Locally" icon="server" color="#6366F1">
**Local installation** - Full control
Install via Docker, PyPI, or source code
</Card>
</CardGroup>
For "Option 1: Managed by Prowler", go directly to the [Configuration Guide](/getting-started/basic-usage/prowler-mcp#hosted-server-configuration-recommended) to set up your Claude Desktop, Cursor, or other MCP client.
**This guide is focused on local installation, "Option 2: Run Locally"**.
## Installation Methods
Choose one of the following installation methods:
<Tabs>
<Tab title="Docker (Recommended)">
### Pull from Docker Hub
The easiest way to run locally is using the official Docker image:
```bash
docker pull prowlercloud/prowler-mcp
```
### Run the Container
```bash
# STDIO mode (for local MCP clients)
docker run --rm -i prowlercloud/prowler-mcp
# HTTP mode (for remote access)
docker run --rm -p 8000:8000 \
prowlercloud/prowler-mcp \
--transport http --host 0.0.0.0 --port 8000
```
### With Environment Variables
```bash
docker run --rm -i \
-e PROWLER_APP_API_KEY="pk_your_api_key" \
-e PROWLER_API_BASE_URL="https://api.prowler.com" \
prowlercloud/prowler-mcp
```
<Info>
**Docker Hub:** [prowlercloud/prowler-mcp](https://hub.docker.com/r/prowlercloud/prowler-mcp)
</Info>
</Tab>
<Tab title="PyPI Package">
### Install via pip
<Warning>
**Coming Soon** - PyPI package will be available shortly
</Warning>
</Tab>
<Tab title="From Source (Development)">
### Install uv
If `uv` is not installed, install it first. Visit [uv documentation](https://docs.astral.sh/uv/) for more installation options.
### Clone the Repository
```bash
git clone https://github.com/prowler-cloud/prowler.git
cd prowler/mcp_server
```
### Install Dependencies
The MCP server uses `uv` for dependency management. Install dependencies with:
```bash
uv sync
```
### Verify Installation
Test that the server is properly installed:
```bash
uv run prowler-mcp --help
```
The help message with available command-line options should appear.
<Note>
**For Development:** This method is recommended if you're developing or contributing to the MCP server.
</Note>
</Tab>
<Tab title="Build Docker Image">
### Prerequisites
Ensure Docker is installed on your system. Visit [Docker documentation](https://docs.docker.com/get-docker/) for more installation options.
### Clone the Repository
```bash
git clone https://github.com/prowler-cloud/prowler.git
cd prowler/mcp_server
```
### Build the Docker Image
```bash
docker build -t prowler-mcp .
```
This creates a Docker image named `prowler-mcp` with all necessary dependencies.
### Verify Installation
Test that the Docker image was built successfully:
```bash
docker run --rm prowler-mcp --help
```
The help message with available command-line options should appear.
</Tab>
</Tabs>
---
## Command Line Options
The Prowler MCP Server supports the following command-line arguments:
```bash
prowler-mcp [--transport {stdio,http}] [--host HOST] [--port PORT]
```
| Argument | Values | Default | Description |
|----------|--------|---------|-------------|
| `--transport` | `stdio`, `http` | `stdio` | Transport method for MCP communication |
| `--host` | Any valid hostname/IP | `127.0.0.1` | Host to bind to (HTTP mode only) |
| `--port` | Port number | `8000` | Port to bind to (HTTP mode only) |
### Examples
```bash
# Default STDIO mode for local MCP client integration
prowler-mcp
# Explicit STDIO mode
prowler-mcp --transport stdio
# HTTP mode on default host and port (127.0.0.1:8000)
prowler-mcp --transport http
# HTTP mode accessible from any network interface
prowler-mcp --transport http --host 0.0.0.0
# HTTP mode with custom port
prowler-mcp --transport http --host 0.0.0.0 --port 8080
```
## Environment Variables
Configure the server using environment variables:
| Variable | Description | Required | Default |
|----------|-------------|----------|---------|
| `PROWLER_APP_API_KEY` | Prowler API key | Only for STDIO mode | - |
| `PROWLER_API_BASE_URL` | Custom Prowler API endpoint | No | `https://api.prowler.com` |
| `PROWLER_MCP_TRANSPORT_MODE` | Default transport mode (overwritten by `--transport` argument) | No | `stdio` |
<CodeGroup>
```bash macOS/Linux
export PROWLER_APP_API_KEY="pk_your_api_key_here"
export PROWLER_API_BASE_URL="https://api.prowler.com"
export PROWLER_MCP_TRANSPORT_MODE="http"
```
```bash Windows PowerShell
$env:PROWLER_APP_API_KEY="pk_your_api_key_here"
$env:PROWLER_API_BASE_URL="https://api.prowler.com"
$env:PROWLER_MCP_TRANSPORT_MODE="http"
```
</CodeGroup>
<Warning>
Never commit your API key to version control. Use environment variables or secure secret management solutions.
</Warning>
### Using Environment Files
For convenience, create a `.env` file in the `mcp_server` directory:
```bash .env
PROWLER_APP_API_KEY=pk_your_api_key_here
PROWLER_API_BASE_URL=https://api.prowler.com
PROWLER_MCP_TRANSPORT_MODE=stdio
```
When using Docker, pass the environment file:
```bash
docker run --rm --env-file .env -it prowler-mcp
```
## Running from Any Location
Run the MCP server from anywhere using `uvx`:
```bash
uvx /path/to/prowler/mcp_server/
```
This is particularly useful when configuring MCP clients that need to launch the server from a specific path.
## Production Deployment
For production deployments that require customization, it is recommended to use the ASGI application that can be found in `prowler_mcp_server.server`. This can be run with uvicorn:
```bash
uvicorn prowler_mcp_server.server:app --host 0.0.0.0 --port 8000
```
For more details on production deployment options, see the [FastMCP production deployment guide](https://gofastmcp.com/deployment/http#production-deployment) and [uvicorn settings](https://www.uvicorn.org/settings/).
### Entrypoint Script
The source tree includes `entrypoint.sh` to simplify switching between the
standard CLI runner and the ASGI app. The first argument selects the mode and
any additional flags are passed straight through:
```bash
# Default CLI experience (prowler-mcp console script)
./entrypoint.sh main --transport http --host 0.0.0.0
# ASGI app via uvicorn
./entrypoint.sh uvicorn --host 0.0.0.0 --port 9000
```
Omitting the mode defaults to `main`, matching the `prowler-mcp` console script.
When `uvicorn` mode is selected, the script exports `PROWLER_MCP_TRANSPORT_MODE=http` automatically.
This is the default entrypoint for the Docker container.
## Next Steps
Now that you have the Prowler MCP Server installed, proceed to configure your MCP client:
<CardGroup cols={1}>
<Card title="Configuration" icon="gear" href="/getting-started/basic-usage/prowler-mcp">
Configure Claude Desktop, Cursor, or other MCP clients
</Card>
</CardGroup>
## Getting Help
If you encounter issues during installation:
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)

Before

Width:  |  Height:  |  Size: 313 KiB

After

Width:  |  Height:  |  Size: 313 KiB

Before

Width:  |  Height:  |  Size: 420 KiB

After

Width:  |  Height:  |  Size: 420 KiB

Some files were not shown because too many files have changed in this diff Show More