mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-06-15 09:59:34 +00:00
Compare commits
94 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 0cb4784187 | |||
| 124676e893 | |||
| e08c2f2605 | |||
| 9758fc36df | |||
| 56bb5e92cc | |||
| 2ef750d133 | |||
| 18f3bc098c | |||
| 67b1983d85 | |||
| a3db23af7d | |||
| 3eaa21f06f | |||
| 5d5c109067 | |||
| c6cb4e4814 | |||
| ab06a09173 | |||
| 9c6c007f73 | |||
| 206f23b5a5 | |||
| 5c9e9bc86a | |||
| 34554d6123 | |||
| 000cb93157 | |||
| 524209bdf2 | |||
| c4a0da8204 | |||
| f0cba0321c | |||
| 79888c9312 | |||
| a79910a694 | |||
| 4cadee7bb1 | |||
| 756d436a2f | |||
| 5e85ef5835 | |||
| 0fa9e2da6c | |||
| ce7510db28 | |||
| 8e3d50c807 | |||
| d8908d2ccc | |||
| 0b9969a723 | |||
| 985d73f44f | |||
| 1d705e22da | |||
| ca55d4ce86 | |||
| 0201073fcb | |||
| 928c556721 | |||
| a653ad7852 | |||
| a3c811f801 | |||
| c85d3e9188 | |||
| 6f394cf9de | |||
| ba765fa07d | |||
| d928ee442f | |||
| 30ab5f52b9 | |||
| c424707e32 | |||
| 92efbe3926 | |||
| 4a61578dd8 | |||
| ec75b5d0a3 | |||
| db5bab51ae | |||
| be476b732a | |||
| 434b37f758 | |||
| c08c27e5c6 | |||
| 8773751779 | |||
| f70a959a49 | |||
| 20314cad8c | |||
| 564ad56d2f | |||
| b2d91c97d8 | |||
| c232195df4 | |||
| b4b9d800a8 | |||
| fc1d3d4a47 | |||
| d4be0f4d7a | |||
| 305339ffb4 | |||
| 272e4547b2 | |||
| 8c3e1b96f9 | |||
| d496f5a58e | |||
| 5789e87f4f | |||
| 1994750151 | |||
| 27304a8007 | |||
| 9761651f8d | |||
| 406aace585 | |||
| f734b249a4 | |||
| ebd5814112 | |||
| 42e816081e | |||
| 741217ce80 | |||
| 5f9ab68bd9 | |||
| fba2854f65 | |||
| 8794515318 | |||
| 335db928dc | |||
| 046baa8eb9 | |||
| ef60ea99c3 | |||
| 1483efa18e | |||
| b74744b135 | |||
| e80eed6baf | |||
| 1ba22f6f45 | |||
| da6b7b89cb | |||
| cc9aa7f7ee | |||
| ecf749fce8 | |||
| 1a7f52fc9c | |||
| b630234cdf | |||
| d6685eec1f | |||
| 86cff92d1f | |||
| 28e81783ef | |||
| 13266b8743 | |||
| 4e143cf013 | |||
| 5cfe140b7b |
@@ -29,6 +29,12 @@ POSTGRES_ADMIN_PASSWORD=postgres
|
||||
POSTGRES_USER=prowler
|
||||
POSTGRES_PASSWORD=postgres
|
||||
POSTGRES_DB=prowler_db
|
||||
# Read replica settings (optional)
|
||||
# POSTGRES_REPLICA_HOST=postgres-db
|
||||
# POSTGRES_REPLICA_PORT=5432
|
||||
# POSTGRES_REPLICA_USER=prowler
|
||||
# POSTGRES_REPLICA_PASSWORD=postgres
|
||||
# POSTGRES_REPLICA_DB=prowler_db
|
||||
|
||||
# Celery-Prowler task settings
|
||||
TASK_RETRY_DELAY_SECONDS=0.1
|
||||
|
||||
+27
-5
@@ -1,6 +1,28 @@
|
||||
# SDK
|
||||
/* @prowler-cloud/sdk
|
||||
/.github/ @prowler-cloud/sdk
|
||||
prowler @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
|
||||
tests @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
|
||||
api @prowler-cloud/api
|
||||
ui @prowler-cloud/ui
|
||||
/prowler/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
|
||||
/tests/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
|
||||
/dashboard/ @prowler-cloud/sdk
|
||||
/docs/ @prowler-cloud/sdk
|
||||
/examples/ @prowler-cloud/sdk
|
||||
/util/ @prowler-cloud/sdk
|
||||
/contrib/ @prowler-cloud/sdk
|
||||
/permissions/ @prowler-cloud/sdk
|
||||
/codecov.yml @prowler-cloud/sdk @prowler-cloud/api
|
||||
|
||||
# API
|
||||
/api/ @prowler-cloud/api
|
||||
|
||||
# UI
|
||||
/ui/ @prowler-cloud/ui
|
||||
|
||||
# AI
|
||||
/mcp_server/ @prowler-cloud/ai
|
||||
|
||||
# Platform
|
||||
/.github/ @prowler-cloud/platform
|
||||
/Makefile @prowler-cloud/platform
|
||||
/kubernetes/ @prowler-cloud/platform
|
||||
**/Dockerfile* @prowler-cloud/platform
|
||||
**/docker-compose*.yml @prowler-cloud/platform
|
||||
**/docker-compose*.yaml @prowler-cloud/platform
|
||||
|
||||
@@ -3,6 +3,41 @@ description: Create a report to help us improve
|
||||
labels: ["bug", "status/needs-triage"]
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Issue search
|
||||
options:
|
||||
- label: I have searched the existing issues and this bug has not been reported yet
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: component
|
||||
attributes:
|
||||
label: Which component is affected?
|
||||
multiple: true
|
||||
options:
|
||||
- Prowler CLI/SDK
|
||||
- Prowler API
|
||||
- Prowler UI
|
||||
- Prowler Dashboard
|
||||
- Prowler MCP Server
|
||||
- Documentation
|
||||
- Other
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: provider
|
||||
attributes:
|
||||
label: Cloud Provider (if applicable)
|
||||
multiple: true
|
||||
options:
|
||||
- AWS
|
||||
- Azure
|
||||
- GCP
|
||||
- Kubernetes
|
||||
- GitHub
|
||||
- Microsoft 365
|
||||
- Not applicable
|
||||
- type: textarea
|
||||
id: reproduce
|
||||
attributes:
|
||||
@@ -78,6 +113,15 @@ body:
|
||||
prowler --version
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: python-version
|
||||
attributes:
|
||||
label: Python version
|
||||
description: Which Python version are you using?
|
||||
placeholder: |-
|
||||
python --version
|
||||
validations:
|
||||
required: true
|
||||
- type: input
|
||||
id: pip-version
|
||||
attributes:
|
||||
|
||||
@@ -1 +1,11 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: 📖 Documentation
|
||||
url: https://docs.prowler.com
|
||||
about: Check our comprehensive documentation for guides and tutorials
|
||||
- name: 💬 GitHub Discussions
|
||||
url: https://github.com/prowler-cloud/prowler/discussions
|
||||
about: Ask questions and discuss with the community
|
||||
- name: 🌟 Prowler Community
|
||||
url: https://goto.prowler.com/slack
|
||||
about: Join our community for support and updates
|
||||
|
||||
@@ -3,6 +3,42 @@ description: Suggest an idea for this project
|
||||
labels: ["feature-request", "status/needs-triage"]
|
||||
|
||||
body:
|
||||
- type: checkboxes
|
||||
id: search
|
||||
attributes:
|
||||
label: Feature search
|
||||
options:
|
||||
- label: I have searched the existing issues and this feature has not been requested yet
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: component
|
||||
attributes:
|
||||
label: Which component would this feature affect?
|
||||
multiple: true
|
||||
options:
|
||||
- Prowler CLI/SDK
|
||||
- Prowler API
|
||||
- Prowler UI
|
||||
- Prowler Dashboard
|
||||
- Prowler MCP Server
|
||||
- Documentation
|
||||
- New component/Integration
|
||||
validations:
|
||||
required: true
|
||||
- type: dropdown
|
||||
id: provider
|
||||
attributes:
|
||||
label: Related to specific cloud provider?
|
||||
multiple: true
|
||||
options:
|
||||
- AWS
|
||||
- Azure
|
||||
- GCP
|
||||
- Kubernetes
|
||||
- GitHub
|
||||
- Microsoft 365
|
||||
- All providers
|
||||
- Not provider-specific
|
||||
- type: textarea
|
||||
id: Problem
|
||||
attributes:
|
||||
@@ -19,6 +55,14 @@ body:
|
||||
description: A clear and concise description of what you want to happen.
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: use-case
|
||||
attributes:
|
||||
label: Use case and benefits
|
||||
description: Who would benefit from this feature and how?
|
||||
placeholder: This would help security teams by...
|
||||
validations:
|
||||
required: true
|
||||
- type: textarea
|
||||
id: Alternatives
|
||||
attributes:
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
name: 'Setup Python with Poetry'
|
||||
description: 'Setup Python environment with Poetry and install dependencies'
|
||||
author: 'Prowler'
|
||||
|
||||
inputs:
|
||||
python-version:
|
||||
description: 'Python version to use'
|
||||
required: true
|
||||
working-directory:
|
||||
description: 'Working directory for Poetry'
|
||||
required: false
|
||||
default: '.'
|
||||
poetry-version:
|
||||
description: 'Poetry version to install'
|
||||
required: false
|
||||
default: '2.1.1'
|
||||
install-dependencies:
|
||||
description: 'Install Python dependencies with Poetry'
|
||||
required: false
|
||||
default: 'true'
|
||||
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Replace @master with current branch in pyproject.toml
|
||||
if: github.event_name == 'pull_request' && github.base_ref == 'master'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
|
||||
echo "Using branch: $BRANCH_NAME"
|
||||
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
|
||||
|
||||
- name: Install poetry
|
||||
shell: bash
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pipx install poetry==${{ inputs.poetry-version }}
|
||||
|
||||
- name: Update SDK resolved_reference to latest commit
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
|
||||
echo "Latest commit hash: $LATEST_COMMIT"
|
||||
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
|
||||
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
|
||||
}' poetry.lock
|
||||
echo "Updated resolved_reference:"
|
||||
grep -A2 -B2 "resolved_reference" poetry.lock
|
||||
|
||||
- name: Update poetry.lock
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: poetry lock
|
||||
|
||||
- name: Set up Python ${{ inputs.python-version }}
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ inputs.python-version }}
|
||||
cache: 'poetry'
|
||||
cache-dependency-path: ${{ inputs.working-directory }}/poetry.lock
|
||||
|
||||
- name: Install Python dependencies
|
||||
if: inputs.install-dependencies == 'true'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
poetry install --no-root
|
||||
poetry run pip list
|
||||
@@ -0,0 +1,152 @@
|
||||
name: 'Container Security Scan with Trivy'
|
||||
description: 'Scans container images for vulnerabilities using Trivy and reports results'
|
||||
author: 'Prowler'
|
||||
|
||||
inputs:
|
||||
image-name:
|
||||
description: 'Container image name to scan'
|
||||
required: true
|
||||
image-tag:
|
||||
description: 'Container image tag to scan'
|
||||
required: true
|
||||
default: ${{ github.sha }}
|
||||
severity:
|
||||
description: 'Severities to scan for (comma-separated)'
|
||||
required: false
|
||||
default: 'CRITICAL,HIGH,MEDIUM,LOW'
|
||||
fail-on-critical:
|
||||
description: 'Fail the build if critical vulnerabilities are found'
|
||||
required: false
|
||||
default: 'false'
|
||||
upload-sarif:
|
||||
description: 'Upload results to GitHub Security tab'
|
||||
required: false
|
||||
default: 'true'
|
||||
create-pr-comment:
|
||||
description: 'Create a comment on the PR with scan results'
|
||||
required: false
|
||||
default: 'true'
|
||||
artifact-retention-days:
|
||||
description: 'Days to retain the Trivy report artifact'
|
||||
required: false
|
||||
default: '2'
|
||||
|
||||
outputs:
|
||||
critical-count:
|
||||
description: 'Number of critical vulnerabilities found'
|
||||
value: ${{ steps.security-check.outputs.critical }}
|
||||
high-count:
|
||||
description: 'Number of high vulnerabilities found'
|
||||
value: ${{ steps.security-check.outputs.high }}
|
||||
total-count:
|
||||
description: 'Total number of vulnerabilities found'
|
||||
value: ${{ steps.security-check.outputs.total }}
|
||||
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Run Trivy vulnerability scan (SARIF)
|
||||
if: inputs.upload-sarif == 'true'
|
||||
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
|
||||
with:
|
||||
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
|
||||
format: 'sarif'
|
||||
output: 'trivy-results.sarif'
|
||||
severity: 'CRITICAL,HIGH'
|
||||
exit-code: '0'
|
||||
|
||||
- name: Upload Trivy results to GitHub Security tab
|
||||
if: inputs.upload-sarif == 'true'
|
||||
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
sarif_file: 'trivy-results.sarif'
|
||||
category: 'trivy-container'
|
||||
|
||||
- name: Run Trivy vulnerability scan (JSON)
|
||||
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
|
||||
with:
|
||||
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
|
||||
format: 'json'
|
||||
output: 'trivy-report.json'
|
||||
severity: ${{ inputs.severity }}
|
||||
exit-code: '0'
|
||||
|
||||
- name: Upload Trivy report artifact
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
if: always()
|
||||
with:
|
||||
name: trivy-scan-report-${{ inputs.image-name }}
|
||||
path: trivy-report.json
|
||||
retention-days: ${{ inputs.artifact-retention-days }}
|
||||
|
||||
- name: Generate security summary
|
||||
id: security-check
|
||||
shell: bash
|
||||
run: |
|
||||
CRITICAL=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="CRITICAL")] | length' trivy-report.json)
|
||||
HIGH=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="HIGH")] | length' trivy-report.json)
|
||||
TOTAL=$(jq '[.Results[]?.Vulnerabilities[]?] | length' trivy-report.json)
|
||||
|
||||
echo "critical=$CRITICAL" >> $GITHUB_OUTPUT
|
||||
echo "high=$HIGH" >> $GITHUB_OUTPUT
|
||||
echo "total=$TOTAL" >> $GITHUB_OUTPUT
|
||||
|
||||
echo "### 🔒 Container Security Scan" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "**Image:** \`${{ inputs.image-name }}:${{ inputs.image-tag }}\`" >> $GITHUB_STEP_SUMMARY
|
||||
echo "" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- 🔴 Critical: $CRITICAL" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- 🟠 High: $HIGH" >> $GITHUB_STEP_SUMMARY
|
||||
echo "- **Total**: $TOTAL" >> $GITHUB_STEP_SUMMARY
|
||||
|
||||
- name: Comment scan results on PR
|
||||
if: inputs.create-pr-comment == 'true' && github.event_name == 'pull_request'
|
||||
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
|
||||
env:
|
||||
IMAGE_NAME: ${{ inputs.image-name }}
|
||||
GITHUB_SHA: ${{ inputs.image-tag }}
|
||||
SEVERITY: ${{ inputs.severity }}
|
||||
with:
|
||||
script: |
|
||||
const comment = require('./.github/scripts/trivy-pr-comment.js');
|
||||
|
||||
// Unique identifier to find our comment
|
||||
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
|
||||
const body = marker + '\n' + comment;
|
||||
|
||||
// Find existing comment
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
});
|
||||
|
||||
const existingComment = comments.find(c => c.body?.includes(marker));
|
||||
|
||||
if (existingComment) {
|
||||
// Update existing comment
|
||||
await github.rest.issues.updateComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
comment_id: existingComment.id,
|
||||
body: body
|
||||
});
|
||||
console.log('✅ Updated existing Trivy scan comment');
|
||||
} else {
|
||||
// Create new comment
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: body
|
||||
});
|
||||
console.log('✅ Created new Trivy scan comment');
|
||||
}
|
||||
|
||||
- name: Check for critical vulnerabilities
|
||||
if: inputs.fail-on-critical == 'true' && steps.security-check.outputs.critical != '0'
|
||||
shell: bash
|
||||
run: |
|
||||
echo "::error::Found ${{ steps.security-check.outputs.critical }} critical vulnerabilities"
|
||||
echo "::warning::Please update packages or use a different base image"
|
||||
exit 1
|
||||
@@ -1,3 +1,12 @@
|
||||
name: "API - CodeQL Config"
|
||||
name: 'API: CodeQL Config'
|
||||
paths:
|
||||
- "api/"
|
||||
- 'api/'
|
||||
|
||||
paths-ignore:
|
||||
- 'api/tests/**'
|
||||
- 'api/**/__pycache__/**'
|
||||
- 'api/**/migrations/**'
|
||||
- 'api/**/*.md'
|
||||
|
||||
queries:
|
||||
- uses: security-and-quality
|
||||
|
||||
@@ -42,6 +42,11 @@ provider/mongodbatlas:
|
||||
- any-glob-to-any-file: "prowler/providers/mongodbatlas/**"
|
||||
- any-glob-to-any-file: "tests/providers/mongodbatlas/**"
|
||||
|
||||
provider/oci:
|
||||
- changed-files:
|
||||
- any-glob-to-any-file: "prowler/providers/oraclecloud/**"
|
||||
- any-glob-to-any-file: "tests/providers/oraclecloud/**"
|
||||
|
||||
github_actions:
|
||||
- changed-files:
|
||||
- any-glob-to-any-file: ".github/workflows/*"
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
const fs = require('fs');
|
||||
|
||||
// Configuration from environment variables
|
||||
const REPORT_FILE = process.env.TRIVY_REPORT_FILE || 'trivy-report.json';
|
||||
const IMAGE_NAME = process.env.IMAGE_NAME || 'container-image';
|
||||
const GITHUB_SHA = process.env.GITHUB_SHA || 'unknown';
|
||||
const GITHUB_REPOSITORY = process.env.GITHUB_REPOSITORY || '';
|
||||
const GITHUB_RUN_ID = process.env.GITHUB_RUN_ID || '';
|
||||
const SEVERITY = process.env.SEVERITY || 'CRITICAL,HIGH,MEDIUM,LOW';
|
||||
|
||||
// Parse severities to scan
|
||||
const scannedSeverities = SEVERITY.split(',').map(s => s.trim());
|
||||
|
||||
// Read and parse the Trivy report
|
||||
const report = JSON.parse(fs.readFileSync(REPORT_FILE, 'utf-8'));
|
||||
|
||||
let vulnCount = 0;
|
||||
let vulnsByType = { CRITICAL: 0, HIGH: 0, MEDIUM: 0, LOW: 0 };
|
||||
let affectedPackages = new Set();
|
||||
|
||||
if (report.Results && Array.isArray(report.Results)) {
|
||||
for (const result of report.Results) {
|
||||
if (result.Vulnerabilities && Array.isArray(result.Vulnerabilities)) {
|
||||
for (const vuln of result.Vulnerabilities) {
|
||||
vulnCount++;
|
||||
if (vulnsByType[vuln.Severity] !== undefined) {
|
||||
vulnsByType[vuln.Severity]++;
|
||||
}
|
||||
if (vuln.PkgName) {
|
||||
affectedPackages.add(vuln.PkgName);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const shortSha = GITHUB_SHA.substring(0, 7);
|
||||
const timestamp = new Date().toISOString().replace('T', ' ').substring(0, 19) + ' UTC';
|
||||
|
||||
// Severity icons and labels
|
||||
const severityConfig = {
|
||||
CRITICAL: { icon: '🔴', label: 'Critical' },
|
||||
HIGH: { icon: '🟠', label: 'High' },
|
||||
MEDIUM: { icon: '🟡', label: 'Medium' },
|
||||
LOW: { icon: '🔵', label: 'Low' }
|
||||
};
|
||||
|
||||
let comment = '## 🔒 Container Security Scan\n\n';
|
||||
comment += `**Image:** \`${IMAGE_NAME}:${shortSha}\`\n`;
|
||||
comment += `**Last scan:** ${timestamp}\n\n`;
|
||||
|
||||
if (vulnCount === 0) {
|
||||
comment += '### ✅ No Vulnerabilities Detected\n\n';
|
||||
comment += 'The container image passed all security checks. No known CVEs were found.\n';
|
||||
} else {
|
||||
comment += '### 📊 Vulnerability Summary\n\n';
|
||||
comment += '| Severity | Count |\n';
|
||||
comment += '|----------|-------|\n';
|
||||
|
||||
// Only show severities that were scanned
|
||||
for (const severity of scannedSeverities) {
|
||||
const config = severityConfig[severity];
|
||||
const count = vulnsByType[severity] || 0;
|
||||
const isBold = (severity === 'CRITICAL' || severity === 'HIGH') && count > 0;
|
||||
const countDisplay = isBold ? `**${count}**` : count;
|
||||
comment += `| ${config.icon} ${config.label} | ${countDisplay} |\n`;
|
||||
}
|
||||
|
||||
comment += `| **Total** | **${vulnCount}** |\n\n`;
|
||||
|
||||
if (affectedPackages.size > 0) {
|
||||
comment += `**${affectedPackages.size}** package(s) affected\n\n`;
|
||||
}
|
||||
|
||||
if (vulnsByType.CRITICAL > 0) {
|
||||
comment += '### ⚠️ Action Required\n\n';
|
||||
comment += '**Critical severity vulnerabilities detected.** These should be addressed before merging:\n';
|
||||
comment += '- Review the detailed scan results\n';
|
||||
comment += '- Update affected packages to patched versions\n';
|
||||
comment += '- Consider using a different base image if updates are unavailable\n\n';
|
||||
} else if (vulnsByType.HIGH > 0) {
|
||||
comment += '### ⚠️ Attention Needed\n\n';
|
||||
comment += '**High severity vulnerabilities found.** Please review and plan remediation:\n';
|
||||
comment += '- Assess the risk and exploitability\n';
|
||||
comment += '- Prioritize updates in the next maintenance cycle\n\n';
|
||||
} else {
|
||||
comment += '### ℹ️ Review Recommended\n\n';
|
||||
comment += 'Medium/Low severity vulnerabilities found. Consider addressing during regular maintenance.\n\n';
|
||||
}
|
||||
}
|
||||
|
||||
comment += '---\n';
|
||||
comment += '📋 **Resources:**\n';
|
||||
|
||||
if (GITHUB_REPOSITORY && GITHUB_RUN_ID) {
|
||||
comment += `- [Download full report](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}) (see artifacts)\n`;
|
||||
}
|
||||
|
||||
comment += '- [View in Security tab](https://github.com/' + (GITHUB_REPOSITORY || 'repository') + '/security/code-scanning)\n';
|
||||
comment += '- Scanned with [Trivy](https://github.com/aquasecurity/trivy)\n';
|
||||
|
||||
module.exports = comment;
|
||||
@@ -1,36 +1,34 @@
|
||||
# For most projects, this workflow file will not need changing; you simply need
|
||||
# to commit it to your repository.
|
||||
#
|
||||
# You may wish to alter this file to override the set of languages analyzed,
|
||||
# or to provide custom queries or build logic.
|
||||
#
|
||||
# ******** NOTE ********
|
||||
# We have attempted to detect the languages in your repository. Please check
|
||||
# the `language` matrix defined below to confirm you have the correct set of
|
||||
# supported CodeQL languages.
|
||||
#
|
||||
name: API - CodeQL
|
||||
name: 'API: CodeQL'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- "api/**"
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-codeql.yml'
|
||||
- '.github/codeql/api-codeql-config.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- "api/**"
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-codeql.yml'
|
||||
- '.github/codeql/api-codeql-config.yml'
|
||||
schedule:
|
||||
- cron: '00 12 * * *'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze
|
||||
name: CodeQL Security Analysis
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
@@ -39,21 +37,20 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: [ 'python' ]
|
||||
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
|
||||
language:
|
||||
- 'python'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
config-file: ./.github/codeql/api-codeql-config.yml
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
config-file: ./.github/codeql/api-codeql-config.yml
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
category: "/language:${{matrix.language}}"
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
category: '/language:${{ matrix.language }}'
|
||||
|
||||
@@ -1,20 +1,30 @@
|
||||
name: API - Pull Request
|
||||
name: 'API: Pull Request'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- ".github/workflows/api-pull-request.yml"
|
||||
- "api/**"
|
||||
- '.github/workflows/api-pull-request.yml'
|
||||
- 'api/**'
|
||||
- '!api/docs/**'
|
||||
- '!api/README.md'
|
||||
- '!api/CHANGELOG.md'
|
||||
pull_request:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- ".github/workflows/api-pull-request.yml"
|
||||
- "api/**"
|
||||
- '.github/workflows/api-pull-request.yml'
|
||||
- 'api/**'
|
||||
- '!api/docs/**'
|
||||
- '!api/README.md'
|
||||
- '!api/CHANGELOG.md'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
POSTGRES_HOST: localhost
|
||||
@@ -29,21 +39,94 @@ env:
|
||||
VALKEY_DB: 0
|
||||
API_WORKING_DIR: ./api
|
||||
IMAGE_NAME: prowler-api
|
||||
IGNORE_FILES: |
|
||||
api/docs/**
|
||||
api/README.md
|
||||
api/CHANGELOG.md
|
||||
|
||||
jobs:
|
||||
test:
|
||||
code-quality:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.12"]
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Poetry check
|
||||
run: poetry check --lock
|
||||
|
||||
- name: Ruff lint
|
||||
run: poetry run ruff check . --exclude contrib
|
||||
|
||||
- name: Ruff format
|
||||
run: poetry run ruff format --check . --exclude contrib
|
||||
|
||||
- name: Pylint
|
||||
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
|
||||
|
||||
security-scans:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Bandit
|
||||
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
|
||||
|
||||
- name: Safety
|
||||
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
|
||||
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
|
||||
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
|
||||
|
||||
- name: Vulture
|
||||
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
|
||||
|
||||
tests:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
# Service containers to run with `test`
|
||||
services:
|
||||
# Label used to access the service container
|
||||
postgres:
|
||||
image: postgres
|
||||
env:
|
||||
@@ -52,7 +135,6 @@ jobs:
|
||||
POSTGRES_USER: ${{ env.POSTGRES_USER }}
|
||||
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
|
||||
POSTGRES_DB: ${{ env.POSTGRES_DB }}
|
||||
# Set health checks to wait until postgres has started
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
@@ -66,7 +148,6 @@ jobs:
|
||||
VALKEY_HOST: ${{ env.VALKEY_HOST }}
|
||||
VALKEY_PORT: ${{ env.VALKEY_PORT }}
|
||||
VALKEY_DB: ${{ env.VALKEY_DB }}
|
||||
# Set health checks to wait until postgres has started
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
@@ -76,158 +157,72 @@ jobs:
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Test if changes are in not ignored paths
|
||||
id: are-non-ignored-files-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
api/**
|
||||
.github/workflows/api-pull-request.yml
|
||||
files_ignore: ${{ env.IGNORE_FILES }}
|
||||
|
||||
- name: Replace @master with current branch in pyproject.toml - Only for pull requests to `master`
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'pull_request' && github.base_ref == 'master'
|
||||
run: |
|
||||
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
|
||||
echo "Using branch: $BRANCH_NAME"
|
||||
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
|
||||
|
||||
- name: Install poetry
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pipx install poetry==2.1.1
|
||||
|
||||
- name: Update SDK's poetry.lock resolved_reference to latest commit - Only for push events to `master`
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/master'
|
||||
run: |
|
||||
# Get the latest commit hash from the prowler-cloud/prowler repository
|
||||
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
|
||||
echo "Latest commit hash: $LATEST_COMMIT"
|
||||
|
||||
# Update the resolved_reference specifically for prowler-cloud/prowler repository
|
||||
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
|
||||
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
|
||||
}' poetry.lock
|
||||
|
||||
# Verify the change was made
|
||||
echo "Updated resolved_reference:"
|
||||
grep -A2 -B2 "resolved_reference" poetry.lock
|
||||
|
||||
- name: Update poetry.lock
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry lock
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: "poetry"
|
||||
working-directory: ./api
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry install --no-root
|
||||
poetry run pip list
|
||||
VERSION=$(curl --silent "https://api.github.com/repos/hadolint/hadolint/releases/latest" | \
|
||||
grep '"tag_name":' | \
|
||||
sed -E 's/.*"v([^"]+)".*/\1/' \
|
||||
) && curl -L -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/v${VERSION}/hadolint-Linux-x86_64" \
|
||||
&& chmod +x /tmp/hadolint
|
||||
|
||||
- name: Poetry check
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry check --lock
|
||||
|
||||
- name: Lint with ruff
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run ruff check . --exclude contrib
|
||||
|
||||
- name: Check Format with ruff
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run ruff format --check . --exclude contrib
|
||||
|
||||
- name: Lint with pylint
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
|
||||
|
||||
- name: Bandit
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
|
||||
|
||||
- name: Safety
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
|
||||
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
|
||||
run: |
|
||||
poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
|
||||
|
||||
- name: Vulture
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
|
||||
|
||||
- name: Hadolint
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
/tmp/hadolint Dockerfile --ignore=DL3013
|
||||
|
||||
- name: Test with pytest
|
||||
working-directory: ./api
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest --cov=./src/backend --cov-report=xml src/backend
|
||||
- name: Run tests with pytest
|
||||
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
|
||||
|
||||
- name: Upload coverage reports to Codecov
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: api
|
||||
test-container-build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Test if changes are in not ignored paths
|
||||
id: are-non-ignored-files-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
dockerfile-lint:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
files: api/**
|
||||
files_ignore: ${{ env.IGNORE_FILES }}
|
||||
dockerfile: api/Dockerfile
|
||||
ignore: DL3013
|
||||
|
||||
container-build-and-scan:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
- name: Build Container
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
|
||||
- name: Build container
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.API_WORKING_DIR }}
|
||||
push: false
|
||||
tags: ${{ env.IMAGE_NAME }}:latest
|
||||
outputs: type=docker
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Scan container with Trivy
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
|
||||
@@ -1,28 +1,35 @@
|
||||
name: Prowler - Automatic Backport
|
||||
name: 'Tools: Backport'
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
branches: ['master']
|
||||
types: ['labeled', 'closed']
|
||||
branches:
|
||||
- 'master'
|
||||
types:
|
||||
- 'labeled'
|
||||
- 'closed'
|
||||
paths:
|
||||
- '.github/workflows/backport.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# The prefix of the label that triggers the backport must not contain the branch name
|
||||
# so, for example, if the branch is 'master', the label should be 'backport-to-<branch>'
|
||||
BACKPORT_LABEL_PREFIX: backport-to-
|
||||
BACKPORT_LABEL_IGNORE: was-backported
|
||||
|
||||
jobs:
|
||||
backport:
|
||||
name: Backport PR
|
||||
if: github.event.pull_request.merged == true && !(contains(github.event.pull_request.labels.*.name, 'backport')) && !(contains(github.event.pull_request.labels.*.name, 'was-backported'))
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
id-token: write
|
||||
pull-requests: write
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Check labels
|
||||
id: preview_label_check
|
||||
id: label_check
|
||||
uses: agilepathway/label-checker@c3d16ad512e7cea5961df85ff2486bb774caf3c5 # v1.6.65
|
||||
with:
|
||||
allow_failure: true
|
||||
@@ -31,17 +38,17 @@ jobs:
|
||||
none_of: ${{ env.BACKPORT_LABEL_IGNORE }}
|
||||
repo_token: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Backport Action
|
||||
if: steps.preview_label_check.outputs.label_check == 'success'
|
||||
- name: Backport PR
|
||||
if: steps.label_check.outputs.label_check == 'success'
|
||||
uses: sorenlouv/backport-github-action@ad888e978060bc1b2798690dd9d03c4036560947 # v9.5.1
|
||||
with:
|
||||
github_token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
auto_backport_label_prefix: ${{ env.BACKPORT_LABEL_PREFIX }}
|
||||
|
||||
- name: Info log
|
||||
if: ${{ success() && steps.preview_label_check.outputs.label_check == 'success' }}
|
||||
- name: Display backport info log
|
||||
if: success() && steps.label_check.outputs.label_check == 'success'
|
||||
run: cat ~/.backport/backport.info.log
|
||||
|
||||
- name: Debug log
|
||||
if: ${{ failure() && steps.preview_label_check.outputs.label_check == 'success' }}
|
||||
- name: Display backport debug log
|
||||
if: failure() && steps.label_check.outputs.label_check == 'success'
|
||||
run: cat ~/.backport/backport.debug.log
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
name: Prowler - Pull Request Documentation Link
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v3'
|
||||
paths:
|
||||
- 'docs/**'
|
||||
- '.github/workflows/build-documentation-on-pr.yml'
|
||||
|
||||
env:
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
|
||||
jobs:
|
||||
documentation-link:
|
||||
name: Documentation Link
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Find existing documentation comment
|
||||
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
|
||||
id: find-comment
|
||||
with:
|
||||
issue-number: ${{ env.PR_NUMBER }}
|
||||
comment-author: 'github-actions[bot]'
|
||||
body-includes: '<!-- prowler-docs-link -->'
|
||||
|
||||
- name: Create or update PR comment with the Prowler Documentation URI
|
||||
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
|
||||
with:
|
||||
comment-id: ${{ steps.find-comment.outputs.comment-id }}
|
||||
issue-number: ${{ env.PR_NUMBER }}
|
||||
body: |
|
||||
<!-- prowler-docs-link -->
|
||||
You can check the documentation for this PR here -> [Prowler Documentation](https://prowler-prowler-docs--${{ env.PR_NUMBER }}.com.readthedocs.build/projects/prowler-open-source/en/${{ env.PR_NUMBER }}/)
|
||||
edit-mode: replace
|
||||
@@ -1,23 +1,31 @@
|
||||
name: Prowler - Conventional Commit
|
||||
name: 'Tools: Conventional Commit'
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types:
|
||||
- "opened"
|
||||
- "edited"
|
||||
- "synchronize"
|
||||
branches:
|
||||
- "master"
|
||||
- "v3"
|
||||
- "v4.*"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v3'
|
||||
- 'v4.*'
|
||||
- 'v5.*'
|
||||
types:
|
||||
- 'opened'
|
||||
- 'edited'
|
||||
- 'synchronize'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
conventional-commit-check:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
|
||||
steps:
|
||||
- name: conventional-commit-check
|
||||
id: conventional-commit-check
|
||||
- name: Check PR title format
|
||||
uses: agenthunt/conventional-commit-checker-action@9e552d650d0e205553ec7792d447929fc78e012b # v2.0.0
|
||||
with:
|
||||
pr-title-regex: '^([^\s(]+)(?:\(([^)]+)\))?: (.+)'
|
||||
pr-title-regex: '^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\([^)]+\))?!?: .+'
|
||||
|
||||
@@ -1,67 +1,70 @@
|
||||
name: Prowler - Create Backport Label
|
||||
name: 'Tools: Backport Label'
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
BACKPORT_LABEL_PREFIX: backport-to-
|
||||
BACKPORT_LABEL_COLOR: B60205
|
||||
|
||||
jobs:
|
||||
create_label:
|
||||
create-label:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: write
|
||||
contents: read
|
||||
issues: write
|
||||
|
||||
steps:
|
||||
- name: Create backport label
|
||||
- name: Create backport label for minor releases
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
OWNER_REPO: ${{ github.repository }}
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
run: |
|
||||
VERSION_ONLY=${RELEASE_TAG#v} # Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
|
||||
RELEASE_TAG="${{ github.event.release.tag_name }}"
|
||||
|
||||
if [ -z "$RELEASE_TAG" ]; then
|
||||
echo "Error: No release tag provided"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Processing release tag: $RELEASE_TAG"
|
||||
|
||||
# Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
|
||||
VERSION_ONLY="${RELEASE_TAG#v}"
|
||||
|
||||
# Check if it's a minor version (X.Y.0)
|
||||
if [[ "$VERSION_ONLY" =~ ^[0-9]+\.[0-9]+\.0$ ]]; then
|
||||
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is a minor version. Proceeding to create backport label."
|
||||
if [[ "$VERSION_ONLY" =~ ^([0-9]+)\.([0-9]+)\.0$ ]]; then
|
||||
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is a minor version. Proceeding to create backport label."
|
||||
|
||||
TWO_DIGIT_VERSION=${VERSION_ONLY%.0} # Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
|
||||
# Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
|
||||
MAJOR="${BASH_REMATCH[1]}"
|
||||
MINOR="${BASH_REMATCH[2]}"
|
||||
TWO_DIGIT_VERSION="${MAJOR}.${MINOR}"
|
||||
|
||||
FINAL_LABEL_NAME="backport-to-v${TWO_DIGIT_VERSION}"
|
||||
FINAL_DESCRIPTION="Backport PR to the v${TWO_DIGIT_VERSION} branch"
|
||||
LABEL_NAME="${BACKPORT_LABEL_PREFIX}v${TWO_DIGIT_VERSION}"
|
||||
LABEL_DESC="Backport PR to the v${TWO_DIGIT_VERSION} branch"
|
||||
LABEL_COLOR="$BACKPORT_LABEL_COLOR"
|
||||
|
||||
echo "Effective label name will be: ${FINAL_LABEL_NAME}"
|
||||
echo "Effective description will be: ${FINAL_DESCRIPTION}"
|
||||
echo "Label name: $LABEL_NAME"
|
||||
echo "Label description: $LABEL_DESC"
|
||||
|
||||
# Check if the label already exists
|
||||
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER_REPO}/labels/${FINAL_LABEL_NAME}")
|
||||
|
||||
if [ "${STATUS_CODE}" -eq 200 ]; then
|
||||
echo "Label '${FINAL_LABEL_NAME}' already exists."
|
||||
elif [ "${STATUS_CODE}" -eq 404 ]; then
|
||||
echo "Label '${FINAL_LABEL_NAME}' does not exist. Creating it..."
|
||||
# Prepare JSON data payload
|
||||
JSON_DATA=$(printf '{"name":"%s","description":"%s","color":"B60205"}' "${FINAL_LABEL_NAME}" "${FINAL_DESCRIPTION}")
|
||||
|
||||
CREATE_STATUS_CODE=$(curl -s -o /tmp/curl_create_response.json -w "%{http_code}" -X POST \
|
||||
-H "Accept: application/vnd.github.v3+json" \
|
||||
-H "Authorization: token ${GITHUB_TOKEN}" \
|
||||
--data "${JSON_DATA}" \
|
||||
"https://api.github.com/repos/${OWNER_REPO}/labels")
|
||||
|
||||
CREATE_RESPONSE_BODY=$(cat /tmp/curl_create_response.json)
|
||||
rm -f /tmp/curl_create_response.json
|
||||
|
||||
if [ "$CREATE_STATUS_CODE" -eq 201 ]; then
|
||||
echo "Label '${FINAL_LABEL_NAME}' created successfully."
|
||||
else
|
||||
echo "Error creating label '${FINAL_LABEL_NAME}'. Status: $CREATE_STATUS_CODE"
|
||||
echo "Response: $CREATE_RESPONSE_BODY"
|
||||
exit 1
|
||||
fi
|
||||
# Check if label already exists
|
||||
if gh label list --repo ${{ github.repository }} --limit 1000 | grep -q "^${LABEL_NAME}[[:space:]]"; then
|
||||
echo "Label '$LABEL_NAME' already exists."
|
||||
else
|
||||
echo "Error checking for label '${FINAL_LABEL_NAME}'. HTTP Status: ${STATUS_CODE}"
|
||||
exit 1
|
||||
echo "Label '$LABEL_NAME' does not exist. Creating it..."
|
||||
gh label create "$LABEL_NAME" \
|
||||
--description "$LABEL_DESC" \
|
||||
--color "$LABEL_COLOR" \
|
||||
--repo ${{ github.repository }}
|
||||
echo "Label '$LABEL_NAME' created successfully."
|
||||
fi
|
||||
else
|
||||
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is not a minor version. Skipping backport label creation."
|
||||
exit 0
|
||||
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is not a minor version. Skipping backport label creation."
|
||||
fi
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: Prowler - Find secrets
|
||||
name: 'Tools: TruffleHog'
|
||||
|
||||
on: pull_request
|
||||
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
name: 'MCP: Container Build and Push'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
paths:
|
||||
- "mcp_server/**"
|
||||
- ".github/workflows/mcp-container-build-push.yml"
|
||||
|
||||
# Uncomment to test this workflow on PRs
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - "master"
|
||||
# paths:
|
||||
# - "mcp_server/**"
|
||||
# - ".github/workflows/mcp-container-build-push.yml"
|
||||
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
STABLE_TAG: stable
|
||||
WORKING_DIRECTORY: ./mcp_server
|
||||
|
||||
# Container registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-mcp
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
|
||||
steps:
|
||||
- name: Calculate short SHA
|
||||
id: set-short-sha
|
||||
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
|
||||
|
||||
container-build-push:
|
||||
needs: setup
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push container (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
|
||||
labels: |
|
||||
org.opencontainers.image.title=Prowler MCP Server
|
||||
org.opencontainers.image.description=Model Context Protocol server for Prowler
|
||||
org.opencontainers.image.vendor=ProwlerPro, Inc.
|
||||
org.opencontainers.image.source=https://github.com/${{ github.repository }}
|
||||
org.opencontainers.image.revision=${{ github.sha }}
|
||||
org.opencontainers.image.created=${{ github.event.head_commit.timestamp }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Build and push container (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
labels: |
|
||||
org.opencontainers.image.title=Prowler MCP Server
|
||||
org.opencontainers.image.description=Model Context Protocol server for Prowler
|
||||
org.opencontainers.image.vendor=ProwlerPro, Inc.
|
||||
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
|
||||
org.opencontainers.image.source=https://github.com/${{ github.repository }}
|
||||
org.opencontainers.image.revision=${{ github.sha }}
|
||||
org.opencontainers.image.created=${{ github.event.release.published_at }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Trigger deployment
|
||||
if: github.event_name == 'push'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.CLOUD_DISPATCH }}
|
||||
event-type: mcp-prowler-deployment
|
||||
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
|
||||
@@ -76,10 +76,12 @@ jobs:
|
||||
UI_VERSION=$(extract_latest_version "ui/CHANGELOG.md")
|
||||
API_VERSION=$(extract_latest_version "api/CHANGELOG.md")
|
||||
SDK_VERSION=$(extract_latest_version "prowler/CHANGELOG.md")
|
||||
MCP_VERSION=$(extract_latest_version "mcp_server/CHANGELOG.md")
|
||||
|
||||
echo "UI_VERSION=${UI_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "API_VERSION=${API_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "SDK_VERSION=${SDK_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "MCP_VERSION=${MCP_VERSION}" >> "${GITHUB_ENV}"
|
||||
|
||||
if [ -n "$UI_VERSION" ]; then
|
||||
echo "Read UI version from changelog: $UI_VERSION"
|
||||
@@ -99,11 +101,18 @@ jobs:
|
||||
echo "Warning: No SDK version found in prowler/CHANGELOG.md"
|
||||
fi
|
||||
|
||||
if [ -n "$MCP_VERSION" ]; then
|
||||
echo "Read MCP version from changelog: $MCP_VERSION"
|
||||
else
|
||||
echo "Warning: No MCP version found in mcp_server/CHANGELOG.md"
|
||||
fi
|
||||
|
||||
echo "Prowler version: $PROWLER_VERSION"
|
||||
echo "Branch name: $BRANCH_NAME"
|
||||
echo "UI version: $UI_VERSION"
|
||||
echo "API version: $API_VERSION"
|
||||
echo "SDK version: $SDK_VERSION"
|
||||
echo "MCP version: $MCP_VERSION"
|
||||
echo "Is minor release: $([ $PATCH_VERSION -eq 0 ] && echo 'true' || echo 'false')"
|
||||
else
|
||||
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
|
||||
@@ -183,7 +192,26 @@ jobs:
|
||||
touch "prowler_changelog.md"
|
||||
fi
|
||||
|
||||
# Combine changelogs in order: UI, API, SDK
|
||||
# MCP has changes if the changelog references this Prowler version
|
||||
# Check if the changelog contains "(Prowler X.Y.Z)" or "(Prowler UNRELEASED)"
|
||||
if [ -f "mcp_server/CHANGELOG.md" ]; then
|
||||
MCP_PROWLER_REF=$(grep -m 1 "^## \[.*\] (Prowler" mcp_server/CHANGELOG.md | sed -E 's/.*\(Prowler ([^)]+)\).*/\1/' | tr -d '[:space:]')
|
||||
if [ "$MCP_PROWLER_REF" = "$PROWLER_VERSION" ] || [ "$MCP_PROWLER_REF" = "UNRELEASED" ]; then
|
||||
echo "HAS_MCP_CHANGES=true" >> $GITHUB_ENV
|
||||
echo "✓ MCP changes detected - Prowler reference: $MCP_PROWLER_REF (version: $MCP_VERSION)"
|
||||
extract_changelog "mcp_server/CHANGELOG.md" "$MCP_VERSION" "mcp_changelog.md"
|
||||
else
|
||||
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
|
||||
echo "ℹ No MCP changes for this release (Prowler reference: $MCP_PROWLER_REF, input: $PROWLER_VERSION)"
|
||||
touch "mcp_changelog.md"
|
||||
fi
|
||||
else
|
||||
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
|
||||
echo "ℹ No MCP changelog found"
|
||||
touch "mcp_changelog.md"
|
||||
fi
|
||||
|
||||
# Combine changelogs in order: UI, API, SDK, MCP
|
||||
> combined_changelog.md
|
||||
|
||||
if [ "$HAS_UI_CHANGES" = "true" ] && [ -s "ui_changelog.md" ]; then
|
||||
@@ -207,6 +235,13 @@ jobs:
|
||||
echo "" >> combined_changelog.md
|
||||
fi
|
||||
|
||||
if [ "$HAS_MCP_CHANGES" = "true" ] && [ -s "mcp_changelog.md" ]; then
|
||||
echo "## MCP" >> combined_changelog.md
|
||||
echo "" >> combined_changelog.md
|
||||
cat mcp_changelog.md >> combined_changelog.md
|
||||
echo "" >> combined_changelog.md
|
||||
fi
|
||||
|
||||
echo "Combined changelog preview:"
|
||||
cat combined_changelog.md
|
||||
|
||||
@@ -367,4 +402,4 @@ jobs:
|
||||
|
||||
- name: Clean up temporary files
|
||||
run: |
|
||||
rm -f prowler_changelog.md api_changelog.md ui_changelog.md combined_changelog.md
|
||||
rm -f prowler_changelog.md api_changelog.md ui_changelog.md mcp_changelog.md combined_changelog.md
|
||||
|
||||
@@ -13,7 +13,7 @@ jobs:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
env:
|
||||
MONITORED_FOLDERS: "api ui prowler dashboard"
|
||||
MONITORED_FOLDERS: "api ui prowler mcp_server"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
@@ -249,6 +249,21 @@ jobs:
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodb_atlas_coverage.xml tests/providers/mongodbatlas
|
||||
|
||||
# Test OCI
|
||||
- name: OCI - Check if any file has changed
|
||||
id: oci-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/oraclecloud/**
|
||||
./tests/providers/oraclecloud/**
|
||||
./poetry.lock
|
||||
|
||||
- name: OCI - Test
|
||||
if: steps.oci-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oci_coverage.xml tests/providers/oraclecloud
|
||||
|
||||
# Common Tests
|
||||
- name: Lib - Test
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
@@ -268,4 +283,4 @@ jobs:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler
|
||||
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./lib_coverage.xml,./config_coverage.xml
|
||||
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./oci_coverage.xml,./lib_coverage.xml,./config_coverage.xml
|
||||
|
||||
@@ -82,12 +82,13 @@ prowler dashboard
|
||||
|
||||
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Stage | Interface |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| AWS | 576 | 82 | 36 | 10 | Official | Stable | UI, API, CLI |
|
||||
| GCP | 79 | 13 | 10 | 3 | Official | Stable | UI, API, CLI |
|
||||
| Azure | 162 | 19 | 11 | 4 | Official | Stable | UI, API, CLI |
|
||||
| AWS | 576 | 82 | 38 | 10 | Official | Stable | UI, API, CLI |
|
||||
| GCP | 79 | 13 | 11 | 3 | Official | Stable | UI, API, CLI |
|
||||
| Azure | 162 | 19 | 12 | 4 | Official | Stable | UI, API, CLI |
|
||||
| Kubernetes | 83 | 7 | 5 | 7 | Official | Stable | UI, API, CLI |
|
||||
| GitHub | 17 | 2 | 1 | 0 | Official | Stable | UI, API, CLI |
|
||||
| M365 | 70 | 7 | 3 | 2 | Official | Stable | UI, API, CLI |
|
||||
| OCI | 51 | 13 | 1 | 10 | Official | Stable | CLI |
|
||||
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | Beta | CLI |
|
||||
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | Beta | CLI |
|
||||
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | Beta | CLI |
|
||||
|
||||
@@ -7,7 +7,14 @@ All notable changes to the **Prowler API** are documented in this file.
|
||||
### Added
|
||||
- Default JWT keys are generated and stored if they are missing from configuration [(#8655)](https://github.com/prowler-cloud/prowler/pull/8655)
|
||||
- `compliance_name` for each compliance [(#7920)](https://github.com/prowler-cloud/prowler/pull/7920)
|
||||
- Support C5 compliance framework for the AWS provider [(#8830)](https://github.com/prowler-cloud/prowler/pull/8830)
|
||||
- Support for M365 Certificate authentication [(#8538)](https://github.com/prowler-cloud/prowler/pull/8538)
|
||||
- API Key support [(#8805)](https://github.com/prowler-cloud/prowler/pull/8805)
|
||||
- SAML role mapping protection for single-admin tenants to prevent accidental lockout [(#8882)](https://github.com/prowler-cloud/prowler/pull/8882)
|
||||
- Support for `passed_findings` and `total_findings` fields in compliance requirement overview for accurate Prowler ThreatScore calculation [(#8582)](https://github.com/prowler-cloud/prowler/pull/8582)
|
||||
- Database read replica support [(#8869)](https://github.com/prowler-cloud/prowler/pull/8869)
|
||||
- Support Common Cloud Controls for AWS, Azure and GCP [(#8000)](https://github.com/prowler-cloud/prowler/pull/8000)
|
||||
- Add `provider_id__in` filter support to findings and findings severity overview endpoints [(#8951)](https://github.com/prowler-cloud/prowler/pull/8951)
|
||||
|
||||
### Changed
|
||||
- Now the MANAGE_ACCOUNT permission is required to modify or read user permissions instead of MANAGE_USERS [(#8281)](https://github.com/prowler-cloud/prowler/pull/8281)
|
||||
|
||||
Generated
+4
-4
@@ -1,4 +1,4 @@
|
||||
# This file is automatically @generated by Poetry 2.1.4 and should not be changed by hand.
|
||||
# This file is automatically @generated by Poetry 2.2.0 and should not be changed by hand.
|
||||
|
||||
[[package]]
|
||||
name = "about-time"
|
||||
@@ -273,14 +273,14 @@ tests-mypy = ["mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" a
|
||||
|
||||
[[package]]
|
||||
name = "authlib"
|
||||
version = "1.6.4"
|
||||
version = "1.6.5"
|
||||
description = "The ultimate Python library in building OAuth and OpenID Connect servers and clients."
|
||||
optional = false
|
||||
python-versions = ">=3.9"
|
||||
groups = ["dev"]
|
||||
files = [
|
||||
{file = "authlib-1.6.4-py2.py3-none-any.whl", hash = "sha256:39313d2a2caac3ecf6d8f95fbebdfd30ae6ea6ae6a6db794d976405fdd9aa796"},
|
||||
{file = "authlib-1.6.4.tar.gz", hash = "sha256:104b0442a43061dc8bc23b133d1d06a2b0a9c2e3e33f34c4338929e816287649"},
|
||||
{file = "authlib-1.6.5-py2.py3-none-any.whl", hash = "sha256:3e0e0507807f842b02175507bdee8957a1d5707fd4afb17c32fb43fee90b6e3a"},
|
||||
{file = "authlib-1.6.5.tar.gz", hash = "sha256:6aaf9c79b7cc96c900f0b284061691c5d4e61221640a948fe690b556a6d6d10b"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
|
||||
@@ -10,6 +10,7 @@ from rest_framework.exceptions import AuthenticationFailed
|
||||
from rest_framework.request import Request
|
||||
from rest_framework_simplejwt.authentication import JWTAuthentication
|
||||
|
||||
from api.db_router import MainRouter
|
||||
from api.models import TenantAPIKey, TenantAPIKeyManager
|
||||
|
||||
|
||||
@@ -19,6 +20,22 @@ class TenantAPIKeyAuthentication(BaseAPIKeyAuth):
|
||||
def __init__(self):
|
||||
self.key_crypto = get_crypto()
|
||||
|
||||
def _authenticate_credentials(self, request, key):
|
||||
"""
|
||||
Override to use admin connection, bypassing RLS during authentication.
|
||||
Delegates to parent after temporarily routing model queries to admin DB.
|
||||
"""
|
||||
# Temporarily point the model's manager to admin database
|
||||
original_objects = self.model.objects
|
||||
self.model.objects = self.model.objects.using(MainRouter.admin_db)
|
||||
|
||||
try:
|
||||
# Call parent method which will now use admin database
|
||||
return super()._authenticate_credentials(request, key)
|
||||
finally:
|
||||
# Restore original manager
|
||||
self.model.objects = original_objects
|
||||
|
||||
def authenticate(self, request: Request):
|
||||
prefixed_key = self.get_key(request)
|
||||
|
||||
@@ -43,13 +60,15 @@ class TenantAPIKeyAuthentication(BaseAPIKeyAuth):
|
||||
api_key_pk = UUID(api_key_pk)
|
||||
|
||||
try:
|
||||
api_key_instance = TenantAPIKey.objects.get(id=api_key_pk, prefix=prefix)
|
||||
api_key_instance = TenantAPIKey.objects.using(MainRouter.admin_db).get(
|
||||
id=api_key_pk, prefix=prefix
|
||||
)
|
||||
except TenantAPIKey.DoesNotExist:
|
||||
raise AuthenticationFailed("Invalid API Key.")
|
||||
|
||||
# Update last_used_at
|
||||
api_key_instance.last_used_at = timezone.now()
|
||||
api_key_instance.save(update_fields=["last_used_at"])
|
||||
api_key_instance.save(update_fields=["last_used_at"], using=MainRouter.admin_db)
|
||||
|
||||
return entity, {
|
||||
"tenant_id": str(api_key_instance.tenant_id),
|
||||
|
||||
@@ -1,13 +1,15 @@
|
||||
from django.conf import settings
|
||||
from django.core.exceptions import ObjectDoesNotExist
|
||||
from django.db import transaction
|
||||
from rest_framework import permissions
|
||||
from rest_framework.exceptions import NotAuthenticated
|
||||
from rest_framework.filters import SearchFilter
|
||||
from rest_framework.permissions import SAFE_METHODS
|
||||
from rest_framework_json_api import filters
|
||||
from rest_framework_json_api.views import ModelViewSet
|
||||
|
||||
from api.authentication import CombinedJWTOrAPIKeyAuthentication
|
||||
from api.db_router import MainRouter
|
||||
from api.db_router import MainRouter, reset_read_db_alias, set_read_db_alias
|
||||
from api.db_utils import POSTGRES_USER_VAR, rls_transaction
|
||||
from api.filters import CustomDjangoFilterBackend
|
||||
from api.models import Role, Tenant
|
||||
@@ -31,6 +33,20 @@ class BaseViewSet(ModelViewSet):
|
||||
ordering_fields = "__all__"
|
||||
ordering = ["id"]
|
||||
|
||||
def _get_request_db_alias(self, request):
|
||||
if request is None:
|
||||
return MainRouter.default_db
|
||||
|
||||
read_alias = (
|
||||
MainRouter.replica_db
|
||||
if request.method in SAFE_METHODS
|
||||
and MainRouter.replica_db in settings.DATABASES
|
||||
else None
|
||||
)
|
||||
if read_alias:
|
||||
return read_alias
|
||||
return MainRouter.default_db
|
||||
|
||||
def initial(self, request, *args, **kwargs):
|
||||
"""
|
||||
Sets required_permissions before permissions are checked.
|
||||
@@ -48,8 +64,21 @@ class BaseViewSet(ModelViewSet):
|
||||
|
||||
class BaseRLSViewSet(BaseViewSet):
|
||||
def dispatch(self, request, *args, **kwargs):
|
||||
with transaction.atomic():
|
||||
return super().dispatch(request, *args, **kwargs)
|
||||
self.db_alias = self._get_request_db_alias(request)
|
||||
alias_token = None
|
||||
try:
|
||||
if self.db_alias != MainRouter.default_db:
|
||||
alias_token = set_read_db_alias(self.db_alias)
|
||||
|
||||
if request is not None:
|
||||
request.db_alias = self.db_alias
|
||||
|
||||
with transaction.atomic(using=self.db_alias):
|
||||
return super().dispatch(request, *args, **kwargs)
|
||||
finally:
|
||||
if alias_token is not None:
|
||||
reset_read_db_alias(alias_token)
|
||||
self.db_alias = MainRouter.default_db
|
||||
|
||||
def initial(self, request, *args, **kwargs):
|
||||
# Ideally, this logic would be in the `.setup()` method but DRF view sets don't call it
|
||||
@@ -61,7 +90,9 @@ class BaseRLSViewSet(BaseViewSet):
|
||||
if tenant_id is None:
|
||||
raise NotAuthenticated("Tenant ID is not present in token")
|
||||
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(
|
||||
tenant_id, using=getattr(self, "db_alias", MainRouter.default_db)
|
||||
):
|
||||
self.request.tenant_id = tenant_id
|
||||
return super().initial(request, *args, **kwargs)
|
||||
|
||||
@@ -73,18 +104,33 @@ class BaseRLSViewSet(BaseViewSet):
|
||||
|
||||
class BaseTenantViewset(BaseViewSet):
|
||||
def dispatch(self, request, *args, **kwargs):
|
||||
with transaction.atomic():
|
||||
tenant = super().dispatch(request, *args, **kwargs)
|
||||
|
||||
self.db_alias = self._get_request_db_alias(request)
|
||||
alias_token = None
|
||||
try:
|
||||
# If the request is a POST, create the admin role
|
||||
if request.method == "POST":
|
||||
isinstance(tenant, dict) and self._create_admin_role(tenant.data["id"])
|
||||
except Exception as e:
|
||||
self._handle_creation_error(e, tenant)
|
||||
raise
|
||||
if self.db_alias != MainRouter.default_db:
|
||||
alias_token = set_read_db_alias(self.db_alias)
|
||||
|
||||
return tenant
|
||||
if request is not None:
|
||||
request.db_alias = self.db_alias
|
||||
|
||||
with transaction.atomic(using=self.db_alias):
|
||||
tenant = super().dispatch(request, *args, **kwargs)
|
||||
|
||||
try:
|
||||
# If the request is a POST, create the admin role
|
||||
if request.method == "POST":
|
||||
isinstance(tenant, dict) and self._create_admin_role(
|
||||
tenant.data["id"]
|
||||
)
|
||||
except Exception as e:
|
||||
self._handle_creation_error(e, tenant)
|
||||
raise
|
||||
|
||||
return tenant
|
||||
finally:
|
||||
if alias_token is not None:
|
||||
reset_read_db_alias(alias_token)
|
||||
self.db_alias = MainRouter.default_db
|
||||
|
||||
def _create_admin_role(self, tenant_id):
|
||||
Role.objects.using(MainRouter.admin_db).create(
|
||||
@@ -117,14 +163,31 @@ class BaseTenantViewset(BaseViewSet):
|
||||
raise NotAuthenticated("Tenant ID is not present in token")
|
||||
|
||||
user_id = str(request.user.id)
|
||||
with rls_transaction(value=user_id, parameter=POSTGRES_USER_VAR):
|
||||
with rls_transaction(
|
||||
value=user_id,
|
||||
parameter=POSTGRES_USER_VAR,
|
||||
using=getattr(self, "db_alias", MainRouter.default_db),
|
||||
):
|
||||
return super().initial(request, *args, **kwargs)
|
||||
|
||||
|
||||
class BaseUserViewset(BaseViewSet):
|
||||
def dispatch(self, request, *args, **kwargs):
|
||||
with transaction.atomic():
|
||||
return super().dispatch(request, *args, **kwargs)
|
||||
self.db_alias = self._get_request_db_alias(request)
|
||||
alias_token = None
|
||||
try:
|
||||
if self.db_alias != MainRouter.default_db:
|
||||
alias_token = set_read_db_alias(self.db_alias)
|
||||
|
||||
if request is not None:
|
||||
request.db_alias = self.db_alias
|
||||
|
||||
with transaction.atomic(using=self.db_alias):
|
||||
return super().dispatch(request, *args, **kwargs)
|
||||
finally:
|
||||
if alias_token is not None:
|
||||
reset_read_db_alias(alias_token)
|
||||
self.db_alias = MainRouter.default_db
|
||||
|
||||
def initial(self, request, *args, **kwargs):
|
||||
# TODO refactor after improving RLS on users
|
||||
@@ -137,6 +200,8 @@ class BaseUserViewset(BaseViewSet):
|
||||
if tenant_id is None:
|
||||
raise NotAuthenticated("Tenant ID is not present in token")
|
||||
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(
|
||||
tenant_id, using=getattr(self, "db_alias", MainRouter.default_db)
|
||||
):
|
||||
self.request.tenant_id = tenant_id
|
||||
return super().initial(request, *args, **kwargs)
|
||||
|
||||
@@ -150,12 +150,16 @@ def generate_scan_compliance(
|
||||
requirement["checks"][check_id] = status
|
||||
requirement["checks_status"][status.lower()] += 1
|
||||
|
||||
if requirement["status"] != "FAIL" and any(
|
||||
value == "FAIL" for value in requirement["checks"].values()
|
||||
):
|
||||
requirement["status"] = "FAIL"
|
||||
compliance_overview[compliance_id]["requirements_status"]["passed"] -= 1
|
||||
compliance_overview[compliance_id]["requirements_status"]["failed"] += 1
|
||||
if requirement["status"] != "FAIL" and any(
|
||||
value == "FAIL" for value in requirement["checks"].values()
|
||||
):
|
||||
requirement["status"] = "FAIL"
|
||||
compliance_overview[compliance_id]["requirements_status"][
|
||||
"passed"
|
||||
] -= 1
|
||||
compliance_overview[compliance_id]["requirements_status"][
|
||||
"failed"
|
||||
] += 1
|
||||
|
||||
|
||||
def generate_compliance_overview_template(prowler_compliance: dict):
|
||||
|
||||
@@ -1,9 +1,31 @@
|
||||
from contextvars import ContextVar
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
ALLOWED_APPS = ("django", "socialaccount", "account", "authtoken", "silk")
|
||||
|
||||
_read_db_alias = ContextVar("read_db_alias", default=None)
|
||||
|
||||
|
||||
def set_read_db_alias(alias: str | None):
|
||||
if not alias:
|
||||
return None
|
||||
return _read_db_alias.set(alias)
|
||||
|
||||
|
||||
def get_read_db_alias() -> str | None:
|
||||
return _read_db_alias.get()
|
||||
|
||||
|
||||
def reset_read_db_alias(token) -> None:
|
||||
if token is not None:
|
||||
_read_db_alias.reset(token)
|
||||
|
||||
|
||||
class MainRouter:
|
||||
default_db = "default"
|
||||
admin_db = "admin"
|
||||
replica_db = "replica"
|
||||
|
||||
def db_for_read(self, model, **hints): # noqa: F841
|
||||
model_table_name = model._meta.db_table
|
||||
@@ -11,6 +33,9 @@ class MainRouter:
|
||||
model_table_name.startswith(f"{app}_") for app in ALLOWED_APPS
|
||||
):
|
||||
return self.admin_db
|
||||
read_alias = get_read_db_alias()
|
||||
if read_alias:
|
||||
return read_alias
|
||||
return None
|
||||
|
||||
def db_for_write(self, model, **hints): # noqa: F841
|
||||
@@ -27,3 +52,8 @@ class MainRouter:
|
||||
if {obj1._state.db, obj2._state.db} <= {self.default_db, self.admin_db}:
|
||||
return True
|
||||
return None
|
||||
|
||||
|
||||
READ_REPLICA_ALIAS = (
|
||||
MainRouter.replica_db if MainRouter.replica_db in settings.DATABASES else None
|
||||
)
|
||||
|
||||
@@ -6,12 +6,14 @@ from datetime import datetime, timedelta, timezone
|
||||
|
||||
from django.conf import settings
|
||||
from django.contrib.auth.models import BaseUserManager
|
||||
from django.db import connection, models, transaction
|
||||
from django.db import DEFAULT_DB_ALIAS, connection, connections, models, transaction
|
||||
from django_celery_beat.models import PeriodicTask
|
||||
from psycopg2 import connect as psycopg2_connect
|
||||
from psycopg2.extensions import AsIs, new_type, register_adapter, register_type
|
||||
from rest_framework_json_api.serializers import ValidationError
|
||||
|
||||
from api.db_router import get_read_db_alias, reset_read_db_alias, set_read_db_alias
|
||||
|
||||
DB_USER = settings.DATABASES["default"]["USER"] if not settings.TESTING else "test"
|
||||
DB_PASSWORD = (
|
||||
settings.DATABASES["default"]["PASSWORD"] if not settings.TESTING else "test"
|
||||
@@ -49,7 +51,11 @@ def psycopg_connection(database_alias: str):
|
||||
|
||||
|
||||
@contextmanager
|
||||
def rls_transaction(value: str, parameter: str = POSTGRES_TENANT_VAR):
|
||||
def rls_transaction(
|
||||
value: str,
|
||||
parameter: str = POSTGRES_TENANT_VAR,
|
||||
using: str | None = None,
|
||||
):
|
||||
"""
|
||||
Creates a new database transaction setting the given configuration value for Postgres RLS. It validates the
|
||||
if the value is a valid UUID.
|
||||
@@ -57,16 +63,32 @@ def rls_transaction(value: str, parameter: str = POSTGRES_TENANT_VAR):
|
||||
Args:
|
||||
value (str): Database configuration parameter value.
|
||||
parameter (str): Database configuration parameter name, by default is 'api.tenant_id'.
|
||||
using (str | None): Optional database alias to run the transaction against. Defaults to the
|
||||
active read alias (if any) or Django's default connection.
|
||||
"""
|
||||
with transaction.atomic():
|
||||
with connection.cursor() as cursor:
|
||||
try:
|
||||
# just in case the value is a UUID object
|
||||
uuid.UUID(str(value))
|
||||
except ValueError:
|
||||
raise ValidationError("Must be a valid UUID")
|
||||
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
|
||||
yield cursor
|
||||
requested_alias = using or get_read_db_alias()
|
||||
db_alias = requested_alias or DEFAULT_DB_ALIAS
|
||||
if db_alias not in connections:
|
||||
db_alias = DEFAULT_DB_ALIAS
|
||||
|
||||
router_token = None
|
||||
try:
|
||||
if db_alias != DEFAULT_DB_ALIAS:
|
||||
router_token = set_read_db_alias(db_alias)
|
||||
|
||||
with transaction.atomic(using=db_alias):
|
||||
conn = connections[db_alias]
|
||||
with conn.cursor() as cursor:
|
||||
try:
|
||||
# just in case the value is a UUID object
|
||||
uuid.UUID(str(value))
|
||||
except ValueError:
|
||||
raise ValidationError("Must be a valid UUID")
|
||||
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
|
||||
yield cursor
|
||||
finally:
|
||||
if router_token is not None:
|
||||
reset_read_db_alias(router_token)
|
||||
|
||||
|
||||
class CustomUserManager(BaseUserManager):
|
||||
|
||||
@@ -765,6 +765,7 @@ class ComplianceOverviewFilter(FilterSet):
|
||||
class ScanSummaryFilter(FilterSet):
|
||||
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
|
||||
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
|
||||
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
|
||||
provider_type = ChoiceFilter(
|
||||
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
|
||||
)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import uuid
|
||||
|
||||
import django.core.validators
|
||||
import django.db.models.deletion
|
||||
import drf_simple_apikey.models
|
||||
from django.conf import settings
|
||||
@@ -20,7 +21,13 @@ class Migration(migrations.Migration):
|
||||
migrations.CreateModel(
|
||||
name="TenantAPIKey",
|
||||
fields=[
|
||||
("name", models.CharField(blank=True, max_length=255, null=True)),
|
||||
(
|
||||
"name",
|
||||
models.CharField(
|
||||
max_length=255,
|
||||
validators=[django.core.validators.MinLengthValidator(3)],
|
||||
),
|
||||
),
|
||||
(
|
||||
"expiry_date",
|
||||
models.DateTimeField(
|
||||
@@ -37,7 +44,7 @@ class Migration(migrations.Migration):
|
||||
help_text="If the API key is revoked, entities cannot use it anymore. (This cannot be undone.)",
|
||||
),
|
||||
),
|
||||
("created", models.DateTimeField(auto_now=True)),
|
||||
("created", models.DateTimeField(auto_now_add=True, editable=False)),
|
||||
(
|
||||
"whitelisted_ips",
|
||||
models.JSONField(
|
||||
@@ -110,7 +117,11 @@ class Migration(migrations.Migration):
|
||||
"constraints": [
|
||||
models.UniqueConstraint(
|
||||
fields=("tenant_id", "prefix"), name="unique_api_key_prefixes"
|
||||
)
|
||||
),
|
||||
models.UniqueConstraint(
|
||||
fields=("tenant_id", "name"),
|
||||
name="unique_api_key_name_per_tenant",
|
||||
),
|
||||
],
|
||||
},
|
||||
),
|
||||
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
# Generated by Django 5.1.12 on 2025-10-07 10:41
|
||||
|
||||
from django.db import migrations, models
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
dependencies = [
|
||||
("api", "0048_api_key"),
|
||||
]
|
||||
operations = [
|
||||
migrations.AddField(
|
||||
model_name="compliancerequirementoverview",
|
||||
name="passed_findings",
|
||||
field=models.IntegerField(default=0),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="compliancerequirementoverview",
|
||||
name="total_findings",
|
||||
field=models.IntegerField(default=0),
|
||||
),
|
||||
]
|
||||
@@ -220,6 +220,8 @@ class Membership(models.Model):
|
||||
|
||||
class TenantAPIKey(AbstractAPIKey, RowLevelSecurityProtectedModel):
|
||||
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
|
||||
name = models.CharField(max_length=100, validators=[MinLengthValidator(3)])
|
||||
created = models.DateTimeField(auto_now_add=True, editable=False)
|
||||
prefix = models.CharField(
|
||||
max_length=11,
|
||||
unique=True,
|
||||
@@ -255,6 +257,10 @@ class TenantAPIKey(AbstractAPIKey, RowLevelSecurityProtectedModel):
|
||||
fields=("tenant_id", "prefix"),
|
||||
name="unique_api_key_prefixes",
|
||||
),
|
||||
models.UniqueConstraint(
|
||||
fields=("tenant_id", "name"),
|
||||
name="unique_api_key_name_per_tenant",
|
||||
),
|
||||
]
|
||||
|
||||
indexes = [
|
||||
@@ -1293,6 +1299,8 @@ class ComplianceRequirementOverview(RowLevelSecurityProtectedModel):
|
||||
passed_checks = models.IntegerField(default=0)
|
||||
failed_checks = models.IntegerField(default=0)
|
||||
total_checks = models.IntegerField(default=0)
|
||||
passed_findings = models.IntegerField(default=0)
|
||||
total_findings = models.IntegerField(default=0)
|
||||
|
||||
scan = models.ForeignKey(
|
||||
Scan,
|
||||
|
||||
@@ -11,11 +11,12 @@ class APIJSONRenderer(JSONRenderer):
|
||||
def render(self, data, accepted_media_type=None, renderer_context=None):
|
||||
request = renderer_context.get("request")
|
||||
tenant_id = getattr(request, "tenant_id", None) if request else None
|
||||
db_alias = getattr(request, "db_alias", None) if request else None
|
||||
include_param_present = "include" in request.query_params if request else False
|
||||
|
||||
# Use rls_transaction if needed for included resources, otherwise do nothing
|
||||
context_manager = (
|
||||
rls_transaction(tenant_id)
|
||||
rls_transaction(tenant_id, using=db_alias)
|
||||
if tenant_id and include_param_present
|
||||
else nullcontext()
|
||||
)
|
||||
|
||||
@@ -86,6 +86,17 @@ paths:
|
||||
description: A search term.
|
||||
schema:
|
||||
type: string
|
||||
- in: query
|
||||
name: include
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enum:
|
||||
- entity
|
||||
description: include query parameter to allow the client to customize which
|
||||
related resources should be returned.
|
||||
explode: false
|
||||
- name: page[number]
|
||||
required: false
|
||||
in: query
|
||||
@@ -186,6 +197,17 @@ paths:
|
||||
format: uuid
|
||||
description: A UUID string identifying this tenant api key.
|
||||
required: true
|
||||
- in: query
|
||||
name: include
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
enum:
|
||||
- entity
|
||||
description: include query parameter to allow the client to customize which
|
||||
related resources should be returned.
|
||||
explode: false
|
||||
tags:
|
||||
- API Keys
|
||||
security:
|
||||
@@ -3589,6 +3611,16 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- in: query
|
||||
name: filter[provider_id__in]
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
format: uuid
|
||||
description: Multiple values may be separated by commas.
|
||||
explode: false
|
||||
style: form
|
||||
- in: query
|
||||
name: filter[provider_type]
|
||||
schema:
|
||||
@@ -3756,6 +3788,16 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- in: query
|
||||
name: filter[provider_id__in]
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
format: uuid
|
||||
description: Multiple values may be separated by commas.
|
||||
explode: false
|
||||
style: form
|
||||
- in: query
|
||||
name: filter[provider_type]
|
||||
schema:
|
||||
@@ -3958,6 +4000,16 @@ paths:
|
||||
schema:
|
||||
type: string
|
||||
format: uuid
|
||||
- in: query
|
||||
name: filter[provider_id__in]
|
||||
schema:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
format: uuid
|
||||
description: Multiple values may be separated by commas.
|
||||
explode: false
|
||||
style: form
|
||||
- in: query
|
||||
name: filter[provider_type]
|
||||
schema:
|
||||
@@ -4706,6 +4758,7 @@ paths:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
x-spec-enum-id: 4c1e219dad1cc0e7
|
||||
enum:
|
||||
- aws
|
||||
- azure
|
||||
@@ -11987,26 +12040,47 @@ components:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory
|
||||
where the application is registered.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
user:
|
||||
type: email
|
||||
description: User microsoft email address.
|
||||
deprecated: true
|
||||
password:
|
||||
type: string
|
||||
description: User password.
|
||||
deprecated: true
|
||||
required:
|
||||
- client_id
|
||||
- client_secret
|
||||
- tenant_id
|
||||
- user
|
||||
- password
|
||||
- type: object
|
||||
title: M365 Certificate Credentials
|
||||
properties:
|
||||
client_id:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory
|
||||
where the application is registered.
|
||||
certificate_content:
|
||||
type: string
|
||||
description: The certificate content in base64 format for
|
||||
certificate-based authentication.
|
||||
required:
|
||||
- client_id
|
||||
- tenant_id
|
||||
- certificate_content
|
||||
- type: object
|
||||
title: GCP Static Credentials
|
||||
properties:
|
||||
@@ -12410,8 +12484,8 @@ components:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
nullable: true
|
||||
maxLength: 255
|
||||
minLength: 3
|
||||
maxLength: 100
|
||||
prefix:
|
||||
type: string
|
||||
readOnly: true
|
||||
@@ -12431,6 +12505,8 @@ components:
|
||||
readOnly: true
|
||||
nullable: true
|
||||
description: Last time this API key was used for authentication
|
||||
required:
|
||||
- name
|
||||
relationships:
|
||||
type: object
|
||||
properties:
|
||||
@@ -13851,26 +13927,47 @@ components:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory where
|
||||
the application is registered.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
user:
|
||||
type: email
|
||||
description: User microsoft email address.
|
||||
deprecated: true
|
||||
password:
|
||||
type: string
|
||||
description: User password.
|
||||
deprecated: true
|
||||
required:
|
||||
- client_id
|
||||
- client_secret
|
||||
- tenant_id
|
||||
- user
|
||||
- password
|
||||
- type: object
|
||||
title: M365 Certificate Credentials
|
||||
properties:
|
||||
client_id:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory where
|
||||
the application is registered.
|
||||
certificate_content:
|
||||
type: string
|
||||
description: The certificate content in base64 format for certificate-based
|
||||
authentication.
|
||||
required:
|
||||
- client_id
|
||||
- tenant_id
|
||||
- certificate_content
|
||||
- type: object
|
||||
title: GCP Static Credentials
|
||||
properties:
|
||||
@@ -14099,26 +14196,47 @@ components:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory
|
||||
where the application is registered.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
user:
|
||||
type: email
|
||||
description: User microsoft email address.
|
||||
deprecated: true
|
||||
password:
|
||||
type: string
|
||||
description: User password.
|
||||
deprecated: true
|
||||
required:
|
||||
- client_id
|
||||
- client_secret
|
||||
- tenant_id
|
||||
- user
|
||||
- password
|
||||
- type: object
|
||||
title: M365 Certificate Credentials
|
||||
properties:
|
||||
client_id:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory
|
||||
where the application is registered.
|
||||
certificate_content:
|
||||
type: string
|
||||
description: The certificate content in base64 format for
|
||||
certificate-based authentication.
|
||||
required:
|
||||
- client_id
|
||||
- tenant_id
|
||||
- certificate_content
|
||||
- type: object
|
||||
title: GCP Static Credentials
|
||||
properties:
|
||||
@@ -14363,26 +14481,47 @@ components:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory where
|
||||
the application is registered.
|
||||
client_secret:
|
||||
type: string
|
||||
description: The client secret associated with the application
|
||||
(client) ID, providing secure access.
|
||||
user:
|
||||
type: email
|
||||
description: User microsoft email address.
|
||||
deprecated: true
|
||||
password:
|
||||
type: string
|
||||
description: User password.
|
||||
deprecated: true
|
||||
required:
|
||||
- client_id
|
||||
- client_secret
|
||||
- tenant_id
|
||||
- user
|
||||
- password
|
||||
- type: object
|
||||
title: M365 Certificate Credentials
|
||||
properties:
|
||||
client_id:
|
||||
type: string
|
||||
description: The Azure application (client) ID for authentication
|
||||
in Azure AD.
|
||||
tenant_id:
|
||||
type: string
|
||||
description: The Azure tenant ID, representing the directory where
|
||||
the application is registered.
|
||||
certificate_content:
|
||||
type: string
|
||||
description: The certificate content in base64 format for certificate-based
|
||||
authentication.
|
||||
required:
|
||||
- client_id
|
||||
- tenant_id
|
||||
- certificate_content
|
||||
- type: object
|
||||
title: GCP Static Credentials
|
||||
properties:
|
||||
@@ -15636,8 +15775,8 @@ components:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
nullable: true
|
||||
maxLength: 255
|
||||
maxLength: 100
|
||||
minLength: 3
|
||||
prefix:
|
||||
type: string
|
||||
readOnly: true
|
||||
@@ -15659,6 +15798,8 @@ components:
|
||||
format: date-time
|
||||
nullable: true
|
||||
description: Last time this API key was used for authentication
|
||||
required:
|
||||
- name
|
||||
relationships:
|
||||
type: object
|
||||
properties:
|
||||
@@ -15705,8 +15846,8 @@ components:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
nullable: true
|
||||
maxLength: 255
|
||||
maxLength: 100
|
||||
minLength: 3
|
||||
prefix:
|
||||
type: string
|
||||
readOnly: true
|
||||
@@ -15732,6 +15873,8 @@ components:
|
||||
api_key:
|
||||
type: string
|
||||
readOnly: true
|
||||
required:
|
||||
- name
|
||||
relationships:
|
||||
type: object
|
||||
properties:
|
||||
@@ -15782,8 +15925,8 @@ components:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
nullable: true
|
||||
maxLength: 255
|
||||
minLength: 3
|
||||
maxLength: 100
|
||||
prefix:
|
||||
type: string
|
||||
readOnly: true
|
||||
@@ -15810,6 +15953,8 @@ components:
|
||||
api_key:
|
||||
type: string
|
||||
readOnly: true
|
||||
required:
|
||||
- name
|
||||
relationships:
|
||||
type: object
|
||||
properties:
|
||||
@@ -15877,8 +16022,8 @@ components:
|
||||
properties:
|
||||
name:
|
||||
type: string
|
||||
nullable: true
|
||||
maxLength: 255
|
||||
maxLength: 100
|
||||
minLength: 3
|
||||
prefix:
|
||||
type: string
|
||||
readOnly: true
|
||||
@@ -15897,6 +16042,8 @@ components:
|
||||
readOnly: true
|
||||
nullable: true
|
||||
description: Last time this API key was used for authentication
|
||||
required:
|
||||
- name
|
||||
relationships:
|
||||
type: object
|
||||
properties:
|
||||
|
||||
@@ -1003,6 +1003,504 @@ class TestCombinedAuthentication:
|
||||
assert api_key.last_used_at is not None
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestAPIKeyRLSBypass:
|
||||
"""Test RLS bypass fix for API key authentication.
|
||||
|
||||
These tests verify that API key authentication works correctly even when
|
||||
RLS context is not set, which is critical since we don't know the tenant_id
|
||||
until we look up the API key (which itself is protected by RLS).
|
||||
|
||||
The fix ensures all database operations during authentication use the admin
|
||||
database, bypassing RLS constraints.
|
||||
"""
|
||||
|
||||
def test_api_key_authentication_without_rls_context(
|
||||
self, create_test_user, tenants_fixture, api_keys_fixture
|
||||
):
|
||||
"""Verify API key authentication works without pre-existing RLS context.
|
||||
|
||||
This is the core fix: authentication must succeed even when prowler.tenant_id
|
||||
is not set, since we need to look up the API key to discover the tenant.
|
||||
"""
|
||||
client = APIClient()
|
||||
api_key = api_keys_fixture[0]
|
||||
|
||||
api_key_headers = get_api_key_header(api_key._raw_key)
|
||||
response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert response.status_code == 200
|
||||
assert "data" in response.json()
|
||||
|
||||
def test_api_key_lookup_uses_admin_database(
|
||||
self, create_test_user, tenants_fixture
|
||||
):
|
||||
"""Verify API key lookup uses admin database during authentication.
|
||||
|
||||
The TenantAPIKey model is RLS-protected, so queries against it would
|
||||
normally fail without prowler.tenant_id set. The fix routes lookups
|
||||
to the admin database which bypasses RLS.
|
||||
"""
|
||||
client = APIClient()
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
role = Role.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
name="Admin DB Test Role",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=create_test_user,
|
||||
role=role,
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
|
||||
api_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Admin DB Test Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=create_test_user,
|
||||
)
|
||||
|
||||
api_key_headers = get_api_key_header(raw_key)
|
||||
response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert response.status_code == 200
|
||||
|
||||
api_key.refresh_from_db()
|
||||
assert api_key.last_used_at is not None
|
||||
|
||||
def test_tenant_context_established_after_authentication(
|
||||
self, create_test_user, tenants_fixture, api_keys_fixture
|
||||
):
|
||||
"""Verify correct tenant context is established after API key auth.
|
||||
|
||||
After authentication, the tenant_id from the API key should be used
|
||||
to set up the proper RLS context for subsequent queries.
|
||||
"""
|
||||
client = APIClient()
|
||||
api_key = api_keys_fixture[0]
|
||||
|
||||
api_key_headers = get_api_key_header(api_key._raw_key)
|
||||
|
||||
# Use tenant-list endpoint to get actual tenant IDs
|
||||
tenant_response = client.get(reverse("tenant-list"), headers=api_key_headers)
|
||||
|
||||
assert tenant_response.status_code == 200
|
||||
tenant_data = tenant_response.json()["data"]
|
||||
tenant_ids = [t["id"] for t in tenant_data]
|
||||
|
||||
# Verify the API key's tenant is in the list of accessible tenants
|
||||
assert str(api_key.tenant_id) in tenant_ids
|
||||
|
||||
def test_concurrent_authentication_different_tenants(self, tenants_fixture):
|
||||
"""Verify multiple API keys from different tenants can authenticate simultaneously.
|
||||
|
||||
This tests that the admin database routing works correctly in concurrent
|
||||
scenarios and doesn't cause tenant isolation issues.
|
||||
"""
|
||||
client = APIClient()
|
||||
|
||||
user1 = User.objects.create_user(
|
||||
name="concurrent_user1",
|
||||
email="concurrent1@test.com",
|
||||
password=TEST_PASSWORD,
|
||||
)
|
||||
user2 = User.objects.create_user(
|
||||
name="concurrent_user2",
|
||||
email="concurrent2@test.com",
|
||||
password=TEST_PASSWORD,
|
||||
)
|
||||
|
||||
tenant1 = tenants_fixture[0]
|
||||
tenant2 = tenants_fixture[1]
|
||||
|
||||
Membership.objects.create(user=user1, tenant=tenant1)
|
||||
Membership.objects.create(user=user2, tenant=tenant2)
|
||||
|
||||
role1 = Role.objects.create(
|
||||
tenant_id=tenant1.id,
|
||||
name="Concurrent Role 1",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
role2 = Role.objects.create(
|
||||
tenant_id=tenant2.id,
|
||||
name="Concurrent Role 2",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user1,
|
||||
role=role1,
|
||||
tenant_id=tenant1.id,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user2,
|
||||
role=role2,
|
||||
tenant_id=tenant2.id,
|
||||
)
|
||||
|
||||
api_key1, raw_key1 = TenantAPIKey.objects.create_api_key(
|
||||
name="Concurrent Key 1",
|
||||
tenant_id=tenant1.id,
|
||||
entity=user1,
|
||||
)
|
||||
api_key2, raw_key2 = TenantAPIKey.objects.create_api_key(
|
||||
name="Concurrent Key 2",
|
||||
tenant_id=tenant2.id,
|
||||
entity=user2,
|
||||
)
|
||||
|
||||
headers1 = get_api_key_header(raw_key1)
|
||||
headers2 = get_api_key_header(raw_key2)
|
||||
|
||||
response1 = client.get(reverse("provider-list"), headers=headers1)
|
||||
response2 = client.get(reverse("provider-list"), headers=headers2)
|
||||
|
||||
assert response1.status_code == 200
|
||||
assert response2.status_code == 200
|
||||
|
||||
api_key1.refresh_from_db()
|
||||
api_key2.refresh_from_db()
|
||||
|
||||
assert api_key1.last_used_at is not None
|
||||
assert api_key2.last_used_at is not None
|
||||
assert api_key1.tenant_id == tenant1.id
|
||||
assert api_key2.tenant_id == tenant2.id
|
||||
|
||||
def test_api_key_update_last_used_uses_admin_db(
|
||||
self, create_test_user, tenants_fixture, api_keys_fixture
|
||||
):
|
||||
"""Verify last_used_at update uses admin database.
|
||||
|
||||
The update to last_used_at during authentication must also use the
|
||||
admin database since it occurs before RLS context is established.
|
||||
"""
|
||||
client = APIClient()
|
||||
api_key = api_keys_fixture[0]
|
||||
|
||||
assert api_key.last_used_at is None
|
||||
|
||||
api_key_headers = get_api_key_header(api_key._raw_key)
|
||||
first_response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert first_response.status_code == 200
|
||||
|
||||
api_key.refresh_from_db()
|
||||
first_timestamp = api_key.last_used_at
|
||||
assert first_timestamp is not None
|
||||
|
||||
time.sleep(0.1)
|
||||
|
||||
second_response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
assert second_response.status_code == 200
|
||||
|
||||
api_key.refresh_from_db()
|
||||
second_timestamp = api_key.last_used_at
|
||||
assert second_timestamp > first_timestamp
|
||||
|
||||
def test_api_key_prefix_lookup_bypasses_rls(
|
||||
self, create_test_user, tenants_fixture
|
||||
):
|
||||
"""Verify prefix-based API key lookup works without RLS context.
|
||||
|
||||
The authentication process splits the key into prefix and encrypted parts,
|
||||
then looks up by prefix. This lookup must work via admin database.
|
||||
"""
|
||||
client = APIClient()
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
role = Role.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
name="Prefix Test Role",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=create_test_user,
|
||||
role=role,
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
|
||||
api_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Prefix Test Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=create_test_user,
|
||||
)
|
||||
|
||||
prefix = raw_key.split(".")[0]
|
||||
assert prefix == api_key.prefix
|
||||
|
||||
api_key_headers = get_api_key_header(raw_key)
|
||||
response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert response.status_code == 200
|
||||
|
||||
def test_expired_api_key_check_uses_admin_db(
|
||||
self, create_test_user, tenants_fixture
|
||||
):
|
||||
"""Verify expired API key validation works via admin database.
|
||||
|
||||
Checking if a key is expired requires reading from TenantAPIKey,
|
||||
which must use admin database during authentication.
|
||||
"""
|
||||
client = APIClient()
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
expired_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Expired Test Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=create_test_user,
|
||||
expiry_date=datetime.now(timezone.utc) - timedelta(days=1),
|
||||
)
|
||||
|
||||
api_key_headers = get_api_key_header(raw_key)
|
||||
response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert response.status_code == 401
|
||||
assert "expired" in response.json()["errors"][0]["detail"].lower()
|
||||
|
||||
def test_revoked_api_key_check_uses_admin_db(
|
||||
self, create_test_user, tenants_fixture
|
||||
):
|
||||
"""Verify revoked API key validation works via admin database.
|
||||
|
||||
Checking if a key is revoked requires reading from TenantAPIKey,
|
||||
which must use admin database during authentication.
|
||||
"""
|
||||
client = APIClient()
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
role = Role.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
name="Revoked Test Role",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=create_test_user,
|
||||
role=role,
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
|
||||
api_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Revoked Test Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=create_test_user,
|
||||
)
|
||||
|
||||
api_key.revoked = True
|
||||
api_key.save()
|
||||
|
||||
api_key_headers = get_api_key_header(raw_key)
|
||||
response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
|
||||
assert response.status_code == 401
|
||||
assert "revoked" in response.json()["errors"][0]["detail"].lower()
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestAPIKeyMultiTenantWorkflows:
|
||||
"""Test complete multi-tenant workflows using API keys.
|
||||
|
||||
These integration tests verify end-to-end scenarios where API keys
|
||||
are used across different tenants and ensure proper isolation.
|
||||
"""
|
||||
|
||||
def test_user_with_multiple_tenant_memberships_api_keys(self, tenants_fixture):
|
||||
"""User with memberships in multiple tenants can use different API keys.
|
||||
|
||||
Tests that a user can have separate API keys for different tenants
|
||||
and each key only accesses resources in its tenant.
|
||||
"""
|
||||
client = APIClient()
|
||||
|
||||
user = User.objects.create_user(
|
||||
name="multi_tenant_user",
|
||||
email="multitenant@test.com",
|
||||
password=TEST_PASSWORD,
|
||||
)
|
||||
|
||||
tenant1 = tenants_fixture[0]
|
||||
tenant2 = tenants_fixture[1]
|
||||
|
||||
Membership.objects.create(user=user, tenant=tenant1)
|
||||
Membership.objects.create(user=user, tenant=tenant2)
|
||||
|
||||
role1 = Role.objects.create(
|
||||
tenant_id=tenant1.id,
|
||||
name="Multi Tenant Role 1",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
role2 = Role.objects.create(
|
||||
tenant_id=tenant2.id,
|
||||
name="Multi Tenant Role 2",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user,
|
||||
role=role1,
|
||||
tenant_id=tenant1.id,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user,
|
||||
role=role2,
|
||||
tenant_id=tenant2.id,
|
||||
)
|
||||
|
||||
key1, raw_key1 = TenantAPIKey.objects.create_api_key(
|
||||
name="Tenant 1 Key",
|
||||
tenant_id=tenant1.id,
|
||||
entity=user,
|
||||
)
|
||||
key2, raw_key2 = TenantAPIKey.objects.create_api_key(
|
||||
name="Tenant 2 Key",
|
||||
tenant_id=tenant2.id,
|
||||
entity=user,
|
||||
)
|
||||
|
||||
headers1 = get_api_key_header(raw_key1)
|
||||
headers2 = get_api_key_header(raw_key2)
|
||||
|
||||
response1 = client.get(reverse("provider-list"), headers=headers1)
|
||||
response2 = client.get(reverse("provider-list"), headers=headers2)
|
||||
|
||||
assert response1.status_code == 200
|
||||
assert response2.status_code == 200
|
||||
|
||||
me_response1 = client.get(reverse("user-me"), headers=headers1)
|
||||
me_response2 = client.get(reverse("user-me"), headers=headers2)
|
||||
|
||||
assert me_response1.status_code == 200
|
||||
assert me_response2.status_code == 200
|
||||
|
||||
assert me_response1.json()["data"]["id"] == str(user.id)
|
||||
assert me_response2.json()["data"]["id"] == str(user.id)
|
||||
|
||||
def test_api_key_cannot_access_different_tenant_resources(
|
||||
self, tenants_fixture, providers_fixture
|
||||
):
|
||||
"""API key from one tenant cannot access resources from another tenant.
|
||||
|
||||
Verifies RLS enforcement after authentication ensures tenant isolation.
|
||||
"""
|
||||
client = APIClient()
|
||||
|
||||
user1 = User.objects.create_user(
|
||||
name="tenant1_user",
|
||||
email="tenant1user@test.com",
|
||||
password=TEST_PASSWORD,
|
||||
)
|
||||
user2 = User.objects.create_user(
|
||||
name="tenant2_user",
|
||||
email="tenant2user@test.com",
|
||||
password=TEST_PASSWORD,
|
||||
)
|
||||
|
||||
tenant1 = tenants_fixture[0]
|
||||
tenant2 = tenants_fixture[1]
|
||||
|
||||
Membership.objects.create(user=user1, tenant=tenant1)
|
||||
Membership.objects.create(user=user2, tenant=tenant2)
|
||||
|
||||
role1 = Role.objects.create(
|
||||
tenant_id=tenant1.id,
|
||||
name="Isolation Test Role 1",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
role2 = Role.objects.create(
|
||||
tenant_id=tenant2.id,
|
||||
name="Isolation Test Role 2",
|
||||
unlimited_visibility=True,
|
||||
manage_account=True,
|
||||
)
|
||||
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user1,
|
||||
role=role1,
|
||||
tenant_id=tenant1.id,
|
||||
)
|
||||
UserRoleRelationship.objects.create(
|
||||
user=user2,
|
||||
role=role2,
|
||||
tenant_id=tenant2.id,
|
||||
)
|
||||
|
||||
key1, raw_key1 = TenantAPIKey.objects.create_api_key(
|
||||
name="Isolation Key 1",
|
||||
tenant_id=tenant1.id,
|
||||
entity=user1,
|
||||
)
|
||||
|
||||
headers1 = get_api_key_header(raw_key1)
|
||||
|
||||
provider_response = client.get(reverse("provider-list"), headers=headers1)
|
||||
assert provider_response.status_code == 200
|
||||
|
||||
providers_data = provider_response.json()["data"]
|
||||
|
||||
if providers_data:
|
||||
for provider in providers_data:
|
||||
provider_tenant_id = str(tenants_fixture[0].id)
|
||||
assert str(tenant2.id) != provider_tenant_id
|
||||
|
||||
def test_api_key_workflow_create_authenticate_revoke(
|
||||
self, create_test_user_rbac, tenants_fixture
|
||||
):
|
||||
"""Complete workflow: create API key via JWT, use it, then revoke via JWT.
|
||||
|
||||
Tests the full lifecycle using both JWT and API key authentication.
|
||||
"""
|
||||
client = APIClient()
|
||||
tenants_fixture[0]
|
||||
|
||||
jwt_access_token, _ = get_api_tokens(
|
||||
client, create_test_user_rbac.email, TEST_PASSWORD
|
||||
)
|
||||
jwt_headers = get_authorization_header(jwt_access_token)
|
||||
|
||||
create_response = client.post(
|
||||
reverse("api-key-list"),
|
||||
data={
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"attributes": {
|
||||
"name": "Workflow Test Key",
|
||||
},
|
||||
}
|
||||
},
|
||||
format="vnd.api+json",
|
||||
headers=jwt_headers,
|
||||
)
|
||||
|
||||
assert create_response.status_code == 201
|
||||
api_key_data = create_response.json()["data"]
|
||||
api_key_id = api_key_data["id"]
|
||||
raw_api_key = api_key_data["attributes"]["api_key"]
|
||||
|
||||
api_key_headers = get_api_key_header(raw_api_key)
|
||||
auth_response = client.get(reverse("provider-list"), headers=api_key_headers)
|
||||
assert auth_response.status_code == 200
|
||||
|
||||
revoke_response = client.delete(
|
||||
reverse("api-key-revoke", kwargs={"pk": api_key_id}),
|
||||
headers=jwt_headers,
|
||||
)
|
||||
assert revoke_response.status_code == 200
|
||||
|
||||
revoked_auth_response = client.get(
|
||||
reverse("provider-list"), headers=api_key_headers
|
||||
)
|
||||
assert revoked_auth_response.status_code == 401
|
||||
assert "revoked" in revoked_auth_response.json()["errors"][0]["detail"].lower()
|
||||
|
||||
|
||||
def get_api_key_header(api_key: str) -> dict:
|
||||
"""Helper to create API key authorization header."""
|
||||
return {"Authorization": f"Api-Key {api_key}"}
|
||||
|
||||
@@ -0,0 +1,384 @@
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from unittest.mock import patch
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from django.test import RequestFactory
|
||||
from rest_framework.exceptions import AuthenticationFailed
|
||||
|
||||
from api.authentication import TenantAPIKeyAuthentication
|
||||
from api.db_router import MainRouter
|
||||
from api.models import TenantAPIKey
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestTenantAPIKeyAuthentication:
|
||||
@pytest.fixture
|
||||
def auth_backend(self):
|
||||
"""Create an instance of TenantAPIKeyAuthentication."""
|
||||
return TenantAPIKeyAuthentication()
|
||||
|
||||
@pytest.fixture
|
||||
def request_factory(self):
|
||||
"""Create a Django request factory."""
|
||||
return RequestFactory()
|
||||
|
||||
def test_authenticate_credentials_uses_admin_database(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that _authenticate_credentials routes queries to admin database."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
# Extract the encrypted key part (after the prefix and separator)
|
||||
_, encrypted_key = raw_key.split(TenantAPIKey.objects.separator, 1)
|
||||
|
||||
# Create a mock request
|
||||
request = request_factory.get("/")
|
||||
|
||||
# Call the method
|
||||
entity, auth_dict = auth_backend._authenticate_credentials(
|
||||
request, encrypted_key
|
||||
)
|
||||
|
||||
# Verify that the entity is the user associated with the API key
|
||||
assert entity == api_key.entity
|
||||
assert entity.id == api_key.entity.id
|
||||
|
||||
def test_authenticate_credentials_restores_manager_on_success(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that the manager is restored after successful authentication."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
_, encrypted_key = raw_key.split(TenantAPIKey.objects.separator, 1)
|
||||
|
||||
# Store the original manager
|
||||
original_manager = TenantAPIKey.objects
|
||||
|
||||
request = request_factory.get("/")
|
||||
|
||||
# Call the method
|
||||
auth_backend._authenticate_credentials(request, encrypted_key)
|
||||
|
||||
# Verify the manager was restored
|
||||
assert TenantAPIKey.objects == original_manager
|
||||
|
||||
def test_authenticate_credentials_restores_manager_on_exception(
|
||||
self, auth_backend, request_factory
|
||||
):
|
||||
"""Test that the manager is restored even when an exception occurs."""
|
||||
# Store the original manager
|
||||
original_manager = TenantAPIKey.objects
|
||||
|
||||
request = request_factory.get("/")
|
||||
|
||||
# Try to authenticate with an invalid key that will raise an exception
|
||||
with pytest.raises(Exception):
|
||||
auth_backend._authenticate_credentials(request, "invalid_encrypted_key")
|
||||
|
||||
# Verify the manager was restored despite the exception
|
||||
assert TenantAPIKey.objects == original_manager
|
||||
|
||||
def test_authenticate_valid_api_key(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test successful authentication with a valid API key."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
# Create a request with the API key in the Authorization header
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# Authenticate
|
||||
entity, auth_dict = auth_backend.authenticate(request)
|
||||
|
||||
# Verify the entity and auth dict
|
||||
assert entity == api_key.entity
|
||||
assert auth_dict["tenant_id"] == str(api_key.tenant_id)
|
||||
assert auth_dict["sub"] == str(api_key.entity.id)
|
||||
assert auth_dict["api_key_prefix"] == api_key.prefix
|
||||
|
||||
# Verify that last_used_at was updated
|
||||
api_key.refresh_from_db()
|
||||
assert api_key.last_used_at is not None
|
||||
assert (datetime.now(timezone.utc) - api_key.last_used_at).seconds < 5
|
||||
|
||||
def test_authenticate_valid_api_key_uses_admin_database(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that authenticate uses admin database for API key lookup."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# Mock the manager's using method to verify it's called with admin_db
|
||||
with patch.object(
|
||||
TenantAPIKey.objects, "using", wraps=TenantAPIKey.objects.using
|
||||
) as mock_using:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
# Verify that .using('admin') was called
|
||||
mock_using.assert_called_with(MainRouter.admin_db)
|
||||
|
||||
def test_authenticate_invalid_key_format_missing_separator(
|
||||
self, auth_backend, request_factory
|
||||
):
|
||||
"""Test authentication fails with invalid API key format (no separator)."""
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = "Api-Key invalid_key_no_separator"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "Invalid API Key."
|
||||
|
||||
def test_authenticate_invalid_key_format_empty_prefix(
|
||||
self, auth_backend, request_factory
|
||||
):
|
||||
"""Test authentication fails with empty prefix."""
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = "Api-Key .encrypted_part"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "Invalid API Key."
|
||||
|
||||
def test_authenticate_invalid_encrypted_key(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test authentication fails with invalid encrypted key."""
|
||||
api_key = api_keys_fixture[0]
|
||||
|
||||
# Create an invalid key with valid prefix but invalid encryption
|
||||
invalid_key = f"{api_key.prefix}.invalid_encrypted_data"
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {invalid_key}"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "Invalid API Key."
|
||||
|
||||
def test_authenticate_revoked_api_key(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test authentication fails with a revoked API key."""
|
||||
# Use the revoked API key (index 2 from fixture)
|
||||
api_key = api_keys_fixture[2]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# The revoked key should fail during credential validation
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "This API Key has been revoked."
|
||||
|
||||
def test_authenticate_expired_api_key(
|
||||
self, auth_backend, create_test_user, tenants_fixture, request_factory
|
||||
):
|
||||
"""Test authentication fails with an expired API key."""
|
||||
tenant = tenants_fixture[0]
|
||||
user = create_test_user
|
||||
|
||||
# Create an expired API key
|
||||
api_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Expired API Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=user,
|
||||
expiry_date=datetime.now(timezone.utc) - timedelta(days=1),
|
||||
)
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "API Key has already expired."
|
||||
|
||||
def test_authenticate_nonexistent_api_key(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test authentication fails when API key doesn't exist in database."""
|
||||
# Create a valid-looking encrypted key with a non-existent UUID
|
||||
api_key = api_keys_fixture[0]
|
||||
non_existent_uuid = str(uuid4())
|
||||
|
||||
# Manually create an encrypted key with a non-existent ID
|
||||
payload = {
|
||||
"_pk": non_existent_uuid,
|
||||
"_exp": (datetime.now(timezone.utc) + timedelta(days=30)).timestamp(),
|
||||
}
|
||||
encrypted_key = auth_backend.key_crypto.generate(payload)
|
||||
fake_key = f"{api_key.prefix}.{encrypted_key}"
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {fake_key}"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "No entity matching this api key."
|
||||
|
||||
def test_authenticate_updates_last_used_at(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that last_used_at is updated on successful authentication."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
# Store the original last_used_at
|
||||
original_last_used = api_key.last_used_at
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# Authenticate
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
# Refresh from database
|
||||
api_key.refresh_from_db()
|
||||
|
||||
# Verify last_used_at was updated
|
||||
assert api_key.last_used_at is not None
|
||||
if original_last_used:
|
||||
assert api_key.last_used_at > original_last_used
|
||||
|
||||
def test_authenticate_saves_to_admin_database(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that the API key save operation uses admin database."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# Mock the save method to verify it's called with using='admin'
|
||||
with patch.object(TenantAPIKey, "save") as mock_save:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
# Verify save was called with using=admin_db
|
||||
mock_save.assert_called_once_with(
|
||||
update_fields=["last_used_at"], using=MainRouter.admin_db
|
||||
)
|
||||
|
||||
def test_authenticate_returns_correct_auth_dict(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that the auth dict contains all required fields."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
entity, auth_dict = auth_backend.authenticate(request)
|
||||
|
||||
# Verify all required fields are present
|
||||
assert "tenant_id" in auth_dict
|
||||
assert "sub" in auth_dict
|
||||
assert "api_key_prefix" in auth_dict
|
||||
|
||||
# Verify values are correct
|
||||
assert auth_dict["tenant_id"] == str(api_key.tenant_id)
|
||||
assert auth_dict["sub"] == str(api_key.entity.id)
|
||||
assert auth_dict["api_key_prefix"] == api_key.prefix
|
||||
|
||||
def test_authenticate_with_multiple_api_keys_same_tenant(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test that authentication works correctly with multiple API keys for the same tenant."""
|
||||
# Test with first API key
|
||||
api_key1 = api_keys_fixture[0]
|
||||
raw_key1 = api_key1._raw_key
|
||||
|
||||
request1 = request_factory.get("/")
|
||||
request1.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key1}"
|
||||
|
||||
entity1, auth_dict1 = auth_backend.authenticate(request1)
|
||||
|
||||
assert entity1 == api_key1.entity
|
||||
assert auth_dict1["api_key_prefix"] == api_key1.prefix
|
||||
|
||||
# Test with second API key
|
||||
api_key2 = api_keys_fixture[1]
|
||||
raw_key2 = api_key2._raw_key
|
||||
|
||||
request2 = request_factory.get("/")
|
||||
request2.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key2}"
|
||||
|
||||
entity2, auth_dict2 = auth_backend.authenticate(request2)
|
||||
|
||||
assert entity2 == api_key2.entity
|
||||
assert auth_dict2["api_key_prefix"] == api_key2.prefix
|
||||
|
||||
# Verify they're different keys but same tenant
|
||||
assert auth_dict1["api_key_prefix"] != auth_dict2["api_key_prefix"]
|
||||
assert auth_dict1["tenant_id"] == auth_dict2["tenant_id"]
|
||||
|
||||
def test_authenticate_with_wrong_prefix_in_db(
|
||||
self, auth_backend, api_keys_fixture, request_factory
|
||||
):
|
||||
"""Test authentication fails when prefix doesn't match database."""
|
||||
api_key = api_keys_fixture[0]
|
||||
raw_key = api_key._raw_key
|
||||
|
||||
# Extract the encrypted part and combine with wrong prefix
|
||||
_, encrypted_part = raw_key.split(TenantAPIKey.objects.separator, 1)
|
||||
wrong_key = f"pk_wrong123.{encrypted_part}"
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {wrong_key}"
|
||||
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "Invalid API Key."
|
||||
|
||||
def test_authenticate_credentials_exception_handling(
|
||||
self, auth_backend, request_factory
|
||||
):
|
||||
"""Test that exceptions in _authenticate_credentials are properly handled."""
|
||||
request = request_factory.get("/")
|
||||
|
||||
# Test with completely invalid data that will cause InvalidToken
|
||||
with pytest.raises(Exception):
|
||||
auth_backend._authenticate_credentials(request, "completely_invalid")
|
||||
|
||||
def test_authenticate_with_expired_timestamp(
|
||||
self, auth_backend, create_test_user, tenants_fixture, request_factory
|
||||
):
|
||||
"""Test that expired timestamp in encrypted key causes authentication failure."""
|
||||
tenant = tenants_fixture[0]
|
||||
user = create_test_user
|
||||
|
||||
# Create an API key with a very short expiry
|
||||
api_key, raw_key = TenantAPIKey.objects.create_api_key(
|
||||
name="Short-lived API Key",
|
||||
tenant_id=tenant.id,
|
||||
entity=user,
|
||||
expiry_date=datetime.now(timezone.utc) + timedelta(seconds=1),
|
||||
)
|
||||
|
||||
# Wait for the key to expire
|
||||
time.sleep(2)
|
||||
|
||||
request = request_factory.get("/")
|
||||
request.META["HTTP_AUTHORIZATION"] = f"Api-Key {raw_key}"
|
||||
|
||||
# Should fail with expired key
|
||||
with pytest.raises(AuthenticationFailed) as exc_info:
|
||||
auth_backend.authenticate(request)
|
||||
|
||||
assert str(exc_info.value.detail) == "API Key has already expired."
|
||||
@@ -46,6 +46,7 @@ from api.models import (
|
||||
SAMLConfiguration,
|
||||
SAMLToken,
|
||||
Scan,
|
||||
ScanSummary,
|
||||
StateChoices,
|
||||
Task,
|
||||
TenantAPIKey,
|
||||
@@ -1870,17 +1871,7 @@ class TestProviderSecretViewSet:
|
||||
"kubeconfig_content": "kubeconfig-content",
|
||||
},
|
||||
),
|
||||
# M365 with STATIC secret - no user or password
|
||||
(
|
||||
Provider.ProviderChoices.M365.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"client_id": "client-id",
|
||||
"client_secret": "client-secret",
|
||||
"tenant_id": "tenant-id",
|
||||
},
|
||||
),
|
||||
# M365 with user only
|
||||
# M365 client secret credentials
|
||||
(
|
||||
Provider.ProviderChoices.M365.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
@@ -1889,27 +1880,17 @@ class TestProviderSecretViewSet:
|
||||
"client_secret": "client-secret",
|
||||
"tenant_id": "tenant-id",
|
||||
"user": "test@domain.com",
|
||||
},
|
||||
),
|
||||
# M365 with password only
|
||||
(
|
||||
Provider.ProviderChoices.M365.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"client_id": "client-id",
|
||||
"client_secret": "client-secret",
|
||||
"tenant_id": "tenant-id",
|
||||
"password": "supersecret",
|
||||
},
|
||||
),
|
||||
# M365 with user and password
|
||||
# M365 certificate credentials (valid base64)
|
||||
(
|
||||
Provider.ProviderChoices.M365.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"client_id": "client-id",
|
||||
"client_secret": "client-secret",
|
||||
"tenant_id": "tenant-id",
|
||||
"certificate_content": "VGVzdCBjZXJ0aWZpY2F0ZSBjb250ZW50",
|
||||
"user": "test@domain.com",
|
||||
"password": "supersecret",
|
||||
},
|
||||
@@ -2279,6 +2260,50 @@ class TestProviderSecretViewSet:
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
def test_m365_provider_secrets_invalid_certificate_base64(
|
||||
self, authenticated_client, providers_fixture
|
||||
):
|
||||
"""Test M365 provider secret creation with invalid base64 certificate content"""
|
||||
# Find M365 provider from fixture
|
||||
m365_provider = None
|
||||
for provider in providers_fixture:
|
||||
if provider.provider == Provider.ProviderChoices.M365.value:
|
||||
m365_provider = provider
|
||||
break
|
||||
|
||||
assert m365_provider is not None, "M365 provider not found in fixture"
|
||||
|
||||
data = {
|
||||
"data": {
|
||||
"type": "provider-secrets",
|
||||
"attributes": {
|
||||
"name": "M365 Certificate Invalid Base64",
|
||||
"secret_type": "static",
|
||||
"secret": {
|
||||
"client_id": "client-id",
|
||||
"tenant_id": "tenant-id",
|
||||
"certificate_content": "invalid-base64-content!@#$%",
|
||||
"user": "test@domain.com",
|
||||
"password": "supersecret",
|
||||
},
|
||||
},
|
||||
"relationships": {
|
||||
"provider": {
|
||||
"data": {"type": "providers", "id": str(m365_provider.id)}
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.post(
|
||||
reverse("providersecret-list"),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "certificate content is not valid base64 encoded data" in str(
|
||||
response.json()
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestScanViewSet:
|
||||
@@ -5742,6 +5767,171 @@ class TestOverviewViewSet:
|
||||
assert service1_data["attributes"]["muted"] == 1
|
||||
assert service2_data["attributes"]["muted"] == 0
|
||||
|
||||
def test_overview_findings_provider_id_in_filter(
|
||||
self, authenticated_client, tenants_fixture, providers_fixture
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider1, provider2, *_ = providers_fixture
|
||||
|
||||
scan1 = Scan.objects.create(
|
||||
name="scan-one",
|
||||
provider=provider1,
|
||||
trigger=Scan.TriggerChoices.MANUAL,
|
||||
state=StateChoices.COMPLETED,
|
||||
tenant=tenant,
|
||||
)
|
||||
scan2 = Scan.objects.create(
|
||||
name="scan-two",
|
||||
provider=provider2,
|
||||
trigger=Scan.TriggerChoices.MANUAL,
|
||||
state=StateChoices.COMPLETED,
|
||||
tenant=tenant,
|
||||
)
|
||||
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan1,
|
||||
check_id="check-provider-one",
|
||||
service="service-a",
|
||||
severity="high",
|
||||
region="region-a",
|
||||
_pass=5,
|
||||
fail=1,
|
||||
muted=2,
|
||||
total=8,
|
||||
new=5,
|
||||
changed=2,
|
||||
unchanged=1,
|
||||
fail_new=1,
|
||||
fail_changed=0,
|
||||
pass_new=3,
|
||||
pass_changed=2,
|
||||
muted_new=1,
|
||||
muted_changed=1,
|
||||
)
|
||||
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan2,
|
||||
check_id="check-provider-two",
|
||||
service="service-b",
|
||||
severity="medium",
|
||||
region="region-b",
|
||||
_pass=2,
|
||||
fail=3,
|
||||
muted=1,
|
||||
total=6,
|
||||
new=3,
|
||||
changed=2,
|
||||
unchanged=1,
|
||||
fail_new=2,
|
||||
fail_changed=1,
|
||||
pass_new=1,
|
||||
pass_changed=1,
|
||||
muted_new=1,
|
||||
muted_changed=0,
|
||||
)
|
||||
|
||||
single_response = authenticated_client.get(
|
||||
reverse("overview-findings"),
|
||||
{"filter[provider_id__in]": str(provider1.id)},
|
||||
)
|
||||
assert single_response.status_code == status.HTTP_200_OK
|
||||
single_attributes = single_response.json()["data"]["attributes"]
|
||||
assert single_attributes["pass"] == 5
|
||||
assert single_attributes["fail"] == 1
|
||||
assert single_attributes["muted"] == 2
|
||||
assert single_attributes["total"] == 8
|
||||
|
||||
combined_response = authenticated_client.get(
|
||||
reverse("overview-findings"),
|
||||
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
|
||||
)
|
||||
assert combined_response.status_code == status.HTTP_200_OK
|
||||
combined_attributes = combined_response.json()["data"]["attributes"]
|
||||
assert combined_attributes["pass"] == 7
|
||||
assert combined_attributes["fail"] == 4
|
||||
assert combined_attributes["muted"] == 3
|
||||
assert combined_attributes["total"] == 14
|
||||
|
||||
def test_overview_findings_severity_provider_id_in_filter(
|
||||
self, authenticated_client, tenants_fixture, providers_fixture
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider1, provider2, *_ = providers_fixture
|
||||
|
||||
scan1 = Scan.objects.create(
|
||||
name="severity-scan-one",
|
||||
provider=provider1,
|
||||
trigger=Scan.TriggerChoices.MANUAL,
|
||||
state=StateChoices.COMPLETED,
|
||||
tenant=tenant,
|
||||
)
|
||||
scan2 = Scan.objects.create(
|
||||
name="severity-scan-two",
|
||||
provider=provider2,
|
||||
trigger=Scan.TriggerChoices.MANUAL,
|
||||
state=StateChoices.COMPLETED,
|
||||
tenant=tenant,
|
||||
)
|
||||
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan1,
|
||||
check_id="severity-check-one",
|
||||
service="service-a",
|
||||
severity="high",
|
||||
region="region-a",
|
||||
_pass=4,
|
||||
fail=4,
|
||||
muted=0,
|
||||
total=8,
|
||||
)
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan1,
|
||||
check_id="severity-check-two",
|
||||
service="service-a",
|
||||
severity="medium",
|
||||
region="region-b",
|
||||
_pass=2,
|
||||
fail=2,
|
||||
muted=0,
|
||||
total=4,
|
||||
)
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan2,
|
||||
check_id="severity-check-three",
|
||||
service="service-b",
|
||||
severity="critical",
|
||||
region="region-c",
|
||||
_pass=1,
|
||||
fail=2,
|
||||
muted=0,
|
||||
total=3,
|
||||
)
|
||||
|
||||
single_response = authenticated_client.get(
|
||||
reverse("overview-findings_severity"),
|
||||
{"filter[provider_id__in]": str(provider1.id)},
|
||||
)
|
||||
assert single_response.status_code == status.HTTP_200_OK
|
||||
single_attributes = single_response.json()["data"]["attributes"]
|
||||
assert single_attributes["high"] == 8
|
||||
assert single_attributes["medium"] == 4
|
||||
assert single_attributes["critical"] == 0
|
||||
|
||||
combined_response = authenticated_client.get(
|
||||
reverse("overview-findings_severity"),
|
||||
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
|
||||
)
|
||||
assert combined_response.status_code == status.HTTP_200_OK
|
||||
combined_attributes = combined_response.json()["data"]["attributes"]
|
||||
assert combined_attributes["high"] == 8
|
||||
assert combined_attributes["medium"] == 4
|
||||
assert combined_attributes["critical"] == 3
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestScheduleViewSet:
|
||||
@@ -5781,10 +5971,12 @@ class TestScheduleViewSet:
|
||||
)
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
@patch("tasks.beat.perform_scheduled_scan_task.apply_async")
|
||||
@patch("api.v1.views.Task.objects.get")
|
||||
def test_schedule_daily_already_scheduled(
|
||||
self,
|
||||
mock_task_get,
|
||||
mock_apply_async,
|
||||
authenticated_client,
|
||||
providers_fixture,
|
||||
tasks_fixture,
|
||||
@@ -5792,6 +5984,7 @@ class TestScheduleViewSet:
|
||||
provider, *_ = providers_fixture
|
||||
prowler_task = tasks_fixture[0]
|
||||
mock_task_get.return_value = prowler_task
|
||||
mock_apply_async.return_value.id = prowler_task.id
|
||||
json_payload = {
|
||||
"provider_id": str(provider.id),
|
||||
}
|
||||
@@ -6892,6 +7085,188 @@ class TestTenantFinishACSView:
|
||||
assert response.status_code == 302
|
||||
assert "sso_saml_failed=true" in response.url
|
||||
|
||||
def test_dispatch_skips_role_mapping_when_single_manage_account_user(
|
||||
self, create_test_user, tenants_fixture, saml_setup, settings, monkeypatch
|
||||
):
|
||||
"""Test that role mapping is skipped when tenant has only one user with MANAGE_ACCOUNT role"""
|
||||
monkeypatch.setenv("SAML_SSO_CALLBACK_URL", "http://localhost/sso-complete")
|
||||
user = create_test_user
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
# Create a single role with manage_account=True for the user
|
||||
admin_role = Role.objects.using(MainRouter.admin_db).create(
|
||||
name="admin",
|
||||
tenant=tenant,
|
||||
manage_account=True,
|
||||
manage_users=True,
|
||||
manage_billing=True,
|
||||
manage_providers=True,
|
||||
manage_integrations=True,
|
||||
manage_scans=True,
|
||||
unlimited_visibility=True,
|
||||
)
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
|
||||
user=user, role=admin_role, tenant_id=tenant.id
|
||||
)
|
||||
|
||||
social_account = SocialAccount(
|
||||
user=user,
|
||||
provider="saml",
|
||||
extra_data={
|
||||
"firstName": ["John"],
|
||||
"lastName": ["Doe"],
|
||||
"organization": ["testing_company"],
|
||||
"userType": ["no_permissions"], # This should be ignored
|
||||
},
|
||||
)
|
||||
|
||||
request = RequestFactory().get(
|
||||
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
|
||||
)
|
||||
request.user = user
|
||||
request.session = {}
|
||||
|
||||
with (
|
||||
patch(
|
||||
"allauth.socialaccount.providers.saml.views.get_app_or_404"
|
||||
) as mock_get_app_or_404,
|
||||
patch(
|
||||
"allauth.socialaccount.models.SocialApp.objects.get"
|
||||
) as mock_socialapp_get,
|
||||
patch(
|
||||
"allauth.socialaccount.models.SocialAccount.objects.get"
|
||||
) as mock_sa_get,
|
||||
patch("api.models.SAMLDomainIndex.objects.get") as mock_saml_domain_get,
|
||||
patch("api.models.SAMLConfiguration.objects.get") as mock_saml_config_get,
|
||||
patch("api.models.User.objects.get") as mock_user_get,
|
||||
):
|
||||
mock_get_app_or_404.return_value = MagicMock(
|
||||
provider="saml", client_id="testtenant", name="Test App", settings={}
|
||||
)
|
||||
mock_sa_get.return_value = social_account
|
||||
mock_socialapp_get.return_value = MagicMock(provider_id="saml")
|
||||
mock_saml_domain_get.return_value = SimpleNamespace(tenant_id=tenant.id)
|
||||
mock_saml_config_get.return_value = MagicMock()
|
||||
mock_user_get.return_value = user
|
||||
|
||||
view = TenantFinishACSView.as_view()
|
||||
response = view(request, organization_slug="testtenant")
|
||||
|
||||
assert response.status_code == 302
|
||||
|
||||
# Verify the admin role is still assigned (not changed to no_permissions)
|
||||
assert (
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db)
|
||||
.filter(user=user, role=admin_role, tenant_id=tenant.id)
|
||||
.exists()
|
||||
)
|
||||
|
||||
# Verify no_permissions role was NOT created in the database
|
||||
assert (
|
||||
not Role.objects.using(MainRouter.admin_db)
|
||||
.filter(name="no_permissions", tenant=tenant)
|
||||
.exists()
|
||||
)
|
||||
|
||||
# Verify no_permissions role was NOT assigned to the user
|
||||
assert not (
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db)
|
||||
.filter(user=user, role__name="no_permissions", tenant_id=tenant.id)
|
||||
.exists()
|
||||
)
|
||||
|
||||
def test_dispatch_applies_role_mapping_when_multiple_manage_account_users(
|
||||
self, create_test_user, tenants_fixture, saml_setup, settings, monkeypatch
|
||||
):
|
||||
"""Test that role mapping is applied when tenant has multiple users with MANAGE_ACCOUNT role"""
|
||||
monkeypatch.setenv("SAML_SSO_CALLBACK_URL", "http://localhost/sso-complete")
|
||||
user = create_test_user
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
# Create a second user with manage_account=True
|
||||
second_admin = User.objects.using(MainRouter.admin_db).create(
|
||||
email="admin2@prowler.com", name="Second Admin"
|
||||
)
|
||||
admin_role = Role.objects.using(MainRouter.admin_db).create(
|
||||
name="admin",
|
||||
tenant=tenant,
|
||||
manage_account=True,
|
||||
manage_users=True,
|
||||
manage_billing=True,
|
||||
manage_providers=True,
|
||||
manage_integrations=True,
|
||||
manage_scans=True,
|
||||
unlimited_visibility=True,
|
||||
)
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
|
||||
user=user, role=admin_role, tenant_id=tenant.id
|
||||
)
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
|
||||
user=second_admin, role=admin_role, tenant_id=tenant.id
|
||||
)
|
||||
|
||||
social_account = SocialAccount(
|
||||
user=user,
|
||||
provider="saml",
|
||||
extra_data={
|
||||
"firstName": ["John"],
|
||||
"lastName": ["Doe"],
|
||||
"organization": ["testing_company"],
|
||||
"userType": ["viewer"], # This SHOULD be applied
|
||||
},
|
||||
)
|
||||
|
||||
request = RequestFactory().get(
|
||||
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
|
||||
)
|
||||
request.user = user
|
||||
request.session = {}
|
||||
|
||||
with (
|
||||
patch(
|
||||
"allauth.socialaccount.providers.saml.views.get_app_or_404"
|
||||
) as mock_get_app_or_404,
|
||||
patch(
|
||||
"allauth.socialaccount.models.SocialApp.objects.get"
|
||||
) as mock_socialapp_get,
|
||||
patch(
|
||||
"allauth.socialaccount.models.SocialAccount.objects.get"
|
||||
) as mock_sa_get,
|
||||
patch("api.models.SAMLDomainIndex.objects.get") as mock_saml_domain_get,
|
||||
patch("api.models.SAMLConfiguration.objects.get") as mock_saml_config_get,
|
||||
patch("api.models.User.objects.get") as mock_user_get,
|
||||
):
|
||||
mock_get_app_or_404.return_value = MagicMock(
|
||||
provider="saml", client_id="testtenant", name="Test App", settings={}
|
||||
)
|
||||
mock_sa_get.return_value = social_account
|
||||
mock_socialapp_get.return_value = MagicMock(provider_id="saml")
|
||||
mock_saml_domain_get.return_value = SimpleNamespace(tenant_id=tenant.id)
|
||||
mock_saml_config_get.return_value = MagicMock()
|
||||
mock_user_get.return_value = user
|
||||
|
||||
view = TenantFinishACSView.as_view()
|
||||
response = view(request, organization_slug="testtenant")
|
||||
|
||||
assert response.status_code == 302
|
||||
|
||||
# Verify the viewer role was created and assigned (role mapping was applied)
|
||||
viewer_role = Role.objects.using(MainRouter.admin_db).get(
|
||||
name="viewer", tenant=tenant
|
||||
)
|
||||
assert (
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db)
|
||||
.filter(user=user, role=viewer_role, tenant_id=tenant.id)
|
||||
.exists()
|
||||
)
|
||||
|
||||
# Verify the admin role was removed (replaced by viewer)
|
||||
assert not (
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db)
|
||||
.filter(user=user, role=admin_role, tenant_id=tenant.id)
|
||||
.exists()
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestLighthouseConfigViewSet:
|
||||
@@ -7530,8 +7905,6 @@ class TestTenantApiKeyViewSet:
|
||||
(
|
||||
[
|
||||
{"name": "New API Key"},
|
||||
{"name": ""},
|
||||
{},
|
||||
]
|
||||
),
|
||||
)
|
||||
@@ -7572,6 +7945,18 @@ class TestTenantApiKeyViewSet:
|
||||
{"name": "Invalid Expiry", "expires_at": "not-a-date"},
|
||||
"expires_at",
|
||||
),
|
||||
(
|
||||
{"name": ""},
|
||||
"name",
|
||||
),
|
||||
(
|
||||
{},
|
||||
"name",
|
||||
),
|
||||
(
|
||||
{"name": "AB"}, # Too short (min length is 3)
|
||||
"name",
|
||||
),
|
||||
]
|
||||
),
|
||||
)
|
||||
@@ -7600,6 +7985,58 @@ class TestTenantApiKeyViewSet:
|
||||
== f"/data/attributes/{error_pointer}"
|
||||
)
|
||||
|
||||
def test_api_keys_create_duplicate_name(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
"""Test creating an API key with a duplicate name fails."""
|
||||
# Use the name of an existing API key
|
||||
existing_name = api_keys_fixture[0].name
|
||||
data = {
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"attributes": {
|
||||
"name": existing_name,
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.post(
|
||||
reverse("api-key-list"),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "errors" in response.json()
|
||||
error_detail = response.json()["errors"][0]["detail"]
|
||||
assert "already exists" in error_detail.lower()
|
||||
|
||||
def test_api_keys_update_duplicate_name(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
"""Test updating an API key with a duplicate name fails."""
|
||||
# Get two different API keys
|
||||
first_api_key = api_keys_fixture[0]
|
||||
second_api_key = api_keys_fixture[1]
|
||||
|
||||
# Try to update the second API key to have the same name as the first one
|
||||
data = {
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": str(second_api_key.id),
|
||||
"attributes": {
|
||||
"name": first_api_key.name,
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.patch(
|
||||
reverse("api-key-detail", kwargs={"pk": second_api_key.id}),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "errors" in response.json()
|
||||
error_detail = response.json()["errors"][0]["detail"]
|
||||
assert "already exists" in error_detail.lower()
|
||||
|
||||
def test_api_keys_create_multiple_unique_prefixes(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
@@ -7677,6 +8114,27 @@ class TestTenantApiKeyViewSet:
|
||||
api_key.refresh_from_db()
|
||||
assert api_key.revoked is True
|
||||
|
||||
def test_api_keys_revoke_preserves_created_field(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
"""Test that revoking an API key preserves the created timestamp."""
|
||||
api_key = api_keys_fixture[0] # Not revoked
|
||||
assert api_key.revoked is False
|
||||
|
||||
# Record the original created timestamp
|
||||
original_created = api_key.created
|
||||
|
||||
response = authenticated_client.delete(
|
||||
reverse("api-key-revoke", kwargs={"pk": api_key.id})
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
|
||||
# Verify in database
|
||||
api_key.refresh_from_db()
|
||||
assert api_key.revoked is True
|
||||
# Verify created field has not changed
|
||||
assert api_key.created == original_created
|
||||
|
||||
def test_api_keys_revoke_already_revoked(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
@@ -8024,6 +8482,53 @@ class TestTenantApiKeyViewSet:
|
||||
assert data["relationships"]["entity"]["data"]["type"] == "users"
|
||||
assert data["relationships"]["entity"]["data"]["id"] == str(api_key.entity.id)
|
||||
|
||||
def test_api_keys_retrieve_with_entity_include(
|
||||
self, authenticated_client, api_keys_fixture
|
||||
):
|
||||
"""Test retrieving API key with ?include=entity returns user data without memberships."""
|
||||
api_key = api_keys_fixture[0]
|
||||
response = authenticated_client.get(
|
||||
reverse("api-key-detail", kwargs={"pk": api_key.id}),
|
||||
{"include": "entity"},
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
response_data = response.json()
|
||||
|
||||
# Verify the main data contains the entity relationship
|
||||
data = response_data["data"]
|
||||
assert "entity" in data["relationships"]
|
||||
assert data["relationships"]["entity"]["data"]["type"] == "users"
|
||||
assert data["relationships"]["entity"]["data"]["id"] == str(api_key.entity.id)
|
||||
|
||||
# Verify included section exists
|
||||
assert "included" in response_data
|
||||
assert len(response_data["included"]) == 1
|
||||
|
||||
# Verify included user data
|
||||
included_user = response_data["included"][0]
|
||||
assert included_user["type"] == "users"
|
||||
assert included_user["id"] == str(api_key.entity.id)
|
||||
|
||||
# Refresh entity from database to get current state
|
||||
# (in case other tests modified the shared session-scoped user fixture)
|
||||
api_key.entity.refresh_from_db()
|
||||
|
||||
# Verify UserIncludeSerializer fields are present
|
||||
user_attrs = included_user["attributes"]
|
||||
assert "name" in user_attrs
|
||||
assert "email" in user_attrs
|
||||
assert "company_name" in user_attrs
|
||||
assert "date_joined" in user_attrs
|
||||
assert user_attrs["name"] == api_key.entity.name
|
||||
assert user_attrs["email"] == api_key.entity.email
|
||||
|
||||
# Verify memberships field is NOT included (excluded by UserIncludeSerializer)
|
||||
assert "memberships" not in user_attrs
|
||||
|
||||
# Verify roles relationship is present
|
||||
assert "relationships" in included_user
|
||||
assert "roles" in included_user["relationships"]
|
||||
|
||||
def test_api_keys_entity_auto_assigned_on_create(
|
||||
self, authenticated_client, create_test_user
|
||||
):
|
||||
|
||||
@@ -105,23 +105,25 @@ from rest_framework_json_api import serializers
|
||||
"type": "string",
|
||||
"description": "The Azure application (client) ID for authentication in Azure AD.",
|
||||
},
|
||||
"client_secret": {
|
||||
"type": "string",
|
||||
"description": "The client secret associated with the application (client) ID, providing "
|
||||
"secure access.",
|
||||
},
|
||||
"tenant_id": {
|
||||
"type": "string",
|
||||
"description": "The Azure tenant ID, representing the directory where the application is "
|
||||
"registered.",
|
||||
},
|
||||
"client_secret": {
|
||||
"type": "string",
|
||||
"description": "The client secret associated with the application (client) ID, providing "
|
||||
"secure access.",
|
||||
},
|
||||
"user": {
|
||||
"type": "email",
|
||||
"description": "User microsoft email address.",
|
||||
"deprecated": True,
|
||||
},
|
||||
"password": {
|
||||
"type": "string",
|
||||
"description": "User password.",
|
||||
"deprecated": True,
|
||||
},
|
||||
},
|
||||
"required": [
|
||||
@@ -132,6 +134,30 @@ from rest_framework_json_api import serializers
|
||||
"password",
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"title": "M365 Certificate Credentials",
|
||||
"properties": {
|
||||
"client_id": {
|
||||
"type": "string",
|
||||
"description": "The Azure application (client) ID for authentication in Azure AD.",
|
||||
},
|
||||
"tenant_id": {
|
||||
"type": "string",
|
||||
"description": "The Azure tenant ID, representing the directory where the application is "
|
||||
"registered.",
|
||||
},
|
||||
"certificate_content": {
|
||||
"type": "string",
|
||||
"description": "The certificate content in base64 format for certificate-based authentication.",
|
||||
},
|
||||
},
|
||||
"required": [
|
||||
"client_id",
|
||||
"tenant_id",
|
||||
"certificate_content",
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"title": "GCP Static Credentials",
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import base64
|
||||
import json
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
@@ -317,6 +318,23 @@ class UserSerializer(BaseSerializerV1):
|
||||
)
|
||||
|
||||
|
||||
class UserIncludeSerializer(UserSerializer):
|
||||
class Meta:
|
||||
model = User
|
||||
fields = [
|
||||
"id",
|
||||
"name",
|
||||
"email",
|
||||
"company_name",
|
||||
"date_joined",
|
||||
"roles",
|
||||
]
|
||||
|
||||
included_serializers = {
|
||||
"roles": "api.v1.serializers.RoleIncludeSerializer",
|
||||
}
|
||||
|
||||
|
||||
class UserCreateSerializer(BaseWriteSerializer):
|
||||
password = serializers.CharField(write_only=True)
|
||||
company_name = serializers.CharField(required=False)
|
||||
@@ -1378,10 +1396,38 @@ class AzureProviderSecret(serializers.Serializer):
|
||||
|
||||
class M365ProviderSecret(serializers.Serializer):
|
||||
client_id = serializers.CharField()
|
||||
client_secret = serializers.CharField()
|
||||
client_secret = serializers.CharField(required=False)
|
||||
tenant_id = serializers.CharField()
|
||||
user = serializers.EmailField(required=False)
|
||||
password = serializers.CharField(required=False)
|
||||
certificate_content = serializers.CharField(required=False)
|
||||
|
||||
def validate(self, attrs):
|
||||
if attrs.get("client_secret") and attrs.get("certificate_content"):
|
||||
raise serializers.ValidationError(
|
||||
"You cannot provide both client_secret and certificate_content."
|
||||
)
|
||||
if not attrs.get("client_secret") and not attrs.get("certificate_content"):
|
||||
raise serializers.ValidationError(
|
||||
"You must provide either client_secret or certificate_content."
|
||||
)
|
||||
return super().validate(attrs)
|
||||
|
||||
def validate_certificate_content(self, certificate_content):
|
||||
"""Validate that M365 certificate content is valid base64 encoded data."""
|
||||
if certificate_content:
|
||||
try:
|
||||
base64.b64decode(certificate_content, validate=True)
|
||||
except Exception as e:
|
||||
raise ValidationError(
|
||||
{
|
||||
"certificate_content": [
|
||||
f"The provided certificate content is not valid base64 encoded data: {str(e)}"
|
||||
]
|
||||
},
|
||||
code="m365-certificate-content",
|
||||
)
|
||||
return certificate_content
|
||||
|
||||
class Meta:
|
||||
resource_name = "provider-secrets"
|
||||
@@ -1974,6 +2020,17 @@ class ComplianceOverviewDetailSerializer(serializers.Serializer):
|
||||
resource_name = "compliance-requirements-details"
|
||||
|
||||
|
||||
class ComplianceOverviewDetailThreatscoreSerializer(ComplianceOverviewDetailSerializer):
|
||||
"""
|
||||
Serializer for detailed compliance requirement information for Threatscore.
|
||||
|
||||
Includes additional fields specific to the Threatscore framework.
|
||||
"""
|
||||
|
||||
passed_findings = serializers.IntegerField()
|
||||
total_findings = serializers.IntegerField()
|
||||
|
||||
|
||||
class ComplianceOverviewAttributesSerializer(serializers.Serializer):
|
||||
id = serializers.CharField()
|
||||
compliance_name = serializers.CharField()
|
||||
@@ -2779,6 +2836,10 @@ class TenantApiKeySerializer(RLSSerializer):
|
||||
"entity",
|
||||
]
|
||||
|
||||
included_serializers = {
|
||||
"entity": "api.v1.serializers.UserIncludeSerializer",
|
||||
}
|
||||
|
||||
|
||||
class TenantApiKeyCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"""Serializer for creating new API keys."""
|
||||
@@ -2811,6 +2872,13 @@ class TenantApiKeyCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"api_key": {"read_only": True},
|
||||
}
|
||||
|
||||
def validate_name(self, value):
|
||||
"""Validate that the name is unique within the tenant."""
|
||||
tenant_id = self.context.get("tenant_id")
|
||||
if TenantAPIKey.objects.filter(tenant_id=tenant_id, name=value).exists():
|
||||
raise ValidationError("An API key with this name already exists.")
|
||||
return value
|
||||
|
||||
def get_api_key(self, obj):
|
||||
"""Return the raw API key if it was stored during creation."""
|
||||
return getattr(obj, "_raw_api_key", None)
|
||||
@@ -2852,3 +2920,14 @@ class TenantApiKeyUpdateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"inserted_at": {"read_only": True},
|
||||
"last_used_at": {"read_only": True},
|
||||
}
|
||||
|
||||
def validate_name(self, value):
|
||||
"""Validate that the name is unique within the tenant, excluding current instance."""
|
||||
tenant_id = self.context.get("tenant_id")
|
||||
if (
|
||||
TenantAPIKey.objects.filter(tenant_id=tenant_id, name=value)
|
||||
.exclude(id=self.instance.id)
|
||||
.exists()
|
||||
):
|
||||
raise ValidationError("An API key with this name already exists.")
|
||||
return value
|
||||
|
||||
@@ -142,6 +142,7 @@ from api.v1.mixins import PaginateByPkMixin, TaskManagementMixin
|
||||
from api.v1.serializers import (
|
||||
ComplianceOverviewAttributesSerializer,
|
||||
ComplianceOverviewDetailSerializer,
|
||||
ComplianceOverviewDetailThreatscoreSerializer,
|
||||
ComplianceOverviewMetadataSerializer,
|
||||
ComplianceOverviewSerializer,
|
||||
FindingDynamicFilterSerializer,
|
||||
@@ -665,36 +666,48 @@ class TenantFinishACSView(FinishACSView):
|
||||
.get(email_domain=email_domain)
|
||||
.tenant
|
||||
)
|
||||
role_name = (
|
||||
extra.get("userType", ["no_permissions"])[0].strip()
|
||||
if extra.get("userType")
|
||||
else "no_permissions"
|
||||
|
||||
# Check if tenant has only one user with MANAGE_ACCOUNT role
|
||||
users_with_manage_account = (
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db)
|
||||
.filter(role__manage_account=True, tenant_id=tenant.id)
|
||||
.values("user")
|
||||
.distinct()
|
||||
.count()
|
||||
)
|
||||
try:
|
||||
role = Role.objects.using(MainRouter.admin_db).get(
|
||||
name=role_name, tenant=tenant
|
||||
|
||||
# Only apply role mapping from userType if tenant does NOT have exactly one user with MANAGE_ACCOUNT
|
||||
if users_with_manage_account != 1:
|
||||
role_name = (
|
||||
extra.get("userType", ["no_permissions"])[0].strip()
|
||||
if extra.get("userType")
|
||||
else "no_permissions"
|
||||
)
|
||||
except Role.DoesNotExist:
|
||||
role = Role.objects.using(MainRouter.admin_db).create(
|
||||
name=role_name,
|
||||
tenant=tenant,
|
||||
manage_users=False,
|
||||
manage_account=False,
|
||||
manage_billing=False,
|
||||
manage_providers=False,
|
||||
manage_integrations=False,
|
||||
manage_scans=False,
|
||||
unlimited_visibility=False,
|
||||
try:
|
||||
role = Role.objects.using(MainRouter.admin_db).get(
|
||||
name=role_name, tenant=tenant
|
||||
)
|
||||
except Role.DoesNotExist:
|
||||
role = Role.objects.using(MainRouter.admin_db).create(
|
||||
name=role_name,
|
||||
tenant=tenant,
|
||||
manage_users=False,
|
||||
manage_account=False,
|
||||
manage_billing=False,
|
||||
manage_providers=False,
|
||||
manage_integrations=False,
|
||||
manage_scans=False,
|
||||
unlimited_visibility=False,
|
||||
)
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).filter(
|
||||
user=user,
|
||||
tenant_id=tenant.id,
|
||||
).delete()
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
|
||||
user=user,
|
||||
role=role,
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).filter(
|
||||
user=user,
|
||||
tenant_id=tenant.id,
|
||||
).delete()
|
||||
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
|
||||
user=user,
|
||||
role=role,
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
membership, _ = Membership.objects.using(MainRouter.admin_db).get_or_create(
|
||||
user=user,
|
||||
tenant=tenant,
|
||||
@@ -3436,15 +3449,16 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
|
||||
)
|
||||
filtered_queryset = self.filter_queryset(self.get_queryset())
|
||||
|
||||
all_requirements = (
|
||||
filtered_queryset.values(
|
||||
"requirement_id", "framework", "version", "description"
|
||||
)
|
||||
.distinct()
|
||||
.annotate(
|
||||
total_instances=Count("id"),
|
||||
manual_count=Count("id", filter=Q(requirement_status="MANUAL")),
|
||||
)
|
||||
all_requirements = filtered_queryset.values(
|
||||
"requirement_id",
|
||||
"framework",
|
||||
"version",
|
||||
"description",
|
||||
).annotate(
|
||||
total_instances=Count("id"),
|
||||
manual_count=Count("id", filter=Q(requirement_status="MANUAL")),
|
||||
passed_findings_sum=Sum("passed_findings"),
|
||||
total_findings_sum=Sum("total_findings"),
|
||||
)
|
||||
|
||||
passed_instances = (
|
||||
@@ -3463,6 +3477,8 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
|
||||
total_instances = requirement["total_instances"]
|
||||
passed_count = passed_counts.get(requirement_id, 0)
|
||||
is_manual = requirement["manual_count"] == total_instances
|
||||
passed_findings = requirement["passed_findings_sum"] or 0
|
||||
total_findings = requirement["total_findings_sum"] or 0
|
||||
if is_manual:
|
||||
requirement_status = "MANUAL"
|
||||
elif passed_count == total_instances:
|
||||
@@ -3477,10 +3493,19 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
|
||||
"version": requirement["version"],
|
||||
"description": requirement["description"],
|
||||
"status": requirement_status,
|
||||
"passed_findings": passed_findings,
|
||||
"total_findings": total_findings,
|
||||
}
|
||||
)
|
||||
|
||||
serializer = self.get_serializer(requirements_summary, many=True)
|
||||
# Use different serializer for threatscore framework
|
||||
if "threatscore" not in compliance_id:
|
||||
serializer = self.get_serializer(requirements_summary, many=True)
|
||||
else:
|
||||
serializer = ComplianceOverviewDetailThreatscoreSerializer(
|
||||
requirements_summary, many=True
|
||||
)
|
||||
|
||||
return Response(serializer.data, status=status.HTTP_200_OK)
|
||||
|
||||
@action(detail=False, methods=["get"], url_name="attributes")
|
||||
|
||||
@@ -5,24 +5,39 @@ DEBUG = env.bool("DJANGO_DEBUG", default=True)
|
||||
ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["*"])
|
||||
|
||||
# Database
|
||||
default_db_name = env("POSTGRES_DB", default="prowler_db")
|
||||
default_db_user = env("POSTGRES_USER", default="prowler_user")
|
||||
default_db_password = env("POSTGRES_PASSWORD", default="prowler")
|
||||
default_db_host = env("POSTGRES_HOST", default="postgres-db")
|
||||
default_db_port = env("POSTGRES_PORT", default="5432")
|
||||
|
||||
DATABASES = {
|
||||
"prowler_user": {
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": env("POSTGRES_DB", default="prowler_db"),
|
||||
"USER": env("POSTGRES_USER", default="prowler_user"),
|
||||
"PASSWORD": env("POSTGRES_PASSWORD", default="prowler"),
|
||||
"HOST": env("POSTGRES_HOST", default="postgres-db"),
|
||||
"PORT": env("POSTGRES_PORT", default="5432"),
|
||||
"NAME": default_db_name,
|
||||
"USER": default_db_user,
|
||||
"PASSWORD": default_db_password,
|
||||
"HOST": default_db_host,
|
||||
"PORT": default_db_port,
|
||||
},
|
||||
"admin": {
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": env("POSTGRES_DB", default="prowler_db"),
|
||||
"NAME": default_db_name,
|
||||
"USER": env("POSTGRES_ADMIN_USER", default="prowler"),
|
||||
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD", default="S3cret"),
|
||||
"HOST": env("POSTGRES_HOST", default="postgres-db"),
|
||||
"PORT": env("POSTGRES_PORT", default="5432"),
|
||||
"HOST": default_db_host,
|
||||
"PORT": default_db_port,
|
||||
},
|
||||
"replica": {
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": env("POSTGRES_REPLICA_DB", default=default_db_name),
|
||||
"USER": env("POSTGRES_REPLICA_USER", default=default_db_user),
|
||||
"PASSWORD": env("POSTGRES_REPLICA_PASSWORD", default=default_db_password),
|
||||
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
|
||||
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
|
||||
},
|
||||
}
|
||||
|
||||
DATABASES["default"] = DATABASES["prowler_user"]
|
||||
|
||||
REST_FRAMEWORK["DEFAULT_RENDERER_CLASSES"] = tuple( # noqa: F405
|
||||
|
||||
@@ -6,22 +6,37 @@ ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["localhost", "127.0.0.
|
||||
|
||||
# Database
|
||||
# TODO Use Django database routers https://docs.djangoproject.com/en/5.0/topics/db/multi-db/#automatic-database-routing
|
||||
default_db_name = env("POSTGRES_DB")
|
||||
default_db_user = env("POSTGRES_USER")
|
||||
default_db_password = env("POSTGRES_PASSWORD")
|
||||
default_db_host = env("POSTGRES_HOST")
|
||||
default_db_port = env("POSTGRES_PORT")
|
||||
|
||||
DATABASES = {
|
||||
"prowler_user": {
|
||||
"ENGINE": "django.db.backends.postgresql",
|
||||
"NAME": env("POSTGRES_DB"),
|
||||
"USER": env("POSTGRES_USER"),
|
||||
"PASSWORD": env("POSTGRES_PASSWORD"),
|
||||
"HOST": env("POSTGRES_HOST"),
|
||||
"PORT": env("POSTGRES_PORT"),
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": default_db_name,
|
||||
"USER": default_db_user,
|
||||
"PASSWORD": default_db_password,
|
||||
"HOST": default_db_host,
|
||||
"PORT": default_db_port,
|
||||
},
|
||||
"admin": {
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": env("POSTGRES_DB"),
|
||||
"NAME": default_db_name,
|
||||
"USER": env("POSTGRES_ADMIN_USER"),
|
||||
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD"),
|
||||
"HOST": env("POSTGRES_HOST"),
|
||||
"PORT": env("POSTGRES_PORT"),
|
||||
"HOST": default_db_host,
|
||||
"PORT": default_db_port,
|
||||
},
|
||||
"replica": {
|
||||
"ENGINE": "psqlextra.backend",
|
||||
"NAME": env("POSTGRES_REPLICA_DB", default=default_db_name),
|
||||
"USER": env("POSTGRES_REPLICA_USER", default=default_db_user),
|
||||
"PASSWORD": env("POSTGRES_REPLICA_PASSWORD", default=default_db_password),
|
||||
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
|
||||
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
|
||||
},
|
||||
}
|
||||
|
||||
DATABASES["default"] = DATABASES["prowler_user"]
|
||||
|
||||
@@ -20,6 +20,10 @@ from prowler.lib.outputs.asff.asff import ASFF
|
||||
from prowler.lib.outputs.compliance.aws_well_architected.aws_well_architected import (
|
||||
AWSWellArchitected,
|
||||
)
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_aws import CCC_AWS
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_azure import CCC_Azure
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_gcp import CCC_GCP
|
||||
from prowler.lib.outputs.compliance.c5.c5_aws import AWSC5
|
||||
from prowler.lib.outputs.compliance.cis.cis_aws import AWSCIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_azure import AzureCIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_gcp import GCPCIS
|
||||
@@ -73,12 +77,15 @@ COMPLIANCE_CLASS_MAP = {
|
||||
(lambda name: name.startswith("iso27001_"), AWSISO27001),
|
||||
(lambda name: name.startswith("kisa"), AWSKISAISMSP),
|
||||
(lambda name: name == "prowler_threatscore_aws", ProwlerThreatScoreAWS),
|
||||
(lambda name: name == "ccc_aws", CCC_AWS),
|
||||
(lambda name: name.startswith("c5_"), AWSC5),
|
||||
],
|
||||
"azure": [
|
||||
(lambda name: name.startswith("cis_"), AzureCIS),
|
||||
(lambda name: name == "mitre_attack_azure", AzureMitreAttack),
|
||||
(lambda name: name.startswith("ens_"), AzureENS),
|
||||
(lambda name: name.startswith("iso27001_"), AzureISO27001),
|
||||
(lambda name: name == "ccc_azure", CCC_Azure),
|
||||
(lambda name: name == "prowler_threatscore_azure", ProwlerThreatScoreAzure),
|
||||
],
|
||||
"gcp": [
|
||||
@@ -87,6 +94,7 @@ COMPLIANCE_CLASS_MAP = {
|
||||
(lambda name: name.startswith("ens_"), GCPENS),
|
||||
(lambda name: name.startswith("iso27001_"), GCPISO27001),
|
||||
(lambda name: name == "prowler_threatscore_gcp", ProwlerThreatScoreGCP),
|
||||
(lambda name: name == "ccc_gcp", CCC_GCP),
|
||||
],
|
||||
"kubernetes": [
|
||||
(lambda name: name.startswith("cis_"), KubernetesCIS),
|
||||
|
||||
@@ -5,6 +5,7 @@ from celery.utils.log import get_task_logger
|
||||
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
|
||||
from tasks.utils import batched
|
||||
|
||||
from api.db_router import READ_REPLICA_ALIAS
|
||||
from api.db_utils import rls_transaction
|
||||
from api.models import Finding, Integration, Provider
|
||||
from api.utils import initialize_prowler_integration, initialize_prowler_provider
|
||||
@@ -289,7 +290,7 @@ def upload_security_hub_integration(
|
||||
has_findings = False
|
||||
batch_number = 0
|
||||
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
qs = (
|
||||
Finding.all_objects.filter(tenant_id=tenant_id, scan_id=scan_id)
|
||||
.order_by("uid")
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
import csv
|
||||
import io
|
||||
import json
|
||||
import time
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
from copy import deepcopy
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any
|
||||
|
||||
from celery.utils.log import get_task_logger
|
||||
from config.settings.celery import CELERY_DEADLOCK_ATTEMPTS
|
||||
@@ -14,8 +18,11 @@ from api.compliance import (
|
||||
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
|
||||
generate_scan_compliance,
|
||||
)
|
||||
from api.db_router import READ_REPLICA_ALIAS, MainRouter
|
||||
from api.db_utils import (
|
||||
create_objects_in_batches,
|
||||
POSTGRES_TENANT_VAR,
|
||||
SET_CONFIG_QUERY,
|
||||
psycopg_connection,
|
||||
rls_transaction,
|
||||
update_objects_in_batches,
|
||||
)
|
||||
@@ -40,6 +47,28 @@ from prowler.lib.scan.scan import Scan as ProwlerScan
|
||||
|
||||
logger = get_task_logger(__name__)
|
||||
|
||||
# Column order must match `ComplianceRequirementOverview` schema in
|
||||
# `api/models.py`. Keep this list minimal but sufficient to populate all
|
||||
# non-nullable fields plus the counters we care about.
|
||||
COMPLIANCE_REQUIREMENT_COPY_COLUMNS = (
|
||||
"id",
|
||||
"tenant_id",
|
||||
"inserted_at",
|
||||
"compliance_id",
|
||||
"framework",
|
||||
"version",
|
||||
"description",
|
||||
"region",
|
||||
"requirement_id",
|
||||
"requirement_status",
|
||||
"passed_checks",
|
||||
"failed_checks",
|
||||
"total_checks",
|
||||
"passed_findings",
|
||||
"total_findings",
|
||||
"scan_id",
|
||||
)
|
||||
|
||||
|
||||
def _create_finding_delta(
|
||||
last_status: FindingStatus | None | str, new_status: FindingStatus | None
|
||||
@@ -107,6 +136,124 @@ def _store_resources(
|
||||
return resource_instance, (resource_instance.uid, resource_instance.region)
|
||||
|
||||
|
||||
def _copy_compliance_requirement_rows(
|
||||
tenant_id: str, rows: list[dict[str, Any]]
|
||||
) -> None:
|
||||
"""Stream compliance requirement rows into Postgres using COPY.
|
||||
|
||||
We leverage the admin connection (when available) to bypass the COPY + RLS
|
||||
restriction, writing only the fields required by
|
||||
``ComplianceRequirementOverview``.
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
rows: List of row dictionaries prepared by
|
||||
:func:`create_compliance_requirements`.
|
||||
"""
|
||||
|
||||
csv_buffer = io.StringIO()
|
||||
writer = csv.writer(csv_buffer)
|
||||
|
||||
datetime_now = datetime.now(tz=timezone.utc)
|
||||
for row in rows:
|
||||
writer.writerow(
|
||||
[
|
||||
str(row.get("id")),
|
||||
str(row.get("tenant_id")),
|
||||
(row.get("inserted_at") or datetime_now).isoformat(),
|
||||
row.get("compliance_id") or "",
|
||||
row.get("framework") or "",
|
||||
row.get("version") or "",
|
||||
row.get("description") or "",
|
||||
row.get("region") or "",
|
||||
row.get("requirement_id") or "",
|
||||
row.get("requirement_status") or "",
|
||||
row.get("passed_checks", 0),
|
||||
row.get("failed_checks", 0),
|
||||
row.get("total_checks", 0),
|
||||
row.get("passed_findings", 0),
|
||||
row.get("total_findings", 0),
|
||||
str(row.get("scan_id")),
|
||||
]
|
||||
)
|
||||
|
||||
csv_buffer.seek(0)
|
||||
copy_sql = (
|
||||
"COPY compliance_requirements_overviews ("
|
||||
+ ", ".join(COMPLIANCE_REQUIREMENT_COPY_COLUMNS)
|
||||
+ ") FROM STDIN WITH (FORMAT CSV, DELIMITER ',', QUOTE '\"', ESCAPE '\"', NULL '\\N')"
|
||||
)
|
||||
|
||||
try:
|
||||
with psycopg_connection(MainRouter.admin_db) as connection:
|
||||
connection.autocommit = False
|
||||
try:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.execute(SET_CONFIG_QUERY, [POSTGRES_TENANT_VAR, tenant_id])
|
||||
cursor.copy_expert(copy_sql, csv_buffer)
|
||||
connection.commit()
|
||||
except Exception:
|
||||
connection.rollback()
|
||||
raise
|
||||
finally:
|
||||
csv_buffer.close()
|
||||
|
||||
|
||||
def _persist_compliance_requirement_rows(
|
||||
tenant_id: str, rows: list[dict[str, Any]]
|
||||
) -> None:
|
||||
"""Persist compliance requirement rows using COPY with ORM fallback.
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
rows: Precomputed row dictionaries that reflect the compliance
|
||||
overview state for a scan.
|
||||
"""
|
||||
if not rows:
|
||||
return
|
||||
|
||||
try:
|
||||
_copy_compliance_requirement_rows(tenant_id, rows)
|
||||
except Exception as error:
|
||||
logger.exception(
|
||||
"COPY bulk insert for compliance requirements failed; falling back to ORM bulk_create",
|
||||
exc_info=error,
|
||||
)
|
||||
fallback_objects = [
|
||||
ComplianceRequirementOverview(
|
||||
id=row["id"],
|
||||
tenant_id=row["tenant_id"],
|
||||
inserted_at=row["inserted_at"],
|
||||
compliance_id=row["compliance_id"],
|
||||
framework=row["framework"],
|
||||
version=row["version"],
|
||||
description=row["description"],
|
||||
region=row["region"],
|
||||
requirement_id=row["requirement_id"],
|
||||
requirement_status=row["requirement_status"],
|
||||
passed_checks=row["passed_checks"],
|
||||
failed_checks=row["failed_checks"],
|
||||
total_checks=row["total_checks"],
|
||||
passed_findings=row.get("passed_findings", 0),
|
||||
total_findings=row.get("total_findings", 0),
|
||||
scan_id=row["scan_id"],
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
with rls_transaction(tenant_id):
|
||||
ComplianceRequirementOverview.objects.bulk_create(
|
||||
fallback_objects, batch_size=500
|
||||
)
|
||||
|
||||
|
||||
def _normalized_compliance_key(framework: str | None, version: str | None) -> str:
|
||||
"""Return normalized identifier used to group compliance totals."""
|
||||
|
||||
normalized_framework = (framework or "").lower().replace("-", "").replace("_", "")
|
||||
normalized_version = (version or "").lower().replace("-", "").replace("_", "")
|
||||
return f"{normalized_framework}{normalized_version}"
|
||||
|
||||
|
||||
def perform_prowler_scan(
|
||||
tenant_id: str,
|
||||
scan_id: str,
|
||||
@@ -143,7 +290,7 @@ def perform_prowler_scan(
|
||||
scan_instance.save()
|
||||
|
||||
# Find the mutelist processor if it exists
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
try:
|
||||
mutelist_processor = Processor.objects.get(
|
||||
tenant_id=tenant_id, processor_type=Processor.ProcessorChoices.MUTELIST
|
||||
@@ -272,7 +419,7 @@ def perform_prowler_scan(
|
||||
unique_resources.add((resource_instance.uid, resource_instance.region))
|
||||
|
||||
# Process finding
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
finding_uid = finding.uid
|
||||
last_first_seen_at = None
|
||||
if finding_uid not in last_status_cache:
|
||||
@@ -305,6 +452,12 @@ def perform_prowler_scan(
|
||||
# If the finding is muted at this time the reason must be the configured Mutelist
|
||||
muted_reason = "Muted by mutelist" if finding.muted else None
|
||||
|
||||
# Increment failed_findings_count cache if the finding status is FAIL and not muted
|
||||
if status == FindingStatus.FAIL and not finding.muted:
|
||||
resource_uid = finding.resource_uid
|
||||
resource_failed_findings_cache[resource_uid] += 1
|
||||
|
||||
with rls_transaction(tenant_id):
|
||||
# Create the finding
|
||||
finding_instance = Finding.objects.create(
|
||||
tenant_id=tenant_id,
|
||||
@@ -325,11 +478,6 @@ def perform_prowler_scan(
|
||||
)
|
||||
finding_instance.add_resources([resource_instance])
|
||||
|
||||
# Increment failed_findings_count cache if the finding status is FAIL and not muted
|
||||
if status == FindingStatus.FAIL and not finding.muted:
|
||||
resource_uid = finding.resource_uid
|
||||
resource_failed_findings_cache[resource_uid] += 1
|
||||
|
||||
# Update scan resource summaries
|
||||
scan_resource_cache.add(
|
||||
(
|
||||
@@ -439,7 +587,7 @@ def aggregate_findings(tenant_id: str, scan_id: str):
|
||||
- muted_new: Muted findings with a delta of 'new'.
|
||||
- muted_changed: Muted findings with a delta of 'changed'.
|
||||
"""
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
findings = Finding.objects.filter(tenant_id=tenant_id, scan_id=scan_id)
|
||||
|
||||
aggregation = findings.values(
|
||||
@@ -582,15 +730,32 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
ValidationError: If tenant_id is not a valid UUID.
|
||||
"""
|
||||
try:
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
scan_instance = Scan.objects.get(pk=scan_id)
|
||||
provider_instance = scan_instance.provider
|
||||
prowler_provider = return_prowler_provider(provider_instance)
|
||||
|
||||
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
|
||||
provider_instance.provider
|
||||
]
|
||||
modeled_threatscore_compliance_id = "ProwlerThreatScore-1.0"
|
||||
threatscore_requirements_by_check: dict[str, set[str]] = {}
|
||||
threatscore_framework = compliance_template.get(
|
||||
modeled_threatscore_compliance_id
|
||||
)
|
||||
if threatscore_framework:
|
||||
for requirement_id, requirement in threatscore_framework[
|
||||
"requirements"
|
||||
].items():
|
||||
for check_id in requirement["checks"]:
|
||||
threatscore_requirements_by_check.setdefault(check_id, set()).add(
|
||||
requirement_id
|
||||
)
|
||||
|
||||
# Get check status data by region from findings
|
||||
findings = (
|
||||
Finding.all_objects.filter(scan_id=scan_id, muted=False)
|
||||
.only("id", "check_id", "status")
|
||||
.only("id", "check_id", "status", "compliance")
|
||||
.prefetch_related(
|
||||
Prefetch(
|
||||
"resources",
|
||||
@@ -601,14 +766,36 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
.iterator(chunk_size=1000)
|
||||
)
|
||||
|
||||
findings_count_by_compliance = {}
|
||||
check_status_by_region = {}
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
for finding in findings:
|
||||
for resource in finding.small_resources:
|
||||
region = resource.region
|
||||
current_status = check_status_by_region.setdefault(region, {})
|
||||
if current_status.get(finding.check_id) != "FAIL":
|
||||
current_status[finding.check_id] = finding.status
|
||||
if modeled_threatscore_compliance_id in finding.compliance:
|
||||
for requirement_id in finding.compliance[
|
||||
modeled_threatscore_compliance_id
|
||||
]:
|
||||
compliance_key = findings_count_by_compliance.setdefault(
|
||||
region, {}
|
||||
).setdefault(
|
||||
modeled_threatscore_compliance_id.lower().replace(
|
||||
"-", ""
|
||||
),
|
||||
{},
|
||||
)
|
||||
if requirement_id not in compliance_key:
|
||||
compliance_key[requirement_id] = {
|
||||
"total": 0,
|
||||
"pass": 0,
|
||||
}
|
||||
|
||||
compliance_key[requirement_id]["total"] += 1
|
||||
if finding.status == "PASS":
|
||||
compliance_key[requirement_id]["pass"] += 1
|
||||
|
||||
try:
|
||||
# Try to get regions from provider
|
||||
@@ -617,11 +804,6 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
# If not available, use regions from findings
|
||||
regions = set(check_status_by_region.keys())
|
||||
|
||||
# Get compliance template for the provider
|
||||
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
|
||||
provider_instance.provider
|
||||
]
|
||||
|
||||
# Create compliance data by region
|
||||
compliance_overview_by_region = {
|
||||
region: deepcopy(compliance_template) for region in regions
|
||||
@@ -640,36 +822,53 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
status,
|
||||
)
|
||||
|
||||
# Prepare compliance requirement objects
|
||||
compliance_requirement_objects = []
|
||||
# Prepare compliance requirement rows
|
||||
compliance_requirement_rows: list[dict[str, Any]] = []
|
||||
utc_datetime_now = datetime.now(tz=timezone.utc)
|
||||
for region, compliance_data in compliance_overview_by_region.items():
|
||||
for compliance_id, compliance in compliance_data.items():
|
||||
modeled_compliance_id = _normalized_compliance_key(
|
||||
compliance["framework"], compliance["version"]
|
||||
)
|
||||
# Create an overview record for each requirement within each compliance framework
|
||||
for requirement_id, requirement in compliance["requirements"].items():
|
||||
compliance_requirement_objects.append(
|
||||
ComplianceRequirementOverview(
|
||||
tenant_id=tenant_id,
|
||||
scan=scan_instance,
|
||||
region=region,
|
||||
compliance_id=compliance_id,
|
||||
framework=compliance["framework"],
|
||||
version=compliance["version"],
|
||||
requirement_id=requirement_id,
|
||||
description=requirement["description"],
|
||||
passed_checks=requirement["checks_status"]["pass"],
|
||||
failed_checks=requirement["checks_status"]["fail"],
|
||||
total_checks=requirement["checks_status"]["total"],
|
||||
requirement_status=requirement["status"],
|
||||
)
|
||||
checks_status = requirement["checks_status"]
|
||||
compliance_requirement_rows.append(
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": utc_datetime_now,
|
||||
"compliance_id": compliance_id,
|
||||
"framework": compliance["framework"],
|
||||
"version": compliance["version"] or "",
|
||||
"description": requirement.get("description") or "",
|
||||
"region": region,
|
||||
"requirement_id": requirement_id,
|
||||
"requirement_status": requirement["status"],
|
||||
"passed_checks": checks_status["pass"],
|
||||
"failed_checks": checks_status["fail"],
|
||||
"total_checks": checks_status["total"],
|
||||
"scan_id": scan_instance.id,
|
||||
"passed_findings": findings_count_by_compliance.get(
|
||||
region, {}
|
||||
)
|
||||
.get(modeled_compliance_id, {})
|
||||
.get(requirement_id, {})
|
||||
.get("pass", 0),
|
||||
"total_findings": findings_count_by_compliance.get(
|
||||
region, {}
|
||||
)
|
||||
.get(modeled_compliance_id, {})
|
||||
.get(requirement_id, {})
|
||||
.get("total", 0),
|
||||
}
|
||||
)
|
||||
|
||||
# Bulk create requirement records
|
||||
create_objects_in_batches(
|
||||
tenant_id, ComplianceRequirementOverview, compliance_requirement_objects
|
||||
)
|
||||
# Bulk create requirement records using PostgreSQL COPY
|
||||
_persist_compliance_requirement_rows(tenant_id, compliance_requirement_rows)
|
||||
|
||||
return {
|
||||
"requirements_created": len(compliance_requirement_objects),
|
||||
"requirements_created": len(compliance_requirement_rows),
|
||||
"regions_processed": list(regions),
|
||||
"compliance_frameworks": (
|
||||
list(compliance_overview_by_region.get(list(regions)[0], {}).keys())
|
||||
|
||||
@@ -34,6 +34,7 @@ from tasks.jobs.scan import (
|
||||
from tasks.utils import batched, get_next_execution_datetime
|
||||
|
||||
from api.compliance import get_compliance_frameworks
|
||||
from api.db_router import READ_REPLICA_ALIAS
|
||||
from api.db_utils import rls_transaction
|
||||
from api.decorators import set_tenant
|
||||
from api.models import Finding, Integration, Provider, Scan, ScanSummary, StateChoices
|
||||
@@ -343,70 +344,73 @@ def generate_outputs_task(scan_id: str, provider_id: str, tenant_id: str):
|
||||
.order_by("uid")
|
||||
.iterator()
|
||||
)
|
||||
for batch, is_last in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
|
||||
fos = [FindingOutput.transform_api_finding(f, prowler_provider) for f in batch]
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
for batch, is_last in batched(qs, DJANGO_FINDINGS_BATCH_SIZE):
|
||||
fos = [
|
||||
FindingOutput.transform_api_finding(f, prowler_provider) for f in batch
|
||||
]
|
||||
|
||||
# Outputs
|
||||
for mode, cfg in OUTPUT_FORMATS_MAPPING.items():
|
||||
# Skip ASFF generation if not needed
|
||||
if mode == "json-asff" and not generate_asff:
|
||||
continue
|
||||
# Outputs
|
||||
for mode, cfg in OUTPUT_FORMATS_MAPPING.items():
|
||||
# Skip ASFF generation if not needed
|
||||
if mode == "json-asff" and not generate_asff:
|
||||
continue
|
||||
|
||||
cls = cfg["class"]
|
||||
suffix = cfg["suffix"]
|
||||
extra = cfg.get("kwargs", {}).copy()
|
||||
if mode == "html":
|
||||
extra.update(provider=prowler_provider, stats=scan_summary)
|
||||
cls = cfg["class"]
|
||||
suffix = cfg["suffix"]
|
||||
extra = cfg.get("kwargs", {}).copy()
|
||||
if mode == "html":
|
||||
extra.update(provider=prowler_provider, stats=scan_summary)
|
||||
|
||||
writer, initialization = get_writer(
|
||||
output_writers,
|
||||
cls,
|
||||
lambda cls=cls, fos=fos, suffix=suffix: cls(
|
||||
findings=fos,
|
||||
file_path=out_dir,
|
||||
file_extension=suffix,
|
||||
from_cli=False,
|
||||
),
|
||||
is_last,
|
||||
)
|
||||
if not initialization:
|
||||
writer.transform(fos)
|
||||
writer.batch_write_data_to_file(**extra)
|
||||
writer._data.clear()
|
||||
writer, initialization = get_writer(
|
||||
output_writers,
|
||||
cls,
|
||||
lambda cls=cls, fos=fos, suffix=suffix: cls(
|
||||
findings=fos,
|
||||
file_path=out_dir,
|
||||
file_extension=suffix,
|
||||
from_cli=False,
|
||||
),
|
||||
is_last,
|
||||
)
|
||||
if not initialization:
|
||||
writer.transform(fos)
|
||||
writer.batch_write_data_to_file(**extra)
|
||||
writer._data.clear()
|
||||
|
||||
# Compliance CSVs
|
||||
for name in frameworks_avail:
|
||||
compliance_obj = frameworks_bulk[name]
|
||||
# Compliance CSVs
|
||||
for name in frameworks_avail:
|
||||
compliance_obj = frameworks_bulk[name]
|
||||
|
||||
klass = GenericCompliance
|
||||
for condition, cls in COMPLIANCE_CLASS_MAP.get(provider_type, []):
|
||||
if condition(name):
|
||||
klass = cls
|
||||
break
|
||||
klass = GenericCompliance
|
||||
for condition, cls in COMPLIANCE_CLASS_MAP.get(provider_type, []):
|
||||
if condition(name):
|
||||
klass = cls
|
||||
break
|
||||
|
||||
filename = f"{comp_dir}_{name}.csv"
|
||||
filename = f"{comp_dir}_{name}.csv"
|
||||
|
||||
writer, initialization = get_writer(
|
||||
compliance_writers,
|
||||
name,
|
||||
lambda klass=klass, fos=fos: klass(
|
||||
findings=fos,
|
||||
compliance=compliance_obj,
|
||||
file_path=filename,
|
||||
from_cli=False,
|
||||
),
|
||||
is_last,
|
||||
)
|
||||
if not initialization:
|
||||
writer.transform(fos, compliance_obj, name)
|
||||
writer.batch_write_data_to_file()
|
||||
writer._data.clear()
|
||||
writer, initialization = get_writer(
|
||||
compliance_writers,
|
||||
name,
|
||||
lambda klass=klass, fos=fos: klass(
|
||||
findings=fos,
|
||||
compliance=compliance_obj,
|
||||
file_path=filename,
|
||||
from_cli=False,
|
||||
),
|
||||
is_last,
|
||||
)
|
||||
if not initialization:
|
||||
writer.transform(fos, compliance_obj, name)
|
||||
writer.batch_write_data_to_file()
|
||||
writer._data.clear()
|
||||
|
||||
compressed = _compress_output_files(out_dir)
|
||||
upload_uri = _upload_to_s3(tenant_id, compressed, scan_id)
|
||||
|
||||
# S3 integrations (need output_directory)
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
s3_integrations = Integration.objects.filter(
|
||||
integrationproviderrelationship__provider_id=provider_id,
|
||||
integration_type=Integration.IntegrationChoices.AMAZON_S3,
|
||||
|
||||
@@ -1,17 +1,22 @@
|
||||
import csv
|
||||
import json
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timezone
|
||||
from io import StringIO
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from tasks.jobs.scan import (
|
||||
_copy_compliance_requirement_rows,
|
||||
_create_finding_delta,
|
||||
_persist_compliance_requirement_rows,
|
||||
_store_resources,
|
||||
create_compliance_requirements,
|
||||
perform_prowler_scan,
|
||||
)
|
||||
from tasks.utils import CustomEncoder
|
||||
|
||||
from api.db_router import MainRouter
|
||||
from api.exceptions import ProviderConnectionError
|
||||
from api.models import Finding, Provider, Resource, Scan, StateChoices, StatusChoices
|
||||
from prowler.lib.check.models import Severity
|
||||
@@ -1045,3 +1050,773 @@ class TestCreateComplianceRequirements:
|
||||
|
||||
assert "requirements_created" in result
|
||||
assert result["requirements_created"] >= 0
|
||||
|
||||
|
||||
class TestComplianceRequirementCopy:
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_streams_csv(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
captured = {}
|
||||
|
||||
def copy_side_effect(sql, file_obj):
|
||||
captured["sql"] = sql
|
||||
captured["data"] = file_obj.read()
|
||||
|
||||
cursor.copy_expert.side_effect = copy_side_effect
|
||||
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": None,
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
mock_psycopg_connection.assert_called_once_with("admin")
|
||||
connection.cursor.assert_called_once()
|
||||
cursor.execute.assert_called_once()
|
||||
cursor.copy_expert.assert_called_once()
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert csv_rows[0][0] == str(row["id"])
|
||||
assert csv_rows[0][5] == ""
|
||||
assert csv_rows[0][-1] == str(row["scan_id"])
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
"tasks.jobs.scan._copy_compliance_requirement_rows",
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
):
|
||||
inserted_at = datetime.now(timezone.utc)
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
tenant_id = row["tenant_id"]
|
||||
|
||||
ctx = MagicMock()
|
||||
ctx.__enter__.return_value = None
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
|
||||
mock_copy.assert_called_once_with(tenant_id, [row])
|
||||
mock_rls_transaction.assert_called_once_with(tenant_id)
|
||||
mock_bulk_create.assert_called_once()
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
objects = args[0]
|
||||
assert len(objects) == 1
|
||||
fallback = objects[0]
|
||||
assert fallback.version == row["version"]
|
||||
assert fallback.compliance_id == row["compliance_id"]
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
|
||||
def test_persist_compliance_requirement_rows_no_rows(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
):
|
||||
_persist_compliance_requirement_rows(str(uuid.uuid4()), [])
|
||||
|
||||
mock_copy.assert_not_called()
|
||||
mock_rls_transaction.assert_not_called()
|
||||
mock_bulk_create.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_multiple_rows(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test COPY with multiple rows to ensure batch processing works correctly."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
captured = {}
|
||||
|
||||
def copy_side_effect(sql, file_obj):
|
||||
captured["sql"] = sql
|
||||
captured["data"] = file_obj.read()
|
||||
|
||||
cursor.copy_expert.side_effect = copy_side_effect
|
||||
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = uuid.uuid4()
|
||||
inserted_at = datetime.now(timezone.utc)
|
||||
|
||||
rows = [
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": "First requirement",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 5,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 5,
|
||||
"scan_id": scan_id,
|
||||
},
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": "Second requirement",
|
||||
"region": "us-west-2",
|
||||
"requirement_id": "req-2",
|
||||
"requirement_status": "FAIL",
|
||||
"passed_checks": 3,
|
||||
"failed_checks": 2,
|
||||
"total_checks": 5,
|
||||
"scan_id": scan_id,
|
||||
},
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "aws_foundational_security_aws",
|
||||
"framework": "AWS-Foundational-Security-Best-Practices",
|
||||
"version": "2.0",
|
||||
"description": "Third requirement",
|
||||
"region": "eu-west-1",
|
||||
"requirement_id": "req-3",
|
||||
"requirement_status": "MANUAL",
|
||||
"passed_checks": 0,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 3,
|
||||
"scan_id": scan_id,
|
||||
},
|
||||
]
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(tenant_id, rows)
|
||||
|
||||
mock_psycopg_connection.assert_called_once_with("admin")
|
||||
connection.cursor.assert_called_once()
|
||||
cursor.execute.assert_called_once()
|
||||
cursor.copy_expert.assert_called_once()
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert len(csv_rows) == 3
|
||||
|
||||
# Validate first row
|
||||
assert csv_rows[0][0] == str(rows[0]["id"])
|
||||
assert csv_rows[0][1] == tenant_id
|
||||
assert csv_rows[0][3] == "cisa_aws"
|
||||
assert csv_rows[0][4] == "CISA"
|
||||
assert csv_rows[0][6] == "First requirement"
|
||||
assert csv_rows[0][7] == "us-east-1"
|
||||
assert csv_rows[0][10] == "5"
|
||||
assert csv_rows[0][11] == "0"
|
||||
assert csv_rows[0][12] == "5"
|
||||
|
||||
# Validate second row
|
||||
assert csv_rows[1][0] == str(rows[1]["id"])
|
||||
assert csv_rows[1][7] == "us-west-2"
|
||||
assert csv_rows[1][9] == "FAIL"
|
||||
assert csv_rows[1][10] == "3"
|
||||
assert csv_rows[1][11] == "2"
|
||||
|
||||
# Validate third row
|
||||
assert csv_rows[2][0] == str(rows[2]["id"])
|
||||
assert csv_rows[2][3] == "aws_foundational_security_aws"
|
||||
assert csv_rows[2][5] == "2.0"
|
||||
assert csv_rows[2][9] == "MANUAL"
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_null_values(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test COPY handles NULL/None values correctly in nullable fields."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
captured = {}
|
||||
|
||||
def copy_side_effect(sql, file_obj):
|
||||
captured["sql"] = sql
|
||||
captured["data"] = file_obj.read()
|
||||
|
||||
cursor.copy_expert.side_effect = copy_side_effect
|
||||
|
||||
# Row with all nullable fields set to None/empty
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "test_framework",
|
||||
"framework": "Test",
|
||||
"version": None, # nullable
|
||||
"description": None, # nullable
|
||||
"region": "",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 0,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 0,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert len(csv_rows) == 1
|
||||
|
||||
# Validate that None values are converted to empty strings in CSV
|
||||
assert csv_rows[0][5] == "" # version
|
||||
assert csv_rows[0][6] == "" # description
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_special_characters(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test COPY correctly escapes special characters in CSV."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
captured = {}
|
||||
|
||||
def copy_side_effect(sql, file_obj):
|
||||
captured["sql"] = sql
|
||||
captured["data"] = file_obj.read()
|
||||
|
||||
cursor.copy_expert.side_effect = copy_side_effect
|
||||
|
||||
# Row with special characters that need escaping
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": 'framework"with"quotes',
|
||||
"framework": "Framework,with,commas",
|
||||
"version": "1.0",
|
||||
"description": 'Description with "quotes", commas, and\nnewlines',
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
# Verify CSV was generated (csv module handles escaping automatically)
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert len(csv_rows) == 1
|
||||
|
||||
# Verify special characters are preserved after CSV parsing
|
||||
assert csv_rows[0][3] == 'framework"with"quotes'
|
||||
assert csv_rows[0][4] == "Framework,with,commas"
|
||||
assert "quotes" in csv_rows[0][6]
|
||||
assert "commas" in csv_rows[0][6]
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_missing_inserted_at(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test COPY uses current datetime when inserted_at is missing."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
captured = {}
|
||||
|
||||
def copy_side_effect(sql, file_obj):
|
||||
captured["sql"] = sql
|
||||
captured["data"] = file_obj.read()
|
||||
|
||||
cursor.copy_expert.side_effect = copy_side_effect
|
||||
|
||||
# Row without inserted_at field
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "test_framework",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
# Note: inserted_at is intentionally missing
|
||||
}
|
||||
|
||||
before_call = datetime.now(timezone.utc)
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
after_call = datetime.now(timezone.utc)
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert len(csv_rows) == 1
|
||||
|
||||
# Verify inserted_at was auto-generated and is a valid ISO datetime
|
||||
inserted_at_str = csv_rows[0][2]
|
||||
inserted_at = datetime.fromisoformat(inserted_at_str)
|
||||
assert before_call <= inserted_at <= after_call
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_transaction_rollback_on_copy_error(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test transaction is rolled back when copy_expert fails."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
# Simulate copy_expert failure
|
||||
cursor.copy_expert.side_effect = Exception("COPY command failed")
|
||||
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
with pytest.raises(Exception, match="COPY command failed"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
# Verify rollback was called
|
||||
connection.rollback.assert_called_once()
|
||||
connection.commit.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_transaction_rollback_on_set_config_error(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test transaction is rolled back when SET_CONFIG fails."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
# Simulate cursor.execute failure
|
||||
cursor.execute.side_effect = Exception("SET prowler.tenant_id failed")
|
||||
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
with pytest.raises(Exception, match="SET prowler.tenant_id failed"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
# Verify rollback was called
|
||||
connection.rollback.assert_called_once()
|
||||
connection.commit.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_commit_on_success(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Test transaction is committed on successful COPY."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
cursor.copy_expert.return_value = None # Success
|
||||
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": str(uuid.uuid4()),
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
|
||||
# Verify commit was called and rollback was not
|
||||
connection.commit.assert_called_once()
|
||||
connection.rollback.assert_not_called()
|
||||
# Verify autocommit was disabled
|
||||
assert connection.autocommit is False
|
||||
|
||||
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
|
||||
def test_persist_compliance_requirement_rows_success(self, mock_copy):
|
||||
"""Test successful COPY path without fallback to ORM."""
|
||||
mock_copy.return_value = None # Success, no exception
|
||||
|
||||
tenant_id = str(uuid.uuid4())
|
||||
rows = [
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": datetime.now(timezone.utc),
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
]
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, rows)
|
||||
|
||||
# Verify COPY was called
|
||||
mock_copy.assert_called_once_with(tenant_id, rows)
|
||||
|
||||
@patch("tasks.jobs.scan.logger")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
"tasks.jobs.scan._copy_compliance_requirement_rows",
|
||||
side_effect=Exception("COPY failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_logging(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_logger
|
||||
):
|
||||
"""Test logger.exception is called when COPY fails and fallback occurs."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": datetime.now(timezone.utc),
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
}
|
||||
|
||||
ctx = MagicMock()
|
||||
ctx.__enter__.return_value = None
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
|
||||
# Verify logger.exception was called
|
||||
mock_logger.exception.assert_called_once()
|
||||
args, kwargs = mock_logger.exception.call_args
|
||||
assert "COPY bulk insert" in args[0]
|
||||
assert "falling back to ORM" in args[0]
|
||||
assert kwargs.get("exc_info") is not None
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
"tasks.jobs.scan._copy_compliance_requirement_rows",
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_multiple_rows(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
):
|
||||
"""Test ORM fallback with multiple rows."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = uuid.uuid4()
|
||||
inserted_at = datetime.now(timezone.utc)
|
||||
|
||||
rows = [
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": "First requirement",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 5,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 5,
|
||||
"scan_id": scan_id,
|
||||
},
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": "Second requirement",
|
||||
"region": "us-west-2",
|
||||
"requirement_id": "req-2",
|
||||
"requirement_status": "FAIL",
|
||||
"passed_checks": 2,
|
||||
"failed_checks": 3,
|
||||
"total_checks": 5,
|
||||
"scan_id": scan_id,
|
||||
},
|
||||
]
|
||||
|
||||
ctx = MagicMock()
|
||||
ctx.__enter__.return_value = None
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, rows)
|
||||
|
||||
mock_copy.assert_called_once_with(tenant_id, rows)
|
||||
mock_rls_transaction.assert_called_once_with(tenant_id)
|
||||
mock_bulk_create.assert_called_once()
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
objects = args[0]
|
||||
assert len(objects) == 2
|
||||
assert kwargs["batch_size"] == 500
|
||||
|
||||
# Validate first object
|
||||
assert objects[0].id == rows[0]["id"]
|
||||
assert objects[0].tenant_id == rows[0]["tenant_id"]
|
||||
assert objects[0].compliance_id == rows[0]["compliance_id"]
|
||||
assert objects[0].framework == rows[0]["framework"]
|
||||
assert objects[0].region == rows[0]["region"]
|
||||
assert objects[0].passed_checks == 5
|
||||
assert objects[0].failed_checks == 0
|
||||
|
||||
# Validate second object
|
||||
assert objects[1].id == rows[1]["id"]
|
||||
assert objects[1].requirement_id == rows[1]["requirement_id"]
|
||||
assert objects[1].requirement_status == rows[1]["requirement_status"]
|
||||
assert objects[1].passed_checks == 2
|
||||
assert objects[1].failed_checks == 3
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
"tasks.jobs.scan._copy_compliance_requirement_rows",
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_all_fields(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
):
|
||||
"""Test ORM fallback correctly maps all fields from row dict to model."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
row_id = uuid.uuid4()
|
||||
scan_id = uuid.uuid4()
|
||||
inserted_at = datetime.now(timezone.utc)
|
||||
|
||||
row = {
|
||||
"id": row_id,
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "aws_foundational_security_aws",
|
||||
"framework": "AWS-Foundational-Security-Best-Practices",
|
||||
"version": "2.0",
|
||||
"description": "Ensure MFA is enabled",
|
||||
"region": "eu-west-1",
|
||||
"requirement_id": "iam.1",
|
||||
"requirement_status": "FAIL",
|
||||
"passed_checks": 10,
|
||||
"failed_checks": 5,
|
||||
"total_checks": 15,
|
||||
"scan_id": scan_id,
|
||||
}
|
||||
|
||||
ctx = MagicMock()
|
||||
ctx.__enter__.return_value = None
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
objects = args[0]
|
||||
assert len(objects) == 1
|
||||
|
||||
obj = objects[0]
|
||||
# Validate ALL fields are correctly mapped
|
||||
assert obj.id == row_id
|
||||
assert obj.tenant_id == tenant_id
|
||||
assert obj.inserted_at == inserted_at
|
||||
assert obj.compliance_id == "aws_foundational_security_aws"
|
||||
assert obj.framework == "AWS-Foundational-Security-Best-Practices"
|
||||
assert obj.version == "2.0"
|
||||
assert obj.description == "Ensure MFA is enabled"
|
||||
assert obj.region == "eu-west-1"
|
||||
assert obj.requirement_id == "iam.1"
|
||||
assert obj.requirement_status == "FAIL"
|
||||
assert obj.passed_checks == 10
|
||||
assert obj.failed_checks == 5
|
||||
assert obj.total_checks == 15
|
||||
assert obj.scan_id == scan_id
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_3_levels
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
data["REQUIREMENTS_DESCRIPTION"] = (
|
||||
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
|
||||
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION"
|
||||
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
|
||||
|
||||
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
|
||||
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
|
||||
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_DESCRIPTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"ACCOUNTID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
]
|
||||
|
||||
return get_section_containers_3_levels(
|
||||
aux,
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"REQUIREMENTS_DESCRIPTION",
|
||||
)
|
||||
@@ -0,0 +1,36 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_3_levels
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
|
||||
data["REQUIREMENTS_ID"] = (
|
||||
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
|
||||
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
|
||||
)
|
||||
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_ID",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"ACCOUNTID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
]
|
||||
|
||||
return get_section_containers_3_levels(
|
||||
aux,
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"REQUIREMENTS_ID",
|
||||
)
|
||||
@@ -0,0 +1,36 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_3_levels
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
|
||||
data["REQUIREMENTS_ID"] = (
|
||||
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
|
||||
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
|
||||
)
|
||||
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_ID",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"ACCOUNTID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
]
|
||||
|
||||
return get_section_containers_3_levels(
|
||||
aux,
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"REQUIREMENTS_ID",
|
||||
)
|
||||
@@ -0,0 +1,36 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_3_levels
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
|
||||
data["REQUIREMENTS_ID"] = (
|
||||
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
|
||||
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
|
||||
)
|
||||
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_ID",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"ACCOUNTID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
]
|
||||
|
||||
return get_section_containers_3_levels(
|
||||
aux,
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"REQUIREMENTS_ID",
|
||||
)
|
||||
@@ -0,0 +1,41 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_cis
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
"""
|
||||
Generate CIS OCI Foundations Benchmark v3.0 compliance table.
|
||||
|
||||
Args:
|
||||
data: DataFrame containing compliance check results with columns:
|
||||
- REQUIREMENTS_ID: CIS requirement ID (e.g., "1.1", "2.1")
|
||||
- REQUIREMENTS_DESCRIPTION: Description of the requirement
|
||||
- REQUIREMENTS_ATTRIBUTES_SECTION: CIS section name
|
||||
- CHECKID: Prowler check identifier
|
||||
- STATUS: Check status (PASS/FAIL)
|
||||
- REGION: OCI region
|
||||
- TENANCYID: OCI tenancy OCID
|
||||
- RESOURCEID: Resource OCID or identifier
|
||||
|
||||
Returns:
|
||||
Section containers organized by CIS sections for dashboard display
|
||||
"""
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_ID",
|
||||
"REQUIREMENTS_DESCRIPTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"TENANCYID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
].copy()
|
||||
|
||||
return get_section_containers_cis(
|
||||
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
|
||||
)
|
||||
@@ -0,0 +1,4 @@
|
||||
.idea/
|
||||
.git/
|
||||
.claude/
|
||||
AGENTS.md
|
||||
@@ -0,0 +1,45 @@
|
||||
# Prowler Documentation
|
||||
|
||||
This repository contains the Prowler Open Source documentation powered by [Mintlify](https://mintlify.com).
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
- **Getting Started**: Overview, installation, and basic usage guides
|
||||
- **User Guide**: Comprehensive guides for Prowler App, CLI, providers, and compliance
|
||||
- **Developer Guide**: Technical documentation for developers contributing to Prowler
|
||||
|
||||
## Local Development
|
||||
|
||||
Install the [Mintlify CLI](https://www.npmjs.com/package/mint) to preview documentation changes locally:
|
||||
|
||||
```bash
|
||||
npm i -g mint
|
||||
```
|
||||
|
||||
Run the following command at the root of your documentation (where `mint.json` is located):
|
||||
|
||||
```bash
|
||||
mint dev
|
||||
```
|
||||
|
||||
View your local preview at `http://localhost:3000`.
|
||||
|
||||
## Publishing Changes
|
||||
|
||||
Changes pushed to the main branch are automatically deployed to production through Mintlify's GitHub integration.
|
||||
|
||||
## Documentation Guidelines
|
||||
|
||||
When contributing to the documentation, please follow the Prowler documentation style guide located in the `.claude` directory.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- If your dev environment isn't running: Run `mint update` to ensure you have the most recent version of the CLI.
|
||||
- If a page loads as a 404: Make sure you are running in a folder with a valid `mint.json` file and that the page path is correctly listed in the navigation.
|
||||
|
||||
## Resources
|
||||
|
||||
- [Prowler GitHub Repository](https://github.com/prowler-cloud/prowler)
|
||||
- [Prowler Documentation](https://docs.prowler.com/)
|
||||
- [Mintlify Documentation](https://mintlify.com/docs)
|
||||
- [Mintlify Community](https://mintlify.com/community)
|
||||
@@ -1,61 +0,0 @@
|
||||
## Access Prowler App
|
||||
|
||||
After [installation](../installation/prowler-app.md), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
|
||||
|
||||
<img src="../../img/sign-up-button.png" alt="Sign Up Button" width="320"/>
|
||||
<img src="../../img/sign-up.png" alt="Sign Up" width="285"/>
|
||||
|
||||
???+ note "User creation and default tenant behavior"
|
||||
|
||||
When creating a new user, the behavior depends on whether an invitation is provided:
|
||||
|
||||
- **Without an invitation**:
|
||||
|
||||
- A new tenant is automatically created.
|
||||
- The new user is assigned to this tenant.
|
||||
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
|
||||
|
||||
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
|
||||
|
||||
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
|
||||
|
||||
## Log In
|
||||
|
||||
Access Prowler App by logging in with **email and password**.
|
||||
|
||||
<img src="../../img/log-in.png" alt="Log In" width="285"/>
|
||||
|
||||
## Add Cloud Provider
|
||||
|
||||
Configure a cloud provider for scanning:
|
||||
|
||||
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
|
||||
2. Select the cloud provider.
|
||||
3. Enter the provider's identifier (Optional: Add an alias):
|
||||
- **AWS**: Account ID
|
||||
- **GCP**: Project ID
|
||||
- **Azure**: Subscription ID
|
||||
- **Kubernetes**: Cluster ID
|
||||
- **M365**: Domain ID
|
||||
4. Follow the guided instructions to add and authenticate your credentials.
|
||||
|
||||
## Start a Scan
|
||||
|
||||
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
|
||||
|
||||
Click `Go to Scans` to monitor progress.
|
||||
|
||||
## View Results
|
||||
|
||||
Review findings during scan execution in the following sections:
|
||||
|
||||
- **Overview** – Provides a high-level summary of your scans.
|
||||
<img src="../../products/img/overview.png" alt="Overview" width="700"/>
|
||||
|
||||
- **Compliance** – Displays compliance insights based on security frameworks.
|
||||
<img src="../../img/compliance.png" alt="Compliance" width="700"/>
|
||||
|
||||
> For detailed usage instructions, refer to the [Prowler App Guide](../tutorials/prowler-app.md).
|
||||
|
||||
???+ note
|
||||
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
|
||||
@@ -1,4 +1,6 @@
|
||||
# Contact Us
|
||||
---
|
||||
title: 'Contact Us'
|
||||
---
|
||||
|
||||
For technical support or any type of inquiries, you are very welcome to:
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# AWS Provider
|
||||
---
|
||||
title: 'AWS Provider'
|
||||
---
|
||||
|
||||
In this page you can find all the details about [Amazon Web Services (AWS)](https://aws.amazon.com/) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](../tutorials/aws/getting-started-aws.md).
|
||||
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](/user-guide/providers/aws/getting-started-aws).
|
||||
|
||||
## AWS Provider Classes Architecture
|
||||
|
||||
The AWS provider implementation follows the general [Provider structure](./provider.md). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the AWS provider.
|
||||
The AWS provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the AWS provider.
|
||||
|
||||
### `AwsProvider` (Main Class)
|
||||
|
||||
@@ -33,7 +35,7 @@ The AWS provider implementation follows the general [Provider structure](./provi
|
||||
### `AWSService` (Service Base Class)
|
||||
|
||||
- **Location:** [`prowler/providers/aws/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/lib/service/service.py)
|
||||
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for AWS.
|
||||
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for AWS.
|
||||
- **Key AWS Responsibilities:**
|
||||
- Receives an `AwsProvider` instance to access session, identity, and configuration.
|
||||
- Manages clients for all services by regions.
|
||||
@@ -52,12 +54,12 @@ The AWS provider implementation follows the general [Provider structure](./provi
|
||||
|
||||
## Specific Patterns in AWS Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/aws/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
|
||||
|
||||
### AWS Service Common Patterns
|
||||
|
||||
@@ -74,12 +76,12 @@ The best reference to understand how to implement a new service is following the
|
||||
|
||||
## Specific Patterns in AWS Checks
|
||||
|
||||
The AWS checks pattern is described in [checks page](./checks.md). You can find all the right now implemented checks:
|
||||
The AWS checks pattern is described in [checks page](/developer-guide/checks). You can find all the right now implemented checks:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/aws/services/s3/s3_bucket_acl_prohibited/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services/s3/s3_bucket_acl_prohibited))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is following the [check creation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
|
||||
The best reference to understand how to implement a new check is following the [check creation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# Azure Provider
|
||||
---
|
||||
title: 'Azure Provider'
|
||||
---
|
||||
|
||||
In this page you can find all the details about [Microsoft Azure](https://azure.microsoft.com/) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](../tutorials/azure/getting-started-azure.md).
|
||||
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](/user-guide/providers/azure/getting-started-azure).
|
||||
|
||||
## Azure Provider Classes Architecture
|
||||
|
||||
The Azure provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the Azure provider.
|
||||
The Azure provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the Azure provider.
|
||||
|
||||
### `AzureProvider` (Main Class)
|
||||
|
||||
@@ -32,7 +34,7 @@ The Azure provider implementation follows the general [Provider structure](./pro
|
||||
### `AzureService` (Service Base Class)
|
||||
|
||||
- **Location:** [`prowler/providers/azure/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/lib/service/service.py)
|
||||
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Azure.
|
||||
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Azure.
|
||||
- **Key Azure Responsibilities:**
|
||||
- Receives an `AzureProvider` instance to access session, identity, and configuration.
|
||||
- Manages clients for all services by subscription.
|
||||
@@ -50,12 +52,12 @@ The Azure provider implementation follows the general [Provider structure](./pro
|
||||
|
||||
## Specific Patterns in Azure Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/azure/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
|
||||
|
||||
### Azure Service Common Patterns
|
||||
|
||||
@@ -68,12 +70,12 @@ The best reference to understand how to implement a new service is following the
|
||||
|
||||
## Specific Patterns in Azure Checks
|
||||
|
||||
The Azure checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
|
||||
The Azure checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is the [Azure check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
|
||||
The best reference to understand how to implement a new check is the [Azure check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
+4
-2
@@ -1,8 +1,10 @@
|
||||
# Check Metadata Guidelines
|
||||
---
|
||||
title: 'Check Metadata Guidelines'
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) section.
|
||||
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) section.
|
||||
|
||||
## Check Title Guidelines
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Prowler Checks
|
||||
---
|
||||
title: 'Prowler Checks'
|
||||
---
|
||||
|
||||
This guide explains how to create new checks in Prowler.
|
||||
|
||||
@@ -12,7 +14,7 @@ The most common high level steps to create a new check are:
|
||||
|
||||
1. Prerequisites:
|
||||
- Verify the check does not already exist by searching [Prowler Hub](https://hub.prowler.com) or checking `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>/`.
|
||||
- Ensure required provider and service exist. If not, follow the [Provider](./provider.md) and [Service](./services.md) documentation to create them.
|
||||
- Ensure required provider and service exist. If not, follow the [Provider](/developer-guide/provider) and [Service](/developer-guide/services) documentation to create them.
|
||||
- Confirm the service has implemented all required methods and attributes for the check (in most cases, you will need to add or modify some methods in the service to get the data you need for the check).
|
||||
2. Navigate to the service directory. The path should be as follows: `prowler/providers/<provider>/services/<service>`.
|
||||
3. Create a check-specific folder. The path should follow this pattern: `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>`. Adhere to the [Naming Format for Checks](#naming-format-for-checks).
|
||||
@@ -20,7 +22,7 @@ The most common high level steps to create a new check are:
|
||||
5. Run the check locally to ensure it works as expected. For checking you can use the CLI in the next way:
|
||||
- To ensure the check has been detected by Prowler: `poetry run python prowler-cli.py <provider> --list-checks | grep <check_name>`.
|
||||
- To run the check, to find possible issues: `poetry run python prowler-cli.py <provider> --log-level ERROR --verbose --check <check_name>`.
|
||||
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](./unit-testing.md) documentation.
|
||||
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](/developer-guide/unit-testing) documentation.
|
||||
7. If the check and its corresponding tests are working as expected, you can submit a PR to Prowler.
|
||||
|
||||
### Naming Format for Checks
|
||||
@@ -39,8 +41,8 @@ The name components are:
|
||||
Each check in Prowler follows a straightforward structure. Within the newly created folder, three files must be added to implement the check logic:
|
||||
|
||||
- `__init__.py` (empty file) – Ensures Python treats the check folder as a package.
|
||||
- `<check_name>.py` (code file) – Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](./checks.md#prowlers-check-code-structure) for more information.
|
||||
- `<check_name>.metadata.json` (metadata file) – Defines the check's metadata for contextual information. Please refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) for more information.
|
||||
- `<check_name>.py` (code file) – Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](/developer-guide/checks#prowlers-check-code-structure) for more information.
|
||||
- `<check_name>.metadata.json` (metadata file) – Defines the check's metadata for contextual information. Please refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) for more information.
|
||||
|
||||
## Prowler's Check Code Structure
|
||||
|
||||
@@ -50,9 +52,10 @@ Below the code for a generic check is presented. It is strongly recommended to c
|
||||
|
||||
Report fields are the most dependent on the provider, consult the `CheckReport<Provider>` class for more information on what can be included in the report [here](https://github.com/prowler-cloud/prowler/blob/master/prowler/lib/check/models.py).
|
||||
|
||||
???+ note
|
||||
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
|
||||
<Note>
|
||||
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
|
||||
|
||||
</Note>
|
||||
```python title="Generic Check Class"
|
||||
# Required Imports
|
||||
# Import the base Check class and the provider-specific CheckReport class
|
||||
@@ -213,7 +216,7 @@ Each check **must** populate the report with an unique identifier for the audite
|
||||
|
||||
### Configurable Checks in Prowler
|
||||
|
||||
See [Configurable Checks](./configurable-checks.md) for detailed information on making checks configurable using the `audit_config` object and configuration file.
|
||||
See [Configurable Checks](/developer-guide/configurable-checks) for detailed information on making checks configurable using the `audit_config` object and configuration file.
|
||||
|
||||
## Metadata Structure for Prowler Checks
|
||||
|
||||
@@ -273,16 +276,17 @@ The `CheckTitle` field must be plain text, clearly and succinctly define **the b
|
||||
|
||||
**Always write the `CheckTitle` to describe the *PASS* case**, the desired secure or compliant state of the resource(s). This helps ensure that findings are easy to interpret and that the title always reflects the best practice being met.
|
||||
|
||||
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](./check-metadata-guidelines.md#check-title-guidelines).
|
||||
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](/developer-guide/check-metadata-guidelines#check-title-guidelines).
|
||||
|
||||
#### CheckType
|
||||
|
||||
???+ warning
|
||||
This field is only applicable to the AWS provider.
|
||||
<Warning>
|
||||
This field is only applicable to the AWS provider.
|
||||
|
||||
</Warning>
|
||||
It follows the [AWS Security Hub Types](https://docs.aws.amazon.com/securityhub/latest/userguide/asff-required-attributes.html#Types) format using the pattern `namespace/category/classifier`.
|
||||
|
||||
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](./check-metadata-guidelines.md#check-type-guidelines-aws-only).
|
||||
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](/developer-guide/check-metadata-guidelines#check-type-guidelines-aws-only).
|
||||
|
||||
#### ServiceName
|
||||
|
||||
@@ -314,13 +318,13 @@ The type of resource being audited. This field helps categorize and organize fin
|
||||
|
||||
A concise, natural language explanation that **clearly describes what the finding means**, focusing on clarity and context rather than technical implementation details. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. This field is limited to maximum 400 characters.
|
||||
|
||||
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](./check-metadata-guidelines.md#description-guidelines).
|
||||
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](/developer-guide/check-metadata-guidelines#description-guidelines).
|
||||
|
||||
#### Risk
|
||||
|
||||
A clear, natural language explanation of **why this finding poses a cybersecurity risk**. Focus on how it may impact confidentiality, integrity, or availability. If those do not apply, describe any relevant operational or financial risks. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. Limit your explanation to 400 characters.
|
||||
|
||||
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](./check-metadata-guidelines.md#risk-guidelines).
|
||||
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](/developer-guide/check-metadata-guidelines#risk-guidelines).
|
||||
|
||||
#### RelatedUrl
|
||||
|
||||
@@ -328,9 +332,10 @@ For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelin
|
||||
|
||||
#### AdditionalURLs
|
||||
|
||||
???+ warning
|
||||
URLs must be valid and not repeated.
|
||||
<Warning>
|
||||
URLs must be valid and not repeated.
|
||||
|
||||
</Warning>
|
||||
A list of official documentation URLs for further reading. These should be authoritative sources that provide additional context, best practices, or detailed information about the security control being checked. Prefer official provider documentation, security standards, or well-established security resources. Avoid third-party blogs or unofficial sources unless they are highly reputable and directly relevant.
|
||||
|
||||
#### Remediation
|
||||
@@ -345,17 +350,17 @@ Provides both code examples and best practice recommendations for addressing the
|
||||
- **Terraform**: HashiCorp Configuration Language (HCL) code with an example of a compliant configuration.
|
||||
- **Other**: Manual steps through web interfaces or other tools to make the finding compliant.
|
||||
|
||||
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](./check-metadata-guidelines.md#remediation-code-guidelines).
|
||||
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](/developer-guide/check-metadata-guidelines#remediation-code-guidelines).
|
||||
|
||||
- **Recommendation**
|
||||
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](./check-metadata-guidelines.md#recommendation-guidelines).
|
||||
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](/developer-guide/check-metadata-guidelines#recommendation-guidelines).
|
||||
- **Url**: [Prowler Hub URL](https://hub.prowler.com/) of the check. This URL is always composed by `https://hub.prowler.com/check/<check_id>`.
|
||||
|
||||
#### Categories
|
||||
|
||||
One or more functional groupings used for execution filtering (e.g., `internet-exposed`). You can define new categories just by adding to this field.
|
||||
|
||||
For the complete list of available categories, see [Categories Guidelines](./check-metadata-guidelines.md#categories-guidelines).
|
||||
For the complete list of available categories, see [Categories Guidelines](/developer-guide/check-metadata-guidelines#categories-guidelines).
|
||||
|
||||
#### DependsOn
|
||||
|
||||
+4
-2
@@ -1,4 +1,6 @@
|
||||
# Configurable Checks in Prowler
|
||||
---
|
||||
title: 'Configurable Checks in Prowler'
|
||||
---
|
||||
|
||||
Prowler empowers users to extend and adapt cloud security coverage by making checks configurable through the use of the `audit_config` object. This approach enables customization of checks to meet specific requirements through a configuration file.
|
||||
|
||||
@@ -41,6 +43,6 @@ When adding a new configurable check to Prowler, update the following files:
|
||||
- **Test Fixtures:** If tests depend on this configuration, add the variable to `tests/config/fixtures/config.yaml`.
|
||||
- **Documentation:** Document the new variable in the list of configurable checks in `docs/tutorials/configuration_file.md`.
|
||||
|
||||
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](../tutorials/configuration_file.md).
|
||||
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](/user-guide/cli/tutorials/configuration_file).
|
||||
|
||||
This approach ensures that checks are easily configurable, making Prowler highly adaptable to different environments and requirements.
|
||||
@@ -1,4 +1,6 @@
|
||||
# Debugging in Prowler
|
||||
---
|
||||
title: 'Debugging in Prowler'
|
||||
---
|
||||
|
||||
Debugging in Prowler simplifies the development process, allowing developers to efficiently inspect and resolve unexpected issues during execution.
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
## Contributing to Documentation
|
||||
|
||||
Prowler documentation is built using `mkdocs`, allowing contributors to easily add or enhance documentation.
|
||||
|
||||
### Installation and Setup
|
||||
|
||||
Install all necessary dependencies using: `poetry install --with docs`.
|
||||
|
||||
1. Install `mkdocs` using your preferred package manager.
|
||||
|
||||
2. Running the Documentation Locally
|
||||
Navigate to the `prowler` repository folder.
|
||||
Start the local documentation server by running: `mkdocs serve`.
|
||||
Open `http://localhost:8000` in your browser to view live updates.
|
||||
|
||||
3. Making Documentation Changes
|
||||
Make all needed changes to docs or add new documents. Edit existing Markdown (.md) files inside `prowler/docs`.
|
||||
To add new sections or files, update the `mkdocs.yaml` file located in the root directory of Prowler’s repository.
|
||||
|
||||
4. Submitting Changes
|
||||
|
||||
Once documentation updates are complete:
|
||||
|
||||
Submit a pull request for review.
|
||||
|
||||
The Prowler team will assess and merge contributions.
|
||||
|
||||
Your efforts help improve Prowler documentation—thank you for contributing!
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: 'Contributing to Documentation'
|
||||
---
|
||||
|
||||
Prowler documentation is built using [Mintlify](https://www.mintlify.com/docs), allowing contributors to easily add or enhance documentation.
|
||||
|
||||
## Installation and Setup
|
||||
|
||||
<Steps>
|
||||
<Step title="Install Mintlify CLI">
|
||||
```bash
|
||||
npm i -g mint
|
||||
```
|
||||
For detailed instructions, check the [Mintlify documentation](https://www.mintlify.com/docs/installation).
|
||||
</Step>
|
||||
|
||||
<Step title="Preview Documentation Locally">
|
||||
Start the local development server to preview changes in real-time.
|
||||
|
||||
```bash
|
||||
mint dev
|
||||
```
|
||||
|
||||
A local preview of your documentation will be available at http://localhost:3000
|
||||
</Step>
|
||||
|
||||
<Step title="Make Documentation Changes">
|
||||
Edit existing Markdown (.mdx) files inside the `docs` directory or add new documents.
|
||||
|
||||
For reference about formatting, check the [Mintlify documentation](https://www.mintlify.com/docs/create/text).
|
||||
|
||||
To add new sections or files, update the [`docs/docs.json`](https://github.com/prowler-cloud/prowler/blob/master/docs/docs.json) file to include them in the navigation.
|
||||
</Step>
|
||||
|
||||
<Step title="Submit Changes">
|
||||
Once documentation updates are complete, submit a pull request for review.
|
||||
|
||||
The Prowler team will assess and merge contributions.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
Your efforts help improve Prowler documentation—thank you for contributing!
|
||||
@@ -1,12 +1,14 @@
|
||||
# Google Cloud Provider
|
||||
---
|
||||
title: 'Google Cloud Provider'
|
||||
---
|
||||
|
||||
This page details the [Google Cloud Platform (GCP)](https://cloud.google.com/) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](../tutorials/gcp/getting-started-gcp.md).
|
||||
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](/user-guide/providers/gcp/getting-started-gcp).
|
||||
|
||||
## GCP Provider Classes Architecture
|
||||
|
||||
The GCP provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
|
||||
The GCP provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
|
||||
|
||||
### Main Class
|
||||
|
||||
@@ -32,7 +34,7 @@ The GCP provider implementation follows the general [Provider structure](./provi
|
||||
### `GCPService` (Service Base Class)
|
||||
|
||||
- **Location:** [`prowler/providers/gcp/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/lib/service/service.py)
|
||||
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for GCP.
|
||||
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for GCP.
|
||||
- **Key GCP Responsibilities:**
|
||||
- Receives a `GcpProvider` instance to access session, identity, and configuration.
|
||||
- Manages clients for all services by project.
|
||||
@@ -95,12 +97,12 @@ def _get_instances(self):
|
||||
|
||||
## Specific Patterns in GCP Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/gcp/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
|
||||
|
||||
### GCP Service Common Patterns
|
||||
|
||||
@@ -117,12 +119,12 @@ The best reference to understand how to implement a new service is following the
|
||||
|
||||
## Specific Patterns in GCP Checks
|
||||
|
||||
The GCP checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
|
||||
The GCP checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
|
||||
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# GitHub Provider
|
||||
---
|
||||
title: 'GitHub Provider'
|
||||
---
|
||||
|
||||
This page details the [GitHub](https://github.com/) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](../tutorials/github/getting-started-github.md).
|
||||
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](/user-guide/providers/github/getting-started-github).
|
||||
|
||||
## GitHub Provider Classes Architecture
|
||||
|
||||
The GitHub provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
|
||||
The GitHub provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
|
||||
|
||||
### `GithubProvider` (Main Class)
|
||||
|
||||
@@ -48,12 +50,12 @@ The GitHub provider implementation follows the general [Provider structure](./pr
|
||||
|
||||
## Specific Patterns in GitHub Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/github/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
|
||||
|
||||
### GitHub Service Common Patterns
|
||||
|
||||
@@ -66,12 +68,12 @@ The best reference to understand how to implement a new service is following the
|
||||
|
||||
## Specific Patterns in GitHub Checks
|
||||
|
||||
The GitHub checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
|
||||
The GitHub checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/github/services/repository/repository_secret_scanning_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services/repository/repository_secret_scanning_enabled))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
|
||||
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
# Integration Tests
|
||||
|
||||
Coming soon ...
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: 'Integration Tests'
|
||||
---
|
||||
|
||||
Coming soon ...
|
||||
@@ -1,4 +1,6 @@
|
||||
# Creating a New Integration
|
||||
---
|
||||
title: 'Creating a New Integration'
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
@@ -151,7 +153,7 @@ Refer to the [Prowler Developer Guide](https://docs.prowler.com/projects/prowler
|
||||
|
||||
# More properties and methods
|
||||
```
|
||||
|
||||
|
||||
* Test Connection Method:
|
||||
|
||||
* Validating Credentials or Tokens
|
||||
@@ -1,4 +1,6 @@
|
||||
# Introduction to developing in Prowler
|
||||
---
|
||||
title: 'Introduction to developing in Prowler'
|
||||
---
|
||||
|
||||
Extending Prowler
|
||||
|
||||
@@ -48,11 +50,12 @@ poetry install --with dev
|
||||
eval $(poetry env activate)
|
||||
```
|
||||
|
||||
???+ important
|
||||
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
|
||||
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
|
||||
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
|
||||
<Warning>
|
||||
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
|
||||
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
|
||||
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
|
||||
|
||||
</Warning>
|
||||
## Contributing to Prowler
|
||||
|
||||
### Ways to Contribute
|
||||
@@ -64,24 +67,24 @@ Here are some ideas for collaborating with Prowler:
|
||||
2. **Expand Prowler's Capabilities**: Prowler is constantly evolving, and you can be a part of its growth. Whether you are adding checks, supporting new services, or introducing integrations, your contributions help improve the tool for everyone. Here is how you can get involved:
|
||||
|
||||
- **Adding New Checks**
|
||||
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](./checks.md).
|
||||
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](/developer-guide/checks).
|
||||
|
||||
- **Adding New Services**
|
||||
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](./services.md). Do not forget to include relevant checks to validate functionality.
|
||||
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](/developer-guide/services). Do not forget to include relevant checks to validate functionality.
|
||||
|
||||
- **Adding New Providers**
|
||||
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](./provider.md). This typically involves setting up new services and checks to ensure compatibility.
|
||||
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](/developer-guide/provider). This typically involves setting up new services and checks to ensure compatibility.
|
||||
|
||||
- **Adding New Output Formats**
|
||||
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](./outputs.md).
|
||||
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](/developer-guide/outputs).
|
||||
|
||||
- **Adding New Integrations**
|
||||
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](./integrations.md).
|
||||
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](/developer-guide/integrations).
|
||||
|
||||
- **Proposing or Implementing Features**
|
||||
Got an idea to make Prowler better? Whether it is a brand-new feature or an enhancement to an existing one, you are welcome to propose it or help implement community-requested improvements.
|
||||
|
||||
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](./documentation.md).
|
||||
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](/developer-guide/documentation).
|
||||
|
||||
4. **Bug Fixes**: If you find any issues or bugs, you can report them in the [GitHub Issues](https://github.com/prowler-cloud/prowler/issues) page and if you want you can also fix them.
|
||||
|
||||
@@ -105,9 +108,10 @@ pre-commit installed at .git/hooks/pre-commit
|
||||
|
||||
Before merging pull requests, several automated checks and utilities ensure code security and updated dependencies:
|
||||
|
||||
???+ note
|
||||
These should have been already installed if `poetry install --with dev` was already run.
|
||||
<Note>
|
||||
These should have been already installed if `poetry install --with dev` was already run.
|
||||
|
||||
</Note>
|
||||
- [`bandit`](https://pypi.org/project/bandit/) for code security review.
|
||||
- [`safety`](https://pypi.org/project/safety/) and [`dependabot`](https://github.com/features/security) for dependencies.
|
||||
- [`hadolint`](https://github.com/hadolint/hadolint) and [`dockle`](https://github.com/goodwithtech/dockle) for container security.
|
||||
@@ -123,9 +127,10 @@ All dependencies are listed in the `pyproject.toml` file.
|
||||
|
||||
For proper code documentation, refer to the following and follow the code documentation practices presented there: [Google Python Style Guide - Comments and Docstrings](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings).
|
||||
|
||||
???+ note
|
||||
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
|
||||
<Note>
|
||||
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
|
||||
|
||||
</Note>
|
||||
### Repository Folder Structure
|
||||
|
||||
Understanding the layout of the Prowler codebase will help you quickly find where to add new features, checks, or integrations. The following is a high-level overview from the root of the repository:
|
||||
@@ -173,4 +178,4 @@ To test Prowler from a specific branch (for example, to try out changes from a p
|
||||
pipx install "git+https://github.com/prowler-cloud/prowler.git@branch-name"
|
||||
```
|
||||
|
||||
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
|
||||
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
|
||||
+10
-8
@@ -1,12 +1,14 @@
|
||||
# Kubernetes Provider
|
||||
---
|
||||
title: 'Kubernetes Provider'
|
||||
---
|
||||
|
||||
This page details the [Kubernetes](https://kubernetes.io/) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](../tutorials/kubernetes/in-cluster.md) or [Non In-Cluster Execution](../tutorials/kubernetes/outside-cluster.md) guides.
|
||||
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](/user-guide/providers/kubernetes/in-cluster) or [Non In-Cluster Execution](/user-guide/providers/kubernetes/outside-cluster) guides.
|
||||
|
||||
## Kubernetes Provider Classes Architecture
|
||||
|
||||
The Kubernetes provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
|
||||
The Kubernetes provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
|
||||
|
||||
### `KubernetesProvider` (Main Class)
|
||||
|
||||
@@ -31,7 +33,7 @@ The Kubernetes provider implementation follows the general [Provider structure](
|
||||
### `KubernetesService` (Service Base Class)
|
||||
|
||||
- **Location:** [`prowler/providers/kubernetes/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/lib/service/service.py)
|
||||
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Kubernetes.
|
||||
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Kubernetes.
|
||||
- **Key Kubernetes Responsibilities:**
|
||||
- Receives a `KubernetesProvider` instance to access session, identity, and configuration.
|
||||
- Manages the Kubernetes API client and context.
|
||||
@@ -50,12 +52,12 @@ The Kubernetes provider implementation follows the general [Provider structure](
|
||||
|
||||
## Specific Patterns in Kubernetes Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/kubernetes/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other already implemented services as reference.
|
||||
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other already implemented services as reference.
|
||||
|
||||
### Kubernetes Service Common Patterns
|
||||
|
||||
@@ -69,12 +71,12 @@ The best reference to understand how to implement a new service is following the
|
||||
|
||||
## Specific Patterns in Kubernetes Checks
|
||||
|
||||
The Kubernetes checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
|
||||
The Kubernetes checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](./checks.md#creating-a-check) and taking other checks as reference.
|
||||
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](/developer-guide/checks#creating-a-check) and taking other checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Extending Prowler Lighthouse AI
|
||||
---
|
||||
title: 'Extending Prowler Lighthouse AI'
|
||||
---
|
||||
|
||||
This guide helps developers customize and extend Prowler Lighthouse AI by adding or modifying AI agents.
|
||||
|
||||
@@ -15,9 +17,10 @@ AI agents fall into two main categories:
|
||||
|
||||
Prowler Lighthouse AI is an autonomous agent - selecting the right tool(s) based on the users query.
|
||||
|
||||
???+ note
|
||||
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
|
||||
<Note>
|
||||
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
|
||||
|
||||
</Note>
|
||||
### LLM Dependency
|
||||
|
||||
The autonomous nature of agents depends on the underlying LLM. Autonomous agents using identical system prompts and tools but powered by different LLM providers might approach user queries differently. Agent with one LLM might solve a problem efficiently, while with another it might take a different route or fail entirely.
|
||||
@@ -30,7 +33,7 @@ Prowler Lighthouse AI uses a multi-agent architecture orchestrated by the [Langg
|
||||
|
||||
### Architecture Components
|
||||
|
||||
<img src="../../tutorials/img/lighthouse-architecture.png" alt="Prowler Lighthouse architecture">
|
||||
<img src="/images/prowler-app/lighthouse-architecture.png" alt="Prowler Lighthouse architecture" />
|
||||
|
||||
Prowler Lighthouse AI integrates with the NextJS application:
|
||||
|
||||
@@ -67,9 +70,10 @@ Modifying the supervisor prompt allows you to:
|
||||
- Modify task delegation to specialized agents
|
||||
- Set up guardrails (query types to answer or decline)
|
||||
|
||||
???+ note
|
||||
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
|
||||
<Note>
|
||||
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
|
||||
|
||||
</Note>
|
||||
### How to Create New Specialized Agents
|
||||
|
||||
The supervisor agent and all specialized agents are defined in the `route.ts` file. The supervisor agent uses [langgraph-supervisor](https://www.npmjs.com/package/@langchain/langgraph-supervisor), while other agents use the prebuilt [create-react-agent](https://langchain-ai.github.io/langgraphjs/how-tos/create-react-agent/).
|
||||
@@ -77,14 +81,16 @@ The supervisor agent and all specialized agents are defined in the `route.ts` fi
|
||||
To add new capabilities or all Lighthouse AI to interact with other APIs, create additional specialized agents:
|
||||
|
||||
1. First determine what the new agent would do. Create a detailed prompt defining the agent's purpose and capabilities. You can see an example from [here](https://github.com/prowler-cloud/prowler/blob/master/ui/lib/lighthouse/prompts.ts#L359-L385).
|
||||
???+ note
|
||||
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
|
||||
<Note>
|
||||
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
|
||||
|
||||
</Note>
|
||||
2. Create necessary tools for the agents to access specific data or perform actions. A tool is a specialized function that extends the capabilities of LLM by allowing it to access external data or APIs. A tool is triggered by LLM based on the description of the tool and the user's query.
|
||||
For example, the description of `getScanTool` is "Fetches detailed information about a specific scan by its ID." If the description doesn't convey what the tool is capable of doing, LLM will not invoke the function. If the description of `getScanTool` was set to something random or not set at all, LLM will not answer queries like "Give me the critical issues from the scan ID xxxxxxxxxxxxxxx"
|
||||
???+ note
|
||||
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
|
||||
<Note>
|
||||
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
|
||||
|
||||
</Note>
|
||||
3. Use the `createReactAgent` function to define a new agent. For example, the rolesAgent name is "roles_agent" and has access to call tools "*getRolesTool*" and "*getRoleTool*"
|
||||
```js
|
||||
const rolesAgent = createReactAgent({
|
||||
@@ -1,12 +1,14 @@
|
||||
# LLM Provider
|
||||
---
|
||||
title: 'LLM Provider'
|
||||
---
|
||||
|
||||
This page details the [Large Language Model (LLM)](https://en.wikipedia.org/wiki/Large_language_model) provider implementation in Prowler.
|
||||
|
||||
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](../tutorials/llm/getting-started-llm.md).
|
||||
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](/user-guide/providers/llm/getting-started-llm).
|
||||
|
||||
## LLM Provider Classes Architecture
|
||||
|
||||
The LLM provider implementation follows the general [Provider structure](./provider.md). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
|
||||
The LLM provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
|
||||
|
||||
### Main Class
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
# Microsoft 365 (M365) Provider
|
||||
---
|
||||
title: 'Microsoft 365 (M365) Provider'
|
||||
---
|
||||
|
||||
This page details the [Microsoft 365 (M365)](https://www.microsoft.com/en-us/microsoft-365) provider implementation in Prowler.
|
||||
|
||||
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](../tutorials/microsoft365/getting-started-m365.md).
|
||||
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](/user-guide/providers/microsoft365/getting-started-m365).
|
||||
|
||||
---
|
||||
|
||||
@@ -15,14 +17,14 @@ By default, Prowler will audit the Microsoft Entra ID tenant and its supported s
|
||||
- **Required modules:**
|
||||
- [ExchangeOnlineManagement](https://www.powershellgallery.com/packages/ExchangeOnlineManagement/3.6.0) (≥ 3.6.0)
|
||||
- [MicrosoftTeams](https://www.powershellgallery.com/packages/MicrosoftTeams/6.6.0) (≥ 6.6.0)
|
||||
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](../tutorials/microsoft365/authentication.md#supported-powershell-versions) and [Needed PowerShell Modules](../tutorials/microsoft365/authentication.md#required-powershell-modules).
|
||||
- For more details and troubleshooting, see [Use of PowerShell in M365](../tutorials/microsoft365/use-of-powershell.md).
|
||||
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](/user-guide/providers/microsoft365/authentication#supported-powershell-versions) and [Needed PowerShell Modules](/user-guide/providers/microsoft365/authentication#required-powershell-modules).
|
||||
- For more details and troubleshooting, see [Use of PowerShell in M365](/user-guide/providers/microsoft365/use-of-powershell).
|
||||
|
||||
---
|
||||
|
||||
## M365 Provider Classes Architecture
|
||||
|
||||
The M365 provider implementation follows the general [Provider structure](./provider.md). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
|
||||
The M365 provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
|
||||
|
||||
### `M365Provider` (Main Class)
|
||||
|
||||
@@ -73,12 +75,12 @@ The M365 provider implementation follows the general [Provider structure](./prov
|
||||
|
||||
## Specific Patterns in M365 Services
|
||||
|
||||
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
|
||||
|
||||
- Directly in the code, in location [`prowler/providers/m365/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services)
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new service is by following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
|
||||
The best reference to understand how to implement a new service is by following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
|
||||
|
||||
### M365 Service Common Patterns
|
||||
|
||||
@@ -92,12 +94,12 @@ The best reference to understand how to implement a new service is by following
|
||||
|
||||
## Specific Patterns in M365 Checks
|
||||
|
||||
The M365 checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
|
||||
The M365 checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
|
||||
|
||||
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/m365/services/entra/entra_users_mfa_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services/entra/entra_users_mfa_enabled))
|
||||
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
|
||||
|
||||
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
|
||||
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
|
||||
|
||||
### Check Report Class
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Create a Custom Output Format
|
||||
---
|
||||
title: 'Create a Custom Output Format'
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Prowler Providers
|
||||
---
|
||||
title: 'Prowler Providers'
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
@@ -14,9 +16,10 @@ A provider is any platform or service that offers resources, data, or functional
|
||||
|
||||
For providers supported by Prowler, refer to [Prowler Hub](https://hub.prowler.com/).
|
||||
|
||||
???+ important
|
||||
There are some custom providers added by the community, like [NHN Cloud](https://www.nhncloud.com/), that are not maintained by the Prowler team, but can be used in the Prowler CLI. They can be checked directly at the [Prowler GitHub repository](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
|
||||
<Warning>
|
||||
There are some custom providers added by the community, like [NHN Cloud](https://www.nhncloud.com/), that are not maintained by the Prowler team, but can be used in the Prowler CLI. They can be checked directly at the [Prowler GitHub repository](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
|
||||
|
||||
</Warning>
|
||||
## Adding a New Provider
|
||||
|
||||
To integrate an unsupported Prowler provider and implement its security checks, create a dedicated folder for all related files (e.g., services, checks)."
|
||||
@@ -31,7 +34,7 @@ Within this folder the following folders are also to be created:
|
||||
- `arguments/arguments.py` – Handles provider-specific argument parsing.
|
||||
- `mutelist/mutelist.py` – Manages the mutelist functionality for the provider.
|
||||
|
||||
- `services` – Stores all [services](./services.md) that the provider offers and want to be audited by [Prowler checks](./checks.md).
|
||||
- `services` – Stores all [services](/developer-guide/services) that the provider offers and want to be audited by [Prowler checks](/developer-guide/checks).
|
||||
|
||||
- `__init__.py` (empty) – Ensures Python recognizes this folder as a package.
|
||||
|
||||
@@ -41,9 +44,10 @@ Within this folder the following folders are also to be created:
|
||||
|
||||
By adhering to this structure, Prowler can effectively support services and security checks for additional providers.
|
||||
|
||||
???+ important
|
||||
If your new provider requires a Python library (such as an official SDK or API client) to connect to its services, make sure to add it as a dependency in the `pyproject.toml` file. This ensures that all contributors and users have the necessary packages installed when working with your provider.
|
||||
<Warning>
|
||||
If your new provider requires a Python library (such as an official SDK or API client) to connect to its services, make sure to add it as a dependency in the `pyproject.toml` file. This ensures that all contributors and users have the necessary packages installed when working with your provider.
|
||||
|
||||
</Warning>
|
||||
## Provider Structure in Prowler
|
||||
|
||||
Prowler's provider architecture is designed to facilitate security audits through a generic service tailored to each provider. This is accomplished by passing the necessary parameters to the constructor, which initializes all required session values.
|
||||
@@ -1,4 +1,6 @@
|
||||
# Renaming Checks in Prowler
|
||||
---
|
||||
title: 'Renaming Checks in Prowler'
|
||||
---
|
||||
|
||||
To rename a check in Prowler, follow these steps when aligning with Check ID structure, fixing typos, or updating check logic that requires a new name.
|
||||
|
||||
+3
-1
@@ -1,4 +1,6 @@
|
||||
# Creating a New Security Compliance Framework in Prowler
|
||||
---
|
||||
title: 'Creating a New Security Compliance Framework in Prowler'
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
@@ -1,15 +1,18 @@
|
||||
# Prowler Services
|
||||
---
|
||||
title: 'Prowler Services'
|
||||
---
|
||||
|
||||
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](./provider.md).
|
||||
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](/developer-guide/provider).
|
||||
|
||||
???+note
|
||||
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](./provider.md) documentation to create it from scratch.
|
||||
<Note>
|
||||
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](/developer-guide/provider) documentation to create it from scratch.
|
||||
|
||||
</Note>
|
||||
## Introduction
|
||||
|
||||
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](./provider.md), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
|
||||
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](/developer-guide/provider), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
|
||||
|
||||
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](./checks.md) to generate the security findings.
|
||||
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](/developer-guide/checks) to generate the security findings.
|
||||
|
||||
## Adding a New Service
|
||||
|
||||
@@ -159,9 +162,10 @@ class <Service>(ServiceParentClass):
|
||||
)
|
||||
```
|
||||
|
||||
???+note
|
||||
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
|
||||
<Note>
|
||||
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
|
||||
|
||||
</Note>
|
||||
#### Resource Models
|
||||
|
||||
Resource models define structured classes used within services to store and process data extracted from API calls. They are defined in the same file as the service class, but outside of the class, usually at the bottom of the file.
|
||||
@@ -231,11 +235,11 @@ Before implementing a new service, verify that Prowler's existing permissions fo
|
||||
|
||||
Provider-Specific Permissions Documentation:
|
||||
|
||||
- [AWS](../tutorials/aws/authentication.md#required-permissions)
|
||||
- [Azure](../tutorials/azure/authentication.md#required-permissions)
|
||||
- [GCP](../tutorials/gcp/authentication.md#required-permissions)
|
||||
- [M365](../tutorials/microsoft365/authentication.md#required-permissions)
|
||||
- [GitHub](../tutorials/github/authentication.md)
|
||||
- [AWS](/user-guide/providers/aws/authentication#required-permissions)
|
||||
- [Azure](/user-guide/providers/azure/authentication#required-permissions)
|
||||
- [GCP](/user-guide/providers/gcp/authentication#required-permissions)
|
||||
- [M365](/user-guide/providers/microsoft365/authentication#required-permissions)
|
||||
- [GitHub](/user-guide/providers/github/authentication)
|
||||
|
||||
## Best Practices
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
# Unit Tests for Prowler Checks
|
||||
---
|
||||
title: 'Unit Tests for Prowler Checks'
|
||||
---
|
||||
|
||||
Unit tests for Prowler checks vary based on the provider being evaluated.
|
||||
|
||||
@@ -39,7 +41,7 @@ To execute the Prowler test suite, install the necessary dependencies listed in
|
||||
|
||||
### Prerequisites
|
||||
|
||||
If you have not installed Prowler yet, refer to the [developer guide introduction](./introduction.md#getting-the-code-and-installing-all-dependencies).
|
||||
If you have not installed Prowler yet, refer to the [developer guide introduction](/developer-guide/introduction#getting-the-code-and-installing-all-dependencies).
|
||||
|
||||
### Executing Tests
|
||||
|
||||
@@ -57,16 +59,18 @@ Other Commands for Running Tests
|
||||
- Running tests for a provider check:
|
||||
`pytest -n auto -vvv -s -x tests/providers/<provider>/services/<service>/<check>`
|
||||
|
||||
???+ note
|
||||
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
|
||||
<Note>
|
||||
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
|
||||
|
||||
</Note>
|
||||
## AWS Testing Approaches
|
||||
|
||||
For AWS provider, different testing approaches apply based on API coverage based on several criteria.
|
||||
|
||||
???+ note
|
||||
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
|
||||
<Note>
|
||||
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
|
||||
|
||||
</Note>
|
||||
- AWS API Calls Covered by [Moto](https://github.com/getmoto/moto):
|
||||
- Service Tests: `@mock_aws`
|
||||
- Checks Tests: `@mock_aws`
|
||||
@@ -205,12 +209,14 @@ class Test_iam_password_policy_uppercase:
|
||||
|
||||
If the IAM service required for testing is not supported by the Moto library, use [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock) to inject objects into the service client.
|
||||
|
||||
???+ warning
|
||||
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
|
||||
<Warning>
|
||||
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
|
||||
|
||||
???+ note
|
||||
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
|
||||
</Warning>
|
||||
<Note>
|
||||
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
|
||||
|
||||
</Note>
|
||||
#### Mocking Service Objects Using MagicMock
|
||||
|
||||
The following code demonstrates how to use MagicMock to create service objects.
|
||||
@@ -377,13 +383,15 @@ class Test_iam_password_policy_uppercase:
|
||||
# Refer to the previous section for the check test, as the implementation remains unchanged.
|
||||
```
|
||||
|
||||
???+ note
|
||||
This example does not use Moto to simplify the setup.
|
||||
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
|
||||
<Note>
|
||||
This example does not use Moto to simplify the setup.
|
||||
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
|
||||
|
||||
???+ note
|
||||
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
|
||||
</Note>
|
||||
<Note>
|
||||
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
|
||||
|
||||
</Note>
|
||||
#### Mocking Several Services
|
||||
|
||||
Since the provider is being mocked, multiple attributes can be configured to customize its behavior:
|
||||
@@ -488,7 +496,7 @@ will cause that the service will be initialised twice:
|
||||
|
||||
Later, when importing `<service>_client.py` at `<check>.py`, Python uses the mocked instance since the patch was applied at the correct reference point.
|
||||
|
||||
In the [next section](./unit-testing.md#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
|
||||
In the [next section](/developer-guide/unit-testing#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
|
||||
|
||||
##### Mocking the Service and the Service Client at the Service Client Level
|
||||
|
||||
@@ -642,9 +650,10 @@ class Test_compute_project_os_login_enabled:
|
||||
|
||||
The testing of Google Cloud Services follows the same principles as the one of Google Cloud checks. While all API calls must be mocked, attribute setup for API calls in this scenario is defined in the fixtures file, specifically within the [fixtures file](https://github.com/prowler-cloud/prowler/blob/master/tests/providers/gcp/gcp_fixtures.py) in the `mock_api_client` function.
|
||||
|
||||
???+ important
|
||||
Every method within a service must be tested to ensure full coverage and accurate validation.
|
||||
<Warning>
|
||||
Every method within a service must be tested to ensure full coverage and accurate validation.
|
||||
|
||||
</Warning>
|
||||
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
|
||||
|
||||
```python title="BigQuery Service Test"
|
||||
@@ -907,9 +916,12 @@ class Test_app_ensure_http_is_redirected_to_https:
|
||||
|
||||
The testing of Azure Services follows the same principles as the one of Google Cloud checks. All API calls are still mocked, but for methods that initialize attributes via an API call, use the [patch](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch) decorator at the beginning of the class to ensure proper mocking.
|
||||
|
||||
???+ important "Remember"
|
||||
Every method within a service must be tested to ensure full coverage and accurate validation.
|
||||
<Warning>
|
||||
**Remember**
|
||||
|
||||
Every method within a service must be tested to ensure full coverage and accurate validation.
|
||||
|
||||
</Warning>
|
||||
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
|
||||
|
||||
```python title="AppInsights Service Test"
|
||||
+433
@@ -0,0 +1,433 @@
|
||||
{
|
||||
"$schema": "https://mintlify.com/docs.json",
|
||||
"theme": "mint",
|
||||
"name": "Prowler Documentation",
|
||||
"colors": {
|
||||
"primary": "#000000",
|
||||
"light": "#10B981",
|
||||
"dark": "#10B981"
|
||||
},
|
||||
"favicon": "/favicon.ico",
|
||||
"logo": {
|
||||
"dark": "/images/prowler-logo-white.png",
|
||||
"light": "/images/prowler-logo-black.png"
|
||||
},
|
||||
"navigation": {
|
||||
"tabs": [
|
||||
{
|
||||
"tab": "Getting Started",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Welcome",
|
||||
"pages": [
|
||||
"introduction"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler Cloud",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-cloud",
|
||||
"getting-started/products/prowler-cloud-pricing",
|
||||
"getting-started/products/prowler-cloud-aws-marketplace",
|
||||
"getting-started/goto/prowler-cloud",
|
||||
"getting-started/goto/prowler-api-reference"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler CLI",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-cli",
|
||||
"getting-started/installation/prowler-cli",
|
||||
"getting-started/basic-usage/prowler-cli"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler App",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-app",
|
||||
"getting-started/installation/prowler-app",
|
||||
"getting-started/basic-usage/prowler-app"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler Lighthouse AI",
|
||||
"pages": [
|
||||
"user-guide/tutorials/prowler-app-lighthouse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler MCP Server",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-mcp",
|
||||
"getting-started/installation/prowler-mcp",
|
||||
"getting-started/basic-usage/prowler-mcp",
|
||||
"getting-started/basic-usage/prowler-mcp-tools"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler Hub",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-hub",
|
||||
"getting-started/goto/prowler-hub"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Prowler vs. Others",
|
||||
"pages": [
|
||||
"getting-started/comparison/index",
|
||||
"getting-started/comparison/awssecurityhub",
|
||||
"getting-started/comparison/gcp",
|
||||
"getting-started/comparison/microsoftdefender",
|
||||
"getting-started/comparison/microsoftsentinel"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Guides",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Prowler Cloud/App",
|
||||
"pages": [
|
||||
"user-guide/tutorials/prowler-app",
|
||||
{
|
||||
"group": "Authentication",
|
||||
"pages": [
|
||||
"user-guide/tutorials/prowler-app-social-login",
|
||||
"user-guide/tutorials/prowler-app-sso"
|
||||
]
|
||||
},
|
||||
"user-guide/tutorials/prowler-app-rbac",
|
||||
"user-guide/providers/prowler-app-api-keys",
|
||||
"user-guide/tutorials/prowler-app-mute-findings",
|
||||
{
|
||||
"group": "Integrations",
|
||||
"expanded": true,
|
||||
"pages": [
|
||||
"user-guide/tutorials/prowler-app-s3-integration",
|
||||
"user-guide/tutorials/prowler-app-security-hub-integration",
|
||||
"user-guide/tutorials/prowler-app-jira-integration"
|
||||
]
|
||||
},
|
||||
"user-guide/tutorials/prowler-app-lighthouse",
|
||||
{
|
||||
"group": "Tutorials",
|
||||
"pages": [
|
||||
"user-guide/tutorials/prowler-app-sso-entra",
|
||||
"user-guide/tutorials/bulk-provider-provisioning",
|
||||
"user-guide/tutorials/aws-organizations-bulk-provisioning"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "CLI",
|
||||
"pages": [
|
||||
"user-guide/cli/tutorials/misc",
|
||||
"user-guide/cli/tutorials/reporting",
|
||||
"user-guide/cli/tutorials/compliance",
|
||||
"user-guide/cli/tutorials/dashboard",
|
||||
"user-guide/cli/tutorials/configuration_file",
|
||||
"user-guide/cli/tutorials/logging",
|
||||
"user-guide/cli/tutorials/mutelist",
|
||||
{
|
||||
"group": "Integrations",
|
||||
"pages": [
|
||||
"user-guide/providers/aws/securityhub",
|
||||
"user-guide/cli/tutorials/integrations",
|
||||
"user-guide/providers/aws/s3"
|
||||
]
|
||||
},
|
||||
"user-guide/cli/tutorials/fixer",
|
||||
"user-guide/cli/tutorials/check-aliases",
|
||||
"user-guide/cli/tutorials/custom-checks-metadata",
|
||||
"user-guide/cli/tutorials/pentesting",
|
||||
"user-guide/cli/tutorials/scan-unused-services",
|
||||
"user-guide/cli/tutorials/quick-inventory",
|
||||
{
|
||||
"group": "Tutorials",
|
||||
"pages": [
|
||||
"user-guide/cli/tutorials/parallel-execution"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Providers",
|
||||
"pages": [
|
||||
{
|
||||
"group": "AWS",
|
||||
"pages": [
|
||||
"user-guide/providers/aws/getting-started-aws",
|
||||
"user-guide/providers/aws/authentication",
|
||||
"user-guide/providers/aws/role-assumption",
|
||||
"user-guide/providers/aws/organizations",
|
||||
"user-guide/providers/aws/regions-and-partitions",
|
||||
"user-guide/providers/aws/tag-based-scan",
|
||||
"user-guide/providers/aws/resource-arn-based-scan",
|
||||
"user-guide/providers/aws/boto3-configuration",
|
||||
"user-guide/providers/aws/threat-detection",
|
||||
"user-guide/providers/aws/cloudshell",
|
||||
"user-guide/providers/aws/multiaccount"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Azure",
|
||||
"pages": [
|
||||
"user-guide/providers/azure/getting-started-azure",
|
||||
"user-guide/providers/azure/authentication",
|
||||
"user-guide/providers/azure/use-non-default-cloud",
|
||||
"user-guide/providers/azure/subscriptions",
|
||||
"user-guide/providers/azure/create-prowler-service-principal"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Google Cloud",
|
||||
"pages": [
|
||||
"user-guide/providers/gcp/getting-started-gcp",
|
||||
"user-guide/providers/gcp/authentication",
|
||||
"user-guide/providers/gcp/projects",
|
||||
"user-guide/providers/gcp/organization",
|
||||
"user-guide/providers/gcp/retry-configuration"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Kubernetes",
|
||||
"pages": [
|
||||
"user-guide/providers/kubernetes/in-cluster",
|
||||
"user-guide/providers/kubernetes/outside-cluster",
|
||||
"user-guide/providers/kubernetes/misc"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Microsoft 365",
|
||||
"pages": [
|
||||
"user-guide/providers/microsoft365/getting-started-m365",
|
||||
"user-guide/providers/microsoft365/authentication",
|
||||
"user-guide/providers/microsoft365/use-of-powershell"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "GitHub",
|
||||
"pages": [
|
||||
"user-guide/providers/github/getting-started-github",
|
||||
"user-guide/providers/github/authentication"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "IaC",
|
||||
"pages": [
|
||||
"user-guide/providers/iac/getting-started-iac",
|
||||
"user-guide/providers/iac/authentication"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "MongoDB Atlas",
|
||||
"pages": [
|
||||
"user-guide/providers/mongodbatlas/getting-started-mongodbatlas",
|
||||
"user-guide/providers/mongodbatlas/authentication"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "LLM",
|
||||
"pages": [
|
||||
"user-guide/providers/llm/getting-started-llm"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Oracle Cloud Infrastructure",
|
||||
"pages": [
|
||||
"user-guide/providers/oci/getting-started-oci",
|
||||
"user-guide/providers/oci/authentication"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Compliance",
|
||||
"pages": [
|
||||
"user-guide/compliance/tutorials/threatscore"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Developer Guide",
|
||||
"groups": [
|
||||
{
|
||||
"group": "Concepts",
|
||||
"pages": [
|
||||
"developer-guide/introduction",
|
||||
"developer-guide/provider",
|
||||
"developer-guide/services",
|
||||
"developer-guide/checks",
|
||||
"developer-guide/outputs",
|
||||
"developer-guide/integrations",
|
||||
"developer-guide/security-compliance-framework",
|
||||
"developer-guide/lighthouse"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Providers",
|
||||
"pages": [
|
||||
"developer-guide/aws-details",
|
||||
"developer-guide/azure-details",
|
||||
"developer-guide/gcp-details",
|
||||
"developer-guide/kubernetes-details",
|
||||
"developer-guide/m365-details",
|
||||
"developer-guide/github-details",
|
||||
"developer-guide/llm-details"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Miscellaneous",
|
||||
"pages": [
|
||||
"developer-guide/documentation",
|
||||
{
|
||||
"group": "Testing",
|
||||
"pages": [
|
||||
"developer-guide/unit-testing",
|
||||
"developer-guide/integration-testing"
|
||||
]
|
||||
},
|
||||
"developer-guide/debugging",
|
||||
"developer-guide/configurable-checks",
|
||||
"developer-guide/renaming-checks",
|
||||
"developer-guide/check-metadata-guidelines"
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Security",
|
||||
"pages": [
|
||||
"security"
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Contact Us",
|
||||
"pages": [
|
||||
"contact"
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "Troubleshooting",
|
||||
"pages": [
|
||||
"troubleshooting"
|
||||
]
|
||||
},
|
||||
{
|
||||
"tab": "About Us",
|
||||
"icon": "/favicon.ico",
|
||||
"href": "https://prowler.com/about#team"
|
||||
},
|
||||
{
|
||||
"tab": "Changelog",
|
||||
"icon": "github",
|
||||
"href": "https://github.com/prowler-cloud/prowler/releases"
|
||||
},
|
||||
{
|
||||
"tab": "Public Roadmap",
|
||||
"href": "https://roadmap.prowler.com/"
|
||||
}
|
||||
],
|
||||
"global": {
|
||||
"anchors": [
|
||||
{
|
||||
"anchor": "GitHub",
|
||||
"href": "https://github.com/prowler-cloud/prowler",
|
||||
"icon": "github"
|
||||
},
|
||||
{
|
||||
"anchor": "Slack",
|
||||
"href": "https://goto.prowler.com/slack",
|
||||
"icon": "slack"
|
||||
},
|
||||
{
|
||||
"anchor": "YouTube",
|
||||
"href": "https://www.youtube.com/@prowlercloud",
|
||||
"icon": "youtube"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"navbar": {
|
||||
"links": [
|
||||
{
|
||||
"label": "Prowler Hub",
|
||||
"href": "https://hub.prowler.com"
|
||||
},
|
||||
{
|
||||
"label": "Prowler Cloud",
|
||||
"href": "https://cloud.prowler.com",
|
||||
"style": "primary"
|
||||
}
|
||||
]
|
||||
},
|
||||
"analytics": {
|
||||
"ga4": {
|
||||
"measurementId": "G-KBKV70W5Y2"
|
||||
}
|
||||
},
|
||||
"feedback": {
|
||||
"thumbsRating": true,
|
||||
"suggestEdit": true,
|
||||
"raiseIssue": true
|
||||
},
|
||||
"footer": {
|
||||
"socials": {
|
||||
"x-twitter": "https://x.com/prowlercloud",
|
||||
"github": "https://github.com/prowler-cloud/prowler",
|
||||
"linkedin": "https://www.linkedin.com/company/prowler-security",
|
||||
"youtube": "https://www.youtube.com/@prowlercloud",
|
||||
"slack": "https://goto.prowler.com/slack",
|
||||
"website": "https://prowler.com"
|
||||
}
|
||||
},
|
||||
"redirects": [
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-lighthouse",
|
||||
"destination": "/user-guide/tutorials/prowler-app-lighthouse"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/developer-guide/introduction",
|
||||
"destination": "/developer-guide/introduction"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/aws/getting-started-aws",
|
||||
"destination": "/user-guide/providers/aws/getting-started-aws"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/azure/getting-started-azure",
|
||||
"destination": "/user-guide/providers/azure/getting-started-azure"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/gcp/getting-started-gcp",
|
||||
"destination": "/user-guide/providers/gcp/getting-started-gcp"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/microsoft365/getting-started-m365",
|
||||
"destination": "/user-guide/providers/microsoft365/getting-started-m365"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/github/getting-started-github",
|
||||
"destination": "/user-guide/providers/github/getting-started-github"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-sso",
|
||||
"destination": "/user-guide/tutorials/prowler-app-sso"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest",
|
||||
"destination": "/introduction"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-saas/en/latest/:slug*",
|
||||
"destination": "https://docs.prowler.pro/en/latest/:slug*"
|
||||
},
|
||||
{
|
||||
"source": "/projects/prowler-open-source/en/latest/tutorials/:slug*",
|
||||
"destination": "/user-guide/tutorials/:slug*"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
---
|
||||
title: 'Basic Usage'
|
||||
---
|
||||
|
||||
## Access Prowler App
|
||||
|
||||
After [installation](/getting-started/installation/prowler-app), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
|
||||
|
||||
<img src="/images/sign-up-button.png" alt="Sign Up Button" width="320" />
|
||||
<img src="/images/sign-up.png" alt="Sign Up" width="285" />
|
||||
|
||||
<Note>
|
||||
**User creation and default tenant behavior**
|
||||
|
||||
|
||||
When creating a new user, the behavior depends on whether an invitation is provided:
|
||||
|
||||
- **Without an invitation**:
|
||||
|
||||
- A new tenant is automatically created.
|
||||
- The new user is assigned to this tenant.
|
||||
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
|
||||
|
||||
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
|
||||
|
||||
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
|
||||
|
||||
</Note>
|
||||
## Log In
|
||||
|
||||
Access Prowler App by logging in with **email and password**.
|
||||
|
||||
<img src="/images/log-in.png" alt="Log In" width="285" />
|
||||
|
||||
## Add Cloud Provider
|
||||
|
||||
Configure a cloud provider for scanning:
|
||||
|
||||
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
|
||||
2. Select the cloud provider.
|
||||
3. Enter the provider's identifier (Optional: Add an alias):
|
||||
- **AWS**: Account ID
|
||||
- **GCP**: Project ID
|
||||
- **Azure**: Subscription ID
|
||||
- **Kubernetes**: Cluster ID
|
||||
- **M365**: Domain ID
|
||||
4. Follow the guided instructions to add and authenticate your credentials.
|
||||
|
||||
## Start a Scan
|
||||
|
||||
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
|
||||
|
||||
Click `Go to Scans` to monitor progress.
|
||||
|
||||
## View Results
|
||||
|
||||
Review findings during scan execution in the following sections:
|
||||
|
||||
- **Overview** – Provides a high-level summary of your scans.
|
||||
<img src="/images/products/overview.png" alt="Overview" width="700" />
|
||||
|
||||
- **Compliance** – Displays compliance insights based on security frameworks.
|
||||
<img src="/images/compliance.png" alt="Compliance" width="700" />
|
||||
|
||||
> For detailed usage instructions, refer to the [Prowler App Guide](/user-guide/tutorials/prowler-app).
|
||||
|
||||
<Note>
|
||||
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
|
||||
|
||||
</Note>
|
||||
+60
-26
@@ -1,18 +1,24 @@
|
||||
---
|
||||
title: 'Basic Usage'
|
||||
---
|
||||
|
||||
## Running Prowler
|
||||
|
||||
Running Prowler requires specifying the provider (e.g `aws`, `gcp`, `azure`, `kubernetes`, `m365`, `github`, `iac` or `mongodbatlas`):
|
||||
|
||||
???+ note
|
||||
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
|
||||
<Note>
|
||||
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
|
||||
|
||||
</Note>
|
||||
```console
|
||||
prowler <provider>
|
||||
```
|
||||

|
||||

|
||||
|
||||
???+ note
|
||||
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
|
||||
<Note>
|
||||
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
|
||||
|
||||
</Note>
|
||||
## Verbose Output
|
||||
|
||||
If you prefer the former verbose output, use: `--verbose`. This allows seeing more info while Prowler is running, minimal output is displayed unless verbosity is enabled.
|
||||
@@ -26,7 +32,7 @@ prowler <provider> -M csv json-asff json-ocsf html
|
||||
```
|
||||
The HTML report is saved in the output directory, alongside other reports. It will look like this:
|
||||
|
||||

|
||||

|
||||
|
||||
## Listing Available Checks and Services
|
||||
|
||||
@@ -58,7 +64,7 @@ prowler kubernetes --excluded-services controllermanager
|
||||
```
|
||||
## Additional Options
|
||||
|
||||
Explore more advanced time-saving execution methods in the [Miscellaneous](../tutorials/misc.md) section.
|
||||
Explore more advanced time-saving execution methods in the [Miscellaneous](/user-guide/cli/tutorials/misc) section.
|
||||
|
||||
Access the help menu and view all available options with `-h`/`--help`:
|
||||
|
||||
@@ -74,10 +80,11 @@ Use a custom AWS profile with `-p`/`--profile` and/or specific AWS regions with
|
||||
prowler aws --profile custom-profile -f us-east-1 eu-south-2
|
||||
```
|
||||
|
||||
???+ note
|
||||
By default, `prowler` will scan all AWS regions.
|
||||
<Note>
|
||||
By default, `prowler` will scan all AWS regions.
|
||||
|
||||
See more details about AWS Authentication in the [Authentication Section](../tutorials/aws/authentication.md) section.
|
||||
</Note>
|
||||
See more details about AWS Authentication in the [Authentication Section](/user-guide/providers/aws/authentication) section.
|
||||
|
||||
## Azure
|
||||
|
||||
@@ -97,7 +104,7 @@ prowler azure --browser-auth --tenant-id "XXXXXXXX"
|
||||
prowler azure --managed-identity-auth
|
||||
```
|
||||
|
||||
See more details about Azure Authentication in the [Authentication Section](../tutorials/azure/authentication.md)
|
||||
See more details about Azure Authentication in the [Authentication Section](/user-guide/providers/azure/authentication)
|
||||
|
||||
By default, Prowler scans all accessible subscriptions. Scan specific subscriptions using the following flag (using az cli auth as example):
|
||||
|
||||
@@ -154,9 +161,10 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
|
||||
```console
|
||||
prowler kubernetes --kubeconfig-file path
|
||||
```
|
||||
???+ note
|
||||
<Note>
|
||||
If no `--kubeconfig-file` is provided, Prowler will use the default KubeConfig file location (`~/.kube/config`).
|
||||
|
||||
</Note>
|
||||
- **In-Cluster Execution**
|
||||
|
||||
To run Prowler inside the cluster, apply the provided YAML configuration to deploy a job in a new namespace:
|
||||
@@ -170,9 +178,10 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
|
||||
kubectl logs prowler-XXXXX --namespace prowler-ns
|
||||
```
|
||||
|
||||
???+ note
|
||||
<Note>
|
||||
By default, Prowler scans all namespaces in the active Kubernetes context. Use the `--context`flag to specify the context to be scanned and `--namespaces` to restrict scanning to specific namespaces.
|
||||
|
||||
</Note>
|
||||
## Microsoft 365
|
||||
|
||||
Microsoft 365 requires specifying the auth method:
|
||||
@@ -182,9 +191,6 @@ Microsoft 365 requires specifying the auth method:
|
||||
# To use service principal authentication for MSGraph and PowerShell modules
|
||||
prowler m365 --sp-env-auth
|
||||
|
||||
# To use both service principal (for MSGraph) and user credentials (for PowerShell modules)
|
||||
prowler m365 --env-auth
|
||||
|
||||
# To use az cli authentication
|
||||
prowler m365 --az-cli-auth
|
||||
|
||||
@@ -193,7 +199,7 @@ prowler m365 --browser-auth --tenant-id "XXXXXXXX"
|
||||
|
||||
```
|
||||
|
||||
See more details about M365 Authentication in the [Authentication Section](../tutorials/microsoft365/authentication.md) section.
|
||||
See more details about M365 Authentication in the [Authentication Section](/user-guide/providers/microsoft365/authentication) section.
|
||||
|
||||
## GitHub
|
||||
|
||||
@@ -214,13 +220,14 @@ Prowler enables security scanning of your **GitHub account**, including **Reposi
|
||||
prowler github --github-app-id app_id --github-app-key app_key
|
||||
```
|
||||
|
||||
???+ note
|
||||
<Note>
|
||||
If no login method is explicitly provided, Prowler will automatically attempt to authenticate using environment variables in the following order of precedence:
|
||||
|
||||
1. `GITHUB_PERSONAL_ACCESS_TOKEN`
|
||||
2. `OAUTH_APP_TOKEN`
|
||||
3. `GITHUB_APP_ID` and `GITHUB_APP_KEY`
|
||||
|
||||
</Note>
|
||||
## Infrastructure as Code (IaC)
|
||||
|
||||
Prowler's Infrastructure as Code (IaC) provider enables you to scan local or remote infrastructure code for security and compliance issues using [Trivy](https://trivy.dev/). This provider supports a wide range of IaC frameworks, allowing you to assess your code before deployment.
|
||||
@@ -247,14 +254,15 @@ prowler iac --scan-path ./my-iac-directory --frameworks terraform kubernetes
|
||||
prowler iac --scan-path ./my-iac-directory --exclude-path ./my-iac-directory/test,./my-iac-directory/examples
|
||||
```
|
||||
|
||||
???+ note
|
||||
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
|
||||
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
|
||||
- The IaC provider does not require cloud authentication for local scans.
|
||||
- It is ideal for CI/CD pipelines and local development environments.
|
||||
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
|
||||
<Note>
|
||||
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
|
||||
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
|
||||
- The IaC provider does not require cloud authentication for local scans.
|
||||
- It is ideal for CI/CD pipelines and local development environments.
|
||||
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
|
||||
|
||||
See more details about IaC scanning in the [IaC Tutorial](../tutorials/iac/getting-started-iac.md) section.
|
||||
</Note>
|
||||
See more details about IaC scanning in the [IaC Tutorial](/user-guide/providers/iac/getting-started-iac) section.
|
||||
|
||||
## MongoDB Atlas
|
||||
|
||||
@@ -279,4 +287,30 @@ You can filter scans to specific organizations or projects:
|
||||
prowler mongodbatlas --atlas-project-id <project_id>
|
||||
```
|
||||
|
||||
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](../tutorials/mongodbatlas/authentication.md)
|
||||
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](/user-guide/providers/mongodbatlas/authentication)
|
||||
|
||||
## Oracle Cloud
|
||||
|
||||
Prowler allows you to scan your Oracle Cloud deployments for security and compliance issues.
|
||||
|
||||
You have two options to authenticate:
|
||||
|
||||
1. OCI Config File Authentication: this config file can be generated using the OCI CLI with the `oci session authenticate` command or created manually using the OCI Console. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#oci-session-authentication).
|
||||
|
||||
```console
|
||||
prowler oci
|
||||
```
|
||||
|
||||
You can add different profiles to the config file to scan different tenancies or regions. In order to scan a specific profile, use the `--profile` flag:
|
||||
|
||||
```console
|
||||
prowler oci --profile <profile_name>
|
||||
```
|
||||
|
||||
2. Instance Principal Authentication: when running Prowler on an OCI Compute instance, you can use Instance Principal authentication. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#instance-principal-authentication).
|
||||
|
||||
```console
|
||||
prowler oci --use-instance-principal
|
||||
```
|
||||
|
||||
See more details about Oracle Cloud Authentication in [Oracle Cloud Authentication](/user-guide/providers/oci/authentication)
|
||||
@@ -0,0 +1,122 @@
|
||||
---
|
||||
title: "Tools Reference"
|
||||
---
|
||||
|
||||
Complete reference guide for all tools available in the Prowler MCP Server. Tools are organized by namespace.
|
||||
|
||||
## Tool Categories Summary
|
||||
|
||||
| Category | Tool Count | Authentication Required |
|
||||
|----------|------------|------------------------|
|
||||
| Prowler Hub | 10 tools | No |
|
||||
| Prowler Documentation | 2 tools | No |
|
||||
| Prowler Cloud/App | 28 tools | Yes |
|
||||
|
||||
## Tool Naming Convention
|
||||
|
||||
All tools follow a consistent naming pattern with prefixes:
|
||||
|
||||
- `prowler_hub_*` - Prowler Hub catalog and compliance tools
|
||||
- `prowler_docs_*` - Prowler documentation search and retrieval
|
||||
- `prowler_app_*` - Prowler Cloud and App (Self-Managed) management tools
|
||||
|
||||
## Prowler Hub Tools
|
||||
|
||||
Access Prowler's security check catalog and compliance frameworks. **No authentication required.**
|
||||
|
||||
### Check Discovery
|
||||
|
||||
- **`prowler_hub_get_checks`** - List security checks with advanced filtering options
|
||||
- **`prowler_hub_get_check_filters`** - Return available filter values for checks (providers, services, severities, categories, compliances)
|
||||
- **`prowler_hub_search_checks`** - Full-text search across check metadata
|
||||
- **`prowler_hub_get_check_raw_metadata`** - Fetch raw check metadata in JSON format
|
||||
|
||||
### Check Code
|
||||
|
||||
- **`prowler_hub_get_check_code`** - Fetch the Python implementation code for a security check
|
||||
- **`prowler_hub_get_check_fixer`** - Fetch the automated fixer code for a check (if available)
|
||||
|
||||
### Compliance Frameworks
|
||||
|
||||
- **`prowler_hub_get_compliance_frameworks`** - List and filter compliance frameworks
|
||||
- **`prowler_hub_search_compliance_frameworks`** - Full-text search across compliance frameworks
|
||||
|
||||
### Provider Information
|
||||
|
||||
- **`prowler_hub_list_providers`** - List Prowler official providers and their services
|
||||
- **`prowler_hub_get_artifacts_count`** - Get total count of checks and frameworks in Prowler Hub
|
||||
|
||||
## Prowler Documentation Tools
|
||||
|
||||
Search and access official Prowler documentation. **No authentication required.**
|
||||
|
||||
- **`prowler_docs_search`** - Search the official Prowler documentation using full-text search
|
||||
- **`prowler_docs_get_document`** - Retrieve the full markdown content of a specific documentation file
|
||||
|
||||
## Prowler Cloud/App Tools
|
||||
|
||||
Manage Prowler Cloud or Prowler App (Self-Managed) features. **Requires authentication.**
|
||||
|
||||
<Note>
|
||||
These tools require a valid API key. See the [Configuration Guide](/getting-started/basic-usage/prowler-mcp) for authentication setup.
|
||||
</Note>
|
||||
|
||||
### Findings Management
|
||||
|
||||
- **`prowler_app_list_findings`** - List security findings with advanced filtering
|
||||
- **`prowler_app_get_finding`** - Get detailed information about a specific finding
|
||||
- **`prowler_app_get_latest_findings`** - Retrieve latest findings from the most recent scans
|
||||
- **`prowler_app_get_findings_metadata`** - Get unique metadata values from filtered findings
|
||||
- **`prowler_app_get_latest_findings_metadata`** - Get metadata from latest findings across all providers
|
||||
|
||||
### Provider Management
|
||||
|
||||
- **`prowler_app_list_providers`** - List all providers with filtering options
|
||||
- **`prowler_app_create_provider`** - Create a new provider in the current tenant
|
||||
- **`prowler_app_get_provider`** - Get detailed information about a specific provider
|
||||
- **`prowler_app_update_provider`** - Update provider details (alias, etc.)
|
||||
- **`prowler_app_delete_provider`** - Delete a specific provider
|
||||
- **`prowler_app_test_provider_connection`** - Test provider connection status
|
||||
|
||||
### Provider Secrets Management
|
||||
|
||||
- **`prowler_app_list_provider_secrets`** - List all provider secrets with filtering
|
||||
- **`prowler_app_add_provider_secret`** - Add or update credentials for a provider
|
||||
- **`prowler_app_get_provider_secret`** - Get detailed information about a provider secret
|
||||
- **`prowler_app_update_provider_secret`** - Update provider secret details
|
||||
- **`prowler_app_delete_provider_secret`** - Delete a provider secret
|
||||
|
||||
### Scan Management
|
||||
|
||||
- **`prowler_app_list_scans`** - List all scans with filtering options
|
||||
- **`prowler_app_create_scan`** - Trigger a manual scan for a specific provider
|
||||
- **`prowler_app_get_scan`** - Get detailed information about a specific scan
|
||||
- **`prowler_app_update_scan`** - Update scan details
|
||||
- **`prowler_app_get_scan_compliance_report`** - Download compliance report as CSV
|
||||
- **`prowler_app_get_scan_report`** - Download ZIP file containing complete scan report
|
||||
|
||||
### Schedule Management
|
||||
|
||||
- **`prowler_app_schedules_daily_scan`** - Create a daily scheduled scan for a provider
|
||||
|
||||
### Processor Management
|
||||
|
||||
- **`prowler_app_processors_list`** - List all processors with filtering
|
||||
- **`prowler_app_processors_create`** - Create a new processor (currently only mute lists supported)
|
||||
- **`prowler_app_processors_retrieve`** - Get processor details by ID
|
||||
- **`prowler_app_processors_partial_update`** - Update processor configuration
|
||||
- **`prowler_app_processors_destroy`** - Delete a processor
|
||||
|
||||
## Usage Tips
|
||||
|
||||
- Use natural language to interact with the tools through your AI assistant
|
||||
- Tools can be combined for complex workflows
|
||||
- Filter options are available on most list tools
|
||||
- Authentication is only required for Prowler Cloud/App tools
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [MCP Protocol Specification](https://modelcontextprotocol.io)
|
||||
- [Prowler API Documentation](https://api.prowler.com/api/v1/docs)
|
||||
- [Prowler Hub API](https://hub.prowler.com/api/docs)
|
||||
- [GitHub Repository](https://github.com/prowler-cloud/prowler)
|
||||
@@ -0,0 +1,247 @@
|
||||
---
|
||||
title: "Configuration"
|
||||
---
|
||||
|
||||
Configure your MCP client to connect to Prowler MCP Server.
|
||||
|
||||
## Step 1: Get Your API Key (Optional)
|
||||
|
||||
<Note>
|
||||
**Authentication is optional**: Prowler Hub and Prowler Documentation features work without authentication. An API key is only required for Prowler Cloud and Prowler App (Self-Managed) features.
|
||||
</Note>
|
||||
|
||||
To use Prowler Cloud or Prowler App (Self-Managed) features. To get the API key, please refer to the [API Keys](/user-guide/providers/prowler-app-api-keys) guide.
|
||||
|
||||
<Warning>
|
||||
Keep the API key secure. Never share it publicly or commit it to version control.
|
||||
</Warning>
|
||||
|
||||
## Step 2: Configure Your MCP Client
|
||||
|
||||
Choose the configuration based on your deployment:
|
||||
|
||||
- **STDIO Mode**: Local installation only (runs as subprocess).
|
||||
- **HTTP Mode**: Prowler Cloud MCP Server or self-hosted Prowler MCP Server.
|
||||
|
||||
### HTTP Mode (Prowler Cloud MCP Server or self-hosted Prowler MCP Server)
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Native HTTP Support (Cursor, VSCode)">
|
||||
**Clients that support HTTP with custom headers natively**
|
||||
|
||||
For example: Cursor, VSCode, LobeChat, etc.
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"url": "https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
|
||||
"headers": {
|
||||
"Authorization": "Bearer pk_your_api_key_here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Using mcp-remote (Claude Desktop)">
|
||||
**For clients without native HTTP support (like Claude Desktop)**
|
||||
|
||||
For example: Claude Desktop.
|
||||
|
||||
**Configuration:**
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"command": "npx",
|
||||
"args": [
|
||||
"mcp-remote",
|
||||
"https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
|
||||
"--header",
|
||||
"Authorization: Bearer ${PROWLER_APP_API_KEY}"
|
||||
],
|
||||
"env": {
|
||||
"PROWLER_APP_API_KEY": "pk_your_api_key_here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Info>
|
||||
The `mcp-remote` tool acts as a bridge for clients that don't support HTTP natively. Learn more at [mcp-remote on npm](https://www.npmjs.com/package/mcp-remote).
|
||||
</Info>
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### STDIO Mode (Local Installation Only)
|
||||
|
||||
STDIO mode is only available when running the MCP server locally.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Using uvx">
|
||||
**Run from source or local installation**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"command": "uvx",
|
||||
"args": ["/absolute/path/to/prowler/mcp_server/"],
|
||||
"env": {
|
||||
"PROWLER_APP_API_KEY": "pk_your_api_key_here",
|
||||
"PROWLER_API_BASE_URL": "https://api.prowler.com"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
Replace `/absolute/path/to/prowler/mcp_server/` with the actual path. The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
|
||||
</Note>
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Using Docker">
|
||||
**Run with Docker image**
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"command": "docker",
|
||||
"args": [
|
||||
"run",
|
||||
"--rm",
|
||||
"-i",
|
||||
"--env",
|
||||
"PROWLER_APP_API_KEY=pk_your_api_key_here",
|
||||
"--env",
|
||||
"PROWLER_API_BASE_URL=https://api.prowler.com",
|
||||
"prowlercloud/prowler-mcp"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
|
||||
</Note>
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Step 3: Start Using Prowler MCP
|
||||
|
||||
Restart your MCP client and start asking questions:
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"Onboard this new AWS account in my Prowler Organization"*
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
Prowler MCP Server supports two authentication methods to connect to Prowler Cloud or Prowler App (Self-Managed):
|
||||
|
||||
### API Key (Recommended)
|
||||
|
||||
Use your Prowler API key directly in the Bearer token:
|
||||
|
||||
```
|
||||
Authorization: Bearer pk_your_api_key_here
|
||||
```
|
||||
|
||||
This is the recommended method for most users.
|
||||
|
||||
### JWT Token
|
||||
|
||||
Alternatively, obtain a JWT token from Prowler:
|
||||
|
||||
```bash
|
||||
curl -X POST https://api.prowler.com/api/v1/tokens \
|
||||
-H "Content-Type: application/vnd.api+json" \
|
||||
-H "Accept: application/vnd.api+json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"type": "tokens",
|
||||
"attributes": {
|
||||
"email": "your-email@example.com",
|
||||
"password": "your-password"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Use the returned JWT token in place of the API key:
|
||||
|
||||
```
|
||||
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
|
||||
```
|
||||
|
||||
<Warning>
|
||||
JWT tokens are only valid for 30 minutes. You need to generate a new token if you want to continue using the MCP server.
|
||||
</Warning>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Not Detected
|
||||
|
||||
- Restart your MCP client after configuration changes
|
||||
- Check the configuration file syntax (valid JSON)
|
||||
- Review client logs for specific error messages
|
||||
- Verify the server URL is correct
|
||||
|
||||
### Authentication Failures
|
||||
|
||||
**Error: Unauthorized (401)**
|
||||
- Verify your API key is correct
|
||||
- Ensure the key hasn't expired
|
||||
- Check you're using the right API endpoint
|
||||
|
||||
### Connection Issues
|
||||
|
||||
**Cannot Reach Server:**
|
||||
- Verify the server URL is correct
|
||||
- Check network connectivity
|
||||
- For local servers, ensure the server is running
|
||||
- Check firewall settings
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Protect Your API Key**
|
||||
- Never commit API keys to version control.
|
||||
- Use environment variables or secure vaults.
|
||||
- Rotate keys regularly.
|
||||
|
||||
2. **Network Security**
|
||||
- Use HTTPS for production deployments.
|
||||
- Restrict network access to the MCP server.
|
||||
- Consider VPN for remote access.
|
||||
|
||||
3. **Least Privilege**
|
||||
- API key gives the permission of the user who created the key, make sure to use the key with the minimal required permissions.
|
||||
- Review the tools that are gonna be used and how they are gonna be used to avoid prompt injections or unintended behavior.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that your MCP server is configured:
|
||||
|
||||
<CardGroup cols={1}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
Need assistance with configuration?
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
title: 'AWS Security Hub'
|
||||
---
|
||||
|
||||
AWS Security Hub remains a managed service designed for centralizing security alerts and compliance status within AWS environments. It integrates with various AWS security services and provides a consolidated view of security findings.
|
||||
|
||||
## Key Features and Strengths
|
||||
|
||||
- **Centralized Dashboard for AWS:** Provides a single pane of glass to monitor and manage security findings from multiple AWS services like GuardDuty, Inspector, and Config.
|
||||
|
||||
- **Compliance Checks:** Automatically checks for compliance against standards like CIS and PCI DSS within AWS environments.
|
||||
|
||||
- **AWS Native Automation:** Offers seamless automation for incident response using AWS Lambda and CloudWatch Events, reducing the time to react to security issues.
|
||||
|
||||
- **User-Friendly Interface:** Accessible via the AWS Management Console, offering a streamlined experience for managing security across AWS accounts.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **AWS-Centric:** Limited to AWS environments, with no direct support for multi-cloud or hybrid environments.
|
||||
|
||||
- **Dependency on AWS Config:** Some of its checks depend on AWS Config, which may not be enabled in all regions or accounts.
|
||||
|
||||
- **Vendor Lock-In:** Tightly coupled with AWS, making it less suitable for organizations with a cloud-agnostic strategy.
|
||||
|
||||
## Prowler
|
||||
|
||||
Prowler is an open-source, multi-cloud security tool that offers extensive customization and flexibility, making it ideal for organizations with complex or multi-cloud environments. Here are the updated features and advantages:
|
||||
|
||||
## Main Advantages of Prowler
|
||||
|
||||
- **Multi-Region and Multi-Account Scanning by Default:**
|
||||
- Prowler is inherently multi-region and can scan multiple AWS accounts without requiring additional configuration or enabling specific services like AWS Config.
|
||||
|
||||
- **Minimal Setup Requirements:**
|
||||
- All Prowler needs is a role with appropriate permissions to start scanning. There’s no need to enable specific services or configure complex setups.
|
||||
|
||||
- **Versatile Execution Environment:**
|
||||
- Prowler can be run from various environments, including a local workstation, container, AWS CloudShell, or even from another AWS account or cloud provider by assuming a role. This flexibility makes it easy to integrate into different operational workflows.
|
||||
|
||||
- **Flexible Results Storage and Sharing:**
|
||||
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis, or locally for easy sharing and discussion. This flexibility is particularly useful for collaborative security assessments.
|
||||
|
||||
- **Customizable Reporting and Analysis:**
|
||||
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. It also supports integration with Amazon QuickSight for in-depth analysis and offers a SaaS model with resource-based pricing, making it adaptable to different organizational needs.
|
||||
|
||||
- **Security Hub Integration for Cost-Effective Operations:**
|
||||
- Prowler can send results directly into Security Hub in any AWS account, including only failed findings. This selective reporting can make Security Hub more cost-effective by reducing the volume of data processed.
|
||||
|
||||
- **Custom Checks and Compliance Frameworks:**
|
||||
- Users can write custom checks, remediations, and compliance frameworks in minutes, tailoring the tool to their specific security policies and operational needs.
|
||||
|
||||
- **Extensive Compliance Support:**
|
||||
- Prowler supports over 27 compliance frameworks out of the box for AWS, providing comprehensive coverage across various regulatory requirements and best practices.
|
||||
|
||||
- **Kubernetes and Multi-Cloud Support:**
|
||||
- Prowler extends its scanning capabilities beyond AWS, offering support for Kubernetes clusters (including EKS), as well as environments in Google Cloud Platform (GCP) and Azure. This multi-cloud capability is essential for organizations with diverse cloud footprints.
|
||||
|
||||
- **All-Region Checks:**
|
||||
- Prowler runs all checks in all regions, regardless of AWS Config resource type support, ensuring comprehensive coverage across your entire AWS environment.
|
||||
|
||||
## Comparison Summary
|
||||
|
||||
### Scope and Environment
|
||||
|
||||
- **Security Hub** is ideal for AWS-centric environments needing a managed service for monitoring and automating security across AWS resources.
|
||||
- **Prowler** is better suited for organizations operating in multi-cloud or hybrid environments, offering flexibility, customization, and support for multiple cloud providers including AWS, Azure, GCP, and Kubernetes.
|
||||
|
||||
### Setup and Maintenance
|
||||
|
||||
- **Security Hub** requires enabling and configuring AWS services by region, per account, and can become more than one person's full-time role – including Config. Security Hub operates only within the AWS ecosystem.
|
||||
- **Prowler** requires minimal setup, only needing appropriate permissions, and can be executed from various environments, making it more versatile in different operational contexts.
|
||||
|
||||
### Customization and Flexibility
|
||||
|
||||
- **Security Hub** offers predefined compliance checks and automation within AWS but is less flexible in terms of customization.
|
||||
- **Prowler** allows for highly customizable checks, remediation actions, and compliance frameworks, with the ability to adapt quickly to organizational needs and regulatory changes.
|
||||
|
||||
### Cost Efficiency
|
||||
|
||||
- **Security Hub** may involve additional costs for processing and storing findings.
|
||||
- **Prowler** can optimize costs by selectively sending failed findings to Security Hub and storing results locally or in S3, which can be more cost-effective.
|
||||
|
||||
### Multi-Cloud and Multi-Region Support
|
||||
|
||||
- **Security Hub** is confined to AWS, with region-specific checks depending on AWS Config.
|
||||
- **Prowler** is inherently multi-region and multi-cloud, offering consistent and comprehensive checks across different cloud environments and regions.
|
||||
|
||||
## Conclusion
|
||||
|
||||
For a CISO or security professional evaluating these tools, the decision between AWS Security Hub and Prowler will depend on the organization’s cloud strategy, compliance needs, and the level of flexibility required:
|
||||
|
||||
- If the organization is heavily invested in AWS and prefers a managed, integrated security service that offers ease of use and automation within the AWS ecosystem, **AWS Security Hub** is the more appropriate choice.
|
||||
|
||||
- If the organization operates in a multi-cloud environment or requires a highly customizable tool that can run comprehensive, multi-region scans across AWS, Azure, GCP, and Kubernetes, **Prowler** provides a more powerful and flexible solution, especially for those needing to adapt quickly to evolving security and compliance requirements.
|
||||
@@ -0,0 +1,97 @@
|
||||
---
|
||||
title: 'GCP Cloud Security Command Center (Cloud SCC)'
|
||||
---
|
||||
|
||||
Google Cloud Security Command Center (Cloud SCC) is a centralized security and risk management platform for Google Cloud Platform (GCP). It provides visibility into assets, vulnerabilities, and threats across GCP environments, helping organizations to manage and improve their security posture.
|
||||
|
||||
## Key Features and Strengths
|
||||
|
||||
- **Centralized Security Visibility:** Cloud SCC provides a single pane of glass to monitor the security and risk status across your GCP resources. It aggregates findings from various GCP security services, such as Security Health Analytics, Web Security Scanner, and Event Threat Detection.
|
||||
|
||||
- **Asset Inventory and Classification:** Cloud SCC offers comprehensive asset discovery and classification across GCP, giving security teams a detailed inventory of their cloud resources, including their configurations and security states.
|
||||
|
||||
- **Threat Detection and Monitoring:** The platform integrates with GCP’s threat detection tools, such as Google’s Event Threat Detection, which analyzes logs for suspicious activities and potential threats.
|
||||
|
||||
- **Compliance Monitoring:** Cloud SCC helps monitor compliance with various regulatory standards by continuously assessing your GCP resources against best practices and security benchmarks.
|
||||
|
||||
- **Automated Remediation:** Cloud SCC can trigger automated responses to security findings through integrations with Google Cloud Functions or other orchestration tools, helping to mitigate risks quickly.
|
||||
|
||||
- **Native GCP Integration:** Cloud SCC is deeply integrated with the GCP ecosystem, offering seamless operation within Google Cloud environments and leveraging Google's extensive security expertise.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **GCP-Centric:** While Cloud SCC is powerful within the GCP ecosystem, it is primarily focused on GCP and does not natively extend to multi-cloud environments without additional tools or connectors.
|
||||
|
||||
- **Cost Considerations:** As a managed service within GCP, costs can scale with the amount of data ingested and the complexity of the environment, especially as additional features or higher volumes of data are utilized.
|
||||
|
||||
- **Dependency on GCP Services:** Cloud SCC's capabilities depend on other GCP services being enabled, such as Security Health Analytics and Web Security Scanner, which may increase overall complexity and cost.
|
||||
|
||||
## Prowler
|
||||
|
||||
Prowler is an open-source, multi-cloud security tool designed to perform detailed security assessments and compliance checks across diverse cloud environments, including AWS, Azure, GCP, and Kubernetes. Here are the key advantages of Prowler when compared to GCP Cloud SCC:
|
||||
|
||||
## Main Advantages of Prowler
|
||||
|
||||
- **Multi-Region and Multi-Account Scanning by Default:**
|
||||
- Prowler inherently supports multi-region and multi-account scanning across multiple cloud providers, including GCP, AWS, Azure, and Kubernetes. It does not require additional configuration to perform these scans, making it immediately useful for organizations operating in multiple cloud environments.
|
||||
|
||||
- **Minimal Setup Requirements:**
|
||||
- Prowler requires only appropriate roles and permissions to start scanning. It doesn’t necessitate enabling specific services within GCP, which can simplify the setup process and reduce dependencies.
|
||||
|
||||
- **Versatile Execution Environment:**
|
||||
- Prowler can be run from various environments, such as a local workstation, container, Google Cloud Shell, or even other cloud providers by assuming a role. This versatility allows for flexible deployment and integration into existing security operations.
|
||||
|
||||
- **Flexible Results Storage and Sharing:**
|
||||
- Prowler results can be stored in an S3 bucket for AWS, Google Cloud Storage (GCS) for GCP, or locally, allowing for quick analysis and easy sharing. This flexibility is particularly advantageous for multi-cloud security assessments and collaborative security processes.
|
||||
|
||||
- **Customizable Reporting and Analysis:**
|
||||
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. These reports can be tailored to specific needs and easily integrated with other security tools or dashboards, providing comprehensive insights across all cloud environments.
|
||||
|
||||
- **SIEM Integration and Cost Efficiency:**
|
||||
- Prowler can be configured to send findings directly into SIEM systems, including those integrated with GCP or other platforms. By sending only failed findings or selected results, Prowler helps manage costs associated with data ingestion and analysis in SIEM platforms.
|
||||
|
||||
- **Custom Checks and Compliance Frameworks:**
|
||||
- Prowler allows for the creation of custom security checks, remediation actions, and compliance frameworks, providing flexibility that can be adapted to the unique security policies and regulatory requirements of an organization.
|
||||
|
||||
- **Extensive Compliance Support:**
|
||||
- Prowler supports over 27 compliance frameworks out of the box, with capabilities to extend these frameworks to GCP environments as well as other cloud platforms. This broad compliance coverage ensures that organizations can maintain adherence to various regulatory requirements.
|
||||
|
||||
- **Kubernetes and Multi-Cloud Support:**
|
||||
- Prowler is designed to support security assessments across cloud environments, including Kubernetes clusters and GCP. This multi-cloud capability is essential for organizations that operate across diverse cloud platforms and require consistent security posture management.
|
||||
|
||||
- **All-Region Checks:**
|
||||
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage across an organization’s cloud resources, regardless of the region or cloud provider.
|
||||
|
||||
## Comparison Summary
|
||||
|
||||
### Scope and Environment
|
||||
|
||||
- **GCP Cloud SCC** is ideal for organizations primarily using GCP, offering a centralized platform for managing security and compliance within the GCP ecosystem.
|
||||
- **Prowler** excels in multi-cloud environments, offering flexibility and comprehensive security checks across AWS, Azure, GCP, and Kubernetes without being confined to a single cloud provider.
|
||||
|
||||
### Setup and Flexibility
|
||||
|
||||
- **GCP Cloud SCC** requires enabling various GCP services and may involve more complex setup, especially for multi-region or multi-account scenarios within GCP.
|
||||
- **Prowler** requires minimal setup and can be deployed quickly across different cloud environments, offering a more straightforward approach to multi-cloud security management.
|
||||
|
||||
### Customization and Compliance
|
||||
|
||||
- **GCP Cloud SCC** provides predefined compliance checks within the GCP environment but may require additional tools or customization for broader or more specific requirements.
|
||||
- **Prowler** allows for extensive customization of security checks, compliance frameworks, and reporting, providing a flexible solution that can be tailored to an organization’s specific needs across various cloud platforms.
|
||||
|
||||
### Cost Efficiency
|
||||
|
||||
- **GCP Cloud SCC** costs can scale with the volume of data processed and the number of enabled services, which may be significant in large or complex environments. SCC pricing is confusing to understand, and starts at $0.071 per vCPU hour for some tiers and depending on the scan service. Take a look at the pricing model [here](https://cloud.google.com/security-command-center/pricing), godspeed.
|
||||
- **Prowler** helps manage costs by allowing selective reporting, such as sending only failed findings to SIEMs, and storing results in cost-effective ways, such as local storage or cloud buckets. Prowler is always $0.001 per resource per day - no per account charge.
|
||||
|
||||
### Multi-Cloud and Multi-Region Support
|
||||
|
||||
- **GCP Cloud SCC** is focused on GCP and may require additional tools for comprehensive multi-cloud support.
|
||||
- **Prowler** is inherently multi-cloud, supporting AWS, Azure, GCP, and Kubernetes out of the box, making it an ideal choice for organizations with diverse cloud footprints.
|
||||
|
||||
## Conclusion
|
||||
|
||||
For a CISO evaluating these tools, the decision between GCP Cloud Security Command Center (Cloud SCC) and Prowler hinges on the organization’s cloud strategy, security management needs, and the level of flexibility and multi-cloud support required:
|
||||
|
||||
- If the organization is heavily invested in GCP and needs a centralized platform that integrates seamlessly with GCP services for asset management, threat detection, and compliance monitoring, **GCP Cloud SCC** is likely the better choice.
|
||||
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
title: 'Comparison'
|
||||
---
|
||||
|
||||
Click to learn more about each cloud security provider and learn how Prowler is differentiated.
|
||||
|
||||
- [AWS Security Hub](/getting-started/comparison/awssecurityhub)
|
||||
- [Microsoft Sentinel](/getting-started/comparison/microsoftsentinel)
|
||||
- [Microsoft Defender](/getting-started/comparison/microsoftdefender)
|
||||
- [Google Cloud Security Command Center](/getting-started/comparison/gcp)
|
||||
@@ -0,0 +1,101 @@
|
||||
---
|
||||
title: 'Microsoft Defender for Cloud'
|
||||
---
|
||||
|
||||
**Use open-source scanning to validate and extend Microsoft Defender for Cloud**
|
||||
|
||||
---
|
||||
|
||||
## **Overview**
|
||||
|
||||
If you're using Microsoft Defender for Cloud to monitor your Azure infrastructure, Prowler can complement it with fully transparent, customizable scans across Azure, AWS, GCP, and Kubernetes. Prowler helps you validate policies, automate compliance, and gain deeper visibility—all from the CLI, API or our Prowler UI.
|
||||
|
||||
You can run Prowler alongside Defender for Cloud to:
|
||||
|
||||
* Double-check security posture with open-source checks.
|
||||
* Customize rules for your organization’s policies.
|
||||
* Bring your own, or community contributed policies.
|
||||
* Automate multi-cloud scans in CI/CD or scheduled jobs.
|
||||
|
||||
---
|
||||
|
||||
## **Why use Prowler with Defender for Cloud**
|
||||
|
||||
Microsoft Defender for Cloud offers centralized dashboards, alerting, and some cross-cloud coverage. Prowler provides full transparency and control over what’s being checked and how those checks work—no vendor lock-in, no surprises.
|
||||
|
||||
Use them together to get:
|
||||
|
||||
* More confidence in your security posture
|
||||
* Checks you can inspect, modify, and version
|
||||
* CLI-first, portable scanning across clouds
|
||||
* Open-source tooling that integrates easily into pipelines and audits
|
||||
|
||||
---
|
||||
|
||||
## **Quickstart**
|
||||
|
||||
Here’s how to install Prowler and run a scan in your Azure account.
|
||||
|
||||
### **1\. Install Prowler**
|
||||
|
||||
```
|
||||
git clone https://github.com/prowler-cloud/prowler
|
||||
cd prowler
|
||||
./install.sh
|
||||
```
|
||||
|
||||
### **2\. Authenticate with Azure**
|
||||
|
||||
Make sure you're signed in and select your subscription:
|
||||
|
||||
```
|
||||
az login
|
||||
export AZURE_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
|
||||
```
|
||||
|
||||
### **3\. Run a scan**
|
||||
|
||||
```
|
||||
./prowler -p Azure -f az-aks -f az-general
|
||||
```
|
||||
|
||||
This will run checks focused on Azure Kubernetes Service (AKS) and general Azure best practices.
|
||||
|
||||
### **4\. Review results**
|
||||
|
||||
```
|
||||
cat output/prowler-output-*.json
|
||||
open output/prowler-output-*.html
|
||||
```
|
||||
|
||||
You can export findings in JSON, CSV, JUnit, HTML, or AWS Security Hub–compatible formats.
|
||||
|
||||
---
|
||||
|
||||
## **Compare capabilities**
|
||||
|
||||
| Feature | Microsoft Defender for Cloud | Prowler |
|
||||
| ----- | ----- | ----- |
|
||||
| Azure-native posture management | ✅ | ✅ |
|
||||
| AWS, GCP, and Kubernetes support | ⚠️ (limited) | ✅ |
|
||||
| Custom policy creation | ❌ | ✅ |
|
||||
| CLI-first, scriptable | ❌ | ✅ |
|
||||
| Open source | ❌ | ✅ |
|
||||
| Compliance mappings (CIS, NIST, etc.) | ✅ (limited control) | ✅ (customizable) |
|
||||
| Exportable detections | ❌ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## **Common use cases**
|
||||
|
||||
**✅ Validate policies**
|
||||
Run Prowler to confirm your Azure policies are configured as expected and compliant with frameworks like CIS or NIST.
|
||||
|
||||
**✅ Automate compliance scans**
|
||||
Schedule regular Prowler scans in your CI/CD pipeline or infrastructure monitoring workflows. Generate reports for auditors or internal reviews.
|
||||
|
||||
**✅ Extend detection coverage**
|
||||
If Defender for Cloud doesn’t cover all the services or resources in your environment, Prowler’s checks fill in the gaps.
|
||||
|
||||
**✅ Build custom checks**
|
||||
Security is never one-size-fits-all. Prowler lets you write your own checks for organization-specific policies.
|
||||
@@ -0,0 +1,93 @@
|
||||
---
|
||||
title: 'Microsoft Sentinel'
|
||||
---
|
||||
|
||||
Microsoft Sentinel is a scalable, cloud-native security information and event management (SIEM) and security orchestration automated response (SOAR) solution. It's designed to collect, detect, investigate, and respond to threats across the enterprise, primarily within the Azure cloud environment but also extending to on-premises and other cloud environments through various connectors.
|
||||
|
||||
## Key Features and Strengths
|
||||
|
||||
- **SIEM and SOAR Capabilities:** Microsoft Sentinel combines SIEM and SOAR functionalities, allowing it to collect and analyze large volumes of data from various sources and automate responses to detected threats.
|
||||
|
||||
- **Native Azure Integration:** As part of the Azure ecosystem, Sentinel integrates seamlessly with Azure services, providing deep visibility and analytics for Azure resources.
|
||||
|
||||
- **Advanced Threat Detection:** Sentinel uses AI and machine learning to detect potential threats and anomalous activities, leveraging Microsoft's extensive threat intelligence network.
|
||||
|
||||
- **Scalability and Flexibility:** Being cloud-native, Sentinel scales automatically to handle increasing data volumes and complexity without requiring extensive infrastructure management.
|
||||
|
||||
- **Customizable Dashboards and Analytics:** Sentinel offers customizable dashboards and analytics, allowing security teams to tailor their views and queries to specific needs.
|
||||
|
||||
- **Multi-Source Data Ingestion:** While focused on Azure, Sentinel can ingest data from multiple sources, including AWS, GCP, on-premises environments, and third-party security products.
|
||||
|
||||
## Limitations
|
||||
|
||||
- **Azure-Centric:** While it supports multi-cloud environments, its primary focus and strengths are within the Azure ecosystem. Integration with other cloud platforms and on-premises environments may require additional connectors and configurations.
|
||||
|
||||
- **Cost Considerations:** As a SIEM tool, Sentinel can become expensive, particularly as data ingestion and analysis volumes grow. The cost model is based on data volume, which can add up quickly in large environments.
|
||||
|
||||
- **Complexity in Customization:** Although Sentinel offers advanced customization, setting up and fine-tuning these customizations can require significant expertise and effort, particularly in multi-cloud environments.
|
||||
|
||||
## Prowler
|
||||
|
||||
Prowler is an open-source, multi-cloud security tool that offers extensive flexibility and customization, making it ideal for organizations that need to maintain a strong security posture across diverse cloud environments. Here are the key advantages of Prowler, particularly when compared to Microsoft Sentinel:
|
||||
|
||||
## Main Advantages of Prowler
|
||||
|
||||
- **Multi-Region and Multi-Account Scanning by Default:**
|
||||
- Prowler is inherently multi-region and multi-account, requiring no additional configuration to scan across these environments. This capability is available out of the box without needing to enable specific services or create complex setups.
|
||||
|
||||
- **Minimal Setup Requirements:**
|
||||
- Prowler requires only a role with appropriate permissions to begin scanning. There’s no need for extensive setup, making it easier and quicker to deploy across various environments.
|
||||
|
||||
- **Versatile Execution Environment:**
|
||||
- Prowler can be run from a local workstation, container, AWS CloudShell, or even from other cloud providers like Azure or GCP by assuming a role. This versatility allows security teams to integrate Prowler into a wide range of operational workflows without being tied to a single cloud environment.
|
||||
|
||||
- **Flexible Results Storage and Sharing:**
|
||||
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis or locally for easy sharing and collaboration. This flexibility is particularly useful for multi-cloud security assessments and incident response.
|
||||
|
||||
- **Customizable Reporting and Analysis:**
|
||||
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. Additionally, it can integrate with Amazon QuickSight for advanced analytics, and offers a SaaS model with resource-based pricing, making it adaptable to various organizational needs.
|
||||
|
||||
- **SIEM Integration and Cost Efficiency:**
|
||||
- While Microsoft Sentinel has a built-in SIEM functionality, Prowler can send results directly into SIEM systems, including Microsoft Sentinel. By sending only failed findings, Prowler can help optimize costs associated with data ingestion and storage in SIEM platforms.
|
||||
|
||||
- **Custom Checks and Compliance Frameworks:**
|
||||
- Prowler enables users to write custom checks, remediations, and compliance frameworks quickly, allowing organizations to adapt the tool to their specific security policies and regulatory requirements.
|
||||
|
||||
- **Extensive Compliance Support:**
|
||||
- Prowler supports over 27 compliance frameworks out of the box, providing comprehensive coverage for AWS environments, which can be extended to multi-cloud scenarios.
|
||||
|
||||
- **Kubernetes and Multi-Cloud Support:**
|
||||
- Prowler is designed to support security assessments beyond AWS, including Kubernetes clusters (including EKS) and environments in Azure and GCP. This capability is critical for organizations that operate across multiple cloud platforms and require consistent security posture management.
|
||||
|
||||
- **All-Region Checks:**
|
||||
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage without the limitations that may be imposed by region-specific configurations or services.
|
||||
|
||||
## Comparison Summary
|
||||
|
||||
### Scope and Environment
|
||||
- **Microsoft Sentinel** is an advanced SIEM/SOAR tool optimized for Azure environments, with support for multi-cloud and on-premises systems through connectors.
|
||||
- **Prowler** is a flexible, multi-cloud security tool that excels in environments where organizations need to manage security across AWS, Azure, GCP, and Kubernetes with minimal setup and high customizability.
|
||||
|
||||
### Setup and Flexibility
|
||||
- **Microsoft Sentinel** requires more setup, especially when integrating with non-Azure environments, and its cost scales with data ingestion.
|
||||
- **Prowler** requires minimal setup and can be easily deployed in any cloud or on-premises environment. Its ability to run from various environments and store results in flexible formats makes it particularly adaptable.
|
||||
|
||||
### Customization and Compliance
|
||||
- **Microsoft Sentinel** offers powerful but complex customization options, primarily within the Azure ecosystem.
|
||||
- **Prowler** provides straightforward customization of security checks, remediation actions, and compliance frameworks, with broad support for multiple compliance standards out of the box.
|
||||
|
||||
### Cost Efficiency
|
||||
- **Microsoft Sentinel** can become costly as data volumes grow, particularly in large or multi-cloud environments.
|
||||
- **Prowler** helps control costs by enabling selective reporting (e.g., sending only failed findings to SIEMs like Sentinel) and storing results in cost-effective ways, such as S3 or locally.
|
||||
|
||||
### Multi-Cloud and Multi-Region Support
|
||||
- **Microsoft Sentinel** supports multi-cloud environments but may require additional configuration and connectors.
|
||||
- **Prowler** is designed for multi-cloud environments from the ground up, with inherent support for AWS, Azure, GCP, Kubernetes, and all regions, making it an ideal tool for organizations with diverse cloud footprints.
|
||||
|
||||
## Conclusion
|
||||
|
||||
For a CISO or security professional evaluating these tools, the decision between Microsoft Sentinel and Prowler hinges on the organization's cloud strategy, SIEM needs, and the level of customization and flexibility required:
|
||||
|
||||
- If the organization is heavily invested in Azure and needs an integrated SIEM/SOAR solution with advanced threat detection, analytics, and automation capabilities, **Microsoft Sentinel** is likely the better choice.
|
||||
|
||||
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: "Prowler API Reference"
|
||||
url: "https://api.prowler.com/api/v1/docs"
|
||||
---
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: "Go to Cloud"
|
||||
url: "https://cloud.prowler.com"
|
||||
---
|
||||
@@ -0,0 +1,4 @@
|
||||
---
|
||||
title: "Go to Hub"
|
||||
url: "https://hub.prowler.com"
|
||||
---
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
title: "Prowler MCP"
|
||||
url: "https://github.com/prowler-cloud/prowler/tree/master/mcp_server"
|
||||
tag: "new!"
|
||||
---
|
||||
+30
-36
@@ -1,14 +1,19 @@
|
||||
---
|
||||
title: 'Installation'
|
||||
---
|
||||
|
||||
### Installation
|
||||
|
||||
Prowler App supports multiple installation methods based on your environment.
|
||||
|
||||
Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed usage instructions.
|
||||
Refer to the [Prowler App Tutorial](/user-guide/tutorials/prowler-app) for detailed usage instructions.
|
||||
|
||||
???+ warning
|
||||
Prowler configuration is based in `.env` files. Every version of Prowler can have differences on that file, so, please, use the file that corresponds with that version or repository branch or tag.
|
||||
|
||||
=== "Docker Compose"
|
||||
<Warning>
|
||||
Prowler configuration is based in `.env` files. Every version of Prowler can have differences on that file, so, please, use the file that corresponds with that version or repository branch or tag.
|
||||
|
||||
</Warning>
|
||||
<Tabs>
|
||||
<Tab title="Docker Compose">
|
||||
_Requirements_:
|
||||
|
||||
* `Docker Compose` installed: https://docs.docker.com/compose/install/.
|
||||
@@ -20,25 +25,8 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
|
||||
curl -LO https://raw.githubusercontent.com/prowler-cloud/prowler/refs/heads/master/.env
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
> Containers are built for `linux/amd64`. If your workstation's architecture is different, please set `DOCKER_DEFAULT_PLATFORM=linux/amd64` in your environment or use the `--platform linux/amd64` flag in the docker command.
|
||||
|
||||
> Enjoy Prowler App at http://localhost:3000 by signing up with your email and password.
|
||||
|
||||
???+ note
|
||||
You can change the environment variables in the `.env` file. Note that it is not recommended to use the default values in production environments.
|
||||
|
||||
???+ note
|
||||
For a secure setup, leave empty or remove `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY` in `.env` before first start. When absent, the API auto‑generates a unique key pair and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate, delete the stored key files and restart the API.
|
||||
|
||||
???+ note
|
||||
There is a development mode available, you can use the file https://github.com/prowler-cloud/prowler/blob/master/docker-compose-dev.yml to run the app in development mode.
|
||||
|
||||
???+ warning
|
||||
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
|
||||
|
||||
=== "GitHub"
|
||||
|
||||
</Tab>
|
||||
<Tab title="GitHub">
|
||||
_Requirements_:
|
||||
|
||||
* `git` installed.
|
||||
@@ -46,8 +34,9 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
|
||||
* `npm` installed: [npm installation](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).
|
||||
* `Docker Compose` installed: https://docs.docker.com/compose/install/.
|
||||
|
||||
???+ warning
|
||||
Make sure to have `api/.env` and `ui/.env.local` files with the required environment variables. You can find the required environment variables in the [`api/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/api/.env.example) and [`ui/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/ui/.env.template) files.
|
||||
<Warning>
|
||||
Make sure to have `api/.env` and `ui/.env.local` files with the required environment variables. You can find the required environment variables in the [`api/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/api/.env.example) and [`ui/.env.template`](https://github.com/prowler-cloud/prowler/blob/master/ui/.env.template) files.
|
||||
</Warning>
|
||||
|
||||
_Commands to run the API_:
|
||||
|
||||
@@ -64,11 +53,12 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
|
||||
gunicorn -c config/guniconf.py config.wsgi:application
|
||||
```
|
||||
|
||||
???+ important
|
||||
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
|
||||
<Warning>
|
||||
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
|
||||
|
||||
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
|
||||
In case you have any doubts, consult the Poetry environment activation guide: https://python-poetry.org/docs/managing-environments/#activating-the-environment
|
||||
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
|
||||
In case you have any doubts, consult the Poetry environment activation guide: https://python-poetry.org/docs/managing-environments/#activating-the-environment
|
||||
</Warning>
|
||||
|
||||
> Now, you can access the API documentation at http://localhost:8080/api/v1/docs.
|
||||
|
||||
@@ -110,10 +100,11 @@ Refer to the [Prowler App Tutorial](../tutorials/prowler-app.md) for detailed us
|
||||
|
||||
> Enjoy Prowler App at http://localhost:3000 by signing up with your email and password.
|
||||
|
||||
???+ warning
|
||||
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
|
||||
|
||||
|
||||
<Warning>
|
||||
Google and GitHub authentication is only available in [Prowler Cloud](https://prowler.com).
|
||||
</Warning>
|
||||
</Tab>
|
||||
</Tabs>
|
||||
### Update Prowler App
|
||||
|
||||
Upgrade Prowler App installation using one of two options:
|
||||
@@ -136,9 +127,12 @@ docker compose pull --policy always
|
||||
The `--policy always` flag ensures that Docker pulls the latest images even if they already exist locally.
|
||||
|
||||
|
||||
???+ note "What Gets Preserved During Upgrade"
|
||||
Everything is preserved, nothing will be deleted after the update.
|
||||
<Note>
|
||||
**What Gets Preserved During Upgrade**
|
||||
|
||||
Everything is preserved, nothing will be deleted after the update.
|
||||
|
||||
</Note>
|
||||
### Troubleshooting
|
||||
|
||||
If containers don't start, check logs for errors:
|
||||
+32
-49
@@ -1,10 +1,13 @@
|
||||
---
|
||||
title: 'Installation'
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/). Install it as a Python package with `Python >= 3.9, <= 3.12`:
|
||||
|
||||
=== "pipx"
|
||||
|
||||
<Tabs>
|
||||
<Tab title="pipx">
|
||||
[pipx](https://pipx.pypa.io/stable/) installs Python applications in isolated environments. Use `pipx` for global installation.
|
||||
|
||||
_Requirements_:
|
||||
@@ -19,17 +22,11 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
pipx install prowler
|
||||
prowler -v
|
||||
```
|
||||
|
||||
Upgrade Prowler to the latest version:
|
||||
|
||||
``` bash
|
||||
pipx upgrade prowler
|
||||
```
|
||||
|
||||
=== "pip"
|
||||
|
||||
???+ warning
|
||||
This method modifies the chosen installation environment. Consider using [pipx](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) for global installation.
|
||||
</Tab>
|
||||
<Tab title="pip">
|
||||
<Warning>
|
||||
This method modifies the chosen installation environment. Consider using [pipx](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) for global installation.
|
||||
</Warning>
|
||||
|
||||
_Requirements_:
|
||||
|
||||
@@ -49,9 +46,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
``` bash
|
||||
pip install --upgrade prowler
|
||||
```
|
||||
|
||||
=== "Docker"
|
||||
|
||||
</Tab>
|
||||
<Tab title="Docker">
|
||||
_Requirements_:
|
||||
|
||||
* Have `docker` installed: https://docs.docker.com/get-docker/.
|
||||
@@ -69,9 +65,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
--env AWS_SECRET_ACCESS_KEY \
|
||||
--env AWS_SESSION_TOKEN toniblyx/prowler:latest
|
||||
```
|
||||
|
||||
=== "GitHub"
|
||||
|
||||
</Tab>
|
||||
<Tab title="GitHub">
|
||||
_Requirements for Developers_:
|
||||
|
||||
* `git`
|
||||
@@ -86,11 +81,12 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
poetry install
|
||||
poetry run python prowler-cli.py -v
|
||||
```
|
||||
???+ note
|
||||
If you want to clone Prowler from Windows, use `git config core.longpaths true` to allow long file paths.
|
||||
|
||||
=== "Amazon Linux 2"
|
||||
|
||||
<Note>
|
||||
If you want to clone Prowler from Windows, use `git config core.longpaths true` to allow long file paths.
|
||||
</Note>
|
||||
</Tab>
|
||||
<Tab title="Amazon Linux 2">
|
||||
_Requirements_:
|
||||
|
||||
* `Python >= 3.9, <= 3.12`
|
||||
@@ -104,9 +100,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
pipx install prowler
|
||||
prowler -v
|
||||
```
|
||||
|
||||
=== "Ubuntu"
|
||||
|
||||
</Tab>
|
||||
<Tab title="Ubuntu">
|
||||
_Requirements_:
|
||||
|
||||
* `Ubuntu 23.04` or above, if you are using an older version of Ubuntu check [pipx installation](https://docs.prowler.com/projects/prowler-open-source/en/latest/#__tabbed_1_1) and ensure you have `Python >= 3.9, <= 3.12`.
|
||||
@@ -122,9 +117,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
pipx install prowler
|
||||
prowler -v
|
||||
```
|
||||
|
||||
=== "Brew"
|
||||
|
||||
</Tab>
|
||||
<Tab title="Brew">
|
||||
_Requirements_:
|
||||
|
||||
* `Brew` installed in your Mac or Linux
|
||||
@@ -136,9 +130,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
brew install prowler
|
||||
prowler -v
|
||||
```
|
||||
|
||||
=== "AWS CloudShell"
|
||||
|
||||
</Tab>
|
||||
<Tab title="AWS CloudShell">
|
||||
After the migration of AWS CloudShell from Amazon Linux 2 to Amazon Linux 2023 [[1]](https://aws.amazon.com/about-aws/whats-new/2023/12/aws-cloudshell-migrated-al2023/) [[2]](https://docs.aws.amazon.com/cloudshell/latest/userguide/cloudshell-AL2023-migration.html), there is no longer a need to manually compile Python 3.9 as it is already included in AL2023. Prowler can thus be easily installed following the generic method of installation via pip. Follow the steps below to successfully execute Prowler v4 in AWS CloudShell:
|
||||
|
||||
_Requirements_:
|
||||
@@ -158,11 +151,11 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
prowler aws
|
||||
```
|
||||
|
||||
???+ note
|
||||
To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/tmp/output/prowler-output-123456789012-20221220191331.csv`
|
||||
|
||||
=== "Azure CloudShell"
|
||||
|
||||
<Note>
|
||||
To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/tmp/output/prowler-output-123456789012-20221220191331.csv`
|
||||
</Note>
|
||||
</Tab>
|
||||
<Tab title="Azure CloudShell">
|
||||
_Requirements_:
|
||||
|
||||
* Open Azure CloudShell `bash`.
|
||||
@@ -176,18 +169,8 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler/).
|
||||
cd /tmp
|
||||
prowler azure --az-cli-auth
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
## Container versions
|
||||
|
||||
The available versions of Prowler CLI are the following:
|
||||
@@ -0,0 +1,276 @@
|
||||
---
|
||||
title: "Installation"
|
||||
---
|
||||
|
||||
There are **two ways** to use Prowler MCP Server:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Option 1: Managed by Prowler" icon="cloud" color="#10B981">
|
||||
**No installation required** - Just configuration
|
||||
|
||||
Use `https://mcp.prowler.com/mcp`
|
||||
</Card>
|
||||
<Card title="Option 2: Run Locally" icon="server" color="#6366F1">
|
||||
**Local installation** - Full control
|
||||
|
||||
Install via Docker, PyPI, or source code
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
|
||||
For "Option 1: Managed by Prowler", go directly to the [Configuration Guide](/getting-started/basic-usage/prowler-mcp#hosted-server-configuration-recommended) to set up your Claude Desktop, Cursor, or other MCP client.
|
||||
**This guide is focused on local installation, "Option 2: Run Locally"**.
|
||||
|
||||
## Installation Methods
|
||||
|
||||
Choose one of the following installation methods:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Docker (Recommended)">
|
||||
### Pull from Docker Hub
|
||||
|
||||
The easiest way to run locally is using the official Docker image:
|
||||
|
||||
```bash
|
||||
docker pull prowlercloud/prowler-mcp
|
||||
```
|
||||
|
||||
### Run the Container
|
||||
|
||||
```bash
|
||||
# STDIO mode (for local MCP clients)
|
||||
docker run --rm -i prowlercloud/prowler-mcp
|
||||
|
||||
# HTTP mode (for remote access)
|
||||
docker run --rm -p 8000:8000 \
|
||||
prowlercloud/prowler-mcp \
|
||||
--transport http --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
### With Environment Variables
|
||||
|
||||
```bash
|
||||
docker run --rm -i \
|
||||
-e PROWLER_APP_API_KEY="pk_your_api_key" \
|
||||
-e PROWLER_API_BASE_URL="https://api.prowler.com" \
|
||||
prowlercloud/prowler-mcp
|
||||
```
|
||||
|
||||
<Info>
|
||||
**Docker Hub:** [prowlercloud/prowler-mcp](https://hub.docker.com/r/prowlercloud/prowler-mcp)
|
||||
</Info>
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="PyPI Package">
|
||||
### Install via pip
|
||||
|
||||
<Warning>
|
||||
**Coming Soon** - PyPI package will be available shortly
|
||||
</Warning>
|
||||
</Tab>
|
||||
|
||||
<Tab title="From Source (Development)">
|
||||
### Install uv
|
||||
|
||||
If `uv` is not installed, install it first. Visit [uv documentation](https://docs.astral.sh/uv/) for more installation options.
|
||||
|
||||
### Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/prowler-cloud/prowler.git
|
||||
cd prowler/mcp_server
|
||||
```
|
||||
|
||||
### Install Dependencies
|
||||
|
||||
The MCP server uses `uv` for dependency management. Install dependencies with:
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
```
|
||||
|
||||
### Verify Installation
|
||||
|
||||
Test that the server is properly installed:
|
||||
|
||||
```bash
|
||||
uv run prowler-mcp --help
|
||||
```
|
||||
|
||||
The help message with available command-line options should appear.
|
||||
|
||||
<Note>
|
||||
**For Development:** This method is recommended if you're developing or contributing to the MCP server.
|
||||
</Note>
|
||||
|
||||
</Tab>
|
||||
|
||||
<Tab title="Build Docker Image">
|
||||
### Prerequisites
|
||||
|
||||
Ensure Docker is installed on your system. Visit [Docker documentation](https://docs.docker.com/get-docker/) for more installation options.
|
||||
|
||||
### Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/prowler-cloud/prowler.git
|
||||
cd prowler/mcp_server
|
||||
```
|
||||
|
||||
### Build the Docker Image
|
||||
|
||||
```bash
|
||||
docker build -t prowler-mcp .
|
||||
```
|
||||
|
||||
This creates a Docker image named `prowler-mcp` with all necessary dependencies.
|
||||
|
||||
### Verify Installation
|
||||
|
||||
Test that the Docker image was built successfully:
|
||||
|
||||
```bash
|
||||
docker run --rm prowler-mcp --help
|
||||
```
|
||||
|
||||
The help message with available command-line options should appear.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
---
|
||||
|
||||
## Command Line Options
|
||||
|
||||
The Prowler MCP Server supports the following command-line arguments:
|
||||
|
||||
```bash
|
||||
prowler-mcp [--transport {stdio,http}] [--host HOST] [--port PORT]
|
||||
```
|
||||
|
||||
| Argument | Values | Default | Description |
|
||||
|----------|--------|---------|-------------|
|
||||
| `--transport` | `stdio`, `http` | `stdio` | Transport method for MCP communication |
|
||||
| `--host` | Any valid hostname/IP | `127.0.0.1` | Host to bind to (HTTP mode only) |
|
||||
| `--port` | Port number | `8000` | Port to bind to (HTTP mode only) |
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
# Default STDIO mode for local MCP client integration
|
||||
prowler-mcp
|
||||
|
||||
# Explicit STDIO mode
|
||||
prowler-mcp --transport stdio
|
||||
|
||||
# HTTP mode on default host and port (127.0.0.1:8000)
|
||||
prowler-mcp --transport http
|
||||
|
||||
# HTTP mode accessible from any network interface
|
||||
prowler-mcp --transport http --host 0.0.0.0
|
||||
|
||||
# HTTP mode with custom port
|
||||
prowler-mcp --transport http --host 0.0.0.0 --port 8080
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Configure the server using environment variables:
|
||||
|
||||
| Variable | Description | Required | Default |
|
||||
|----------|-------------|----------|---------|
|
||||
| `PROWLER_APP_API_KEY` | Prowler API key | Only for STDIO mode | - |
|
||||
| `PROWLER_API_BASE_URL` | Custom Prowler API endpoint | No | `https://api.prowler.com` |
|
||||
| `PROWLER_MCP_TRANSPORT_MODE` | Default transport mode (overwritten by `--transport` argument) | No | `stdio` |
|
||||
|
||||
<CodeGroup>
|
||||
```bash macOS/Linux
|
||||
export PROWLER_APP_API_KEY="pk_your_api_key_here"
|
||||
export PROWLER_API_BASE_URL="https://api.prowler.com"
|
||||
export PROWLER_MCP_TRANSPORT_MODE="http"
|
||||
```
|
||||
|
||||
```bash Windows PowerShell
|
||||
$env:PROWLER_APP_API_KEY="pk_your_api_key_here"
|
||||
$env:PROWLER_API_BASE_URL="https://api.prowler.com"
|
||||
$env:PROWLER_MCP_TRANSPORT_MODE="http"
|
||||
```
|
||||
</CodeGroup>
|
||||
|
||||
<Warning>
|
||||
Never commit your API key to version control. Use environment variables or secure secret management solutions.
|
||||
</Warning>
|
||||
|
||||
### Using Environment Files
|
||||
|
||||
For convenience, create a `.env` file in the `mcp_server` directory:
|
||||
|
||||
```bash .env
|
||||
PROWLER_APP_API_KEY=pk_your_api_key_here
|
||||
PROWLER_API_BASE_URL=https://api.prowler.com
|
||||
PROWLER_MCP_TRANSPORT_MODE=stdio
|
||||
```
|
||||
|
||||
When using Docker, pass the environment file:
|
||||
|
||||
```bash
|
||||
docker run --rm --env-file .env -it prowler-mcp
|
||||
```
|
||||
|
||||
## Running from Any Location
|
||||
|
||||
Run the MCP server from anywhere using `uvx`:
|
||||
|
||||
```bash
|
||||
uvx /path/to/prowler/mcp_server/
|
||||
```
|
||||
|
||||
This is particularly useful when configuring MCP clients that need to launch the server from a specific path.
|
||||
|
||||
## Production Deployment
|
||||
|
||||
For production deployments that require customization, it is recommended to use the ASGI application that can be found in `prowler_mcp_server.server`. This can be run with uvicorn:
|
||||
|
||||
```bash
|
||||
uvicorn prowler_mcp_server.server:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
For more details on production deployment options, see the [FastMCP production deployment guide](https://gofastmcp.com/deployment/http#production-deployment) and [uvicorn settings](https://www.uvicorn.org/settings/).
|
||||
|
||||
### Entrypoint Script
|
||||
|
||||
The source tree includes `entrypoint.sh` to simplify switching between the
|
||||
standard CLI runner and the ASGI app. The first argument selects the mode and
|
||||
any additional flags are passed straight through:
|
||||
|
||||
```bash
|
||||
# Default CLI experience (prowler-mcp console script)
|
||||
./entrypoint.sh main --transport http --host 0.0.0.0
|
||||
|
||||
# ASGI app via uvicorn
|
||||
./entrypoint.sh uvicorn --host 0.0.0.0 --port 9000
|
||||
```
|
||||
|
||||
Omitting the mode defaults to `main`, matching the `prowler-mcp` console script.
|
||||
When `uvicorn` mode is selected, the script exports `PROWLER_MCP_TRANSPORT_MODE=http` automatically.
|
||||
|
||||
This is the default entrypoint for the Docker container.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that you have the Prowler MCP Server installed, proceed to configure your MCP client:
|
||||
|
||||
<CardGroup cols={1}>
|
||||
<Card title="Configuration" icon="gear" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configure Claude Desktop, Cursor, or other MCP clients
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
If you encounter issues during installation:
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
|
Before Width: | Height: | Size: 313 KiB After Width: | Height: | Size: 313 KiB |
|
Before Width: | Height: | Size: 420 KiB After Width: | Height: | Size: 420 KiB |
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user