Compare commits

..

2 Commits

Author SHA1 Message Date
Alan Buscaglia 8e40c2ceb1 refactor(ui): move modal buttons to shared UI components
Move modal-buttons.tsx from api-keys feature to shared UI custom
components as custom-modal-buttons.tsx for reusability across the app.
Update import in revoke-api-key-modal.
2025-10-14 11:30:53 +02:00
Alan Buscaglia 71aec6aede refactor(api-keys): enhance UI components and validation
Improve API key management UI with better component architecture:
- Migrate to FormButtons component from form library for consistency
- Add conditional revoke button (hide for revoked/expired keys)
- Improve modal button styling with icon support
- Update column headers to sentence case for better readability
- Add AlertTitle support in Alert component
- Enhance form validation feedback across all modals
2025-10-14 11:30:53 +02:00
1498 changed files with 5335 additions and 81954 deletions
+5 -27
View File
@@ -1,28 +1,6 @@
# SDK
/* @prowler-cloud/sdk
/prowler/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
/tests/ @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
/dashboard/ @prowler-cloud/sdk
/docs/ @prowler-cloud/sdk
/examples/ @prowler-cloud/sdk
/util/ @prowler-cloud/sdk
/contrib/ @prowler-cloud/sdk
/permissions/ @prowler-cloud/sdk
/codecov.yml @prowler-cloud/sdk @prowler-cloud/api
# API
/api/ @prowler-cloud/api
# UI
/ui/ @prowler-cloud/ui
# AI
/mcp_server/ @prowler-cloud/ai
# Platform
/.github/ @prowler-cloud/platform
/Makefile @prowler-cloud/platform
/kubernetes/ @prowler-cloud/platform
**/Dockerfile* @prowler-cloud/platform
**/docker-compose*.yml @prowler-cloud/platform
**/docker-compose*.yaml @prowler-cloud/platform
/.github/ @prowler-cloud/sdk
prowler @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
tests @prowler-cloud/sdk @prowler-cloud/detection-and-remediation
api @prowler-cloud/api
ui @prowler-cloud/ui
-44
View File
@@ -3,41 +3,6 @@ description: Create a report to help us improve
labels: ["bug", "status/needs-triage"]
body:
- type: checkboxes
id: search
attributes:
label: Issue search
options:
- label: I have searched the existing issues and this bug has not been reported yet
required: true
- type: dropdown
id: component
attributes:
label: Which component is affected?
multiple: true
options:
- Prowler CLI/SDK
- Prowler API
- Prowler UI
- Prowler Dashboard
- Prowler MCP Server
- Documentation
- Other
validations:
required: true
- type: dropdown
id: provider
attributes:
label: Cloud Provider (if applicable)
multiple: true
options:
- AWS
- Azure
- GCP
- Kubernetes
- GitHub
- Microsoft 365
- Not applicable
- type: textarea
id: reproduce
attributes:
@@ -113,15 +78,6 @@ body:
prowler --version
validations:
required: true
- type: input
id: python-version
attributes:
label: Python version
description: Which Python version are you using?
placeholder: |-
python --version
validations:
required: true
- type: input
id: pip-version
attributes:
-10
View File
@@ -1,11 +1 @@
blank_issues_enabled: false
contact_links:
- name: 📖 Documentation
url: https://docs.prowler.com
about: Check our comprehensive documentation for guides and tutorials
- name: 💬 GitHub Discussions
url: https://github.com/prowler-cloud/prowler/discussions
about: Ask questions and discuss with the community
- name: 🌟 Prowler Community
url: https://goto.prowler.com/slack
about: Join our community for support and updates
@@ -3,42 +3,6 @@ description: Suggest an idea for this project
labels: ["feature-request", "status/needs-triage"]
body:
- type: checkboxes
id: search
attributes:
label: Feature search
options:
- label: I have searched the existing issues and this feature has not been requested yet or is already in our [Public Roadmap](https://roadmap.prowler.com/roadmap)
required: true
- type: dropdown
id: component
attributes:
label: Which component would this feature affect?
multiple: true
options:
- Prowler CLI/SDK
- Prowler API
- Prowler UI
- Prowler Dashboard
- Prowler MCP Server
- Documentation
- New component/Integration
validations:
required: true
- type: dropdown
id: provider
attributes:
label: Related to specific cloud provider?
multiple: true
options:
- AWS
- Azure
- GCP
- Kubernetes
- GitHub
- Microsoft 365
- All providers
- Not provider-specific
- type: textarea
id: Problem
attributes:
@@ -55,14 +19,6 @@ body:
description: A clear and concise description of what you want to happen.
validations:
required: true
- type: textarea
id: use-case
attributes:
label: Use case and benefits
description: Who would benefit from this feature and how?
placeholder: This would help security teams by...
validations:
required: true
- type: textarea
id: Alternatives
attributes:
@@ -1,71 +0,0 @@
name: 'Setup Python with Poetry'
description: 'Setup Python environment with Poetry and install dependencies'
author: 'Prowler'
inputs:
python-version:
description: 'Python version to use'
required: true
working-directory:
description: 'Working directory for Poetry'
required: false
default: '.'
poetry-version:
description: 'Poetry version to install'
required: false
default: '2.1.1'
install-dependencies:
description: 'Install Python dependencies with Poetry'
required: false
default: 'true'
runs:
using: 'composite'
steps:
- name: Replace @master with current branch in pyproject.toml
if: github.event_name == 'pull_request' && github.base_ref == 'master'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
echo "Using branch: $BRANCH_NAME"
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
- name: Install poetry
shell: bash
run: |
python -m pip install --upgrade pip
pipx install poetry==${{ inputs.poetry-version }}
- name: Update SDK resolved_reference to latest commit
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
echo "Latest commit hash: $LATEST_COMMIT"
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
}' poetry.lock
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock
shell: bash
working-directory: ${{ inputs.working-directory }}
run: poetry lock
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ inputs.python-version }}
cache: 'poetry'
cache-dependency-path: ${{ inputs.working-directory }}/poetry.lock
- name: Install Python dependencies
if: inputs.install-dependencies == 'true'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
poetry install --no-root
poetry run pip list
-152
View File
@@ -1,152 +0,0 @@
name: 'Container Security Scan with Trivy'
description: 'Scans container images for vulnerabilities using Trivy and reports results'
author: 'Prowler'
inputs:
image-name:
description: 'Container image name to scan'
required: true
image-tag:
description: 'Container image tag to scan'
required: true
default: ${{ github.sha }}
severity:
description: 'Severities to scan for (comma-separated)'
required: false
default: 'CRITICAL,HIGH,MEDIUM,LOW'
fail-on-critical:
description: 'Fail the build if critical vulnerabilities are found'
required: false
default: 'false'
upload-sarif:
description: 'Upload results to GitHub Security tab'
required: false
default: 'true'
create-pr-comment:
description: 'Create a comment on the PR with scan results'
required: false
default: 'true'
artifact-retention-days:
description: 'Days to retain the Trivy report artifact'
required: false
default: '2'
outputs:
critical-count:
description: 'Number of critical vulnerabilities found'
value: ${{ steps.security-check.outputs.critical }}
high-count:
description: 'Number of high vulnerabilities found'
value: ${{ steps.security-check.outputs.high }}
total-count:
description: 'Total number of vulnerabilities found'
value: ${{ steps.security-check.outputs.total }}
runs:
using: 'composite'
steps:
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true'
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '0'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true'
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
sarif_file: 'trivy-results.sarif'
category: 'trivy-container'
- name: Run Trivy vulnerability scan (JSON)
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'json'
output: 'trivy-report.json'
severity: ${{ inputs.severity }}
exit-code: '0'
- name: Upload Trivy report artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: always()
with:
name: trivy-scan-report-${{ inputs.image-name }}
path: trivy-report.json
retention-days: ${{ inputs.artifact-retention-days }}
- name: Generate security summary
id: security-check
shell: bash
run: |
CRITICAL=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="CRITICAL")] | length' trivy-report.json)
HIGH=$(jq '[.Results[]?.Vulnerabilities[]? | select(.Severity=="HIGH")] | length' trivy-report.json)
TOTAL=$(jq '[.Results[]?.Vulnerabilities[]?] | length' trivy-report.json)
echo "critical=$CRITICAL" >> $GITHUB_OUTPUT
echo "high=$HIGH" >> $GITHUB_OUTPUT
echo "total=$TOTAL" >> $GITHUB_OUTPUT
echo "### 🔒 Container Security Scan" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image:** \`${{ inputs.image-name }}:${{ inputs.image-tag }}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "- 🔴 Critical: $CRITICAL" >> $GITHUB_STEP_SUMMARY
echo "- 🟠 High: $HIGH" >> $GITHUB_STEP_SUMMARY
echo "- **Total**: $TOTAL" >> $GITHUB_STEP_SUMMARY
- name: Comment scan results on PR
if: inputs.create-pr-comment == 'true' && github.event_name == 'pull_request'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
IMAGE_NAME: ${{ inputs.image-name }}
GITHUB_SHA: ${{ inputs.image-tag }}
SEVERITY: ${{ inputs.severity }}
with:
script: |
const comment = require('./.github/scripts/trivy-pr-comment.js');
// Unique identifier to find our comment
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
const body = marker + '\n' + comment;
// Find existing comment
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existingComment = comments.find(c => c.body?.includes(marker));
if (existingComment) {
// Update existing comment
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existingComment.id,
body: body
});
console.log('✅ Updated existing Trivy scan comment');
} else {
// Create new comment
await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: body
});
console.log('✅ Created new Trivy scan comment');
}
- name: Check for critical vulnerabilities
if: inputs.fail-on-critical == 'true' && steps.security-check.outputs.critical != '0'
shell: bash
run: |
echo "::error::Found ${{ steps.security-check.outputs.critical }} critical vulnerabilities"
echo "::warning::Please update packages or use a different base image"
exit 1
+2 -11
View File
@@ -1,12 +1,3 @@
name: 'API: CodeQL Config'
name: "API - CodeQL Config"
paths:
- 'api/'
paths-ignore:
- 'api/tests/**'
- 'api/**/__pycache__/**'
- 'api/**/migrations/**'
- 'api/**/*.md'
queries:
- uses: security-and-quality
- "api/"
-5
View File
@@ -42,11 +42,6 @@ provider/mongodbatlas:
- any-glob-to-any-file: "prowler/providers/mongodbatlas/**"
- any-glob-to-any-file: "tests/providers/mongodbatlas/**"
provider/oci:
- changed-files:
- any-glob-to-any-file: "prowler/providers/oraclecloud/**"
- any-glob-to-any-file: "tests/providers/oraclecloud/**"
github_actions:
- changed-files:
- any-glob-to-any-file: ".github/workflows/*"
-102
View File
@@ -1,102 +0,0 @@
const fs = require('fs');
// Configuration from environment variables
const REPORT_FILE = process.env.TRIVY_REPORT_FILE || 'trivy-report.json';
const IMAGE_NAME = process.env.IMAGE_NAME || 'container-image';
const GITHUB_SHA = process.env.GITHUB_SHA || 'unknown';
const GITHUB_REPOSITORY = process.env.GITHUB_REPOSITORY || '';
const GITHUB_RUN_ID = process.env.GITHUB_RUN_ID || '';
const SEVERITY = process.env.SEVERITY || 'CRITICAL,HIGH,MEDIUM,LOW';
// Parse severities to scan
const scannedSeverities = SEVERITY.split(',').map(s => s.trim());
// Read and parse the Trivy report
const report = JSON.parse(fs.readFileSync(REPORT_FILE, 'utf-8'));
let vulnCount = 0;
let vulnsByType = { CRITICAL: 0, HIGH: 0, MEDIUM: 0, LOW: 0 };
let affectedPackages = new Set();
if (report.Results && Array.isArray(report.Results)) {
for (const result of report.Results) {
if (result.Vulnerabilities && Array.isArray(result.Vulnerabilities)) {
for (const vuln of result.Vulnerabilities) {
vulnCount++;
if (vulnsByType[vuln.Severity] !== undefined) {
vulnsByType[vuln.Severity]++;
}
if (vuln.PkgName) {
affectedPackages.add(vuln.PkgName);
}
}
}
}
}
const shortSha = GITHUB_SHA.substring(0, 7);
const timestamp = new Date().toISOString().replace('T', ' ').substring(0, 19) + ' UTC';
// Severity icons and labels
const severityConfig = {
CRITICAL: { icon: '🔴', label: 'Critical' },
HIGH: { icon: '🟠', label: 'High' },
MEDIUM: { icon: '🟡', label: 'Medium' },
LOW: { icon: '🔵', label: 'Low' }
};
let comment = '## 🔒 Container Security Scan\n\n';
comment += `**Image:** \`${IMAGE_NAME}:${shortSha}\`\n`;
comment += `**Last scan:** ${timestamp}\n\n`;
if (vulnCount === 0) {
comment += '### ✅ No Vulnerabilities Detected\n\n';
comment += 'The container image passed all security checks. No known CVEs were found.\n';
} else {
comment += '### 📊 Vulnerability Summary\n\n';
comment += '| Severity | Count |\n';
comment += '|----------|-------|\n';
// Only show severities that were scanned
for (const severity of scannedSeverities) {
const config = severityConfig[severity];
const count = vulnsByType[severity] || 0;
const isBold = (severity === 'CRITICAL' || severity === 'HIGH') && count > 0;
const countDisplay = isBold ? `**${count}**` : count;
comment += `| ${config.icon} ${config.label} | ${countDisplay} |\n`;
}
comment += `| **Total** | **${vulnCount}** |\n\n`;
if (affectedPackages.size > 0) {
comment += `**${affectedPackages.size}** package(s) affected\n\n`;
}
if (vulnsByType.CRITICAL > 0) {
comment += '### ⚠️ Action Required\n\n';
comment += '**Critical severity vulnerabilities detected.** These should be addressed before merging:\n';
comment += '- Review the detailed scan results\n';
comment += '- Update affected packages to patched versions\n';
comment += '- Consider using a different base image if updates are unavailable\n\n';
} else if (vulnsByType.HIGH > 0) {
comment += '### ⚠️ Attention Needed\n\n';
comment += '**High severity vulnerabilities found.** Please review and plan remediation:\n';
comment += '- Assess the risk and exploitability\n';
comment += '- Prioritize updates in the next maintenance cycle\n\n';
} else {
comment += '### ️ Review Recommended\n\n';
comment += 'Medium/Low severity vulnerabilities found. Consider addressing during regular maintenance.\n\n';
}
}
comment += '---\n';
comment += '📋 **Resources:**\n';
if (GITHUB_REPOSITORY && GITHUB_RUN_ID) {
comment += `- [Download full report](https://github.com/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}) (see artifacts)\n`;
}
comment += '- [View in Security tab](https://github.com/' + (GITHUB_REPOSITORY || 'repository') + '/security/code-scanning)\n';
comment += '- Scanned with [Trivy](https://github.com/aquasecurity/trivy)\n';
module.exports = comment;
+33 -30
View File
@@ -1,34 +1,36 @@
name: 'API: CodeQL'
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: API - CodeQL
on:
push:
branches:
- 'master'
- 'v5.*'
- "master"
- "v5.*"
paths:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- "api/**"
pull_request:
branches:
- 'master'
- 'v5.*'
- "master"
- "v5.*"
paths:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- "api/**"
schedule:
- cron: '00 12 * * *'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
analyze:
name: CodeQL Security Analysis
name: Analyze
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
actions: read
contents: read
@@ -37,20 +39,21 @@ jobs:
strategy:
fail-fast: false
matrix:
language:
- 'python'
language: [ 'python' ]
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
category: '/language:${{ matrix.language }}'
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
category: "/language:${{matrix.language}}"
+151 -146
View File
@@ -1,30 +1,20 @@
name: 'API: Pull Request'
name: API - Pull Request
on:
push:
branches:
- 'master'
- 'v5.*'
- "master"
- "v5.*"
paths:
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
- ".github/workflows/api-pull-request.yml"
- "api/**"
pull_request:
branches:
- 'master'
- 'v5.*'
- "master"
- "v5.*"
paths:
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
- ".github/workflows/api-pull-request.yml"
- "api/**"
env:
POSTGRES_HOST: localhost
@@ -39,94 +29,21 @@ env:
VALKEY_DB: 0
API_WORKING_DIR: ./api
IMAGE_NAME: prowler-api
IGNORE_FILES: |
api/docs/**
api/README.md
api/CHANGELOG.md
jobs:
code-quality:
if: github.repository == 'prowler-cloud/prowler'
test:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Poetry check
run: poetry check --lock
- name: Ruff lint
run: poetry run ruff check . --exclude contrib
- name: Ruff format
run: poetry run ruff format --check . --exclude contrib
- name: Pylint
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Bandit
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
tests:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
python-version: ["3.12"]
# Service containers to run with `test`
services:
# Label used to access the service container
postgres:
image: postgres
env:
@@ -135,6 +52,7 @@ jobs:
POSTGRES_USER: ${{ env.POSTGRES_USER }}
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_DB: ${{ env.POSTGRES_DB }}
# Set health checks to wait until postgres has started
ports:
- 5432:5432
options: >-
@@ -148,6 +66,7 @@ jobs:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
VALKEY_DB: ${{ env.VALKEY_DB }}
# Set health checks to wait until postgres has started
ports:
- 6379:6379
options: >-
@@ -157,72 +76,158 @@ jobs:
--health-retries 5
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
- name: Test if changes are in not ignored paths
id: are-non-ignored-files-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
.github/workflows/api-pull-request.yml
files_ignore: ${{ env.IGNORE_FILES }}
- name: Replace @master with current branch in pyproject.toml - Only for pull requests to `master`
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'pull_request' && github.base_ref == 'master'
run: |
BRANCH_NAME="${GITHUB_HEAD_REF:-${GITHUB_REF_NAME}}"
echo "Using branch: $BRANCH_NAME"
sed -i "s|@master|@$BRANCH_NAME|g" pyproject.toml
- name: Install poetry
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
python -m pip install --upgrade pip
pipx install poetry==2.1.1
- name: Update SDK's poetry.lock resolved_reference to latest commit - Only for push events to `master`
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true' && github.event_name == 'push' && github.ref == 'refs/heads/master'
run: |
# Get the latest commit hash from the prowler-cloud/prowler repository
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
echo "Latest commit hash: $LATEST_COMMIT"
# Update the resolved_reference specifically for prowler-cloud/prowler repository
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
}' poetry.lock
# Verify the change was made
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry lock
- name: Set up Python ${{ matrix.python-version }}
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
cache: "poetry"
- name: Run tests with pytest
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Install dependencies
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry install --no-root
poetry run pip list
VERSION=$(curl --silent "https://api.github.com/repos/hadolint/hadolint/releases/latest" | \
grep '"tag_name":' | \
sed -E 's/.*"v([^"]+)".*/\1/' \
) && curl -L -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/v${VERSION}/hadolint-Linux-x86_64" \
&& chmod +x /tmp/hadolint
- name: Poetry check
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry check --lock
- name: Lint with ruff
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run ruff check . --exclude contrib
- name: Check Format with ruff
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run ruff format --check . --exclude contrib
- name: Lint with pylint
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
- name: Bandit
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: |
poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
- name: Hadolint
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
/tmp/hadolint Dockerfile --ignore=DL3013
- name: Test with pytest
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Upload coverage reports to Codecov
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: api
dockerfile-lint:
if: github.repository == 'prowler-cloud/prowler'
test-container-build:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Lint Dockerfile with Hadolint
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
- name: Test if changes are in not ignored paths
id: are-non-ignored-files-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
dockerfile: api/Dockerfile
ignore: DL3013
container-build-and-scan:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
files: api/**
files_ignore: ${{ env.IGNORE_FILES }}
- name: Set up Docker Buildx
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build container
- name: Build Container
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.API_WORKING_DIR }}
push: false
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
tags: ${{ env.IMAGE_NAME }}:latest
outputs: type=docker
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan container with Trivy
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
+15 -22
View File
@@ -1,35 +1,28 @@
name: 'Tools: Backport'
name: Prowler - Automatic Backport
on:
pull_request_target:
branches:
- 'master'
types:
- 'labeled'
- 'closed'
paths:
- '.github/workflows/backport.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: false
branches: ['master']
types: ['labeled', 'closed']
env:
# The prefix of the label that triggers the backport must not contain the branch name
# so, for example, if the branch is 'master', the label should be 'backport-to-<branch>'
BACKPORT_LABEL_PREFIX: backport-to-
BACKPORT_LABEL_IGNORE: was-backported
jobs:
backport:
name: Backport PR
if: github.event.pull_request.merged == true && !(contains(github.event.pull_request.labels.*.name, 'backport')) && !(contains(github.event.pull_request.labels.*.name, 'was-backported'))
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: write
id-token: write
pull-requests: write
contents: write
steps:
- name: Check labels
id: label_check
id: preview_label_check
uses: agilepathway/label-checker@c3d16ad512e7cea5961df85ff2486bb774caf3c5 # v1.6.65
with:
allow_failure: true
@@ -38,17 +31,17 @@ jobs:
none_of: ${{ env.BACKPORT_LABEL_IGNORE }}
repo_token: ${{ secrets.GITHUB_TOKEN }}
- name: Backport PR
if: steps.label_check.outputs.label_check == 'success'
- name: Backport Action
if: steps.preview_label_check.outputs.label_check == 'success'
uses: sorenlouv/backport-github-action@ad888e978060bc1b2798690dd9d03c4036560947 # v9.5.1
with:
github_token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
auto_backport_label_prefix: ${{ env.BACKPORT_LABEL_PREFIX }}
- name: Display backport info log
if: success() && steps.label_check.outputs.label_check == 'success'
- name: Info log
if: ${{ success() && steps.preview_label_check.outputs.label_check == 'success' }}
run: cat ~/.backport/backport.info.log
- name: Display backport debug log
if: failure() && steps.label_check.outputs.label_check == 'success'
- name: Debug log
if: ${{ failure() && steps.preview_label_check.outputs.label_check == 'success' }}
run: cat ~/.backport/backport.debug.log
@@ -0,0 +1,36 @@
name: Prowler - Pull Request Documentation Link
on:
pull_request:
branches:
- 'master'
- 'v3'
paths:
- 'docs/**'
- '.github/workflows/build-documentation-on-pr.yml'
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
jobs:
documentation-link:
name: Documentation Link
runs-on: ubuntu-latest
steps:
- name: Find existing documentation comment
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-comment
with:
issue-number: ${{ env.PR_NUMBER }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- prowler-docs-link -->'
- name: Create or update PR comment with the Prowler Documentation URI
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ env.PR_NUMBER }}
body: |
<!-- prowler-docs-link -->
You can check the documentation for this PR here -> [Prowler Documentation](https://prowler-prowler-docs--${{ env.PR_NUMBER }}.com.readthedocs.build/projects/prowler-open-source/en/${{ env.PR_NUMBER }}/)
edit-mode: replace
+13 -20
View File
@@ -1,31 +1,24 @@
name: 'Tools: Conventional Commit'
name: Prowler - Conventional Commit
on:
pull_request:
branches:
- 'master'
- 'v3'
- 'v4.*'
- 'v5.*'
types:
- 'opened'
- 'edited'
- 'synchronize'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
- "opened"
- "edited"
- "synchronize"
branches:
- "master"
- "v3"
- "v4.*"
- "v5.*"
jobs:
conventional-commit-check:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: read
steps:
- name: Check PR title format
- name: conventional-commit-check
id: conventional-commit-check
uses: agenthunt/conventional-commit-checker-action@9e552d650d0e205553ec7792d447929fc78e012b # v2.0.0
with:
pr-title-regex: '^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\([^)]+\))?!?: .+'
pr-title-regex: '^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\([^)]+\))?!?: .+'
+46 -49
View File
@@ -1,70 +1,67 @@
name: 'Tools: Backport Label'
name: Prowler - Create Backport Label
on:
release:
types:
- 'published'
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
cancel-in-progress: false
env:
BACKPORT_LABEL_PREFIX: backport-to-
BACKPORT_LABEL_COLOR: B60205
types: [published]
jobs:
create-label:
create_label:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
contents: write
issues: write
steps:
- name: Create backport label for minor releases
- name: Create backport label
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
OWNER_REPO: ${{ github.repository }}
run: |
RELEASE_TAG="${{ github.event.release.tag_name }}"
if [ -z "$RELEASE_TAG" ]; then
echo "Error: No release tag provided"
exit 1
fi
echo "Processing release tag: $RELEASE_TAG"
# Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
VERSION_ONLY="${RELEASE_TAG#v}"
VERSION_ONLY=${RELEASE_TAG#v} # Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
# Check if it's a minor version (X.Y.0)
if [[ "$VERSION_ONLY" =~ ^([0-9]+)\.([0-9]+)\.0$ ]]; then
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is a minor version. Proceeding to create backport label."
if [[ "$VERSION_ONLY" =~ ^[0-9]+\.[0-9]+\.0$ ]]; then
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is a minor version. Proceeding to create backport label."
# Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
MAJOR="${BASH_REMATCH[1]}"
MINOR="${BASH_REMATCH[2]}"
TWO_DIGIT_VERSION="${MAJOR}.${MINOR}"
TWO_DIGIT_VERSION=${VERSION_ONLY%.0} # Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
LABEL_NAME="${BACKPORT_LABEL_PREFIX}v${TWO_DIGIT_VERSION}"
LABEL_DESC="Backport PR to the v${TWO_DIGIT_VERSION} branch"
LABEL_COLOR="$BACKPORT_LABEL_COLOR"
FINAL_LABEL_NAME="backport-to-v${TWO_DIGIT_VERSION}"
FINAL_DESCRIPTION="Backport PR to the v${TWO_DIGIT_VERSION} branch"
echo "Label name: $LABEL_NAME"
echo "Label description: $LABEL_DESC"
echo "Effective label name will be: ${FINAL_LABEL_NAME}"
echo "Effective description will be: ${FINAL_DESCRIPTION}"
# Check if label already exists
if gh label list --repo ${{ github.repository }} --limit 1000 | grep -q "^${LABEL_NAME}[[:space:]]"; then
echo "Label '$LABEL_NAME' already exists."
# Check if the label already exists
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER_REPO}/labels/${FINAL_LABEL_NAME}")
if [ "${STATUS_CODE}" -eq 200 ]; then
echo "Label '${FINAL_LABEL_NAME}' already exists."
elif [ "${STATUS_CODE}" -eq 404 ]; then
echo "Label '${FINAL_LABEL_NAME}' does not exist. Creating it..."
# Prepare JSON data payload
JSON_DATA=$(printf '{"name":"%s","description":"%s","color":"B60205"}' "${FINAL_LABEL_NAME}" "${FINAL_DESCRIPTION}")
CREATE_STATUS_CODE=$(curl -s -o /tmp/curl_create_response.json -w "%{http_code}" -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token ${GITHUB_TOKEN}" \
--data "${JSON_DATA}" \
"https://api.github.com/repos/${OWNER_REPO}/labels")
CREATE_RESPONSE_BODY=$(cat /tmp/curl_create_response.json)
rm -f /tmp/curl_create_response.json
if [ "$CREATE_STATUS_CODE" -eq 201 ]; then
echo "Label '${FINAL_LABEL_NAME}' created successfully."
else
echo "Error creating label '${FINAL_LABEL_NAME}'. Status: $CREATE_STATUS_CODE"
echo "Response: $CREATE_RESPONSE_BODY"
exit 1
fi
else
echo "Label '$LABEL_NAME' does not exist. Creating it..."
gh label create "$LABEL_NAME" \
--description "$LABEL_DESC" \
--color "$LABEL_COLOR" \
--repo ${{ github.repository }}
echo "Label '$LABEL_NAME' created successfully."
echo "Error checking for label '${FINAL_LABEL_NAME}'. HTTP Status: ${STATUS_CODE}"
exit 1
fi
else
echo "Release $RELEASE_TAG (version $VERSION_ONLY) is not a minor version. Skipping backport label creation."
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is not a minor version. Skipping backport label creation."
exit 0
fi
+10 -24
View File
@@ -1,33 +1,19 @@
name: 'Tools: TruffleHog'
name: Prowler - Find secrets
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on: pull_request
jobs:
scan-secrets:
trufflehog:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 0
- name: Scan for secrets with TruffleHog
uses: trufflesecurity/trufflehog@ad6fc8fb446b8fafbf7ea8193d2d6bfd42f45690 # v3.90.11
- name: TruffleHog OSS
uses: trufflesecurity/trufflehog@466da5b0bb161144f6afca9afe5d57975828c410 # v3.90.8
with:
extra_args: '--results=verified,unknown'
path: ./
base: ${{ github.event.repository.default_branch }}
head: HEAD
extra_args: --only-verified
+8 -20
View File
@@ -1,29 +1,17 @@
name: 'Tools: PR Labeler'
name: Prowler - PR Labeler
on:
pull_request_target:
branches:
- 'master'
- 'v5.*'
types:
- 'opened'
- 'reopened'
- 'synchronize'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
pull_request_target:
branches:
- "master"
- "v3"
- "v4.*"
jobs:
labeler:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
runs-on: ubuntu-latest
steps:
- name: Apply labels to PR
uses: actions/labeler@634933edcd8ababfe52f92936142cc22ac488b1b # v6.0.1
with:
sync-labels: true
- uses: actions/labeler@634933edcd8ababfe52f92936142cc22ac488b1b # v6.0.1
@@ -1,110 +0,0 @@
name: 'MCP: Container Build and Push'
on:
push:
branches:
- 'master'
paths:
- 'mcp_server/**'
- '.github/workflows/mcp-container-build-push.yml'
release:
types:
- 'published'
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./mcp_server
# Container registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-mcp
jobs:
setup:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
steps:
- name: Calculate short SHA
id: set-short-sha
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
container-build-push:
needs: setup
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push MCP container (latest)
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.head_commit.timestamp }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push MCP container (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.release.published_at }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Trigger MCP deployment
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: mcp-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
@@ -0,0 +1,90 @@
name: MCP Server - Build and Push containers
on:
push:
branches:
- "master"
paths:
- "mcp_server/**"
- ".github/workflows/mcp-server-build-push-containers.yml"
# Uncomment the below code to test this action on PRs
# pull_request:
# branches:
# - "master"
# paths:
# - "mcp_server/**"
# - ".github/workflows/mcp-server-build-push-containers.yml"
release:
types: [published]
env:
# Tags
LATEST_TAG: latest
WORKING_DIRECTORY: ./mcp_server
# Container Registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-mcp
jobs:
repository-check:
name: Repository check
runs-on: ubuntu-latest
outputs:
is_repo: ${{ steps.repository_check.outputs.is_repo }}
steps:
- name: Repository check
id: repository_check
working-directory: /tmp
run: |
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
then
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
else
echo "This action only runs for prowler-cloud/prowler"
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
fi
container-build-push:
needs: repository-check
if: needs.repository-check.outputs.is_repo == 'true'
runs-on: ubuntu-latest
defaults:
run:
working-directory: ${{ env.WORKING_DIRECTORY }}
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set short git commit SHA
id: vars
run: |
shortSha=$(git rev-parse --short ${{ github.sha }})
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push container image (latest)
# Comment the following line for testing
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
# Set push: false for testing
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.SHORT_SHA }}
cache-from: type=gha
cache-to: type=gha,mode=max
-103
View File
@@ -1,103 +0,0 @@
name: 'Tools: Check Changelog'
on:
pull_request:
types:
- 'opened'
- 'synchronize'
- 'reopened'
- 'labeled'
- 'unlabeled'
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
check-changelog:
if: contains(github.event.pull_request.labels.*.name, 'no-changelog') == false
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
env:
MONITORED_FOLDERS: 'api ui prowler mcp_server'
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 0
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
ui/**
prowler/**
mcp_server/**
- name: Check for folder changes and changelog presence
id: check-folders
run: |
missing_changelogs=""
# Check api folder
if [[ "${{ steps.changed-files.outputs.any_changed }}" == "true" ]]; then
for folder in $MONITORED_FOLDERS; do
# Get files changed in this folder
changed_in_folder=$(echo "${{ steps.changed-files.outputs.all_changed_files }}" | tr ' ' '\n' | grep "^${folder}/" || true)
if [ -n "$changed_in_folder" ]; then
echo "Detected changes in ${folder}/"
# Check if CHANGELOG.md was updated
if ! echo "$changed_in_folder" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "No changelog update found for ${folder}/"
missing_changelogs="${missing_changelogs}- \`${folder}\`"$'\n'
fi
fi
done
fi
{
echo "missing_changelogs<<EOF"
echo -e "${missing_changelogs}"
echo "EOF"
} >> $GITHUB_OUTPUT
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
id: find-comment
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- changelog-check -->'
- name: Update PR comment with changelog status
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
edit-mode: replace
body: |
<!-- changelog-check -->
${{ steps.check-folders.outputs.missing_changelogs != '' && format('⚠️ **Changes detected in the following folders without a corresponding update to the `CHANGELOG.md`:**
{0}
Please add an entry to the corresponding `CHANGELOG.md` file to maintain a clear history of changes.', steps.check-folders.outputs.missing_changelogs) || '✅ All necessary `CHANGELOG.md` files have been updated.' }}
- name: Fail if changelog is missing
if: steps.check-folders.outputs.missing_changelogs != ''
run: |
echo "::error::Missing changelog updates in some folders"
exit 1
+104 -52
View File
@@ -1,40 +1,42 @@
name: 'Tools: PR Conflict Checker'
name: Prowler - PR Conflict Checker
on:
pull_request_target:
pull_request:
types:
- 'opened'
- 'synchronize'
- 'reopened'
- opened
- synchronize
- reopened
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true
- "master"
- "v5.*"
# Leaving this commented until we find a way to run it for forks but in Prowler's context
# pull_request_target:
# types:
# - opened
# - synchronize
# - reopened
# branches:
# - "master"
# - "v5.*"
jobs:
check-conflicts:
conflict-checker:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
issues: write
steps:
- name: Checkout PR head
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: '**'
files: |
**
- name: Check for conflict markers
id: conflict-check
@@ -49,10 +51,10 @@ jobs:
if [ -f "$file" ]; then
echo "Checking file: $file"
# Look for conflict markers (more precise regex)
if grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$file" 2>/dev/null; then
# Look for conflict markers
if grep -l "^<<<<<<<\|^=======\|^>>>>>>>" "$file" 2>/dev/null; then
echo "Conflict markers found in: $file"
CONFLICT_FILES="${CONFLICT_FILES}- \`${file}\`"$'\n'
CONFLICT_FILES="$CONFLICT_FILES$file "
HAS_CONFLICTS=true
fi
fi
@@ -60,64 +62,114 @@ jobs:
if [ "$HAS_CONFLICTS" = true ]; then
echo "has_conflicts=true" >> $GITHUB_OUTPUT
{
echo "conflict_files<<EOF"
echo "$CONFLICT_FILES"
echo "EOF"
} >> $GITHUB_OUTPUT
echo "Conflict markers detected"
echo "conflict_files=$CONFLICT_FILES" >> $GITHUB_OUTPUT
echo "Conflict markers detected in files: $CONFLICT_FILES"
else
echo "has_conflicts=false" >> $GITHUB_OUTPUT
echo "No conflict markers found in changed files"
fi
- name: Manage conflict label
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HAS_CONFLICTS: ${{ steps.conflict-check.outputs.has_conflicts }}
run: |
LABEL_NAME="has-conflicts"
- name: Add conflict label
if: steps.conflict-check.outputs.has_conflicts == 'true'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
script: |
const { data: labels } = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
# Add or remove label based on conflict status
if [ "$HAS_CONFLICTS" = "true" ]; then
echo "Adding conflict label to PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --add-label "$LABEL_NAME" --repo ${{ github.repository }} || true
else
echo "Removing conflict label from PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --remove-label "$LABEL_NAME" --repo ${{ github.repository }} || true
fi
const hasConflictLabel = labels.some(label => label.name === 'has-conflicts');
- name: Find existing comment
if (!hasConflictLabel) {
await github.rest.issues.addLabels({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
labels: ['has-conflicts']
});
console.log('Added has-conflicts label');
} else {
console.log('has-conflicts label already exists');
}
- name: Remove conflict label
if: steps.conflict-check.outputs.has_conflicts == 'false'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
with:
github-token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
script: |
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
name: 'has-conflicts'
});
console.log('Removed has-conflicts label');
} catch (error) {
if (error.status === 404) {
console.log('has-conflicts label was not present');
} else {
throw error;
}
}
- name: Find existing conflict comment
if: steps.conflict-check.outputs.has_conflicts == 'true'
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-comment
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- conflict-checker-comment -->'
body-regex: '(⚠️ \*\*Conflict Markers Detected\*\*|✅ \*\*Conflict Markers Resolved\*\*)'
- name: Create or update comment
- name: Create or update conflict comment
if: steps.conflict-check.outputs.has_conflicts == 'true'
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
edit-mode: replace
body: |
<!-- conflict-checker-comment -->
${{ steps.conflict-check.outputs.has_conflicts == 'true' && '⚠️ **Conflict Markers Detected**' || '✅ **Conflict Markers Resolved**' }}
⚠️ **Conflict Markers Detected**
${{ steps.conflict-check.outputs.has_conflicts == 'true' && format('This pull request contains unresolved conflict markers in the following files:
{0}
This pull request contains unresolved conflict markers in the following files:
```
${{ steps.conflict-check.outputs.conflict_files }}
```
Please resolve these conflicts by:
1. Locating the conflict markers: `<<<<<<<`, `=======`, and `>>>>>>>`
2. Manually editing the files to resolve the conflicts
3. Removing all conflict markers
4. Committing and pushing the changes', steps.conflict-check.outputs.conflict_files) || 'All conflict markers have been successfully resolved in this pull request.' }}
4. Committing and pushing the changes
- name: Find existing conflict comment when resolved
if: steps.conflict-check.outputs.has_conflicts == 'false'
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad # v4.0.0
id: find-resolved-comment
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-regex: '(⚠️ \*\*Conflict Markers Detected\*\*|✅ \*\*Conflict Markers Resolved\*\*)'
- name: Update comment when conflicts resolved
if: steps.conflict-check.outputs.has_conflicts == 'false' && steps.find-resolved-comment.outputs.comment-id != ''
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
comment-id: ${{ steps.find-resolved-comment.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
edit-mode: replace
body: |
✅ **Conflict Markers Resolved**
All conflict markers have been successfully resolved in this pull request.
- name: Fail workflow if conflicts detected
if: steps.conflict-check.outputs.has_conflicts == 'true'
run: |
echo "::error::Workflow failed due to conflict markers detected in the PR"
echo "::error::Workflow failed due to conflict markers in files: ${{ steps.conflict-check.outputs.conflict_files }}"
exit 1
@@ -1,6 +1,6 @@
name: 'Tools: Prepare Release'
name: Prowler - Release Preparation
run-name: 'Prepare Release for Prowler ${{ inputs.prowler_version }}'
run-name: Prowler Release Preparation for ${{ inputs.prowler_version }}
on:
workflow_dispatch:
@@ -10,23 +10,18 @@ on:
required: true
type: string
concurrency:
group: ${{ github.workflow }}-${{ inputs.prowler_version }}
cancel-in-progress: false
env:
PROWLER_VERSION: ${{ inputs.prowler_version }}
PROWLER_VERSION: ${{ github.event.inputs.prowler_version }}
jobs:
prepare-release:
if: github.event_name == 'workflow_dispatch' && github.repository == 'prowler-cloud/prowler'
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
pull-requests: write
steps:
- name: Checkout repository
- name: Checkout code
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 0
@@ -39,15 +34,15 @@ jobs:
- name: Install Poetry
run: |
python3 -m pip install --user poetry==2.1.1
python3 -m pip install --user poetry
echo "$HOME/.local/bin" >> $GITHUB_PATH
- name: Configure Git
run: |
git config --global user.name 'prowler-bot'
git config --global user.email '179230569+prowler-bot@users.noreply.github.com'
git config --global user.name "prowler-bot"
git config --global user.email "179230569+prowler-bot@users.noreply.github.com"
- name: Parse version and read changelogs
- name: Parse version and determine branch
run: |
# Validate version format (reusing pattern from sdk-bump-version.yml)
if [[ $PROWLER_VERSION =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
@@ -81,12 +76,10 @@ jobs:
UI_VERSION=$(extract_latest_version "ui/CHANGELOG.md")
API_VERSION=$(extract_latest_version "api/CHANGELOG.md")
SDK_VERSION=$(extract_latest_version "prowler/CHANGELOG.md")
MCP_VERSION=$(extract_latest_version "mcp_server/CHANGELOG.md")
echo "UI_VERSION=${UI_VERSION}" >> "${GITHUB_ENV}"
echo "API_VERSION=${API_VERSION}" >> "${GITHUB_ENV}"
echo "SDK_VERSION=${SDK_VERSION}" >> "${GITHUB_ENV}"
echo "MCP_VERSION=${MCP_VERSION}" >> "${GITHUB_ENV}"
if [ -n "$UI_VERSION" ]; then
echo "Read UI version from changelog: $UI_VERSION"
@@ -106,25 +99,18 @@ jobs:
echo "Warning: No SDK version found in prowler/CHANGELOG.md"
fi
if [ -n "$MCP_VERSION" ]; then
echo "Read MCP version from changelog: $MCP_VERSION"
else
echo "Warning: No MCP version found in mcp_server/CHANGELOG.md"
fi
echo "Prowler version: $PROWLER_VERSION"
echo "Branch name: $BRANCH_NAME"
echo "UI version: $UI_VERSION"
echo "API version: $API_VERSION"
echo "SDK version: $SDK_VERSION"
echo "MCP version: $MCP_VERSION"
echo "Is minor release: $([ $PATCH_VERSION -eq 0 ] && echo 'true' || echo 'false')"
else
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
exit 1
fi
- name: Extract and combine changelog entries
- name: Extract changelog entries
run: |
set -e
@@ -197,26 +183,7 @@ jobs:
touch "prowler_changelog.md"
fi
# MCP has changes if the changelog references this Prowler version
# Check if the changelog contains "(Prowler X.Y.Z)" or "(Prowler UNRELEASED)"
if [ -f "mcp_server/CHANGELOG.md" ]; then
MCP_PROWLER_REF=$(grep -m 1 "^## \[.*\] (Prowler" mcp_server/CHANGELOG.md | sed -E 's/.*\(Prowler ([^)]+)\).*/\1/' | tr -d '[:space:]')
if [ "$MCP_PROWLER_REF" = "$PROWLER_VERSION" ] || [ "$MCP_PROWLER_REF" = "UNRELEASED" ]; then
echo "HAS_MCP_CHANGES=true" >> $GITHUB_ENV
echo "✓ MCP changes detected - Prowler reference: $MCP_PROWLER_REF (version: $MCP_VERSION)"
extract_changelog "mcp_server/CHANGELOG.md" "$MCP_VERSION" "mcp_changelog.md"
else
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
echo " No MCP changes for this release (Prowler reference: $MCP_PROWLER_REF, input: $PROWLER_VERSION)"
touch "mcp_changelog.md"
fi
else
echo "HAS_MCP_CHANGES=false" >> $GITHUB_ENV
echo " No MCP changelog found"
touch "mcp_changelog.md"
fi
# Combine changelogs in order: UI, API, SDK, MCP
# Combine changelogs in order: UI, API, SDK
> combined_changelog.md
if [ "$HAS_UI_CHANGES" = "true" ] && [ -s "ui_changelog.md" ]; then
@@ -240,17 +207,10 @@ jobs:
echo "" >> combined_changelog.md
fi
if [ "$HAS_MCP_CHANGES" = "true" ] && [ -s "mcp_changelog.md" ]; then
echo "## MCP" >> combined_changelog.md
echo "" >> combined_changelog.md
cat mcp_changelog.md >> combined_changelog.md
echo "" >> combined_changelog.md
fi
echo "Combined changelog preview:"
cat combined_changelog.md
- name: Checkout release branch for patch release
- name: Checkout existing branch for patch release
if: ${{ env.PATCH_VERSION != '0' }}
run: |
echo "Patch release detected, checking out existing branch $BRANCH_NAME..."
@@ -265,7 +225,7 @@ jobs:
exit 1
fi
- name: Verify SDK version in pyproject.toml
- name: Verify version in pyproject.toml
run: |
CURRENT_VERSION=$(grep '^version = ' pyproject.toml | sed -E 's/version = "([^"]+)"/\1/' | tr -d '[:space:]')
PROWLER_VERSION_TRIMMED=$(echo "$PROWLER_VERSION" | tr -d '[:space:]')
@@ -275,7 +235,7 @@ jobs:
fi
echo "✓ pyproject.toml version: $CURRENT_VERSION"
- name: Verify SDK version in prowler/config/config.py
- name: Verify version in prowler/config/config.py
run: |
CURRENT_VERSION=$(grep '^prowler_version = ' prowler/config/config.py | sed -E 's/prowler_version = "([^"]+)"/\1/' | tr -d '[:space:]')
PROWLER_VERSION_TRIMMED=$(echo "$PROWLER_VERSION" | tr -d '[:space:]')
@@ -285,7 +245,7 @@ jobs:
fi
echo "✓ prowler/config/config.py version: $CURRENT_VERSION"
- name: Verify API version in api/pyproject.toml
- name: Verify version in api/pyproject.toml
if: ${{ env.HAS_API_CHANGES == 'true' }}
run: |
CURRENT_API_VERSION=$(grep '^version = ' api/pyproject.toml | sed -E 's/version = "([^"]+)"/\1/' | tr -d '[:space:]')
@@ -296,7 +256,7 @@ jobs:
fi
echo "✓ api/pyproject.toml version: $CURRENT_API_VERSION"
- name: Verify API prowler dependency in api/pyproject.toml
- name: Verify prowler dependency in api/pyproject.toml
if: ${{ env.PATCH_VERSION != '0' && env.HAS_API_CHANGES == 'true' }}
run: |
CURRENT_PROWLER_REF=$(grep 'prowler @ git+https://github.com/prowler-cloud/prowler.git@' api/pyproject.toml | sed -E 's/.*@([^"]+)".*/\1/' | tr -d '[:space:]')
@@ -307,7 +267,7 @@ jobs:
fi
echo "✓ api/pyproject.toml prowler dependency: $CURRENT_PROWLER_REF"
- name: Verify API version in api/src/backend/api/v1/views.py
- name: Verify version in api/src/backend/api/v1/views.py
if: ${{ env.HAS_API_CHANGES == 'true' }}
run: |
CURRENT_API_VERSION=$(grep 'spectacular_settings.VERSION = ' api/src/backend/api/v1/views.py | sed -E 's/.*spectacular_settings.VERSION = "([^"]+)".*/\1/' | tr -d '[:space:]')
@@ -318,7 +278,7 @@ jobs:
fi
echo "✓ api/src/backend/api/v1/views.py version: $CURRENT_API_VERSION"
- name: Checkout release branch for minor release
- name: Checkout existing release branch for minor release
if: ${{ env.PATCH_VERSION == '0' }}
run: |
echo "Minor release detected (patch = 0), checking out existing branch $BRANCH_NAME..."
@@ -330,7 +290,7 @@ jobs:
exit 1
fi
- name: Update API prowler dependency for minor release
- name: Prepare prowler dependency update for minor release
if: ${{ env.PATCH_VERSION == '0' }}
run: |
CURRENT_PROWLER_REF=$(grep 'prowler @ git+https://github.com/prowler-cloud/prowler.git@' api/pyproject.toml | sed -E 's/.*@([^"]+)".*/\1/' | tr -d '[:space:]')
@@ -367,7 +327,7 @@ jobs:
echo "✓ Prepared prowler dependency update to: $UPDATED_PROWLER_REF"
- name: Create PR for API dependency update
- name: Create Pull Request against release branch
if: ${{ env.PATCH_VERSION == '0' }}
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
@@ -407,4 +367,4 @@ jobs:
- name: Clean up temporary files
run: |
rm -f prowler_changelog.md api_changelog.md ui_changelog.md mcp_changelog.md combined_changelog.md
rm -f prowler_changelog.md api_changelog.md ui_changelog.md combined_changelog.md
@@ -0,0 +1,77 @@
name: Prowler - Check Changelog
on:
pull_request:
types: [opened, synchronize, reopened, labeled, unlabeled]
jobs:
check-changelog:
if: contains(github.event.pull_request.labels.*.name, 'no-changelog') == false
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
pull-requests: write
env:
MONITORED_FOLDERS: "api ui prowler dashboard"
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 0
- name: Get list of changed files
id: changed_files
run: |
git fetch origin ${{ github.base_ref }}
git diff --name-only origin/${{ github.base_ref }}...HEAD > changed_files.txt
cat changed_files.txt
- name: Check for folder changes and changelog presence
id: check_folders
run: |
missing_changelogs=""
for folder in $MONITORED_FOLDERS; do
if grep -q "^${folder}/" changed_files.txt; then
echo "Detected changes in ${folder}/"
if ! grep -q "^${folder}/CHANGELOG.md$" changed_files.txt; then
echo "No changelog update found for ${folder}/"
missing_changelogs="${missing_changelogs}- \`${folder}\`\n"
fi
fi
done
echo "missing_changelogs<<EOF" >> $GITHUB_OUTPUT
echo -e "${missing_changelogs}" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
id: find_comment
uses: peter-evans/find-comment@b30e6a3c0ed37e7c023ccd3f1db5c6c0b0c23aad #v4.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: 'github-actions[bot]'
body-includes: '<!-- changelog-check -->'
- name: Update PR comment with changelog status
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find_comment.outputs.comment-id }}
edit-mode: replace
body: |
<!-- changelog-check -->
${{ steps.check_folders.outputs.missing_changelogs != '' && format('⚠️ **Changes detected in the following folders without a corresponding update to the `CHANGELOG.md`:**
{0}
Please add an entry to the corresponding `CHANGELOG.md` file to maintain a clear history of changes.', steps.check_folders.outputs.missing_changelogs) || '✅ All necessary `CHANGELOG.md` files have been updated. Great job! 🎉' }}
- name: Fail if changelog is missing
if: steps.check_folders.outputs.missing_changelogs != ''
run: |
echo "ERROR: Missing changelog updates in some folders."
exit 1
@@ -1,31 +1,27 @@
name: 'Tools: PR Merged'
name: Prowler - Merged Pull Request
on:
pull_request_target:
branches:
- 'master'
types:
- 'closed'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: false
branches: ['master']
types: ['closed']
jobs:
trigger-cloud-pull-request:
name: Trigger Cloud Pull Request
if: github.event.pull_request.merged == true && github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Calculate short commit SHA
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: ${{ github.event.pull_request.merge_commit_sha }}
- name: Set short git commit SHA
id: vars
run: |
SHORT_SHA="${{ github.event.pull_request.merge_commit_sha }}"
echo "SHORT_SHA=${SHORT_SHA::7}" >> $GITHUB_ENV
shortSha=$(git rev-parse --short ${{ github.event.pull_request.merge_commit_sha }})
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
- name: Trigger Cloud repository pull request
- name: Trigger pull request
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
@@ -35,12 +31,8 @@ jobs:
{
"PROWLER_COMMIT_SHA": "${{ github.event.pull_request.merge_commit_sha }}",
"PROWLER_COMMIT_SHORT_SHA": "${{ env.SHORT_SHA }}",
"PROWLER_PR_NUMBER": "${{ github.event.pull_request.number }}",
"PROWLER_PR_TITLE": ${{ toJson(github.event.pull_request.title) }},
"PROWLER_PR_LABELS": ${{ toJson(github.event.pull_request.labels.*.name) }},
"PROWLER_PR_BODY": ${{ toJson(github.event.pull_request.body) }},
"PROWLER_PR_URL": ${{ toJson(github.event.pull_request.html_url) }},
"PROWLER_PR_MERGED_BY": "${{ github.event.pull_request.merged_by.login }}",
"PROWLER_PR_BASE_BRANCH": "${{ github.event.pull_request.base.ref }}",
"PROWLER_PR_HEAD_BRANCH": "${{ github.event.pull_request.head.ref }}"
"PROWLER_PR_URL": ${{ toJson(github.event.pull_request.html_url) }}
}
+1 -16
View File
@@ -249,21 +249,6 @@ jobs:
run: |
poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodb_atlas_coverage.xml tests/providers/mongodbatlas
# Test OCI
- name: OCI - Check if any file has changed
id: oci-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/oraclecloud/**
./tests/providers/oraclecloud/**
./poetry.lock
- name: OCI - Test
if: steps.oci-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oci_coverage.xml tests/providers/oraclecloud
# Common Tests
- name: Lib - Test
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
@@ -283,4 +268,4 @@ jobs:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./oci_coverage.xml,./lib_coverage.xml,./config_coverage.xml
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./lib_coverage.xml,./config_coverage.xml
-3
View File
@@ -83,6 +83,3 @@ CLAUDE.md
# MCP Server
mcp_server/prowler_mcp_server/prowler_app/server.py
mcp_server/prowler_mcp_server/prowler_app/utils/schema.yaml
# Compliance report
*.pdf
+1 -9
View File
@@ -46,14 +46,6 @@ help: ## Show this help.
@echo "Prowler Makefile"
@awk 'BEGIN {FS = ":.*##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n"} /^[a-zA-Z_-]+:.*?##/ { printf " \033[36m%-15s\033[0m %s\n", $$1, $$2 } /^##@/ { printf "\n\033[1m%s\033[0m\n", substr($$0, 5) } ' $(MAKEFILE_LIST)
##@ Build no cache
build-no-cache-dev:
docker compose -f docker-compose-dev.yml build --no-cache api-dev worker-dev worker-beat
##@ Development Environment
run-api-dev: ## Start development environment with API, PostgreSQL, Valkey, and workers
docker compose -f docker-compose-dev.yml up api-dev postgres valkey worker-dev worker-beat
##@ Development Environment
build-and-run-api-dev: build-no-cache-dev run-api-dev
docker compose -f docker-compose-dev.yml up api-dev postgres valkey worker-dev worker-beat --build
+3 -4
View File
@@ -82,13 +82,12 @@ prowler dashboard
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Stage | Interface |
|---|---|---|---|---|---|---|---|
| AWS | 576 | 82 | 38 | 10 | Official | Stable | UI, API, CLI |
| GCP | 79 | 13 | 11 | 3 | Official | Stable | UI, API, CLI |
| Azure | 162 | 19 | 12 | 4 | Official | Stable | UI, API, CLI |
| AWS | 576 | 82 | 36 | 10 | Official | Stable | UI, API, CLI |
| GCP | 79 | 13 | 10 | 3 | Official | Stable | UI, API, CLI |
| Azure | 162 | 19 | 11 | 4 | Official | Stable | UI, API, CLI |
| Kubernetes | 83 | 7 | 5 | 7 | Official | Stable | UI, API, CLI |
| GitHub | 17 | 2 | 1 | 0 | Official | Stable | UI, API, CLI |
| M365 | 70 | 7 | 3 | 2 | Official | Stable | UI, API, CLI |
| OCI | 51 | 13 | 1 | 10 | Official | Stable | CLI |
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | Beta | CLI |
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | Beta | CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | Beta | CLI |
+1 -6
View File
@@ -2,21 +2,16 @@
All notable changes to the **Prowler API** are documented in this file.
## [1.14.0] (Prowler 5.13.0)
## [1.14.0] (Prowler UNRELEASED)
### Added
- Default JWT keys are generated and stored if they are missing from configuration [(#8655)](https://github.com/prowler-cloud/prowler/pull/8655)
- `compliance_name` for each compliance [(#7920)](https://github.com/prowler-cloud/prowler/pull/7920)
- Support C5 compliance framework for the AWS provider [(#8830)](https://github.com/prowler-cloud/prowler/pull/8830)
- Support for M365 Certificate authentication [(#8538)](https://github.com/prowler-cloud/prowler/pull/8538)
- API Key support [(#8805)](https://github.com/prowler-cloud/prowler/pull/8805)
- SAML role mapping protection for single-admin tenants to prevent accidental lockout [(#8882)](https://github.com/prowler-cloud/prowler/pull/8882)
- Support for `passed_findings` and `total_findings` fields in compliance requirement overview for accurate Prowler ThreatScore calculation [(#8582)](https://github.com/prowler-cloud/prowler/pull/8582)
- PDF reporting for Prowler ThreatScore [(#8867)](https://github.com/prowler-cloud/prowler/pull/8867)
- Database read replica support [(#8869)](https://github.com/prowler-cloud/prowler/pull/8869)
- Support for configuring multiple LLM providers [(#8772)](https://github.com/prowler-cloud/prowler/pull/8772)
- Support Common Cloud Controls for AWS, Azure and GCP [(#8000)](https://github.com/prowler-cloud/prowler/pull/8000)
- Add `provider_id__in` filter support to findings and findings severity overview endpoints [(#8951)](https://github.com/prowler-cloud/prowler/pull/8951)
### Changed
- Now the MANAGE_ACCOUNT permission is required to modify or read user permissions instead of MANAGE_USERS [(#8281)](https://github.com/prowler-cloud/prowler/pull/8281)
+5 -532
View File
@@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 2.2.0 and should not be changed by hand.
# This file is automatically @generated by Poetry 2.1.4 and should not be changed by hand.
[[package]]
name = "about-time"
@@ -273,14 +273,14 @@ tests-mypy = ["mypy (>=1.11.1) ; platform_python_implementation == \"CPython\" a
[[package]]
name = "authlib"
version = "1.6.5"
version = "1.6.4"
description = "The ultimate Python library in building OAuth and OpenID Connect servers and clients."
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "authlib-1.6.5-py2.py3-none-any.whl", hash = "sha256:3e0e0507807f842b02175507bdee8957a1d5707fd4afb17c32fb43fee90b6e3a"},
{file = "authlib-1.6.5.tar.gz", hash = "sha256:6aaf9c79b7cc96c900f0b284061691c5d4e61221640a948fe690b556a6d6d10b"},
{file = "authlib-1.6.4-py2.py3-none-any.whl", hash = "sha256:39313d2a2caac3ecf6d8f95fbebdfd30ae6ea6ae6a6db794d976405fdd9aa796"},
{file = "authlib-1.6.4.tar.gz", hash = "sha256:104b0442a43061dc8bc23b133d1d06a2b0a9c2e3e33f34c4338929e816287649"},
]
[package.dependencies]
@@ -1256,98 +1256,6 @@ files = [
{file = "contextlib2-21.6.0.tar.gz", hash = "sha256:ab1e2bfe1d01d968e1b7e8d9023bc51ef3509bba217bb730cee3827e1ee82869"},
]
[[package]]
name = "contourpy"
version = "1.3.3"
description = "Python library for calculating contours of 2D quadrilateral grids"
optional = false
python-versions = ">=3.11"
groups = ["main"]
files = [
{file = "contourpy-1.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:709a48ef9a690e1343202916450bc48b9e51c049b089c7f79a267b46cffcdaa1"},
{file = "contourpy-1.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:23416f38bfd74d5d28ab8429cc4d63fa67d5068bd711a85edb1c3fb0c3e2f381"},
{file = "contourpy-1.3.3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:929ddf8c4c7f348e4c0a5a3a714b5c8542ffaa8c22954862a46ca1813b667ee7"},
{file = "contourpy-1.3.3-cp311-cp311-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9e999574eddae35f1312c2b4b717b7885d4edd6cb46700e04f7f02db454e67c1"},
{file = "contourpy-1.3.3-cp311-cp311-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0bf67e0e3f482cb69779dd3061b534eb35ac9b17f163d851e2a547d56dba0a3a"},
{file = "contourpy-1.3.3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:51e79c1f7470158e838808d4a996fa9bac72c498e93d8ebe5119bc1e6becb0db"},
{file = "contourpy-1.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:598c3aaece21c503615fd59c92a3598b428b2f01bfb4b8ca9c4edeecc2438620"},
{file = "contourpy-1.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:322ab1c99b008dad206d406bb61d014cf0174df491ae9d9d0fac6a6fda4f977f"},
{file = "contourpy-1.3.3-cp311-cp311-win32.whl", hash = "sha256:fd907ae12cd483cd83e414b12941c632a969171bf90fc937d0c9f268a31cafff"},
{file = "contourpy-1.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:3519428f6be58431c56581f1694ba8e50626f2dd550af225f82fb5f5814d2a42"},
{file = "contourpy-1.3.3-cp311-cp311-win_arm64.whl", hash = "sha256:15ff10bfada4bf92ec8b31c62bf7c1834c244019b4a33095a68000d7075df470"},
{file = "contourpy-1.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b08a32ea2f8e42cf1d4be3169a98dd4be32bafe4f22b6c4cb4ba810fa9e5d2cb"},
{file = "contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:556dba8fb6f5d8742f2923fe9457dbdd51e1049c4a43fd3986a0b14a1d815fc6"},
{file = "contourpy-1.3.3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92d9abc807cf7d0e047b95ca5d957cf4792fcd04e920ca70d48add15c1a90ea7"},
{file = "contourpy-1.3.3-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b2e8faa0ed68cb29af51edd8e24798bb661eac3bd9f65420c1887b6ca89987c8"},
{file = "contourpy-1.3.3-cp312-cp312-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:626d60935cf668e70a5ce6ff184fd713e9683fb458898e4249b63be9e28286ea"},
{file = "contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4d00e655fcef08aba35ec9610536bfe90267d7ab5ba944f7032549c55a146da1"},
{file = "contourpy-1.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:451e71b5a7d597379ef572de31eeb909a87246974d960049a9848c3bc6c41bf7"},
{file = "contourpy-1.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:459c1f020cd59fcfe6650180678a9993932d80d44ccde1fa1868977438f0b411"},
{file = "contourpy-1.3.3-cp312-cp312-win32.whl", hash = "sha256:023b44101dfe49d7d53932be418477dba359649246075c996866106da069af69"},
{file = "contourpy-1.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:8153b8bfc11e1e4d75bcb0bff1db232f9e10b274e0929de9d608027e0d34ff8b"},
{file = "contourpy-1.3.3-cp312-cp312-win_arm64.whl", hash = "sha256:07ce5ed73ecdc4a03ffe3e1b3e3c1166db35ae7584be76f65dbbe28a7791b0cc"},
{file = "contourpy-1.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:177fb367556747a686509d6fef71d221a4b198a3905fe824430e5ea0fda54eb5"},
{file = "contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d002b6f00d73d69333dac9d0b8d5e84d9724ff9ef044fd63c5986e62b7c9e1b1"},
{file = "contourpy-1.3.3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:348ac1f5d4f1d66d3322420f01d42e43122f43616e0f194fc1c9f5d830c5b286"},
{file = "contourpy-1.3.3-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:655456777ff65c2c548b7c454af9c6f33f16c8884f11083244b5819cc214f1b5"},
{file = "contourpy-1.3.3-cp313-cp313-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:644a6853d15b2512d67881586bd03f462c7ab755db95f16f14d7e238f2852c67"},
{file = "contourpy-1.3.3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4debd64f124ca62069f313a9cb86656ff087786016d76927ae2cf37846b006c9"},
{file = "contourpy-1.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a15459b0f4615b00bbd1e91f1b9e19b7e63aea7483d03d804186f278c0af2659"},
{file = "contourpy-1.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ca0fdcd73925568ca027e0b17ab07aad764be4706d0a925b89227e447d9737b7"},
{file = "contourpy-1.3.3-cp313-cp313-win32.whl", hash = "sha256:b20c7c9a3bf701366556e1b1984ed2d0cedf999903c51311417cf5f591d8c78d"},
{file = "contourpy-1.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:1cadd8b8969f060ba45ed7c1b714fe69185812ab43bd6b86a9123fe8f99c3263"},
{file = "contourpy-1.3.3-cp313-cp313-win_arm64.whl", hash = "sha256:fd914713266421b7536de2bfa8181aa8c699432b6763a0ea64195ebe28bff6a9"},
{file = "contourpy-1.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:88df9880d507169449d434c293467418b9f6cbe82edd19284aa0409e7fdb933d"},
{file = "contourpy-1.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:d06bb1f751ba5d417047db62bca3c8fde202b8c11fb50742ab3ab962c81e8216"},
{file = "contourpy-1.3.3-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e4e6b05a45525357e382909a4c1600444e2a45b4795163d3b22669285591c1ae"},
{file = "contourpy-1.3.3-cp313-cp313t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ab3074b48c4e2cf1a960e6bbeb7f04566bf36b1861d5c9d4d8ac04b82e38ba20"},
{file = "contourpy-1.3.3-cp313-cp313t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c3d53c796f8647d6deb1abe867daeb66dcc8a97e8455efa729516b997b8ed99"},
{file = "contourpy-1.3.3-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:50ed930df7289ff2a8d7afeb9603f8289e5704755c7e5c3bbd929c90c817164b"},
{file = "contourpy-1.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:4feffb6537d64b84877da813a5c30f1422ea5739566abf0bd18065ac040e120a"},
{file = "contourpy-1.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:2b7e9480ffe2b0cd2e787e4df64270e3a0440d9db8dc823312e2c940c167df7e"},
{file = "contourpy-1.3.3-cp313-cp313t-win32.whl", hash = "sha256:283edd842a01e3dcd435b1c5116798d661378d83d36d337b8dde1d16a5fc9ba3"},
{file = "contourpy-1.3.3-cp313-cp313t-win_amd64.whl", hash = "sha256:87acf5963fc2b34825e5b6b048f40e3635dd547f590b04d2ab317c2619ef7ae8"},
{file = "contourpy-1.3.3-cp313-cp313t-win_arm64.whl", hash = "sha256:3c30273eb2a55024ff31ba7d052dde990d7d8e5450f4bbb6e913558b3d6c2301"},
{file = "contourpy-1.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fde6c716d51c04b1c25d0b90364d0be954624a0ee9d60e23e850e8d48353d07a"},
{file = "contourpy-1.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:cbedb772ed74ff5be440fa8eee9bd49f64f6e3fc09436d9c7d8f1c287b121d77"},
{file = "contourpy-1.3.3-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:22e9b1bd7a9b1d652cd77388465dc358dafcd2e217d35552424aa4f996f524f5"},
{file = "contourpy-1.3.3-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a22738912262aa3e254e4f3cb079a95a67132fc5a063890e224393596902f5a4"},
{file = "contourpy-1.3.3-cp314-cp314-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:afe5a512f31ee6bd7d0dda52ec9864c984ca3d66664444f2d72e0dc4eb832e36"},
{file = "contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f64836de09927cba6f79dcd00fdd7d5329f3fccc633468507079c829ca4db4e3"},
{file = "contourpy-1.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:1fd43c3be4c8e5fd6e4f2baeae35ae18176cf2e5cced681cca908addf1cdd53b"},
{file = "contourpy-1.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6afc576f7b33cf00996e5c1102dc2a8f7cc89e39c0b55df93a0b78c1bd992b36"},
{file = "contourpy-1.3.3-cp314-cp314-win32.whl", hash = "sha256:66c8a43a4f7b8df8b71ee1840e4211a3c8d93b214b213f590e18a1beca458f7d"},
{file = "contourpy-1.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:cf9022ef053f2694e31d630feaacb21ea24224be1c3ad0520b13d844274614fd"},
{file = "contourpy-1.3.3-cp314-cp314-win_arm64.whl", hash = "sha256:95b181891b4c71de4bb404c6621e7e2390745f887f2a026b2d99e92c17892339"},
{file = "contourpy-1.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:33c82d0138c0a062380332c861387650c82e4cf1747aaa6938b9b6516762e772"},
{file = "contourpy-1.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ea37e7b45949df430fe649e5de8351c423430046a2af20b1c1961cae3afcda77"},
{file = "contourpy-1.3.3-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d304906ecc71672e9c89e87c4675dc5c2645e1f4269a5063b99b0bb29f232d13"},
{file = "contourpy-1.3.3-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ca658cd1a680a5c9ea96dc61cdbae1e85c8f25849843aa799dfd3cb370ad4fbe"},
{file = "contourpy-1.3.3-cp314-cp314t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ab2fd90904c503739a75b7c8c5c01160130ba67944a7b77bbf36ef8054576e7f"},
{file = "contourpy-1.3.3-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b7301b89040075c30e5768810bc96a8e8d78085b47d8be6e4c3f5a0b4ed478a0"},
{file = "contourpy-1.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2a2a8b627d5cc6b7c41a4beff6c5ad5eb848c88255fda4a8745f7e901b32d8e4"},
{file = "contourpy-1.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:fd6ec6be509c787f1caf6b247f0b1ca598bef13f4ddeaa126b7658215529ba0f"},
{file = "contourpy-1.3.3-cp314-cp314t-win32.whl", hash = "sha256:e74a9a0f5e3fff48fb5a7f2fd2b9b70a3fe014a67522f79b7cca4c0c7e43c9ae"},
{file = "contourpy-1.3.3-cp314-cp314t-win_amd64.whl", hash = "sha256:13b68d6a62db8eafaebb8039218921399baf6e47bf85006fd8529f2a08ef33fc"},
{file = "contourpy-1.3.3-cp314-cp314t-win_arm64.whl", hash = "sha256:b7448cb5a725bb1e35ce88771b86fba35ef418952474492cf7c764059933ff8b"},
{file = "contourpy-1.3.3-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:cd5dfcaeb10f7b7f9dc8941717c6c2ade08f587be2226222c12b25f0483ed497"},
{file = "contourpy-1.3.3-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:0c1fc238306b35f246d61a1d416a627348b5cf0648648a031e14bb8705fcdfe8"},
{file = "contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:70f9aad7de812d6541d29d2bbf8feb22ff7e1c299523db288004e3157ff4674e"},
{file = "contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5ed3657edf08512fc3fe81b510e35c2012fbd3081d2e26160f27ca28affec989"},
{file = "contourpy-1.3.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:3d1a3799d62d45c18bafd41c5fa05120b96a28079f2393af559b843d1a966a77"},
{file = "contourpy-1.3.3.tar.gz", hash = "sha256:083e12155b210502d0bca491432bb04d56dc3432f95a979b429f2848c3dbe880"},
]
[package.dependencies]
numpy = ">=1.25"
[package.extras]
bokeh = ["bokeh", "selenium"]
docs = ["furo", "sphinx (>=7.2)", "sphinx-copybutton"]
mypy = ["bokeh", "contourpy[bokeh,docs]", "docutils-stubs", "mypy (==1.17.0)", "types-Pillow"]
test = ["Pillow", "contourpy[test-no-images]", "matplotlib"]
test-no-images = ["pytest", "pytest-cov", "pytest-rerunfailures", "pytest-xdist", "wurlitzer"]
[[package]]
name = "coverage"
version = "7.5.4"
@@ -1482,22 +1390,6 @@ ssh = ["bcrypt (>=3.1.5)"]
test = ["certifi (>=2024)", "cryptography-vectors (==44.0.1)", "pretend (>=0.7)", "pytest (>=7.4.0)", "pytest-benchmark (>=4.0)", "pytest-cov (>=2.10.1)", "pytest-xdist (>=3.5.0)"]
test-randomorder = ["pytest-randomly"]
[[package]]
name = "cycler"
version = "0.12.1"
description = "Composable style cycles"
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30"},
{file = "cycler-0.12.1.tar.gz", hash = "sha256:88bb128f02ba341da8ef447245a9e138fae777f6a23943da4540077d3601eb1c"},
]
[package.extras]
docs = ["ipython", "matplotlib", "numpydoc", "sphinx"]
tests = ["pytest", "pytest-cov", "pytest-xdist"]
[[package]]
name = "dash"
version = "3.1.1"
@@ -2228,87 +2120,6 @@ werkzeug = ">=3.1.0"
async = ["asgiref (>=3.2)"]
dotenv = ["python-dotenv"]
[[package]]
name = "fonttools"
version = "4.60.1"
description = "Tools to manipulate font files"
optional = false
python-versions = ">=3.9"
groups = ["main"]
files = [
{file = "fonttools-4.60.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:9a52f254ce051e196b8fe2af4634c2d2f02c981756c6464dc192f1b6050b4e28"},
{file = "fonttools-4.60.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:c7420a2696a44650120cdd269a5d2e56a477e2bfa9d95e86229059beb1c19e15"},
{file = "fonttools-4.60.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee0c0b3b35b34f782afc673d503167157094a16f442ace7c6c5e0ca80b08f50c"},
{file = "fonttools-4.60.1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:282dafa55f9659e8999110bd8ed422ebe1c8aecd0dc396550b038e6c9a08b8ea"},
{file = "fonttools-4.60.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:4ba4bd646e86de16160f0fb72e31c3b9b7d0721c3e5b26b9fa2fc931dfdb2652"},
{file = "fonttools-4.60.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:0b0835ed15dd5b40d726bb61c846a688f5b4ce2208ec68779bc81860adb5851a"},
{file = "fonttools-4.60.1-cp310-cp310-win32.whl", hash = "sha256:1525796c3ffe27bb6268ed2a1bb0dcf214d561dfaf04728abf01489eb5339dce"},
{file = "fonttools-4.60.1-cp310-cp310-win_amd64.whl", hash = "sha256:268ecda8ca6cb5c4f044b1fb9b3b376e8cd1b361cef275082429dc4174907038"},
{file = "fonttools-4.60.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7b4c32e232a71f63a5d00259ca3d88345ce2a43295bb049d21061f338124246f"},
{file = "fonttools-4.60.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:3630e86c484263eaac71d117085d509cbcf7b18f677906824e4bace598fb70d2"},
{file = "fonttools-4.60.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5c1015318e4fec75dd4943ad5f6a206d9727adf97410d58b7e32ab644a807914"},
{file = "fonttools-4.60.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:e6c58beb17380f7c2ea181ea11e7db8c0ceb474c9dd45f48e71e2cb577d146a1"},
{file = "fonttools-4.60.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ec3681a0cb34c255d76dd9d865a55f260164adb9fa02628415cdc2d43ee2c05d"},
{file = "fonttools-4.60.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f4b5c37a5f40e4d733d3bbaaef082149bee5a5ea3156a785ff64d949bd1353fa"},
{file = "fonttools-4.60.1-cp311-cp311-win32.whl", hash = "sha256:398447f3d8c0c786cbf1209711e79080a40761eb44b27cdafffb48f52bcec258"},
{file = "fonttools-4.60.1-cp311-cp311-win_amd64.whl", hash = "sha256:d066ea419f719ed87bc2c99a4a4bfd77c2e5949cb724588b9dd58f3fd90b92bf"},
{file = "fonttools-4.60.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:7b0c6d57ab00dae9529f3faf187f2254ea0aa1e04215cf2f1a8ec277c96661bc"},
{file = "fonttools-4.60.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:839565cbf14645952d933853e8ade66a463684ed6ed6c9345d0faf1f0e868877"},
{file = "fonttools-4.60.1-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:8177ec9676ea6e1793c8a084a90b65a9f778771998eb919d05db6d4b1c0b114c"},
{file = "fonttools-4.60.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:996a4d1834524adbb423385d5a629b868ef9d774670856c63c9a0408a3063401"},
{file = "fonttools-4.60.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a46b2f450bc79e06ef3b6394f0c68660529ed51692606ad7f953fc2e448bc903"},
{file = "fonttools-4.60.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6ec722ee589e89a89f5b7574f5c45604030aa6ae24cb2c751e2707193b466fed"},
{file = "fonttools-4.60.1-cp312-cp312-win32.whl", hash = "sha256:b2cf105cee600d2de04ca3cfa1f74f1127f8455b71dbad02b9da6ec266e116d6"},
{file = "fonttools-4.60.1-cp312-cp312-win_amd64.whl", hash = "sha256:992775c9fbe2cf794786fa0ffca7f09f564ba3499b8fe9f2f80bd7197db60383"},
{file = "fonttools-4.60.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:6f68576bb4bbf6060c7ab047b1574a1ebe5c50a17de62830079967b211059ebb"},
{file = "fonttools-4.60.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:eedacb5c5d22b7097482fa834bda0dafa3d914a4e829ec83cdea2a01f8c813c4"},
{file = "fonttools-4.60.1-cp313-cp313-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b33a7884fabd72bdf5f910d0cf46be50dce86a0362a65cfc746a4168c67eb96c"},
{file = "fonttools-4.60.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2409d5fb7b55fd70f715e6d34e7a6e4f7511b8ad29a49d6df225ee76da76dd77"},
{file = "fonttools-4.60.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:c8651e0d4b3bdeda6602b85fdc2abbefc1b41e573ecb37b6779c4ca50753a199"},
{file = "fonttools-4.60.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:145daa14bf24824b677b9357c5e44fd8895c2a8f53596e1b9ea3496081dc692c"},
{file = "fonttools-4.60.1-cp313-cp313-win32.whl", hash = "sha256:2299df884c11162617a66b7c316957d74a18e3758c0274762d2cc87df7bc0272"},
{file = "fonttools-4.60.1-cp313-cp313-win_amd64.whl", hash = "sha256:a3db56f153bd4c5c2b619ab02c5db5192e222150ce5a1bc10f16164714bc39ac"},
{file = "fonttools-4.60.1-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:a884aef09d45ba1206712c7dbda5829562d3fea7726935d3289d343232ecb0d3"},
{file = "fonttools-4.60.1-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8a44788d9d91df72d1a5eac49b31aeb887a5f4aab761b4cffc4196c74907ea85"},
{file = "fonttools-4.60.1-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:e852d9dda9f93ad3651ae1e3bb770eac544ec93c3807888798eccddf84596537"},
{file = "fonttools-4.60.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:154cb6ee417e417bf5f7c42fe25858c9140c26f647c7347c06f0cc2d47eff003"},
{file = "fonttools-4.60.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:5664fd1a9ea7f244487ac8f10340c4e37664675e8667d6fee420766e0fb3cf08"},
{file = "fonttools-4.60.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:583b7f8e3c49486e4d489ad1deacfb8d5be54a8ef34d6df824f6a171f8511d99"},
{file = "fonttools-4.60.1-cp314-cp314-win32.whl", hash = "sha256:66929e2ea2810c6533a5184f938502cfdaea4bc3efb7130d8cc02e1c1b4108d6"},
{file = "fonttools-4.60.1-cp314-cp314-win_amd64.whl", hash = "sha256:f3d5be054c461d6a2268831f04091dc82753176f6ea06dc6047a5e168265a987"},
{file = "fonttools-4.60.1-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:b6379e7546ba4ae4b18f8ae2b9bc5960936007a1c0e30b342f662577e8bc3299"},
{file = "fonttools-4.60.1-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9d0ced62b59e0430b3690dbc5373df1c2aa7585e9a8ce38eff87f0fd993c5b01"},
{file = "fonttools-4.60.1-cp314-cp314t-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:875cb7764708b3132637f6c5fb385b16eeba0f7ac9fa45a69d35e09b47045801"},
{file = "fonttools-4.60.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a184b2ea57b13680ab6d5fbde99ccef152c95c06746cb7718c583abd8f945ccc"},
{file = "fonttools-4.60.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:026290e4ec76583881763fac284aca67365e0be9f13a7fb137257096114cb3bc"},
{file = "fonttools-4.60.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:f0e8817c7d1a0c2eedebf57ef9a9896f3ea23324769a9a2061a80fe8852705ed"},
{file = "fonttools-4.60.1-cp314-cp314t-win32.whl", hash = "sha256:1410155d0e764a4615774e5c2c6fc516259fe3eca5882f034eb9bfdbee056259"},
{file = "fonttools-4.60.1-cp314-cp314t-win_amd64.whl", hash = "sha256:022beaea4b73a70295b688f817ddc24ed3e3418b5036ffcd5658141184ef0d0c"},
{file = "fonttools-4.60.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:122e1a8ada290423c493491d002f622b1992b1ab0b488c68e31c413390dc7eb2"},
{file = "fonttools-4.60.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:a140761c4ff63d0cb9256ac752f230460ee225ccef4ad8f68affc723c88e2036"},
{file = "fonttools-4.60.1-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0eae96373e4b7c9e45d099d7a523444e3554360927225c1cdae221a58a45b856"},
{file = "fonttools-4.60.1-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:596ecaca36367027d525b3b426d8a8208169d09edcf8c7506aceb3a38bfb55c7"},
{file = "fonttools-4.60.1-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:2ee06fc57512144d8b0445194c2da9f190f61ad51e230f14836286470c99f854"},
{file = "fonttools-4.60.1-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:b42d86938e8dda1cd9a1a87a6d82f1818eaf933348429653559a458d027446da"},
{file = "fonttools-4.60.1-cp39-cp39-win32.whl", hash = "sha256:8b4eb332f9501cb1cd3d4d099374a1e1306783ff95489a1026bde9eb02ccc34a"},
{file = "fonttools-4.60.1-cp39-cp39-win_amd64.whl", hash = "sha256:7473a8ed9ed09aeaa191301244a5a9dbe46fe0bf54f9d6cd21d83044c3321217"},
{file = "fonttools-4.60.1-py3-none-any.whl", hash = "sha256:906306ac7afe2156fcf0042173d6ebbb05416af70f6b370967b47f8f00103bbb"},
{file = "fonttools-4.60.1.tar.gz", hash = "sha256:ef00af0439ebfee806b25f24c8f92109157ff3fac5731dc7867957812e87b8d9"},
]
[package.extras]
all = ["brotli (>=1.0.1) ; platform_python_implementation == \"CPython\"", "brotlicffi (>=0.8.0) ; platform_python_implementation != \"CPython\"", "lxml (>=4.0)", "lz4 (>=1.7.4.2)", "matplotlib", "munkres ; platform_python_implementation == \"PyPy\"", "pycairo", "scipy ; platform_python_implementation != \"PyPy\"", "skia-pathops (>=0.5.0)", "sympy", "uharfbuzz (>=0.23.0)", "unicodedata2 (>=15.1.0) ; python_version <= \"3.12\"", "xattr ; sys_platform == \"darwin\"", "zopfli (>=0.1.4)"]
graphite = ["lz4 (>=1.7.4.2)"]
interpolatable = ["munkres ; platform_python_implementation == \"PyPy\"", "pycairo", "scipy ; platform_python_implementation != \"PyPy\""]
lxml = ["lxml (>=4.0)"]
pathops = ["skia-pathops (>=0.5.0)"]
plot = ["matplotlib"]
repacker = ["uharfbuzz (>=0.23.0)"]
symfont = ["sympy"]
type1 = ["xattr ; sys_platform == \"darwin\""]
unicode = ["unicodedata2 (>=15.1.0) ; python_version <= \"3.12\""]
woff = ["brotli (>=1.0.1) ; platform_python_implementation == \"CPython\"", "brotlicffi (>=0.8.0) ; platform_python_implementation != \"CPython\"", "zopfli (>=0.1.4)"]
[[package]]
name = "freezegun"
version = "1.5.1"
@@ -2976,117 +2787,6 @@ files = [
[package.dependencies]
referencing = ">=0.31.0"
[[package]]
name = "kiwisolver"
version = "1.4.9"
description = "A fast implementation of the Cassowary constraint solver"
optional = false
python-versions = ">=3.10"
groups = ["main"]
files = [
{file = "kiwisolver-1.4.9-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:b4b4d74bda2b8ebf4da5bd42af11d02d04428b2c32846e4c2c93219df8a7987b"},
{file = "kiwisolver-1.4.9-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:fb3b8132019ea572f4611d770991000d7f58127560c4889729248eb5852a102f"},
{file = "kiwisolver-1.4.9-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:84fd60810829c27ae375114cd379da1fa65e6918e1da405f356a775d49a62bcf"},
{file = "kiwisolver-1.4.9-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:b78efa4c6e804ecdf727e580dbb9cba85624d2e1c6b5cb059c66290063bd99a9"},
{file = "kiwisolver-1.4.9-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d4efec7bcf21671db6a3294ff301d2fc861c31faa3c8740d1a94689234d1b415"},
{file = "kiwisolver-1.4.9-cp310-cp310-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:90f47e70293fc3688b71271100a1a5453aa9944a81d27ff779c108372cf5567b"},
{file = "kiwisolver-1.4.9-cp310-cp310-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:8fdca1def57a2e88ef339de1737a1449d6dbf5fab184c54a1fca01d541317154"},
{file = "kiwisolver-1.4.9-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:9cf554f21be770f5111a1690d42313e140355e687e05cf82cb23d0a721a64a48"},
{file = "kiwisolver-1.4.9-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:fc1795ac5cd0510207482c3d1d3ed781143383b8cfd36f5c645f3897ce066220"},
{file = "kiwisolver-1.4.9-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:ccd09f20ccdbbd341b21a67ab50a119b64a403b09288c27481575105283c1586"},
{file = "kiwisolver-1.4.9-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:540c7c72324d864406a009d72f5d6856f49693db95d1fbb46cf86febef873634"},
{file = "kiwisolver-1.4.9-cp310-cp310-win_amd64.whl", hash = "sha256:ede8c6d533bc6601a47ad4046080d36b8fc99f81e6f1c17b0ac3c2dc91ac7611"},
{file = "kiwisolver-1.4.9-cp310-cp310-win_arm64.whl", hash = "sha256:7b4da0d01ac866a57dd61ac258c5607b4cd677f63abaec7b148354d2b2cdd536"},
{file = "kiwisolver-1.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:eb14a5da6dc7642b0f3a18f13654847cd8b7a2550e2645a5bda677862b03ba16"},
{file = "kiwisolver-1.4.9-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:39a219e1c81ae3b103643d2aedb90f1ef22650deb266ff12a19e7773f3e5f089"},
{file = "kiwisolver-1.4.9-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2405a7d98604b87f3fc28b1716783534b1b4b8510d8142adca34ee0bc3c87543"},
{file = "kiwisolver-1.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:dc1ae486f9abcef254b5618dfb4113dd49f94c68e3e027d03cf0143f3f772b61"},
{file = "kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a1f570ce4d62d718dce3f179ee78dac3b545ac16c0c04bb363b7607a949c0d1"},
{file = "kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb27e7b78d716c591e88e0a09a2139c6577865d7f2e152488c2cc6257f460872"},
{file = "kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:15163165efc2f627eb9687ea5f3a28137217d217ac4024893d753f46bce9de26"},
{file = "kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:bdee92c56a71d2b24c33a7d4c2856bd6419d017e08caa7802d2963870e315028"},
{file = "kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:412f287c55a6f54b0650bd9b6dce5aceddb95864a1a90c87af16979d37c89771"},
{file = "kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:2c93f00dcba2eea70af2be5f11a830a742fe6b579a1d4e00f47760ef13be247a"},
{file = "kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f117e1a089d9411663a3207ba874f31be9ac8eaa5b533787024dc07aeb74f464"},
{file = "kiwisolver-1.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:be6a04e6c79819c9a8c2373317d19a96048e5a3f90bec587787e86a1153883c2"},
{file = "kiwisolver-1.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:0ae37737256ba2de764ddc12aed4956460277f00c4996d51a197e72f62f5eec7"},
{file = "kiwisolver-1.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:ac5a486ac389dddcc5bef4f365b6ae3ffff2c433324fb38dd35e3fab7c957999"},
{file = "kiwisolver-1.4.9-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f2ba92255faa7309d06fe44c3a4a97efe1c8d640c2a79a5ef728b685762a6fd2"},
{file = "kiwisolver-1.4.9-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4a2899935e724dd1074cb568ce7ac0dce28b2cd6ab539c8e001a8578eb106d14"},
{file = "kiwisolver-1.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f6008a4919fdbc0b0097089f67a1eb55d950ed7e90ce2cc3e640abadd2757a04"},
{file = "kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:67bb8b474b4181770f926f7b7d2f8c0248cbcb78b660fdd41a47054b28d2a752"},
{file = "kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2327a4a30d3ee07d2fbe2e7933e8a37c591663b96ce42a00bc67461a87d7df77"},
{file = "kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7a08b491ec91b1d5053ac177afe5290adacf1f0f6307d771ccac5de30592d198"},
{file = "kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d8fc5c867c22b828001b6a38d2eaeb88160bf5783c6cb4a5e440efc981ce286d"},
{file = "kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:3b3115b2581ea35bb6d1f24a4c90af37e5d9b49dcff267eeed14c3893c5b86ab"},
{file = "kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:858e4c22fb075920b96a291928cb7dea5644e94c0ee4fcd5af7e865655e4ccf2"},
{file = "kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ed0fecd28cc62c54b262e3736f8bb2512d8dcfdc2bcf08be5f47f96bf405b145"},
{file = "kiwisolver-1.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:f68208a520c3d86ea51acf688a3e3002615a7f0238002cccc17affecc86a8a54"},
{file = "kiwisolver-1.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:2c1a4f57df73965f3f14df20b80ee29e6a7930a57d2d9e8491a25f676e197c60"},
{file = "kiwisolver-1.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a5d0432ccf1c7ab14f9949eec60c5d1f924f17c037e9f8b33352fa05799359b8"},
{file = "kiwisolver-1.4.9-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:efb3a45b35622bb6c16dbfab491a8f5a391fe0e9d45ef32f4df85658232ca0e2"},
{file = "kiwisolver-1.4.9-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1a12cf6398e8a0a001a059747a1cbf24705e18fe413bc22de7b3d15c67cffe3f"},
{file = "kiwisolver-1.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b67e6efbf68e077dd71d1a6b37e43e1a99d0bff1a3d51867d45ee8908b931098"},
{file = "kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5656aa670507437af0207645273ccdfee4f14bacd7f7c67a4306d0dcaeaf6eed"},
{file = "kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bfc08add558155345129c7803b3671cf195e6a56e7a12f3dde7c57d9b417f525"},
{file = "kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:40092754720b174e6ccf9e845d0d8c7d8e12c3d71e7fc35f55f3813e96376f78"},
{file = "kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:497d05f29a1300d14e02e6441cf0f5ee81c1ff5a304b0d9fb77423974684e08b"},
{file = "kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:bdd1a81a1860476eb41ac4bc1e07b3f07259e6d55bbf739b79c8aaedcf512799"},
{file = "kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:e6b93f13371d341afee3be9f7c5964e3fe61d5fa30f6a30eb49856935dfe4fc3"},
{file = "kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:d75aa530ccfaa593da12834b86a0724f58bff12706659baa9227c2ccaa06264c"},
{file = "kiwisolver-1.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:dd0a578400839256df88c16abddf9ba14813ec5f21362e1fe65022e00c883d4d"},
{file = "kiwisolver-1.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:d4188e73af84ca82468f09cadc5ac4db578109e52acb4518d8154698d3a87ca2"},
{file = "kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:5a0f2724dfd4e3b3ac5a82436a8e6fd16baa7d507117e4279b660fe8ca38a3a1"},
{file = "kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:1b11d6a633e4ed84fc0ddafd4ebfd8ea49b3f25082c04ad12b8315c11d504dc1"},
{file = "kiwisolver-1.4.9-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:61874cdb0a36016354853593cffc38e56fc9ca5aa97d2c05d3dcf6922cd55a11"},
{file = "kiwisolver-1.4.9-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:60c439763a969a6af93b4881db0eed8fadf93ee98e18cbc35bc8da868d0c4f0c"},
{file = "kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92a2f997387a1b79a75e7803aa7ded2cfbe2823852ccf1ba3bcf613b62ae3197"},
{file = "kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a31d512c812daea6d8b3be3b2bfcbeb091dbb09177706569bcfc6240dcf8b41c"},
{file = "kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:52a15b0f35dad39862d376df10c5230155243a2c1a436e39eb55623ccbd68185"},
{file = "kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a30fd6fdef1430fd9e1ba7b3398b5ee4e2887783917a687d86ba69985fb08748"},
{file = "kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:cc9617b46837c6468197b5945e196ee9ca43057bb7d9d1ae688101e4e1dddf64"},
{file = "kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:0ab74e19f6a2b027ea4f845a78827969af45ce790e6cb3e1ebab71bdf9f215ff"},
{file = "kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:dba5ee5d3981160c28d5490f0d1b7ed730c22470ff7f6cc26cfcfaacb9896a07"},
{file = "kiwisolver-1.4.9-cp313-cp313t-win_arm64.whl", hash = "sha256:0749fd8f4218ad2e851e11cc4dc05c7cbc0cbc4267bdfdb31782e65aace4ee9c"},
{file = "kiwisolver-1.4.9-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:9928fe1eb816d11ae170885a74d074f57af3a0d65777ca47e9aeb854a1fba386"},
{file = "kiwisolver-1.4.9-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:d0005b053977e7b43388ddec89fa567f43d4f6d5c2c0affe57de5ebf290dc552"},
{file = "kiwisolver-1.4.9-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:2635d352d67458b66fd0667c14cb1d4145e9560d503219034a18a87e971ce4f3"},
{file = "kiwisolver-1.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:767c23ad1c58c9e827b649a9ab7809fd5fd9db266a9cf02b0e926ddc2c680d58"},
{file = "kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72d0eb9fba308b8311685c2268cf7d0a0639a6cd027d8128659f72bdd8a024b4"},
{file = "kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f68e4f3eeca8fb22cc3d731f9715a13b652795ef657a13df1ad0c7dc0e9731df"},
{file = "kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d84cd4061ae292d8ac367b2c3fa3aad11cb8625a95d135fe93f286f914f3f5a6"},
{file = "kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a60ea74330b91bd22a29638940d115df9dc00af5035a9a2a6ad9399ffb4ceca5"},
{file = "kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:ce6a3a4e106cf35c2d9c4fa17c05ce0b180db622736845d4315519397a77beaf"},
{file = "kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:77937e5e2a38a7b48eef0585114fe7930346993a88060d0bf886086d2aa49ef5"},
{file = "kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:24c175051354f4a28c5d6a31c93906dc653e2bf234e8a4bbfb964892078898ce"},
{file = "kiwisolver-1.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:0763515d4df10edf6d06a3c19734e2566368980d21ebec439f33f9eb936c07b7"},
{file = "kiwisolver-1.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:0e4e2bf29574a6a7b7f6cb5fa69293b9f96c928949ac4a53ba3f525dffb87f9c"},
{file = "kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d976bbb382b202f71c67f77b0ac11244021cfa3f7dfd9e562eefcea2df711548"},
{file = "kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2489e4e5d7ef9a1c300a5e0196e43d9c739f066ef23270607d45aba368b91f2d"},
{file = "kiwisolver-1.4.9-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:e2ea9f7ab7fbf18fffb1b5434ce7c69a07582f7acc7717720f1d69f3e806f90c"},
{file = "kiwisolver-1.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b34e51affded8faee0dfdb705416153819d8ea9250bbbf7ea1b249bdeb5f1122"},
{file = "kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8aacd3d4b33b772542b2e01beb50187536967b514b00003bdda7589722d2a64"},
{file = "kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7cf974dd4e35fa315563ac99d6287a1024e4dc2077b8a7d7cd3d2fb65d283134"},
{file = "kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:85bd218b5ecfbee8c8a82e121802dcb519a86044c9c3b2e4aef02fa05c6da370"},
{file = "kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:0856e241c2d3df4efef7c04a1e46b1936b6120c9bcf36dd216e3acd84bc4fb21"},
{file = "kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:9af39d6551f97d31a4deebeac6f45b156f9755ddc59c07b402c148f5dbb6482a"},
{file = "kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:bb4ae2b57fc1d8cbd1cf7b1d9913803681ffa903e7488012be5b76dedf49297f"},
{file = "kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:aedff62918805fb62d43a4aa2ecd4482c380dc76cd31bd7c8878588a61bd0369"},
{file = "kiwisolver-1.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:1fa333e8b2ce4d9660f2cda9c0e1b6bafcfb2457a9d259faa82289e73ec24891"},
{file = "kiwisolver-1.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:4a48a2ce79d65d363597ef7b567ce3d14d68783d2b2263d98db3d9477805ba32"},
{file = "kiwisolver-1.4.9-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:4d1d9e582ad4d63062d34077a9a1e9f3c34088a2ec5135b1f7190c07cf366527"},
{file = "kiwisolver-1.4.9-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:deed0c7258ceb4c44ad5ec7d9918f9f14fd05b2be86378d86cf50e63d1e7b771"},
{file = "kiwisolver-1.4.9-pp310-pypy310_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0a590506f303f512dff6b7f75fd2fd18e16943efee932008fe7140e5fa91d80e"},
{file = "kiwisolver-1.4.9-pp310-pypy310_pp73-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e09c2279a4d01f099f52d5c4b3d9e208e91edcbd1a175c9662a8b16e000fece9"},
{file = "kiwisolver-1.4.9-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:c9e7cdf45d594ee04d5be1b24dd9d49f3d1590959b2271fb30b5ca2b262c00fb"},
{file = "kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:720e05574713db64c356e86732c0f3c5252818d05f9df320f0ad8380641acea5"},
{file = "kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:17680d737d5335b552994a2008fab4c851bcd7de33094a82067ef3a576ff02fa"},
{file = "kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:85b5352f94e490c028926ea567fc569c52ec79ce131dadb968d3853e809518c2"},
{file = "kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:464415881e4801295659462c49461a24fb107c140de781d55518c4b80cb6790f"},
{file = "kiwisolver-1.4.9-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:fb940820c63a9590d31d88b815e7a3aa5915cad3ce735ab45f0c730b39547de1"},
{file = "kiwisolver-1.4.9.tar.gz", hash = "sha256:c3b22c26c6fd6811b0ae8363b95ca8ce4ea3c202d3d0975b2914310ceb1bcc4d"},
]
[[package]]
name = "kombu"
version = "5.5.4"
@@ -3437,85 +3137,6 @@ dev = ["marshmallow[tests]", "pre-commit (>=3.5,<5.0)", "tox"]
docs = ["autodocsumm (==0.2.14)", "furo (==2024.8.6)", "sphinx (==8.1.3)", "sphinx-copybutton (==0.5.2)", "sphinx-issues (==5.0.0)", "sphinxext-opengraph (==0.9.1)"]
tests = ["pytest", "simplejson"]
[[package]]
name = "matplotlib"
version = "3.10.6"
description = "Python plotting package"
optional = false
python-versions = ">=3.10"
groups = ["main"]
files = [
{file = "matplotlib-3.10.6-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:bc7316c306d97463a9866b89d5cc217824e799fa0de346c8f68f4f3d27c8693d"},
{file = "matplotlib-3.10.6-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:d00932b0d160ef03f59f9c0e16d1e3ac89646f7785165ce6ad40c842db16cc2e"},
{file = "matplotlib-3.10.6-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8fa4c43d6bfdbfec09c733bca8667de11bfa4970e8324c471f3a3632a0301c15"},
{file = "matplotlib-3.10.6-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ea117a9c1627acaa04dbf36265691921b999cbf515a015298e54e1a12c3af837"},
{file = "matplotlib-3.10.6-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:08fc803293b4e1694ee325896030de97f74c141ccff0be886bb5915269247676"},
{file = "matplotlib-3.10.6-cp310-cp310-win_amd64.whl", hash = "sha256:2adf92d9b7527fbfb8818e050260f0ebaa460f79d61546374ce73506c9421d09"},
{file = "matplotlib-3.10.6-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:905b60d1cb0ee604ce65b297b61cf8be9f4e6cfecf95a3fe1c388b5266bc8f4f"},
{file = "matplotlib-3.10.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7bac38d816637343e53d7185d0c66677ff30ffb131044a81898b5792c956ba76"},
{file = "matplotlib-3.10.6-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:942a8de2b5bfff1de31d95722f702e2966b8a7e31f4e68f7cd963c7cd8861cf6"},
{file = "matplotlib-3.10.6-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a3276c85370bc0dfca051ec65c5817d1e0f8f5ce1b7787528ec8ed2d524bbc2f"},
{file = "matplotlib-3.10.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:9df5851b219225731f564e4b9e7f2ac1e13c9e6481f941b5631a0f8e2d9387ce"},
{file = "matplotlib-3.10.6-cp311-cp311-win_amd64.whl", hash = "sha256:abb5d9478625dd9c9eb51a06d39aae71eda749ae9b3138afb23eb38824026c7e"},
{file = "matplotlib-3.10.6-cp311-cp311-win_arm64.whl", hash = "sha256:886f989ccfae63659183173bb3fced7fd65e9eb793c3cc21c273add368536951"},
{file = "matplotlib-3.10.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:31ca662df6a80bd426f871105fdd69db7543e28e73a9f2afe80de7e531eb2347"},
{file = "matplotlib-3.10.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1678bb61d897bb4ac4757b5ecfb02bfb3fddf7f808000fb81e09c510712fda75"},
{file = "matplotlib-3.10.6-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:56cd2d20842f58c03d2d6e6c1f1cf5548ad6f66b91e1e48f814e4fb5abd1cb95"},
{file = "matplotlib-3.10.6-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:662df55604a2f9a45435566d6e2660e41efe83cd94f4288dfbf1e6d1eae4b0bb"},
{file = "matplotlib-3.10.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:08f141d55148cd1fc870c3387d70ca4df16dee10e909b3b038782bd4bda6ea07"},
{file = "matplotlib-3.10.6-cp312-cp312-win_amd64.whl", hash = "sha256:590f5925c2d650b5c9d813c5b3b5fc53f2929c3f8ef463e4ecfa7e052044fb2b"},
{file = "matplotlib-3.10.6-cp312-cp312-win_arm64.whl", hash = "sha256:f44c8d264a71609c79a78d50349e724f5d5fc3684ead7c2a473665ee63d868aa"},
{file = "matplotlib-3.10.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:819e409653c1106c8deaf62e6de6b8611449c2cd9939acb0d7d4e57a3d95cc7a"},
{file = "matplotlib-3.10.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:59c8ac8382fefb9cb71308dde16a7c487432f5255d8f1fd32473523abecfecdf"},
{file = "matplotlib-3.10.6-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:84e82d9e0fd70c70bc55739defbd8055c54300750cbacf4740c9673a24d6933a"},
{file = "matplotlib-3.10.6-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:25f7a3eb42d6c1c56e89eacd495661fc815ffc08d9da750bca766771c0fd9110"},
{file = "matplotlib-3.10.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f9c862d91ec0b7842920a4cfdaaec29662195301914ea54c33e01f1a28d014b2"},
{file = "matplotlib-3.10.6-cp313-cp313-win_amd64.whl", hash = "sha256:1b53bd6337eba483e2e7d29c5ab10eee644bc3a2491ec67cc55f7b44583ffb18"},
{file = "matplotlib-3.10.6-cp313-cp313-win_arm64.whl", hash = "sha256:cbd5eb50b7058b2892ce45c2f4e92557f395c9991f5c886d1bb74a1582e70fd6"},
{file = "matplotlib-3.10.6-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:acc86dd6e0e695c095001a7fccff158c49e45e0758fdf5dcdbb0103318b59c9f"},
{file = "matplotlib-3.10.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:e228cd2ffb8f88b7d0b29e37f68ca9aaf83e33821f24a5ccc4f082dd8396bc27"},
{file = "matplotlib-3.10.6-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:658bc91894adeab669cf4bb4a186d049948262987e80f0857216387d7435d833"},
{file = "matplotlib-3.10.6-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8913b7474f6dd83ac444c9459c91f7f0f2859e839f41d642691b104e0af056aa"},
{file = "matplotlib-3.10.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:091cea22e059b89f6d7d1a18e2c33a7376c26eee60e401d92a4d6726c4e12706"},
{file = "matplotlib-3.10.6-cp313-cp313t-win_amd64.whl", hash = "sha256:491e25e02a23d7207629d942c666924a6b61e007a48177fdd231a0097b7f507e"},
{file = "matplotlib-3.10.6-cp313-cp313t-win_arm64.whl", hash = "sha256:3d80d60d4e54cda462e2cd9a086d85cd9f20943ead92f575ce86885a43a565d5"},
{file = "matplotlib-3.10.6-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:70aaf890ce1d0efd482df969b28a5b30ea0b891224bb315810a3940f67182899"},
{file = "matplotlib-3.10.6-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1565aae810ab79cb72e402b22facfa6501365e73ebab70a0fdfb98488d2c3c0c"},
{file = "matplotlib-3.10.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f3b23315a01981689aa4e1a179dbf6ef9fbd17143c3eea77548c2ecfb0499438"},
{file = "matplotlib-3.10.6-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:30fdd37edf41a4e6785f9b37969de57aea770696cb637d9946eb37470c94a453"},
{file = "matplotlib-3.10.6-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:bc31e693da1c08012c764b053e702c1855378e04102238e6a5ee6a7117c53a47"},
{file = "matplotlib-3.10.6-cp314-cp314-win_amd64.whl", hash = "sha256:05be9bdaa8b242bc6ff96330d18c52f1fc59c6fb3a4dd411d953d67e7e1baf98"},
{file = "matplotlib-3.10.6-cp314-cp314-win_arm64.whl", hash = "sha256:f56a0d1ab05d34c628592435781d185cd99630bdfd76822cd686fb5a0aecd43a"},
{file = "matplotlib-3.10.6-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:94f0b4cacb23763b64b5dace50d5b7bfe98710fed5f0cef5c08135a03399d98b"},
{file = "matplotlib-3.10.6-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:cc332891306b9fb39462673d8225d1b824c89783fee82840a709f96714f17a5c"},
{file = "matplotlib-3.10.6-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee1d607b3fb1590deb04b69f02ea1d53ed0b0bf75b2b1a5745f269afcbd3cdd3"},
{file = "matplotlib-3.10.6-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:376a624a218116461696b27b2bbf7a8945053e6d799f6502fc03226d077807bf"},
{file = "matplotlib-3.10.6-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:83847b47f6524c34b4f2d3ce726bb0541c48c8e7692729865c3df75bfa0f495a"},
{file = "matplotlib-3.10.6-cp314-cp314t-win_amd64.whl", hash = "sha256:c7e0518e0d223683532a07f4b512e2e0729b62674f1b3a1a69869f98e6b1c7e3"},
{file = "matplotlib-3.10.6-cp314-cp314t-win_arm64.whl", hash = "sha256:4dd83e029f5b4801eeb87c64efd80e732452781c16a9cf7415b7b63ec8f374d7"},
{file = "matplotlib-3.10.6-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:13fcd07ccf17e354398358e0307a1f53f5325dca22982556ddb9c52837b5af41"},
{file = "matplotlib-3.10.6-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:470fc846d59d1406e34fa4c32ba371039cd12c2fe86801159a965956f2575bd1"},
{file = "matplotlib-3.10.6-pp310-pypy310_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f7173f8551b88f4ef810a94adae3128c2530e0d07529f7141be7f8d8c365f051"},
{file = "matplotlib-3.10.6-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:f2d684c3204fa62421bbf770ddfebc6b50130f9cad65531eeba19236d73bb488"},
{file = "matplotlib-3.10.6-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:6f4a69196e663a41d12a728fab8751177215357906436804217d6d9cf0d4d6cf"},
{file = "matplotlib-3.10.6-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:4d6ca6ef03dfd269f4ead566ec6f3fb9becf8dab146fb999022ed85ee9f6b3eb"},
{file = "matplotlib-3.10.6.tar.gz", hash = "sha256:ec01b645840dd1996df21ee37f208cd8ba57644779fa20464010638013d3203c"},
]
[package.dependencies]
contourpy = ">=1.0.1"
cycler = ">=0.10"
fonttools = ">=4.22.0"
kiwisolver = ">=1.3.1"
numpy = ">=1.23"
packaging = ">=20.0"
pillow = ">=8"
pyparsing = ">=2.3.1"
python-dateutil = ">=2.7"
[package.extras]
dev = ["meson-python (>=0.13.1,<0.17.0)", "pybind11 (>=2.13.2,!=2.13.3)", "setuptools (>=64)", "setuptools_scm (>=7)"]
[[package]]
name = "mccabe"
version = "0.7.0"
@@ -4236,131 +3857,6 @@ files = [
[package.dependencies]
setuptools = "*"
[[package]]
name = "pillow"
version = "11.3.0"
description = "Python Imaging Library (Fork)"
optional = false
python-versions = ">=3.9"
groups = ["main"]
files = [
{file = "pillow-11.3.0-cp310-cp310-macosx_10_10_x86_64.whl", hash = "sha256:1b9c17fd4ace828b3003dfd1e30bff24863e0eb59b535e8f80194d9cc7ecf860"},
{file = "pillow-11.3.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:65dc69160114cdd0ca0f35cb434633c75e8e7fad4cf855177a05bf38678f73ad"},
{file = "pillow-11.3.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7107195ddc914f656c7fc8e4a5e1c25f32e9236ea3ea860f257b0436011fddd0"},
{file = "pillow-11.3.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:cc3e831b563b3114baac7ec2ee86819eb03caa1a2cef0b481a5675b59c4fe23b"},
{file = "pillow-11.3.0-cp310-cp310-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f1f182ebd2303acf8c380a54f615ec883322593320a9b00438eb842c1f37ae50"},
{file = "pillow-11.3.0-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4445fa62e15936a028672fd48c4c11a66d641d2c05726c7ec1f8ba6a572036ae"},
{file = "pillow-11.3.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:71f511f6b3b91dd543282477be45a033e4845a40278fa8dcdbfdb07109bf18f9"},
{file = "pillow-11.3.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:040a5b691b0713e1f6cbe222e0f4f74cd233421e105850ae3b3c0ceda520f42e"},
{file = "pillow-11.3.0-cp310-cp310-win32.whl", hash = "sha256:89bd777bc6624fe4115e9fac3352c79ed60f3bb18651420635f26e643e3dd1f6"},
{file = "pillow-11.3.0-cp310-cp310-win_amd64.whl", hash = "sha256:19d2ff547c75b8e3ff46f4d9ef969a06c30ab2d4263a9e287733aa8b2429ce8f"},
{file = "pillow-11.3.0-cp310-cp310-win_arm64.whl", hash = "sha256:819931d25e57b513242859ce1876c58c59dc31587847bf74cfe06b2e0cb22d2f"},
{file = "pillow-11.3.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:1cd110edf822773368b396281a2293aeb91c90a2db00d78ea43e7e861631b722"},
{file = "pillow-11.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:9c412fddd1b77a75aa904615ebaa6001f169b26fd467b4be93aded278266b288"},
{file = "pillow-11.3.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7d1aa4de119a0ecac0a34a9c8bde33f34022e2e8f99104e47a3ca392fd60e37d"},
{file = "pillow-11.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:91da1d88226663594e3f6b4b8c3c8d85bd504117d043740a8e0ec449087cc494"},
{file = "pillow-11.3.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:643f189248837533073c405ec2f0bb250ba54598cf80e8c1e043381a60632f58"},
{file = "pillow-11.3.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:106064daa23a745510dabce1d84f29137a37224831d88eb4ce94bb187b1d7e5f"},
{file = "pillow-11.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:cd8ff254faf15591e724dc7c4ddb6bf4793efcbe13802a4ae3e863cd300b493e"},
{file = "pillow-11.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:932c754c2d51ad2b2271fd01c3d121daaa35e27efae2a616f77bf164bc0b3e94"},
{file = "pillow-11.3.0-cp311-cp311-win32.whl", hash = "sha256:b4b8f3efc8d530a1544e5962bd6b403d5f7fe8b9e08227c6b255f98ad82b4ba0"},
{file = "pillow-11.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:1a992e86b0dd7aeb1f053cd506508c0999d710a8f07b4c791c63843fc6a807ac"},
{file = "pillow-11.3.0-cp311-cp311-win_arm64.whl", hash = "sha256:30807c931ff7c095620fe04448e2c2fc673fcbb1ffe2a7da3fb39613489b1ddd"},
{file = "pillow-11.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:fdae223722da47b024b867c1ea0be64e0df702c5e0a60e27daad39bf960dd1e4"},
{file = "pillow-11.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:921bd305b10e82b4d1f5e802b6850677f965d8394203d182f078873851dada69"},
{file = "pillow-11.3.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:eb76541cba2f958032d79d143b98a3a6b3ea87f0959bbe256c0b5e416599fd5d"},
{file = "pillow-11.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:67172f2944ebba3d4a7b54f2e95c786a3a50c21b88456329314caaa28cda70f6"},
{file = "pillow-11.3.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:97f07ed9f56a3b9b5f49d3661dc9607484e85c67e27f3e8be2c7d28ca032fec7"},
{file = "pillow-11.3.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:676b2815362456b5b3216b4fd5bd89d362100dc6f4945154ff172e206a22c024"},
{file = "pillow-11.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3e184b2f26ff146363dd07bde8b711833d7b0202e27d13540bfe2e35a323a809"},
{file = "pillow-11.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6be31e3fc9a621e071bc17bb7de63b85cbe0bfae91bb0363c893cbe67247780d"},
{file = "pillow-11.3.0-cp312-cp312-win32.whl", hash = "sha256:7b161756381f0918e05e7cb8a371fff367e807770f8fe92ecb20d905d0e1c149"},
{file = "pillow-11.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a6444696fce635783440b7f7a9fc24b3ad10a9ea3f0ab66c5905be1c19ccf17d"},
{file = "pillow-11.3.0-cp312-cp312-win_arm64.whl", hash = "sha256:2aceea54f957dd4448264f9bf40875da0415c83eb85f55069d89c0ed436e3542"},
{file = "pillow-11.3.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:1c627742b539bba4309df89171356fcb3cc5a9178355b2727d1b74a6cf155fbd"},
{file = "pillow-11.3.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:30b7c02f3899d10f13d7a48163c8969e4e653f8b43416d23d13d1bbfdc93b9f8"},
{file = "pillow-11.3.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:7859a4cc7c9295f5838015d8cc0a9c215b77e43d07a25e460f35cf516df8626f"},
{file = "pillow-11.3.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ec1ee50470b0d050984394423d96325b744d55c701a439d2bd66089bff963d3c"},
{file = "pillow-11.3.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:7db51d222548ccfd274e4572fdbf3e810a5e66b00608862f947b163e613b67dd"},
{file = "pillow-11.3.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:2d6fcc902a24ac74495df63faad1884282239265c6839a0a6416d33faedfae7e"},
{file = "pillow-11.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f0f5d8f4a08090c6d6d578351a2b91acf519a54986c055af27e7a93feae6d3f1"},
{file = "pillow-11.3.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c37d8ba9411d6003bba9e518db0db0c58a680ab9fe5179f040b0463644bc9805"},
{file = "pillow-11.3.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:13f87d581e71d9189ab21fe0efb5a23e9f28552d5be6979e84001d3b8505abe8"},
{file = "pillow-11.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:023f6d2d11784a465f09fd09a34b150ea4672e85fb3d05931d89f373ab14abb2"},
{file = "pillow-11.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:45dfc51ac5975b938e9809451c51734124e73b04d0f0ac621649821a63852e7b"},
{file = "pillow-11.3.0-cp313-cp313-win32.whl", hash = "sha256:a4d336baed65d50d37b88ca5b60c0fa9d81e3a87d4a7930d3880d1624d5b31f3"},
{file = "pillow-11.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:0bce5c4fd0921f99d2e858dc4d4d64193407e1b99478bc5cacecba2311abde51"},
{file = "pillow-11.3.0-cp313-cp313-win_arm64.whl", hash = "sha256:1904e1264881f682f02b7f8167935cce37bc97db457f8e7849dc3a6a52b99580"},
{file = "pillow-11.3.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:4c834a3921375c48ee6b9624061076bc0a32a60b5532b322cc0ea64e639dd50e"},
{file = "pillow-11.3.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:5e05688ccef30ea69b9317a9ead994b93975104a677a36a8ed8106be9260aa6d"},
{file = "pillow-11.3.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:1019b04af07fc0163e2810167918cb5add8d74674b6267616021ab558dc98ced"},
{file = "pillow-11.3.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f944255db153ebb2b19c51fe85dd99ef0ce494123f21b9db4877ffdfc5590c7c"},
{file = "pillow-11.3.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1f85acb69adf2aaee8b7da124efebbdb959a104db34d3a2cb0f3793dbae422a8"},
{file = "pillow-11.3.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:05f6ecbeff5005399bb48d198f098a9b4b6bdf27b8487c7f38ca16eeb070cd59"},
{file = "pillow-11.3.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a7bc6e6fd0395bc052f16b1a8670859964dbd7003bd0af2ff08342eb6e442cfe"},
{file = "pillow-11.3.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:83e1b0161c9d148125083a35c1c5a89db5b7054834fd4387499e06552035236c"},
{file = "pillow-11.3.0-cp313-cp313t-win32.whl", hash = "sha256:2a3117c06b8fb646639dce83694f2f9eac405472713fcb1ae887469c0d4f6788"},
{file = "pillow-11.3.0-cp313-cp313t-win_amd64.whl", hash = "sha256:857844335c95bea93fb39e0fa2726b4d9d758850b34075a7e3ff4f4fa3aa3b31"},
{file = "pillow-11.3.0-cp313-cp313t-win_arm64.whl", hash = "sha256:8797edc41f3e8536ae4b10897ee2f637235c94f27404cac7297f7b607dd0716e"},
{file = "pillow-11.3.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:d9da3df5f9ea2a89b81bb6087177fb1f4d1c7146d583a3fe5c672c0d94e55e12"},
{file = "pillow-11.3.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:0b275ff9b04df7b640c59ec5a3cb113eefd3795a8df80bac69646ef699c6981a"},
{file = "pillow-11.3.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:0743841cabd3dba6a83f38a92672cccbd69af56e3e91777b0ee7f4dba4385632"},
{file = "pillow-11.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:2465a69cf967b8b49ee1b96d76718cd98c4e925414ead59fdf75cf0fd07df673"},
{file = "pillow-11.3.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:41742638139424703b4d01665b807c6468e23e699e8e90cffefe291c5832b027"},
{file = "pillow-11.3.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:93efb0b4de7e340d99057415c749175e24c8864302369e05914682ba642e5d77"},
{file = "pillow-11.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7966e38dcd0fa11ca390aed7c6f20454443581d758242023cf36fcb319b1a874"},
{file = "pillow-11.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:98a9afa7b9007c67ed84c57c9e0ad86a6000da96eaa638e4f8abe5b65ff83f0a"},
{file = "pillow-11.3.0-cp314-cp314-win32.whl", hash = "sha256:02a723e6bf909e7cea0dac1b0e0310be9d7650cd66222a5f1c571455c0a45214"},
{file = "pillow-11.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:a418486160228f64dd9e9efcd132679b7a02a5f22c982c78b6fc7dab3fefb635"},
{file = "pillow-11.3.0-cp314-cp314-win_arm64.whl", hash = "sha256:155658efb5e044669c08896c0c44231c5e9abcaadbc5cd3648df2f7c0b96b9a6"},
{file = "pillow-11.3.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:59a03cdf019efbfeeed910bf79c7c93255c3d54bc45898ac2a4140071b02b4ae"},
{file = "pillow-11.3.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:f8a5827f84d973d8636e9dc5764af4f0cf2318d26744b3d902931701b0d46653"},
{file = "pillow-11.3.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:ee92f2fd10f4adc4b43d07ec5e779932b4eb3dbfbc34790ada5a6669bc095aa6"},
{file = "pillow-11.3.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c96d333dcf42d01f47b37e0979b6bd73ec91eae18614864622d9b87bbd5bbf36"},
{file = "pillow-11.3.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4c96f993ab8c98460cd0c001447bff6194403e8b1d7e149ade5f00594918128b"},
{file = "pillow-11.3.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:41342b64afeba938edb034d122b2dda5db2139b9a4af999729ba8818e0056477"},
{file = "pillow-11.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:068d9c39a2d1b358eb9f245ce7ab1b5c3246c7c8c7d9ba58cfa5b43146c06e50"},
{file = "pillow-11.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:a1bc6ba083b145187f648b667e05a2534ecc4b9f2784c2cbe3089e44868f2b9b"},
{file = "pillow-11.3.0-cp314-cp314t-win32.whl", hash = "sha256:118ca10c0d60b06d006be10a501fd6bbdfef559251ed31b794668ed569c87e12"},
{file = "pillow-11.3.0-cp314-cp314t-win_amd64.whl", hash = "sha256:8924748b688aa210d79883357d102cd64690e56b923a186f35a82cbc10f997db"},
{file = "pillow-11.3.0-cp314-cp314t-win_arm64.whl", hash = "sha256:79ea0d14d3ebad43ec77ad5272e6ff9bba5b679ef73375ea760261207fa8e0aa"},
{file = "pillow-11.3.0-cp39-cp39-macosx_10_10_x86_64.whl", hash = "sha256:48d254f8a4c776de343051023eb61ffe818299eeac478da55227d96e241de53f"},
{file = "pillow-11.3.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:7aee118e30a4cf54fdd873bd3a29de51e29105ab11f9aad8c32123f58c8f8081"},
{file = "pillow-11.3.0-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:23cff760a9049c502721bdb743a7cb3e03365fafcdfc2ef9784610714166e5a4"},
{file = "pillow-11.3.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6359a3bc43f57d5b375d1ad54a0074318a0844d11b76abccf478c37c986d3cfc"},
{file = "pillow-11.3.0-cp39-cp39-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:092c80c76635f5ecb10f3f83d76716165c96f5229addbd1ec2bdbbda7d496e06"},
{file = "pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cadc9e0ea0a2431124cde7e1697106471fc4c1da01530e679b2391c37d3fbb3a"},
{file = "pillow-11.3.0-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:6a418691000f2a418c9135a7cf0d797c1bb7d9a485e61fe8e7722845b95ef978"},
{file = "pillow-11.3.0-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:97afb3a00b65cc0804d1c7abddbf090a81eaac02768af58cbdcaaa0a931e0b6d"},
{file = "pillow-11.3.0-cp39-cp39-win32.whl", hash = "sha256:ea944117a7974ae78059fcc1800e5d3295172bb97035c0c1d9345fca1419da71"},
{file = "pillow-11.3.0-cp39-cp39-win_amd64.whl", hash = "sha256:e5c5858ad8ec655450a7c7df532e9842cf8df7cc349df7225c60d5d348c8aada"},
{file = "pillow-11.3.0-cp39-cp39-win_arm64.whl", hash = "sha256:6abdbfd3aea42be05702a8dd98832329c167ee84400a1d1f61ab11437f1717eb"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:3cee80663f29e3843b68199b9d6f4f54bd1d4a6b59bdd91bceefc51238bcb967"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:b5f56c3f344f2ccaf0dd875d3e180f631dc60a51b314295a3e681fe8cf851fbe"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e67d793d180c9df62f1f40aee3accca4829d3794c95098887edc18af4b8b780c"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d000f46e2917c705e9fb93a3606ee4a819d1e3aa7a9b442f6444f07e77cf5e25"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:527b37216b6ac3a12d7838dc3bd75208ec57c1c6d11ef01902266a5a0c14fc27"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:be5463ac478b623b9dd3937afd7fb7ab3d79dd290a28e2b6df292dc75063eb8a"},
{file = "pillow-11.3.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:8dc70ca24c110503e16918a658b869019126ecfe03109b754c402daff12b3d9f"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:7c8ec7a017ad1bd562f93dbd8505763e688d388cde6e4a010ae1486916e713e6"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:9ab6ae226de48019caa8074894544af5b53a117ccb9d3b3dcb2871464c829438"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:fe27fb049cdcca11f11a7bfda64043c37b30e6b91f10cb5bab275806c32f6ab3"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:465b9e8844e3c3519a983d58b80be3f668e2a7a5db97f2784e7079fbc9f9822c"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5418b53c0d59b3824d05e029669efa023bbef0f3e92e75ec8428f3799487f361"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:504b6f59505f08ae014f724b6207ff6222662aab5cc9542577fb084ed0676ac7"},
{file = "pillow-11.3.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:c84d689db21a1c397d001aa08241044aa2069e7587b398c8cc63020390b1c1b8"},
{file = "pillow-11.3.0.tar.gz", hash = "sha256:3828ee7586cd0b2091b6209e5ad53e20d0649bbe87164a459d0676e035e8f523"},
]
[package.extras]
docs = ["furo", "olefile", "sphinx (>=8.2)", "sphinx-autobuild", "sphinx-copybutton", "sphinx-inline-tabs", "sphinxext-opengraph"]
fpx = ["olefile"]
mic = ["olefile"]
test-arrow = ["pyarrow"]
tests = ["check-manifest", "coverage (>=7.4.2)", "defusedxml", "markdown2", "olefile", "packaging", "pyroma", "pytest", "pytest-cov", "pytest-timeout", "pytest-xdist", "trove-classifiers (>=2024.10.12)"]
typing = ["typing-extensions ; python_version < \"3.10\""]
xmp = ["defusedxml"]
[[package]]
name = "platformdirs"
version = "4.3.8"
@@ -5520,29 +5016,6 @@ attrs = ">=22.2.0"
rpds-py = ">=0.7.0"
typing-extensions = {version = ">=4.4.0", markers = "python_version < \"3.13\""}
[[package]]
name = "reportlab"
version = "4.4.4"
description = "The Reportlab Toolkit"
optional = false
python-versions = "<4,>=3.9"
groups = ["main"]
files = [
{file = "reportlab-4.4.4-py3-none-any.whl", hash = "sha256:299b3b0534e7202bb94ed2ddcd7179b818dcda7de9d8518a57c85a58a1ebaadb"},
{file = "reportlab-4.4.4.tar.gz", hash = "sha256:cb2f658b7f4a15be2cc68f7203aa67faef67213edd4f2d4bdd3eb20dab75a80d"},
]
[package.dependencies]
charset-normalizer = "*"
pillow = ">=9.0.0"
[package.extras]
accel = ["rl_accel (>=0.9.0,<1.1)"]
bidi = ["rlbidi"]
pycairo = ["freetype-py (>=2.3.0,<2.4)", "rlPyCairo (>=0.2.0,<1)"]
renderpm = ["rl_renderPM (>=4.0.3,<4.1)"]
shaping = ["uharfbuzz"]
[[package]]
name = "requests"
version = "2.32.5"
@@ -6786,4 +6259,4 @@ type = ["pytest-mypy"]
[metadata]
lock-version = "2.1"
python-versions = ">=3.11,<3.13"
content-hash = "3c9164d668d37d6373eb5200bbe768232ead934d9312b9c68046b1df922789f3"
content-hash = "03442fd4673006c5a74374f90f53621fd1c9d117279fe6cc0355ef833eb7f9bb"
+1 -3
View File
@@ -33,9 +33,7 @@ dependencies = [
"xmlsec==1.3.14",
"h2 (==4.3.0)",
"markdown (>=3.9,<4.0)",
"drf-simple-apikey (==2.2.1)",
"matplotlib (>=3.10.6,<4.0.0)",
"reportlab (>=4.4.4,<5.0.0)"
"drf-simple-apikey (==2.2.1)"
]
description = "Prowler's API (Django/DRF)"
license = "Apache-2.0"
-45
View File
@@ -27,8 +27,6 @@ from api.models import (
Finding,
Integration,
Invitation,
LighthouseProviderConfiguration,
LighthouseProviderModels,
Membership,
OverviewStatusChoices,
PermissionChoices,
@@ -767,7 +765,6 @@ class ComplianceOverviewFilter(FilterSet):
class ScanSummaryFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="scan__provider__provider", choices=Provider.ProviderChoices.choices
)
@@ -930,45 +927,3 @@ class TenantApiKeyFilter(FilterSet):
"revoked": ["exact"],
"name": ["exact", "icontains"],
}
class LighthouseProviderConfigFilter(FilterSet):
provider_type = ChoiceFilter(
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
field_name="provider_type",
lookup_expr="in",
)
is_active = BooleanFilter()
class Meta:
model = LighthouseProviderConfiguration
fields = {
"provider_type": ["exact", "in"],
"is_active": ["exact"],
}
class LighthouseProviderModelsFilter(FilterSet):
provider_type = ChoiceFilter(
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
field_name="provider_configuration__provider_type",
)
provider_type__in = ChoiceInFilter(
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
field_name="provider_configuration__provider_type",
lookup_expr="in",
)
# Allow filtering by model id
model_id = CharFilter(field_name="model_id", lookup_expr="exact")
model_id__icontains = CharFilter(field_name="model_id", lookup_expr="icontains")
model_id__in = CharInFilter(field_name="model_id", lookup_expr="in")
class Meta:
model = LighthouseProviderModels
fields = {
"model_id": ["exact", "icontains", "in"],
}
@@ -24,7 +24,7 @@ class Migration(migrations.Migration):
(
"name",
models.CharField(
max_length=100,
max_length=255,
validators=[django.core.validators.MinLengthValidator(3)],
),
),
@@ -44,7 +44,7 @@ class Migration(migrations.Migration):
help_text="If the API key is revoked, entities cannot use it anymore. (This cannot be undone.)",
),
),
("created", models.DateTimeField(auto_now_add=True, editable=False)),
("created", models.DateTimeField(auto_now=True)),
(
"whitelisted_ips",
models.JSONField(
@@ -1,266 +0,0 @@
# Generated by Django 5.1.12 on 2025-10-09 07:50
import json
import logging
import uuid
import django.db.models.deletion
from config.custom_logging import BackendLogger
from cryptography.fernet import Fernet
from django.conf import settings
from django.db import migrations, models
import api.rls
from api.db_router import MainRouter
logger = logging.getLogger(BackendLogger.API)
def migrate_lighthouse_configs_forward(apps, schema_editor):
"""
Migrate data from old LighthouseConfiguration to new multi-provider models.
Old system: one LighthouseConfiguration per tenant (always OpenAI).
"""
LighthouseConfiguration = apps.get_model("api", "LighthouseConfiguration")
LighthouseProviderConfiguration = apps.get_model(
"api", "LighthouseProviderConfiguration"
)
LighthouseTenantConfiguration = apps.get_model(
"api", "LighthouseTenantConfiguration"
)
LighthouseProviderModels = apps.get_model("api", "LighthouseProviderModels")
fernet = Fernet(settings.SECRETS_ENCRYPTION_KEY.encode())
# Migrate only tenants that actually have a LighthouseConfiguration
for old_config in (
LighthouseConfiguration.objects.using(MainRouter.admin_db)
.select_related("tenant")
.all()
):
tenant = old_config.tenant
tenant_id = str(tenant.id)
try:
# Create OpenAI provider configuration for this tenant
api_key_decrypted = fernet.decrypt(bytes(old_config.api_key)).decode()
credentials_encrypted = fernet.encrypt(
json.dumps({"api_key": api_key_decrypted}).encode()
)
provider_config = LighthouseProviderConfiguration.objects.using(
MainRouter.admin_db
).create(
tenant=tenant,
provider_type="openai",
credentials=credentials_encrypted,
is_active=old_config.is_active,
)
# Create tenant configuration from old values
LighthouseTenantConfiguration.objects.using(MainRouter.admin_db).create(
tenant=tenant,
business_context=old_config.business_context or "",
default_provider="openai",
default_models={"openai": old_config.model},
)
# Create initial provider model record
LighthouseProviderModels.objects.using(MainRouter.admin_db).create(
tenant=tenant,
provider_configuration=provider_config,
model_id=old_config.model,
model_name=old_config.model,
default_parameters={},
)
except Exception:
logger.exception(
"Failed to migrate lighthouse config for tenant %s", tenant_id
)
continue
class Migration(migrations.Migration):
dependencies = [
("api", "0049_compliancerequirementoverview_passed_failed_findings"),
]
operations = [
migrations.CreateModel(
name="LighthouseProviderConfiguration",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
(
"provider_type",
models.CharField(
choices=[("openai", "OpenAI")],
help_text="LLM provider name",
max_length=50,
),
),
("base_url", models.URLField(blank=True, null=True)),
(
"credentials",
models.BinaryField(
help_text="Encrypted JSON credentials for the provider"
),
),
("is_active", models.BooleanField(default=True)),
],
options={
"db_table": "lighthouse_provider_configurations",
"abstract": False,
},
),
migrations.CreateModel(
name="LighthouseProviderModels",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
("model_id", models.CharField(max_length=100)),
("model_name", models.CharField(max_length=100)),
("default_parameters", models.JSONField(blank=True, default=dict)),
],
options={
"db_table": "lighthouse_provider_models",
"abstract": False,
},
),
migrations.CreateModel(
name="LighthouseTenantConfiguration",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
("business_context", models.TextField(blank=True, default="")),
("default_provider", models.CharField(blank=True, max_length=50)),
("default_models", models.JSONField(blank=True, default=dict)),
],
options={
"db_table": "lighthouse_tenant_config",
"abstract": False,
},
),
migrations.AddField(
model_name="lighthouseproviderconfiguration",
name="tenant",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
migrations.AddField(
model_name="lighthouseprovidermodels",
name="provider_configuration",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="available_models",
to="api.lighthouseproviderconfiguration",
),
),
migrations.AddField(
model_name="lighthouseprovidermodels",
name="tenant",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
migrations.AddField(
model_name="lighthousetenantconfiguration",
name="tenant",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
migrations.AddIndex(
model_name="lighthouseproviderconfiguration",
index=models.Index(
fields=["tenant_id", "provider_type"], name="lh_pc_tenant_type_idx"
),
),
migrations.AddConstraint(
model_name="lighthouseproviderconfiguration",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_lighthouseproviderconfiguration",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.AddConstraint(
model_name="lighthouseproviderconfiguration",
constraint=models.UniqueConstraint(
fields=("tenant_id", "provider_type"),
name="unique_provider_config_per_tenant",
),
),
migrations.AddIndex(
model_name="lighthouseprovidermodels",
index=models.Index(
fields=["tenant_id", "provider_configuration"],
name="lh_prov_models_cfg_idx",
),
),
migrations.AddConstraint(
model_name="lighthouseprovidermodels",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_lighthouseprovidermodels",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.AddConstraint(
model_name="lighthouseprovidermodels",
constraint=models.UniqueConstraint(
fields=("tenant_id", "provider_configuration", "model_id"),
name="unique_provider_model_per_configuration",
),
),
migrations.AddConstraint(
model_name="lighthousetenantconfiguration",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_lighthousetenantconfiguration",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.AddConstraint(
model_name="lighthousetenantconfiguration",
constraint=models.UniqueConstraint(
fields=("tenant_id",), name="unique_tenant_lighthouse_config"
),
),
# Migrate data from old LighthouseConfiguration to new tables
# This runs after all tables, indexes, and constraints are created
# The old Lighthouse configuration table is not removed, so reverse_code is noop
# During rollbacks, the old Lighthouse configuration remains intact while the new tables are removed
migrations.RunPython(
migrate_lighthouse_configs_forward,
reverse_code=migrations.RunPython.noop,
),
]
@@ -1,21 +0,0 @@
# Generated by Django 5.1.12 on 2025-10-14 11:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0050_lighthouse_multi_llm"),
]
operations = [
migrations.AlterField(
model_name="lighthouseproviderconfiguration",
name="provider_type",
field=models.CharField(
choices=[("openai", "OpenAI"), ("bedrock", "AWS Bedrock")],
help_text="LLM provider name",
max_length=50,
),
)
]
+25 -183
View File
@@ -221,7 +221,6 @@ class Membership(models.Model):
class TenantAPIKey(AbstractAPIKey, RowLevelSecurityProtectedModel):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
name = models.CharField(max_length=100, validators=[MinLengthValidator(3)])
created = models.DateTimeField(auto_now_add=True, editable=False)
prefix = models.CharField(
max_length=11,
unique=True,
@@ -1873,6 +1872,22 @@ class LighthouseConfiguration(RowLevelSecurityProtectedModel):
def clean(self):
super().clean()
# Validate temperature
if not 0 <= self.temperature <= 1:
raise ModelValidationError(
detail="Temperature must be between 0 and 1",
code="invalid_temperature",
pointer="/data/attributes/temperature",
)
# Validate max_tokens
if not 500 <= self.max_tokens <= 5000:
raise ModelValidationError(
detail="Max tokens must be between 500 and 5000",
code="invalid_max_tokens",
pointer="/data/attributes/max_tokens",
)
@property
def api_key_decoded(self):
"""Return the decrypted API key, or None if unavailable or invalid."""
@@ -1897,6 +1912,15 @@ class LighthouseConfiguration(RowLevelSecurityProtectedModel):
code="invalid_api_key",
pointer="/data/attributes/api_key",
)
# Validate OpenAI API key format
openai_key_pattern = r"^sk-[\w-]+T3BlbkFJ[\w-]+$"
if not re.match(openai_key_pattern, value):
raise ModelValidationError(
detail="Invalid OpenAI API key format.",
code="invalid_api_key",
pointer="/data/attributes/api_key",
)
self.api_key = fernet.encrypt(value.encode())
def save(self, *args, **kwargs):
@@ -1959,185 +1983,3 @@ class Processor(RowLevelSecurityProtectedModel):
class JSONAPIMeta:
resource_name = "processors"
class LighthouseProviderConfiguration(RowLevelSecurityProtectedModel):
"""
Per-tenant configuration for an LLM provider (credentials, base URL, activation).
One configuration per provider type per tenant.
"""
class LLMProviderChoices(models.TextChoices):
OPENAI = "openai", _("OpenAI")
BEDROCK = "bedrock", _("AWS Bedrock")
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
provider_type = models.CharField(
max_length=50,
choices=LLMProviderChoices.choices,
help_text="LLM provider name",
)
# For OpenAI-compatible providers
base_url = models.URLField(blank=True, null=True)
# Encrypted JSON for provider-specific auth
credentials = models.BinaryField(
blank=False, null=False, help_text="Encrypted JSON credentials for the provider"
)
is_active = models.BooleanField(default=True)
def __str__(self):
return f"{self.get_provider_type_display()} ({self.tenant_id})"
def clean(self):
super().clean()
@property
def credentials_decoded(self):
if not self.credentials:
return None
try:
decrypted_data = fernet.decrypt(bytes(self.credentials))
return json.loads(decrypted_data.decode())
except (InvalidToken, json.JSONDecodeError) as e:
logger.warning("Failed to decrypt provider credentials: %s", e)
return None
except Exception as e:
logger.exception(
"Unexpected error while decrypting provider credentials: %s", e
)
return None
@credentials_decoded.setter
def credentials_decoded(self, value):
"""
Set and encrypt credentials (assumes serializer performed validation).
"""
if not value:
raise ModelValidationError(
detail="Credentials are required",
code="invalid_credentials",
pointer="/data/attributes/credentials",
)
self.credentials = fernet.encrypt(json.dumps(value).encode())
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "lighthouse_provider_configurations"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
models.UniqueConstraint(
fields=["tenant_id", "provider_type"],
name="unique_provider_config_per_tenant",
),
]
indexes = [
models.Index(
fields=["tenant_id", "provider_type"],
name="lh_pc_tenant_type_idx",
),
]
class JSONAPIMeta:
resource_name = "lighthouse-providers"
class LighthouseTenantConfiguration(RowLevelSecurityProtectedModel):
"""
Tenant-level Lighthouse settings (business context and defaults).
One record per tenant.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
business_context = models.TextField(blank=True, default="")
# Preferred provider key (e.g., "openai", "bedrock", "openai_compatible")
default_provider = models.CharField(max_length=50, blank=True)
# Mapping of provider -> model id, e.g., {"openai": "gpt-4o", "bedrock": "anthropic.claude-v2"}
default_models = models.JSONField(default=dict, blank=True)
def __str__(self):
return f"Lighthouse Tenant Config for {self.tenant_id}"
def clean(self):
super().clean()
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "lighthouse_tenant_config"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
models.UniqueConstraint(
fields=["tenant_id"], name="unique_tenant_lighthouse_config"
),
]
class JSONAPIMeta:
resource_name = "lighthouse-config"
class LighthouseProviderModels(RowLevelSecurityProtectedModel):
"""
Per-tenant, per-provider configuration list of available LLM models.
RLS-protected; populated via provider API using tenant-scoped credentials.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
# Scope to a specific provider configuration within a tenant
provider_configuration = models.ForeignKey(
LighthouseProviderConfiguration,
on_delete=models.CASCADE,
related_name="available_models",
)
model_id = models.CharField(max_length=100)
# Human-friendly model name
model_name = models.CharField(max_length=100)
# Model-specific default parameters (e.g., temperature, max_tokens)
default_parameters = models.JSONField(default=dict, blank=True)
def __str__(self):
return f"{self.provider_configuration.provider_type}:{self.model_id} ({self.tenant_id})"
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "lighthouse_provider_models"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
models.UniqueConstraint(
fields=["tenant_id", "provider_configuration", "model_id"],
name="unique_provider_model_per_configuration",
),
]
indexes = [
models.Index(
fields=["tenant_id", "provider_configuration"],
name="lh_prov_models_cfg_idx",
),
]
+1 -38
View File
@@ -6,14 +6,7 @@ from django.dispatch import receiver
from django_celery_results.backends.database import DatabaseBackend
from api.db_utils import delete_related_daily_task
from api.models import (
LighthouseProviderConfiguration,
LighthouseTenantConfiguration,
Membership,
Provider,
TenantAPIKey,
User,
)
from api.models import Membership, Provider, TenantAPIKey, User
def create_task_result_on_publish(sender=None, headers=None, **kwargs): # noqa: F841
@@ -63,33 +56,3 @@ def revoke_membership_api_keys(sender, instance, **kwargs): # noqa: F841
TenantAPIKey.objects.filter(
entity=instance.user, tenant_id=instance.tenant.id
).update(revoked=True)
@receiver(pre_delete, sender=LighthouseProviderConfiguration)
def cleanup_lighthouse_defaults_before_delete(sender, instance, **kwargs): # noqa: F841
"""
Ensure tenant Lighthouse defaults do not reference a soon-to-be-deleted provider.
This runs for both per-instance deletes and queryset (bulk) deletes.
"""
try:
tenant_cfg = LighthouseTenantConfiguration.objects.get(
tenant_id=instance.tenant_id
)
except LighthouseTenantConfiguration.DoesNotExist:
return
updated = False
defaults = tenant_cfg.default_models or {}
if instance.provider_type in defaults:
defaults.pop(instance.provider_type, None)
tenant_cfg.default_models = defaults
updated = True
if tenant_cfg.default_provider == instance.provider_type:
tenant_cfg.default_provider = ""
updated = True
if updated:
tenant_cfg.save()
File diff suppressed because it is too large Load Diff
-914
View File
@@ -35,9 +35,6 @@ from api.db_router import MainRouter
from api.models import (
Integration,
Invitation,
LighthouseProviderConfiguration,
LighthouseProviderModels,
LighthouseTenantConfiguration,
Membership,
Processor,
Provider,
@@ -49,7 +46,6 @@ from api.models import (
SAMLConfiguration,
SAMLToken,
Scan,
ScanSummary,
StateChoices,
Task,
TenantAPIKey,
@@ -2692,55 +2688,6 @@ class TestScanViewSet:
== "There is a problem with credentials."
)
@patch("api.v1.views.ScanViewSet._get_task_status")
@patch("api.v1.views.get_s3_client")
@patch("api.v1.views.env.str")
def test_threatscore_s3_wildcard(
self,
mock_env_str,
mock_get_s3_client,
mock_get_task_status,
authenticated_client,
scans_fixture,
):
"""
When the threatscore endpoint is called with an S3 output_location,
the view should list objects in S3 using wildcard pattern matching,
retrieve the matching PDF file, and return it with HTTP 200 and proper headers.
"""
scan = scans_fixture[0]
scan.state = StateChoices.COMPLETED
bucket = "test-bucket"
zip_key = "tenant-id/scan-id/prowler-output-foo.zip"
scan.output_location = f"s3://{bucket}/{zip_key}"
scan.save()
pdf_key = os.path.join(
os.path.dirname(zip_key),
"threatscore",
"prowler-output-123_threatscore_report.pdf",
)
mock_s3_client = Mock()
mock_s3_client.list_objects_v2.return_value = {"Contents": [{"Key": pdf_key}]}
mock_s3_client.get_object.return_value = {"Body": io.BytesIO(b"pdf-bytes")}
mock_env_str.return_value = bucket
mock_get_s3_client.return_value = mock_s3_client
mock_get_task_status.return_value = None
url = reverse("scan-threatscore", kwargs={"pk": scan.id})
response = authenticated_client.get(url)
assert response.status_code == status.HTTP_200_OK
assert response["Content-Type"] == "application/pdf"
assert response["Content-Disposition"].endswith(
'"prowler-output-123_threatscore_report.pdf"'
)
assert response.content == b"pdf-bytes"
mock_s3_client.list_objects_v2.assert_called_once()
mock_s3_client.get_object.assert_called_once_with(Bucket=bucket, Key=pdf_key)
def test_report_s3_success(self, authenticated_client, scans_fixture, monkeypatch):
"""
When output_location is an S3 URL and the S3 client returns the file successfully,
@@ -5819,171 +5766,6 @@ class TestOverviewViewSet:
assert service1_data["attributes"]["muted"] == 1
assert service2_data["attributes"]["muted"] == 0
def test_overview_findings_provider_id_in_filter(
self, authenticated_client, tenants_fixture, providers_fixture
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
scan1 = Scan.objects.create(
name="scan-one",
provider=provider1,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
scan2 = Scan.objects.create(
name="scan-two",
provider=provider2,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="check-provider-one",
service="service-a",
severity="high",
region="region-a",
_pass=5,
fail=1,
muted=2,
total=8,
new=5,
changed=2,
unchanged=1,
fail_new=1,
fail_changed=0,
pass_new=3,
pass_changed=2,
muted_new=1,
muted_changed=1,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan2,
check_id="check-provider-two",
service="service-b",
severity="medium",
region="region-b",
_pass=2,
fail=3,
muted=1,
total=6,
new=3,
changed=2,
unchanged=1,
fail_new=2,
fail_changed=1,
pass_new=1,
pass_changed=1,
muted_new=1,
muted_changed=0,
)
single_response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_id__in]": str(provider1.id)},
)
assert single_response.status_code == status.HTTP_200_OK
single_attributes = single_response.json()["data"]["attributes"]
assert single_attributes["pass"] == 5
assert single_attributes["fail"] == 1
assert single_attributes["muted"] == 2
assert single_attributes["total"] == 8
combined_response = authenticated_client.get(
reverse("overview-findings"),
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
)
assert combined_response.status_code == status.HTTP_200_OK
combined_attributes = combined_response.json()["data"]["attributes"]
assert combined_attributes["pass"] == 7
assert combined_attributes["fail"] == 4
assert combined_attributes["muted"] == 3
assert combined_attributes["total"] == 14
def test_overview_findings_severity_provider_id_in_filter(
self, authenticated_client, tenants_fixture, providers_fixture
):
tenant = tenants_fixture[0]
provider1, provider2, *_ = providers_fixture
scan1 = Scan.objects.create(
name="severity-scan-one",
provider=provider1,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
scan2 = Scan.objects.create(
name="severity-scan-two",
provider=provider2,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant=tenant,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="severity-check-one",
service="service-a",
severity="high",
region="region-a",
_pass=4,
fail=4,
muted=0,
total=8,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan1,
check_id="severity-check-two",
service="service-a",
severity="medium",
region="region-b",
_pass=2,
fail=2,
muted=0,
total=4,
)
ScanSummary.objects.create(
tenant=tenant,
scan=scan2,
check_id="severity-check-three",
service="service-b",
severity="critical",
region="region-c",
_pass=1,
fail=2,
muted=0,
total=3,
)
single_response = authenticated_client.get(
reverse("overview-findings_severity"),
{"filter[provider_id__in]": str(provider1.id)},
)
assert single_response.status_code == status.HTTP_200_OK
single_attributes = single_response.json()["data"]["attributes"]
assert single_attributes["high"] == 8
assert single_attributes["medium"] == 4
assert single_attributes["critical"] == 0
combined_response = authenticated_client.get(
reverse("overview-findings_severity"),
{"filter[provider_id__in]": f"{provider1.id},{provider2.id}"},
)
assert combined_response.status_code == status.HTTP_200_OK
combined_attributes = combined_response.json()["data"]["attributes"]
assert combined_attributes["high"] == 8
assert combined_attributes["medium"] == 4
assert combined_attributes["critical"] == 3
@pytest.mark.django_db
class TestScheduleViewSet:
@@ -8166,27 +7948,6 @@ class TestTenantApiKeyViewSet:
api_key.refresh_from_db()
assert api_key.revoked is True
def test_api_keys_revoke_preserves_created_field(
self, authenticated_client, api_keys_fixture
):
"""Test that revoking an API key preserves the created timestamp."""
api_key = api_keys_fixture[0] # Not revoked
assert api_key.revoked is False
# Record the original created timestamp
original_created = api_key.created
response = authenticated_client.delete(
reverse("api-key-revoke", kwargs={"pk": api_key.id})
)
assert response.status_code == status.HTTP_200_OK
# Verify in database
api_key.refresh_from_db()
assert api_key.revoked is True
# Verify created field has not changed
assert api_key.created == original_created
def test_api_keys_revoke_already_revoked(
self, authenticated_client, api_keys_fixture
):
@@ -8706,678 +8467,3 @@ class TestTenantApiKeyViewSet:
# Verify error object structure
error = response_data["errors"][0]
assert "detail" in error or "title" in error
@pytest.mark.django_db
class TestLighthouseTenantConfigViewSet:
"""Test Lighthouse tenant configuration endpoint (singleton pattern)"""
def test_lighthouse_tenant_config_create_via_patch(self, authenticated_client):
"""Test creating a tenant config successfully via PATCH (upsert)"""
payload = {
"data": {
"type": "lighthouse-config",
"attributes": {
"business_context": "Test business context for security analysis",
"default_provider": "",
"default_models": {},
},
}
}
response = authenticated_client.patch(
reverse("lighthouse-config"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert (
data["attributes"]["business_context"]
== "Test business context for security analysis"
)
assert data["attributes"]["default_provider"] == ""
assert data["attributes"]["default_models"] == {}
def test_lighthouse_tenant_config_upsert_behavior(self, authenticated_client):
"""Test that PATCH creates config if not exists and updates if exists (upsert)"""
payload = {
"data": {
"type": "lighthouse-config",
"attributes": {
"business_context": "First config",
},
}
}
# First PATCH creates the config
response = authenticated_client.patch(
reverse("lighthouse-config"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_200_OK
first_data = response.json()["data"]
assert first_data["attributes"]["business_context"] == "First config"
# Second PATCH updates the same config (not creating a duplicate)
payload["data"]["attributes"]["business_context"] = "Updated config"
response = authenticated_client.patch(
reverse("lighthouse-config"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_200_OK
second_data = response.json()["data"]
assert second_data["attributes"]["business_context"] == "Updated config"
# Verify it's the same config (same ID)
assert first_data["id"] == second_data["id"]
@patch("openai.OpenAI")
def test_lighthouse_tenant_config_retrieve(
self, mock_openai_client, authenticated_client, tenants_fixture
):
"""Test retrieving the singleton tenant config with proper provider and model validation"""
# Mock OpenAI client and models response
mock_models_response = Mock()
mock_models_response.data = [
Mock(id="gpt-4o"),
Mock(id="gpt-4o-mini"),
Mock(id="gpt-5"),
]
mock_openai_client.return_value.models.list.return_value = mock_models_response
# Create OpenAI provider configuration
provider_config = LighthouseProviderConfiguration.objects.create(
tenant_id=tenants_fixture[0].id,
provider_type="openai",
credentials=b'{"api_key": "sk-test1234567890T3BlbkFJtest1234567890"}',
is_active=True,
)
# Create provider models (simulating refresh)
LighthouseProviderModels.objects.create(
tenant_id=tenants_fixture[0].id,
provider_configuration=provider_config,
model_id="gpt-4o",
default_parameters={},
)
LighthouseProviderModels.objects.create(
tenant_id=tenants_fixture[0].id,
provider_configuration=provider_config,
model_id="gpt-4o-mini",
default_parameters={},
)
# Create tenant configuration with valid provider and model
config = LighthouseTenantConfiguration.objects.create(
tenant_id=tenants_fixture[0].id,
business_context="Test context",
default_provider="openai",
default_models={"openai": "gpt-4o"},
)
# Retrieve and verify the configuration
response = authenticated_client.get(reverse("lighthouse-config"))
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert data["id"] == str(config.id)
assert data["attributes"]["business_context"] == "Test context"
assert data["attributes"]["default_provider"] == "openai"
assert data["attributes"]["default_models"] == {"openai": "gpt-4o"}
def test_lighthouse_tenant_config_retrieve_not_found(self, authenticated_client):
"""Test GET when config doesn't exist returns 404"""
response = authenticated_client.get(reverse("lighthouse-config"))
assert response.status_code == status.HTTP_404_NOT_FOUND
assert "not found" in response.json()["errors"][0]["detail"].lower()
def test_lighthouse_tenant_config_partial_update(
self, authenticated_client, tenants_fixture
):
"""Test updating tenant config fields"""
from api.models import LighthouseTenantConfiguration
# Create config first
config = LighthouseTenantConfiguration.objects.create(
tenant_id=tenants_fixture[0].id,
business_context="Original context",
default_provider="",
default_models={},
)
# Update it
payload = {
"data": {
"type": "lighthouse-config",
"attributes": {
"business_context": "Updated context for cloud security",
},
}
}
response = authenticated_client.patch(
reverse("lighthouse-config"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_200_OK
# Verify update
config.refresh_from_db()
assert config.business_context == "Updated context for cloud security"
def test_lighthouse_tenant_config_update_invalid_provider(
self, authenticated_client, tenants_fixture
):
"""Test validation fails when default_provider is not configured and active"""
from api.models import LighthouseTenantConfiguration
# Create config first
LighthouseTenantConfiguration.objects.create(
tenant_id=tenants_fixture[0].id,
business_context="Test",
)
# Try to set invalid provider
payload = {
"data": {
"type": "lighthouse-config",
"attributes": {
"default_provider": "nonexistent-provider",
},
}
}
response = authenticated_client.patch(
reverse("lighthouse-config"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert "provider" in response.json()["errors"][0]["detail"].lower()
def test_lighthouse_tenant_config_update_invalid_json_format(
self, authenticated_client, tenants_fixture
):
"""Test that invalid JSON payload is rejected"""
from api.models import LighthouseTenantConfiguration
# Create config first
LighthouseTenantConfiguration.objects.create(
tenant_id=tenants_fixture[0].id,
business_context="Test",
)
# Send invalid JSON
response = authenticated_client.patch(
reverse("lighthouse-config"),
data="invalid json",
content_type=API_JSON_CONTENT_TYPE,
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
@pytest.mark.django_db
class TestLighthouseProviderConfigViewSet:
"""Tests for LighthouseProviderConfiguration create validations"""
def test_invalid_provider_type(self, authenticated_client):
"""Add invalid provider (testprovider) should error"""
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "testprovider",
"credentials": {"api_key": "sk-testT3BlbkFJkey"},
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_400_BAD_REQUEST
def test_openai_missing_credentials(self, authenticated_client):
"""OpenAI provider without credentials should error"""
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_400_BAD_REQUEST
@pytest.mark.parametrize(
"credentials",
[
{}, # empty credentials
{"token": "sk-testT3BlbkFJkey"}, # wrong key name
{"api_key": "ks-invalid-format"}, # wrong format
],
)
def test_openai_invalid_credentials(self, authenticated_client, credentials):
"""OpenAI provider with invalid credentials should error"""
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": credentials,
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_400_BAD_REQUEST
def test_openai_valid_credentials_success(self, authenticated_client):
"""OpenAI provider with valid sk-xxx format should succeed"""
valid_key = "sk-abc123T3BlbkFJxyz456"
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": {"api_key": valid_key},
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_201_CREATED
data = resp.json()["data"]
masked_creds = data["attributes"].get("credentials")
assert masked_creds is not None
assert "api_key" in masked_creds
assert masked_creds["api_key"] == ("*" * len(valid_key))
def test_openai_provider_duplicate_per_tenant(self, authenticated_client):
"""If an OpenAI provider exists for tenant, creating again should error"""
valid_key = "sk-dup123T3BlbkFJdup456"
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": {"api_key": valid_key},
},
}
}
# First creation succeeds
resp1 = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp1.status_code == status.HTTP_201_CREATED
# Second creation should fail with validation error
resp2 = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp2.status_code == status.HTTP_400_BAD_REQUEST
assert "already exists" in str(resp2.json()).lower()
def test_openai_patch_base_url_and_is_active(self, authenticated_client):
"""After creating, should be able to patch base_url and is_active"""
valid_key = "sk-patch123T3BlbkFJpatch456"
create_payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": {"api_key": valid_key},
},
}
}
create_resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=create_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert create_resp.status_code == status.HTTP_201_CREATED
provider_id = create_resp.json()["data"]["id"]
patch_payload = {
"data": {
"type": "lighthouse-providers",
"id": provider_id,
"attributes": {
"base_url": "https://api.example.com/v1",
"is_active": False,
},
}
}
patch_resp = authenticated_client.patch(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
data=patch_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert patch_resp.status_code == status.HTTP_200_OK
updated = patch_resp.json()["data"]["attributes"]
assert updated["base_url"] == "https://api.example.com/v1"
assert updated["is_active"] is False
def test_openai_patch_invalid_credentials(self, authenticated_client):
"""PATCH with invalid credentials.api_key should error (400)"""
valid_key = "sk-ok123T3BlbkFJok456"
create_payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": {"api_key": valid_key},
},
}
}
create_resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=create_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert create_resp.status_code == status.HTTP_201_CREATED
provider_id = create_resp.json()["data"]["id"]
# Try patch with invalid api_key format
patch_payload = {
"data": {
"type": "lighthouse-providers",
"id": provider_id,
"attributes": {
"credentials": {"api_key": "ks-invalid-format"},
},
}
}
patch_resp = authenticated_client.patch(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
data=patch_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert patch_resp.status_code == status.HTTP_400_BAD_REQUEST
def test_openai_get_masking_and_fields_filter(self, authenticated_client):
valid_key = "sk-get123T3BlbkFJget456"
create_payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "openai",
"credentials": {"api_key": valid_key},
},
}
}
create_resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=create_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert create_resp.status_code == status.HTTP_201_CREATED
provider_id = create_resp.json()["data"]["id"]
# Default GET should return masked credentials
get_resp = authenticated_client.get(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
)
assert get_resp.status_code == status.HTTP_200_OK
masked = get_resp.json()["data"]["attributes"]["credentials"]["api_key"]
assert masked == ("*" * len(valid_key))
# Fields filter should return decrypted credentials structure
get_full = authenticated_client.get(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
+ "?fields[lighthouse-providers]=credentials"
)
assert get_full.status_code == status.HTTP_200_OK
creds = get_full.json()["data"]["attributes"]["credentials"]
assert creds["api_key"] == valid_key
def test_delete_provider_updates_tenant_defaults(
self, authenticated_client, tenants_fixture
):
"""Deleting a provider config should clear tenant default_provider and its default_model entry."""
tenant = tenants_fixture[0]
# Create provider configuration to delete
provider = LighthouseProviderConfiguration.objects.create(
tenant_id=tenant.id,
provider_type="openai",
credentials=b'{"api_key":"sk-test123T3BlbkFJ"}',
is_active=True,
)
# Seed tenant defaults referencing the provider we will delete
cfg = LighthouseTenantConfiguration.objects.create(
tenant_id=tenant.id,
business_context="Test",
default_provider="openai",
default_models={"openai": "gpt-4o", "other": "model-x"},
)
# Delete via API and validate response
url = reverse("lighthouse-providers-detail", kwargs={"pk": str(provider.id)})
resp = authenticated_client.delete(url)
assert resp.status_code in (
status.HTTP_204_NO_CONTENT,
status.HTTP_200_OK,
)
# Tenant defaults should be updated
cfg.refresh_from_db()
assert cfg.default_provider == ""
assert "openai" not in cfg.default_models
# Unrelated entries should remain untouched
assert cfg.default_models.get("other") == "model-x"
@pytest.mark.parametrize(
"credentials",
[
{}, # empty credentials
{
"access_key_id": "AKIAIOSFODNN7EXAMPLE"
}, # missing secret_access_key and region
{
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}, # missing access_key_id and region
{
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
}, # missing region
{ # invalid access_key_id format (not starting with AKIA)
"access_key_id": "ABCD0123456789ABCDEF",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-east-1",
},
{ # invalid access_key_id format (wrong length)
"access_key_id": "AKIAIOSFODNN7EXAMPL",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-east-1",
},
{ # invalid secret_access_key format (wrong length)
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEK",
"region": "us-east-1",
},
{ # invalid region format
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "invalid-region",
},
{ # invalid region format (uppercase)
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "US-EAST-1",
},
],
)
def test_bedrock_invalid_credentials(self, authenticated_client, credentials):
"""Bedrock provider with invalid credentials should error"""
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "bedrock",
"credentials": credentials,
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_400_BAD_REQUEST
def test_bedrock_valid_credentials_success(self, authenticated_client):
"""Bedrock provider with valid AWS credentials should succeed and mask credentials"""
valid_credentials = {
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-east-1",
}
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "bedrock",
"credentials": valid_credentials,
},
}
}
resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp.status_code == status.HTTP_201_CREATED
data = resp.json()["data"]
# Verify credentials are returned masked
masked_creds = data["attributes"].get("credentials")
assert masked_creds is not None
assert "access_key_id" in masked_creds
assert "secret_access_key" in masked_creds
assert "region" in masked_creds
# Verify all characters are masked with asterisks
assert all(c == "*" for c in masked_creds["access_key_id"])
assert all(c == "*" for c in masked_creds["secret_access_key"])
def test_bedrock_provider_duplicate_per_tenant(self, authenticated_client):
"""Creating a second Bedrock provider for same tenant should fail"""
valid_credentials = {
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "us-west-2",
}
payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "bedrock",
"credentials": valid_credentials,
},
}
}
# First creation succeeds
resp1 = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp1.status_code == status.HTTP_201_CREATED
# Second creation should fail with validation error
resp2 = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert resp2.status_code == status.HTTP_400_BAD_REQUEST
assert "already exists" in str(resp2.json()).lower()
def test_bedrock_patch_credentials_and_fields_filter(self, authenticated_client):
"""PATCH credentials and verify fields filter returns decrypted values"""
valid_credentials = {
"access_key_id": "AKIAIOSFODNN7EXAMPLE",
"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"region": "eu-west-1",
}
create_payload = {
"data": {
"type": "lighthouse-providers",
"attributes": {
"provider_type": "bedrock",
"credentials": valid_credentials,
},
}
}
create_resp = authenticated_client.post(
reverse("lighthouse-providers-list"),
data=create_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert create_resp.status_code == status.HTTP_201_CREATED
provider_id = create_resp.json()["data"]["id"]
# Update credentials with new valid ones
new_credentials = {
"access_key_id": "AKIAZZZZZZZZZZZZZZZZ",
"secret_access_key": "aBcDeFgHiJkLmNoPqRsTuVwXyZ0123456789+/==",
"region": "ap-south-1",
}
patch_payload = {
"data": {
"type": "lighthouse-providers",
"id": provider_id,
"attributes": {
"credentials": new_credentials,
"is_active": False,
},
}
}
patch_resp = authenticated_client.patch(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
data=patch_payload,
content_type=API_JSON_CONTENT_TYPE,
)
assert patch_resp.status_code == status.HTTP_200_OK
updated = patch_resp.json()["data"]["attributes"]
assert updated["is_active"] is False
# Default GET should return masked credentials
get_resp = authenticated_client.get(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
)
assert get_resp.status_code == status.HTTP_200_OK
masked = get_resp.json()["data"]["attributes"]["credentials"]
assert all(c == "*" for c in masked["access_key_id"])
assert all(c == "*" for c in masked["secret_access_key"])
# Fields filter should return decrypted credentials
get_full = authenticated_client.get(
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
+ "?fields[lighthouse-providers]=credentials"
)
assert get_full.status_code == status.HTTP_200_OK
creds = get_full.json()["data"]["attributes"]["credentials"]
assert creds["access_key_id"] == new_credentials["access_key_id"]
assert creds["secret_access_key"] == new_credentials["secret_access_key"]
assert creds["region"] == new_credentials["region"]
@@ -1,52 +0,0 @@
import re
from rest_framework_json_api import serializers
class OpenAICredentialsSerializer(serializers.Serializer):
api_key = serializers.CharField()
def validate_api_key(self, value: str) -> str:
pattern = r"^sk-[\w-]+$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError("Invalid OpenAI API key format.")
return value
class BedrockCredentialsSerializer(serializers.Serializer):
"""
Serializer for AWS Bedrock credentials validation.
Validates long-term AWS credentials (AKIA) and region format.
"""
access_key_id = serializers.CharField()
secret_access_key = serializers.CharField()
region = serializers.CharField()
def validate_access_key_id(self, value: str) -> str:
"""Validate AWS access key ID format (AKIA for long-term credentials)."""
pattern = r"^AKIA[0-9A-Z]{16}$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS access key ID format. Must be AKIA followed by 16 alphanumeric characters."
)
return value
def validate_secret_access_key(self, value: str) -> str:
"""Validate AWS secret access key format (40 base64 characters)."""
pattern = r"^[A-Za-z0-9/+=]{40}$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS secret access key format. Must be 40 base64 characters."
)
return value
def validate_region(self, value: str) -> str:
"""Validate AWS region format."""
pattern = r"^[a-z]{2}-[a-z]+-\d+$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS region format. Expected format like 'us-east-1' or 'eu-west-2'."
)
return value
@@ -105,25 +105,23 @@ from rest_framework_json_api import serializers
"type": "string",
"description": "The Azure application (client) ID for authentication in Azure AD.",
},
"tenant_id": {
"type": "string",
"description": "The Azure tenant ID, representing the directory where the application is "
"registered.",
},
"client_secret": {
"type": "string",
"description": "The client secret associated with the application (client) ID, providing "
"secure access.",
},
"tenant_id": {
"type": "string",
"description": "The Azure tenant ID, representing the directory where the application is "
"registered.",
},
"user": {
"type": "email",
"description": "User microsoft email address.",
"deprecated": True,
"description": "Deprecated: User microsoft email address.",
},
"password": {
"type": "string",
"description": "User password.",
"deprecated": True,
"description": "Deprecated: User password.",
},
},
"required": [
@@ -151,11 +149,21 @@ from rest_framework_json_api import serializers
"type": "string",
"description": "The certificate content in base64 format for certificate-based authentication.",
},
"user": {
"type": "email",
"description": "User microsoft email address.",
},
"password": {
"type": "string",
"description": "User password.",
},
},
"required": [
"client_id",
"tenant_id",
"certificate_content",
"user",
"password",
],
},
{
-419
View File
@@ -6,10 +6,8 @@ from django.conf import settings
from django.contrib.auth import authenticate
from django.contrib.auth.models import update_last_login
from django.contrib.auth.password_validation import validate_password
from django.db import IntegrityError
from drf_spectacular.utils import extend_schema_field
from jwt.exceptions import InvalidKeyError
from rest_framework.reverse import reverse
from rest_framework.validators import UniqueTogetherValidator
from rest_framework_json_api import serializers
from rest_framework_json_api.relations import SerializerMethodResourceRelatedField
@@ -27,9 +25,6 @@ from api.models import (
Invitation,
InvitationRoleRelationship,
LighthouseConfiguration,
LighthouseProviderConfiguration,
LighthouseProviderModels,
LighthouseTenantConfiguration,
Membership,
Processor,
Provider,
@@ -59,10 +54,6 @@ from api.v1.serializer_utils.integrations import (
S3ConfigSerializer,
SecurityHubConfigSerializer,
)
from api.v1.serializer_utils.lighthouse import (
BedrockCredentialsSerializer,
OpenAICredentialsSerializer,
)
from api.v1.serializer_utils.processors import ProcessorConfigField
from api.v1.serializer_utils.providers import ProviderSecretField
from prowler.lib.mutelist.mutelist import Mutelist
@@ -2759,16 +2750,6 @@ class LighthouseConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
"updated_at": {"read_only": True},
}
def validate_temperature(self, value):
if not 0 <= value <= 1:
raise ValidationError("Temperature must be between 0 and 1.")
return value
def validate_max_tokens(self, value):
if not 500 <= value <= 5000:
raise ValidationError("Max tokens must be between 500 and 5000.")
return value
def validate(self, attrs):
tenant_id = self.context.get("request").tenant_id
if LighthouseConfiguration.objects.filter(tenant_id=tenant_id).exists():
@@ -2777,11 +2758,6 @@ class LighthouseConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
"tenant_id": "Lighthouse configuration already exists for this tenant."
}
)
api_key = attrs.get("api_key")
if api_key is not None:
OpenAICredentialsSerializer(data={"api_key": api_key}).is_valid(
raise_exception=True
)
return super().validate(attrs)
def create(self, validated_data):
@@ -2826,24 +2802,6 @@ class LighthouseConfigUpdateSerializer(BaseWriteSerializer):
"max_tokens": {"required": False},
}
def validate_temperature(self, value):
if not 0 <= value <= 1:
raise ValidationError("Temperature must be between 0 and 1.")
return value
def validate_max_tokens(self, value):
if not 500 <= value <= 5000:
raise ValidationError("Max tokens must be between 500 and 5000.")
return value
def validate(self, attrs):
api_key = attrs.get("api_key", None)
if api_key is not None:
OpenAICredentialsSerializer(data={"api_key": api_key}).is_valid(
raise_exception=True
)
return super().validate(attrs)
def update(self, instance, validated_data):
api_key = validated_data.pop("api_key", None)
instance = super().update(instance, validated_data)
@@ -2973,380 +2931,3 @@ class TenantApiKeyUpdateSerializer(RLSSerializer, BaseWriteSerializer):
):
raise ValidationError("An API key with this name already exists.")
return value
# Lighthouse: Provider configurations
class LighthouseProviderConfigSerializer(RLSSerializer):
"""
Read serializer for LighthouseProviderConfiguration.
"""
# Decrypted credentials are only returned in to_representation when requested
credentials = serializers.JSONField(required=False, read_only=True)
class Meta:
model = LighthouseProviderConfiguration
fields = [
"id",
"inserted_at",
"updated_at",
"provider_type",
"base_url",
"is_active",
"credentials",
"url",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"is_active": {"read_only": True},
"url": {"read_only": True, "view_name": "lighthouse-providers-detail"},
}
class JSONAPIMeta:
resource_name = "lighthouse-providers"
def to_representation(self, instance):
data = super().to_representation(instance)
# Support JSON:API fields filter: fields[lighthouse-providers]=credentials,base_url
fields_param = self.context.get("request", None) and self.context[
"request"
].query_params.get("fields[lighthouse-providers]", "")
creds = instance.credentials_decoded
requested_fields = (
[f.strip() for f in fields_param.split(",")] if fields_param else []
)
if "credentials" in requested_fields:
# Return full decrypted credentials JSON
data["credentials"] = creds
else:
# Return masked credentials by default
def mask_value(value):
if isinstance(value, str):
return "*" * len(value)
if isinstance(value, dict):
return {k: mask_value(v) for k, v in value.items()}
if isinstance(value, list):
return [mask_value(v) for v in value]
return value
# Always return masked credentials, even if creds is None
if creds is not None:
data["credentials"] = mask_value(creds)
else:
# If credentials_decoded returns None, return None for credentials field
data["credentials"] = None
return data
class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
"""
Create serializer for LighthouseProviderConfiguration.
Accepts credentials as JSON; stored encrypted via credentials_decoded.
"""
credentials = serializers.JSONField(write_only=True, required=True)
class Meta:
model = LighthouseProviderConfiguration
fields = [
"provider_type",
"base_url",
"credentials",
"is_active",
]
extra_kwargs = {
"is_active": {"required": False},
"base_url": {"required": False, "allow_null": True},
}
def create(self, validated_data):
credentials = validated_data.pop("credentials")
instance = LighthouseProviderConfiguration(**validated_data)
instance.tenant_id = self.context.get("tenant_id")
instance.credentials_decoded = credentials
try:
instance.save()
return instance
except IntegrityError:
raise ValidationError(
{
"provider_type": "Configuration for this provider already exists for the tenant."
}
)
def validate(self, attrs):
provider_type = attrs.get("provider_type")
credentials = attrs.get("credentials") or {}
if provider_type == LighthouseProviderConfiguration.LLMProviderChoices.OPENAI:
try:
OpenAICredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
provider_type == LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
try:
BedrockCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
return super().validate(attrs)
class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
"""
Update serializer for LighthouseProviderConfiguration.
"""
credentials = serializers.JSONField(write_only=True, required=False)
class Meta:
model = LighthouseProviderConfiguration
fields = [
"id",
"provider_type",
"base_url",
"credentials",
"is_active",
]
extra_kwargs = {
"id": {"read_only": True},
"provider_type": {"read_only": True},
"base_url": {"required": False, "allow_null": True},
"is_active": {"required": False},
}
def update(self, instance, validated_data):
credentials = validated_data.pop("credentials", None)
for attr, value in validated_data.items():
setattr(instance, attr, value)
if credentials is not None:
instance.credentials_decoded = credentials
instance.save()
return instance
def validate(self, attrs):
provider_type = getattr(self.instance, "provider_type", None)
credentials = attrs.get("credentials", None)
if (
credentials is not None
and provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
):
try:
OpenAICredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
credentials is not None
and provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
try:
BedrockCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
return super().validate(attrs)
# Lighthouse: Tenant configuration
class LighthouseTenantConfigSerializer(RLSSerializer):
"""
Read serializer for LighthouseTenantConfiguration.
"""
# Build singleton URL without pk
url = serializers.SerializerMethodField()
def get_url(self, obj):
request = self.context.get("request")
return reverse("lighthouse-config", request=request)
class Meta:
model = LighthouseTenantConfiguration
fields = [
"id",
"inserted_at",
"updated_at",
"business_context",
"default_provider",
"default_models",
"url",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"url": {"read_only": True},
}
class LighthouseTenantConfigUpdateSerializer(BaseWriteSerializer):
class Meta:
model = LighthouseTenantConfiguration
fields = [
"id",
"business_context",
"default_provider",
"default_models",
]
extra_kwargs = {
"id": {"read_only": True},
}
def validate(self, attrs):
request = self.context.get("request")
tenant_id = self.context.get("tenant_id") or (
getattr(request, "tenant_id", None) if request else None
)
default_provider = attrs.get(
"default_provider", getattr(self.instance, "default_provider", "")
)
default_models = attrs.get(
"default_models", getattr(self.instance, "default_models", {})
)
if default_provider:
supported = set(LighthouseProviderConfiguration.LLMProviderChoices.values)
if default_provider not in supported:
raise ValidationError(
{"default_provider": f"Unsupported provider '{default_provider}'."}
)
if not LighthouseProviderConfiguration.objects.filter(
tenant_id=tenant_id, provider_type=default_provider, is_active=True
).exists():
raise ValidationError(
{
"default_provider": f"No active configuration found for '{default_provider}'."
}
)
if default_models is not None and not isinstance(default_models, dict):
raise ValidationError(
{"default_models": "Must be an object mapping provider -> model_id."}
)
for provider_type, model_id in (default_models or {}).items():
provider_cfg = LighthouseProviderConfiguration.objects.filter(
tenant_id=tenant_id, provider_type=provider_type, is_active=True
).first()
if not provider_cfg:
raise ValidationError(
{
"default_models": f"No active configuration for provider '{provider_type}'."
}
)
if not LighthouseProviderModels.objects.filter(
tenant_id=tenant_id,
provider_configuration=provider_cfg,
model_id=model_id,
).exists():
raise ValidationError(
{
"default_models": f"Invalid model '{model_id}' for provider '{provider_type}'."
}
)
return super().validate(attrs)
# Lighthouse: Provider models
class LighthouseProviderModelsSerializer(RLSSerializer):
"""
Read serializer for LighthouseProviderModels.
"""
provider_configuration = serializers.ResourceRelatedField(read_only=True)
class Meta:
model = LighthouseProviderModels
fields = [
"id",
"inserted_at",
"updated_at",
"provider_configuration",
"model_id",
"model_name",
"default_parameters",
"url",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"url": {"read_only": True, "view_name": "lighthouse-models-detail"},
}
class LighthouseProviderModelsCreateSerializer(RLSSerializer, BaseWriteSerializer):
provider_configuration = serializers.ResourceRelatedField(
queryset=LighthouseProviderConfiguration.objects.all()
)
class Meta:
model = LighthouseProviderModels
fields = [
"provider_configuration",
"model_id",
"default_parameters",
]
extra_kwargs = {
"default_parameters": {"required": False},
}
class LighthouseProviderModelsUpdateSerializer(BaseWriteSerializer):
class Meta:
model = LighthouseProviderModels
fields = [
"id",
"default_parameters",
]
extra_kwargs = {
"id": {"read_only": True},
}
+1 -22
View File
@@ -17,9 +17,6 @@ from api.v1.views import (
InvitationAcceptViewSet,
InvitationViewSet,
LighthouseConfigViewSet,
LighthouseProviderConfigViewSet,
LighthouseProviderModelsViewSet,
LighthouseTenantConfigViewSet,
MembershipViewSet,
OverviewViewSet,
ProcessorViewSet,
@@ -37,12 +34,12 @@ from api.v1.views import (
ScheduleViewSet,
SchemaView,
TaskViewSet,
TenantApiKeyViewSet,
TenantFinishACSView,
TenantMembersViewSet,
TenantViewSet,
UserRoleRelationshipView,
UserViewSet,
TenantApiKeyViewSet,
)
router = routers.DefaultRouter(trailing_slash=False)
@@ -70,16 +67,6 @@ router.register(
basename="lighthouseconfiguration",
)
router.register(r"api-keys", TenantApiKeyViewSet, basename="api-key")
router.register(
r"lighthouse/providers",
LighthouseProviderConfigViewSet,
basename="lighthouse-providers",
)
router.register(
r"lighthouse/models",
LighthouseProviderModelsViewSet,
basename="lighthouse-models",
)
tenants_router = routers.NestedSimpleRouter(router, r"tenants", lookup="tenant")
tenants_router.register(
@@ -150,14 +137,6 @@ urlpatterns = [
),
name="provider_group-providers-relationship",
),
# Lighthouse tenant config as singleton endpoint
path(
"lighthouse/configuration",
LighthouseTenantConfigViewSet.as_view(
{"get": "list", "patch": "partial_update"}
),
name="lighthouse-config",
),
# API endpoint to start SAML SSO flow
path(
"auth/saml/initiate/", SAMLInitiateAPIView.as_view(), name="api_saml_initiate"
+1 -340
View File
@@ -1,6 +1,4 @@
import fnmatch
import glob
import json
import logging
import os
from datetime import datetime, timedelta, timezone
@@ -61,13 +59,11 @@ from tasks.tasks import (
backfill_scan_resource_summaries_task,
check_integration_connection_task,
check_lighthouse_connection_task,
check_lighthouse_provider_connection_task,
check_provider_connection_task,
delete_provider_task,
delete_tenant_task,
jira_integration_task,
perform_scan_task,
refresh_lighthouse_provider_models_task,
)
from api.base_views import BaseRLSViewSet, BaseTenantViewset, BaseUserViewset
@@ -87,8 +83,6 @@ from api.filters import (
InvitationFilter,
LatestFindingFilter,
LatestResourceFilter,
LighthouseProviderConfigFilter,
LighthouseProviderModelsFilter,
MembershipFilter,
ProcessorFilter,
ProviderFilter,
@@ -111,9 +105,6 @@ from api.models import (
Integration,
Invitation,
LighthouseConfiguration,
LighthouseProviderConfiguration,
LighthouseProviderModels,
LighthouseTenantConfiguration,
Membership,
Processor,
Provider,
@@ -168,12 +159,6 @@ from api.v1.serializers import (
LighthouseConfigCreateSerializer,
LighthouseConfigSerializer,
LighthouseConfigUpdateSerializer,
LighthouseProviderConfigCreateSerializer,
LighthouseProviderConfigSerializer,
LighthouseProviderConfigUpdateSerializer,
LighthouseProviderModelsSerializer,
LighthouseTenantConfigSerializer,
LighthouseTenantConfigUpdateSerializer,
MembershipSerializer,
OverviewFindingSerializer,
OverviewProviderSerializer,
@@ -1608,25 +1593,6 @@ class ProviderViewSet(BaseRLSViewSet):
},
request=None,
),
threatscore=extend_schema(
tags=["Scan"],
summary="Retrieve threatscore report",
description="Download a specific threatscore report (e.g., 'prowler_threatscore_aws') as a PDF file.",
request=None,
responses={
200: OpenApiResponse(
description="PDF file containing the threatscore report"
),
202: OpenApiResponse(description="The task is in progress"),
401: OpenApiResponse(
description="API key missing or user not Authenticated"
),
403: OpenApiResponse(description="There is a problem with credentials"),
404: OpenApiResponse(
description="The scan has no threatscore reports, or the threatscore report generation task has not started yet"
),
},
),
)
@method_decorator(CACHE_DECORATOR, name="list")
@method_decorator(CACHE_DECORATOR, name="retrieve")
@@ -1683,9 +1649,6 @@ class ScanViewSet(BaseRLSViewSet):
if hasattr(self, "response_serializer_class"):
return self.response_serializer_class
return ScanComplianceReportSerializer
elif self.action == "threatscore":
if hasattr(self, "response_serializer_class"):
return self.response_serializer_class
return super().get_serializer_class()
def partial_update(self, request, *args, **kwargs):
@@ -1790,18 +1753,7 @@ class ScanViewSet(BaseRLSViewSet):
status=status.HTTP_502_BAD_GATEWAY,
)
contents = resp.get("Contents", [])
keys = []
for obj in contents:
key = obj["Key"]
key_basename = os.path.basename(key)
if any(ch in suffix for ch in ("*", "?", "[")):
if fnmatch.fnmatch(key_basename, suffix):
keys.append(key)
elif key_basename == suffix:
keys.append(key)
elif key.endswith(suffix):
# Backward compatibility if suffix already includes directories
keys.append(key)
keys = [obj["Key"] for obj in contents if obj["Key"].endswith(suffix)]
if not keys:
return Response(
{
@@ -1928,45 +1880,6 @@ class ScanViewSet(BaseRLSViewSet):
content, filename = loader
return self._serve_file(content, filename, "text/csv")
@action(
detail=True,
methods=["get"],
url_name="threatscore",
)
def threatscore(self, request, pk=None):
scan = self.get_object()
running_resp = self._get_task_status(scan)
if running_resp:
return running_resp
if not scan.output_location:
return Response(
{
"detail": "The scan has no reports, or the threatscore report generation task has not started yet."
},
status=status.HTTP_404_NOT_FOUND,
)
if scan.output_location.startswith("s3://"):
bucket = env.str("DJANGO_OUTPUT_S3_AWS_OUTPUT_BUCKET", "")
key_prefix = scan.output_location.removeprefix(f"s3://{bucket}/")
prefix = os.path.join(
os.path.dirname(key_prefix),
"threatscore",
"*_threatscore_report.pdf",
)
loader = self._load_file(prefix, s3=True, bucket=bucket, list_objects=True)
else:
base = os.path.dirname(scan.output_location)
pattern = os.path.join(base, "threatscore", "*_threatscore_report.pdf")
loader = self._load_file(pattern, s3=False)
if isinstance(loader, Response):
return loader
content, filename = loader
return self._serve_file(content, filename, "application/pdf")
def create(self, request, *args, **kwargs):
input_serializer = self.get_serializer(data=request.data)
input_serializer.is_valid(raise_exception=True)
@@ -4191,25 +4104,21 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
tags=["Lighthouse AI"],
summary="List all Lighthouse AI configurations",
description="Retrieve a list of all Lighthouse AI configurations.",
deprecated=True,
),
create=extend_schema(
tags=["Lighthouse AI"],
summary="Create a new Lighthouse AI configuration",
description="Create a new Lighthouse AI configuration with the specified details.",
deprecated=True,
),
partial_update=extend_schema(
tags=["Lighthouse AI"],
summary="Partially update a Lighthouse AI configuration",
description="Update certain fields of an existing Lighthouse AI configuration.",
deprecated=True,
),
destroy=extend_schema(
tags=["Lighthouse AI"],
summary="Delete a Lighthouse AI configuration",
description="Remove a Lighthouse AI configuration by its ID.",
deprecated=True,
),
connection=extend_schema(
tags=["Lighthouse AI"],
@@ -4217,7 +4126,6 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
description="Verify the connection to the OpenAI API for a specific Lighthouse AI configuration.",
request=None,
responses={202: OpenApiResponse(response=TaskSerializer)},
deprecated=True,
),
)
class LighthouseConfigViewSet(BaseRLSViewSet):
@@ -4268,253 +4176,6 @@ class LighthouseConfigViewSet(BaseRLSViewSet):
)
@extend_schema_view(
list=extend_schema(
tags=["Lighthouse AI"],
summary="List all LLM provider configs",
description="Retrieve all LLM provider configurations for the current tenant",
),
retrieve=extend_schema(
tags=["Lighthouse AI"],
summary="Retrieve LLM provider config",
description="Get details for a specific provider configuration in the current tenant.",
),
create=extend_schema(
tags=["Lighthouse AI"],
summary="Create LLM provider config",
description="Create a per-tenant configuration for an LLM provider. Only one configuration per provider type is allowed per tenant.",
),
partial_update=extend_schema(
tags=["Lighthouse AI"],
summary="Update LLM provider config",
description="Partially update a provider configuration (e.g., base_url, is_active).",
),
destroy=extend_schema(
tags=["Lighthouse AI"],
summary="Delete LLM provider config",
description="Delete a provider configuration. Any tenant defaults that reference this provider are cleared during deletion.",
),
)
class LighthouseProviderConfigViewSet(BaseRLSViewSet):
queryset = LighthouseProviderConfiguration.objects.all()
serializer_class = LighthouseProviderConfigSerializer
http_method_names = ["get", "post", "patch", "delete"]
filterset_class = LighthouseProviderConfigFilter
def get_queryset(self):
if getattr(self, "swagger_fake_view", False):
return LighthouseProviderConfiguration.objects.none()
return LighthouseProviderConfiguration.objects.filter(
tenant_id=self.request.tenant_id
)
def get_serializer_class(self):
if self.action == "create":
return LighthouseProviderConfigCreateSerializer
elif self.action == "partial_update":
return LighthouseProviderConfigUpdateSerializer
elif self.action in ["connection", "refresh_models"]:
return TaskSerializer
return super().get_serializer_class()
def create(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
instance = serializer.save()
read_serializer = LighthouseProviderConfigSerializer(
instance, context=self.get_serializer_context()
)
headers = self.get_success_headers(read_serializer.data)
return Response(
data=read_serializer.data,
status=status.HTTP_201_CREATED,
headers=headers,
)
def partial_update(self, request, *args, **kwargs):
instance = self.get_object()
serializer = self.get_serializer(
instance,
data=request.data,
partial=True,
context=self.get_serializer_context(),
)
serializer.is_valid(raise_exception=True)
serializer.save()
read_serializer = LighthouseProviderConfigSerializer(
instance, context=self.get_serializer_context()
)
return Response(data=read_serializer.data, status=status.HTTP_200_OK)
@extend_schema(
tags=["Lighthouse AI"],
summary="Check LLM provider connection",
description="Validate provider credentials asynchronously and toggle is_active. Supports OpenAI and AWS Bedrock providers.",
request=None,
responses={202: OpenApiResponse(response=TaskSerializer)},
)
@action(detail=True, methods=["post"], url_name="connection")
def connection(self, request, pk=None):
instance = self.get_object()
with transaction.atomic():
task = check_lighthouse_provider_connection_task.delay(
provider_config_id=str(instance.id), tenant_id=self.request.tenant_id
)
prowler_task = Task.objects.get(id=task.id)
serializer = TaskSerializer(prowler_task)
return Response(
data=serializer.data,
status=status.HTTP_202_ACCEPTED,
headers={
"Content-Location": reverse(
"task-detail", kwargs={"pk": prowler_task.id}
)
},
)
@extend_schema(
tags=["Lighthouse AI"],
summary="Refresh LLM models catalog",
description="Fetch available models for this provider configuration and upsert into catalog. Supports OpenAI and AWS Bedrock providers.",
request=None,
responses={202: OpenApiResponse(response=TaskSerializer)},
)
@action(
detail=True,
methods=["post"],
url_path="refresh-models",
url_name="refresh-models",
)
def refresh_models(self, request, pk=None):
instance = self.get_object()
with transaction.atomic():
task = refresh_lighthouse_provider_models_task.delay(
provider_config_id=str(instance.id), tenant_id=self.request.tenant_id
)
prowler_task = Task.objects.get(id=task.id)
serializer = TaskSerializer(prowler_task)
return Response(
data=serializer.data,
status=status.HTTP_202_ACCEPTED,
headers={
"Content-Location": reverse(
"task-detail", kwargs={"pk": prowler_task.id}
)
},
)
@extend_schema_view(
list=extend_schema(
tags=["Lighthouse AI"],
summary="Get Lighthouse AI Tenant config",
description="Retrieve current tenant-level Lighthouse AI settings. Returns a single configuration object.",
),
partial_update=extend_schema(
tags=["Lighthouse AI"],
summary="Update Lighthouse AI Tenant config",
description="Update tenant-level settings. Validates that the default provider is configured and active and that default model IDs exist for the chosen providers. Auto-creates configuration if it doesn't exist.",
),
)
class LighthouseTenantConfigViewSet(BaseRLSViewSet):
"""
Singleton endpoint for tenant-level Lighthouse AI configuration.
This viewset implements a true singleton pattern:
- GET returns the single configuration object (or 404 if not found)
- PATCH updates/creates the configuration (upsert semantics)
- No ID is required in the URL
"""
queryset = LighthouseTenantConfiguration.objects.all()
serializer_class = LighthouseTenantConfigSerializer
http_method_names = ["get", "patch"]
def get_queryset(self):
if getattr(self, "swagger_fake_view", False):
return LighthouseTenantConfiguration.objects.none()
return LighthouseTenantConfiguration.objects.filter(
tenant_id=self.request.tenant_id
)
def get_serializer_class(self):
if self.action == "partial_update":
return LighthouseTenantConfigUpdateSerializer
return super().get_serializer_class()
def get_object(self):
"""Retrieve the singleton instance for the current tenant."""
obj = LighthouseTenantConfiguration.objects.filter(
tenant_id=self.request.tenant_id
).first()
if obj is None:
raise NotFound("Tenant Lighthouse configuration not found")
self.check_object_permissions(self.request, obj)
return obj
def list(self, request, *args, **kwargs):
"""GET endpoint for singleton - returns single object, not an array."""
instance = self.get_object()
serializer = self.get_serializer(instance)
return Response(serializer.data)
def partial_update(self, request, *args, **kwargs):
"""PATCH endpoint for singleton - no pk required. Auto-creates if not exists."""
# Auto-create tenant config if it doesn't exist (upsert semantics)
instance, created = LighthouseTenantConfiguration.objects.get_or_create(
tenant_id=self.request.tenant_id,
defaults={},
)
# Extract attributes from JSON:API payload
try:
payload = json.loads(request.body)
attributes = payload.get("data", {}).get("attributes", {})
except (json.JSONDecodeError, AttributeError):
raise ValidationError("Invalid JSON:API payload")
serializer = self.get_serializer(instance, data=attributes, partial=True)
serializer.is_valid(raise_exception=True)
serializer.save()
read_serializer = LighthouseTenantConfigSerializer(
instance, context=self.get_serializer_context()
)
return Response(read_serializer.data, status=status.HTTP_200_OK)
@extend_schema_view(
list=extend_schema(
tags=["Lighthouse AI"],
summary="List all LLM models",
description="List available LLM models per configured provider for the current tenant.",
),
retrieve=extend_schema(
tags=["Lighthouse AI"],
summary="Retrieve LLM model details",
description="Get details for a specific LLM model.",
),
)
class LighthouseProviderModelsViewSet(BaseRLSViewSet):
queryset = LighthouseProviderModels.objects.all()
serializer_class = LighthouseProviderModelsSerializer
filterset_class = LighthouseProviderModelsFilter
# Expose as read-only catalog collection
http_method_names = ["get"]
def get_queryset(self):
if getattr(self, "swagger_fake_view", False):
return LighthouseProviderModels.objects.none()
return LighthouseProviderModels.objects.filter(tenant_id=self.request.tenant_id)
def get_serializer_class(self):
return super().get_serializer_class()
@extend_schema_view(
list=extend_schema(
tags=["Processor"],
Binary file not shown.

Before

Width:  |  Height:  |  Size: 24 KiB

+28 -38
View File
@@ -20,10 +20,6 @@ from prowler.lib.outputs.asff.asff import ASFF
from prowler.lib.outputs.compliance.aws_well_architected.aws_well_architected import (
AWSWellArchitected,
)
from prowler.lib.outputs.compliance.c5.c5_aws import AWSC5
from prowler.lib.outputs.compliance.ccc.ccc_aws import CCC_AWS
from prowler.lib.outputs.compliance.ccc.ccc_azure import CCC_Azure
from prowler.lib.outputs.compliance.ccc.ccc_gcp import CCC_GCP
from prowler.lib.outputs.compliance.cis.cis_aws import AWSCIS
from prowler.lib.outputs.compliance.cis.cis_azure import AzureCIS
from prowler.lib.outputs.compliance.cis.cis_gcp import GCPCIS
@@ -77,15 +73,12 @@ COMPLIANCE_CLASS_MAP = {
(lambda name: name.startswith("iso27001_"), AWSISO27001),
(lambda name: name.startswith("kisa"), AWSKISAISMSP),
(lambda name: name == "prowler_threatscore_aws", ProwlerThreatScoreAWS),
(lambda name: name == "ccc_aws", CCC_AWS),
(lambda name: name.startswith("c5_"), AWSC5),
],
"azure": [
(lambda name: name.startswith("cis_"), AzureCIS),
(lambda name: name == "mitre_attack_azure", AzureMitreAttack),
(lambda name: name.startswith("ens_"), AzureENS),
(lambda name: name.startswith("iso27001_"), AzureISO27001),
(lambda name: name == "ccc_azure", CCC_Azure),
(lambda name: name == "prowler_threatscore_azure", ProwlerThreatScoreAzure),
],
"gcp": [
@@ -94,7 +87,6 @@ COMPLIANCE_CLASS_MAP = {
(lambda name: name.startswith("ens_"), GCPENS),
(lambda name: name.startswith("iso27001_"), GCPISO27001),
(lambda name: name == "prowler_threatscore_gcp", ProwlerThreatScoreGCP),
(lambda name: name == "ccc_gcp", CCC_GCP),
],
"kubernetes": [
(lambda name: name.startswith("cis_"), KubernetesCIS),
@@ -183,21 +175,18 @@ def get_s3_client():
return s3_client
def _upload_to_s3(
tenant_id: str, scan_id: str, local_path: str, relative_key: str
) -> str | None:
def _upload_to_s3(tenant_id: str, zip_path: str, scan_id: str) -> str | None:
"""
Upload a local artifact to an S3 bucket under the tenant/scan prefix.
Upload the specified ZIP file to an S3 bucket.
If the S3 bucket environment variables are not configured,
the function returns None without performing an upload.
Args:
tenant_id (str): The tenant identifier used as the first segment of the S3 key.
scan_id (str): The scan identifier used as the second segment of the S3 key.
local_path (str): Filesystem path to the artifact to upload.
relative_key (str): Object key relative to `<tenant_id>/<scan_id>/`.
tenant_id (str): The tenant identifier, used as part of the S3 key prefix.
zip_path (str): The local file system path to the ZIP file to be uploaded.
scan_id (str): The scan identifier, used as part of the S3 key prefix.
Returns:
str | None: S3 URI of the uploaded artifact, or None if the upload is skipped.
str: The S3 URI of the uploaded file (e.g., "s3://<bucket>/<key>") if successful.
None: If the required environment variables for the S3 bucket are not set.
Raises:
botocore.exceptions.ClientError: If the upload attempt to S3 fails for any reason.
"""
@@ -205,26 +194,34 @@ def _upload_to_s3(
if not bucket:
return
if not relative_key:
return
if not os.path.isfile(local_path):
return
try:
s3 = get_s3_client()
s3_key = f"{tenant_id}/{scan_id}/{relative_key}"
s3.upload_file(Filename=local_path, Bucket=bucket, Key=s3_key)
# Upload the ZIP file (outputs) to the S3 bucket
zip_key = f"{tenant_id}/{scan_id}/{os.path.basename(zip_path)}"
s3.upload_file(
Filename=zip_path,
Bucket=bucket,
Key=zip_key,
)
return f"s3://{base.DJANGO_OUTPUT_S3_AWS_OUTPUT_BUCKET}/{s3_key}"
# Upload the compliance directory to the S3 bucket
compliance_dir = os.path.join(os.path.dirname(zip_path), "compliance")
for filename in os.listdir(compliance_dir):
local_path = os.path.join(compliance_dir, filename)
if not os.path.isfile(local_path):
continue
file_key = f"{tenant_id}/{scan_id}/compliance/{filename}"
s3.upload_file(Filename=local_path, Bucket=bucket, Key=file_key)
return f"s3://{base.DJANGO_OUTPUT_S3_AWS_OUTPUT_BUCKET}/{zip_key}"
except (ClientError, NoCredentialsError, ParamValidationError, ValueError) as e:
logger.error(f"S3 upload failed: {str(e)}")
def _generate_output_directory(
output_directory, prowler_provider: object, tenant_id: str, scan_id: str
) -> tuple[str, str, str]:
) -> tuple[str, str]:
"""
Generate a file system path for the output directory of a prowler scan.
@@ -251,7 +248,6 @@ def _generate_output_directory(
>>> _generate_output_directory("/tmp", "aws", "tenant-1234", "scan-5678")
'/tmp/tenant-1234/aws/scan-5678/prowler-output-2023-02-15T12:34:56',
'/tmp/tenant-1234/aws/scan-5678/compliance/prowler-output-2023-02-15T12:34:56'
'/tmp/tenant-1234/aws/scan-5678/threatscore/prowler-output-2023-02-15T12:34:56'
"""
# Sanitize the prowler provider name to ensure it is a valid directory name
prowler_provider_sanitized = re.sub(r"[^\w\-]", "-", prowler_provider)
@@ -272,10 +268,4 @@ def _generate_output_directory(
)
os.makedirs("/".join(compliance_path.split("/")[:-1]), exist_ok=True)
threatscore_path = (
f"{output_directory}/{tenant_id}/{scan_id}/threatscore/prowler-output-"
f"{prowler_provider_sanitized}-{timestamp}"
)
os.makedirs("/".join(threatscore_path.split("/")[:-1]), exist_ok=True)
return path, compliance_path, threatscore_path
return path, compliance_path
@@ -1,372 +0,0 @@
from typing import Dict
import boto3
import openai
from botocore.exceptions import BotoCoreError, ClientError
from celery.utils.log import get_task_logger
from api.models import LighthouseProviderConfiguration, LighthouseProviderModels
logger = get_task_logger(__name__)
def _extract_openai_api_key(
provider_cfg: LighthouseProviderConfiguration,
) -> str | None:
"""
Safely extract the OpenAI API key from a provider configuration.
Args:
provider_cfg (LighthouseProviderConfiguration): The provider configuration instance
containing the credentials.
Returns:
str | None: The API key string if present and valid, otherwise None.
"""
creds = provider_cfg.credentials_decoded
if not isinstance(creds, dict):
return None
api_key = creds.get("api_key")
if not isinstance(api_key, str) or not api_key:
return None
return api_key
def _extract_bedrock_credentials(
provider_cfg: LighthouseProviderConfiguration,
) -> Dict[str, str] | None:
"""
Safely extract AWS Bedrock credentials from a provider configuration.
Args:
provider_cfg (LighthouseProviderConfiguration): The provider configuration instance
containing the credentials.
Returns:
Dict[str, str] | None: Dictionary with 'access_key_id', 'secret_access_key', and
'region' if present and valid, otherwise None.
"""
creds = provider_cfg.credentials_decoded
if not isinstance(creds, dict):
return None
access_key_id = creds.get("access_key_id")
secret_access_key = creds.get("secret_access_key")
region = creds.get("region")
# Validate all required fields are present and are strings
if (
not isinstance(access_key_id, str)
or not access_key_id
or not isinstance(secret_access_key, str)
or not secret_access_key
or not isinstance(region, str)
or not region
):
return None
return {
"access_key_id": access_key_id,
"secret_access_key": secret_access_key,
"region": region,
}
def check_lighthouse_provider_connection(provider_config_id: str) -> Dict:
"""
Validate a Lighthouse provider configuration by calling the provider API and
toggle its active state accordingly.
Args:
provider_config_id: The primary key of the `LighthouseProviderConfiguration`
to validate.
Returns:
dict: A result dictionary with the following keys:
- "connected" (bool): Whether the provider credentials are valid.
- "error" (str | None): The error message when not connected, otherwise None.
Side Effects:
- Updates and persists `is_active` on the `LighthouseProviderConfiguration`.
Raises:
LighthouseProviderConfiguration.DoesNotExist: If no configuration exists with the given ID.
"""
provider_cfg = LighthouseProviderConfiguration.objects.get(pk=provider_config_id)
try:
if (
provider_cfg.provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
):
api_key = _extract_openai_api_key(provider_cfg)
if not api_key:
provider_cfg.is_active = False
provider_cfg.save()
return {"connected": False, "error": "API key is invalid or missing"}
# Test connection by listing models
client = openai.OpenAI(api_key=api_key)
_ = client.models.list()
elif (
provider_cfg.provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
bedrock_creds = _extract_bedrock_credentials(provider_cfg)
if not bedrock_creds:
provider_cfg.is_active = False
provider_cfg.save()
return {
"connected": False,
"error": "AWS credentials are invalid or missing",
}
# Test connection by listing foundation models
bedrock_client = boto3.client(
"bedrock",
aws_access_key_id=bedrock_creds["access_key_id"],
aws_secret_access_key=bedrock_creds["secret_access_key"],
region_name=bedrock_creds["region"],
)
_ = bedrock_client.list_foundation_models()
else:
return {"connected": False, "error": "Unsupported provider type"}
# Connection successful
provider_cfg.is_active = True
provider_cfg.save()
return {"connected": True, "error": None}
except Exception as e:
logger.warning(
"%s connection check failed: %s", provider_cfg.provider_type, str(e)
)
provider_cfg.is_active = False
provider_cfg.save()
return {"connected": False, "error": str(e)}
def _fetch_openai_models(api_key: str) -> Dict[str, str]:
"""
Fetch available models from OpenAI API.
Args:
api_key: OpenAI API key for authentication.
Returns:
Dict mapping model_id to model_name. For OpenAI, both are the same
as the API doesn't provide separate display names.
Raises:
Exception: If the API call fails.
"""
client = openai.OpenAI(api_key=api_key)
models = client.models.list()
# OpenAI uses model.id for both ID and display name
return {m.id: m.id for m in getattr(models, "data", [])}
def _fetch_bedrock_models(bedrock_creds: Dict[str, str]) -> Dict[str, str]:
"""
Fetch available models from AWS Bedrock with entitlement verification.
This function:
1. Lists foundation models with TEXT modality support
2. Lists inference profiles with TEXT modality support
3. Verifies user has entitlement access to each model
Args:
bedrock_creds: Dictionary with 'access_key_id', 'secret_access_key', and 'region'.
Returns:
Dict mapping model_id to model_name for all accessible models.
Raises:
BotoCoreError, ClientError: If AWS API calls fail.
"""
bedrock_client = boto3.client(
"bedrock",
aws_access_key_id=bedrock_creds["access_key_id"],
aws_secret_access_key=bedrock_creds["secret_access_key"],
region_name=bedrock_creds["region"],
)
models_to_check: Dict[str, str] = {}
# Step 1: Get foundation models with TEXT modality
foundation_response = bedrock_client.list_foundation_models()
model_summaries = foundation_response.get("modelSummaries", [])
for model in model_summaries:
# Check if model supports TEXT input and output modality
input_modalities = model.get("inputModalities", [])
output_modalities = model.get("outputModalities", [])
if "TEXT" not in input_modalities or "TEXT" not in output_modalities:
continue
model_id = model.get("modelId")
if not model_id:
continue
inference_types = model.get("inferenceTypesSupported", [])
# Only include models with ON_DEMAND inference support
if "ON_DEMAND" in inference_types:
models_to_check[model_id] = model["modelName"]
# Step 2: Get inference profiles
try:
inference_profiles_response = bedrock_client.list_inference_profiles()
inference_profiles = inference_profiles_response.get(
"inferenceProfileSummaries", []
)
for profile in inference_profiles:
# Check if profile supports TEXT modality
input_modalities = profile.get("inputModalities", [])
output_modalities = profile.get("outputModalities", [])
if "TEXT" not in input_modalities or "TEXT" not in output_modalities:
continue
profile_id = profile.get("inferenceProfileId")
if profile_id:
models_to_check[profile_id] = profile["inferenceProfileName"]
except (BotoCoreError, ClientError) as e:
logger.info(
"Could not fetch inference profiles in %s: %s",
bedrock_creds["region"],
str(e),
)
# Step 3: Verify entitlement availability for each model
available_models: Dict[str, str] = {}
for model_id, model_name in models_to_check.items():
try:
availability = bedrock_client.get_foundation_model_availability(
modelId=model_id
)
entitlement = availability.get("entitlementAvailability")
# Only include models user has access to
if entitlement == "AVAILABLE":
available_models[model_id] = model_name
else:
logger.debug(
"Skipping model %s - entitlement status: %s", model_id, entitlement
)
except (BotoCoreError, ClientError) as e:
logger.debug(
"Could not check availability for model %s: %s", model_id, str(e)
)
continue
return available_models
def refresh_lighthouse_provider_models(provider_config_id: str) -> Dict:
"""
Refresh the catalog of models for a Lighthouse provider configuration.
Fetches the current list of models from the provider, upserts entries into
`LighthouseProviderModels`, and deletes stale entries no longer returned.
Args:
provider_config_id: The primary key of the `LighthouseProviderConfiguration`
whose models should be refreshed.
Returns:
dict: A result dictionary with the following keys on success:
- "created" (int): Number of new model rows created.
- "updated" (int): Number of existing model rows updated.
- "deleted" (int): Number of stale model rows removed.
If an error occurs, the dictionary will contain an "error" (str) field instead.
Raises:
LighthouseProviderConfiguration.DoesNotExist: If no configuration exists with the given ID.
"""
provider_cfg = LighthouseProviderConfiguration.objects.get(pk=provider_config_id)
fetched_models: Dict[str, str] = {}
# Fetch models from the appropriate provider
try:
if (
provider_cfg.provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
):
api_key = _extract_openai_api_key(provider_cfg)
if not api_key:
return {
"created": 0,
"updated": 0,
"deleted": 0,
"error": "API key is invalid or missing",
}
fetched_models = _fetch_openai_models(api_key)
elif (
provider_cfg.provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
bedrock_creds = _extract_bedrock_credentials(provider_cfg)
if not bedrock_creds:
return {
"created": 0,
"updated": 0,
"deleted": 0,
"error": "AWS credentials are invalid or missing",
}
fetched_models = _fetch_bedrock_models(bedrock_creds)
else:
return {
"created": 0,
"updated": 0,
"deleted": 0,
"error": "Unsupported provider type",
}
except Exception as e:
logger.warning(
"Unexpected error refreshing %s models: %s",
provider_cfg.provider_type,
str(e),
)
return {"created": 0, "updated": 0, "deleted": 0, "error": str(e)}
# Upsert models into the catalog
created = 0
updated = 0
for model_id, model_name in fetched_models.items():
obj, was_created = LighthouseProviderModels.objects.update_or_create(
tenant_id=provider_cfg.tenant_id,
provider_configuration=provider_cfg,
model_id=model_id,
defaults={
"model_name": model_name,
"default_parameters": {},
},
)
if was_created:
created += 1
else:
updated += 1
# Delete stale models not present anymore
deleted, _ = (
LighthouseProviderModels.objects.filter(
tenant_id=provider_cfg.tenant_id, provider_configuration=provider_cfg
)
.exclude(model_id__in=fetched_models.keys())
.delete()
)
return {"created": created, "updated": updated, "deleted": deleted}
File diff suppressed because it is too large Load Diff
+6 -70
View File
@@ -1,4 +1,3 @@
import os
from datetime import datetime, timedelta, timezone
from pathlib import Path
from shutil import rmtree
@@ -27,11 +26,6 @@ from tasks.jobs.integrations import (
upload_s3_integration,
upload_security_hub_integration,
)
from tasks.jobs.lighthouse_providers import (
check_lighthouse_provider_connection,
refresh_lighthouse_provider_models,
)
from tasks.jobs.report import generate_threatscore_report_job
from tasks.jobs.scan import (
aggregate_findings,
create_compliance_requirements,
@@ -70,15 +64,10 @@ def _perform_scan_complete_tasks(tenant_id: str, scan_id: str, provider_id: str)
generate_outputs_task.si(
scan_id=scan_id, provider_id=provider_id, tenant_id=tenant_id
),
group(
generate_threatscore_report_task.si(
tenant_id=tenant_id, scan_id=scan_id, provider_id=provider_id
),
check_integrations_task.si(
tenant_id=tenant_id,
provider_id=provider_id,
scan_id=scan_id,
),
check_integrations_task.si(
tenant_id=tenant_id,
provider_id=provider_id,
scan_id=scan_id,
),
).apply_async()
@@ -315,7 +304,7 @@ def generate_outputs_task(scan_id: str, provider_id: str, tenant_id: str):
frameworks_bulk = Compliance.get_bulk(provider_type)
frameworks_avail = get_compliance_frameworks(provider_type)
out_dir, comp_dir, _ = _generate_output_directory(
out_dir, comp_dir = _generate_output_directory(
DJANGO_TMP_OUTPUT_DIRECTORY, provider_uid, tenant_id, scan_id
)
@@ -418,24 +407,7 @@ def generate_outputs_task(scan_id: str, provider_id: str, tenant_id: str):
writer._data.clear()
compressed = _compress_output_files(out_dir)
upload_uri = _upload_to_s3(
tenant_id,
scan_id,
compressed,
os.path.basename(compressed),
)
compliance_dir_path = Path(comp_dir).parent
if compliance_dir_path.exists():
for artifact_path in sorted(compliance_dir_path.iterdir()):
if artifact_path.is_file():
_upload_to_s3(
tenant_id,
scan_id,
str(artifact_path),
f"compliance/{artifact_path.name}",
)
upload_uri = _upload_to_s3(tenant_id, compressed, scan_id)
# S3 integrations (need output_directory)
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
@@ -528,24 +500,6 @@ def check_lighthouse_connection_task(lighthouse_config_id: str, tenant_id: str =
return check_lighthouse_connection(lighthouse_config_id=lighthouse_config_id)
@shared_task(base=RLSTask, name="lighthouse-provider-connection-check")
@set_tenant
def check_lighthouse_provider_connection_task(
provider_config_id: str, tenant_id: str | None = None
) -> dict:
"""Task wrapper to validate provider credentials and set is_active."""
return check_lighthouse_provider_connection(provider_config_id=provider_config_id)
@shared_task(base=RLSTask, name="lighthouse-provider-models-refresh")
@set_tenant
def refresh_lighthouse_provider_models_task(
provider_config_id: str, tenant_id: str | None = None
) -> dict:
"""Task wrapper to refresh provider models catalog for the given configuration."""
return refresh_lighthouse_provider_models(provider_config_id=provider_config_id)
@shared_task(name="integration-check")
def check_integrations_task(tenant_id: str, provider_id: str, scan_id: str = None):
"""
@@ -663,21 +617,3 @@ def jira_integration_task(
return send_findings_to_jira(
tenant_id, integration_id, project_key, issue_type, finding_ids
)
@shared_task(
base=RLSTask,
name="scan-threatscore-report",
queue="scan-reports",
)
def generate_threatscore_report_task(tenant_id: str, scan_id: str, provider_id: str):
"""
Task to generate a threatscore report for a given scan.
Args:
tenant_id (str): The tenant identifier.
scan_id (str): The scan identifier.
provider_id (str): The provider identifier.
"""
return generate_threatscore_report_job(
tenant_id=tenant_id, scan_id=scan_id, provider_id=provider_id
)
+10 -32
View File
@@ -72,26 +72,17 @@ class TestOutputs:
client_mock = MagicMock()
mock_get_client.return_value = client_mock
result = _upload_to_s3(
"tenant-id",
"scan-id",
str(zip_path),
"outputs.zip",
)
result = _upload_to_s3("tenant-id", str(zip_path), "scan-id")
expected_uri = "s3://test-bucket/tenant-id/scan-id/outputs.zip"
assert result == expected_uri
client_mock.upload_file.assert_called_once_with(
Filename=str(zip_path),
Bucket="test-bucket",
Key="tenant-id/scan-id/outputs.zip",
)
assert client_mock.upload_file.call_count == 2
@patch("tasks.jobs.export.get_s3_client")
@patch("tasks.jobs.export.base")
def test_upload_to_s3_missing_bucket(self, mock_base, mock_get_client):
mock_base.DJANGO_OUTPUT_S3_AWS_OUTPUT_BUCKET = ""
result = _upload_to_s3("tenant", "scan", "/tmp/fake.zip", "fake.zip")
result = _upload_to_s3("tenant", "/tmp/fake.zip", "scan")
assert result is None
@patch("tasks.jobs.export.get_s3_client")
@@ -110,15 +101,11 @@ class TestOutputs:
client_mock = MagicMock()
mock_get_client.return_value = client_mock
result = _upload_to_s3(
"tenant",
"scan",
str(compliance_dir / "subdir"),
"compliance/subdir",
)
result = _upload_to_s3("tenant", str(zip_path), "scan")
assert result is None
client_mock.upload_file.assert_not_called()
expected_uri = "s3://test-bucket/tenant/scan/results.zip"
assert result == expected_uri
client_mock.upload_file.assert_called_once()
@patch(
"tasks.jobs.export.get_s3_client",
@@ -139,12 +126,7 @@ class TestOutputs:
compliance_dir.mkdir()
(compliance_dir / "report.csv").write_text("csv")
_upload_to_s3(
"tenant",
"scan",
str(zip_path),
"zipfile.zip",
)
_upload_to_s3("tenant", str(zip_path), "scan")
mock_logger.assert_called()
@patch("tasks.jobs.export.rls_transaction")
@@ -168,17 +150,15 @@ class TestOutputs:
provider = "aws"
expected_timestamp = "20230615103045"
path, compliance, threatscore = _generate_output_directory(
path, compliance = _generate_output_directory(
base_dir, provider, tenant_id, scan_id
)
assert os.path.isdir(os.path.dirname(path))
assert os.path.isdir(os.path.dirname(compliance))
assert os.path.isdir(os.path.dirname(threatscore))
assert path.endswith(f"{provider}-{expected_timestamp}")
assert compliance.endswith(f"{provider}-{expected_timestamp}")
assert threatscore.endswith(f"{provider}-{expected_timestamp}")
@patch("tasks.jobs.export.rls_transaction")
@patch("tasks.jobs.export.Scan")
@@ -201,14 +181,12 @@ class TestOutputs:
provider = "aws/test@check"
expected_timestamp = "20230615103045"
path, compliance, threatscore = _generate_output_directory(
path, compliance = _generate_output_directory(
base_dir, provider, tenant_id, scan_id
)
assert os.path.isdir(os.path.dirname(path))
assert os.path.isdir(os.path.dirname(compliance))
assert os.path.isdir(os.path.dirname(threatscore))
assert path.endswith(f"aws-test-check-{expected_timestamp}")
assert compliance.endswith(f"aws-test-check-{expected_timestamp}")
assert threatscore.endswith(f"aws-test-check-{expected_timestamp}")
-963
View File
@@ -1,963 +0,0 @@
import uuid
from pathlib import Path
from unittest.mock import MagicMock, patch
import matplotlib
import pytest
from tasks.jobs.report import (
_aggregate_requirement_statistics_from_database,
_calculate_requirements_data_from_statistics,
_load_findings_for_requirement_checks,
generate_threatscore_report,
generate_threatscore_report_job,
)
from tasks.tasks import generate_threatscore_report_task
from api.models import Finding, StatusChoices
from prowler.lib.check.models import Severity
matplotlib.use("Agg") # Use non-interactive backend for tests
@pytest.mark.django_db
class TestGenerateThreatscoreReport:
def setup_method(self):
self.scan_id = str(uuid.uuid4())
self.provider_id = str(uuid.uuid4())
self.tenant_id = str(uuid.uuid4())
def test_no_findings_returns_early(self):
with patch("tasks.jobs.report.ScanSummary.objects.filter") as mock_filter:
mock_filter.return_value.exists.return_value = False
result = generate_threatscore_report_job(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
assert result == {"upload": False}
mock_filter.assert_called_once_with(scan_id=self.scan_id)
@patch("tasks.jobs.report.rmtree")
@patch("tasks.jobs.report._upload_to_s3")
@patch("tasks.jobs.report.generate_threatscore_report")
@patch("tasks.jobs.report._generate_output_directory")
@patch("tasks.jobs.report.Provider.objects.get")
@patch("tasks.jobs.report.ScanSummary.objects.filter")
def test_generate_threatscore_report_happy_path(
self,
mock_scan_summary_filter,
mock_provider_get,
mock_generate_output_directory,
mock_generate_report,
mock_upload,
mock_rmtree,
):
mock_scan_summary_filter.return_value.exists.return_value = True
mock_provider = MagicMock()
mock_provider.uid = "provider-uid"
mock_provider.provider = "aws"
mock_provider_get.return_value = mock_provider
mock_generate_output_directory.return_value = (
"/tmp/output",
"/tmp/compressed",
"/tmp/threatscore_path",
)
mock_upload.return_value = "s3://bucket/threatscore_report.pdf"
result = generate_threatscore_report_job(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
assert result == {"upload": True}
mock_generate_report.assert_called_once_with(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
compliance_id="prowler_threatscore_aws",
output_path="/tmp/threatscore_path_threatscore_report.pdf",
provider_id=self.provider_id,
only_failed=True,
min_risk_level=4,
)
mock_upload.assert_called_once_with(
self.tenant_id,
self.scan_id,
"/tmp/threatscore_path_threatscore_report.pdf",
"threatscore/threatscore_path_threatscore_report.pdf",
)
mock_rmtree.assert_called_once_with(
Path("/tmp/threatscore_path_threatscore_report.pdf").parent,
ignore_errors=True,
)
def test_generate_threatscore_report_fails_upload(self):
with (
patch("tasks.jobs.report.ScanSummary.objects.filter") as mock_filter,
patch("tasks.jobs.report.Provider.objects.get") as mock_provider_get,
patch("tasks.jobs.report._generate_output_directory") as mock_gen_dir,
patch("tasks.jobs.report.generate_threatscore_report"),
patch("tasks.jobs.report._upload_to_s3", return_value=None),
):
mock_filter.return_value.exists.return_value = True
# Mock provider
mock_provider = MagicMock()
mock_provider.uid = "aws-provider-uid"
mock_provider.provider = "aws"
mock_provider_get.return_value = mock_provider
mock_gen_dir.return_value = (
"/tmp/output",
"/tmp/compressed",
"/tmp/threatscore_path",
)
result = generate_threatscore_report_job(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
assert result == {"upload": False}
def test_generate_threatscore_report_logs_rmtree_exception(self, caplog):
with (
patch("tasks.jobs.report.ScanSummary.objects.filter") as mock_filter,
patch("tasks.jobs.report.Provider.objects.get") as mock_provider_get,
patch("tasks.jobs.report._generate_output_directory") as mock_gen_dir,
patch("tasks.jobs.report.generate_threatscore_report"),
patch(
"tasks.jobs.report._upload_to_s3", return_value="s3://bucket/report.pdf"
),
patch(
"tasks.jobs.report.rmtree", side_effect=Exception("Test deletion error")
),
):
mock_filter.return_value.exists.return_value = True
# Mock provider
mock_provider = MagicMock()
mock_provider.uid = "aws-provider-uid"
mock_provider.provider = "aws"
mock_provider_get.return_value = mock_provider
mock_gen_dir.return_value = (
"/tmp/output",
"/tmp/compressed",
"/tmp/threatscore_path",
)
with caplog.at_level("ERROR"):
generate_threatscore_report_job(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
assert "Error deleting output files" in caplog.text
def test_generate_threatscore_report_azure_provider(self):
with (
patch("tasks.jobs.report.ScanSummary.objects.filter") as mock_filter,
patch("tasks.jobs.report.Provider.objects.get") as mock_provider_get,
patch("tasks.jobs.report._generate_output_directory") as mock_gen_dir,
patch("tasks.jobs.report.generate_threatscore_report") as mock_generate,
patch(
"tasks.jobs.report._upload_to_s3", return_value="s3://bucket/report.pdf"
),
patch("tasks.jobs.report.rmtree"),
):
mock_filter.return_value.exists.return_value = True
mock_provider = MagicMock()
mock_provider.uid = "azure-provider-uid"
mock_provider.provider = "azure"
mock_provider_get.return_value = mock_provider
mock_gen_dir.return_value = (
"/tmp/output",
"/tmp/compressed",
"/tmp/threatscore_path",
)
generate_threatscore_report_job(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
mock_generate.assert_called_once_with(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
compliance_id="prowler_threatscore_azure",
output_path="/tmp/threatscore_path_threatscore_report.pdf",
provider_id=self.provider_id,
only_failed=True,
min_risk_level=4,
)
@pytest.mark.django_db
class TestAggregateRequirementStatistics:
"""Test suite for _aggregate_requirement_statistics_from_database function."""
def test_aggregates_findings_correctly(self, tenants_fixture, scans_fixture):
"""Verify correct pass/total counts per check are aggregated from database."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
# Create findings with different check_ids and statuses
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-1",
check_id="check_1",
status=StatusChoices.PASS,
severity=Severity.high,
impact=Severity.high,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-2",
check_id="check_1",
status=StatusChoices.FAIL,
severity=Severity.high,
impact=Severity.high,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-3",
check_id="check_2",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
result = _aggregate_requirement_statistics_from_database(
str(tenant.id), str(scan.id)
)
assert result == {
"check_1": {"passed": 1, "total": 2},
"check_2": {"passed": 1, "total": 1},
}
def test_handles_empty_scan(self, tenants_fixture, scans_fixture):
"""Return empty dict when no findings exist for the scan."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
result = _aggregate_requirement_statistics_from_database(
str(tenant.id), str(scan.id)
)
assert result == {}
def test_multiple_findings_same_check(self, tenants_fixture, scans_fixture):
"""Aggregate multiple findings for same check_id correctly."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
# Create 5 findings for same check, 3 passed
for i in range(3):
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid=f"finding-pass-{i}",
check_id="check_same",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
for i in range(2):
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid=f"finding-fail-{i}",
check_id="check_same",
status=StatusChoices.FAIL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
result = _aggregate_requirement_statistics_from_database(
str(tenant.id), str(scan.id)
)
assert result == {"check_same": {"passed": 3, "total": 5}}
def test_only_failed_findings(self, tenants_fixture, scans_fixture):
"""Correctly count when all findings are FAIL status."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-fail-1",
check_id="check_fail",
status=StatusChoices.FAIL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-fail-2",
check_id="check_fail",
status=StatusChoices.FAIL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
result = _aggregate_requirement_statistics_from_database(
str(tenant.id), str(scan.id)
)
assert result == {"check_fail": {"passed": 0, "total": 2}}
def test_mixed_statuses(self, tenants_fixture, scans_fixture):
"""Test with PASS, FAIL, and MANUAL statuses mixed."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-pass",
check_id="check_mixed",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-fail",
check_id="check_mixed",
status=StatusChoices.FAIL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-manual",
check_id="check_mixed",
status=StatusChoices.MANUAL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
result = _aggregate_requirement_statistics_from_database(
str(tenant.id), str(scan.id)
)
# Only PASS status is counted as passed
assert result == {"check_mixed": {"passed": 1, "total": 3}}
@pytest.mark.django_db
class TestLoadFindingsForChecks:
"""Test suite for _load_findings_for_requirement_checks function."""
def test_loads_only_requested_checks(
self, tenants_fixture, scans_fixture, providers_fixture
):
"""Verify only findings for specified check_ids are loaded."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
providers_fixture[0]
# Create findings with different check_ids
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-1",
check_id="check_requested",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-2",
check_id="check_not_requested",
status=StatusChoices.FAIL,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
mock_provider = MagicMock()
with patch(
"tasks.jobs.report.FindingOutput.transform_api_finding"
) as mock_transform:
mock_finding_output = MagicMock()
mock_finding_output.check_id = "check_requested"
mock_transform.return_value = mock_finding_output
result = _load_findings_for_requirement_checks(
str(tenant.id), str(scan.id), ["check_requested"], mock_provider
)
# Only one finding should be loaded
assert "check_requested" in result
assert "check_not_requested" not in result
assert len(result["check_requested"]) == 1
assert mock_transform.call_count == 1
def test_empty_check_ids_returns_empty(
self, tenants_fixture, scans_fixture, providers_fixture
):
"""Return empty dict when check_ids list is empty."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
mock_provider = MagicMock()
result = _load_findings_for_requirement_checks(
str(tenant.id), str(scan.id), [], mock_provider
)
assert result == {}
def test_groups_by_check_id(
self, tenants_fixture, scans_fixture, providers_fixture
):
"""Multiple findings for same check are grouped correctly."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
# Create multiple findings for same check
for i in range(3):
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid=f"finding-{i}",
check_id="check_group",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
mock_provider = MagicMock()
with patch(
"tasks.jobs.report.FindingOutput.transform_api_finding"
) as mock_transform:
mock_finding_output = MagicMock()
mock_finding_output.check_id = "check_group"
mock_transform.return_value = mock_finding_output
result = _load_findings_for_requirement_checks(
str(tenant.id), str(scan.id), ["check_group"], mock_provider
)
assert len(result["check_group"]) == 3
def test_transforms_to_finding_output(
self, tenants_fixture, scans_fixture, providers_fixture
):
"""Findings are transformed using FindingOutput.transform_api_finding."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid="finding-transform",
check_id="check_transform",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
mock_provider = MagicMock()
with patch(
"tasks.jobs.report.FindingOutput.transform_api_finding"
) as mock_transform:
mock_finding_output = MagicMock()
mock_finding_output.check_id = "check_transform"
mock_transform.return_value = mock_finding_output
result = _load_findings_for_requirement_checks(
str(tenant.id), str(scan.id), ["check_transform"], mock_provider
)
# Verify transform was called
mock_transform.assert_called_once()
# Verify the transformed output is in the result
assert result["check_transform"][0] == mock_finding_output
def test_batched_iteration(self, tenants_fixture, scans_fixture, providers_fixture):
"""Works correctly with multiple batches of findings."""
tenant = tenants_fixture[0]
scan = scans_fixture[0]
# Create enough findings to ensure batching (assuming batch size > 1)
for i in range(10):
Finding.objects.create(
tenant_id=tenant.id,
scan=scan,
uid=f"finding-batch-{i}",
check_id="check_batch",
status=StatusChoices.PASS,
severity=Severity.medium,
impact=Severity.medium,
check_metadata={},
raw_result={},
)
mock_provider = MagicMock()
with patch(
"tasks.jobs.report.FindingOutput.transform_api_finding"
) as mock_transform:
mock_finding_output = MagicMock()
mock_finding_output.check_id = "check_batch"
mock_transform.return_value = mock_finding_output
result = _load_findings_for_requirement_checks(
str(tenant.id), str(scan.id), ["check_batch"], mock_provider
)
# All 10 findings should be loaded regardless of batching
assert len(result["check_batch"]) == 10
assert mock_transform.call_count == 10
@pytest.mark.django_db
class TestCalculateRequirementsData:
"""Test suite for _calculate_requirements_data_from_statistics function."""
def test_requirement_status_all_pass(self):
"""Status is PASS when all findings for requirement checks pass."""
mock_compliance = MagicMock()
mock_compliance.Framework = "TestFramework"
mock_compliance.Version = "1.0"
mock_requirement = MagicMock()
mock_requirement.Id = "req_1"
mock_requirement.Description = "Test requirement"
mock_requirement.Checks = ["check_1", "check_2"]
mock_requirement.Attributes = [MagicMock()]
mock_compliance.Requirements = [mock_requirement]
requirement_statistics = {
"check_1": {"passed": 5, "total": 5},
"check_2": {"passed": 3, "total": 3},
}
attributes_by_id, requirements_list = (
_calculate_requirements_data_from_statistics(
mock_compliance, requirement_statistics
)
)
assert len(requirements_list) == 1
assert requirements_list[0]["attributes"]["status"] == StatusChoices.PASS
assert requirements_list[0]["attributes"]["passed_findings"] == 8
assert requirements_list[0]["attributes"]["total_findings"] == 8
def test_requirement_status_some_fail(self):
"""Status is FAIL when some findings fail."""
mock_compliance = MagicMock()
mock_compliance.Framework = "TestFramework"
mock_compliance.Version = "1.0"
mock_requirement = MagicMock()
mock_requirement.Id = "req_2"
mock_requirement.Description = "Test requirement with failures"
mock_requirement.Checks = ["check_3"]
mock_requirement.Attributes = [MagicMock()]
mock_compliance.Requirements = [mock_requirement]
requirement_statistics = {
"check_3": {"passed": 2, "total": 5},
}
attributes_by_id, requirements_list = (
_calculate_requirements_data_from_statistics(
mock_compliance, requirement_statistics
)
)
assert len(requirements_list) == 1
assert requirements_list[0]["attributes"]["status"] == StatusChoices.FAIL
assert requirements_list[0]["attributes"]["passed_findings"] == 2
assert requirements_list[0]["attributes"]["total_findings"] == 5
def test_requirement_status_no_findings(self):
"""Status is MANUAL when no findings exist for requirement."""
mock_compliance = MagicMock()
mock_compliance.Framework = "TestFramework"
mock_compliance.Version = "1.0"
mock_requirement = MagicMock()
mock_requirement.Id = "req_3"
mock_requirement.Description = "Manual requirement"
mock_requirement.Checks = ["check_nonexistent"]
mock_requirement.Attributes = [MagicMock()]
mock_compliance.Requirements = [mock_requirement]
requirement_statistics = {}
attributes_by_id, requirements_list = (
_calculate_requirements_data_from_statistics(
mock_compliance, requirement_statistics
)
)
assert len(requirements_list) == 1
assert requirements_list[0]["attributes"]["status"] == StatusChoices.MANUAL
assert requirements_list[0]["attributes"]["passed_findings"] == 0
assert requirements_list[0]["attributes"]["total_findings"] == 0
def test_aggregates_multiple_checks(self):
"""Correctly sum stats across multiple checks in requirement."""
mock_compliance = MagicMock()
mock_compliance.Framework = "TestFramework"
mock_compliance.Version = "1.0"
mock_requirement = MagicMock()
mock_requirement.Id = "req_4"
mock_requirement.Description = "Multi-check requirement"
mock_requirement.Checks = ["check_a", "check_b", "check_c"]
mock_requirement.Attributes = [MagicMock()]
mock_compliance.Requirements = [mock_requirement]
requirement_statistics = {
"check_a": {"passed": 10, "total": 15},
"check_b": {"passed": 5, "total": 10},
"check_c": {"passed": 0, "total": 5},
}
attributes_by_id, requirements_list = (
_calculate_requirements_data_from_statistics(
mock_compliance, requirement_statistics
)
)
assert len(requirements_list) == 1
# 10 + 5 + 0 = 15 passed
assert requirements_list[0]["attributes"]["passed_findings"] == 15
# 15 + 10 + 5 = 30 total
assert requirements_list[0]["attributes"]["total_findings"] == 30
# Not all passed, so should be FAIL
assert requirements_list[0]["attributes"]["status"] == StatusChoices.FAIL
def test_returns_correct_structure(self):
"""Verify tuple structure and dict keys are correct."""
mock_compliance = MagicMock()
mock_compliance.Framework = "TestFramework"
mock_compliance.Version = "1.0"
mock_attribute = MagicMock()
mock_requirement = MagicMock()
mock_requirement.Id = "req_5"
mock_requirement.Description = "Structure test"
mock_requirement.Checks = ["check_struct"]
mock_requirement.Attributes = [mock_attribute]
mock_compliance.Requirements = [mock_requirement]
requirement_statistics = {"check_struct": {"passed": 1, "total": 1}}
attributes_by_id, requirements_list = (
_calculate_requirements_data_from_statistics(
mock_compliance, requirement_statistics
)
)
# Verify attributes_by_id structure
assert "req_5" in attributes_by_id
assert "attributes" in attributes_by_id["req_5"]
assert "description" in attributes_by_id["req_5"]
assert "req_attributes" in attributes_by_id["req_5"]["attributes"]
assert "checks" in attributes_by_id["req_5"]["attributes"]
# Verify requirements_list structure
assert len(requirements_list) == 1
req = requirements_list[0]
assert "id" in req
assert "attributes" in req
assert "framework" in req["attributes"]
assert "version" in req["attributes"]
assert "status" in req["attributes"]
assert "description" in req["attributes"]
assert "passed_findings" in req["attributes"]
assert "total_findings" in req["attributes"]
@pytest.mark.django_db
class TestGenerateThreatscoreReportFunction:
def setup_method(self):
self.scan_id = str(uuid.uuid4())
self.provider_id = str(uuid.uuid4())
self.tenant_id = str(uuid.uuid4())
self.compliance_id = "prowler_threatscore_aws"
self.output_path = "/tmp/test_threatscore_report.pdf"
@patch("tasks.jobs.report.initialize_prowler_provider")
@patch("tasks.jobs.report.Provider.objects.get")
@patch("tasks.jobs.report.Compliance.get_bulk")
@patch("tasks.jobs.report._aggregate_requirement_statistics_from_database")
@patch("tasks.jobs.report._calculate_requirements_data_from_statistics")
@patch("tasks.jobs.report._load_findings_for_requirement_checks")
@patch("tasks.jobs.report.SimpleDocTemplate")
@patch("tasks.jobs.report.Image")
@patch("tasks.jobs.report.Spacer")
@patch("tasks.jobs.report.Paragraph")
@patch("tasks.jobs.report.PageBreak")
@patch("tasks.jobs.report.Table")
@patch("tasks.jobs.report.TableStyle")
@patch("tasks.jobs.report.plt.subplots")
@patch("tasks.jobs.report.plt.savefig")
@patch("tasks.jobs.report.io.BytesIO")
def test_generate_threatscore_report_success(
self,
mock_bytesio,
mock_savefig,
mock_subplots,
mock_table_style,
mock_table,
mock_page_break,
mock_paragraph,
mock_spacer,
mock_image,
mock_doc_template,
mock_load_findings,
mock_calculate_requirements,
mock_aggregate_statistics,
mock_compliance_get_bulk,
mock_provider_get,
mock_initialize_provider,
):
"""Test the updated generate_threatscore_report using new memory-efficient architecture."""
mock_provider = MagicMock()
mock_provider.provider = "aws"
mock_provider_get.return_value = mock_provider
prowler_provider = MagicMock()
mock_initialize_provider.return_value = prowler_provider
# Mock compliance object with requirements
mock_compliance_obj = MagicMock()
mock_compliance_obj.Framework = "ProwlerThreatScore"
mock_compliance_obj.Version = "1.0"
mock_compliance_obj.Description = "Test Description"
# Configure requirement with properly set numeric attributes for chart generation
mock_requirement = MagicMock()
mock_requirement.Id = "req_1"
mock_requirement.Description = "Test requirement"
mock_requirement.Checks = ["check_1"]
# Create a properly configured attribute mock with numeric values
mock_requirement_attr = MagicMock()
mock_requirement_attr.Section = "1. IAM"
mock_requirement_attr.SubSection = "1.1 Identity"
mock_requirement_attr.Title = "Test Requirement Title"
mock_requirement_attr.LevelOfRisk = 3
mock_requirement_attr.Weight = 100
mock_requirement_attr.AttributeDescription = "Test requirement description"
mock_requirement_attr.AdditionalInformation = "Additional test information"
mock_requirement.Attributes = [mock_requirement_attr]
mock_compliance_obj.Requirements = [mock_requirement]
mock_compliance_get_bulk.return_value = {
self.compliance_id: mock_compliance_obj
}
# Mock the aggregated statistics from database
mock_aggregate_statistics.return_value = {"check_1": {"passed": 5, "total": 10}}
# Mock the calculated requirements data with properly configured attributes
mock_attributes_by_id = {
"req_1": {
"attributes": {
"req_attributes": [mock_requirement_attr],
"checks": ["check_1"],
},
"description": "Test requirement",
}
}
mock_requirements_list = [
{
"id": "req_1",
"attributes": {
"framework": "ProwlerThreatScore",
"version": "1.0",
"status": StatusChoices.FAIL,
"description": "Test requirement",
"passed_findings": 5,
"total_findings": 10,
},
}
]
mock_calculate_requirements.return_value = (
mock_attributes_by_id,
mock_requirements_list,
)
# Mock the on-demand loaded findings
mock_finding_output = MagicMock()
mock_finding_output.check_id = "check_1"
mock_finding_output.status = "FAIL"
mock_finding_output.metadata = MagicMock()
mock_finding_output.metadata.CheckTitle = "Test Check"
mock_finding_output.metadata.Severity = "HIGH"
mock_finding_output.resource_name = "test-resource"
mock_finding_output.region = "us-east-1"
mock_load_findings.return_value = {"check_1": [mock_finding_output]}
# Mock PDF generation components
mock_doc = MagicMock()
mock_doc_template.return_value = mock_doc
mock_fig, mock_ax = MagicMock(), MagicMock()
mock_subplots.return_value = (mock_fig, mock_ax)
mock_buffer = MagicMock()
mock_bytesio.return_value = mock_buffer
mock_image.return_value = MagicMock()
mock_spacer.return_value = MagicMock()
mock_paragraph.return_value = MagicMock()
mock_page_break.return_value = MagicMock()
mock_table.return_value = MagicMock()
mock_table_style.return_value = MagicMock()
# Execute the function
generate_threatscore_report(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
compliance_id=self.compliance_id,
output_path=self.output_path,
provider_id=self.provider_id,
only_failed=True,
min_risk_level=4,
)
# Verify the new workflow was followed
mock_provider_get.assert_called_once_with(id=self.provider_id)
mock_initialize_provider.assert_called_once_with(mock_provider)
mock_compliance_get_bulk.assert_called_once_with("aws")
# Verify the new functions were called in correct order with correct parameters
mock_aggregate_statistics.assert_called_once_with(self.tenant_id, self.scan_id)
mock_calculate_requirements.assert_called_once_with(
mock_compliance_obj, {"check_1": {"passed": 5, "total": 10}}
)
mock_load_findings.assert_called_once_with(
self.tenant_id, self.scan_id, ["check_1"], prowler_provider
)
# Verify PDF was built
mock_doc_template.assert_called_once()
mock_doc.build.assert_called_once()
@patch("tasks.jobs.report.initialize_prowler_provider")
@patch("tasks.jobs.report.Provider.objects.get")
@patch("tasks.jobs.report.Compliance.get_bulk")
@patch("tasks.jobs.report.Finding.all_objects.filter")
def test_generate_threatscore_report_exception_handling(
self,
mock_finding_filter,
mock_compliance_get_bulk,
mock_provider_get,
mock_initialize_provider,
):
mock_provider_get.side_effect = Exception("Provider not found")
with pytest.raises(Exception, match="Provider not found"):
generate_threatscore_report(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
compliance_id=self.compliance_id,
output_path=self.output_path,
provider_id=self.provider_id,
only_failed=True,
min_risk_level=4,
)
@pytest.mark.django_db
class TestGenerateThreatscoreReportTask:
def setup_method(self):
self.scan_id = str(uuid.uuid4())
self.provider_id = str(uuid.uuid4())
self.tenant_id = str(uuid.uuid4())
@patch("tasks.tasks.generate_threatscore_report_job")
def test_generate_threatscore_report_task_calls_job(self, mock_generate_job):
mock_generate_job.return_value = {"upload": True}
result = generate_threatscore_report_task(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
assert result == {"upload": True}
mock_generate_job.assert_called_once_with(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
@patch("tasks.tasks.generate_threatscore_report_job")
def test_generate_threatscore_report_task_handles_job_exception(
self, mock_generate_job
):
mock_generate_job.side_effect = Exception("Job failed")
with pytest.raises(Exception, match="Job failed"):
generate_threatscore_report_task(
tenant_id=self.tenant_id,
scan_id=self.scan_id,
provider_id=self.provider_id,
)
+59 -127
View File
@@ -98,11 +98,7 @@ class TestGenerateOutputs:
),
patch(
"tasks.tasks._generate_output_directory",
return_value=(
"/tmp/test/out-dir",
"/tmp/test/comp-dir",
"/tmp/test/threat-dir",
),
return_value=("out-dir", "comp-dir"),
),
patch("tasks.tasks.Scan.all_objects.filter") as mock_scan_update,
patch("tasks.tasks.rmtree"),
@@ -130,8 +126,7 @@ class TestGenerateOutputs:
patch("tasks.tasks.get_compliance_frameworks"),
patch("tasks.tasks.Finding.all_objects.filter") as mock_findings,
patch(
"tasks.tasks._generate_output_directory",
return_value=("/tmp/test/out", "/tmp/test/comp", "/tmp/test/threat"),
"tasks.tasks._generate_output_directory", return_value=("out", "comp")
),
patch("tasks.tasks.FindingOutput._transform_findings_stats"),
patch("tasks.tasks.FindingOutput.transform_api_finding"),
@@ -173,35 +168,15 @@ class TestGenerateOutputs:
mock_finding_output = MagicMock()
mock_finding_output.compliance = {"cis": ["requirement-1", "requirement-2"]}
html_writer_mock = MagicMock()
html_writer_mock._data = []
html_writer_mock.close_file = False
html_writer_mock.transform = MagicMock()
html_writer_mock.batch_write_data_to_file = MagicMock()
compliance_writer_mock = MagicMock()
compliance_writer_mock._data = []
compliance_writer_mock.close_file = False
compliance_writer_mock.transform = MagicMock()
compliance_writer_mock.batch_write_data_to_file = MagicMock()
# Create a mock class that returns our mock instance when called
mock_compliance_class = MagicMock(return_value=compliance_writer_mock)
mock_provider = MagicMock()
mock_provider.provider = "aws"
mock_provider.uid = "test-provider-uid"
with (
patch("tasks.tasks.ScanSummary.objects.filter") as mock_filter,
patch("tasks.tasks.Provider.objects.get", return_value=mock_provider),
patch("tasks.tasks.Provider.objects.get"),
patch("tasks.tasks.initialize_prowler_provider"),
patch("tasks.tasks.Compliance.get_bulk", return_value={"cis": MagicMock()}),
patch("tasks.tasks.get_compliance_frameworks", return_value=["cis"]),
patch("tasks.tasks.Finding.all_objects.filter") as mock_findings,
patch(
"tasks.tasks._generate_output_directory",
return_value=("/tmp/test/out", "/tmp/test/comp", "/tmp/test/threat"),
"tasks.tasks._generate_output_directory", return_value=("out", "comp")
),
patch(
"tasks.tasks.FindingOutput._transform_findings_stats",
@@ -215,20 +190,6 @@ class TestGenerateOutputs:
patch("tasks.tasks._upload_to_s3", return_value="s3://bucket/f.zip"),
patch("tasks.tasks.Scan.all_objects.filter"),
patch("tasks.tasks.rmtree"),
patch(
"tasks.tasks.OUTPUT_FORMATS_MAPPING",
{
"html": {
"class": lambda *args, **kwargs: html_writer_mock,
"suffix": ".html",
"kwargs": {},
}
},
),
patch(
"tasks.tasks.COMPLIANCE_CLASS_MAP",
{"aws": [(lambda x: True, mock_compliance_class)]},
),
):
mock_filter.return_value.exists.return_value = True
mock_findings.return_value.order_by.return_value.iterator.return_value = [
@@ -236,12 +197,29 @@ class TestGenerateOutputs:
True,
]
generate_outputs_task(
scan_id=self.scan_id,
provider_id=self.provider_id,
tenant_id=self.tenant_id,
)
html_writer_mock.batch_write_data_to_file.assert_called_once()
html_writer_mock = MagicMock()
with (
patch(
"tasks.tasks.OUTPUT_FORMATS_MAPPING",
{
"html": {
"class": lambda *args, **kwargs: html_writer_mock,
"suffix": ".html",
"kwargs": {},
}
},
),
patch(
"tasks.tasks.COMPLIANCE_CLASS_MAP",
{"aws": [(lambda x: True, MagicMock())]},
),
):
generate_outputs_task(
scan_id=self.scan_id,
provider_id=self.provider_id,
tenant_id=self.tenant_id,
)
html_writer_mock.batch_write_data_to_file.assert_called_once()
def test_transform_called_only_on_second_batch(self):
raw1 = MagicMock()
@@ -278,11 +256,7 @@ class TestGenerateOutputs:
),
patch(
"tasks.tasks._generate_output_directory",
return_value=(
"/tmp/test/outdir",
"/tmp/test/compdir",
"/tmp/test/threatdir",
),
return_value=("outdir", "compdir"),
),
patch("tasks.tasks._compress_output_files", return_value="outdir.zip"),
patch("tasks.tasks._upload_to_s3", return_value="s3://bucket/outdir.zip"),
@@ -329,14 +303,12 @@ class TestGenerateOutputs:
def __init__(self, *args, **kwargs):
self.transform_calls = []
self._data = []
self.close_file = False
writer_instances.append(self)
def transform(self, fos, comp_obj, name):
self.transform_calls.append((fos, comp_obj, name))
def batch_write_data_to_file(self):
# Mock implementation - do nothing
pass
two_batches = [
@@ -357,11 +329,7 @@ class TestGenerateOutputs:
patch("tasks.tasks.get_compliance_frameworks", return_value=["cis"]),
patch(
"tasks.tasks._generate_output_directory",
return_value=(
"/tmp/test/outdir",
"/tmp/test/compdir",
"/tmp/test/threatdir",
),
return_value=("outdir", "compdir"),
),
patch("tasks.tasks.FindingOutput._transform_findings_stats"),
patch(
@@ -400,35 +368,15 @@ class TestGenerateOutputs:
mock_finding_output = MagicMock()
mock_finding_output.compliance = {"cis": ["requirement-1", "requirement-2"]}
json_writer_mock = MagicMock()
json_writer_mock._data = []
json_writer_mock.close_file = False
json_writer_mock.transform = MagicMock()
json_writer_mock.batch_write_data_to_file = MagicMock()
compliance_writer_mock = MagicMock()
compliance_writer_mock._data = []
compliance_writer_mock.close_file = False
compliance_writer_mock.transform = MagicMock()
compliance_writer_mock.batch_write_data_to_file = MagicMock()
# Create a mock class that returns our mock instance when called
mock_compliance_class = MagicMock(return_value=compliance_writer_mock)
mock_provider = MagicMock()
mock_provider.provider = "aws"
mock_provider.uid = "test-provider-uid"
with (
patch("tasks.tasks.ScanSummary.objects.filter") as mock_filter,
patch("tasks.tasks.Provider.objects.get", return_value=mock_provider),
patch("tasks.tasks.Provider.objects.get"),
patch("tasks.tasks.initialize_prowler_provider"),
patch("tasks.tasks.Compliance.get_bulk", return_value={"cis": MagicMock()}),
patch("tasks.tasks.get_compliance_frameworks", return_value=["cis"]),
patch("tasks.tasks.Finding.all_objects.filter") as mock_findings,
patch(
"tasks.tasks._generate_output_directory",
return_value=("/tmp/test/out", "/tmp/test/comp", "/tmp/test/threat"),
"tasks.tasks._generate_output_directory", return_value=("out", "comp")
),
patch(
"tasks.tasks.FindingOutput._transform_findings_stats",
@@ -442,20 +390,6 @@ class TestGenerateOutputs:
patch("tasks.tasks._upload_to_s3", return_value="s3://bucket/file.zip"),
patch("tasks.tasks.Scan.all_objects.filter"),
patch("tasks.tasks.rmtree", side_effect=Exception("Test deletion error")),
patch(
"tasks.tasks.OUTPUT_FORMATS_MAPPING",
{
"json": {
"class": lambda *args, **kwargs: json_writer_mock,
"suffix": ".json",
"kwargs": {},
}
},
),
patch(
"tasks.tasks.COMPLIANCE_CLASS_MAP",
{"aws": [(lambda x: True, mock_compliance_class)]},
),
):
mock_filter.return_value.exists.return_value = True
mock_findings.return_value.order_by.return_value.iterator.return_value = [
@@ -463,13 +397,29 @@ class TestGenerateOutputs:
True,
]
with caplog.at_level("ERROR"):
generate_outputs_task(
scan_id=self.scan_id,
provider_id=self.provider_id,
tenant_id=self.tenant_id,
)
assert "Error deleting output files" in caplog.text
with (
patch(
"tasks.tasks.OUTPUT_FORMATS_MAPPING",
{
"json": {
"class": lambda *args, **kwargs: MagicMock(),
"suffix": ".json",
"kwargs": {},
}
},
),
patch(
"tasks.tasks.COMPLIANCE_CLASS_MAP",
{"aws": [(lambda x: True, MagicMock())]},
),
):
with caplog.at_level("ERROR"):
generate_outputs_task(
scan_id=self.scan_id,
provider_id=self.provider_id,
tenant_id=self.tenant_id,
)
assert "Error deleting output files" in caplog.text
@patch("tasks.tasks.rls_transaction")
@patch("tasks.tasks.Integration.objects.filter")
@@ -485,8 +435,7 @@ class TestGenerateOutputs:
patch("tasks.tasks.get_compliance_frameworks", return_value=[]),
patch("tasks.tasks.Finding.all_objects.filter") as mock_findings,
patch(
"tasks.tasks._generate_output_directory",
return_value=("/tmp/test/out", "/tmp/test/comp", "/tmp/test/threat"),
"tasks.tasks._generate_output_directory", return_value=("out", "comp")
),
patch("tasks.tasks.FindingOutput._transform_findings_stats"),
patch("tasks.tasks.FindingOutput.transform_api_finding"),
@@ -527,15 +476,8 @@ class TestScanCompleteTasks:
@patch("tasks.tasks.create_compliance_requirements_task.apply_async")
@patch("tasks.tasks.perform_scan_summary_task.si")
@patch("tasks.tasks.generate_outputs_task.si")
@patch("tasks.tasks.generate_threatscore_report_task.si")
@patch("tasks.tasks.check_integrations_task.si")
def test_scan_complete_tasks(
self,
mock_check_integrations_task,
mock_threatscore_task,
mock_outputs_task,
mock_scan_summary_task,
mock_compliance_tasks,
self, mock_outputs_task, mock_scan_summary_task, mock_compliance_tasks
):
_perform_scan_complete_tasks("tenant-id", "scan-id", "provider-id")
mock_compliance_tasks.assert_called_once_with(
@@ -550,16 +492,6 @@ class TestScanCompleteTasks:
provider_id="provider-id",
tenant_id="tenant-id",
)
mock_threatscore_task.assert_called_once_with(
tenant_id="tenant-id",
scan_id="scan-id",
provider_id="provider-id",
)
mock_check_integrations_task.assert_called_once_with(
tenant_id="tenant-id",
provider_id="provider-id",
scan_id="scan-id",
)
@pytest.mark.django_db
@@ -730,7 +662,7 @@ class TestCheckIntegrationsTask:
mock_initialize_provider.return_value = MagicMock()
mock_compliance_bulk.return_value = {}
mock_get_frameworks.return_value = []
mock_generate_dir.return_value = ("out-dir", "comp-dir", "threat-dir")
mock_generate_dir.return_value = ("out-dir", "comp-dir")
mock_transform_stats.return_value = {"stats": "data"}
# Mock findings
@@ -855,7 +787,7 @@ class TestCheckIntegrationsTask:
mock_initialize_provider.return_value = MagicMock()
mock_compliance_bulk.return_value = {}
mock_get_frameworks.return_value = []
mock_generate_dir.return_value = ("out-dir", "comp-dir", "threat-dir")
mock_generate_dir.return_value = ("out-dir", "comp-dir")
mock_transform_stats.return_value = {"stats": "data"}
# Mock findings
@@ -971,7 +903,7 @@ class TestCheckIntegrationsTask:
mock_initialize_provider.return_value = MagicMock()
mock_compliance_bulk.return_value = {}
mock_get_frameworks.return_value = []
mock_generate_dir.return_value = ("out-dir", "comp-dir", "threat-dir")
mock_generate_dir.return_value = ("out-dir", "comp-dir")
mock_transform_stats.return_value = {"stats": "data"}
# Mock findings
-43
View File
@@ -1,43 +0,0 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_DESCRIPTION"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SECTION"
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
aux = data[
[
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_DESCRIPTION",
)
-36
View File
@@ -1,36 +0,0 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
-36
View File
@@ -1,36 +0,0 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
-36
View File
@@ -1,36 +0,0 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_ID"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_ID"] = data["REQUIREMENTS_ID"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
-41
View File
@@ -1,41 +0,0 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
warnings.filterwarnings("ignore")
def get_table(data):
"""
Generate CIS OCI Foundations Benchmark v3.0 compliance table.
Args:
data: DataFrame containing compliance check results with columns:
- REQUIREMENTS_ID: CIS requirement ID (e.g., "1.1", "2.1")
- REQUIREMENTS_DESCRIPTION: Description of the requirement
- REQUIREMENTS_ATTRIBUTES_SECTION: CIS section name
- CHECKID: Prowler check identifier
- STATUS: Check status (PASS/FAIL)
- REGION: OCI region
- TENANCYID: OCI tenancy OCID
- RESOURCEID: Resource OCID or identifier
Returns:
Section containers organized by CIS sections for dashboard display
"""
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"TENANCYID",
"RESOURCEID",
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
)
-4
View File
@@ -1,4 +0,0 @@
.idea/
.git/
.claude/
AGENTS.md
-45
View File
@@ -1,45 +0,0 @@
# Prowler Documentation
This repository contains the Prowler Open Source documentation powered by [Mintlify](https://mintlify.com).
## Documentation Structure
- **Getting Started**: Overview, installation, and basic usage guides
- **User Guide**: Comprehensive guides for Prowler App, CLI, providers, and compliance
- **Developer Guide**: Technical documentation for developers contributing to Prowler
## Local Development
Install the [Mintlify CLI](https://www.npmjs.com/package/mint) to preview documentation changes locally:
```bash
npm i -g mint
```
Run the following command at the root of your documentation (where `mint.json` is located):
```bash
mint dev
```
View your local preview at `http://localhost:3000`.
## Publishing Changes
Changes pushed to the main branch are automatically deployed to production through Mintlify's GitHub integration.
## Documentation Guidelines
When contributing to the documentation, please follow the Prowler documentation style guide located in the `.claude` directory.
## Troubleshooting
- If your dev environment isn't running: Run `mint update` to ensure you have the most recent version of the CLI.
- If a page loads as a 404: Make sure you are running in a folder with a valid `mint.json` file and that the page path is correctly listed in the navigation.
## Resources
- [Prowler GitHub Repository](https://github.com/prowler-cloud/prowler)
- [Prowler Documentation](https://docs.prowler.com/)
- [Mintlify Documentation](https://mintlify.com/docs)
- [Mintlify Community](https://mintlify.com/community)
+61
View File
@@ -0,0 +1,61 @@
## Access Prowler App
After [installation](../installation/prowler-app.md), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
<img src="../../img/sign-up-button.png" alt="Sign Up Button" width="320"/>
<img src="../../img/sign-up.png" alt="Sign Up" width="285"/>
???+ note "User creation and default tenant behavior"
When creating a new user, the behavior depends on whether an invitation is provided:
- **Without an invitation**:
- A new tenant is automatically created.
- The new user is assigned to this tenant.
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
## Log In
Access Prowler App by logging in with **email and password**.
<img src="../../img/log-in.png" alt="Log In" width="285"/>
## Add Cloud Provider
Configure a cloud provider for scanning:
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
2. Select the cloud provider.
3. Enter the provider's identifier (Optional: Add an alias):
- **AWS**: Account ID
- **GCP**: Project ID
- **Azure**: Subscription ID
- **Kubernetes**: Cluster ID
- **M365**: Domain ID
4. Follow the guided instructions to add and authenticate your credentials.
## Start a Scan
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
Click `Go to Scans` to monitor progress.
## View Results
Review findings during scan execution in the following sections:
- **Overview** Provides a high-level summary of your scans.
<img src="../../products/img/overview.png" alt="Overview" width="700"/>
- **Compliance** Displays compliance insights based on security frameworks.
<img src="../../img/compliance.png" alt="Compliance" width="700"/>
> For detailed usage instructions, refer to the [Prowler App Guide](../tutorials/prowler-app.md).
???+ note
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
@@ -1,24 +1,18 @@
---
title: 'Basic Usage'
---
## Running Prowler
Running Prowler requires specifying the provider (e.g `aws`, `gcp`, `azure`, `kubernetes`, `m365`, `github`, `iac` or `mongodbatlas`):
<Note>
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
???+ note
If no provider is specified, AWS is used by default for backward compatibility with Prowler v2.
</Note>
```console
prowler <provider>
```
![Prowler Execution](/images/short-display.png)
![Prowler Execution](../img/short-display.png)
<Note>
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
???+ note
Running the `prowler` command without options will uses environment variable credentials. Refer to the Authentication section of each provider for credential configuration details.
</Note>
## Verbose Output
If you prefer the former verbose output, use: `--verbose`. This allows seeing more info while Prowler is running, minimal output is displayed unless verbosity is enabled.
@@ -32,7 +26,7 @@ prowler <provider> -M csv json-asff json-ocsf html
```
The HTML report is saved in the output directory, alongside other reports. It will look like this:
![Prowler Execution](/images/html-output.png)
![Prowler Execution](../img/html-output.png)
## Listing Available Checks and Services
@@ -64,7 +58,7 @@ prowler kubernetes --excluded-services controllermanager
```
## Additional Options
Explore more advanced time-saving execution methods in the [Miscellaneous](/user-guide/cli/tutorials/misc) section.
Explore more advanced time-saving execution methods in the [Miscellaneous](../tutorials/misc.md) section.
Access the help menu and view all available options with `-h`/`--help`:
@@ -80,11 +74,10 @@ Use a custom AWS profile with `-p`/`--profile` and/or specific AWS regions with
prowler aws --profile custom-profile -f us-east-1 eu-south-2
```
<Note>
By default, `prowler` will scan all AWS regions.
???+ note
By default, `prowler` will scan all AWS regions.
</Note>
See more details about AWS Authentication in the [Authentication Section](/user-guide/providers/aws/authentication) section.
See more details about AWS Authentication in the [Authentication Section](../tutorials/aws/authentication.md) section.
## Azure
@@ -104,7 +97,7 @@ prowler azure --browser-auth --tenant-id "XXXXXXXX"
prowler azure --managed-identity-auth
```
See more details about Azure Authentication in the [Authentication Section](/user-guide/providers/azure/authentication)
See more details about Azure Authentication in the [Authentication Section](../tutorials/azure/authentication.md)
By default, Prowler scans all accessible subscriptions. Scan specific subscriptions using the following flag (using az cli auth as example):
@@ -161,10 +154,9 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
```console
prowler kubernetes --kubeconfig-file path
```
<Note>
???+ note
If no `--kubeconfig-file` is provided, Prowler will use the default KubeConfig file location (`~/.kube/config`).
</Note>
- **In-Cluster Execution**
To run Prowler inside the cluster, apply the provided YAML configuration to deploy a job in a new namespace:
@@ -178,10 +170,9 @@ Prowler enables security scanning of Kubernetes clusters, supporting both **in-c
kubectl logs prowler-XXXXX --namespace prowler-ns
```
<Note>
???+ note
By default, Prowler scans all namespaces in the active Kubernetes context. Use the `--context`flag to specify the context to be scanned and `--namespaces` to restrict scanning to specific namespaces.
</Note>
## Microsoft 365
Microsoft 365 requires specifying the auth method:
@@ -199,7 +190,7 @@ prowler m365 --browser-auth --tenant-id "XXXXXXXX"
```
See more details about M365 Authentication in the [Authentication Section](/user-guide/providers/microsoft365/authentication) section.
See more details about M365 Authentication in the [Authentication Section](../tutorials/microsoft365/authentication.md) section.
## GitHub
@@ -220,14 +211,13 @@ Prowler enables security scanning of your **GitHub account**, including **Reposi
prowler github --github-app-id app_id --github-app-key app_key
```
<Note>
???+ note
If no login method is explicitly provided, Prowler will automatically attempt to authenticate using environment variables in the following order of precedence:
1. `GITHUB_PERSONAL_ACCESS_TOKEN`
2. `OAUTH_APP_TOKEN`
3. `GITHUB_APP_ID` and `GITHUB_APP_KEY`
</Note>
## Infrastructure as Code (IaC)
Prowler's Infrastructure as Code (IaC) provider enables you to scan local or remote infrastructure code for security and compliance issues using [Trivy](https://trivy.dev/). This provider supports a wide range of IaC frameworks, allowing you to assess your code before deployment.
@@ -254,15 +244,14 @@ prowler iac --scan-path ./my-iac-directory --frameworks terraform kubernetes
prowler iac --scan-path ./my-iac-directory --exclude-path ./my-iac-directory/test,./my-iac-directory/examples
```
<Note>
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
- The IaC provider does not require cloud authentication for local scans.
- It is ideal for CI/CD pipelines and local development environments.
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
???+ note
- `--scan-path` and `--scan-repository-url` are mutually exclusive; only one can be specified at a time.
- For remote repository scans, authentication can be provided via CLI flags or environment variables (`GITHUB_OAUTH_APP_TOKEN`, `GITHUB_USERNAME`, `GITHUB_PERSONAL_ACCESS_TOKEN`). CLI flags take precedence.
- The IaC provider does not require cloud authentication for local scans.
- It is ideal for CI/CD pipelines and local development environments.
- For more details on supported scanners, see the [Trivy documentation](https://trivy.dev/latest/docs/scanner/vulnerability/)
</Note>
See more details about IaC scanning in the [IaC Tutorial](/user-guide/providers/iac/getting-started-iac) section.
See more details about IaC scanning in the [IaC Tutorial](../tutorials/iac/getting-started-iac.md) section.
## MongoDB Atlas
@@ -287,30 +276,4 @@ You can filter scans to specific organizations or projects:
prowler mongodbatlas --atlas-project-id <project_id>
```
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](/user-guide/providers/mongodbatlas/authentication)
## Oracle Cloud
Prowler allows you to scan your Oracle Cloud deployments for security and compliance issues.
You have two options to authenticate:
1. OCI Config File Authentication: this config file can be generated using the OCI CLI with the `oci session authenticate` command or created manually using the OCI Console. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#oci-session-authentication).
```console
prowler oci
```
You can add different profiles to the config file to scan different tenancies or regions. In order to scan a specific profile, use the `--profile` flag:
```console
prowler oci --profile <profile_name>
```
2. Instance Principal Authentication: when running Prowler on an OCI Compute instance, you can use Instance Principal authentication. For more details, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication#instance-principal-authentication).
```console
prowler oci --use-instance-principal
```
See more details about Oracle Cloud Authentication in [Oracle Cloud Authentication](/user-guide/providers/oci/authentication)
See more details about MongoDB Atlas Authentication in [MongoDB Atlas Authentication](../tutorials/mongodbatlas/authentication.md)
+1 -3
View File
@@ -1,6 +1,4 @@
---
title: 'Contact Us'
---
# Contact Us
For technical support or any type of inquiries, you are very welcome to:
@@ -1,14 +1,12 @@
---
title: 'AWS Provider'
---
# AWS Provider
In this page you can find all the details about [Amazon Web Services (AWS)](https://aws.amazon.com/) provider implementation in Prowler.
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](/user-guide/providers/aws/getting-started-aws).
By default, Prowler will audit just one account and organization settings per scan. To configure it, follow the [AWS getting started guide](../tutorials/aws/getting-started-aws.md).
## AWS Provider Classes Architecture
The AWS provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the AWS provider.
The AWS provider implementation follows the general [Provider structure](./provider.md). This section focuses on the AWS-specific implementation, highlighting how the generic provider concepts are realized for AWS in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the AWS provider.
### `AwsProvider` (Main Class)
@@ -35,7 +33,7 @@ The AWS provider implementation follows the general [Provider structure](/develo
### `AWSService` (Service Base Class)
- **Location:** [`prowler/providers/aws/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/lib/service/service.py)
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for AWS.
- **Purpose:** Abstract base class that all AWS service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for AWS.
- **Key AWS Responsibilities:**
- Receives an `AwsProvider` instance to access session, identity, and configuration.
- Manages clients for all services by regions.
@@ -54,12 +52,12 @@ The AWS provider implementation follows the general [Provider structure](/develo
## Specific Patterns in AWS Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the right now implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/aws/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services)
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all AWS services.
### AWS Service Common Patterns
@@ -76,12 +74,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in AWS Checks
The AWS checks pattern is described in [checks page](/developer-guide/checks). You can find all the right now implemented checks:
The AWS checks pattern is described in [checks page](./checks.md). You can find all the right now implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/aws/services/s3/s3_bucket_acl_prohibited/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/aws/services/s3/s3_bucket_acl_prohibited))
- In the [Prowler Hub](https://hub.prowler.com/). For a more human-readable view.
The best reference to understand how to implement a new check is following the [check creation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is following the [check creation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,14 +1,12 @@
---
title: 'Azure Provider'
---
# Azure Provider
In this page you can find all the details about [Microsoft Azure](https://azure.microsoft.com/) provider implementation in Prowler.
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](/user-guide/providers/azure/getting-started-azure).
By default, Prowler will audit all the subscriptions that it is able to list in the Microsoft Entra tenant, and tenant Entra ID service. To configure it, follow the [Azure getting started guide](../tutorials/azure/getting-started-azure.md).
## Azure Provider Classes Architecture
The Azure provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider). In next subsection you can find a list of the main classes of the Azure provider.
The Azure provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Azure-specific implementation, highlighting how the generic provider concepts are realized for Azure in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md). In next subsection you can find a list of the main classes of the Azure provider.
### `AzureProvider` (Main Class)
@@ -34,7 +32,7 @@ The Azure provider implementation follows the general [Provider structure](/deve
### `AzureService` (Service Base Class)
- **Location:** [`prowler/providers/azure/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/lib/service/service.py)
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Azure.
- **Purpose:** Abstract base class that all Azure service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Azure.
- **Key Azure Responsibilities:**
- Receives an `AzureProvider` instance to access session, identity, and configuration.
- Manages clients for all services by subscription.
@@ -52,12 +50,12 @@ The Azure provider implementation follows the general [Provider structure](/deve
## Specific Patterns in Azure Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/azure/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all Azure services.
### Azure Service Common Patterns
@@ -70,12 +68,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in Azure Checks
The Azure checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
The Azure checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/azure/services/storage/storage_blob_public_access_level_is_disabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is the [Azure check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is the [Azure check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,10 +1,8 @@
---
title: 'Check Metadata Guidelines'
---
# Check Metadata Guidelines
## Introduction
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) section.
This guide provides comprehensive guidelines for creating check metadata in Prowler. For basic information on check metadata structure, refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) section.
## Check Title Guidelines
@@ -1,6 +1,4 @@
---
title: 'Prowler Checks'
---
# Prowler Checks
This guide explains how to create new checks in Prowler.
@@ -14,7 +12,7 @@ The most common high level steps to create a new check are:
1. Prerequisites:
- Verify the check does not already exist by searching [Prowler Hub](https://hub.prowler.com) or checking `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>/`.
- Ensure required provider and service exist. If not, follow the [Provider](/developer-guide/provider) and [Service](/developer-guide/services) documentation to create them.
- Ensure required provider and service exist. If not, follow the [Provider](./provider.md) and [Service](./services.md) documentation to create them.
- Confirm the service has implemented all required methods and attributes for the check (in most cases, you will need to add or modify some methods in the service to get the data you need for the check).
2. Navigate to the service directory. The path should be as follows: `prowler/providers/<provider>/services/<service>`.
3. Create a check-specific folder. The path should follow this pattern: `prowler/providers/<provider>/services/<service>/<check_name_want_to_implement>`. Adhere to the [Naming Format for Checks](#naming-format-for-checks).
@@ -22,7 +20,7 @@ The most common high level steps to create a new check are:
5. Run the check locally to ensure it works as expected. For checking you can use the CLI in the next way:
- To ensure the check has been detected by Prowler: `poetry run python prowler-cli.py <provider> --list-checks | grep <check_name>`.
- To run the check, to find possible issues: `poetry run python prowler-cli.py <provider> --log-level ERROR --verbose --check <check_name>`.
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](/developer-guide/unit-testing) documentation.
6. Create comprehensive tests for the check that cover multiple scenarios including both PASS (compliant) and FAIL (non-compliant) cases. For detailed information about test structure and implementation guidelines, refer to the [Testing](./unit-testing.md) documentation.
7. If the check and its corresponding tests are working as expected, you can submit a PR to Prowler.
### Naming Format for Checks
@@ -41,8 +39,8 @@ The name components are:
Each check in Prowler follows a straightforward structure. Within the newly created folder, three files must be added to implement the check logic:
- `__init__.py` (empty file) Ensures Python treats the check folder as a package.
- `<check_name>.py` (code file) Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](/developer-guide/checks#prowlers-check-code-structure) for more information.
- `<check_name>.metadata.json` (metadata file) Defines the check's metadata for contextual information. Please refer to the [check metadata](/developer-guide/checks#metadata-structure-for-prowler-checks) for more information.
- `<check_name>.py` (code file) Contains the check logic, following the prescribed format. Please refer to the [prowler's check code structure](./checks.md#prowlers-check-code-structure) for more information.
- `<check_name>.metadata.json` (metadata file) Defines the check's metadata for contextual information. Please refer to the [check metadata](./checks.md#metadata-structure-for-prowler-checks) for more information.
## Prowler's Check Code Structure
@@ -52,10 +50,9 @@ Below the code for a generic check is presented. It is strongly recommended to c
Report fields are the most dependent on the provider, consult the `CheckReport<Provider>` class for more information on what can be included in the report [here](https://github.com/prowler-cloud/prowler/blob/master/prowler/lib/check/models.py).
<Note>
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
???+ note
Legacy providers (AWS, Azure, GCP, Kubernetes) follow the `Check_Report_<Provider>` naming convention. This is not recommended for current instances. Newer providers adopt the `CheckReport<Provider>` naming convention. Learn more at [Prowler Code](https://github.com/prowler-cloud/prowler/tree/master/prowler/lib/check/models.py).
</Note>
```python title="Generic Check Class"
# Required Imports
# Import the base Check class and the provider-specific CheckReport class
@@ -216,7 +213,7 @@ Each check **must** populate the report with an unique identifier for the audite
### Configurable Checks in Prowler
See [Configurable Checks](/developer-guide/configurable-checks) for detailed information on making checks configurable using the `audit_config` object and configuration file.
See [Configurable Checks](./configurable-checks.md) for detailed information on making checks configurable using the `audit_config` object and configuration file.
## Metadata Structure for Prowler Checks
@@ -276,17 +273,16 @@ The `CheckTitle` field must be plain text, clearly and succinctly define **the b
**Always write the `CheckTitle` to describe the *PASS* case**, the desired secure or compliant state of the resource(s). This helps ensure that findings are easy to interpret and that the title always reflects the best practice being met.
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](/developer-guide/check-metadata-guidelines#check-title-guidelines).
For detailed guidelines on writing effective check titles, including how to determine singular vs. plural scope and common mistakes to avoid, see [Check Title Guidelines](./check-metadata-guidelines.md#check-title-guidelines).
#### CheckType
<Warning>
This field is only applicable to the AWS provider.
???+ warning
This field is only applicable to the AWS provider.
</Warning>
It follows the [AWS Security Hub Types](https://docs.aws.amazon.com/securityhub/latest/userguide/asff-required-attributes.html#Types) format using the pattern `namespace/category/classifier`.
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](/developer-guide/check-metadata-guidelines#check-type-guidelines-aws-only).
For the complete AWS Security Hub selection guidelines, see [Check Type Guidelines](./check-metadata-guidelines.md#check-type-guidelines-aws-only).
#### ServiceName
@@ -318,13 +314,13 @@ The type of resource being audited. This field helps categorize and organize fin
A concise, natural language explanation that **clearly describes what the finding means**, focusing on clarity and context rather than technical implementation details. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. This field is limited to maximum 400 characters.
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](/developer-guide/check-metadata-guidelines#description-guidelines).
For detailed writing guidelines and common mistakes to avoid, see [Description Guidelines](./check-metadata-guidelines.md#description-guidelines).
#### Risk
A clear, natural language explanation of **why this finding poses a cybersecurity risk**. Focus on how it may impact confidentiality, integrity, or availability. If those do not apply, describe any relevant operational or financial risks. Use simple paragraphs with line breaks if needed, but avoid sections, code blocks, or complex formatting. Limit your explanation to 400 characters.
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](/developer-guide/check-metadata-guidelines#risk-guidelines).
For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelines](./check-metadata-guidelines.md#risk-guidelines).
#### RelatedUrl
@@ -332,10 +328,9 @@ For detailed writing guidelines and common mistakes to avoid, see [Risk Guidelin
#### AdditionalURLs
<Warning>
URLs must be valid and not repeated.
???+ warning
URLs must be valid and not repeated.
</Warning>
A list of official documentation URLs for further reading. These should be authoritative sources that provide additional context, best practices, or detailed information about the security control being checked. Prefer official provider documentation, security standards, or well-established security resources. Avoid third-party blogs or unofficial sources unless they are highly reputable and directly relevant.
#### Remediation
@@ -350,17 +345,17 @@ Provides both code examples and best practice recommendations for addressing the
- **Terraform**: HashiCorp Configuration Language (HCL) code with an example of a compliant configuration.
- **Other**: Manual steps through web interfaces or other tools to make the finding compliant.
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](/developer-guide/check-metadata-guidelines#remediation-code-guidelines).
For detailed guidelines on writing remediation code, see [Remediation Code Guidelines](./check-metadata-guidelines.md#remediation-code-guidelines).
- **Recommendation**
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](/developer-guide/check-metadata-guidelines#recommendation-guidelines).
- **Text**: Generic best practice guidance in natural language using Markdown format (maximum 400 characters). For writing guidelines, see [Recommendation Guidelines](./check-metadata-guidelines.md#recommendation-guidelines).
- **Url**: [Prowler Hub URL](https://hub.prowler.com/) of the check. This URL is always composed by `https://hub.prowler.com/check/<check_id>`.
#### Categories
One or more functional groupings used for execution filtering (e.g., `internet-exposed`). You can define new categories just by adding to this field.
For the complete list of available categories, see [Categories Guidelines](/developer-guide/check-metadata-guidelines#categories-guidelines).
For the complete list of available categories, see [Categories Guidelines](./check-metadata-guidelines.md#categories-guidelines).
#### DependsOn
@@ -1,6 +1,4 @@
---
title: 'Configurable Checks in Prowler'
---
# Configurable Checks in Prowler
Prowler empowers users to extend and adapt cloud security coverage by making checks configurable through the use of the `audit_config` object. This approach enables customization of checks to meet specific requirements through a configuration file.
@@ -43,6 +41,6 @@ When adding a new configurable check to Prowler, update the following files:
- **Test Fixtures:** If tests depend on this configuration, add the variable to `tests/config/fixtures/config.yaml`.
- **Documentation:** Document the new variable in the list of configurable checks in `docs/tutorials/configuration_file.md`.
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](/user-guide/cli/tutorials/configuration_file).
For a complete list of checks that already support configuration, see the [Configuration File Tutorial](../tutorials/configuration_file.md).
This approach ensures that checks are easily configurable, making Prowler highly adaptable to different environments and requirements.
@@ -1,6 +1,4 @@
---
title: 'Debugging in Prowler'
---
# Debugging in Prowler
Debugging in Prowler simplifies the development process, allowing developers to efficiently inspect and resolve unexpected issues during execution.
+28
View File
@@ -0,0 +1,28 @@
## Contributing to Documentation
Prowler documentation is built using `mkdocs`, allowing contributors to easily add or enhance documentation.
### Installation and Setup
Install all necessary dependencies using: `poetry install --with docs`.
1. Install `mkdocs` using your preferred package manager.
2. Running the Documentation Locally
Navigate to the `prowler` repository folder.
Start the local documentation server by running: `mkdocs serve`.
Open `http://localhost:8000` in your browser to view live updates.
3. Making Documentation Changes
Make all needed changes to docs or add new documents. Edit existing Markdown (.md) files inside `prowler/docs`.
To add new sections or files, update the `mkdocs.yaml` file located in the root directory of Prowlers repository.
4. Submitting Changes
Once documentation updates are complete:
Submit a pull request for review.
The Prowler team will assess and merge contributions.
Your efforts help improve Prowler documentation—thank you for contributing!
-42
View File
@@ -1,42 +0,0 @@
---
title: 'Contributing to Documentation'
---
Prowler documentation is built using [Mintlify](https://www.mintlify.com/docs), allowing contributors to easily add or enhance documentation.
## Installation and Setup
<Steps>
<Step title="Install Mintlify CLI">
```bash
npm i -g mint
```
For detailed instructions, check the [Mintlify documentation](https://www.mintlify.com/docs/installation).
</Step>
<Step title="Preview Documentation Locally">
Start the local development server to preview changes in real-time.
```bash
mint dev
```
A local preview of your documentation will be available at http://localhost:3000
</Step>
<Step title="Make Documentation Changes">
Edit existing Markdown (.mdx) files inside the `docs` directory or add new documents.
For reference about formatting, check the [Mintlify documentation](https://www.mintlify.com/docs/create/text).
To add new sections or files, update the [`docs/docs.json`](https://github.com/prowler-cloud/prowler/blob/master/docs/docs.json) file to include them in the navigation.
</Step>
<Step title="Submit Changes">
Once documentation updates are complete, submit a pull request for review.
The Prowler team will assess and merge contributions.
</Step>
</Steps>
Your efforts help improve Prowler documentation—thank you for contributing!
@@ -1,14 +1,12 @@
---
title: 'Google Cloud Provider'
---
# Google Cloud Provider
This page details the [Google Cloud Platform (GCP)](https://cloud.google.com/) provider implementation in Prowler.
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](/user-guide/providers/gcp/getting-started-gcp).
By default, Prowler will audit all the GCP projects that the authenticated identity can access. To configure it, follow the [GCP getting started guide](../tutorials/gcp/getting-started-gcp.md).
## GCP Provider Classes Architecture
The GCP provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
The GCP provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GCP-specific implementation, highlighting how the generic provider concepts are realized for GCP in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
### Main Class
@@ -34,7 +32,7 @@ The GCP provider implementation follows the general [Provider structure](/develo
### `GCPService` (Service Base Class)
- **Location:** [`prowler/providers/gcp/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/lib/service/service.py)
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for GCP.
- **Purpose:** Abstract base class that all GCP service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for GCP.
- **Key GCP Responsibilities:**
- Receives a `GcpProvider` instance to access session, identity, and configuration.
- Manages clients for all services by project.
@@ -97,12 +95,12 @@ def _get_instances(self):
## Specific Patterns in GCP Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/gcp/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other services already implemented as reference. In next subsection you can find a list of common patterns that are used accross all GCP services.
### GCP Service Common Patterns
@@ -119,12 +117,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in GCP Checks
The GCP checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks:
The GCP checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/gcp/services/iam/iam_sa_user_managed_key_unused))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](/developer-guide/checks#creating-a-check) and taking other similar checks as reference.
The best reference to understand how to implement a new check is following the [GCP check implementation documentation](./checks.md#creating-a-check) and taking other similar checks as reference.
### Check Report Class
@@ -1,14 +1,12 @@
---
title: 'GitHub Provider'
---
# GitHub Provider
This page details the [GitHub](https://github.com/) provider implementation in Prowler.
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](/user-guide/providers/github/getting-started-github).
By default, Prowler will audit the GitHub account - scanning all repositories, organizations, and applications that your configured credentials can access. To configure it, follow the [GitHub getting started guide](../tutorials/github/getting-started-github.md).
## GitHub Provider Classes Architecture
The GitHub provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
The GitHub provider implementation follows the general [Provider structure](./provider.md). This section focuses on the GitHub-specific implementation, highlighting how the generic provider concepts are realized for GitHub in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
### `GithubProvider` (Main Class)
@@ -50,12 +48,12 @@ The GitHub provider implementation follows the general [Provider structure](/dev
## Specific Patterns in GitHub Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/github/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
### GitHub Service Common Patterns
@@ -68,12 +66,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in GitHub Checks
The GitHub checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
The GitHub checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/github/services/repository/repository_secret_scanning_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/github/services/repository/repository_secret_scanning_enabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
The best reference to understand how to implement a new check is the [GitHub check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
### Check Report Class
@@ -0,0 +1,3 @@
# Integration Tests
Coming soon ...
@@ -1,5 +0,0 @@
---
title: 'Integration Tests'
---
Coming soon ...
@@ -1,6 +1,4 @@
---
title: 'Creating a New Integration'
---
# Creating a New Integration
## Introduction
@@ -153,7 +151,7 @@ Refer to the [Prowler Developer Guide](https://docs.prowler.com/projects/prowler
# More properties and methods
```
* Test Connection Method:
* Validating Credentials or Tokens
@@ -1,6 +1,4 @@
---
title: 'Introduction to developing in Prowler'
---
# Introduction to developing in Prowler
Extending Prowler
@@ -50,12 +48,11 @@ poetry install --with dev
eval $(poetry env activate)
```
<Warning>
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
???+ important
Starting from Poetry v2.0.0, `poetry shell` has been deprecated in favor of `poetry env activate`.
If your poetry version is below 2.0.0 you must keep using `poetry shell` to activate your environment.
In case you have any doubts, consult the [Poetry environment activation guide](https://python-poetry.org/docs/managing-environments/#activating-the-environment).
</Warning>
## Contributing to Prowler
### Ways to Contribute
@@ -67,24 +64,24 @@ Here are some ideas for collaborating with Prowler:
2. **Expand Prowler's Capabilities**: Prowler is constantly evolving, and you can be a part of its growth. Whether you are adding checks, supporting new services, or introducing integrations, your contributions help improve the tool for everyone. Here is how you can get involved:
- **Adding New Checks**
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](/developer-guide/checks).
Want to improve Prowler's detection capabilities for your favorite cloud provider? You can contribute by writing new checks. To get started, follow the [create a new check guide](./checks.md).
- **Adding New Services**
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](/developer-guide/services). Do not forget to include relevant checks to validate functionality.
One key service for your favorite cloud provider is missing? Add it to Prowler! To add a new service, check out the [create a new service guide](./services.md). Do not forget to include relevant checks to validate functionality.
- **Adding New Providers**
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](/developer-guide/provider). This typically involves setting up new services and checks to ensure compatibility.
If you would like to extend Prowler to work with a new cloud provider, follow the [create a new provider guide](./provider.md). This typically involves setting up new services and checks to ensure compatibility.
- **Adding New Output Formats**
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](/developer-guide/outputs).
Want to tailor how results are displayed or exported? You can add custom output formats by following the [create a new output format guide](./outputs.md).
- **Adding New Integrations**
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](/developer-guide/integrations).
Prowler can work with other tools and platforms through integrations. If you would like to add one, see the [create a new integration guide](./integrations.md).
- **Proposing or Implementing Features**
Got an idea to make Prowler better? Whether it is a brand-new feature or an enhancement to an existing one, you are welcome to propose it or help implement community-requested improvements.
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](/developer-guide/documentation).
3. **Improve Documentation**: Help make Prowler more accessible by enhancing our documentation, fixing typos, or adding examples/tutorials. See the tutorial of how we write our documentation [here](./documentation.md).
4. **Bug Fixes**: If you find any issues or bugs, you can report them in the [GitHub Issues](https://github.com/prowler-cloud/prowler/issues) page and if you want you can also fix them.
@@ -108,10 +105,9 @@ pre-commit installed at .git/hooks/pre-commit
Before merging pull requests, several automated checks and utilities ensure code security and updated dependencies:
<Note>
These should have been already installed if `poetry install --with dev` was already run.
???+ note
These should have been already installed if `poetry install --with dev` was already run.
</Note>
- [`bandit`](https://pypi.org/project/bandit/) for code security review.
- [`safety`](https://pypi.org/project/safety/) and [`dependabot`](https://github.com/features/security) for dependencies.
- [`hadolint`](https://github.com/hadolint/hadolint) and [`dockle`](https://github.com/goodwithtech/dockle) for container security.
@@ -127,10 +123,9 @@ All dependencies are listed in the `pyproject.toml` file.
For proper code documentation, refer to the following and follow the code documentation practices presented there: [Google Python Style Guide - Comments and Docstrings](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings).
<Note>
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
???+ note
If you encounter issues when committing to the Prowler repository, use the `--no-verify` flag with the `git commit` command.
</Note>
### Repository Folder Structure
Understanding the layout of the Prowler codebase will help you quickly find where to add new features, checks, or integrations. The following is a high-level overview from the root of the repository:
@@ -178,4 +173,4 @@ To test Prowler from a specific branch (for example, to try out changes from a p
pipx install "git+https://github.com/prowler-cloud/prowler.git@branch-name"
```
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
Replace `branch-name` with the name of the branch you want to test. This will install Prowler in an isolated environment, allowing you to try out the changes safely.
@@ -1,14 +1,12 @@
---
title: 'Kubernetes Provider'
---
# Kubernetes Provider
This page details the [Kubernetes](https://kubernetes.io/) provider implementation in Prowler.
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](/user-guide/providers/kubernetes/in-cluster) or [Non In-Cluster Execution](/user-guide/providers/kubernetes/outside-cluster) guides.
By default, Prowler will audit all namespaces in the Kubernetes cluster accessible by the configured context. To configure it, see the [In-Cluster Execution](../tutorials/kubernetes/in-cluster.md) or [Non In-Cluster Execution](../tutorials/kubernetes/outside-cluster.md) guides.
## Kubernetes Provider Classes Architecture
The Kubernetes provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
The Kubernetes provider implementation follows the general [Provider structure](./provider.md). This section focuses on the Kubernetes-specific implementation, highlighting how the generic provider concepts are realized for Kubernetes in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
### `KubernetesProvider` (Main Class)
@@ -33,7 +31,7 @@ The Kubernetes provider implementation follows the general [Provider structure](
### `KubernetesService` (Service Base Class)
- **Location:** [`prowler/providers/kubernetes/lib/service/service.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/lib/service/service.py)
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](/developer-guide/services#service-base-class)) specifically for Kubernetes.
- **Purpose:** Abstract base class that all Kubernetes service-specific classes inherit from. This implements the generic service pattern (described in [service page](./services.md#service-base-class)) specifically for Kubernetes.
- **Key Kubernetes Responsibilities:**
- Receives a `KubernetesProvider` instance to access session, identity, and configuration.
- Manages the Kubernetes API client and context.
@@ -52,12 +50,12 @@ The Kubernetes provider implementation follows the general [Provider structure](
## Specific Patterns in Kubernetes Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/kubernetes/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and taking other already implemented services as reference.
The best reference to understand how to implement a new service is following the [service implementation documentation](./services.md#adding-a-new-service) and taking other already implemented services as reference.
### Kubernetes Service Common Patterns
@@ -71,12 +69,12 @@ The best reference to understand how to implement a new service is following the
## Specific Patterns in Kubernetes Checks
The Kubernetes checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
The Kubernetes checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/kubernetes/services/rbac/rbac_minimize_wildcard_use_roles))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](/developer-guide/checks#creating-a-check) and taking other checks as reference.
The best reference to understand how to implement a new check is following the [Kubernetes check implementation documentation](./checks.md#creating-a-check) and taking other checks as reference.
### Check Report Class
@@ -1,6 +1,4 @@
---
title: 'Extending Prowler Lighthouse AI'
---
# Extending Prowler Lighthouse AI
This guide helps developers customize and extend Prowler Lighthouse AI by adding or modifying AI agents.
@@ -17,10 +15,9 @@ AI agents fall into two main categories:
Prowler Lighthouse AI is an autonomous agent - selecting the right tool(s) based on the users query.
<Note>
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
???+ note
To learn more about AI agents, read [Anthropic's blog post on building effective agents](https://www.anthropic.com/engineering/building-effective-agents).
</Note>
### LLM Dependency
The autonomous nature of agents depends on the underlying LLM. Autonomous agents using identical system prompts and tools but powered by different LLM providers might approach user queries differently. Agent with one LLM might solve a problem efficiently, while with another it might take a different route or fail entirely.
@@ -33,7 +30,7 @@ Prowler Lighthouse AI uses a multi-agent architecture orchestrated by the [Langg
### Architecture Components
<img src="/images/prowler-app/lighthouse-architecture.png" alt="Prowler Lighthouse architecture" />
<img src="../../tutorials/img/lighthouse-architecture.png" alt="Prowler Lighthouse architecture">
Prowler Lighthouse AI integrates with the NextJS application:
@@ -70,10 +67,9 @@ Modifying the supervisor prompt allows you to:
- Modify task delegation to specialized agents
- Set up guardrails (query types to answer or decline)
<Note>
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
???+ note
The supervisor agent should not have its own tools. This design keeps the system modular and maintainable.
</Note>
### How to Create New Specialized Agents
The supervisor agent and all specialized agents are defined in the `route.ts` file. The supervisor agent uses [langgraph-supervisor](https://www.npmjs.com/package/@langchain/langgraph-supervisor), while other agents use the prebuilt [create-react-agent](https://langchain-ai.github.io/langgraphjs/how-tos/create-react-agent/).
@@ -81,16 +77,14 @@ The supervisor agent and all specialized agents are defined in the `route.ts` fi
To add new capabilities or all Lighthouse AI to interact with other APIs, create additional specialized agents:
1. First determine what the new agent would do. Create a detailed prompt defining the agent's purpose and capabilities. You can see an example from [here](https://github.com/prowler-cloud/prowler/blob/master/ui/lib/lighthouse/prompts.ts#L359-L385).
<Note>
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
???+ note
Ensure that the new agent's capabilities don't collide with existing agents. For example, if there's already a *findings_agent* that talks to findings APIs don't create a new agent to do the same.
</Note>
2. Create necessary tools for the agents to access specific data or perform actions. A tool is a specialized function that extends the capabilities of LLM by allowing it to access external data or APIs. A tool is triggered by LLM based on the description of the tool and the user's query.
For example, the description of `getScanTool` is "Fetches detailed information about a specific scan by its ID." If the description doesn't convey what the tool is capable of doing, LLM will not invoke the function. If the description of `getScanTool` was set to something random or not set at all, LLM will not answer queries like "Give me the critical issues from the scan ID xxxxxxxxxxxxxxx"
<Note>
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
???+ note
Ensure that one tool is added to one agent only. Adding tools is optional. There can be agents with no tools at all.
</Note>
3. Use the `createReactAgent` function to define a new agent. For example, the rolesAgent name is "roles_agent" and has access to call tools "*getRolesTool*" and "*getRoleTool*"
```js
const rolesAgent = createReactAgent({
@@ -1,14 +1,12 @@
---
title: 'LLM Provider'
---
# LLM Provider
This page details the [Large Language Model (LLM)](https://en.wikipedia.org/wiki/Large_language_model) provider implementation in Prowler.
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](/user-guide/providers/llm/getting-started-llm).
The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](../tutorials/llm/getting-started-llm.md).
## LLM Provider Classes Architecture
The LLM provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
The LLM provider implementation follows the general [Provider structure](./provider.md). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
### Main Class
@@ -1,10 +1,8 @@
---
title: 'Microsoft 365 (M365) Provider'
---
# Microsoft 365 (M365) Provider
This page details the [Microsoft 365 (M365)](https://www.microsoft.com/en-us/microsoft-365) provider implementation in Prowler.
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](/user-guide/providers/microsoft365/getting-started-m365).
By default, Prowler will audit the Microsoft Entra ID tenant and its supported services. To configure it, follow the [M365 getting started guide](../tutorials/microsoft365/getting-started-m365.md).
---
@@ -17,14 +15,14 @@ By default, Prowler will audit the Microsoft Entra ID tenant and its supported s
- **Required modules:**
- [ExchangeOnlineManagement](https://www.powershellgallery.com/packages/ExchangeOnlineManagement/3.6.0) (≥ 3.6.0)
- [MicrosoftTeams](https://www.powershellgallery.com/packages/MicrosoftTeams/6.6.0) (≥ 6.6.0)
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](/user-guide/providers/microsoft365/authentication#supported-powershell-versions) and [Needed PowerShell Modules](/user-guide/providers/microsoft365/authentication#required-powershell-modules).
- For more details and troubleshooting, see [Use of PowerShell in M365](/user-guide/providers/microsoft365/use-of-powershell).
- If you use Prowler Cloud or the official containers, PowerShell is pre-installed. For local or pip installations, you must install PowerShell and the modules yourself. See [Authentication: Supported PowerShell Versions](../tutorials/microsoft365/authentication.md#supported-powershell-versions) and [Needed PowerShell Modules](../tutorials/microsoft365/authentication.md#required-powershell-modules).
- For more details and troubleshooting, see [Use of PowerShell in M365](../tutorials/microsoft365/use-of-powershell.md).
---
## M365 Provider Classes Architecture
The M365 provider implementation follows the general [Provider structure](/developer-guide/provider). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](/developer-guide/provider).
The M365 provider implementation follows the general [Provider structure](./provider.md). This section focuses on the M365-specific implementation, highlighting how the generic provider concepts are realized for M365 in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
### `M365Provider` (Main Class)
@@ -75,12 +73,12 @@ The M365 provider implementation follows the general [Provider structure](/devel
## Specific Patterns in M365 Services
The generic service pattern is described in [service page](/developer-guide/services#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
The generic service pattern is described in [service page](./services.md#service-structure-and-initialisation). You can find all the currently implemented services in the following locations:
- Directly in the code, in location [`prowler/providers/m365/services/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services)
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new service is by following the [service implementation documentation](/developer-guide/services#adding-a-new-service) and by taking other already implemented services as reference.
The best reference to understand how to implement a new service is by following the [service implementation documentation](./services.md#adding-a-new-service) and by taking other already implemented services as reference.
### M365 Service Common Patterns
@@ -94,12 +92,12 @@ The best reference to understand how to implement a new service is by following
## Specific Patterns in M365 Checks
The M365 checks pattern is described in [checks page](/developer-guide/checks). You can find all the currently implemented checks in:
The M365 checks pattern is described in [checks page](./checks.md). You can find all the currently implemented checks in:
- Directly in the code, within each service folder, each check has its own folder named after the name of the check. (e.g. [`prowler/providers/m365/services/entra/entra_users_mfa_enabled/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers/m365/services/entra/entra_users_mfa_enabled))
- In the [Prowler Hub](https://hub.prowler.com/) for a more human-readable view.
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](/developer-guide/checks#creating-a-check) and by taking other checks as reference.
The best reference to understand how to implement a new check is following the [M365 check implementation documentation](./checks.md#creating-a-check) and by taking other checks as reference.
### Check Report Class
@@ -1,6 +1,4 @@
---
title: 'Create a Custom Output Format'
---
# Create a Custom Output Format
## Introduction
+172
View File
@@ -0,0 +1,172 @@
# Prowler Providers
## Introduction
Providers form the backbone of Prowler, enabling security assessments across various cloud environments.
A provider is any platform or service that offers resources, data, or functionality that can be audited for security and compliance. This includes:
- Cloud Infrastructure Providers (like Amazon Web Services, Microsoft Azure, and Google Cloud)
- Software as a Service (SaaS) Platforms (like Microsoft 365)
- Development Platforms (like GitHub)
- Container Orchestration Platforms (like Kubernetes)
- Database-as-a-Service Platforms (like MongoDB Atlas)
For providers supported by Prowler, refer to [Prowler Hub](https://hub.prowler.com/).
???+ important
There are some custom providers added by the community, like [NHN Cloud](https://www.nhncloud.com/), that are not maintained by the Prowler team, but can be used in the Prowler CLI. They can be checked directly at the [Prowler GitHub repository](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
## Adding a New Provider
To integrate an unsupported Prowler provider and implement its security checks, create a dedicated folder for all related files (e.g., services, checks)."
This folder must be placed within [`prowler/providers/<new_provider_name>/`](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers).
Within this folder the following folders are also to be created:
- `lib` Stores additional utility functions and core files required by every provider. The following files and subfolders are commonly found in every provider's `lib` folder:
- `service/service.py` Provides a generic service class to be inherited by all services.
- `arguments/arguments.py` Handles provider-specific argument parsing.
- `mutelist/mutelist.py` Manages the mutelist functionality for the provider.
- `services` Stores all [services](./services.md) that the provider offers and want to be audited by [Prowler checks](./checks.md).
- `__init__.py` (empty) Ensures Python recognizes this folder as a package.
- `<new_provider_name>_provider.py` Defines authentication logic, configurations, and other provider-specific data.
- `models.py` Contains necessary models for the new provider.
By adhering to this structure, Prowler can effectively support services and security checks for additional providers.
???+ important
If your new provider requires a Python library (such as an official SDK or API client) to connect to its services, make sure to add it as a dependency in the `pyproject.toml` file. This ensures that all contributors and users have the necessary packages installed when working with your provider.
## Provider Structure in Prowler
Prowler's provider architecture is designed to facilitate security audits through a generic service tailored to each provider. This is accomplished by passing the necessary parameters to the constructor, which initializes all required session values.
### Base Class
All Prowler providers inherit from the same base class located in [`prowler/providers/common/provider.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/common/provider.py). It is an [abstract base class](https://docs.python.org/3/library/abc.html) that defines the interface for all provider classes.
### Provider Class
#### Provider Implementation Guidance
Given the complexity and variability of providers, use existing provider implementations as templates when developing new integrations.
- [AWS](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/aws_provider.py)
- [GCP](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/gcp_provider.py)
- [Azure](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/azure_provider.py)
- [Kubernetes](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/kubernetes_provider.py)
- [Microsoft365](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/microsoft365/microsoft365_provider.py)
- [GitHub](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/github/github_provider.py)
- [MongoDB Atlas](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/mongodbatlas/mongodbatlas_provider.py)
### Basic Provider Implementation: Pseudocode Example
To simplify understanding, the following pseudocode outlines the fundamental structure of a provider, including library imports necessary for authentication.
```python title="Provider Example Class"
# Library Imports for Authentication
# When implementing authentication for a provider, import the required libraries.
from prowler.config.config import load_and_validate_config_file
from prowler.lib.logger import logger
from prowler.lib.mutelist.mutelist import parse_mutelist_file
from prowler.lib.utils.utils import print_boxes
from prowler.providers.common.models import Audit_Metadata
from prowler.providers.common.provider import Provider
from prowler.providers.<new_provider_name>.models import (
# All provider models needed.
ProviderSessionModel,
ProviderIdentityModel,
ProviderOutputOptionsModel
)
class NewProvider(Provider):
# All properties from the class, some of which are properties in the base class.
_type: str = "<provider_name>"
_session: <ProviderSessionModel>
_identity: <ProviderIdentityModel>
_audit_config: dict
_output_options: ProviderOutputOptionsModel
_mutelist: dict
audit_metadata: Audit_Metadata
def __init__(self, arguments):
"""
Initializes the NewProvider instance.
Args:
arguments (dict): A dictionary containing configuration arguments.
"""
logger.info("Setting <NewProviderName> provider ...")
# Initializing the Provider Session
# Steps:
# - Retrieve Account Information
# - Extract relevant account identifiers (subscriptions, projects, or other service references) from the provided arguments.
# Establish a Session
# Use the method enforced by the parent class to set up the session:
self._session = self.setup_session(credentials_file)
# Define Provider Identity
# Assign the identity class, typically provided by the Python provider library:
self._identity = <ProviderIdentityModel>()
# Configure the Provider
# Set the provider-specific configuration.
self._audit_config = load_and_validate_config_file(
self._type, arguments.config_file
)
# All the enforced properties by the parent class.
@property
def identity(self):
return self._identity
@property
def session(self):
return self._session
@property
def type(self):
return self._type
@property
def audit_config(self):
return self._audit_config
@property
def output_options(self):
return self._output_options
def setup_session(self, <all_needed_for_auth>):
"""
Sets up the Provider session.
Args:
<all_needed_for_auth> Can include all necessary arguments to set up the session
Returns:
Credentials necessary to communicate with the provider.
"""
pass
"""
This method is enforced by parent class and is used to print all relevant
information during the prowler execution as a header of execution.
Displaying Account Information with Color Formatting. In Prowler, Account IDs, usernames, and other identifiers are typically displayed using color formatting provided by the colorama module (Fore).
"""
def print_credentials(self):
pass
```
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,4 @@
---
title: 'Renaming Checks in Prowler'
---
# Renaming Checks in Prowler
To rename a check in Prowler, follow these steps when aligning with Check ID structure, fixing typos, or updating check logic that requires a new name.
@@ -1,6 +1,4 @@
---
title: 'Creating a New Security Compliance Framework in Prowler'
---
# Creating a New Security Compliance Framework in Prowler
## Introduction
+250
View File
@@ -0,0 +1,250 @@
# Prowler Services
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](./provider.md).
???+note
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](./provider.md) documentation to create it from scratch.
## Introduction
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](./provider.md), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](./checks.md) to generate the security findings.
## Adding a New Service
To create a new service, a new folder must be created inside the specific provider following this pattern: `prowler/providers/<provider>/services/<new_service_name>/`.
Within this folder the following files are also to be created:
- `__init__.py` (empty) Ensures Python recognizes this folder as a package.
- `<new_service_name>_service.py` Contains all the logic and API calls of the service.
- `<new_service_name>_client_.py` Contains the initialization of the freshly created service's class so that the checks can use it.
Once the files are create, you can check that the service has been created by running the following command: `poetry run python prowler-cli.py <provider> --list-services | grep <new_service_name>`.
## Service Structure and Initialisation
The Prowler's service structure is as outlined below. To initialise it, just import the service client in a check.
### Service Base Class
All Prowler provider service should inherit from a common base class to avoid code duplication. This base class handles initialization and storage of functions and objects needed across services. The exact implementation depends on the provider's API requirements, but the following are the most common responsibilities:
- Initialize/store clients to interact with the provider's API.
- Store the audit and fixer configuration.
- Implement threading logic where applicable.
For reference, the base classes for each provider can be checked here:
- [AWS Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/lib/service/service.py)
- [GCP Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/lib/service/service.py)
- [Azure Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/lib/service/service.py)
- [Kubernetes Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/lib/service/service.py)
- [M365 Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/m365/lib/service/service.py)
- [GitHub Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/github/lib/service/service.py)
### Service Class
Due to the complexity and differences across provider APIs, the following example demonstrates best practices for structuring a service in Prowler.
File `<new_service_name>_service.py`:
```python title="Example Service Class"
from datetime import datetime
from typing import Optional
# To use the Pydantic's BaseModel.
from pydantic import BaseModel
# Prowler logging library.
from prowler.lib.logger import logger
# Provider parent class.
from prowler.providers.<provider>.lib.service.service import ServiceParentClass
# Create a class for the Service.
class <Service>(ServiceParentClass):
def __init__(self, provider: Provider):
"""Initialize the Service Class
Args:
provider: Prowler Provider object.
"""
# Call Service Parent Class __init__.
# The __class__.__name__ is used to obtain it automatically.
# From the Service Class name, but a custom one can be passed.
# String in case the provider's API service name is different.
super().__init__(__class__.__name__, provider)
# Create an empty dictionary of items to be gathered, using the unique ID as the dictionary's key, e.g., instances.
self.<items> = {}
# If parallelization can be carried out by regions or locations, the function __threading_call__ to be used must be implemented in the Service Parent Class.
# If it is not implemented, you can make it in a sequential way, just calling the function.
self.__threading_call__(self.__describe_<items>__)
# If it is needed you can create another function to retrieve more data from the items.
# Here we are using the second parameter of the __threading_call__ function to create one thread per item.
# You can also make it sequential without using the __threading_call__ function iterating over the items inside the function.
self.__threading_call__(self.__describe_<item>__, self.<items>.values())
# In case of use the __threading_call__ function, you have to pass the regional_client to the function, as a parameter.
def __describe_<items>__(self, regional_client):
"""Get all <items> and store in the self.<items> dictionary
Args:
regional_client: Regional client object.
"""
logger.info("<Service> - Describing <Items>...")
# A try-except block must be created in each function.
try:
# If pagination is supported by the provider, is always better to use it, call to the provider API to retrieve the desired data.
describe_<items>_paginator = regional_client.get_paginator("describe_<items>")
# Paginator to get every item.
for page in describe_<items>_paginator.paginate():
# Another try-except within the for loop to continue iterating in case something unexpected happens.
try:
for <item> in page["<Items>"]:
# Adding Retrieved Resources to the Object
# Once the resource has been retrieved, it must be included in the previously created object to ensure proper data handling within the service.
self.<items>[<item_unique_id>] =
<Item>(
arn=stack["<item_arn>"],
name=stack["<item_name>"],
tags=stack.get("Tags", []),
region=regional_client.region,
)
except Exception as error:
logger.error(
f"{<provider_specific_field>} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
# Logging Errors in Exception Handling
# When handling exceptions, use the following approach to log errors appropriately based on the cloud provider being used:
except Exception as error:
# Depending on each provider we can must use different fields in the logger, e.g.: AWS: regional_client.region or self.region, GCP: project_id and location, Azure: subscription
logger.error(
f"{<provider_specific_field>} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
def __describe_<item>__(self, item: <Item>):
"""Get details for a <item>
Args:
item: Item object.
"""
logger.info("<Service> - Describing <Item> to get specific details...")
# A try-except block must be created in each function.
try:
<item>_details = self.regional_clients[<item>.region].describe_<item>(
<Attribute>=<item>.name
)
# E.g., check if item is Public. This case is important: if values are being retrieved from a dictionary, the function "dict.get()" must be used with a default value in case this value is not present.
<item>.public = <item>_details.get("Public", False)
except Exception as error:
# Fields for logging errors with relevant item information, e.g.: AWS: <item>.region, GCP: <item>.project_id, Azure: <item>.region
logger.error(
f"{<item>.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
```
???+note
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
#### Resource Models
Resource models define structured classes used within services to store and process data extracted from API calls. They are defined in the same file as the service class, but outside of the class, usually at the bottom of the file.
Prowler leverages Pydantic's [BaseModel](https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel) to enforce data validation.
```python title="Service Model"
# Implementation Approach
# Each service class should include custom model classes using Pydantic's BaseModel for the resources being audited.
class <Item>(BaseModel):
"""<Item> holds a <Service> <Item>"""
id: str
"""<Items>[].id"""
name: str
"""<Items>[].name"""
region: str
"""<Items>[].region"""
public: bool
"""<Items>[].public"""
# Optional attributes can be created set to None by default.
tags: Optional[list]
"""<Items>[].tags"""
```
#### Service Attributes
*Optimized Data Storage with Python Dictionaries*
Each group of resources within a service should be structured as a Python [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries) to enable efficient lookups. The dictionary lookup operation has [O(1) complexity](https://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common_functions), and lookups are constantly executed.
*Assigning Unique Identifiers*
Each dictionary key must be a unique ID to identify the resource in a univocal way.
Example:
```python
self.virtual_machines = {}
self.virtual_machines["vm-01234567890abcdef"] = VirtualMachine()
```
### Service Client
Each Prowler service requires a service client to use the service in the checks.
The following is the `<new_service_name>_client.py` file, which contains the initialization of the freshly created service's class so that service checks can use it. This file is almost the same for all the services among the providers:
```python
from prowler.providers.common.provider import Provider
from prowler.providers.<provider>.services.<new_service_name>.<new_service_name>_service import <Service>
<new_service_name>_client = <Service>(Provider.get_global_provider())
```
## Provider Permissions in Prowler
Before implementing a new service, verify that Prowler's existing permissions for each provider are sufficient. If additional permissions are required, refer to the relevant documentation and update accordingly.
Provider-Specific Permissions Documentation:
- [AWS](../tutorials/aws/authentication.md#required-permissions)
- [Azure](../tutorials/azure/authentication.md#required-permissions)
- [GCP](../tutorials/gcp/authentication.md#required-permissions)
- [M365](../tutorials/microsoft365/authentication.md#required-permissions)
- [GitHub](../tutorials/github/authentication.md)
## Best Practices
- When available in the provider, use threading or parallelization utilities for all methods that can be parallelized by to maximize performance and reduce scan time.
- Define a Pydantic `BaseModel` for every resource you manage, and use these models for all resource data handling.
- Log every major step (start, success, error) in resource discovery and attribute collection for traceability and debugging; include as much context as possible.
- Catch and log all exceptions, providing detailed context (region, subscription, resource, error type, line number) to aid troubleshooting.
- Use consistent naming for resource containers, unique identifiers, and model attributes to improve code readability and maintainability.
- Add docstrings to every method and comments to explain any service-specific logic, especially where provider APIs behave differently or have quirks.
- Collect and store resource tags and additional attributes to support richer checks and reporting.
- Leverage shared utility helpers for session setup, identifier parsing, and other cross-cutting concerns to avoid code duplication. This kind of code is typically stored in a `lib` folder in the service folder.
- Keep code modular, maintainable, and well-documented for ease of extension and troubleshooting.
-550
View File
@@ -1,550 +0,0 @@
---
title: 'Prowler Services'
---
Here you can find how to create a new service, or to complement an existing one, for a [Prowler Provider](/developer-guide/provider).
<Note>
First ensure that the provider you want to add the service is already created. It can be checked [here](https://github.com/prowler-cloud/prowler/tree/master/prowler/providers). If the provider is not present, please refer to the [Provider](./provider.md) documentation to create it from scratch.
</Note>
## Introduction
In Prowler, a **service** represents a specific solution or resource offered by one of the supported [Prowler Providers](/developer-guide/provider), for example, [EC2](https://aws.amazon.com/ec2/) in AWS, or [Microsoft Exchange](https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-online) in M365. Services are the building blocks that allow Prowler interact directly with the various resources exposed by each provider.
Each service is implemented as a class that encapsulates all the logic, data models, and API interactions required to gather and store information about that service's resources. All of this data is used by the [Prowler checks](/developer-guide/checks) to generate the security findings.
## Adding a New Service
To create a new service, a new folder must be created inside the specific provider following this pattern: `prowler/providers/<provider>/services/<new_service_name>/`.
Within this folder the following files are also to be created:
- `__init__.py` (empty) Ensures Python recognizes this folder as a package.
- `<new_service_name>_service.py` Contains all the logic and API calls of the service.
- `<new_service_name>_client_.py` Contains the initialization of the freshly created service's class so that the checks can use it.
Once the files are create, you can check that the service has been created by running the following command: `poetry run python prowler-cli.py <provider> --list-services | grep <new_service_name>`.
## Service Structure and Initialisation
The Prowler's service structure is as outlined below. To initialise it, just import the service client in a check.
### Service Base Class
All Prowler provider service should inherit from a common base class to avoid code duplication. This base class handles initialization and storage of functions and objects needed across services. The exact implementation depends on the provider's API requirements, but the following are the most common responsibilities:
- Initialize/store clients to interact with the provider's API.
- Store the audit and fixer configuration.
- Implement threading logic where applicable.
For reference, the base classes for each provider can be checked here:
- [AWS Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/aws/lib/service/service.py)
- [GCP Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/gcp/lib/service/service.py)
- [Azure Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/azure/lib/service/service.py)
- [Kubernetes Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/kubernetes/lib/service/service.py)
- [M365 Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/m365/lib/service/service.py)
- [GitHub Service Base Class](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/github/lib/service/service.py)
### Service Class
Due to the complexity and differences across provider APIs, the following example demonstrates best practices for structuring a service in Prowler.
File `<new_service_name>_service.py`:
```python title="Example Service Class"
from datetime import datetime
from typing import Optional
# To use the Pydantic's BaseModel.
from pydantic import BaseModel
# Prowler logging library.
from prowler.lib.logger import logger
# Provider parent class.
from prowler.providers.<provider>.lib.service.service import ServiceParentClass
# Create a class for the Service.
class <Service>(ServiceParentClass):
def __init__(self, provider: Provider):
"""Initialize the Service Class
Args:
provider: Prowler Provider object.
"""
# Call Service Parent Class __init__.
# The __class__.__name__ is used to obtain it automatically.
# From the Service Class name, but a custom one can be passed.
# String in case the provider's API service name is different.
super().__init__(__class__.__name__, provider)
# Create an empty dictionary of items to be gathered, using the unique ID as the dictionary's key, e.g., instances.
self.<items> = {}
# If parallelization can be carried out by regions or locations, the function __threading_call__ to be used must be implemented in the Service Parent Class.
# If it is not implemented, you can make it in a sequential way, just calling the function.
self.__threading_call__(self.__describe_<items>__)
# If it is needed you can create another function to retrieve more data from the items.
# Here we are using the second parameter of the __threading_call__ function to create one thread per item.
# You can also make it sequential without using the __threading_call__ function iterating over the items inside the function.
self.__threading_call__(self.__describe_<item>__, self.<items>.values())
# In case of use the __threading_call__ function, you have to pass the regional_client to the function, as a parameter.
def __describe_<items>__(self, regional_client):
"""Get all <items> and store in the self.<items> dictionary
Args:
regional_client: Regional client object.
"""
logger.info("<Service> - Describing <Items>...")
# A try-except block must be created in each function.
try:
# If pagination is supported by the provider, is always better to use it, call to the provider API to retrieve the desired data.
describe_<items>_paginator = regional_client.get_paginator("describe_<items>")
# Paginator to get every item.
for page in describe_<items>_paginator.paginate():
# Another try-except within the for loop to continue iterating in case something unexpected happens.
try:
for <item> in page["<Items>"]:
# Adding Retrieved Resources to the Object
# Once the resource has been retrieved, it must be included in the previously created object to ensure proper data handling within the service.
self.<items>[<item_unique_id>] =
<Item>(
arn=stack["<item_arn>"],
name=stack["<item_name>"],
tags=stack.get("Tags", []),
region=regional_client.region,
)
except Exception as error:
logger.error(
f"{<provider_specific_field>} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
# Logging Errors in Exception Handling
# When handling exceptions, use the following approach to log errors appropriately based on the cloud provider being used:
except Exception as error:
# Depending on each provider we can must use different fields in the logger, e.g.: AWS: regional_client.region or self.region, GCP: project_id and location, Azure: subscription
logger.error(
f"{<provider_specific_field>} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
def __describe_<item>__(self, item: <Item>):
"""Get details for a <item>
Args:
item: Item object.
"""
logger.info("<Service> - Describing <Item> to get specific details...")
# A try-except block must be created in each function.
try:
<item>_details = self.regional_clients[<item>.region].describe_<item>(
<Attribute>=<item>.name
)
# E.g., check if item is Public. This case is important: if values are being retrieved from a dictionary, the function "dict.get()" must be used with a default value in case this value is not present.
<item>.public = <item>_details.get("Public", False)
except Exception as error:
# Fields for logging errors with relevant item information, e.g.: AWS: <item>.region, GCP: <item>.project_id, Azure: <item>.region
logger.error(
f"{<item>.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
)
```
<Note>
To prevent false findings, when Prowler fails to retrieve items due to Access Denied or similar errors, the affected item's value is set to `None`.
</Note>
#### Resource Models
Resource models define structured classes used within services to store and process data extracted from API calls. They are defined in the same file as the service class, but outside of the class, usually at the bottom of the file.
Prowler leverages Pydantic's [BaseModel](https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel) to enforce data validation.
```python title="Service Model"
# Implementation Approach
# Each service class should include custom model classes using Pydantic's BaseModel for the resources being audited.
class <Item>(BaseModel):
"""<Item> holds a <Service> <Item>"""
id: str
"""<Items>[].id"""
name: str
"""<Items>[].name"""
region: str
"""<Items>[].region"""
public: bool
"""<Items>[].public"""
# Optional attributes can be created set to None by default.
tags: Optional[list]
"""<Items>[].tags"""
```
#### Service Attributes
_Optimized Data Storage with Python Dictionaries_
Each group of resources within a service should be structured as a Python [dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries) to enable efficient lookups. The dictionary lookup operation has [O(1) complexity](https://en.wikipedia.org/wiki/Big_O_notation#Orders_of_common_functions), and lookups are constantly executed.
_Assigning Unique Identifiers_
Each dictionary key must be a unique ID to identify the resource in a univocal way.
Example:
```python
self.virtual_machines = {}
self.virtual_machines["vm-01234567890abcdef"] = VirtualMachine()
```
### Service Client
Each Prowler service requires a service client to use the service in the checks.
The following is the `<new_service_name>_client.py` file, which contains the initialization of the freshly created service's class so that service checks can use it. This file is almost the same for all the services among the providers:
```python
from prowler.providers.common.provider import Provider
from prowler.providers.<provider>.services.<new_service_name>.<new_service_name>_service import <Service>
<new_service_name>_client = <Service>(Provider.get_global_provider())
```
## Provider Permissions in Prowler
Before implementing a new service, verify that Prowler's existing permissions for each provider are sufficient. If additional permissions are required, refer to the relevant documentation and update accordingly.
Provider-Specific Permissions Documentation:
- [AWS](/user-guide/providers/aws/authentication#required-permissions)
- [Azure](/user-guide/providers/azure/authentication#required-permissions)
- [GCP](/user-guide/providers/gcp/authentication#required-permissions)
- [M365](/user-guide/providers/microsoft365/authentication#required-permissions)
- [GitHub](/user-guide/providers/github/authentication)
## Service Architecture and Cross-Service Communication
### Core Principle: Service Isolation with Client Communication
Each service must contain **ONLY** the information unique to that specific service. When a check requires information from multiple services, it must use the **client objects** of other services rather than directly accessing their data structures.
This architecture ensures:
- **Loose coupling** between services
- **Clear separation of concerns**
- **Maintainable and testable code**
- **Consistent data access patterns**
### Cross-Service Communication Pattern
Instead of services directly accessing each other's internal data, checks should import and use client objects:
**❌ INCORRECT - Direct data access:**
```python
# DON'T DO THIS
from prowler.providers.aws.services.cloudtrail.cloudtrail_service import cloudtrail_service
from prowler.providers.aws.services.s3.s3_service import s3_service
class cloudtrail_bucket_requires_mfa_delete(Check):
def execute(self):
# WRONG: Directly accessing service data
for trail in cloudtrail_service.trails.values():
for bucket in s3_service.buckets.values():
# Direct access violates separation of concerns
```
**✅ CORRECT - Client-based communication:**
```python
# DO THIS INSTEAD
from prowler.providers.aws.services.cloudtrail.cloudtrail_client import cloudtrail_client
from prowler.providers.aws.services.s3.s3_client import s3_client
class cloudtrail_bucket_requires_mfa_delete(Check):
def execute(self):
# CORRECT: Using client objects for cross-service communication
for trail in cloudtrail_client.trails.values():
trail_bucket = trail.s3_bucket
for bucket in s3_client.buckets.values():
if trail_bucket == bucket.name:
# Use bucket properties through s3_client
if bucket.mfa_delete:
# Implementation logic
```
### Real-World Example: CloudTrail + S3 Integration
This example demonstrates how CloudTrail checks validate S3 bucket configurations:
```python
from prowler.lib.check.models import Check, Check_Report_AWS
from prowler.providers.aws.services.cloudtrail.cloudtrail_client import cloudtrail_client
from prowler.providers.aws.services.s3.s3_client import s3_client
class cloudtrail_bucket_requires_mfa_delete(Check):
def execute(self):
findings = []
if cloudtrail_client.trails is not None:
for trail in cloudtrail_client.trails.values():
if trail.is_logging:
trail_bucket_is_in_account = False
trail_bucket = trail.s3_bucket
# Cross-service communication: CloudTrail check uses S3 client
for bucket in s3_client.buckets.values():
if trail_bucket == bucket.name:
trail_bucket_is_in_account = True
if bucket.mfa_delete:
report.status = "PASS"
report.status_extended = f"Trail {trail.name} bucket ({trail_bucket}) has MFA delete enabled."
# Handle cross-account scenarios
if not trail_bucket_is_in_account:
report.status = "MANUAL"
report.status_extended = f"Trail {trail.name} bucket ({trail_bucket}) is a cross-account bucket or out of Prowler's audit scope, please check it manually."
findings.append(report)
return findings
```
**Key Benefits:**
- **CloudTrail service** only contains CloudTrail-specific data (trails, configurations)
- **S3 service** only contains S3-specific data (buckets, policies, ACLs)
- **Check logic** orchestrates between services using their public client interfaces
- **Cross-account detection** is handled gracefully when resources span accounts
### Service Consolidation Guidelines
**When to combine services in the same file:**
Implement multiple services as **separate classes in the same file** when two services are **practically the same** or one is a **direct extension** of another.
**Example: S3 and S3Control**
S3Control is an extension of S3 that provides account-level controls and access points. Both are implemented in `s3_service.py`:
```python
# File: prowler/providers/aws/services/s3/s3_service.py
class S3(AWSService):
"""Standard S3 service for bucket operations"""
def __init__(self, provider):
super().__init__(__class__.__name__, provider)
self.buckets = {}
self.regions_with_buckets = []
# S3-specific initialization
self._list_buckets(provider)
self._get_bucket_versioning()
# ... other S3-specific operations
class S3Control(AWSService):
"""S3Control service for account-level and access point operations"""
def __init__(self, provider):
super().__init__(__class__.__name__, provider)
self.account_public_access_block = None
self.access_points = {}
# S3Control-specific initialization
self._get_public_access_block()
self._list_access_points()
# ... other S3Control-specific operations
```
**Separate client files:**
```python
# File: prowler/providers/aws/services/s3/s3_client.py
from prowler.providers.aws.services.s3.s3_service import S3
s3_client = S3(Provider.get_global_provider())
# File: prowler/providers/aws/services/s3/s3control_client.py
from prowler.providers.aws.services.s3.s3_service import S3Control
s3control_client = S3Control(Provider.get_global_provider())
```
**When NOT to consolidate services:**
Keep services separate when they:
- **Operate on different resource types** (EC2 vs RDS)
- **Have different authentication mechanisms** (different API endpoints)
- **Serve different operational domains** (IAM vs CloudTrail)
- **Have different regional behaviors** (global vs regional services)
### Cross-Service Dependencies Guidelines
**1. Always use client imports:**
```python
# Correct pattern
from prowler.providers.aws.services.service_a.service_a_client import service_a_client
from prowler.providers.aws.services.service_b.service_b_client import service_b_client
```
**2. Handle missing resources gracefully:**
```python
# Handle cross-service scenarios
resource_found_in_account = False
for external_resource in other_service_client.resources.values():
if target_resource_id == external_resource.id:
resource_found_in_account = True
# Process found resource
break
if not resource_found_in_account:
# Handle cross-account or missing resource scenarios
report.status = "MANUAL"
report.status_extended = "Resource is cross-account or out of audit scope"
```
**3. Document cross-service dependencies:**
```python
class check_with_dependencies(Check):
"""
Check Description
Dependencies:
- service_a_client: For primary resource information
- service_b_client: For related resource validation
- service_c_client: For policy analysis
"""
```
## Regional Service Implementation
When implementing services for regional providers (like AWS, Azure, GCP), special considerations are needed to handle resource discovery across multiple geographic locations. This section provides a complete guide using AWS as the reference example.
### Regional vs Non-Regional Services
**Regional Services:** Require iteration across multiple geographic locations where resources may exist (e.g., EC2 instances, VPC, RDS databases).
**Non-Regional/Global Services:** Operate at a global or tenant level without regional concepts (e.g., IAM users, Route53 hosted zones).
### AWS Regional Implementation Example
AWS is the perfect example of a regional provider. Here's how Prowler handles AWS's regional architecture:
```python
# File: prowler/providers/aws/services/ec2/ec2_service.py
class EC2(AWSService):
def __init__(self, provider):
super().__init__(__class__.__name__, provider)
self.instances = {}
self.security_groups = {}
# Regional resource discovery across all AWS regions
self.__threading_call__(self._describe_instances)
self.__threading_call__(self._describe_security_groups)
def _describe_instances(self, regional_client):
"""Discover EC2 instances in a specific region"""
try:
describe_instances_paginator = regional_client.get_paginator("describe_instances")
for page in describe_instances_paginator.paginate():
for reservation in page["Reservations"]:
for instance in reservation["Instances"]:
# Each instance includes its region
self.instances[instance["InstanceId"]] = Instance(
id=instance["InstanceId"],
region=regional_client.region,
state=instance["State"]["Name"],
# ... other properties
)
except Exception as error:
logger.error(f"Failed to describe instances in {regional_client.region}: {error}")
```
#### Regional Check Execution
```python
# File: prowler/providers/aws/services/ec2/ec2_instance_public_ip/ec2_instance_public_ip.py
class ec2_instance_public_ip(Check):
def execute(self):
findings = []
# Automatically iterates across ALL AWS regions where instances exist
for instance in ec2_client.instances.values():
report = Check_Report_AWS(metadata=self.metadata(), resource=instance)
report.region = instance.region # Critical: region attribution
report.resource_arn = f"arn:aws:ec2:{instance.region}:{instance.account_id}:instance/{instance.id}"
if instance.public_ip:
report.status = "FAIL"
report.status_extended = f"Instance {instance.id} in {instance.region} has public IP {instance.public_ip}"
else:
report.status = "PASS"
report.status_extended = f"Instance {instance.id} in {instance.region} does not have a public IP"
findings.append(report)
return findings
```
#### Key AWS Regional Features
**Region-Specific ARNs:**
```
arn:aws:ec2:us-east-1:123456789012:instance/i-1234567890abcdef0
arn:aws:s3:eu-west-1:123456789012:bucket/my-bucket
arn:aws:rds:ap-southeast-2:123456789012:db:my-database
```
**Parallel Processing:**
- Each region processed independently in separate threads
- Failed regions don't affect other regions
- User can filter specific regions: `-f us-east-1`
**Global vs Regional Services:**
- **Regional**: EC2, RDS, VPC (require region iteration)
- **Global**: IAM, Route53, CloudFront (single `us-east-1` call)
This architecture allows Prowler to efficiently scan AWS accounts with resources spread across multiple regions while maintaining performance and error isolation.
### Regional Service Best Practices
1. **Use Threading for Regional Discovery**: Leverage the `__threading_call__` method to parallelize resource discovery across regions
2. **Store Region Information**: Always include region metadata in resource objects for proper attribution
3. **Handle Regional Failures Gracefully**: Ensure that failures in one region don't affect others
4. **Optimize for Performance**: Use paginated calls and efficient data structures for large-scale resource discovery
5. **Support Region Filtering**: Allow users to limit scans to specific regions for focused audits
## Best Practices
- When available in the provider, use threading or parallelization utilities for all methods that can be parallelized by to maximize performance and reduce scan time.
- Define a Pydantic `BaseModel` for every resource you manage, and use these models for all resource data handling.
- Log every major step (start, success, error) in resource discovery and attribute collection for traceability and debugging; include as much context as possible.
- Catch and log all exceptions, providing detailed context (region, subscription, resource, error type, line number) to aid troubleshooting.
- Use consistent naming for resource containers, unique identifiers, and model attributes to improve code readability and maintainability.
- Add docstrings to every method and comments to explain any service-specific logic, especially where provider APIs behave differently or have quirks.
- Collect and store resource tags and additional attributes to support richer checks and reporting.
- Leverage shared utility helpers for session setup, identifier parsing, and other cross-cutting concerns to avoid code duplication. This kind of code is typically stored in a `lib` folder in the service folder.
- Keep code modular, maintainable, and well-documented for ease of extension and troubleshooting.
- **Each service should contain only information unique to that specific service** - use client objects for cross-service communication.
- **Handle cross-account and missing resources gracefully** when checks span multiple services.
@@ -1,6 +1,4 @@
---
title: 'Unit Tests for Prowler Checks'
---
# Unit Tests for Prowler Checks
Unit tests for Prowler checks vary based on the provider being evaluated.
@@ -41,7 +39,7 @@ To execute the Prowler test suite, install the necessary dependencies listed in
### Prerequisites
If you have not installed Prowler yet, refer to the [developer guide introduction](/developer-guide/introduction#getting-the-code-and-installing-all-dependencies).
If you have not installed Prowler yet, refer to the [developer guide introduction](./introduction.md#getting-the-code-and-installing-all-dependencies).
### Executing Tests
@@ -59,18 +57,16 @@ Other Commands for Running Tests
- Running tests for a provider check:
`pytest -n auto -vvv -s -x tests/providers/<provider>/services/<service>/<check>`
<Note>
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
???+ note
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) for more details.
</Note>
## AWS Testing Approaches
For AWS provider, different testing approaches apply based on API coverage based on several criteria.
<Note>
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
???+ note
Prowler leverages and contributes to the[Moto](https://github.com/getmoto/moto) library for mocking AWS infrastructure in tests.
</Note>
- AWS API Calls Covered by [Moto](https://github.com/getmoto/moto):
- Service Tests: `@mock_aws`
- Checks Tests: `@mock_aws`
@@ -209,14 +205,12 @@ class Test_iam_password_policy_uppercase:
If the IAM service required for testing is not supported by the Moto library, use [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock) to inject objects into the service client.
<Warning>
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
???+ warning
As stated above, direct service instantiation must be avoided to prevent actual AWS API calls.
</Warning>
<Note>
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
???+ note
The example below demonstrates the IAM GetAccountPasswordPolicy API, which is covered by Moto, but is used for instructional purposes only.
</Note>
#### Mocking Service Objects Using MagicMock
The following code demonstrates how to use MagicMock to create service objects.
@@ -383,15 +377,13 @@ class Test_iam_password_policy_uppercase:
# Refer to the previous section for the check test, as the implementation remains unchanged.
```
<Note>
This example does not use Moto to simplify the setup.
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
???+ note
This example does not use Moto to simplify the setup.
However, if additional `moto` decorators are applied alongside the patch, Moto will automatically intercept the call to `orig(self, operation_name, kwarg)`.
</Note>
<Note>
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
???+ note
The source of the above implementation can be found here:[Patch Other Services with Moto](https://docs.getmoto.org/en/latest/docs/services/patching\_other\_services.html)
</Note>
#### Mocking Several Services
Since the provider is being mocked, multiple attributes can be configured to customize its behavior:
@@ -496,7 +488,7 @@ will cause that the service will be initialised twice:
Later, when importing `<service>_client.py` at `<check>.py`, Python uses the mocked instance since the patch was applied at the correct reference point.
In the [next section](/developer-guide/unit-testing#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
In the [next section](./unit-testing.md#mocking-the-service-and-the-service-client-at-the-service-client-level) we will explore an improved approach to mock objects.
##### Mocking the Service and the Service Client at the Service Client Level
@@ -650,10 +642,9 @@ class Test_compute_project_os_login_enabled:
The testing of Google Cloud Services follows the same principles as the one of Google Cloud checks. While all API calls must be mocked, attribute setup for API calls in this scenario is defined in the fixtures file, specifically within the [fixtures file](https://github.com/prowler-cloud/prowler/blob/master/tests/providers/gcp/gcp_fixtures.py) in the `mock_api_client` function.
<Warning>
Every method within a service must be tested to ensure full coverage and accurate validation.
???+ important
Every method within a service must be tested to ensure full coverage and accurate validation.
</Warning>
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
```python title="BigQuery Service Test"
@@ -916,12 +907,9 @@ class Test_app_ensure_http_is_redirected_to_https:
The testing of Azure Services follows the same principles as the one of Google Cloud checks. All API calls are still mocked, but for methods that initialize attributes via an API call, use the [patch](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch) decorator at the beginning of the class to ensure proper mocking.
<Warning>
**Remember**
???+ important "Remember"
Every method within a service must be tested to ensure full coverage and accurate validation.
Every method within a service must be tested to ensure full coverage and accurate validation.
</Warning>
The following example presents a real testing class, but includes additional comments for educational purposes, explaining key concepts and implementation details.
```python title="AppInsights Service Test"
-433
View File
@@ -1,433 +0,0 @@
{
"$schema": "https://mintlify.com/docs.json",
"theme": "mint",
"name": "Prowler Documentation",
"colors": {
"primary": "#000000",
"light": "#10B981",
"dark": "#10B981"
},
"favicon": "/favicon.ico",
"logo": {
"dark": "/images/prowler-logo-white.png",
"light": "/images/prowler-logo-black.png"
},
"navigation": {
"tabs": [
{
"tab": "Getting Started",
"groups": [
{
"group": "Welcome",
"pages": [
"introduction"
]
},
{
"group": "Prowler Cloud",
"pages": [
"getting-started/products/prowler-cloud",
"getting-started/products/prowler-cloud-pricing",
"getting-started/products/prowler-cloud-aws-marketplace",
"getting-started/goto/prowler-cloud",
"getting-started/goto/prowler-api-reference"
]
},
{
"group": "Prowler CLI",
"pages": [
"getting-started/products/prowler-cli",
"getting-started/installation/prowler-cli",
"getting-started/basic-usage/prowler-cli"
]
},
{
"group": "Prowler App",
"pages": [
"getting-started/products/prowler-app",
"getting-started/installation/prowler-app",
"getting-started/basic-usage/prowler-app"
]
},
{
"group": "Prowler Lighthouse AI",
"pages": [
"user-guide/tutorials/prowler-app-lighthouse"
]
},
{
"group": "Prowler MCP Server",
"pages": [
"getting-started/products/prowler-mcp",
"getting-started/installation/prowler-mcp",
"getting-started/basic-usage/prowler-mcp",
"getting-started/basic-usage/prowler-mcp-tools"
]
},
{
"group": "Prowler Hub",
"pages": [
"getting-started/products/prowler-hub",
"getting-started/goto/prowler-hub"
]
},
{
"group": "Prowler vs. Others",
"pages": [
"getting-started/comparison/index",
"getting-started/comparison/awssecurityhub",
"getting-started/comparison/gcp",
"getting-started/comparison/microsoftdefender",
"getting-started/comparison/microsoftsentinel"
]
}
]
},
{
"tab": "Guides",
"groups": [
{
"group": "Prowler Cloud/App",
"pages": [
"user-guide/tutorials/prowler-app",
{
"group": "Authentication",
"pages": [
"user-guide/tutorials/prowler-app-social-login",
"user-guide/tutorials/prowler-app-sso"
]
},
"user-guide/tutorials/prowler-app-rbac",
"user-guide/providers/prowler-app-api-keys",
"user-guide/tutorials/prowler-app-mute-findings",
{
"group": "Integrations",
"expanded": true,
"pages": [
"user-guide/tutorials/prowler-app-s3-integration",
"user-guide/tutorials/prowler-app-security-hub-integration",
"user-guide/tutorials/prowler-app-jira-integration"
]
},
"user-guide/tutorials/prowler-app-lighthouse",
{
"group": "Tutorials",
"pages": [
"user-guide/tutorials/prowler-app-sso-entra",
"user-guide/tutorials/bulk-provider-provisioning",
"user-guide/tutorials/aws-organizations-bulk-provisioning"
]
}
]
},
{
"group": "CLI",
"pages": [
"user-guide/cli/tutorials/misc",
"user-guide/cli/tutorials/reporting",
"user-guide/cli/tutorials/compliance",
"user-guide/cli/tutorials/dashboard",
"user-guide/cli/tutorials/configuration_file",
"user-guide/cli/tutorials/logging",
"user-guide/cli/tutorials/mutelist",
{
"group": "Integrations",
"pages": [
"user-guide/providers/aws/securityhub",
"user-guide/cli/tutorials/integrations",
"user-guide/providers/aws/s3"
]
},
"user-guide/cli/tutorials/fixer",
"user-guide/cli/tutorials/check-aliases",
"user-guide/cli/tutorials/custom-checks-metadata",
"user-guide/cli/tutorials/pentesting",
"user-guide/cli/tutorials/scan-unused-services",
"user-guide/cli/tutorials/quick-inventory",
{
"group": "Tutorials",
"pages": [
"user-guide/cli/tutorials/parallel-execution"
]
}
]
},
{
"group": "Providers",
"pages": [
{
"group": "AWS",
"pages": [
"user-guide/providers/aws/getting-started-aws",
"user-guide/providers/aws/authentication",
"user-guide/providers/aws/role-assumption",
"user-guide/providers/aws/organizations",
"user-guide/providers/aws/regions-and-partitions",
"user-guide/providers/aws/tag-based-scan",
"user-guide/providers/aws/resource-arn-based-scan",
"user-guide/providers/aws/boto3-configuration",
"user-guide/providers/aws/threat-detection",
"user-guide/providers/aws/cloudshell",
"user-guide/providers/aws/multiaccount"
]
},
{
"group": "Azure",
"pages": [
"user-guide/providers/azure/getting-started-azure",
"user-guide/providers/azure/authentication",
"user-guide/providers/azure/use-non-default-cloud",
"user-guide/providers/azure/subscriptions",
"user-guide/providers/azure/create-prowler-service-principal"
]
},
{
"group": "Google Cloud",
"pages": [
"user-guide/providers/gcp/getting-started-gcp",
"user-guide/providers/gcp/authentication",
"user-guide/providers/gcp/projects",
"user-guide/providers/gcp/organization",
"user-guide/providers/gcp/retry-configuration"
]
},
{
"group": "Kubernetes",
"pages": [
"user-guide/providers/kubernetes/in-cluster",
"user-guide/providers/kubernetes/outside-cluster",
"user-guide/providers/kubernetes/misc"
]
},
{
"group": "Microsoft 365",
"pages": [
"user-guide/providers/microsoft365/getting-started-m365",
"user-guide/providers/microsoft365/authentication",
"user-guide/providers/microsoft365/use-of-powershell"
]
},
{
"group": "GitHub",
"pages": [
"user-guide/providers/github/getting-started-github",
"user-guide/providers/github/authentication"
]
},
{
"group": "IaC",
"pages": [
"user-guide/providers/iac/getting-started-iac",
"user-guide/providers/iac/authentication"
]
},
{
"group": "MongoDB Atlas",
"pages": [
"user-guide/providers/mongodbatlas/getting-started-mongodbatlas",
"user-guide/providers/mongodbatlas/authentication"
]
},
{
"group": "LLM",
"pages": [
"user-guide/providers/llm/getting-started-llm"
]
},
{
"group": "Oracle Cloud Infrastructure",
"pages": [
"user-guide/providers/oci/getting-started-oci",
"user-guide/providers/oci/authentication"
]
}
]
},
{
"group": "Compliance",
"pages": [
"user-guide/compliance/tutorials/threatscore"
]
}
]
},
{
"tab": "Developer Guide",
"groups": [
{
"group": "Concepts",
"pages": [
"developer-guide/introduction",
"developer-guide/provider",
"developer-guide/services",
"developer-guide/checks",
"developer-guide/outputs",
"developer-guide/integrations",
"developer-guide/security-compliance-framework",
"developer-guide/lighthouse"
]
},
{
"group": "Providers",
"pages": [
"developer-guide/aws-details",
"developer-guide/azure-details",
"developer-guide/gcp-details",
"developer-guide/kubernetes-details",
"developer-guide/m365-details",
"developer-guide/github-details",
"developer-guide/llm-details"
]
},
{
"group": "Miscellaneous",
"pages": [
"developer-guide/documentation",
{
"group": "Testing",
"pages": [
"developer-guide/unit-testing",
"developer-guide/integration-testing"
]
},
"developer-guide/debugging",
"developer-guide/configurable-checks",
"developer-guide/renaming-checks",
"developer-guide/check-metadata-guidelines"
]
}
]
},
{
"tab": "Security",
"pages": [
"security"
]
},
{
"tab": "Contact Us",
"pages": [
"contact"
]
},
{
"tab": "Troubleshooting",
"pages": [
"troubleshooting"
]
},
{
"tab": "About Us",
"icon": "/favicon.ico",
"href": "https://prowler.com/about#team"
},
{
"tab": "Changelog",
"icon": "github",
"href": "https://github.com/prowler-cloud/prowler/releases"
},
{
"tab": "Public Roadmap",
"href": "https://roadmap.prowler.com/"
}
],
"global": {
"anchors": [
{
"anchor": "GitHub",
"href": "https://github.com/prowler-cloud/prowler",
"icon": "github"
},
{
"anchor": "Slack",
"href": "https://goto.prowler.com/slack",
"icon": "slack"
},
{
"anchor": "YouTube",
"href": "https://www.youtube.com/@prowlercloud",
"icon": "youtube"
}
]
}
},
"navbar": {
"links": [
{
"label": "Prowler Hub",
"href": "https://hub.prowler.com"
},
{
"label": "Prowler Cloud",
"href": "https://cloud.prowler.com",
"style": "primary"
}
]
},
"analytics": {
"ga4": {
"measurementId": "G-KBKV70W5Y2"
}
},
"feedback": {
"thumbsRating": true,
"suggestEdit": true,
"raiseIssue": true
},
"footer": {
"socials": {
"x-twitter": "https://x.com/prowlercloud",
"github": "https://github.com/prowler-cloud/prowler",
"linkedin": "https://www.linkedin.com/company/prowler-security",
"youtube": "https://www.youtube.com/@prowlercloud",
"slack": "https://goto.prowler.com/slack",
"website": "https://prowler.com"
}
},
"redirects": [
{
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-lighthouse",
"destination": "/user-guide/tutorials/prowler-app-lighthouse"
},
{
"source": "/projects/prowler-open-source/en/latest/developer-guide/introduction",
"destination": "/developer-guide/introduction"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/aws/getting-started-aws",
"destination": "/user-guide/providers/aws/getting-started-aws"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/azure/getting-started-azure",
"destination": "/user-guide/providers/azure/getting-started-azure"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/gcp/getting-started-gcp",
"destination": "/user-guide/providers/gcp/getting-started-gcp"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/microsoft365/getting-started-m365",
"destination": "/user-guide/providers/microsoft365/getting-started-m365"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/github/getting-started-github",
"destination": "/user-guide/providers/github/getting-started-github"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/prowler-app-sso",
"destination": "/user-guide/tutorials/prowler-app-sso"
},
{
"source": "/projects/prowler-open-source/en/latest",
"destination": "/introduction"
},
{
"source": "/projects/prowler-saas/en/latest/:slug*",
"destination": "https://docs.prowler.pro/en/latest/:slug*"
},
{
"source": "/projects/prowler-open-source/en/latest/tutorials/:slug*",
"destination": "/user-guide/tutorials/:slug*"
}
]
}
@@ -1,70 +0,0 @@
---
title: 'Basic Usage'
---
## Access Prowler App
After [installation](/getting-started/installation/prowler-app), navigate to [http://localhost:3000](http://localhost:3000) and sign up with email and password.
<img src="/images/sign-up-button.png" alt="Sign Up Button" width="320" />
<img src="/images/sign-up.png" alt="Sign Up" width="285" />
<Note>
**User creation and default tenant behavior**
When creating a new user, the behavior depends on whether an invitation is provided:
- **Without an invitation**:
- A new tenant is automatically created.
- The new user is assigned to this tenant.
- A set of **RBAC admin permissions** is generated and assigned to the user for the newly-created tenant.
- **With an invitation**: The user is added to the specified tenant with the permissions defined in the invitation.
This mechanism ensures that the first user in a newly created tenant has administrative permissions within that tenant.
</Note>
## Log In
Access Prowler App by logging in with **email and password**.
<img src="/images/log-in.png" alt="Log In" width="285" />
## Add Cloud Provider
Configure a cloud provider for scanning:
1. Navigate to `Settings > Cloud Providers` and click `Add Account`.
2. Select the cloud provider.
3. Enter the provider's identifier (Optional: Add an alias):
- **AWS**: Account ID
- **GCP**: Project ID
- **Azure**: Subscription ID
- **Kubernetes**: Cluster ID
- **M365**: Domain ID
4. Follow the guided instructions to add and authenticate your credentials.
## Start a Scan
Once credentials are successfully added and validated, Prowler initiates a scan of your cloud environment.
Click `Go to Scans` to monitor progress.
## View Results
Review findings during scan execution in the following sections:
- **Overview** Provides a high-level summary of your scans.
<img src="/images/products/overview.png" alt="Overview" width="700" />
- **Compliance** Displays compliance insights based on security frameworks.
<img src="/images/compliance.png" alt="Compliance" width="700" />
> For detailed usage instructions, refer to the [Prowler App Guide](/user-guide/tutorials/prowler-app).
<Note>
Prowler will automatically scan all configured providers every **24 hours**, ensuring your cloud environment stays continuously monitored.
</Note>
@@ -1,122 +0,0 @@
---
title: "Tools Reference"
---
Complete reference guide for all tools available in the Prowler MCP Server. Tools are organized by namespace.
## Tool Categories Summary
| Category | Tool Count | Authentication Required |
|----------|------------|------------------------|
| Prowler Hub | 10 tools | No |
| Prowler Documentation | 2 tools | No |
| Prowler Cloud/App | 28 tools | Yes |
## Tool Naming Convention
All tools follow a consistent naming pattern with prefixes:
- `prowler_hub_*` - Prowler Hub catalog and compliance tools
- `prowler_docs_*` - Prowler documentation search and retrieval
- `prowler_app_*` - Prowler Cloud and App (Self-Managed) management tools
## Prowler Hub Tools
Access Prowler's security check catalog and compliance frameworks. **No authentication required.**
### Check Discovery
- **`prowler_hub_get_checks`** - List security checks with advanced filtering options
- **`prowler_hub_get_check_filters`** - Return available filter values for checks (providers, services, severities, categories, compliances)
- **`prowler_hub_search_checks`** - Full-text search across check metadata
- **`prowler_hub_get_check_raw_metadata`** - Fetch raw check metadata in JSON format
### Check Code
- **`prowler_hub_get_check_code`** - Fetch the Python implementation code for a security check
- **`prowler_hub_get_check_fixer`** - Fetch the automated fixer code for a check (if available)
### Compliance Frameworks
- **`prowler_hub_get_compliance_frameworks`** - List and filter compliance frameworks
- **`prowler_hub_search_compliance_frameworks`** - Full-text search across compliance frameworks
### Provider Information
- **`prowler_hub_list_providers`** - List Prowler official providers and their services
- **`prowler_hub_get_artifacts_count`** - Get total count of checks and frameworks in Prowler Hub
## Prowler Documentation Tools
Search and access official Prowler documentation. **No authentication required.**
- **`prowler_docs_search`** - Search the official Prowler documentation using full-text search
- **`prowler_docs_get_document`** - Retrieve the full markdown content of a specific documentation file
## Prowler Cloud/App Tools
Manage Prowler Cloud or Prowler App (Self-Managed) features. **Requires authentication.**
<Note>
These tools require a valid API key. See the [Configuration Guide](/getting-started/basic-usage/prowler-mcp) for authentication setup.
</Note>
### Findings Management
- **`prowler_app_list_findings`** - List security findings with advanced filtering
- **`prowler_app_get_finding`** - Get detailed information about a specific finding
- **`prowler_app_get_latest_findings`** - Retrieve latest findings from the most recent scans
- **`prowler_app_get_findings_metadata`** - Get unique metadata values from filtered findings
- **`prowler_app_get_latest_findings_metadata`** - Get metadata from latest findings across all providers
### Provider Management
- **`prowler_app_list_providers`** - List all providers with filtering options
- **`prowler_app_create_provider`** - Create a new provider in the current tenant
- **`prowler_app_get_provider`** - Get detailed information about a specific provider
- **`prowler_app_update_provider`** - Update provider details (alias, etc.)
- **`prowler_app_delete_provider`** - Delete a specific provider
- **`prowler_app_test_provider_connection`** - Test provider connection status
### Provider Secrets Management
- **`prowler_app_list_provider_secrets`** - List all provider secrets with filtering
- **`prowler_app_add_provider_secret`** - Add or update credentials for a provider
- **`prowler_app_get_provider_secret`** - Get detailed information about a provider secret
- **`prowler_app_update_provider_secret`** - Update provider secret details
- **`prowler_app_delete_provider_secret`** - Delete a provider secret
### Scan Management
- **`prowler_app_list_scans`** - List all scans with filtering options
- **`prowler_app_create_scan`** - Trigger a manual scan for a specific provider
- **`prowler_app_get_scan`** - Get detailed information about a specific scan
- **`prowler_app_update_scan`** - Update scan details
- **`prowler_app_get_scan_compliance_report`** - Download compliance report as CSV
- **`prowler_app_get_scan_report`** - Download ZIP file containing complete scan report
### Schedule Management
- **`prowler_app_schedules_daily_scan`** - Create a daily scheduled scan for a provider
### Processor Management
- **`prowler_app_processors_list`** - List all processors with filtering
- **`prowler_app_processors_create`** - Create a new processor (currently only mute lists supported)
- **`prowler_app_processors_retrieve`** - Get processor details by ID
- **`prowler_app_processors_partial_update`** - Update processor configuration
- **`prowler_app_processors_destroy`** - Delete a processor
## Usage Tips
- Use natural language to interact with the tools through your AI assistant
- Tools can be combined for complex workflows
- Filter options are available on most list tools
- Authentication is only required for Prowler Cloud/App tools
## Additional Resources
- [MCP Protocol Specification](https://modelcontextprotocol.io)
- [Prowler API Documentation](https://api.prowler.com/api/v1/docs)
- [Prowler Hub API](https://hub.prowler.com/api/docs)
- [GitHub Repository](https://github.com/prowler-cloud/prowler)
@@ -1,247 +0,0 @@
---
title: "Configuration"
---
Configure your MCP client to connect to Prowler MCP Server.
## Step 1: Get Your API Key (Optional)
<Note>
**Authentication is optional**: Prowler Hub and Prowler Documentation features work without authentication. An API key is only required for Prowler Cloud and Prowler App (Self-Managed) features.
</Note>
To use Prowler Cloud or Prowler App (Self-Managed) features. To get the API key, please refer to the [API Keys](/user-guide/providers/prowler-app-api-keys) guide.
<Warning>
Keep the API key secure. Never share it publicly or commit it to version control.
</Warning>
## Step 2: Configure Your MCP Client
Choose the configuration based on your deployment:
- **STDIO Mode**: Local installation only (runs as subprocess).
- **HTTP Mode**: Prowler Cloud MCP Server or self-hosted Prowler MCP Server.
### HTTP Mode (Prowler Cloud MCP Server or self-hosted Prowler MCP Server)
<Tabs>
<Tab title="Native HTTP Support (Cursor, VSCode)">
**Clients that support HTTP with custom headers natively**
For example: Cursor, VSCode, LobeChat, etc.
**Configuration:**
```json
{
"mcpServers": {
"prowler": {
"url": "https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
"headers": {
"Authorization": "Bearer pk_your_api_key_here"
}
}
}
}
```
</Tab>
<Tab title="Using mcp-remote (Claude Desktop)">
**For clients without native HTTP support (like Claude Desktop)**
For example: Claude Desktop.
**Configuration:**
```json
{
"mcpServers": {
"prowler": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.prowler.com/mcp", // or your self-hosted Prowler MCP Server URL
"--header",
"Authorization: Bearer ${PROWLER_APP_API_KEY}"
],
"env": {
"PROWLER_APP_API_KEY": "pk_your_api_key_here"
}
}
}
}
```
<Info>
The `mcp-remote` tool acts as a bridge for clients that don't support HTTP natively. Learn more at [mcp-remote on npm](https://www.npmjs.com/package/mcp-remote).
</Info>
</Tab>
</Tabs>
### STDIO Mode (Local Installation Only)
STDIO mode is only available when running the MCP server locally.
<Tabs>
<Tab title="Using uvx">
**Run from source or local installation**
```json
{
"mcpServers": {
"prowler": {
"command": "uvx",
"args": ["/absolute/path/to/prowler/mcp_server/"],
"env": {
"PROWLER_APP_API_KEY": "pk_your_api_key_here",
"PROWLER_API_BASE_URL": "https://api.prowler.com"
}
}
}
}
```
<Note>
Replace `/absolute/path/to/prowler/mcp_server/` with the actual path. The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
</Note>
</Tab>
<Tab title="Using Docker">
**Run with Docker image**
```json
{
"mcpServers": {
"prowler": {
"command": "docker",
"args": [
"run",
"--rm",
"-i",
"--env",
"PROWLER_APP_API_KEY=pk_your_api_key_here",
"--env",
"PROWLER_API_BASE_URL=https://api.prowler.com",
"prowlercloud/prowler-mcp"
]
}
}
}
```
<Note>
The `PROWLER_API_BASE_URL` is optional and defaults to Prowler Cloud API.
</Note>
</Tab>
</Tabs>
## Step 3: Start Using Prowler MCP
Restart your MCP client and start asking questions:
- *"Show me all critical findings from my AWS accounts"*
- *"What does the S3 bucket public access check do?"*
- *"Onboard this new AWS account in my Prowler Organization"*
## Authentication Methods
Prowler MCP Server supports two authentication methods to connect to Prowler Cloud or Prowler App (Self-Managed):
### API Key (Recommended)
Use your Prowler API key directly in the Bearer token:
```
Authorization: Bearer pk_your_api_key_here
```
This is the recommended method for most users.
### JWT Token
Alternatively, obtain a JWT token from Prowler:
```bash
curl -X POST https://api.prowler.com/api/v1/tokens \
-H "Content-Type: application/vnd.api+json" \
-H "Accept: application/vnd.api+json" \
-d '{
"data": {
"type": "tokens",
"attributes": {
"email": "your-email@example.com",
"password": "your-password"
}
}
}'
```
Use the returned JWT token in place of the API key:
```
Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
```
<Warning>
JWT tokens are only valid for 30 minutes. You need to generate a new token if you want to continue using the MCP server.
</Warning>
## Troubleshooting
### Server Not Detected
- Restart your MCP client after configuration changes
- Check the configuration file syntax (valid JSON)
- Review client logs for specific error messages
- Verify the server URL is correct
### Authentication Failures
**Error: Unauthorized (401)**
- Verify your API key is correct
- Ensure the key hasn't expired
- Check you're using the right API endpoint
### Connection Issues
**Cannot Reach Server:**
- Verify the server URL is correct
- Check network connectivity
- For local servers, ensure the server is running
- Check firewall settings
## Security Best Practices
1. **Protect Your API Key**
- Never commit API keys to version control.
- Use environment variables or secure vaults.
- Rotate keys regularly.
2. **Network Security**
- Use HTTPS for production deployments.
- Restrict network access to the MCP server.
- Consider VPN for remote access.
3. **Least Privilege**
- API key gives the permission of the user who created the key, make sure to use the key with the minimal required permissions.
- Review the tools that are gonna be used and how they are gonna be used to avoid prompt injections or unintended behavior.
## Next Steps
Now that your MCP server is configured:
<CardGroup cols={1}>
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
Explore all available tools
</Card>
</CardGroup>
## Getting Help
Need assistance with configuration?
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
@@ -1,94 +0,0 @@
---
title: 'AWS Security Hub'
---
AWS Security Hub remains a managed service designed for centralizing security alerts and compliance status within AWS environments. It integrates with various AWS security services and provides a consolidated view of security findings.
## Key Features and Strengths
- **Centralized Dashboard for AWS:** Provides a single pane of glass to monitor and manage security findings from multiple AWS services like GuardDuty, Inspector, and Config.
- **Compliance Checks:** Automatically checks for compliance against standards like CIS and PCI DSS within AWS environments.
- **AWS Native Automation:** Offers seamless automation for incident response using AWS Lambda and CloudWatch Events, reducing the time to react to security issues.
- **User-Friendly Interface:** Accessible via the AWS Management Console, offering a streamlined experience for managing security across AWS accounts.
## Limitations
- **AWS-Centric:** Limited to AWS environments, with no direct support for multi-cloud or hybrid environments.
- **Dependency on AWS Config:** Some of its checks depend on AWS Config, which may not be enabled in all regions or accounts.
- **Vendor Lock-In:** Tightly coupled with AWS, making it less suitable for organizations with a cloud-agnostic strategy.
## Prowler
Prowler is an open-source, multi-cloud security tool that offers extensive customization and flexibility, making it ideal for organizations with complex or multi-cloud environments. Here are the updated features and advantages:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler is inherently multi-region and can scan multiple AWS accounts without requiring additional configuration or enabling specific services like AWS Config.
- **Minimal Setup Requirements:**
- All Prowler needs is a role with appropriate permissions to start scanning. Theres no need to enable specific services or configure complex setups.
- **Versatile Execution Environment:**
- Prowler can be run from various environments, including a local workstation, container, AWS CloudShell, or even from another AWS account or cloud provider by assuming a role. This flexibility makes it easy to integrate into different operational workflows.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis, or locally for easy sharing and discussion. This flexibility is particularly useful for collaborative security assessments.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. It also supports integration with Amazon QuickSight for in-depth analysis and offers a SaaS model with resource-based pricing, making it adaptable to different organizational needs.
- **Security Hub Integration for Cost-Effective Operations:**
- Prowler can send results directly into Security Hub in any AWS account, including only failed findings. This selective reporting can make Security Hub more cost-effective by reducing the volume of data processed.
- **Custom Checks and Compliance Frameworks:**
- Users can write custom checks, remediations, and compliance frameworks in minutes, tailoring the tool to their specific security policies and operational needs.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box for AWS, providing comprehensive coverage across various regulatory requirements and best practices.
- **Kubernetes and Multi-Cloud Support:**
- Prowler extends its scanning capabilities beyond AWS, offering support for Kubernetes clusters (including EKS), as well as environments in Google Cloud Platform (GCP) and Azure. This multi-cloud capability is essential for organizations with diverse cloud footprints.
- **All-Region Checks:**
- Prowler runs all checks in all regions, regardless of AWS Config resource type support, ensuring comprehensive coverage across your entire AWS environment.
## Comparison Summary
### Scope and Environment
- **Security Hub** is ideal for AWS-centric environments needing a managed service for monitoring and automating security across AWS resources.
- **Prowler** is better suited for organizations operating in multi-cloud or hybrid environments, offering flexibility, customization, and support for multiple cloud providers including AWS, Azure, GCP, and Kubernetes.
### Setup and Maintenance
- **Security Hub** requires enabling and configuring AWS services by region, per account, and can become more than one person's full-time role including Config. Security Hub operates only within the AWS ecosystem.
- **Prowler** requires minimal setup, only needing appropriate permissions, and can be executed from various environments, making it more versatile in different operational contexts.
### Customization and Flexibility
- **Security Hub** offers predefined compliance checks and automation within AWS but is less flexible in terms of customization.
- **Prowler** allows for highly customizable checks, remediation actions, and compliance frameworks, with the ability to adapt quickly to organizational needs and regulatory changes.
### Cost Efficiency
- **Security Hub** may involve additional costs for processing and storing findings.
- **Prowler** can optimize costs by selectively sending failed findings to Security Hub and storing results locally or in S3, which can be more cost-effective.
### Multi-Cloud and Multi-Region Support
- **Security Hub** is confined to AWS, with region-specific checks depending on AWS Config.
- **Prowler** is inherently multi-region and multi-cloud, offering consistent and comprehensive checks across different cloud environments and regions.
## Conclusion
For a CISO or security professional evaluating these tools, the decision between AWS Security Hub and Prowler will depend on the organizations cloud strategy, compliance needs, and the level of flexibility required:
- If the organization is heavily invested in AWS and prefers a managed, integrated security service that offers ease of use and automation within the AWS ecosystem, **AWS Security Hub** is the more appropriate choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool that can run comprehensive, multi-region scans across AWS, Azure, GCP, and Kubernetes, **Prowler** provides a more powerful and flexible solution, especially for those needing to adapt quickly to evolving security and compliance requirements.
-97
View File
@@ -1,97 +0,0 @@
---
title: 'GCP Cloud Security Command Center (Cloud SCC)'
---
Google Cloud Security Command Center (Cloud SCC) is a centralized security and risk management platform for Google Cloud Platform (GCP). It provides visibility into assets, vulnerabilities, and threats across GCP environments, helping organizations to manage and improve their security posture.
## Key Features and Strengths
- **Centralized Security Visibility:** Cloud SCC provides a single pane of glass to monitor the security and risk status across your GCP resources. It aggregates findings from various GCP security services, such as Security Health Analytics, Web Security Scanner, and Event Threat Detection.
- **Asset Inventory and Classification:** Cloud SCC offers comprehensive asset discovery and classification across GCP, giving security teams a detailed inventory of their cloud resources, including their configurations and security states.
- **Threat Detection and Monitoring:** The platform integrates with GCPs threat detection tools, such as Googles Event Threat Detection, which analyzes logs for suspicious activities and potential threats.
- **Compliance Monitoring:** Cloud SCC helps monitor compliance with various regulatory standards by continuously assessing your GCP resources against best practices and security benchmarks.
- **Automated Remediation:** Cloud SCC can trigger automated responses to security findings through integrations with Google Cloud Functions or other orchestration tools, helping to mitigate risks quickly.
- **Native GCP Integration:** Cloud SCC is deeply integrated with the GCP ecosystem, offering seamless operation within Google Cloud environments and leveraging Google's extensive security expertise.
## Limitations
- **GCP-Centric:** While Cloud SCC is powerful within the GCP ecosystem, it is primarily focused on GCP and does not natively extend to multi-cloud environments without additional tools or connectors.
- **Cost Considerations:** As a managed service within GCP, costs can scale with the amount of data ingested and the complexity of the environment, especially as additional features or higher volumes of data are utilized.
- **Dependency on GCP Services:** Cloud SCC's capabilities depend on other GCP services being enabled, such as Security Health Analytics and Web Security Scanner, which may increase overall complexity and cost.
## Prowler
Prowler is an open-source, multi-cloud security tool designed to perform detailed security assessments and compliance checks across diverse cloud environments, including AWS, Azure, GCP, and Kubernetes. Here are the key advantages of Prowler when compared to GCP Cloud SCC:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler inherently supports multi-region and multi-account scanning across multiple cloud providers, including GCP, AWS, Azure, and Kubernetes. It does not require additional configuration to perform these scans, making it immediately useful for organizations operating in multiple cloud environments.
- **Minimal Setup Requirements:**
- Prowler requires only appropriate roles and permissions to start scanning. It doesnt necessitate enabling specific services within GCP, which can simplify the setup process and reduce dependencies.
- **Versatile Execution Environment:**
- Prowler can be run from various environments, such as a local workstation, container, Google Cloud Shell, or even other cloud providers by assuming a role. This versatility allows for flexible deployment and integration into existing security operations.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored in an S3 bucket for AWS, Google Cloud Storage (GCS) for GCP, or locally, allowing for quick analysis and easy sharing. This flexibility is particularly advantageous for multi-cloud security assessments and collaborative security processes.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. These reports can be tailored to specific needs and easily integrated with other security tools or dashboards, providing comprehensive insights across all cloud environments.
- **SIEM Integration and Cost Efficiency:**
- Prowler can be configured to send findings directly into SIEM systems, including those integrated with GCP or other platforms. By sending only failed findings or selected results, Prowler helps manage costs associated with data ingestion and analysis in SIEM platforms.
- **Custom Checks and Compliance Frameworks:**
- Prowler allows for the creation of custom security checks, remediation actions, and compliance frameworks, providing flexibility that can be adapted to the unique security policies and regulatory requirements of an organization.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box, with capabilities to extend these frameworks to GCP environments as well as other cloud platforms. This broad compliance coverage ensures that organizations can maintain adherence to various regulatory requirements.
- **Kubernetes and Multi-Cloud Support:**
- Prowler is designed to support security assessments across cloud environments, including Kubernetes clusters and GCP. This multi-cloud capability is essential for organizations that operate across diverse cloud platforms and require consistent security posture management.
- **All-Region Checks:**
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage across an organizations cloud resources, regardless of the region or cloud provider.
## Comparison Summary
### Scope and Environment
- **GCP Cloud SCC** is ideal for organizations primarily using GCP, offering a centralized platform for managing security and compliance within the GCP ecosystem.
- **Prowler** excels in multi-cloud environments, offering flexibility and comprehensive security checks across AWS, Azure, GCP, and Kubernetes without being confined to a single cloud provider.
### Setup and Flexibility
- **GCP Cloud SCC** requires enabling various GCP services and may involve more complex setup, especially for multi-region or multi-account scenarios within GCP.
- **Prowler** requires minimal setup and can be deployed quickly across different cloud environments, offering a more straightforward approach to multi-cloud security management.
### Customization and Compliance
- **GCP Cloud SCC** provides predefined compliance checks within the GCP environment but may require additional tools or customization for broader or more specific requirements.
- **Prowler** allows for extensive customization of security checks, compliance frameworks, and reporting, providing a flexible solution that can be tailored to an organizations specific needs across various cloud platforms.
### Cost Efficiency
- **GCP Cloud SCC** costs can scale with the volume of data processed and the number of enabled services, which may be significant in large or complex environments. SCC pricing is confusing to understand, and starts at $0.071 per vCPU hour for some tiers and depending on the scan service. Take a look at the pricing model [here](https://cloud.google.com/security-command-center/pricing), godspeed.
- **Prowler** helps manage costs by allowing selective reporting, such as sending only failed findings to SIEMs, and storing results in cost-effective ways, such as local storage or cloud buckets. Prowler is always $0.001 per resource per day - no per account charge.
### Multi-Cloud and Multi-Region Support
- **GCP Cloud SCC** is focused on GCP and may require additional tools for comprehensive multi-cloud support.
- **Prowler** is inherently multi-cloud, supporting AWS, Azure, GCP, and Kubernetes out of the box, making it an ideal choice for organizations with diverse cloud footprints.
## Conclusion
For a CISO evaluating these tools, the decision between GCP Cloud Security Command Center (Cloud SCC) and Prowler hinges on the organizations cloud strategy, security management needs, and the level of flexibility and multi-cloud support required:
- If the organization is heavily invested in GCP and needs a centralized platform that integrates seamlessly with GCP services for asset management, threat detection, and compliance monitoring, **GCP Cloud SCC** is likely the better choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
-10
View File
@@ -1,10 +0,0 @@
---
title: 'Comparison'
---
Click to learn more about each cloud security provider and learn how Prowler is differentiated.
- [AWS Security Hub](/getting-started/comparison/awssecurityhub)
- [Microsoft Sentinel](/getting-started/comparison/microsoftsentinel)
- [Microsoft Defender](/getting-started/comparison/microsoftdefender)
- [Google Cloud Security Command Center](/getting-started/comparison/gcp)
@@ -1,101 +0,0 @@
---
title: 'Microsoft Defender for Cloud'
---
**Use open-source scanning to validate and extend Microsoft Defender for Cloud**
---
## **Overview**
If you're using Microsoft Defender for Cloud to monitor your Azure infrastructure, Prowler can complement it with fully transparent, customizable scans across Azure, AWS, GCP, and Kubernetes. Prowler helps you validate policies, automate compliance, and gain deeper visibility—all from the CLI, API or our Prowler UI.
You can run Prowler alongside Defender for Cloud to:
* Double-check security posture with open-source checks.
* Customize rules for your organizations policies.
* Bring your own, or community contributed policies.
* Automate multi-cloud scans in CI/CD or scheduled jobs.
---
## **Why use Prowler with Defender for Cloud**
Microsoft Defender for Cloud offers centralized dashboards, alerting, and some cross-cloud coverage. Prowler provides full transparency and control over whats being checked and how those checks work—no vendor lock-in, no surprises.
Use them together to get:
* More confidence in your security posture
* Checks you can inspect, modify, and version
* CLI-first, portable scanning across clouds
* Open-source tooling that integrates easily into pipelines and audits
---
## **Quickstart**
Heres how to install Prowler and run a scan in your Azure account.
### **1\. Install Prowler**
```
git clone https://github.com/prowler-cloud/prowler
cd prowler
./install.sh
```
### **2\. Authenticate with Azure**
Make sure you're signed in and select your subscription:
```
az login
export AZURE_SUBSCRIPTION_ID=$(az account show --query id -o tsv)
```
### **3\. Run a scan**
```
./prowler -p Azure -f az-aks -f az-general
```
This will run checks focused on Azure Kubernetes Service (AKS) and general Azure best practices.
### **4\. Review results**
```
cat output/prowler-output-*.json
open output/prowler-output-*.html
```
You can export findings in JSON, CSV, JUnit, HTML, or AWS Security Hubcompatible formats.
---
## **Compare capabilities**
| Feature | Microsoft Defender for Cloud | Prowler |
| ----- | ----- | ----- |
| Azure-native posture management | ✅ | ✅ |
| AWS, GCP, and Kubernetes support | ⚠️ (limited) | ✅ |
| Custom policy creation | ❌ | ✅ |
| CLI-first, scriptable | ❌ | ✅ |
| Open source | ❌ | ✅ |
| Compliance mappings (CIS, NIST, etc.) | ✅ (limited control) | ✅ (customizable) |
| Exportable detections | ❌ | ✅ |
---
## **Common use cases**
**✅ Validate policies**
Run Prowler to confirm your Azure policies are configured as expected and compliant with frameworks like CIS or NIST.
**✅ Automate compliance scans**
Schedule regular Prowler scans in your CI/CD pipeline or infrastructure monitoring workflows. Generate reports for auditors or internal reviews.
**✅ Extend detection coverage**
If Defender for Cloud doesnt cover all the services or resources in your environment, Prowlers checks fill in the gaps.
**✅ Build custom checks**
Security is never one-size-fits-all. Prowler lets you write your own checks for organization-specific policies.
@@ -1,93 +0,0 @@
---
title: 'Microsoft Sentinel'
---
Microsoft Sentinel is a scalable, cloud-native security information and event management (SIEM) and security orchestration automated response (SOAR) solution. It's designed to collect, detect, investigate, and respond to threats across the enterprise, primarily within the Azure cloud environment but also extending to on-premises and other cloud environments through various connectors.
## Key Features and Strengths
- **SIEM and SOAR Capabilities:** Microsoft Sentinel combines SIEM and SOAR functionalities, allowing it to collect and analyze large volumes of data from various sources and automate responses to detected threats.
- **Native Azure Integration:** As part of the Azure ecosystem, Sentinel integrates seamlessly with Azure services, providing deep visibility and analytics for Azure resources.
- **Advanced Threat Detection:** Sentinel uses AI and machine learning to detect potential threats and anomalous activities, leveraging Microsoft's extensive threat intelligence network.
- **Scalability and Flexibility:** Being cloud-native, Sentinel scales automatically to handle increasing data volumes and complexity without requiring extensive infrastructure management.
- **Customizable Dashboards and Analytics:** Sentinel offers customizable dashboards and analytics, allowing security teams to tailor their views and queries to specific needs.
- **Multi-Source Data Ingestion:** While focused on Azure, Sentinel can ingest data from multiple sources, including AWS, GCP, on-premises environments, and third-party security products.
## Limitations
- **Azure-Centric:** While it supports multi-cloud environments, its primary focus and strengths are within the Azure ecosystem. Integration with other cloud platforms and on-premises environments may require additional connectors and configurations.
- **Cost Considerations:** As a SIEM tool, Sentinel can become expensive, particularly as data ingestion and analysis volumes grow. The cost model is based on data volume, which can add up quickly in large environments.
- **Complexity in Customization:** Although Sentinel offers advanced customization, setting up and fine-tuning these customizations can require significant expertise and effort, particularly in multi-cloud environments.
## Prowler
Prowler is an open-source, multi-cloud security tool that offers extensive flexibility and customization, making it ideal for organizations that need to maintain a strong security posture across diverse cloud environments. Here are the key advantages of Prowler, particularly when compared to Microsoft Sentinel:
## Main Advantages of Prowler
- **Multi-Region and Multi-Account Scanning by Default:**
- Prowler is inherently multi-region and multi-account, requiring no additional configuration to scan across these environments. This capability is available out of the box without needing to enable specific services or create complex setups.
- **Minimal Setup Requirements:**
- Prowler requires only a role with appropriate permissions to begin scanning. Theres no need for extensive setup, making it easier and quicker to deploy across various environments.
- **Versatile Execution Environment:**
- Prowler can be run from a local workstation, container, AWS CloudShell, or even from other cloud providers like Azure or GCP by assuming a role. This versatility allows security teams to integrate Prowler into a wide range of operational workflows without being tied to a single cloud environment.
- **Flexible Results Storage and Sharing:**
- Prowler results can be stored directly into an S3 bucket, allowing for quick analysis or locally for easy sharing and collaboration. This flexibility is particularly useful for multi-cloud security assessments and incident response.
- **Customizable Reporting and Analysis:**
- Prowler supports exporting results in multiple formats, including JSON, CSV, OCSF format, and static HTML reports. Additionally, it can integrate with Amazon QuickSight for advanced analytics, and offers a SaaS model with resource-based pricing, making it adaptable to various organizational needs.
- **SIEM Integration and Cost Efficiency:**
- While Microsoft Sentinel has a built-in SIEM functionality, Prowler can send results directly into SIEM systems, including Microsoft Sentinel. By sending only failed findings, Prowler can help optimize costs associated with data ingestion and storage in SIEM platforms.
- **Custom Checks and Compliance Frameworks:**
- Prowler enables users to write custom checks, remediations, and compliance frameworks quickly, allowing organizations to adapt the tool to their specific security policies and regulatory requirements.
- **Extensive Compliance Support:**
- Prowler supports over 27 compliance frameworks out of the box, providing comprehensive coverage for AWS environments, which can be extended to multi-cloud scenarios.
- **Kubernetes and Multi-Cloud Support:**
- Prowler is designed to support security assessments beyond AWS, including Kubernetes clusters (including EKS) and environments in Azure and GCP. This capability is critical for organizations that operate across multiple cloud platforms and require consistent security posture management.
- **All-Region Checks:**
- Prowler runs all checks in all regions by default, ensuring comprehensive coverage without the limitations that may be imposed by region-specific configurations or services.
## Comparison Summary
### Scope and Environment
- **Microsoft Sentinel** is an advanced SIEM/SOAR tool optimized for Azure environments, with support for multi-cloud and on-premises systems through connectors.
- **Prowler** is a flexible, multi-cloud security tool that excels in environments where organizations need to manage security across AWS, Azure, GCP, and Kubernetes with minimal setup and high customizability.
### Setup and Flexibility
- **Microsoft Sentinel** requires more setup, especially when integrating with non-Azure environments, and its cost scales with data ingestion.
- **Prowler** requires minimal setup and can be easily deployed in any cloud or on-premises environment. Its ability to run from various environments and store results in flexible formats makes it particularly adaptable.
### Customization and Compliance
- **Microsoft Sentinel** offers powerful but complex customization options, primarily within the Azure ecosystem.
- **Prowler** provides straightforward customization of security checks, remediation actions, and compliance frameworks, with broad support for multiple compliance standards out of the box.
### Cost Efficiency
- **Microsoft Sentinel** can become costly as data volumes grow, particularly in large or multi-cloud environments.
- **Prowler** helps control costs by enabling selective reporting (e.g., sending only failed findings to SIEMs like Sentinel) and storing results in cost-effective ways, such as S3 or locally.
### Multi-Cloud and Multi-Region Support
- **Microsoft Sentinel** supports multi-cloud environments but may require additional configuration and connectors.
- **Prowler** is designed for multi-cloud environments from the ground up, with inherent support for AWS, Azure, GCP, Kubernetes, and all regions, making it an ideal tool for organizations with diverse cloud footprints.
## Conclusion
For a CISO or security professional evaluating these tools, the decision between Microsoft Sentinel and Prowler hinges on the organization's cloud strategy, SIEM needs, and the level of customization and flexibility required:
- If the organization is heavily invested in Azure and needs an integrated SIEM/SOAR solution with advanced threat detection, analytics, and automation capabilities, **Microsoft Sentinel** is likely the better choice.
- If the organization operates in a multi-cloud environment or requires a highly customizable tool for performing detailed security assessments across AWS, Azure, GCP, and Kubernetes, **Prowler** offers a more flexible and cost-effective solution, especially for those needing quick deployment, minimal setup, and the ability to manage security across diverse cloud environments.
@@ -1,4 +0,0 @@
---
title: "Prowler API Reference"
url: "https://api.prowler.com/api/v1/docs"
---

Some files were not shown because too many files have changed in this diff Show More