mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-05-13 15:50:55 +00:00
Compare commits
83 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 77089eba57 | |||
| f831171a21 | |||
| 2740d73fe7 | |||
| 1c906b37cd | |||
| 98056b7c85 | |||
| f15ef0d16c | |||
| c42ce6242f | |||
| 702d652de1 | |||
| fff02073cf | |||
| 23e3ea4a41 | |||
| f9afb50ed9 | |||
| 3b95aad6ce | |||
| ac5737d8c4 | |||
| a452c8c3eb | |||
| aa8be0b2fe | |||
| 46bf8e0fef | |||
| c0df0cd1a8 | |||
| 80d58a7b50 | |||
| 2c28d74598 | |||
| 4feab1be55 | |||
| 5bc9b09490 | |||
| fcf817618a | |||
| cad97f25ac | |||
| b854563854 | |||
| 573975f3fe | |||
| f4081f92a1 | |||
| 374496e7ff | |||
| 2a9c2b926d | |||
| f2f1e6bce6 | |||
| 25c823076f | |||
| 6ff559c0d4 | |||
| 899db55f56 | |||
| 22d801ade2 | |||
| 1dc6d41198 | |||
| 456712a0ef | |||
| 885ee62062 | |||
| bbeccaf085 | |||
| d1aca5641a | |||
| 3b7eba64aa | |||
| e9e0797642 | |||
| aaa5abdead | |||
| 0a2749b716 | |||
| 8f8bf63086 | |||
| ea27817a2c | |||
| 9068e6bcd0 | |||
| a4907d8098 | |||
| caee7830a5 | |||
| 65d2989bea | |||
| 6c34945829 | |||
| ce859ddd1f | |||
| 0ca059b45b | |||
| dad100b87a | |||
| 662296aa0e | |||
| b6d49416f0 | |||
| 42be77e82e | |||
| 63169289b0 | |||
| 43d310356d | |||
| 59ae503681 | |||
| bd62f56df4 | |||
| 90fbad16b9 | |||
| affd0c5ffb | |||
| 929bbe3550 | |||
| eb7ef4a8b9 | |||
| 017e19ac18 | |||
| be7680786a | |||
| efba5d2a8d | |||
| 44431a56de | |||
| 969ca8863a | |||
| 03c6f98db4 | |||
| 8ebefb8aa1 | |||
| c3694fdc5b | |||
| df10bc0c4c | |||
| e694b0f634 | |||
| 81e3f87003 | |||
| 7ffe2aeec9 | |||
| 672aa6eb2f | |||
| 2e999f55f9 | |||
| 18998b8867 | |||
| ff4a186df6 | |||
| b8dab5e0ed | |||
| 0b3142f7a8 | |||
| f5dc0c9ee0 | |||
| a230809095 |
@@ -22,8 +22,8 @@ inputs:
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Replace @master with current branch in pyproject.toml
|
||||
if: github.event_name == 'pull_request' && github.base_ref == 'master'
|
||||
- name: Replace @master with current branch in pyproject.toml (prowler repo only)
|
||||
if: github.event_name == 'pull_request' && github.base_ref == 'master' && github.repository == 'prowler-cloud/prowler'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
@@ -37,8 +37,8 @@ runs:
|
||||
python -m pip install --upgrade pip
|
||||
pipx install poetry==${{ inputs.poetry-version }}
|
||||
|
||||
- name: Update SDK resolved_reference to latest commit
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
|
||||
- name: Update poetry.lock with latest Prowler commit
|
||||
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
@@ -50,7 +50,21 @@ runs:
|
||||
echo "Updated resolved_reference:"
|
||||
grep -A2 -B2 "resolved_reference" poetry.lock
|
||||
|
||||
- name: Update poetry.lock
|
||||
- name: Update SDK resolved_reference to latest commit (prowler repo on push)
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/master' && github.repository == 'prowler-cloud/prowler'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
|
||||
echo "Latest commit hash: $LATEST_COMMIT"
|
||||
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
|
||||
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
|
||||
}' poetry.lock
|
||||
echo "Updated resolved_reference:"
|
||||
grep -A2 -B2 "resolved_reference" poetry.lock
|
||||
|
||||
- name: Update poetry.lock (prowler repo only)
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: poetry lock
|
||||
@@ -69,3 +83,11 @@ runs:
|
||||
run: |
|
||||
poetry install --no-root
|
||||
poetry run pip list
|
||||
|
||||
- name: Update Prowler Cloud API Client
|
||||
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
poetry remove prowler-cloud-api-client
|
||||
poetry add ./prowler-cloud-api-client
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
# Slack Notification Action
|
||||
|
||||
A generic and flexible GitHub composite action for sending Slack notifications using JSON template files. Supports both standalone messages and message updates, with automatic status detection.
|
||||
|
||||
## Features
|
||||
|
||||
- **Template-based**: All messages use JSON template files for consistency
|
||||
- **Automatic status detection**: Pass `step-outcome` to auto-calculate success/failure
|
||||
- **Message updates**: Supports updating existing messages (using `chat.update`)
|
||||
- **Simple API**: Clean and minimal interface
|
||||
- **Reusable**: Use across all workflows and scenarios
|
||||
- **Maintainable**: Centralized message templates
|
||||
|
||||
## Use Cases
|
||||
|
||||
1. **Container releases**: Track push start and completion with automatic status
|
||||
2. **Deployments**: Track deployment progress with rich Block Kit formatting
|
||||
3. **Custom notifications**: Any scenario where you need to notify Slack
|
||||
|
||||
## Inputs
|
||||
|
||||
| Input | Description | Required | Default |
|
||||
|-------|-------------|----------|---------|
|
||||
| `slack-bot-token` | Slack bot token for authentication | Yes | - |
|
||||
| `payload-file-path` | Path to JSON file with the Slack message payload | Yes | - |
|
||||
| `update-ts` | Message timestamp to update (leave empty for new messages) | No | `''` |
|
||||
| `step-outcome` | Step outcome for automatic status detection (sets STATUS_EMOJI and STATUS_TEXT env vars) | No | `''` |
|
||||
|
||||
## Outputs
|
||||
|
||||
| Output | Description |
|
||||
|--------|-------------|
|
||||
| `ts` | Timestamp of the Slack message (use for updates) |
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Example 1: Container Release with Automatic Status Detection
|
||||
|
||||
Using JSON template files with automatic status detection:
|
||||
|
||||
```yaml
|
||||
# Send start notification
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
# Do the work
|
||||
- name: Build and push container
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@...
|
||||
with:
|
||||
push: true
|
||||
tags: ...
|
||||
|
||||
# Send completion notification with automatic status detection
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
step-outcome: ${{ steps.container-push.outcome }}
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- No status calculation needed in workflow
|
||||
- Reusable template files
|
||||
- Clean and concise
|
||||
- Automatic `STATUS_EMOJI` and `STATUS_TEXT` env vars set by action
|
||||
- Consistent message format across all workflows
|
||||
|
||||
### Example 2: Deployment with Message Update Pattern
|
||||
|
||||
```yaml
|
||||
# Send initial deployment message
|
||||
- name: Notify deployment started
|
||||
id: slack-start
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
ENVIRONMENT: PRODUCTION
|
||||
COMMIT_HASH: ${{ github.sha }}
|
||||
VERSION_DEPLOYED: latest
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
GITHUB_WORKFLOW: ${{ github.workflow }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/deployment-started.json"
|
||||
|
||||
# Run deployment
|
||||
- name: Deploy
|
||||
id: deploy
|
||||
run: terraform apply -auto-approve
|
||||
|
||||
# Determine additional status variables
|
||||
- name: Determine deployment status
|
||||
if: always()
|
||||
id: deploy-status
|
||||
run: |
|
||||
if [[ "${{ steps.deploy.outcome }}" == "success" ]]; then
|
||||
echo "STATUS_COLOR=28a745" >> $GITHUB_ENV
|
||||
echo "STATUS=Completed" >> $GITHUB_ENV
|
||||
else
|
||||
echo "STATUS_COLOR=fc3434" >> $GITHUB_ENV
|
||||
echo "STATUS=Failed" >> $GITHUB_ENV
|
||||
fi
|
||||
|
||||
# Update the same message with final status
|
||||
- name: Update deployment notification
|
||||
if: always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
MESSAGE_TS: ${{ steps.slack-start.outputs.ts }}
|
||||
COMPONENT: API
|
||||
ENVIRONMENT: PRODUCTION
|
||||
COMMIT_HASH: ${{ github.sha }}
|
||||
VERSION_DEPLOYED: latest
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
GITHUB_WORKFLOW: ${{ github.workflow }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
STATUS: ${{ env.STATUS }}
|
||||
STATUS_COLOR: ${{ env.STATUS_COLOR }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
update-ts: ${{ steps.slack-start.outputs.ts }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/deployment-completed.json"
|
||||
step-outcome: ${{ steps.deploy.outcome }}
|
||||
```
|
||||
|
||||
## Automatic Status Detection
|
||||
|
||||
When you provide `step-outcome` input, the action automatically sets these environment variables:
|
||||
|
||||
| Outcome | STATUS_EMOJI | STATUS_TEXT |
|
||||
|---------|--------------|-------------|
|
||||
| success | `[✓]` | `completed successfully!` |
|
||||
| failure | `[✗]` | `failed` |
|
||||
|
||||
These variables are then available in your payload template files.
|
||||
|
||||
## Template File Format
|
||||
|
||||
All template files must be valid JSON and support environment variable substitution. Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"channel": "$SLACK_CHANNEL_ID",
|
||||
"text": "$STATUS_EMOJI $COMPONENT container release $RELEASE_TAG push $STATUS_TEXT <$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|View run>"
|
||||
}
|
||||
```
|
||||
|
||||
See available templates in [`.github/scripts/slack-messages/`](../../scripts/slack-messages/).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Slack Bot Token with scopes: `chat:write`, `chat:write.public`
|
||||
- Slack Channel ID where messages will be posted
|
||||
- JSON template files for your messages
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Consistency**: All notifications use standardized templates
|
||||
- **Automatic status handling**: No need to calculate success/failure in workflows
|
||||
- **Clean workflows**: Minimal boilerplate code
|
||||
- **Reusable templates**: One template for all components
|
||||
- **Easy to maintain**: Change template once, applies everywhere
|
||||
- **Version controlled**: All message formats in git
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [Slack Block Kit Builder](https://app.slack.com/block-kit-builder)
|
||||
- [Slack API Method Documentation](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
|
||||
- [Message templates documentation](../../scripts/slack-messages/README.md)
|
||||
@@ -0,0 +1,68 @@
|
||||
name: 'Slack Notification'
|
||||
description: 'Generic action to send Slack notifications with optional message updates and automatic status detection'
|
||||
inputs:
|
||||
slack-bot-token:
|
||||
description: 'Slack bot token for authentication'
|
||||
required: true
|
||||
payload-file-path:
|
||||
description: 'Path to JSON file with the Slack message payload'
|
||||
required: true
|
||||
update-ts:
|
||||
description: 'Message timestamp to update (only for updates, leave empty for new messages)'
|
||||
required: false
|
||||
default: ''
|
||||
step-outcome:
|
||||
description: 'Outcome of a step to determine status (success/failure) - automatically sets STATUS_EMOJI, STATUS_TEXT, and STATUS_COLOR env vars'
|
||||
required: false
|
||||
default: ''
|
||||
outputs:
|
||||
ts:
|
||||
description: 'Timestamp of the Slack message'
|
||||
value: ${{ steps.slack-notification.outputs.ts }}
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Determine status
|
||||
id: status
|
||||
shell: bash
|
||||
run: |
|
||||
if [[ "${{ inputs.step-outcome }}" == "success" ]]; then
|
||||
echo "STATUS_EMOJI=[✓]" >> $GITHUB_ENV
|
||||
echo "STATUS_TEXT=succeeded" >> $GITHUB_ENV
|
||||
echo "STATUS_COLOR=6aa84f" >> $GITHUB_ENV
|
||||
elif [[ "${{ inputs.step-outcome }}" == "failure" ]]; then
|
||||
echo "STATUS_EMOJI=[✗]" >> $GITHUB_ENV
|
||||
echo "STATUS_TEXT=failed" >> $GITHUB_ENV
|
||||
echo "STATUS_COLOR=fc3434" >> $GITHUB_ENV
|
||||
else
|
||||
# No outcome provided - pending/in progress state
|
||||
echo "STATUS_COLOR=dbab09" >> $GITHUB_ENV
|
||||
fi
|
||||
|
||||
- name: Send Slack notification (new message)
|
||||
if: inputs.update-ts == ''
|
||||
id: slack-notification-post
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
with:
|
||||
method: chat.postMessage
|
||||
token: ${{ inputs.slack-bot-token }}
|
||||
payload-file-path: ${{ inputs.payload-file-path }}
|
||||
|
||||
- name: Update Slack notification
|
||||
if: inputs.update-ts != ''
|
||||
id: slack-notification-update
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
with:
|
||||
method: chat.update
|
||||
token: ${{ inputs.slack-bot-token }}
|
||||
payload-file-path: ${{ inputs.payload-file-path }}
|
||||
|
||||
- name: Set output
|
||||
id: slack-notification
|
||||
shell: bash
|
||||
run: |
|
||||
if [[ "${{ inputs.update-ts }}" == "" ]]; then
|
||||
echo "ts=${{ steps.slack-notification-post.outputs.ts }}" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "ts=${{ inputs.update-ts }}" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
@@ -45,22 +45,13 @@ outputs:
|
||||
runs:
|
||||
using: 'composite'
|
||||
steps:
|
||||
- name: Run Trivy vulnerability scan (SARIF)
|
||||
if: inputs.upload-sarif == 'true'
|
||||
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
|
||||
- name: Cache Trivy vulnerability database
|
||||
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
|
||||
with:
|
||||
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
|
||||
format: 'sarif'
|
||||
output: 'trivy-results.sarif'
|
||||
severity: 'CRITICAL,HIGH'
|
||||
exit-code: '0'
|
||||
|
||||
- name: Upload Trivy results to GitHub Security tab
|
||||
if: inputs.upload-sarif == 'true'
|
||||
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
sarif_file: 'trivy-results.sarif'
|
||||
category: 'trivy-container'
|
||||
path: ~/.cache/trivy
|
||||
key: trivy-db-${{ runner.os }}-${{ github.run_id }}
|
||||
restore-keys: |
|
||||
trivy-db-${{ runner.os }}-
|
||||
|
||||
- name: Run Trivy vulnerability scan (JSON)
|
||||
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
|
||||
@@ -70,6 +61,27 @@ runs:
|
||||
output: 'trivy-report.json'
|
||||
severity: ${{ inputs.severity }}
|
||||
exit-code: '0'
|
||||
scanners: 'vuln'
|
||||
timeout: '5m'
|
||||
|
||||
- name: Run Trivy vulnerability scan (SARIF)
|
||||
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
|
||||
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
|
||||
with:
|
||||
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
|
||||
format: 'sarif'
|
||||
output: 'trivy-results.sarif'
|
||||
severity: 'CRITICAL,HIGH'
|
||||
exit-code: '0'
|
||||
scanners: 'vuln'
|
||||
timeout: '5m'
|
||||
|
||||
- name: Upload Trivy results to GitHub Security tab
|
||||
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
|
||||
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
sarif_file: 'trivy-results.sarif'
|
||||
category: 'trivy-container'
|
||||
|
||||
- name: Upload Trivy report artifact
|
||||
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
|
||||
@@ -109,20 +121,20 @@ runs:
|
||||
with:
|
||||
script: |
|
||||
const comment = require('./.github/scripts/trivy-pr-comment.js');
|
||||
|
||||
|
||||
// Unique identifier to find our comment
|
||||
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
|
||||
const body = marker + '\n' + comment;
|
||||
|
||||
|
||||
// Find existing comment
|
||||
const { data: comments } = await github.rest.issues.listComments({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
});
|
||||
|
||||
|
||||
const existingComment = comments.find(c => c.body?.includes(marker));
|
||||
|
||||
|
||||
if (existingComment) {
|
||||
// Update existing comment
|
||||
await github.rest.issues.updateComment({
|
||||
|
||||
@@ -1,4 +1,18 @@
|
||||
name: "SDK - CodeQL Config"
|
||||
name: 'SDK: CodeQL Config'
|
||||
paths:
|
||||
- 'prowler/'
|
||||
|
||||
paths-ignore:
|
||||
- "api/"
|
||||
- "ui/"
|
||||
- 'api/'
|
||||
- 'ui/'
|
||||
- 'dashboard/'
|
||||
- 'mcp_server/'
|
||||
- 'tests/**'
|
||||
- 'util/**'
|
||||
- 'contrib/**'
|
||||
- 'examples/**'
|
||||
- 'prowler/**/__pycache__/**'
|
||||
- 'prowler/**/*.md'
|
||||
|
||||
queries:
|
||||
- uses: security-and-quality
|
||||
|
||||
@@ -1,3 +1,17 @@
|
||||
name: "UI - CodeQL Config"
|
||||
name: 'UI: CodeQL Config'
|
||||
paths:
|
||||
- "ui/"
|
||||
- 'ui/'
|
||||
|
||||
paths-ignore:
|
||||
- 'ui/node_modules/**'
|
||||
- 'ui/.next/**'
|
||||
- 'ui/out/**'
|
||||
- 'ui/tests/**'
|
||||
- 'ui/**/*.test.ts'
|
||||
- 'ui/**/*.test.tsx'
|
||||
- 'ui/**/*.spec.ts'
|
||||
- 'ui/**/*.spec.tsx'
|
||||
- 'ui/**/*.md'
|
||||
|
||||
queries:
|
||||
- uses: security-and-quality
|
||||
|
||||
@@ -0,0 +1,462 @@
|
||||
# Slack Message Templates
|
||||
|
||||
This directory contains reusable message templates for Slack notifications sent from GitHub Actions workflows.
|
||||
|
||||
## Usage
|
||||
|
||||
These JSON templates are used with the `slackapi/slack-github-action` using the Slack API method (`chat.postMessage` and `chat.update`). All templates support rich Block Kit formatting and message updates.
|
||||
|
||||
### Available Templates
|
||||
|
||||
**Container Releases**
|
||||
- `container-release-started.json`: Simple one-line notification when container push starts
|
||||
- `container-release-completed.json`: Simple one-line notification when container release completes
|
||||
|
||||
**Deployments**
|
||||
- `deployment-started.json`: Deployment start notification with Block Kit formatting
|
||||
- `deployment-completed.json`: Deployment completion notification (updates the start message)
|
||||
|
||||
All templates use the Slack API method and require a Slack Bot Token.
|
||||
|
||||
## Setup Requirements
|
||||
|
||||
1. Create a Slack App (or use existing)
|
||||
2. Add Bot Token Scopes: `chat:write`, `chat:write.public`
|
||||
3. Install the app to your workspace
|
||||
4. Get the Bot Token from OAuth & Permissions page
|
||||
5. Add secrets:
|
||||
- `SLACK_BOT_TOKEN`: Your bot token
|
||||
- `SLACK_CHANNEL_ID`: The channel ID where messages will be posted
|
||||
|
||||
Reference: [Sending data using a Slack API method](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### Required Secrets (GitHub Secrets)
|
||||
- `SLACK_BOT_TOKEN`: Passed as `token` parameter to the action (not as env variable)
|
||||
- `SLACK_CHANNEL_ID`: Used in payload as env variable
|
||||
|
||||
### Container Release Variables (configured as env)
|
||||
- `COMPONENT`: Component name (e.g., "API", "SDK", "UI", "MCP")
|
||||
- `RELEASE_TAG` / `PROWLER_VERSION`: The release tag or version being deployed
|
||||
- `GITHUB_SERVER_URL`: Provided by GitHub context
|
||||
- `GITHUB_REPOSITORY`: Provided by GitHub context
|
||||
- `GITHUB_RUN_ID`: Provided by GitHub context
|
||||
- `STATUS_EMOJI`: Status symbol (calculated: `[✓]` for success, `[✗]` for failure)
|
||||
- `STATUS_TEXT`: Status text (calculated: "completed successfully!" or "failed")
|
||||
|
||||
### Deployment Variables (configured as env)
|
||||
- `COMPONENT`: Component name (e.g., "API", "SDK", "UI", "MCP")
|
||||
- `ENVIRONMENT`: Environment name (e.g., "DEVELOPMENT", "PRODUCTION")
|
||||
- `COMMIT_HASH`: Commit hash being deployed
|
||||
- `VERSION_DEPLOYED`: Version being deployed
|
||||
- `GITHUB_ACTOR`: User who triggered the workflow
|
||||
- `GITHUB_WORKFLOW`: Workflow name
|
||||
- `GITHUB_SERVER_URL`: Provided by GitHub context
|
||||
- `GITHUB_REPOSITORY`: Provided by GitHub context
|
||||
- `GITHUB_RUN_ID`: Provided by GitHub context
|
||||
|
||||
All other variables (MESSAGE_TS, STATUS, STATUS_COLOR, STATUS_EMOJI, etc.) are calculated internally within the workflow and should NOT be configured as environment variables.
|
||||
|
||||
## Example Workflow Usage
|
||||
|
||||
### Using the Generic Slack Notification Action (Recommended)
|
||||
|
||||
**Recommended approach**: Use the generic reusable action `.github/actions/slack-notification` which provides maximum flexibility:
|
||||
|
||||
#### Example 1: Container Release (Start + Completion)
|
||||
|
||||
```yaml
|
||||
# Send start notification
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload: |
|
||||
{
|
||||
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
|
||||
"text": "API container release ${{ env.RELEASE_TAG }} push started... <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
|
||||
}
|
||||
|
||||
# Build and push container
|
||||
- name: Build and push container
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@...
|
||||
with:
|
||||
push: true
|
||||
tags: ...
|
||||
|
||||
# Calculate status
|
||||
- name: Determine push status
|
||||
if: github.event_name == 'release' && always()
|
||||
id: push-status
|
||||
run: |
|
||||
if [[ "${{ steps.container-push.outcome }}" == "success" ]]; then
|
||||
echo "emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
echo "text=completed successfully!" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
echo "text=failed" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# Send completion notification
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload: |
|
||||
{
|
||||
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
|
||||
"text": "${{ steps.push-status.outputs.emoji }} API container release ${{ env.RELEASE_TAG }} push ${{ steps.push-status.outputs.text }} <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
|
||||
}
|
||||
```
|
||||
|
||||
#### Example 2: Simple One-Time Message
|
||||
|
||||
```yaml
|
||||
- name: Send notification
|
||||
uses: ./.github/actions/slack-notification
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload: |
|
||||
{
|
||||
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
|
||||
"text": "Deployment completed successfully!"
|
||||
}
|
||||
```
|
||||
|
||||
#### Example 3: Deployment with Message Update Pattern
|
||||
|
||||
```yaml
|
||||
# Send initial deployment message
|
||||
- name: Notify deployment started
|
||||
id: slack-start
|
||||
uses: ./.github/actions/slack-notification
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload: |
|
||||
{
|
||||
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
|
||||
"text": "API deployment to PRODUCTION started",
|
||||
"attachments": [
|
||||
{
|
||||
"color": "dbab09",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "header",
|
||||
"text": {
|
||||
"type": "plain_text",
|
||||
"text": "API | Deployment to PRODUCTION"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "section",
|
||||
"fields": [
|
||||
{
|
||||
"type": "mrkdwn",
|
||||
"text": "*Status:*\nIn Progress"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
# Run deployment
|
||||
- name: Deploy
|
||||
id: deploy
|
||||
run: terraform apply -auto-approve
|
||||
|
||||
# Calculate status
|
||||
- name: Determine status
|
||||
if: always()
|
||||
id: status
|
||||
run: |
|
||||
if [[ "${{ steps.deploy.outcome }}" == "success" ]]; then
|
||||
echo "color=28a745" >> $GITHUB_OUTPUT
|
||||
echo "emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
echo "status=Completed" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "color=fc3434" >> $GITHUB_OUTPUT
|
||||
echo "emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
echo "status=Failed" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# Update the same message with final status
|
||||
- name: Update deployment notification
|
||||
if: always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
update-ts: ${{ steps.slack-start.outputs.ts }}
|
||||
payload: |
|
||||
{
|
||||
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
|
||||
"ts": "${{ steps.slack-start.outputs.ts }}",
|
||||
"text": "${{ steps.status.outputs.emoji }} API deployment to PRODUCTION ${{ steps.status.outputs.status }}",
|
||||
"attachments": [
|
||||
{
|
||||
"color": "${{ steps.status.outputs.color }}",
|
||||
"blocks": [
|
||||
{
|
||||
"type": "header",
|
||||
"text": {
|
||||
"type": "plain_text",
|
||||
"text": "API | Deployment to PRODUCTION"
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "section",
|
||||
"fields": [
|
||||
{
|
||||
"type": "mrkdwn",
|
||||
"text": "*Status:*\n${{ steps.status.outputs.emoji }} ${{ steps.status.outputs.status }}"
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Benefits of using the generic action:**
|
||||
- Maximum flexibility: Build any payload you need directly in the workflow
|
||||
- No template files needed: Everything inline
|
||||
- Supports all scenarios: one-time messages, start/update patterns, rich Block Kit
|
||||
- Easy to customize per use case
|
||||
- Generic: Works for containers, deployments, or any notification type
|
||||
|
||||
For more details, see [Slack Notification Action](../../actions/slack-notification/README.md).
|
||||
|
||||
### Using Message Templates (Alternative Approach)
|
||||
|
||||
Simple one-line notifications for container releases:
|
||||
|
||||
```yaml
|
||||
# Step 1: Notify when push starts
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
method: chat.postMessage
|
||||
token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
# Step 2: Build and push container
|
||||
- name: Build and push container
|
||||
id: container-push
|
||||
uses: docker/build-push-action@...
|
||||
with:
|
||||
push: true
|
||||
tags: ...
|
||||
|
||||
# Step 3: Determine push status
|
||||
- name: Determine push status
|
||||
if: github.event_name == 'release' && always()
|
||||
id: push-status
|
||||
run: |
|
||||
if [[ "${{ steps.container-push.outcome }}" == "success" ]]; then
|
||||
echo "status-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
echo "status-text=completed successfully!" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
echo "status-text=failed" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# Step 4: Notify when push completes (success or failure)
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
STATUS_EMOJI: ${{ steps.push-status.outputs.status-emoji }}
|
||||
STATUS_TEXT: ${{ steps.push-status.outputs.status-text }}
|
||||
with:
|
||||
method: chat.postMessage
|
||||
token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
```
|
||||
|
||||
### Deployment with Update Pattern
|
||||
|
||||
For deployments that start with one message and update it with the final status:
|
||||
|
||||
```yaml
|
||||
# Step 1: Send deployment start notification
|
||||
- name: Notify Deployment Start
|
||||
id: slack-notification-start
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
COMPONENT: API
|
||||
ENVIRONMENT: PRODUCTION
|
||||
COMMIT_HASH: ${{ github.sha }}
|
||||
VERSION_DEPLOYED: latest
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
GITHUB_WORKFLOW: ${{ github.workflow }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
method: chat.postMessage
|
||||
token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/deployment-started.json"
|
||||
|
||||
# Step 2: Run your deployment steps
|
||||
- name: Terraform Plan
|
||||
id: terraform-plan
|
||||
run: terraform plan
|
||||
|
||||
- name: Terraform Apply
|
||||
id: terraform-apply
|
||||
run: terraform apply -auto-approve
|
||||
|
||||
# Step 3: Determine status (calculated internally, not configured)
|
||||
- name: Determine Status
|
||||
if: always()
|
||||
id: determine-status
|
||||
run: |
|
||||
if [[ "${{ steps.terraform-apply.outcome }}" == "success" ]]; then
|
||||
echo "status=Completed" >> $GITHUB_OUTPUT
|
||||
echo "status-color=28a745" >> $GITHUB_OUTPUT
|
||||
echo "status-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
echo "plan-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
echo "apply-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
elif [[ "${{ steps.terraform-plan.outcome }}" == "failure" || "${{ steps.terraform-apply.outcome }}" == "failure" ]]; then
|
||||
echo "status=Failed" >> $GITHUB_OUTPUT
|
||||
echo "status-color=fc3434" >> $GITHUB_OUTPUT
|
||||
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
if [[ "${{ steps.terraform-plan.outcome }}" == "failure" ]]; then
|
||||
echo "plan-emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "plan-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
if [[ "${{ steps.terraform-apply.outcome }}" == "failure" ]]; then
|
||||
echo "apply-emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
else
|
||||
echo "apply-emoji=[✓]" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
else
|
||||
echo "status=Failed" >> $GITHUB_OUTPUT
|
||||
echo "status-color=fc3434" >> $GITHUB_OUTPUT
|
||||
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
|
||||
echo "plan-emoji=[?]" >> $GITHUB_OUTPUT
|
||||
echo "apply-emoji=[?]" >> $GITHUB_OUTPUT
|
||||
fi
|
||||
|
||||
# Step 4: Update the same Slack message (using calculated values)
|
||||
- name: Notify Deployment Result
|
||||
if: always()
|
||||
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
|
||||
MESSAGE_TS: ${{ steps.slack-notification-start.outputs.ts }}
|
||||
COMPONENT: API
|
||||
ENVIRONMENT: PRODUCTION
|
||||
COMMIT_HASH: ${{ github.sha }}
|
||||
VERSION_DEPLOYED: latest
|
||||
GITHUB_ACTOR: ${{ github.actor }}
|
||||
GITHUB_WORKFLOW: ${{ github.workflow }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
STATUS: ${{ steps.determine-status.outputs.status }}
|
||||
STATUS_COLOR: ${{ steps.determine-status.outputs.status-color }}
|
||||
STATUS_EMOJI: ${{ steps.determine-status.outputs.status-emoji }}
|
||||
PLAN_EMOJI: ${{ steps.determine-status.outputs.plan-emoji }}
|
||||
APPLY_EMOJI: ${{ steps.determine-status.outputs.apply-emoji }}
|
||||
TERRAFORM_PLAN_OUTCOME: ${{ steps.terraform-plan.outcome }}
|
||||
TERRAFORM_APPLY_OUTCOME: ${{ steps.terraform-apply.outcome }}
|
||||
with:
|
||||
method: chat.update
|
||||
token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/deployment-completed.json"
|
||||
```
|
||||
|
||||
**Note**: Variables like `STATUS`, `STATUS_COLOR`, `STATUS_EMOJI`, `PLAN_EMOJI`, `APPLY_EMOJI` are calculated by the `determine-status` step based on the outcomes of previous steps. They should NOT be manually configured.
|
||||
|
||||
## Key Features
|
||||
|
||||
### Benefits of Using Slack API Method
|
||||
|
||||
- **Rich Block Kit Formatting**: Full support for Slack's Block Kit including headers, sections, fields, colors, and attachments
|
||||
- **Message Updates**: Update the same message instead of posting multiple messages (using `chat.update` with `ts`)
|
||||
- **Consistent Experience**: Same look and feel as Prowler Cloud notifications
|
||||
- **Flexible**: Easy to customize message appearance by editing JSON templates
|
||||
|
||||
### Differences from Webhook Method
|
||||
|
||||
| Feature | webhook-trigger | Slack API (chat.postMessage) |
|
||||
|---------|-----------------|------------------------------|
|
||||
| Setup | Workflow Builder webhook | Slack Bot Token + Channel ID |
|
||||
| Formatting | Plain text/simple | Full Block Kit support |
|
||||
| Message Update | No | Yes (with chat.update) |
|
||||
| Authentication | Webhook URL | Bot Token |
|
||||
| Scopes Required | None | chat:write, chat:write.public |
|
||||
|
||||
## Message Appearance
|
||||
|
||||
### Container Release (Simple One-Line)
|
||||
|
||||
**Start message:**
|
||||
```
|
||||
API container release 4.5.0 push started... View run
|
||||
```
|
||||
|
||||
**Completion message (success):**
|
||||
```
|
||||
[✓] API container release 4.5.0 push completed successfully! View run
|
||||
```
|
||||
|
||||
**Completion message (failure):**
|
||||
```
|
||||
[✗] API container release 4.5.0 push failed View run
|
||||
```
|
||||
|
||||
All messages are simple one-liners with a clickable "View run" link. The completion message adapts to show success `[✓]` or failure `[✗]` based on the outcome of the container push.
|
||||
|
||||
### Deployment Start
|
||||
- Header: Component and environment
|
||||
- Yellow bar (color: `dbab09`)
|
||||
- Status: In Progress
|
||||
- Details: Commit, version, actor, workflow
|
||||
- Link: Direct link to deployment run
|
||||
|
||||
### Deployment Completion
|
||||
- Header: Component and environment
|
||||
- Green bar for success (color: `28a745`) / Red bar for failure (color: `fc3434`)
|
||||
- Status: [✓] Completed or [✗] Failed
|
||||
- Details: All deployment info plus terraform outcomes
|
||||
- Link: Direct link to deployment run
|
||||
|
||||
## Adding New Templates
|
||||
|
||||
1. Create a new JSON file with Block Kit structure
|
||||
2. Use environment variable placeholders (e.g., `$VAR_NAME`)
|
||||
3. Include `channel` and `text` fields (required)
|
||||
4. Add `blocks` or `attachments` for rich formatting
|
||||
5. For update templates, include `ts` field as `$MESSAGE_TS`
|
||||
6. Document the template in this README
|
||||
7. Reference it in your workflow using `payload-file-path`
|
||||
|
||||
## Reference
|
||||
|
||||
- [Slack Block Kit Builder](https://app.slack.com/block-kit-builder)
|
||||
- [Slack API Method Documentation](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"channel": "$SLACK_CHANNEL_ID",
|
||||
"text": "$STATUS_EMOJI $COMPONENT container release $RELEASE_TAG push $STATUS_TEXT <$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|View run>"
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"channel": "$SLACK_CHANNEL_ID",
|
||||
"text": "$COMPONENT container release $RELEASE_TAG push started... <$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|View run>"
|
||||
}
|
||||
@@ -1,115 +0,0 @@
|
||||
name: API - Build and Push containers
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
paths:
|
||||
- "api/**"
|
||||
- "prowler/**"
|
||||
- ".github/workflows/api-build-lint-push-containers.yml"
|
||||
|
||||
# Uncomment the code below to test this action on PRs
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - "master"
|
||||
# paths:
|
||||
# - "api/**"
|
||||
# - ".github/workflows/api-build-lint-push-containers.yml"
|
||||
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
env:
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
STABLE_TAG: stable
|
||||
|
||||
WORKING_DIRECTORY: ./api
|
||||
|
||||
# Container Registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-api
|
||||
|
||||
jobs:
|
||||
repository-check:
|
||||
name: Repository check
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
is_repo: ${{ steps.repository_check.outputs.is_repo }}
|
||||
steps:
|
||||
- name: Repository check
|
||||
id: repository_check
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
|
||||
then
|
||||
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
|
||||
else
|
||||
echo "This action only runs for prowler-cloud/prowler"
|
||||
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
|
||||
fi
|
||||
|
||||
# Build Prowler OSS container
|
||||
container-build-push:
|
||||
needs: repository-check
|
||||
if: needs.repository-check.outputs.is_repo == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ env.WORKING_DIRECTORY }}
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Set short git commit SHA
|
||||
id: vars
|
||||
run: |
|
||||
shortSha=$(git rev-parse --short ${{ github.sha }})
|
||||
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push container image (latest)
|
||||
# Comment the following line for testing
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
# Set push: false for testing
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.SHORT_SHA }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Build and push container image (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Trigger deployment
|
||||
if: github.event_name == 'push'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.CLOUD_DISPATCH }}
|
||||
event-type: prowler-api-deploy
|
||||
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ env.SHORT_SHA }}"}'
|
||||
@@ -0,0 +1,74 @@
|
||||
name: 'API: Code Quality'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-code-quality.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-code-quality.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
API_WORKING_DIR: ./api
|
||||
|
||||
jobs:
|
||||
api-code-quality:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for API changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
api/docs/**
|
||||
api/README.md
|
||||
api/CHANGELOG.md
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Poetry check
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry check --lock
|
||||
|
||||
- name: Ruff lint
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run ruff check . --exclude contrib
|
||||
|
||||
- name: Ruff format
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run ruff format --check . --exclude contrib
|
||||
|
||||
- name: Pylint
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
|
||||
@@ -0,0 +1,135 @@
|
||||
name: 'API: Container Build and Push'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- 'prowler/**'
|
||||
- '.github/workflows/api-build-lint-push-containers.yml'
|
||||
release:
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
STABLE_TAG: stable
|
||||
WORKING_DIRECTORY: ./api
|
||||
|
||||
# Container registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-api
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
|
||||
steps:
|
||||
- name: Calculate short SHA
|
||||
id: set-short-sha
|
||||
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
|
||||
|
||||
container-build-push:
|
||||
needs: setup
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push API container (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
- name: Build and push API container (release)
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: API
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
step-outcome: ${{ steps.container-push.outcome }}
|
||||
|
||||
trigger-deployment:
|
||||
if: github.event_name == 'push'
|
||||
needs: [setup, container-build-push]
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Trigger API deployment
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.CLOUD_DISPATCH }}
|
||||
event-type: api-prowler-deployment
|
||||
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
|
||||
@@ -0,0 +1,94 @@
|
||||
name: 'API: Container Checks'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-container-checks.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-container-checks.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
API_WORKING_DIR: ./api
|
||||
IMAGE_NAME: prowler-api
|
||||
|
||||
jobs:
|
||||
api-dockerfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check if Dockerfile changed
|
||||
id: dockerfile-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: api/Dockerfile
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
if: steps.dockerfile-changed.outputs.any_changed == 'true'
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
dockerfile: api/Dockerfile
|
||||
ignore: DL3013
|
||||
|
||||
api-container-build-and-scan:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for API changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
api/docs/**
|
||||
api/README.md
|
||||
api/CHANGELOG.md
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build container
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.API_WORKING_DIR }}
|
||||
push: false
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Scan container with Trivy
|
||||
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
@@ -1,228 +0,0 @@
|
||||
name: 'API: Pull Request'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- '.github/workflows/api-pull-request.yml'
|
||||
- 'api/**'
|
||||
- '!api/docs/**'
|
||||
- '!api/README.md'
|
||||
- '!api/CHANGELOG.md'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- '.github/workflows/api-pull-request.yml'
|
||||
- 'api/**'
|
||||
- '!api/docs/**'
|
||||
- '!api/README.md'
|
||||
- '!api/CHANGELOG.md'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
POSTGRES_HOST: localhost
|
||||
POSTGRES_PORT: 5432
|
||||
POSTGRES_ADMIN_USER: prowler
|
||||
POSTGRES_ADMIN_PASSWORD: S3cret
|
||||
POSTGRES_USER: prowler_user
|
||||
POSTGRES_PASSWORD: prowler
|
||||
POSTGRES_DB: postgres-db
|
||||
VALKEY_HOST: localhost
|
||||
VALKEY_PORT: 6379
|
||||
VALKEY_DB: 0
|
||||
API_WORKING_DIR: ./api
|
||||
IMAGE_NAME: prowler-api
|
||||
|
||||
jobs:
|
||||
code-quality:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Poetry check
|
||||
run: poetry check --lock
|
||||
|
||||
- name: Ruff lint
|
||||
run: poetry run ruff check . --exclude contrib
|
||||
|
||||
- name: Ruff format
|
||||
run: poetry run ruff format --check . --exclude contrib
|
||||
|
||||
- name: Pylint
|
||||
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
|
||||
|
||||
security-scans:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Bandit
|
||||
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
|
||||
|
||||
- name: Safety
|
||||
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
|
||||
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
|
||||
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
|
||||
|
||||
- name: Vulture
|
||||
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
|
||||
|
||||
tests:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres
|
||||
env:
|
||||
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
|
||||
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
|
||||
POSTGRES_USER: ${{ env.POSTGRES_USER }}
|
||||
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
|
||||
POSTGRES_DB: ${{ env.POSTGRES_DB }}
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
valkey:
|
||||
image: valkey/valkey:7-alpine3.19
|
||||
env:
|
||||
VALKEY_HOST: ${{ env.VALKEY_HOST }}
|
||||
VALKEY_PORT: ${{ env.VALKEY_PORT }}
|
||||
VALKEY_DB: ${{ env.VALKEY_DB }}
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "valkey-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Run tests with pytest
|
||||
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
|
||||
|
||||
- name: Upload coverage reports to Codecov
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: api
|
||||
|
||||
dockerfile-lint:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
dockerfile: api/Dockerfile
|
||||
ignore: DL3013
|
||||
|
||||
container-build-and-scan:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build container
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.API_WORKING_DIR }}
|
||||
push: false
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Scan container with Trivy
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
@@ -0,0 +1,72 @@
|
||||
name: 'API: Security'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-security.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-security.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
API_WORKING_DIR: ./api
|
||||
|
||||
jobs:
|
||||
api-security-scans:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for API changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
api/docs/**
|
||||
api/README.md
|
||||
api/CHANGELOG.md
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Bandit
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
|
||||
|
||||
- name: Safety
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
|
||||
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
|
||||
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
|
||||
|
||||
- name: Vulture
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
|
||||
@@ -0,0 +1,110 @@
|
||||
name: 'API: Tests'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-tests.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'api/**'
|
||||
- '.github/workflows/api-tests.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
POSTGRES_HOST: localhost
|
||||
POSTGRES_PORT: 5432
|
||||
POSTGRES_ADMIN_USER: prowler
|
||||
POSTGRES_ADMIN_PASSWORD: S3cret
|
||||
POSTGRES_USER: prowler_user
|
||||
POSTGRES_PASSWORD: prowler
|
||||
POSTGRES_DB: postgres-db
|
||||
VALKEY_HOST: localhost
|
||||
VALKEY_PORT: 6379
|
||||
VALKEY_DB: 0
|
||||
API_WORKING_DIR: ./api
|
||||
|
||||
jobs:
|
||||
api-tests:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.12'
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./api
|
||||
|
||||
services:
|
||||
postgres:
|
||||
image: postgres
|
||||
env:
|
||||
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
|
||||
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
|
||||
POSTGRES_USER: ${{ env.POSTGRES_USER }}
|
||||
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
|
||||
POSTGRES_DB: ${{ env.POSTGRES_DB }}
|
||||
ports:
|
||||
- 5432:5432
|
||||
options: >-
|
||||
--health-cmd pg_isready
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
valkey:
|
||||
image: valkey/valkey:7-alpine3.19
|
||||
env:
|
||||
VALKEY_HOST: ${{ env.VALKEY_HOST }}
|
||||
VALKEY_PORT: ${{ env.VALKEY_PORT }}
|
||||
VALKEY_DB: ${{ env.VALKEY_DB }}
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "valkey-cli ping"
|
||||
--health-interval 10s
|
||||
--health-timeout 5s
|
||||
--health-retries 5
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for API changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
api/docs/**
|
||||
api/README.md
|
||||
api/CHANGELOG.md
|
||||
|
||||
- name: Setup Python with Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/setup-python-poetry
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
working-directory: ./api
|
||||
|
||||
- name: Run tests with pytest
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
|
||||
|
||||
- name: Upload coverage reports to Codecov
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: api
|
||||
@@ -7,8 +7,6 @@ on:
|
||||
types:
|
||||
- 'labeled'
|
||||
- 'closed'
|
||||
paths:
|
||||
- '.github/workflows/backport.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
name: Community PR labelling
|
||||
|
||||
on:
|
||||
# We need "write" permissions on the PR to be able to add a label.
|
||||
pull_request_target: # We need this to have labelling permissions. There are no user inputs here, so we should be fine.
|
||||
types:
|
||||
- opened
|
||||
|
||||
permissions: {}
|
||||
|
||||
jobs:
|
||||
label-if-community:
|
||||
name: Add 'community' label if the PR is from a community contributor
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Check if author is org member
|
||||
id: check_membership
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
AUTHOR: ${{ github.event.pull_request.user.login }}
|
||||
ORG: ${{ github.repository_owner }}
|
||||
run: |
|
||||
echo "Checking if $AUTHOR is a member of $ORG"
|
||||
if gh api --method GET "orgs/$ORG/members/$AUTHOR" >/dev/null 2>&1; then
|
||||
echo "is_member=true" >> $GITHUB_OUTPUT
|
||||
echo "$AUTHOR is an organization member"
|
||||
else
|
||||
echo "is_member=false" >> $GITHUB_OUTPUT
|
||||
echo "$AUTHOR is not an organization member"
|
||||
fi
|
||||
|
||||
- name: Add community label
|
||||
if: steps.check_membership.outputs.is_member == 'false'
|
||||
env:
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
run: |
|
||||
echo "Adding 'community' label to PR #$PR_NUMBER"
|
||||
gh pr edit "$PR_NUMBER" --add-label community
|
||||
@@ -16,7 +16,7 @@ permissions:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# Tags
|
||||
@@ -80,8 +80,23 @@ jobs:
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: MCP
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
- name: Build and push MCP container (release)
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
@@ -100,8 +115,31 @@ jobs:
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: MCP
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
step-outcome: ${{ steps.container-push.outcome }}
|
||||
|
||||
trigger-deployment:
|
||||
if: github.event_name == 'push'
|
||||
needs: [setup, container-build-push]
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Trigger MCP deployment
|
||||
if: github.event_name == 'push'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
name: 'MCP: Container Checks'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'mcp_server/**'
|
||||
- '.github/workflows/mcp-container-checks.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'mcp_server/**'
|
||||
- '.github/workflows/mcp-container-checks.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
MCP_WORKING_DIR: ./mcp_server
|
||||
IMAGE_NAME: prowler-mcp
|
||||
|
||||
jobs:
|
||||
mcp-dockerfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check if Dockerfile changed
|
||||
id: dockerfile-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: mcp_server/Dockerfile
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
if: steps.dockerfile-changed.outputs.any_changed == 'true'
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
dockerfile: mcp_server/Dockerfile
|
||||
|
||||
mcp-container-build-and-scan:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for MCP changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
mcp_server/README.md
|
||||
mcp_server/CHANGELOG.md
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build MCP container
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.MCP_WORKING_DIR }}
|
||||
push: false
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Scan MCP container with Trivy
|
||||
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
@@ -150,8 +150,8 @@ jobs:
|
||||
# Remove --- separators
|
||||
sed -i '/^---$/d' "$output_file"
|
||||
|
||||
# Remove trailing empty lines
|
||||
sed -i '/^$/d' "$output_file"
|
||||
# Remove only trailing empty lines (not all empty lines)
|
||||
sed -i -e :a -e '/^\s*$/d;N;ba' "$output_file"
|
||||
}
|
||||
|
||||
# Calculate expected versions for this release
|
||||
@@ -247,6 +247,11 @@ jobs:
|
||||
echo "" >> combined_changelog.md
|
||||
fi
|
||||
|
||||
# Add fallback message if no changelogs were added
|
||||
if [ ! -s combined_changelog.md ]; then
|
||||
echo "No component changes detected for this release." >> combined_changelog.md
|
||||
fi
|
||||
|
||||
echo "Combined changelog preview:"
|
||||
cat combined_changelog.md
|
||||
|
||||
@@ -336,13 +341,6 @@ jobs:
|
||||
CURRENT_PROWLER_REF=$(grep 'prowler @ git+https://github.com/prowler-cloud/prowler.git@' api/pyproject.toml | sed -E 's/.*@([^"]+)".*/\1/' | tr -d '[:space:]')
|
||||
BRANCH_NAME_TRIMMED=$(echo "$BRANCH_NAME" | tr -d '[:space:]')
|
||||
|
||||
# Create a temporary branch for the PR from the minor version branch
|
||||
TEMP_BRANCH="update-api-dependency-$BRANCH_NAME_TRIMMED-$(date +%s)"
|
||||
echo "TEMP_BRANCH=$TEMP_BRANCH" >> $GITHUB_ENV
|
||||
|
||||
# Create temp branch from the current minor version branch
|
||||
git checkout -b "$TEMP_BRANCH"
|
||||
|
||||
# Minor release: update the dependency to use the release branch
|
||||
echo "Updating prowler dependency from '$CURRENT_PROWLER_REF' to '$BRANCH_NAME_TRIMMED'"
|
||||
sed -i "s|prowler @ git+https://github.com/prowler-cloud/prowler.git@[^\"]*\"|prowler @ git+https://github.com/prowler-cloud/prowler.git@$BRANCH_NAME_TRIMMED\"|" api/pyproject.toml
|
||||
@@ -360,11 +358,6 @@ jobs:
|
||||
poetry lock
|
||||
cd ..
|
||||
|
||||
# Commit and push the temporary branch
|
||||
git add api/pyproject.toml api/poetry.lock
|
||||
git commit -m "chore(api): update prowler dependency to $BRANCH_NAME_TRIMMED for release $PROWLER_VERSION"
|
||||
git push origin "$TEMP_BRANCH"
|
||||
|
||||
echo "✓ Prepared prowler dependency update to: $UPDATED_PROWLER_REF"
|
||||
|
||||
- name: Create PR for API dependency update
|
||||
@@ -372,8 +365,12 @@ jobs:
|
||||
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
branch: ${{ env.TEMP_BRANCH }}
|
||||
commit-message: 'chore(api): update prowler dependency to ${{ env.BRANCH_NAME }} for release ${{ env.PROWLER_VERSION }}'
|
||||
branch: update-api-dependency-${{ env.BRANCH_NAME }}-${{ github.run_number }}
|
||||
base: ${{ env.BRANCH_NAME }}
|
||||
add-paths: |
|
||||
api/pyproject.toml
|
||||
api/poetry.lock
|
||||
title: "chore(api): Update prowler dependency to ${{ env.BRANCH_NAME }} for release ${{ env.PROWLER_VERSION }}"
|
||||
body: |
|
||||
### Description
|
||||
@@ -406,5 +403,6 @@ jobs:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
|
||||
- name: Clean up temporary files
|
||||
if: always()
|
||||
run: |
|
||||
rm -f prowler_changelog.md api_changelog.md ui_changelog.md mcp_changelog.md combined_changelog.md
|
||||
|
||||
@@ -1,202 +0,0 @@
|
||||
name: SDK - Build and Push containers
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
# For `v3-latest`
|
||||
- "v3"
|
||||
# For `v4-latest`
|
||||
- "v4.6"
|
||||
# For `latest`
|
||||
- "master"
|
||||
paths-ignore:
|
||||
- ".github/**"
|
||||
- "README.md"
|
||||
- "docs/**"
|
||||
- "ui/**"
|
||||
- "api/**"
|
||||
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
env:
|
||||
# AWS Configuration
|
||||
AWS_REGION_STG: eu-west-1
|
||||
AWS_REGION_PLATFORM: eu-west-1
|
||||
AWS_REGION: us-east-1
|
||||
|
||||
# Container's configuration
|
||||
IMAGE_NAME: prowler
|
||||
DOCKERFILE_PATH: ./Dockerfile
|
||||
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
STABLE_TAG: stable
|
||||
# The RELEASE_TAG is set during runtime in releases
|
||||
RELEASE_TAG: ""
|
||||
# The PROWLER_VERSION and PROWLER_VERSION_MAJOR are set during runtime in releases
|
||||
PROWLER_VERSION: ""
|
||||
PROWLER_VERSION_MAJOR: ""
|
||||
# TEMPORARY_TAG: temporary
|
||||
|
||||
# Python configuration
|
||||
PYTHON_VERSION: 3.12
|
||||
|
||||
# Container Registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler
|
||||
|
||||
jobs:
|
||||
# Build Prowler OSS container
|
||||
container-build-push:
|
||||
# needs: dockerfile-linter
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
prowler_version_major: ${{ steps.get-prowler-version.outputs.PROWLER_VERSION_MAJOR }}
|
||||
prowler_version: ${{ steps.get-prowler-version.outputs.PROWLER_VERSION }}
|
||||
env:
|
||||
POETRY_VIRTUALENVS_CREATE: "false"
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
|
||||
- name: Install Poetry
|
||||
run: |
|
||||
pipx install poetry==2.*
|
||||
pipx inject poetry poetry-bumpversion
|
||||
|
||||
- name: Get Prowler version
|
||||
id: get-prowler-version
|
||||
run: |
|
||||
PROWLER_VERSION="$(poetry version -s 2>/dev/null)"
|
||||
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
# Store prowler version major just for the release
|
||||
PROWLER_VERSION_MAJOR="${PROWLER_VERSION%%.*}"
|
||||
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_ENV}"
|
||||
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
case ${PROWLER_VERSION_MAJOR} in
|
||||
3)
|
||||
echo "LATEST_TAG=v3-latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=v3-stable" >> "${GITHUB_ENV}"
|
||||
;;
|
||||
|
||||
|
||||
4)
|
||||
echo "LATEST_TAG=v4-latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=v4-stable" >> "${GITHUB_ENV}"
|
||||
;;
|
||||
|
||||
5)
|
||||
echo "LATEST_TAG=latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=stable" >> "${GITHUB_ENV}"
|
||||
;;
|
||||
|
||||
*)
|
||||
# Fallback if any other version is present
|
||||
echo "Releasing another Prowler major version, aborting..."
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Login to Public ECR
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
registry: public.ecr.aws
|
||||
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
|
||||
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
|
||||
env:
|
||||
AWS_REGION: ${{ env.AWS_REGION }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push container image (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
file: ${{ env.DOCKERFILE_PATH }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Build and push container image (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
# Use local context to get changes
|
||||
# https://github.com/docker/build-push-action#path-context
|
||||
context: .
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.PROWLER_VERSION }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
file: ${{ env.DOCKERFILE_PATH }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
# - name: Push README to Docker Hub (toniblyx)
|
||||
# uses: peter-evans/dockerhub-description@432a30c9e07499fd01da9f8a49f0faf9e0ca5b77 # v4.0.2
|
||||
# with:
|
||||
# username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
# password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
# repository: ${{ env.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}
|
||||
# readme-filepath: ./README.md
|
||||
#
|
||||
# - name: Push README to Docker Hub (prowlercloud)
|
||||
# uses: peter-evans/dockerhub-description@432a30c9e07499fd01da9f8a49f0faf9e0ca5b77 # v4.0.2
|
||||
# with:
|
||||
# username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
# password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
# repository: ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}
|
||||
# readme-filepath: ./README.md
|
||||
|
||||
dispatch-action:
|
||||
needs: container-build-push
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Get latest commit info (latest)
|
||||
if: github.event_name == 'push'
|
||||
run: |
|
||||
LATEST_COMMIT_HASH=$(echo ${{ github.event.after }} | cut -b -7)
|
||||
echo "LATEST_COMMIT_HASH=${LATEST_COMMIT_HASH}" >> $GITHUB_ENV
|
||||
|
||||
- name: Dispatch event (latest)
|
||||
if: github.event_name == 'push' && needs.container-build-push.outputs.prowler_version_major == '3'
|
||||
run: |
|
||||
curl https://api.github.com/repos/${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}/dispatches \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
--data '{"event_type":"dispatch","client_payload":{"version":"v3-latest", "tag": "${{ env.LATEST_COMMIT_HASH }}"}}'
|
||||
|
||||
- name: Dispatch event (release)
|
||||
if: github.event_name == 'release' && needs.container-build-push.outputs.prowler_version_major == '3'
|
||||
run: |
|
||||
curl https://api.github.com/repos/${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}/dispatches \
|
||||
-H "Accept: application/vnd.github+json" \
|
||||
-H "Authorization: Bearer ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}" \
|
||||
-H "X-GitHub-Api-Version: 2022-11-28" \
|
||||
--data '{"event_type":"dispatch","client_payload":{"version":"release", "tag":"${{ needs.container-build-push.outputs.prowler_version }}"}}'
|
||||
@@ -1,146 +1,218 @@
|
||||
name: SDK - Bump Version
|
||||
name: 'SDK: Bump Version'
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
PROWLER_VERSION: ${{ github.event.release.tag_name }}
|
||||
BASE_BRANCH: master
|
||||
|
||||
jobs:
|
||||
bump-version:
|
||||
name: Bump Version
|
||||
detect-release-type:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
outputs:
|
||||
is_minor: ${{ steps.detect.outputs.is_minor }}
|
||||
is_patch: ${{ steps.detect.outputs.is_patch }}
|
||||
major_version: ${{ steps.detect.outputs.major_version }}
|
||||
minor_version: ${{ steps.detect.outputs.minor_version }}
|
||||
patch_version: ${{ steps.detect.outputs.patch_version }}
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Get Prowler version
|
||||
shell: bash
|
||||
- name: Detect release type and parse version
|
||||
id: detect
|
||||
run: |
|
||||
if [[ $PROWLER_VERSION =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
|
||||
MAJOR_VERSION=${BASH_REMATCH[1]}
|
||||
MINOR_VERSION=${BASH_REMATCH[2]}
|
||||
FIX_VERSION=${BASH_REMATCH[3]}
|
||||
PATCH_VERSION=${BASH_REMATCH[3]}
|
||||
|
||||
# Export version components to GitHub environment
|
||||
echo "MAJOR_VERSION=${MAJOR_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "MINOR_VERSION=${MINOR_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "FIX_VERSION=${FIX_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "major_version=${MAJOR_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
echo "minor_version=${MINOR_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
echo "patch_version=${PATCH_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
if (( MAJOR_VERSION == 5 )); then
|
||||
if (( FIX_VERSION == 0 )); then
|
||||
echo "Minor Release: $PROWLER_VERSION"
|
||||
if (( MAJOR_VERSION != 5 )); then
|
||||
echo "::error::Releasing another Prowler major version, aborting..."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Set up next minor version for master
|
||||
BUMP_VERSION_TO=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).${FIX_VERSION}
|
||||
echo "BUMP_VERSION_TO=${BUMP_VERSION_TO}" >> "${GITHUB_ENV}"
|
||||
|
||||
TARGET_BRANCH=${BASE_BRANCH}
|
||||
echo "TARGET_BRANCH=${TARGET_BRANCH}" >> "${GITHUB_ENV}"
|
||||
|
||||
# Set up patch version for version branch
|
||||
PATCH_VERSION_TO=${MAJOR_VERSION}.${MINOR_VERSION}.1
|
||||
echo "PATCH_VERSION_TO=${PATCH_VERSION_TO}" >> "${GITHUB_ENV}"
|
||||
|
||||
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
|
||||
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
|
||||
|
||||
echo "Bumping to next minor version: ${BUMP_VERSION_TO} in branch ${TARGET_BRANCH}"
|
||||
echo "Bumping to next patch version: ${PATCH_VERSION_TO} in branch ${VERSION_BRANCH}"
|
||||
else
|
||||
echo "Patch Release: $PROWLER_VERSION"
|
||||
|
||||
BUMP_VERSION_TO=${MAJOR_VERSION}.${MINOR_VERSION}.$((FIX_VERSION + 1))
|
||||
echo "BUMP_VERSION_TO=${BUMP_VERSION_TO}" >> "${GITHUB_ENV}"
|
||||
|
||||
TARGET_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
|
||||
echo "TARGET_BRANCH=${TARGET_BRANCH}" >> "${GITHUB_ENV}"
|
||||
|
||||
echo "Bumping to next patch version: ${BUMP_VERSION_TO} in branch ${TARGET_BRANCH}"
|
||||
fi
|
||||
if (( PATCH_VERSION == 0 )); then
|
||||
echo "is_minor=true" >> "${GITHUB_OUTPUT}"
|
||||
echo "is_patch=false" >> "${GITHUB_OUTPUT}"
|
||||
echo "✓ Minor release detected: $PROWLER_VERSION"
|
||||
else
|
||||
echo "Releasing another Prowler major version, aborting..."
|
||||
exit 1
|
||||
echo "is_minor=false" >> "${GITHUB_OUTPUT}"
|
||||
echo "is_patch=true" >> "${GITHUB_OUTPUT}"
|
||||
echo "✓ Patch release detected: $PROWLER_VERSION"
|
||||
fi
|
||||
else
|
||||
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
|
||||
echo "::error::Invalid version syntax: '$PROWLER_VERSION' (must be X.Y.Z)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Bump versions in files
|
||||
bump-minor-version:
|
||||
needs: detect-release-type
|
||||
if: needs.detect-release-type.outputs.is_minor == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Calculate next minor version
|
||||
run: |
|
||||
echo "Using PROWLER_VERSION=$PROWLER_VERSION"
|
||||
echo "Using BUMP_VERSION_TO=$BUMP_VERSION_TO"
|
||||
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
|
||||
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
|
||||
|
||||
set -e
|
||||
NEXT_MINOR_VERSION=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).0
|
||||
echo "NEXT_MINOR_VERSION=${NEXT_MINOR_VERSION}" >> "${GITHUB_ENV}"
|
||||
|
||||
echo "Bumping version in pyproject.toml ..."
|
||||
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${BUMP_VERSION_TO}\"|" pyproject.toml
|
||||
echo "Current version: $PROWLER_VERSION"
|
||||
echo "Next minor version: $NEXT_MINOR_VERSION"
|
||||
|
||||
echo "Bumping version in prowler/config/config.py ..."
|
||||
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${BUMP_VERSION_TO}\"|" prowler/config/config.py
|
||||
- name: Bump versions in files for master
|
||||
run: |
|
||||
set -e
|
||||
|
||||
echo "Bumping version in .env ..."
|
||||
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${BUMP_VERSION_TO}|" .env
|
||||
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${NEXT_MINOR_VERSION}\"|" pyproject.toml
|
||||
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${NEXT_MINOR_VERSION}\"|" prowler/config/config.py
|
||||
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${NEXT_MINOR_VERSION}|" .env
|
||||
|
||||
git --no-pager diff
|
||||
echo "Files modified:"
|
||||
git --no-pager diff
|
||||
|
||||
- name: Create Pull Request
|
||||
- name: Create PR for next minor version to master
|
||||
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
|
||||
with:
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
base: ${{ env.TARGET_BRANCH }}
|
||||
commit-message: "chore(release): Bump version to v${{ env.BUMP_VERSION_TO }}"
|
||||
branch: "version-bump-to-v${{ env.BUMP_VERSION_TO }}"
|
||||
title: "chore(release): Bump version to v${{ env.BUMP_VERSION_TO }}"
|
||||
labels: no-changelog
|
||||
body: |
|
||||
### Description
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
base: master
|
||||
commit-message: 'chore(release): Bump version to v${{ env.NEXT_MINOR_VERSION }}'
|
||||
branch: version-bump-to-v${{ env.NEXT_MINOR_VERSION }}
|
||||
title: 'chore(release): Bump version to v${{ env.NEXT_MINOR_VERSION }}'
|
||||
labels: no-changelog
|
||||
body: |
|
||||
### Description
|
||||
|
||||
Bump Prowler version to v${{ env.BUMP_VERSION_TO }}
|
||||
Bump Prowler version to v${{ env.NEXT_MINOR_VERSION }} after releasing v${{ env.PROWLER_VERSION }}.
|
||||
|
||||
### License
|
||||
### License
|
||||
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
|
||||
- name: Handle patch version for minor release
|
||||
if: env.FIX_VERSION == '0'
|
||||
- name: Checkout version branch
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
with:
|
||||
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
|
||||
|
||||
- name: Calculate first patch version
|
||||
run: |
|
||||
echo "Using PROWLER_VERSION=$PROWLER_VERSION"
|
||||
echo "Using PATCH_VERSION_TO=$PATCH_VERSION_TO"
|
||||
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
|
||||
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
|
||||
|
||||
set -e
|
||||
FIRST_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.1
|
||||
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
|
||||
|
||||
echo "Bumping version in pyproject.toml ..."
|
||||
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${PATCH_VERSION_TO}\"|" pyproject.toml
|
||||
echo "FIRST_PATCH_VERSION=${FIRST_PATCH_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
|
||||
|
||||
echo "Bumping version in prowler/config/config.py ..."
|
||||
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${PATCH_VERSION_TO}\"|" prowler/config/config.py
|
||||
echo "First patch version: $FIRST_PATCH_VERSION"
|
||||
echo "Version branch: $VERSION_BRANCH"
|
||||
|
||||
echo "Bumping version in .env ..."
|
||||
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PATCH_VERSION_TO}|" .env
|
||||
- name: Bump versions in files for version branch
|
||||
run: |
|
||||
set -e
|
||||
|
||||
git --no-pager diff
|
||||
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${FIRST_PATCH_VERSION}\"|" pyproject.toml
|
||||
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${FIRST_PATCH_VERSION}\"|" prowler/config/config.py
|
||||
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${FIRST_PATCH_VERSION}|" .env
|
||||
|
||||
- name: Create Pull Request for patch version
|
||||
if: env.FIX_VERSION == '0'
|
||||
echo "Files modified:"
|
||||
git --no-pager diff
|
||||
|
||||
- name: Create PR for first patch version to version branch
|
||||
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
|
||||
with:
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
base: ${{ env.VERSION_BRANCH }}
|
||||
commit-message: "chore(release): Bump version to v${{ env.PATCH_VERSION_TO }}"
|
||||
branch: "version-bump-to-v${{ env.PATCH_VERSION_TO }}"
|
||||
title: "chore(release): Bump version to v${{ env.PATCH_VERSION_TO }}"
|
||||
labels: no-changelog
|
||||
body: |
|
||||
### Description
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
base: ${{ env.VERSION_BRANCH }}
|
||||
commit-message: 'chore(release): Bump version to v${{ env.FIRST_PATCH_VERSION }}'
|
||||
branch: version-bump-to-v${{ env.FIRST_PATCH_VERSION }}
|
||||
title: 'chore(release): Bump version to v${{ env.FIRST_PATCH_VERSION }}'
|
||||
labels: no-changelog
|
||||
body: |
|
||||
### Description
|
||||
|
||||
Bump Prowler version to v${{ env.PATCH_VERSION_TO }}
|
||||
Bump Prowler version to v${{ env.FIRST_PATCH_VERSION }} in version branch after releasing v${{ env.PROWLER_VERSION }}.
|
||||
|
||||
### License
|
||||
### License
|
||||
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
|
||||
bump-patch-version:
|
||||
needs: detect-release-type
|
||||
if: needs.detect-release-type.outputs.is_patch == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Calculate next patch version
|
||||
run: |
|
||||
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
|
||||
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
|
||||
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
|
||||
|
||||
NEXT_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.$((PATCH_VERSION + 1))
|
||||
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
|
||||
|
||||
echo "NEXT_PATCH_VERSION=${NEXT_PATCH_VERSION}" >> "${GITHUB_ENV}"
|
||||
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
|
||||
|
||||
echo "Current version: $PROWLER_VERSION"
|
||||
echo "Next patch version: $NEXT_PATCH_VERSION"
|
||||
echo "Target branch: $VERSION_BRANCH"
|
||||
|
||||
- name: Bump versions in files for version branch
|
||||
run: |
|
||||
set -e
|
||||
|
||||
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${NEXT_PATCH_VERSION}\"|" pyproject.toml
|
||||
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${NEXT_PATCH_VERSION}\"|" prowler/config/config.py
|
||||
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${NEXT_PATCH_VERSION}|" .env
|
||||
|
||||
echo "Files modified:"
|
||||
git --no-pager diff
|
||||
|
||||
- name: Create PR for next patch version to version branch
|
||||
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
|
||||
with:
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
base: ${{ env.VERSION_BRANCH }}
|
||||
commit-message: 'chore(release): Bump version to v${{ env.NEXT_PATCH_VERSION }}'
|
||||
branch: version-bump-to-v${{ env.NEXT_PATCH_VERSION }}
|
||||
title: 'chore(release): Bump version to v${{ env.NEXT_PATCH_VERSION }}'
|
||||
labels: no-changelog
|
||||
body: |
|
||||
### Description
|
||||
|
||||
Bump Prowler version to v${{ env.NEXT_PATCH_VERSION }} after releasing v${{ env.PROWLER_VERSION }}.
|
||||
|
||||
### License
|
||||
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
name: 'SDK: Code Quality'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-code-quality.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-code-quality.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
sdk-code-quality:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.9'
|
||||
- '3.10'
|
||||
- '3.11'
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for SDK changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
prowler/CHANGELOG.md
|
||||
docs/**
|
||||
permissions/**
|
||||
api/**
|
||||
ui/**
|
||||
dashboard/**
|
||||
mcp_server/**
|
||||
README.md
|
||||
mkdocs.yml
|
||||
.backportrc.json
|
||||
.env
|
||||
docker-compose*
|
||||
examples/**
|
||||
.gitignore
|
||||
contrib/**
|
||||
|
||||
- name: Install Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'poetry'
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry install --no-root
|
||||
poetry run pip list
|
||||
|
||||
- name: Check Poetry lock file
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry check --lock
|
||||
|
||||
- name: Lint with flake8
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude contrib,ui,api
|
||||
|
||||
- name: Check format with black
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run black --exclude api ui --check .
|
||||
|
||||
- name: Lint with pylint
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn prowler/
|
||||
@@ -1,44 +1,40 @@
|
||||
# For most projects, this workflow file will not need changing; you simply need
|
||||
# to commit it to your repository.
|
||||
#
|
||||
# You may wish to alter this file to override the set of languages analyzed,
|
||||
# or to provide custom queries or build logic.
|
||||
#
|
||||
# ******** NOTE ********
|
||||
# We have attempted to detect the languages in your repository. Please check
|
||||
# the `language` matrix defined below to confirm you have the correct set of
|
||||
# supported CodeQL languages.
|
||||
#
|
||||
name: SDK - CodeQL
|
||||
name: 'SDK: CodeQL'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v3"
|
||||
- "v4.*"
|
||||
- "v5.*"
|
||||
paths-ignore:
|
||||
- 'ui/**'
|
||||
- 'api/**'
|
||||
- '.github/**'
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- '.github/workflows/sdk-codeql.yml'
|
||||
- '.github/codeql/sdk-codeql-config.yml'
|
||||
- '!prowler/CHANGELOG.md'
|
||||
pull_request:
|
||||
branches:
|
||||
- "master"
|
||||
- "v3"
|
||||
- "v4.*"
|
||||
- "v5.*"
|
||||
paths-ignore:
|
||||
- 'ui/**'
|
||||
- 'api/**'
|
||||
- '.github/**'
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- '.github/workflows/sdk-codeql.yml'
|
||||
- '.github/codeql/sdk-codeql-config.yml'
|
||||
- '!prowler/CHANGELOG.md'
|
||||
schedule:
|
||||
- cron: '00 12 * * *'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze
|
||||
name: CodeQL Security Analysis
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
@@ -47,21 +43,20 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: [ 'python' ]
|
||||
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
|
||||
language:
|
||||
- 'python'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
config-file: ./.github/codeql/sdk-codeql-config.yml
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
languages: ${{ matrix.language }}
|
||||
config-file: ./.github/codeql/sdk-codeql-config.yml
|
||||
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
category: "/language:${{matrix.language}}"
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
category: '/language:${{ matrix.language }}'
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
name: 'SDK: Container Build and Push'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'v3' # For v3-latest
|
||||
- 'v4.6' # For v4-latest
|
||||
- 'master' # For latest
|
||||
paths-ignore:
|
||||
- '.github/**'
|
||||
- '!.github/workflows/sdk-container-build-push.yml'
|
||||
- 'README.md'
|
||||
- 'docs/**'
|
||||
- 'ui/**'
|
||||
- 'api/**'
|
||||
release:
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# Container configuration
|
||||
IMAGE_NAME: prowler
|
||||
DOCKERFILE_PATH: ./Dockerfile
|
||||
|
||||
# Python configuration
|
||||
PYTHON_VERSION: '3.12'
|
||||
|
||||
# Tags (dynamically set based on version)
|
||||
LATEST_TAG: latest
|
||||
STABLE_TAG: stable
|
||||
|
||||
# Container registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler
|
||||
|
||||
# AWS configuration (for ECR)
|
||||
AWS_REGION: us-east-1
|
||||
|
||||
jobs:
|
||||
container-build-push:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
outputs:
|
||||
prowler_version: ${{ steps.get-prowler-version.outputs.prowler_version }}
|
||||
prowler_version_major: ${{ steps.get-prowler-version.outputs.prowler_version_major }}
|
||||
env:
|
||||
POETRY_VIRTUALENVS_CREATE: 'false'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Set up Python ${{ env.PYTHON_VERSION }}
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
|
||||
- name: Install Poetry
|
||||
run: |
|
||||
pipx install poetry==2.1.1
|
||||
pipx inject poetry poetry-bumpversion
|
||||
|
||||
- name: Get Prowler version and set tags
|
||||
id: get-prowler-version
|
||||
run: |
|
||||
PROWLER_VERSION="$(poetry version -s 2>/dev/null)"
|
||||
echo "prowler_version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_ENV}"
|
||||
|
||||
# Extract major version
|
||||
PROWLER_VERSION_MAJOR="${PROWLER_VERSION%%.*}"
|
||||
echo "prowler_version_major=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_OUTPUT}"
|
||||
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_ENV}"
|
||||
|
||||
# Set version-specific tags
|
||||
case ${PROWLER_VERSION_MAJOR} in
|
||||
3)
|
||||
echo "LATEST_TAG=v3-latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=v3-stable" >> "${GITHUB_ENV}"
|
||||
echo "✓ Prowler v3 detected - tags: v3-latest, v3-stable"
|
||||
;;
|
||||
4)
|
||||
echo "LATEST_TAG=v4-latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=v4-stable" >> "${GITHUB_ENV}"
|
||||
echo "✓ Prowler v4 detected - tags: v4-latest, v4-stable"
|
||||
;;
|
||||
5)
|
||||
echo "LATEST_TAG=latest" >> "${GITHUB_ENV}"
|
||||
echo "STABLE_TAG=stable" >> "${GITHUB_ENV}"
|
||||
echo "✓ Prowler v5 detected - tags: latest, stable"
|
||||
;;
|
||||
*)
|
||||
echo "::error::Unsupported Prowler major version: ${PROWLER_VERSION_MAJOR}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Login to Public ECR
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
registry: public.ecr.aws
|
||||
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
|
||||
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
|
||||
env:
|
||||
AWS_REGION: ${{ env.AWS_REGION }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push SDK container (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: .
|
||||
file: ${{ env.DOCKERFILE_PATH }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: SDK
|
||||
RELEASE_TAG: ${{ env.PROWLER_VERSION }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
- name: Build and push SDK container (release)
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: .
|
||||
file: ${{ env.DOCKERFILE_PATH }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
|
||||
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
|
||||
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.PROWLER_VERSION }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: SDK
|
||||
RELEASE_TAG: ${{ env.PROWLER_VERSION }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
step-outcome: ${{ steps.container-push.outcome }}
|
||||
|
||||
dispatch-v3-deployment:
|
||||
if: needs.container-build-push.outputs.prowler_version_major == '3'
|
||||
needs: container-build-push
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Calculate short SHA
|
||||
id: short-sha
|
||||
run: echo "short_sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
|
||||
|
||||
- name: Dispatch v3 deployment (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}
|
||||
event-type: dispatch
|
||||
client-payload: '{"version":"v3-latest","tag":"${{ steps.short-sha.outputs.short_sha }}"}'
|
||||
|
||||
- name: Dispatch v3 deployment (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}
|
||||
event-type: dispatch
|
||||
client-payload: '{"version":"release","tag":"${{ needs.container-build-push.outputs.prowler_version }}"}'
|
||||
@@ -0,0 +1,109 @@
|
||||
name: 'SDK: Container Checks'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'Dockerfile'
|
||||
- '.github/workflows/sdk-container-checks.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'Dockerfile'
|
||||
- '.github/workflows/sdk-container-checks.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
IMAGE_NAME: prowler
|
||||
|
||||
jobs:
|
||||
sdk-dockerfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check if Dockerfile changed
|
||||
id: dockerfile-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: Dockerfile
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
if: steps.dockerfile-changed.outputs.any_changed == 'true'
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
dockerfile: Dockerfile
|
||||
ignore: DL3013
|
||||
|
||||
sdk-container-build-and-scan:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for SDK changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
prowler/CHANGELOG.md
|
||||
docs/**
|
||||
permissions/**
|
||||
api/**
|
||||
ui/**
|
||||
dashboard/**
|
||||
mcp_server/**
|
||||
README.md
|
||||
mkdocs.yml
|
||||
.backportrc.json
|
||||
.env
|
||||
docker-compose*
|
||||
examples/**
|
||||
.gitignore
|
||||
contrib/**
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build SDK container
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: .
|
||||
push: false
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Scan SDK container with Trivy
|
||||
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
@@ -1,286 +0,0 @@
|
||||
name: SDK - Pull Request
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v3"
|
||||
- "v4.*"
|
||||
- "v5.*"
|
||||
pull_request:
|
||||
branches:
|
||||
- "master"
|
||||
- "v3"
|
||||
- "v4.*"
|
||||
- "v5.*"
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version: ["3.9", "3.10", "3.11", "3.12"]
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Test if changes are in not ignored paths
|
||||
id: are-non-ignored-files-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: ./**
|
||||
files_ignore: |
|
||||
.github/**
|
||||
docs/**
|
||||
permissions/**
|
||||
api/**
|
||||
ui/**
|
||||
prowler/CHANGELOG.md
|
||||
README.md
|
||||
mkdocs.yml
|
||||
.backportrc.json
|
||||
.env
|
||||
docker-compose*
|
||||
examples/**
|
||||
.gitignore
|
||||
|
||||
- name: Install poetry
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: "poetry"
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry install --no-root
|
||||
poetry run pip list
|
||||
VERSION=$(curl --silent "https://api.github.com/repos/hadolint/hadolint/releases/latest" | \
|
||||
grep '"tag_name":' | \
|
||||
sed -E 's/.*"v([^"]+)".*/\1/' \
|
||||
) && curl -L -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/v${VERSION}/hadolint-Linux-x86_64" \
|
||||
&& chmod +x /tmp/hadolint
|
||||
|
||||
- name: Poetry check
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry check --lock
|
||||
|
||||
- name: Lint with flake8
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude contrib,ui,api
|
||||
|
||||
- name: Checking format with black
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run black --exclude api ui --check .
|
||||
|
||||
- name: Lint with pylint
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pylint --disable=W,C,R,E -j 0 -rn -sn prowler/
|
||||
|
||||
- name: Bandit
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run bandit -q -lll -x '*_test.py,./contrib/,./api/,./ui' -r .
|
||||
|
||||
- name: Safety
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run safety check --ignore 70612 -r pyproject.toml
|
||||
|
||||
- name: Vulture
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run vulture --exclude "contrib,api,ui" --min-confidence 100 .
|
||||
|
||||
- name: Dockerfile - Check if Dockerfile has changed
|
||||
id: dockerfile-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
Dockerfile
|
||||
|
||||
- name: Hadolint
|
||||
if: steps.dockerfile-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
/tmp/hadolint Dockerfile --ignore=DL3013
|
||||
|
||||
# Test AWS
|
||||
- name: AWS - Check if any file has changed
|
||||
id: aws-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/aws/**
|
||||
./tests/providers/aws/**
|
||||
./poetry.lock
|
||||
|
||||
- name: AWS - Test
|
||||
if: steps.aws-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml tests/providers/aws
|
||||
|
||||
# Test Azure
|
||||
- name: Azure - Check if any file has changed
|
||||
id: azure-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/azure/**
|
||||
./tests/providers/azure/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Azure - Test
|
||||
if: steps.azure-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/azure --cov-report=xml:azure_coverage.xml tests/providers/azure
|
||||
|
||||
# Test GCP
|
||||
- name: GCP - Check if any file has changed
|
||||
id: gcp-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/gcp/**
|
||||
./tests/providers/gcp/**
|
||||
./poetry.lock
|
||||
|
||||
- name: GCP - Test
|
||||
if: steps.gcp-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/gcp --cov-report=xml:gcp_coverage.xml tests/providers/gcp
|
||||
|
||||
# Test Kubernetes
|
||||
- name: Kubernetes - Check if any file has changed
|
||||
id: kubernetes-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/kubernetes/**
|
||||
./tests/providers/kubernetes/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Kubernetes - Test
|
||||
if: steps.kubernetes-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/kubernetes --cov-report=xml:kubernetes_coverage.xml tests/providers/kubernetes
|
||||
|
||||
# Test GitHub
|
||||
- name: GitHub - Check if any file has changed
|
||||
id: github-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/github/**
|
||||
./tests/providers/github/**
|
||||
./poetry.lock
|
||||
|
||||
- name: GitHub - Test
|
||||
if: steps.github-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/github --cov-report=xml:github_coverage.xml tests/providers/github
|
||||
|
||||
# Test NHN
|
||||
- name: NHN - Check if any file has changed
|
||||
id: nhn-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/nhn/**
|
||||
./tests/providers/nhn/**
|
||||
./poetry.lock
|
||||
|
||||
- name: NHN - Test
|
||||
if: steps.nhn-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/nhn --cov-report=xml:nhn_coverage.xml tests/providers/nhn
|
||||
|
||||
# Test M365
|
||||
- name: M365 - Check if any file has changed
|
||||
id: m365-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/m365/**
|
||||
./tests/providers/m365/**
|
||||
./poetry.lock
|
||||
|
||||
- name: M365 - Test
|
||||
if: steps.m365-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/m365 --cov-report=xml:m365_coverage.xml tests/providers/m365
|
||||
|
||||
# Test IaC
|
||||
- name: IaC - Check if any file has changed
|
||||
id: iac-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/iac/**
|
||||
./tests/providers/iac/**
|
||||
./poetry.lock
|
||||
|
||||
- name: IaC - Test
|
||||
if: steps.iac-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/iac --cov-report=xml:iac_coverage.xml tests/providers/iac
|
||||
|
||||
# Test MongoDB Atlas
|
||||
- name: MongoDB Atlas - Check if any file has changed
|
||||
id: mongodb-atlas-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/mongodbatlas/**
|
||||
./tests/providers/mongodbatlas/**
|
||||
.poetry.lock
|
||||
|
||||
- name: MongoDB Atlas - Test
|
||||
if: steps.mongodb-atlas-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodb_atlas_coverage.xml tests/providers/mongodbatlas
|
||||
|
||||
# Test OCI
|
||||
- name: OCI - Check if any file has changed
|
||||
id: oci-changed-files
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/providers/oraclecloud/**
|
||||
./tests/providers/oraclecloud/**
|
||||
./poetry.lock
|
||||
|
||||
- name: OCI - Test
|
||||
if: steps.oci-changed-files.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oci_coverage.xml tests/providers/oraclecloud
|
||||
|
||||
# Common Tests
|
||||
- name: Lib - Test
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/lib --cov-report=xml:lib_coverage.xml tests/lib
|
||||
|
||||
- name: Config - Test
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
run: |
|
||||
poetry run pytest -n auto --cov=./prowler/config --cov-report=xml:config_coverage.xml tests/config
|
||||
|
||||
# Codecov
|
||||
- name: Upload coverage reports to Codecov
|
||||
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler
|
||||
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./oci_coverage.xml,./lib_coverage.xml,./config_coverage.xml
|
||||
@@ -1,98 +1,119 @@
|
||||
name: SDK - PyPI release
|
||||
name: 'SDK: PyPI Release'
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
PYTHON_VERSION: 3.11
|
||||
# CACHE: "poetry"
|
||||
PYTHON_VERSION: '3.12'
|
||||
|
||||
jobs:
|
||||
repository-check:
|
||||
name: Repository check
|
||||
validate-release:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
outputs:
|
||||
is_repo: ${{ steps.repository_check.outputs.is_repo }}
|
||||
steps:
|
||||
- name: Repository check
|
||||
id: repository_check
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
|
||||
then
|
||||
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
|
||||
else
|
||||
echo "This action only runs for prowler-cloud/prowler"
|
||||
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
|
||||
fi
|
||||
prowler_version: ${{ steps.parse-version.outputs.version }}
|
||||
major_version: ${{ steps.parse-version.outputs.major }}
|
||||
|
||||
release-prowler-job:
|
||||
runs-on: ubuntu-latest
|
||||
needs: repository-check
|
||||
if: needs.repository-check.outputs.is_repo == 'true'
|
||||
env:
|
||||
POETRY_VIRTUALENVS_CREATE: "false"
|
||||
name: Release Prowler to PyPI
|
||||
steps:
|
||||
- name: Repository check
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
if [[ "${{ github.repository }}" != "prowler-cloud/prowler" ]]; then
|
||||
echo "This action only runs for prowler-cloud/prowler"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Get Prowler version
|
||||
- name: Parse and validate version
|
||||
id: parse-version
|
||||
run: |
|
||||
PROWLER_VERSION="${{ env.RELEASE_TAG }}"
|
||||
echo "version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
case ${PROWLER_VERSION%%.*} in
|
||||
3)
|
||||
echo "Releasing Prowler v3 with tag ${PROWLER_VERSION}"
|
||||
# Extract major version
|
||||
MAJOR_VERSION="${PROWLER_VERSION%%.*}"
|
||||
echo "major=${MAJOR_VERSION}" >> "${GITHUB_OUTPUT}"
|
||||
|
||||
# Validate major version
|
||||
case ${MAJOR_VERSION} in
|
||||
3|4|5)
|
||||
echo "✓ Releasing Prowler v${MAJOR_VERSION} with tag ${PROWLER_VERSION}"
|
||||
;;
|
||||
4)
|
||||
echo "Releasing Prowler v4 with tag ${PROWLER_VERSION}"
|
||||
;;
|
||||
5)
|
||||
echo "Releasing Prowler v5 with tag ${PROWLER_VERSION}"
|
||||
;;
|
||||
*)
|
||||
echo "Releasing another Prowler major version, aborting..."
|
||||
*)
|
||||
echo "::error::Unsupported Prowler major version: ${MAJOR_VERSION}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
publish-prowler:
|
||||
needs: validate-release
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
environment:
|
||||
name: pypi-prowler
|
||||
url: https://pypi.org/project/prowler/${{ needs.validate-release.outputs.prowler_version }}/
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
pipx install poetry==2.1.1
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Setup Python
|
||||
- name: Install Poetry
|
||||
run: pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python ${{ env.PYTHON_VERSION }}
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
# cache: ${{ env.CACHE }}
|
||||
cache: 'poetry'
|
||||
|
||||
- name: Build Prowler package
|
||||
run: |
|
||||
poetry build
|
||||
run: poetry build
|
||||
|
||||
- name: Publish Prowler package to PyPI
|
||||
run: |
|
||||
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
|
||||
poetry publish
|
||||
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
|
||||
with:
|
||||
print-hash: true
|
||||
|
||||
- name: Replicate PyPI package
|
||||
publish-prowler-cloud:
|
||||
needs: validate-release
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
environment:
|
||||
name: pypi-prowler-cloud
|
||||
url: https://pypi.org/project/prowler-cloud/${{ needs.validate-release.outputs.prowler_version }}/
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Install Poetry
|
||||
run: pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python ${{ env.PYTHON_VERSION }}
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'poetry'
|
||||
|
||||
- name: Install toml package
|
||||
run: pip install toml
|
||||
|
||||
- name: Replicate PyPI package for prowler-cloud
|
||||
run: |
|
||||
rm -rf ./dist && rm -rf ./build && rm -rf prowler.egg-info
|
||||
pip install toml
|
||||
rm -rf ./dist ./build prowler.egg-info
|
||||
python util/replicate_pypi_package.py
|
||||
poetry build
|
||||
|
||||
- name: Build prowler-cloud package
|
||||
run: poetry build
|
||||
|
||||
- name: Publish prowler-cloud package to PyPI
|
||||
run: |
|
||||
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
|
||||
poetry publish
|
||||
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
|
||||
with:
|
||||
print-hash: true
|
||||
|
||||
@@ -1,68 +1,90 @@
|
||||
# This is a basic workflow to help you get started with Actions
|
||||
|
||||
name: SDK - Refresh AWS services' regions
|
||||
name: 'SDK: Refresh AWS Regions'
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 9 * * 1" # runs at 09:00 UTC every Monday
|
||||
- cron: '0 9 * * 1' # Every Monday at 09:00 UTC
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
GITHUB_BRANCH: "master"
|
||||
AWS_REGION_DEV: us-east-1
|
||||
PYTHON_VERSION: '3.12'
|
||||
AWS_REGION: 'us-east-1'
|
||||
|
||||
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
|
||||
jobs:
|
||||
# This workflow contains a single job called "build"
|
||||
build:
|
||||
# The type of runner that the job will run on
|
||||
refresh-aws-regions:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
id-token: write
|
||||
pull-requests: write
|
||||
contents: write
|
||||
# Steps represent a sequence of tasks that will be executed as part of the job
|
||||
steps:
|
||||
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
with:
|
||||
ref: ${{ env.GITHUB_BRANCH }}
|
||||
|
||||
- name: setup python
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
with:
|
||||
ref: 'master'
|
||||
|
||||
- name: Set up Python ${{ env.PYTHON_VERSION }}
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: 3.9 #install the python needed
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
cache: 'pip'
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install boto3
|
||||
run: pip install boto3
|
||||
|
||||
- name: Configure AWS Credentials -- DEV
|
||||
- name: Configure AWS credentials
|
||||
uses: aws-actions/configure-aws-credentials@a03048d87541d1d9fcf2ecf528a4a65ba9bd7838 # v5.0.0
|
||||
with:
|
||||
aws-region: ${{ env.AWS_REGION_DEV }}
|
||||
aws-region: ${{ env.AWS_REGION }}
|
||||
role-to-assume: ${{ secrets.DEV_IAM_ROLE_ARN }}
|
||||
role-session-name: refresh-AWS-regions-dev
|
||||
role-session-name: prowler-refresh-aws-regions
|
||||
|
||||
# Runs a single command using the runners shell
|
||||
- name: Run a one-line script
|
||||
run: python3 util/update_aws_services_regions.py
|
||||
- name: Update AWS services regions
|
||||
run: python util/update_aws_services_regions.py
|
||||
|
||||
# Create pull request
|
||||
- name: Create Pull Request
|
||||
- name: Create pull request
|
||||
id: create-pr
|
||||
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
|
||||
with:
|
||||
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
commit-message: "feat(regions_update): Update regions for AWS services"
|
||||
branch: "aws-services-regions-updated-${{ github.sha }}"
|
||||
labels: "status/waiting-for-revision, severity/low, provider/aws, no-changelog"
|
||||
title: "chore(regions_update): Changes in regions for AWS services"
|
||||
author: 'prowler-bot <179230569+prowler-bot@users.noreply.github.com>'
|
||||
committer: 'github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>'
|
||||
commit-message: 'feat(aws): update regions for AWS services'
|
||||
branch: 'aws-regions-update-${{ github.run_number }}'
|
||||
title: 'feat(aws): Update regions for AWS services'
|
||||
labels: |
|
||||
status/waiting-for-revision
|
||||
severity/low
|
||||
provider/aws
|
||||
no-changelog
|
||||
body: |
|
||||
### Description
|
||||
|
||||
This PR updates the regions for AWS services.
|
||||
Automated update of AWS service regions from the official AWS IP ranges.
|
||||
|
||||
**Trigger:** ${{ github.event_name == 'schedule' && 'Scheduled (weekly)' || github.event_name == 'workflow_dispatch' && 'Manual' || 'Workflow update' }}
|
||||
**Run:** [#${{ github.run_number }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
|
||||
|
||||
### Checklist
|
||||
|
||||
- [x] This is an automated update from AWS official sources
|
||||
- [x] No manual review of region data required
|
||||
|
||||
### License
|
||||
|
||||
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
|
||||
|
||||
- name: PR creation result
|
||||
run: |
|
||||
if [[ "${{ steps.create-pr.outputs.pull-request-number }}" ]]; then
|
||||
echo "✓ Pull request #${{ steps.create-pr.outputs.pull-request-number }} created successfully"
|
||||
echo "URL: ${{ steps.create-pr.outputs.pull-request-url }}"
|
||||
else
|
||||
echo "✓ No changes detected - AWS regions are up to date"
|
||||
fi
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
name: 'SDK: Security'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-security.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-security.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
sdk-security-scans:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for SDK changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
prowler/CHANGELOG.md
|
||||
docs/**
|
||||
permissions/**
|
||||
api/**
|
||||
ui/**
|
||||
dashboard/**
|
||||
mcp_server/**
|
||||
README.md
|
||||
mkdocs.yml
|
||||
.backportrc.json
|
||||
.env
|
||||
docker-compose*
|
||||
examples/**
|
||||
.gitignore
|
||||
contrib/**
|
||||
|
||||
- name: Install Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python 3.12
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: '3.12'
|
||||
cache: 'poetry'
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry install --no-root
|
||||
|
||||
- name: Security scan with Bandit
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run bandit -q -lll -x '*_test.py,./contrib/,./api/,./ui' -r .
|
||||
|
||||
- name: Security scan with Safety
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run safety check --ignore 70612 -r pyproject.toml
|
||||
|
||||
- name: Dead code detection with Vulture
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry run vulture --exclude "contrib,api,ui" --min-confidence 100 .
|
||||
@@ -0,0 +1,369 @@
|
||||
name: 'SDK: Tests'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-tests.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'prowler/**'
|
||||
- 'tests/**'
|
||||
- 'pyproject.toml'
|
||||
- 'poetry.lock'
|
||||
- '.github/workflows/sdk-tests.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
sdk-tests:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 120
|
||||
permissions:
|
||||
contents: read
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- '3.9'
|
||||
- '3.10'
|
||||
- '3.11'
|
||||
- '3.12'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for SDK changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
prowler/CHANGELOG.md
|
||||
docs/**
|
||||
permissions/**
|
||||
api/**
|
||||
ui/**
|
||||
dashboard/**
|
||||
mcp_server/**
|
||||
README.md
|
||||
mkdocs.yml
|
||||
.backportrc.json
|
||||
.env
|
||||
docker-compose*
|
||||
examples/**
|
||||
.gitignore
|
||||
contrib/**
|
||||
|
||||
- name: Install Poetry
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: pipx install poetry==2.1.1
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
cache: 'poetry'
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: poetry install --no-root
|
||||
|
||||
# AWS Provider
|
||||
- name: Check if AWS files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-aws
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/aws/**
|
||||
./tests/**/aws/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run AWS tests
|
||||
if: steps.changed-aws.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml tests/providers/aws
|
||||
|
||||
- name: Upload AWS coverage to Codecov
|
||||
if: steps.changed-aws.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-aws
|
||||
files: ./aws_coverage.xml
|
||||
|
||||
# Azure Provider
|
||||
- name: Check if Azure files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-azure
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/azure/**
|
||||
./tests/**/azure/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run Azure tests
|
||||
if: steps.changed-azure.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/azure --cov-report=xml:azure_coverage.xml tests/providers/azure
|
||||
|
||||
- name: Upload Azure coverage to Codecov
|
||||
if: steps.changed-azure.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-azure
|
||||
files: ./azure_coverage.xml
|
||||
|
||||
# GCP Provider
|
||||
- name: Check if GCP files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-gcp
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/gcp/**
|
||||
./tests/**/gcp/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run GCP tests
|
||||
if: steps.changed-gcp.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/gcp --cov-report=xml:gcp_coverage.xml tests/providers/gcp
|
||||
|
||||
- name: Upload GCP coverage to Codecov
|
||||
if: steps.changed-gcp.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-gcp
|
||||
files: ./gcp_coverage.xml
|
||||
|
||||
# Kubernetes Provider
|
||||
- name: Check if Kubernetes files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-kubernetes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/kubernetes/**
|
||||
./tests/**/kubernetes/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run Kubernetes tests
|
||||
if: steps.changed-kubernetes.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/kubernetes --cov-report=xml:kubernetes_coverage.xml tests/providers/kubernetes
|
||||
|
||||
- name: Upload Kubernetes coverage to Codecov
|
||||
if: steps.changed-kubernetes.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-kubernetes
|
||||
files: ./kubernetes_coverage.xml
|
||||
|
||||
# GitHub Provider
|
||||
- name: Check if GitHub files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-github
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/github/**
|
||||
./tests/**/github/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run GitHub tests
|
||||
if: steps.changed-github.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/github --cov-report=xml:github_coverage.xml tests/providers/github
|
||||
|
||||
- name: Upload GitHub coverage to Codecov
|
||||
if: steps.changed-github.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-github
|
||||
files: ./github_coverage.xml
|
||||
|
||||
# NHN Provider
|
||||
- name: Check if NHN files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-nhn
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/nhn/**
|
||||
./tests/**/nhn/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run NHN tests
|
||||
if: steps.changed-nhn.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/nhn --cov-report=xml:nhn_coverage.xml tests/providers/nhn
|
||||
|
||||
- name: Upload NHN coverage to Codecov
|
||||
if: steps.changed-nhn.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-nhn
|
||||
files: ./nhn_coverage.xml
|
||||
|
||||
# M365 Provider
|
||||
- name: Check if M365 files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-m365
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/m365/**
|
||||
./tests/**/m365/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run M365 tests
|
||||
if: steps.changed-m365.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/m365 --cov-report=xml:m365_coverage.xml tests/providers/m365
|
||||
|
||||
- name: Upload M365 coverage to Codecov
|
||||
if: steps.changed-m365.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-m365
|
||||
files: ./m365_coverage.xml
|
||||
|
||||
# IaC Provider
|
||||
- name: Check if IaC files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-iac
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/iac/**
|
||||
./tests/**/iac/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run IaC tests
|
||||
if: steps.changed-iac.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/iac --cov-report=xml:iac_coverage.xml tests/providers/iac
|
||||
|
||||
- name: Upload IaC coverage to Codecov
|
||||
if: steps.changed-iac.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-iac
|
||||
files: ./iac_coverage.xml
|
||||
|
||||
# MongoDB Atlas Provider
|
||||
- name: Check if MongoDB Atlas files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-mongodbatlas
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/mongodbatlas/**
|
||||
./tests/**/mongodbatlas/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run MongoDB Atlas tests
|
||||
if: steps.changed-mongodbatlas.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodbatlas_coverage.xml tests/providers/mongodbatlas
|
||||
|
||||
- name: Upload MongoDB Atlas coverage to Codecov
|
||||
if: steps.changed-mongodbatlas.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-mongodbatlas
|
||||
files: ./mongodbatlas_coverage.xml
|
||||
|
||||
# OCI Provider
|
||||
- name: Check if OCI files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-oraclecloud
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/**/oraclecloud/**
|
||||
./tests/**/oraclecloud/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run OCI tests
|
||||
if: steps.changed-oraclecloud.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oraclecloud_coverage.xml tests/providers/oraclecloud
|
||||
|
||||
- name: Upload OCI coverage to Codecov
|
||||
if: steps.changed-oraclecloud.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-oraclecloud
|
||||
files: ./oraclecloud_coverage.xml
|
||||
|
||||
# Lib
|
||||
- name: Check if Lib files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-lib
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/lib/**
|
||||
./tests/lib/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run Lib tests
|
||||
if: steps.changed-lib.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/lib --cov-report=xml:lib_coverage.xml tests/lib
|
||||
|
||||
- name: Upload Lib coverage to Codecov
|
||||
if: steps.changed-lib.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-lib
|
||||
files: ./lib_coverage.xml
|
||||
|
||||
# Config
|
||||
- name: Check if Config files changed
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
id: changed-config
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: |
|
||||
./prowler/config/**
|
||||
./tests/config/**
|
||||
./poetry.lock
|
||||
|
||||
- name: Run Config tests
|
||||
if: steps.changed-config.outputs.any_changed == 'true'
|
||||
run: poetry run pytest -n auto --cov=./prowler/config --cov-report=xml:config_coverage.xml tests/config
|
||||
|
||||
- name: Upload Config coverage to Codecov
|
||||
if: steps.changed-config.outputs.any_changed == 'true'
|
||||
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
|
||||
env:
|
||||
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
|
||||
with:
|
||||
flags: prowler-py${{ matrix.python-version }}-config
|
||||
files: ./config_coverage.xml
|
||||
@@ -1,121 +0,0 @@
|
||||
name: UI - Build and Push containers
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
paths:
|
||||
- "ui/**"
|
||||
- ".github/workflows/ui-build-lint-push-containers.yml"
|
||||
|
||||
# Uncomment the below code to test this action on PRs
|
||||
# pull_request:
|
||||
# branches:
|
||||
# - "master"
|
||||
# paths:
|
||||
# - "ui/**"
|
||||
# - ".github/workflows/ui-build-lint-push-containers.yml"
|
||||
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
env:
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
STABLE_TAG: stable
|
||||
|
||||
WORKING_DIRECTORY: ./ui
|
||||
|
||||
# Container Registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-ui
|
||||
NEXT_PUBLIC_API_BASE_URL: http://prowler-api:8080/api/v1
|
||||
|
||||
jobs:
|
||||
repository-check:
|
||||
name: Repository check
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
is_repo: ${{ steps.repository_check.outputs.is_repo }}
|
||||
steps:
|
||||
- name: Repository check
|
||||
id: repository_check
|
||||
working-directory: /tmp
|
||||
run: |
|
||||
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
|
||||
then
|
||||
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
|
||||
else
|
||||
echo "This action only runs for prowler-cloud/prowler"
|
||||
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
|
||||
fi
|
||||
|
||||
# Build Prowler OSS container
|
||||
container-build-push:
|
||||
needs: repository-check
|
||||
if: needs.repository-check.outputs.is_repo == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ env.WORKING_DIRECTORY }}
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Set short git commit SHA
|
||||
id: vars
|
||||
run: |
|
||||
shortSha=$(git rev-parse --short ${{ github.sha }})
|
||||
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push container image (latest)
|
||||
# Comment the following line for testing
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
build-args: |
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=${{ env.SHORT_SHA }}
|
||||
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
|
||||
# Set push: false for testing
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.SHORT_SHA }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Build and push container image (release)
|
||||
if: github.event_name == 'release'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
build-args: |
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${{ env.RELEASE_TAG }}
|
||||
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Trigger deployment
|
||||
if: github.event_name == 'push'
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.CLOUD_DISPATCH }}
|
||||
event-type: prowler-ui-deploy
|
||||
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ env.SHORT_SHA }}"}'
|
||||
@@ -1,36 +1,36 @@
|
||||
# For most projects, this workflow file will not need changing; you simply need
|
||||
# to commit it to your repository.
|
||||
#
|
||||
# You may wish to alter this file to override the set of languages analyzed,
|
||||
# or to provide custom queries or build logic.
|
||||
#
|
||||
# ******** NOTE ********
|
||||
# We have attempted to detect the languages in your repository. Please check
|
||||
# the `language` matrix defined below to confirm you have the correct set of
|
||||
# supported CodeQL languages.
|
||||
#
|
||||
name: UI - CodeQL
|
||||
name: 'UI: CodeQL'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- "ui/**"
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-codeql.yml'
|
||||
- '.github/codeql/ui-codeql-config.yml'
|
||||
- '!ui/CHANGELOG.md'
|
||||
pull_request:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- "ui/**"
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-codeql.yml'
|
||||
- '.github/codeql/ui-codeql-config.yml'
|
||||
- '!ui/CHANGELOG.md'
|
||||
schedule:
|
||||
- cron: "00 12 * * *"
|
||||
- cron: '00 12 * * *'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
analyze:
|
||||
name: Analyze
|
||||
name: CodeQL Security Analysis
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
@@ -39,14 +39,13 @@ jobs:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
language: ["javascript"]
|
||||
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
|
||||
language:
|
||||
- 'javascript-typescript'
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
# Initializes the CodeQL tools for scanning.
|
||||
- name: Initialize CodeQL
|
||||
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
@@ -56,4 +55,4 @@ jobs:
|
||||
- name: Perform CodeQL Analysis
|
||||
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
|
||||
with:
|
||||
category: "/language:${{matrix.language}}"
|
||||
category: '/language:${{ matrix.language }}'
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
name: 'UI: Container Build and Push'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
paths:
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-container-build-push.yml'
|
||||
release:
|
||||
types:
|
||||
- 'published'
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
# Tags
|
||||
LATEST_TAG: latest
|
||||
RELEASE_TAG: ${{ github.event.release.tag_name }}
|
||||
STABLE_TAG: stable
|
||||
WORKING_DIRECTORY: ./ui
|
||||
|
||||
# Container registries
|
||||
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
|
||||
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-ui
|
||||
|
||||
# Build args
|
||||
NEXT_PUBLIC_API_BASE_URL: http://prowler-api:8080/api/v1
|
||||
|
||||
jobs:
|
||||
setup:
|
||||
if: github.repository == 'prowler-cloud/prowler'
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
|
||||
steps:
|
||||
- name: Calculate short SHA
|
||||
id: set-short-sha
|
||||
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
|
||||
|
||||
container-build-push:
|
||||
needs: setup
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Login to DockerHub
|
||||
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build and push UI container (latest)
|
||||
if: github.event_name == 'push'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
build-args: |
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=${{ needs.setup.outputs.short-sha }}
|
||||
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push started
|
||||
if: github.event_name == 'release'
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: UI
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
|
||||
|
||||
- name: Build and push UI container (release)
|
||||
if: github.event_name == 'release'
|
||||
id: container-push
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.WORKING_DIRECTORY }}
|
||||
build-args: |
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${{ env.RELEASE_TAG }}
|
||||
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
|
||||
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
- name: Notify container push completed
|
||||
if: github.event_name == 'release' && always()
|
||||
uses: ./.github/actions/slack-notification
|
||||
env:
|
||||
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
|
||||
COMPONENT: UI
|
||||
RELEASE_TAG: ${{ env.RELEASE_TAG }}
|
||||
GITHUB_SERVER_URL: ${{ github.server_url }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_RUN_ID: ${{ github.run_id }}
|
||||
with:
|
||||
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
|
||||
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
|
||||
step-outcome: ${{ steps.container-push.outcome }}
|
||||
|
||||
trigger-deployment:
|
||||
if: github.event_name == 'push'
|
||||
needs: [setup, container-build-push]
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Trigger UI deployment
|
||||
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
|
||||
with:
|
||||
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
|
||||
repository: ${{ secrets.CLOUD_DISPATCH }}
|
||||
event-type: ui-prowler-deployment
|
||||
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
|
||||
@@ -0,0 +1,96 @@
|
||||
name: 'UI: Container Checks'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-container-checks.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-container-checks.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
UI_WORKING_DIR: ./ui
|
||||
IMAGE_NAME: prowler-ui
|
||||
|
||||
jobs:
|
||||
ui-dockerfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check if Dockerfile changed
|
||||
id: dockerfile-changed
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files: ui/Dockerfile
|
||||
|
||||
- name: Lint Dockerfile with Hadolint
|
||||
if: steps.dockerfile-changed.outputs.any_changed == 'true'
|
||||
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
|
||||
with:
|
||||
dockerfile: ui/Dockerfile
|
||||
ignore: DL3018
|
||||
|
||||
ui-container-build-and-scan:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 30
|
||||
permissions:
|
||||
contents: read
|
||||
security-events: write
|
||||
pull-requests: write
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for UI changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
ui/CHANGELOG.md
|
||||
ui/README.md
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
|
||||
- name: Build UI container
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.UI_WORKING_DIR }}
|
||||
target: prod
|
||||
push: false
|
||||
load: true
|
||||
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
build-args: |
|
||||
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_51LwpXXXX
|
||||
|
||||
- name: Scan UI container with Trivy
|
||||
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: ./.github/actions/trivy-scan
|
||||
with:
|
||||
image-name: ${{ env.IMAGE_NAME }}
|
||||
image-tag: ${{ github.sha }}
|
||||
fail-on-critical: 'false'
|
||||
severity: 'CRITICAL'
|
||||
@@ -18,6 +18,13 @@ jobs:
|
||||
AUTH_TRUST_HOST: true
|
||||
NEXTAUTH_URL: 'http://localhost:3000'
|
||||
NEXT_PUBLIC_API_BASE_URL: 'http://localhost:8080/api/v1'
|
||||
E2E_ADMIN_USER: ${{ secrets.E2E_ADMIN_USER }}
|
||||
E2E_ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD }}
|
||||
E2E_AWS_PROVIDER_ACCOUNT_ID: ${{ secrets.E2E_AWS_PROVIDER_ACCOUNT_ID }}
|
||||
E2E_AWS_PROVIDER_ACCESS_KEY: ${{ secrets.E2E_AWS_PROVIDER_ACCESS_KEY }}
|
||||
E2E_AWS_PROVIDER_SECRET_KEY: ${{ secrets.E2E_AWS_PROVIDER_SECRET_KEY }}
|
||||
E2E_AWS_PROVIDER_ROLE_ARN: ${{ secrets.E2E_AWS_PROVIDER_ROLE_ARN }}
|
||||
E2E_NEW_PASSWORD: ${{ secrets.E2E_NEW_PASSWORD }}
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
name: UI - Pull Request
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- "master"
|
||||
- "v5.*"
|
||||
paths:
|
||||
- ".github/workflows/ui-pull-request.yml"
|
||||
- "ui/**"
|
||||
pull_request:
|
||||
branches:
|
||||
- master
|
||||
- "v5.*"
|
||||
paths:
|
||||
- 'ui/**'
|
||||
env:
|
||||
UI_WORKING_DIR: ./ui
|
||||
IMAGE_NAME: prowler-ui
|
||||
|
||||
jobs:
|
||||
test-and-coverage:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest]
|
||||
node-version: [20.x]
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
with:
|
||||
persist-credentials: false
|
||||
- name: Setup Node.js ${{ matrix.node-version }}
|
||||
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
|
||||
with:
|
||||
node-version: ${{ matrix.node-version }}
|
||||
cache: 'npm'
|
||||
cache-dependency-path: './ui/package-lock.json'
|
||||
- name: Install dependencies
|
||||
working-directory: ./ui
|
||||
run: npm ci
|
||||
- name: Run Healthcheck
|
||||
working-directory: ./ui
|
||||
run: npm run healthcheck
|
||||
- name: Build the application
|
||||
working-directory: ./ui
|
||||
run: npm run build
|
||||
|
||||
test-container-build:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
|
||||
- name: Build Container
|
||||
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
|
||||
with:
|
||||
context: ${{ env.UI_WORKING_DIR }}
|
||||
# Always build using `prod` target
|
||||
target: prod
|
||||
push: false
|
||||
tags: ${{ env.IMAGE_NAME }}:latest
|
||||
outputs: type=docker
|
||||
build-args: |
|
||||
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_51LwpXXXX
|
||||
@@ -0,0 +1,67 @@
|
||||
name: 'UI: Tests'
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-tests.yml'
|
||||
pull_request:
|
||||
branches:
|
||||
- 'master'
|
||||
- 'v5.*'
|
||||
paths:
|
||||
- 'ui/**'
|
||||
- '.github/workflows/ui-tests.yml'
|
||||
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
UI_WORKING_DIR: ./ui
|
||||
NODE_VERSION: '20.x'
|
||||
|
||||
jobs:
|
||||
ui-tests:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
permissions:
|
||||
contents: read
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./ui
|
||||
|
||||
steps:
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
|
||||
|
||||
- name: Check for UI changes
|
||||
id: check-changes
|
||||
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
|
||||
with:
|
||||
files_ignore: |
|
||||
ui/CHANGELOG.md
|
||||
ui/README.md
|
||||
|
||||
- name: Setup Node.js ${{ env.NODE_VERSION }}
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
cache: 'npm'
|
||||
cache-dependency-path: './ui/package-lock.json'
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: npm ci
|
||||
|
||||
- name: Run healthcheck
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: npm run healthcheck
|
||||
|
||||
- name: Build application
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
run: npm run build
|
||||
@@ -39,6 +39,12 @@ secrets-*/
|
||||
# JUnit Reports
|
||||
junit-reports/
|
||||
|
||||
# Test and coverage artifacts
|
||||
*_coverage.xml
|
||||
pytest_*.xml
|
||||
.coverage
|
||||
htmlcov/
|
||||
|
||||
# VSCode files
|
||||
.vscode/
|
||||
|
||||
|
||||
+1
-1
@@ -10,4 +10,4 @@
|
||||
Want some swag as appreciation for your contribution?
|
||||
|
||||
# Prowler Developer Guide
|
||||
https://docs.prowler.com/projects/prowler-open-source/en/latest/developer-guide/introduction/
|
||||
https://goto.prowler.com/devguide
|
||||
|
||||
@@ -88,7 +88,7 @@ prowler dashboard
|
||||
| Kubernetes | 83 | 7 | 5 | 7 | Official | Stable | UI, API, CLI |
|
||||
| GitHub | 17 | 2 | 1 | 0 | Official | Stable | UI, API, CLI |
|
||||
| M365 | 70 | 7 | 3 | 2 | Official | Stable | UI, API, CLI |
|
||||
| OCI | 51 | 13 | 1 | 10 | Official | Stable | CLI |
|
||||
| OCI | 51 | 13 | 1 | 10 | Official | Stable | UI, API, CLI |
|
||||
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | Beta | CLI |
|
||||
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | Beta | CLI |
|
||||
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | Beta | CLI |
|
||||
|
||||
@@ -2,6 +2,22 @@
|
||||
|
||||
All notable changes to the **Prowler API** are documented in this file.
|
||||
|
||||
## [1.15.0] (Prowler UNRELEASED)
|
||||
|
||||
### Added
|
||||
- Extend `GET /api/v1/providers` with provider-type filters and optional pagination disable to support the new Overview filters [(#8975)](https://github.com/prowler-cloud/prowler/pull/8975)
|
||||
- New endpoint to retrieve the number of providers grouped by provider type [(#8975)](https://github.com/prowler-cloud/prowler/pull/8975)
|
||||
- Support for configuring multiple LLM providers [(#8772)](https://github.com/prowler-cloud/prowler/pull/8772)
|
||||
- Support C5 compliance framework for Azure provider [(#9081)](https://github.com/prowler-cloud/prowler/pull/9081)
|
||||
- Support for Oracle Cloud Infrastructure (OCI) provider [(#8927)](https://github.com/prowler-cloud/prowler/pull/8927)
|
||||
- Support C5 compliance framework for the GCP provider [(#9097)](https://github.com/prowler-cloud/prowler/pull/9097)
|
||||
|
||||
## [1.14.1] (Prowler 5.13.1)
|
||||
|
||||
### Fixed
|
||||
- `/api/v1/overviews/providers` collapses data by provider type so the UI receives a single aggregated record per cloud family even when multiple accounts exist [(#9053)](https://github.com/prowler-cloud/prowler/pull/9053)
|
||||
- Security Hub integrations stop failing when they read relationships via the replica by allowing replica relations and saving updates through the primary [(#9080)](https://github.com/prowler-cloud/prowler/pull/9080)
|
||||
|
||||
## [1.14.0] (Prowler 5.13.0)
|
||||
|
||||
### Added
|
||||
|
||||
+1
-1
@@ -43,7 +43,7 @@ name = "prowler-api"
|
||||
package-mode = false
|
||||
# Needed for the SDK compatibility
|
||||
requires-python = ">=3.11,<3.13"
|
||||
version = "1.14.0"
|
||||
version = "1.15.0"
|
||||
|
||||
[project.scripts]
|
||||
celery = "src.backend.config.settings.celery"
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
# from django.contrib import admin
|
||||
|
||||
# Register your models here.
|
||||
@@ -9,6 +9,25 @@ PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE = {}
|
||||
PROWLER_CHECKS = {}
|
||||
AVAILABLE_COMPLIANCE_FRAMEWORKS = {}
|
||||
|
||||
# Map API provider names to Prowler directory names
|
||||
# This is needed because the OCI provider directory is 'oraclecloud' but the provider type is 'oci'
|
||||
PROVIDER_NAME_MAPPING = {
|
||||
"oci": "oraclecloud",
|
||||
}
|
||||
|
||||
|
||||
def get_prowler_provider_name(provider_type: str) -> str:
|
||||
"""
|
||||
Map API provider type to Prowler provider directory name.
|
||||
|
||||
Args:
|
||||
provider_type: The provider type from the API (e.g., 'oci', 'aws', 'azure')
|
||||
|
||||
Returns:
|
||||
The provider name used in Prowler's directory structure (e.g., 'oraclecloud', 'aws', 'azure')
|
||||
"""
|
||||
return PROVIDER_NAME_MAPPING.get(provider_type, provider_type)
|
||||
|
||||
|
||||
def get_compliance_frameworks(provider_type: Provider.ProviderChoices) -> list[str]:
|
||||
"""
|
||||
@@ -28,8 +47,9 @@ def get_compliance_frameworks(provider_type: Provider.ProviderChoices) -> list[s
|
||||
"""
|
||||
global AVAILABLE_COMPLIANCE_FRAMEWORKS
|
||||
if provider_type not in AVAILABLE_COMPLIANCE_FRAMEWORKS:
|
||||
prowler_provider_name = get_prowler_provider_name(provider_type)
|
||||
AVAILABLE_COMPLIANCE_FRAMEWORKS[provider_type] = (
|
||||
get_available_compliance_frameworks(provider_type)
|
||||
get_available_compliance_frameworks(prowler_provider_name)
|
||||
)
|
||||
|
||||
return AVAILABLE_COMPLIANCE_FRAMEWORKS[provider_type]
|
||||
@@ -49,7 +69,8 @@ def get_prowler_provider_checks(provider_type: Provider.ProviderChoices):
|
||||
Returns:
|
||||
Iterable[str]: An iterable of check IDs associated with the specified provider type.
|
||||
"""
|
||||
return CheckMetadata.get_bulk(provider_type).keys()
|
||||
prowler_provider_name = get_prowler_provider_name(provider_type)
|
||||
return CheckMetadata.get_bulk(prowler_provider_name).keys()
|
||||
|
||||
|
||||
def get_prowler_provider_compliance(provider_type: Provider.ProviderChoices) -> dict:
|
||||
@@ -67,7 +88,8 @@ def get_prowler_provider_compliance(provider_type: Provider.ProviderChoices) ->
|
||||
dict: A dictionary mapping compliance framework names to their respective
|
||||
Compliance objects for the specified provider.
|
||||
"""
|
||||
return Compliance.get_bulk(provider_type)
|
||||
prowler_provider_name = get_prowler_provider_name(provider_type)
|
||||
return Compliance.get_bulk(prowler_provider_name)
|
||||
|
||||
|
||||
def load_prowler_compliance():
|
||||
|
||||
@@ -48,8 +48,9 @@ class MainRouter:
|
||||
return db == self.admin_db
|
||||
|
||||
def allow_relation(self, obj1, obj2, **hints): # noqa: F841
|
||||
# Allow relations if both objects are in either "default" or "admin" db connectors
|
||||
if {obj1._state.db, obj2._state.db} <= {self.default_db, self.admin_db}:
|
||||
# Allow relations when both objects originate from allowed connectors
|
||||
allowed_dbs = {self.default_db, self.admin_db, self.replica_db}
|
||||
if {obj1._state.db, obj2._state.db} <= allowed_dbs:
|
||||
return True
|
||||
return None
|
||||
|
||||
|
||||
@@ -27,6 +27,8 @@ from api.models import (
|
||||
Finding,
|
||||
Integration,
|
||||
Invitation,
|
||||
LighthouseProviderConfiguration,
|
||||
LighthouseProviderModels,
|
||||
Membership,
|
||||
OverviewStatusChoices,
|
||||
PermissionChoices,
|
||||
@@ -245,6 +247,14 @@ class ProviderFilter(FilterSet):
|
||||
choices=Provider.ProviderChoices.choices,
|
||||
lookup_expr="in",
|
||||
)
|
||||
provider_type = ChoiceFilter(
|
||||
choices=Provider.ProviderChoices.choices, field_name="provider"
|
||||
)
|
||||
provider_type__in = ChoiceInFilter(
|
||||
field_name="provider",
|
||||
choices=Provider.ProviderChoices.choices,
|
||||
lookup_expr="in",
|
||||
)
|
||||
|
||||
class Meta:
|
||||
model = Provider
|
||||
@@ -928,3 +938,45 @@ class TenantApiKeyFilter(FilterSet):
|
||||
"revoked": ["exact"],
|
||||
"name": ["exact", "icontains"],
|
||||
}
|
||||
|
||||
|
||||
class LighthouseProviderConfigFilter(FilterSet):
|
||||
provider_type = ChoiceFilter(
|
||||
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices
|
||||
)
|
||||
provider_type__in = ChoiceInFilter(
|
||||
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
|
||||
field_name="provider_type",
|
||||
lookup_expr="in",
|
||||
)
|
||||
is_active = BooleanFilter()
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderConfiguration
|
||||
fields = {
|
||||
"provider_type": ["exact", "in"],
|
||||
"is_active": ["exact"],
|
||||
}
|
||||
|
||||
|
||||
class LighthouseProviderModelsFilter(FilterSet):
|
||||
provider_type = ChoiceFilter(
|
||||
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
|
||||
field_name="provider_configuration__provider_type",
|
||||
)
|
||||
provider_type__in = ChoiceInFilter(
|
||||
choices=LighthouseProviderConfiguration.LLMProviderChoices.choices,
|
||||
field_name="provider_configuration__provider_type",
|
||||
lookup_expr="in",
|
||||
)
|
||||
|
||||
# Allow filtering by model id
|
||||
model_id = CharFilter(field_name="model_id", lookup_expr="exact")
|
||||
model_id__icontains = CharFilter(field_name="model_id", lookup_expr="icontains")
|
||||
model_id__in = CharInFilter(field_name="model_id", lookup_expr="in")
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderModels
|
||||
fields = {
|
||||
"model_id": ["exact", "icontains", "in"],
|
||||
}
|
||||
|
||||
@@ -0,0 +1,266 @@
|
||||
# Generated by Django 5.1.12 on 2025-10-09 07:50
|
||||
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
|
||||
import django.db.models.deletion
|
||||
from config.custom_logging import BackendLogger
|
||||
from cryptography.fernet import Fernet
|
||||
from django.conf import settings
|
||||
from django.db import migrations, models
|
||||
|
||||
import api.rls
|
||||
from api.db_router import MainRouter
|
||||
|
||||
logger = logging.getLogger(BackendLogger.API)
|
||||
|
||||
|
||||
def migrate_lighthouse_configs_forward(apps, schema_editor):
|
||||
"""
|
||||
Migrate data from old LighthouseConfiguration to new multi-provider models.
|
||||
Old system: one LighthouseConfiguration per tenant (always OpenAI).
|
||||
"""
|
||||
LighthouseConfiguration = apps.get_model("api", "LighthouseConfiguration")
|
||||
LighthouseProviderConfiguration = apps.get_model(
|
||||
"api", "LighthouseProviderConfiguration"
|
||||
)
|
||||
LighthouseTenantConfiguration = apps.get_model(
|
||||
"api", "LighthouseTenantConfiguration"
|
||||
)
|
||||
LighthouseProviderModels = apps.get_model("api", "LighthouseProviderModels")
|
||||
|
||||
fernet = Fernet(settings.SECRETS_ENCRYPTION_KEY.encode())
|
||||
|
||||
# Migrate only tenants that actually have a LighthouseConfiguration
|
||||
for old_config in (
|
||||
LighthouseConfiguration.objects.using(MainRouter.admin_db)
|
||||
.select_related("tenant")
|
||||
.all()
|
||||
):
|
||||
tenant = old_config.tenant
|
||||
tenant_id = str(tenant.id)
|
||||
|
||||
try:
|
||||
# Create OpenAI provider configuration for this tenant
|
||||
api_key_decrypted = fernet.decrypt(bytes(old_config.api_key)).decode()
|
||||
credentials_encrypted = fernet.encrypt(
|
||||
json.dumps({"api_key": api_key_decrypted}).encode()
|
||||
)
|
||||
provider_config = LighthouseProviderConfiguration.objects.using(
|
||||
MainRouter.admin_db
|
||||
).create(
|
||||
tenant=tenant,
|
||||
provider_type="openai",
|
||||
credentials=credentials_encrypted,
|
||||
is_active=old_config.is_active,
|
||||
)
|
||||
|
||||
# Create tenant configuration from old values
|
||||
LighthouseTenantConfiguration.objects.using(MainRouter.admin_db).create(
|
||||
tenant=tenant,
|
||||
business_context=old_config.business_context or "",
|
||||
default_provider="openai",
|
||||
default_models={"openai": old_config.model},
|
||||
)
|
||||
|
||||
# Create initial provider model record
|
||||
LighthouseProviderModels.objects.using(MainRouter.admin_db).create(
|
||||
tenant=tenant,
|
||||
provider_configuration=provider_config,
|
||||
model_id=old_config.model,
|
||||
model_name=old_config.model,
|
||||
default_parameters={},
|
||||
)
|
||||
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Failed to migrate lighthouse config for tenant %s", tenant_id
|
||||
)
|
||||
continue
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
dependencies = [
|
||||
("api", "0049_compliancerequirementoverview_passed_failed_findings"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.CreateModel(
|
||||
name="LighthouseProviderConfiguration",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.UUIDField(
|
||||
default=uuid.uuid4,
|
||||
editable=False,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
),
|
||||
),
|
||||
("inserted_at", models.DateTimeField(auto_now_add=True)),
|
||||
("updated_at", models.DateTimeField(auto_now=True)),
|
||||
(
|
||||
"provider_type",
|
||||
models.CharField(
|
||||
choices=[("openai", "OpenAI")],
|
||||
help_text="LLM provider name",
|
||||
max_length=50,
|
||||
),
|
||||
),
|
||||
("base_url", models.URLField(blank=True, null=True)),
|
||||
(
|
||||
"credentials",
|
||||
models.BinaryField(
|
||||
help_text="Encrypted JSON credentials for the provider"
|
||||
),
|
||||
),
|
||||
("is_active", models.BooleanField(default=True)),
|
||||
],
|
||||
options={
|
||||
"db_table": "lighthouse_provider_configurations",
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
migrations.CreateModel(
|
||||
name="LighthouseProviderModels",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.UUIDField(
|
||||
default=uuid.uuid4,
|
||||
editable=False,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
),
|
||||
),
|
||||
("inserted_at", models.DateTimeField(auto_now_add=True)),
|
||||
("updated_at", models.DateTimeField(auto_now=True)),
|
||||
("model_id", models.CharField(max_length=100)),
|
||||
("model_name", models.CharField(max_length=100)),
|
||||
("default_parameters", models.JSONField(blank=True, default=dict)),
|
||||
],
|
||||
options={
|
||||
"db_table": "lighthouse_provider_models",
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
migrations.CreateModel(
|
||||
name="LighthouseTenantConfiguration",
|
||||
fields=[
|
||||
(
|
||||
"id",
|
||||
models.UUIDField(
|
||||
default=uuid.uuid4,
|
||||
editable=False,
|
||||
primary_key=True,
|
||||
serialize=False,
|
||||
),
|
||||
),
|
||||
("inserted_at", models.DateTimeField(auto_now_add=True)),
|
||||
("updated_at", models.DateTimeField(auto_now=True)),
|
||||
("business_context", models.TextField(blank=True, default="")),
|
||||
("default_provider", models.CharField(blank=True, max_length=50)),
|
||||
("default_models", models.JSONField(blank=True, default=dict)),
|
||||
],
|
||||
options={
|
||||
"db_table": "lighthouse_tenant_config",
|
||||
"abstract": False,
|
||||
},
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="lighthouseproviderconfiguration",
|
||||
name="tenant",
|
||||
field=models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
|
||||
),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="lighthouseprovidermodels",
|
||||
name="provider_configuration",
|
||||
field=models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE,
|
||||
related_name="available_models",
|
||||
to="api.lighthouseproviderconfiguration",
|
||||
),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="lighthouseprovidermodels",
|
||||
name="tenant",
|
||||
field=models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
|
||||
),
|
||||
),
|
||||
migrations.AddField(
|
||||
model_name="lighthousetenantconfiguration",
|
||||
name="tenant",
|
||||
field=models.ForeignKey(
|
||||
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
|
||||
),
|
||||
),
|
||||
migrations.AddIndex(
|
||||
model_name="lighthouseproviderconfiguration",
|
||||
index=models.Index(
|
||||
fields=["tenant_id", "provider_type"], name="lh_pc_tenant_type_idx"
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthouseproviderconfiguration",
|
||||
constraint=api.rls.RowLevelSecurityConstraint(
|
||||
"tenant_id",
|
||||
name="rls_on_lighthouseproviderconfiguration",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthouseproviderconfiguration",
|
||||
constraint=models.UniqueConstraint(
|
||||
fields=("tenant_id", "provider_type"),
|
||||
name="unique_provider_config_per_tenant",
|
||||
),
|
||||
),
|
||||
migrations.AddIndex(
|
||||
model_name="lighthouseprovidermodels",
|
||||
index=models.Index(
|
||||
fields=["tenant_id", "provider_configuration"],
|
||||
name="lh_prov_models_cfg_idx",
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthouseprovidermodels",
|
||||
constraint=api.rls.RowLevelSecurityConstraint(
|
||||
"tenant_id",
|
||||
name="rls_on_lighthouseprovidermodels",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthouseprovidermodels",
|
||||
constraint=models.UniqueConstraint(
|
||||
fields=("tenant_id", "provider_configuration", "model_id"),
|
||||
name="unique_provider_model_per_configuration",
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthousetenantconfiguration",
|
||||
constraint=api.rls.RowLevelSecurityConstraint(
|
||||
"tenant_id",
|
||||
name="rls_on_lighthousetenantconfiguration",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
),
|
||||
migrations.AddConstraint(
|
||||
model_name="lighthousetenantconfiguration",
|
||||
constraint=models.UniqueConstraint(
|
||||
fields=("tenant_id",), name="unique_tenant_lighthouse_config"
|
||||
),
|
||||
),
|
||||
# Migrate data from old LighthouseConfiguration to new tables
|
||||
# This runs after all tables, indexes, and constraints are created
|
||||
# The old Lighthouse configuration table is not removed, so reverse_code is noop
|
||||
# During rollbacks, the old Lighthouse configuration remains intact while the new tables are removed
|
||||
migrations.RunPython(
|
||||
migrate_lighthouse_configs_forward,
|
||||
reverse_code=migrations.RunPython.noop,
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,34 @@
|
||||
# Generated by Django 5.1.7 on 2025-10-14 00:00
|
||||
|
||||
from django.db import migrations
|
||||
|
||||
import api.db_utils
|
||||
|
||||
|
||||
class Migration(migrations.Migration):
|
||||
dependencies = [
|
||||
("api", "0050_lighthouse_multi_llm"),
|
||||
]
|
||||
|
||||
operations = [
|
||||
migrations.AlterField(
|
||||
model_name="provider",
|
||||
name="provider",
|
||||
field=api.db_utils.ProviderEnumField(
|
||||
choices=[
|
||||
("aws", "AWS"),
|
||||
("azure", "Azure"),
|
||||
("gcp", "GCP"),
|
||||
("kubernetes", "Kubernetes"),
|
||||
("m365", "M365"),
|
||||
("github", "GitHub"),
|
||||
("oci", "Oracle Cloud Infrastructure"),
|
||||
],
|
||||
default="aws",
|
||||
),
|
||||
),
|
||||
migrations.RunSQL(
|
||||
"ALTER TYPE provider ADD VALUE IF NOT EXISTS 'oci';",
|
||||
reverse_sql=migrations.RunSQL.noop,
|
||||
),
|
||||
]
|
||||
+197
-25
@@ -284,6 +284,7 @@ class Provider(RowLevelSecurityProtectedModel):
|
||||
KUBERNETES = "kubernetes", _("Kubernetes")
|
||||
M365 = "m365", _("M365")
|
||||
GITHUB = "github", _("GitHub")
|
||||
OCI = "oci", _("Oracle Cloud Infrastructure")
|
||||
|
||||
@staticmethod
|
||||
def validate_aws_uid(value):
|
||||
@@ -354,6 +355,18 @@ class Provider(RowLevelSecurityProtectedModel):
|
||||
pointer="/data/attributes/uid",
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def validate_oci_uid(value):
|
||||
if not re.match(
|
||||
r"^ocid1\.([a-z0-9_-]+)\.([a-z0-9_-]+)\.([a-z0-9_-]*)\.([a-z0-9]+)$", value
|
||||
):
|
||||
raise ModelValidationError(
|
||||
detail="Oracle Cloud Infrastructure provider ID must be a valid tenancy OCID in the format: "
|
||||
"ocid1.<resource_type>.<realm>.<region>.<unique_id>",
|
||||
code="oci-uid",
|
||||
pointer="/data/attributes/uid",
|
||||
)
|
||||
|
||||
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
|
||||
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
|
||||
updated_at = models.DateTimeField(auto_now=True, editable=False)
|
||||
@@ -1873,22 +1886,6 @@ class LighthouseConfiguration(RowLevelSecurityProtectedModel):
|
||||
def clean(self):
|
||||
super().clean()
|
||||
|
||||
# Validate temperature
|
||||
if not 0 <= self.temperature <= 1:
|
||||
raise ModelValidationError(
|
||||
detail="Temperature must be between 0 and 1",
|
||||
code="invalid_temperature",
|
||||
pointer="/data/attributes/temperature",
|
||||
)
|
||||
|
||||
# Validate max_tokens
|
||||
if not 500 <= self.max_tokens <= 5000:
|
||||
raise ModelValidationError(
|
||||
detail="Max tokens must be between 500 and 5000",
|
||||
code="invalid_max_tokens",
|
||||
pointer="/data/attributes/max_tokens",
|
||||
)
|
||||
|
||||
@property
|
||||
def api_key_decoded(self):
|
||||
"""Return the decrypted API key, or None if unavailable or invalid."""
|
||||
@@ -1913,15 +1910,6 @@ class LighthouseConfiguration(RowLevelSecurityProtectedModel):
|
||||
code="invalid_api_key",
|
||||
pointer="/data/attributes/api_key",
|
||||
)
|
||||
|
||||
# Validate OpenAI API key format
|
||||
openai_key_pattern = r"^sk-[\w-]+T3BlbkFJ[\w-]+$"
|
||||
if not re.match(openai_key_pattern, value):
|
||||
raise ModelValidationError(
|
||||
detail="Invalid OpenAI API key format.",
|
||||
code="invalid_api_key",
|
||||
pointer="/data/attributes/api_key",
|
||||
)
|
||||
self.api_key = fernet.encrypt(value.encode())
|
||||
|
||||
def save(self, *args, **kwargs):
|
||||
@@ -1984,3 +1972,187 @@ class Processor(RowLevelSecurityProtectedModel):
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "processors"
|
||||
|
||||
|
||||
class LighthouseProviderConfiguration(RowLevelSecurityProtectedModel):
|
||||
"""
|
||||
Per-tenant configuration for an LLM provider (credentials, base URL, activation).
|
||||
|
||||
One configuration per provider type per tenant.
|
||||
"""
|
||||
|
||||
class LLMProviderChoices(models.TextChoices):
|
||||
OPENAI = "openai", _("OpenAI")
|
||||
|
||||
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
|
||||
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
|
||||
updated_at = models.DateTimeField(auto_now=True, editable=False)
|
||||
|
||||
provider_type = models.CharField(
|
||||
max_length=50,
|
||||
choices=LLMProviderChoices.choices,
|
||||
help_text="LLM provider name",
|
||||
)
|
||||
|
||||
# For OpenAI-compatible providers
|
||||
base_url = models.URLField(blank=True, null=True)
|
||||
|
||||
# Encrypted JSON for provider-specific auth
|
||||
credentials = models.BinaryField(
|
||||
blank=False, null=False, help_text="Encrypted JSON credentials for the provider"
|
||||
)
|
||||
|
||||
is_active = models.BooleanField(default=True)
|
||||
|
||||
def __str__(self):
|
||||
return f"{self.get_provider_type_display()} ({self.tenant_id})"
|
||||
|
||||
def clean(self):
|
||||
super().clean()
|
||||
|
||||
@property
|
||||
def credentials_decoded(self):
|
||||
if not self.credentials:
|
||||
return None
|
||||
try:
|
||||
decrypted_data = fernet.decrypt(bytes(self.credentials))
|
||||
return json.loads(decrypted_data.decode())
|
||||
except (InvalidToken, json.JSONDecodeError) as e:
|
||||
logger.warning("Failed to decrypt provider credentials: %s", e)
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.exception(
|
||||
"Unexpected error while decrypting provider credentials: %s", e
|
||||
)
|
||||
return None
|
||||
|
||||
@credentials_decoded.setter
|
||||
def credentials_decoded(self, value):
|
||||
"""
|
||||
Set and encrypt credentials (assumes serializer performed validation).
|
||||
"""
|
||||
if not value:
|
||||
raise ModelValidationError(
|
||||
detail="Credentials are required",
|
||||
code="invalid_credentials",
|
||||
pointer="/data/attributes/credentials",
|
||||
)
|
||||
self.credentials = fernet.encrypt(json.dumps(value).encode())
|
||||
|
||||
class Meta(RowLevelSecurityProtectedModel.Meta):
|
||||
db_table = "lighthouse_provider_configurations"
|
||||
|
||||
constraints = [
|
||||
RowLevelSecurityConstraint(
|
||||
field="tenant_id",
|
||||
name="rls_on_%(class)s",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
models.UniqueConstraint(
|
||||
fields=["tenant_id", "provider_type"],
|
||||
name="unique_provider_config_per_tenant",
|
||||
),
|
||||
]
|
||||
|
||||
indexes = [
|
||||
models.Index(
|
||||
fields=["tenant_id", "provider_type"],
|
||||
name="lh_pc_tenant_type_idx",
|
||||
),
|
||||
]
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "lighthouse-providers"
|
||||
|
||||
|
||||
class LighthouseTenantConfiguration(RowLevelSecurityProtectedModel):
|
||||
"""
|
||||
Tenant-level Lighthouse settings (business context and defaults).
|
||||
One record per tenant.
|
||||
"""
|
||||
|
||||
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
|
||||
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
|
||||
updated_at = models.DateTimeField(auto_now=True, editable=False)
|
||||
|
||||
business_context = models.TextField(blank=True, default="")
|
||||
|
||||
# Preferred provider key (e.g., "openai", "bedrock", "openai_compatible")
|
||||
default_provider = models.CharField(max_length=50, blank=True)
|
||||
|
||||
# Mapping of provider -> model id, e.g., {"openai": "gpt-4o", "bedrock": "anthropic.claude-v2"}
|
||||
default_models = models.JSONField(default=dict, blank=True)
|
||||
|
||||
def __str__(self):
|
||||
return f"Lighthouse Tenant Config for {self.tenant_id}"
|
||||
|
||||
def clean(self):
|
||||
super().clean()
|
||||
|
||||
class Meta(RowLevelSecurityProtectedModel.Meta):
|
||||
db_table = "lighthouse_tenant_config"
|
||||
|
||||
constraints = [
|
||||
RowLevelSecurityConstraint(
|
||||
field="tenant_id",
|
||||
name="rls_on_%(class)s",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
models.UniqueConstraint(
|
||||
fields=["tenant_id"], name="unique_tenant_lighthouse_config"
|
||||
),
|
||||
]
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "lighthouse-configurations"
|
||||
|
||||
|
||||
class LighthouseProviderModels(RowLevelSecurityProtectedModel):
|
||||
"""
|
||||
Per-tenant, per-provider configuration list of available LLM models.
|
||||
RLS-protected; populated via provider API using tenant-scoped credentials.
|
||||
"""
|
||||
|
||||
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
|
||||
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
|
||||
updated_at = models.DateTimeField(auto_now=True, editable=False)
|
||||
|
||||
# Scope to a specific provider configuration within a tenant
|
||||
provider_configuration = models.ForeignKey(
|
||||
LighthouseProviderConfiguration,
|
||||
on_delete=models.CASCADE,
|
||||
related_name="available_models",
|
||||
)
|
||||
model_id = models.CharField(max_length=100)
|
||||
|
||||
# Human-friendly model name
|
||||
model_name = models.CharField(max_length=100)
|
||||
|
||||
# Model-specific default parameters (e.g., temperature, max_tokens)
|
||||
default_parameters = models.JSONField(default=dict, blank=True)
|
||||
|
||||
def __str__(self):
|
||||
return f"{self.provider_configuration.provider_type}:{self.model_id} ({self.tenant_id})"
|
||||
|
||||
class Meta(RowLevelSecurityProtectedModel.Meta):
|
||||
db_table = "lighthouse_provider_models"
|
||||
constraints = [
|
||||
RowLevelSecurityConstraint(
|
||||
field="tenant_id",
|
||||
name="rls_on_%(class)s",
|
||||
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
|
||||
),
|
||||
models.UniqueConstraint(
|
||||
fields=["tenant_id", "provider_configuration", "model_id"],
|
||||
name="unique_provider_model_per_configuration",
|
||||
),
|
||||
]
|
||||
indexes = [
|
||||
models.Index(
|
||||
fields=["tenant_id", "provider_configuration"],
|
||||
name="lh_prov_models_cfg_idx",
|
||||
),
|
||||
]
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "lighthouse-models"
|
||||
|
||||
@@ -6,7 +6,14 @@ from django.dispatch import receiver
|
||||
from django_celery_results.backends.database import DatabaseBackend
|
||||
|
||||
from api.db_utils import delete_related_daily_task
|
||||
from api.models import Membership, Provider, TenantAPIKey, User
|
||||
from api.models import (
|
||||
LighthouseProviderConfiguration,
|
||||
LighthouseTenantConfiguration,
|
||||
Membership,
|
||||
Provider,
|
||||
TenantAPIKey,
|
||||
User,
|
||||
)
|
||||
|
||||
|
||||
def create_task_result_on_publish(sender=None, headers=None, **kwargs): # noqa: F841
|
||||
@@ -56,3 +63,33 @@ def revoke_membership_api_keys(sender, instance, **kwargs): # noqa: F841
|
||||
TenantAPIKey.objects.filter(
|
||||
entity=instance.user, tenant_id=instance.tenant.id
|
||||
).update(revoked=True)
|
||||
|
||||
|
||||
@receiver(pre_delete, sender=LighthouseProviderConfiguration)
|
||||
def cleanup_lighthouse_defaults_before_delete(sender, instance, **kwargs): # noqa: F841
|
||||
"""
|
||||
Ensure tenant Lighthouse defaults do not reference a soon-to-be-deleted provider.
|
||||
|
||||
This runs for both per-instance deletes and queryset (bulk) deletes.
|
||||
"""
|
||||
try:
|
||||
tenant_cfg = LighthouseTenantConfiguration.objects.get(
|
||||
tenant_id=instance.tenant_id
|
||||
)
|
||||
except LighthouseTenantConfiguration.DoesNotExist:
|
||||
return
|
||||
|
||||
updated = False
|
||||
defaults = tenant_cfg.default_models or {}
|
||||
|
||||
if instance.provider_type in defaults:
|
||||
defaults.pop(instance.provider_type, None)
|
||||
tenant_cfg.default_models = defaults
|
||||
updated = True
|
||||
|
||||
if tenant_cfg.default_provider == instance.provider_type:
|
||||
tenant_cfg.default_provider = ""
|
||||
updated = True
|
||||
|
||||
if updated:
|
||||
tenant_cfg.save()
|
||||
|
||||
+1417
-32
File diff suppressed because it is too large
Load Diff
@@ -22,6 +22,7 @@ from prowler.providers.azure.azure_provider import AzureProvider
|
||||
from prowler.providers.gcp.gcp_provider import GcpProvider
|
||||
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
|
||||
from prowler.providers.m365.m365_provider import M365Provider
|
||||
from prowler.providers.oraclecloud.oci_provider import OciProvider
|
||||
|
||||
|
||||
class TestMergeDicts:
|
||||
@@ -108,6 +109,7 @@ class TestReturnProwlerProvider:
|
||||
(Provider.ProviderChoices.AZURE.value, AzureProvider),
|
||||
(Provider.ProviderChoices.KUBERNETES.value, KubernetesProvider),
|
||||
(Provider.ProviderChoices.M365.value, M365Provider),
|
||||
(Provider.ProviderChoices.OCI.value, OciProvider),
|
||||
],
|
||||
)
|
||||
def test_return_prowler_provider(self, provider_type, expected_provider):
|
||||
@@ -203,6 +205,10 @@ class TestGetProwlerProviderKwargs:
|
||||
Provider.ProviderChoices.GITHUB.value,
|
||||
{"organizations": ["provider_uid"]},
|
||||
),
|
||||
(
|
||||
Provider.ProviderChoices.OCI.value,
|
||||
{},
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_get_prowler_provider_kwargs(self, provider_type, expected_extra_kwargs):
|
||||
|
||||
@@ -23,6 +23,7 @@ from conftest import (
|
||||
today_after_n_days,
|
||||
)
|
||||
from django.conf import settings
|
||||
from django.db.models import Count
|
||||
from django.http import JsonResponse
|
||||
from django.test import RequestFactory
|
||||
from django.urls import reverse
|
||||
@@ -35,12 +36,16 @@ from api.db_router import MainRouter
|
||||
from api.models import (
|
||||
Integration,
|
||||
Invitation,
|
||||
LighthouseProviderConfiguration,
|
||||
LighthouseProviderModels,
|
||||
LighthouseTenantConfiguration,
|
||||
Membership,
|
||||
Processor,
|
||||
Provider,
|
||||
ProviderGroup,
|
||||
ProviderGroupMembership,
|
||||
ProviderSecret,
|
||||
Resource,
|
||||
Role,
|
||||
RoleProviderGroupRelationship,
|
||||
SAMLConfiguration,
|
||||
@@ -943,6 +948,74 @@ class TestProviderViewSet:
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert len(response.json()["data"]) == len(providers_fixture)
|
||||
|
||||
def test_providers_filter_provider_type(
|
||||
self, authenticated_client, providers_fixture
|
||||
):
|
||||
response = authenticated_client.get(
|
||||
reverse("provider-list"), {"filter[provider_type]": "aws"}
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
data = response.json()["data"]
|
||||
assert len(data) == 2
|
||||
assert all(item["attributes"]["provider"] == "aws" for item in data)
|
||||
|
||||
def test_providers_filter_provider_type_in(
|
||||
self, authenticated_client, providers_fixture
|
||||
):
|
||||
response = authenticated_client.get(
|
||||
reverse("provider-list"), {"filter[provider_type__in]": "aws,gcp"}
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
data = response.json()["data"]
|
||||
assert len(data) == 3
|
||||
assert {"aws", "gcp"} >= {item["attributes"]["provider"] for item in data}
|
||||
|
||||
def test_providers_filter_provider_type_invalid(
|
||||
self, authenticated_client, providers_fixture
|
||||
):
|
||||
response = authenticated_client.get(
|
||||
reverse("provider-list"), {"filter[provider_type]": "invalid"}
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
def test_providers_disable_pagination(
|
||||
self, authenticated_client, providers_fixture, tenants_fixture
|
||||
):
|
||||
tenant, *_ = tenants_fixture
|
||||
existing_count = Provider.objects.filter(tenant_id=tenant.id).count()
|
||||
target_total = settings.REST_FRAMEWORK["PAGE_SIZE"] + 1
|
||||
additional_needed = max(0, target_total - existing_count)
|
||||
|
||||
base_uid = 200000000000
|
||||
for index in range(additional_needed):
|
||||
Provider.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
provider=Provider.ProviderChoices.AWS,
|
||||
uid=f"{base_uid + index:012d}",
|
||||
alias=f"aws_extra_{index}",
|
||||
)
|
||||
|
||||
total_providers = Provider.objects.filter(tenant_id=tenant.id).count()
|
||||
|
||||
paginated_response = authenticated_client.get(reverse("provider-list"))
|
||||
assert paginated_response.status_code == status.HTTP_200_OK
|
||||
paginated_data = paginated_response.json()["data"]
|
||||
assert len(paginated_data) == min(
|
||||
settings.REST_FRAMEWORK["PAGE_SIZE"], total_providers
|
||||
)
|
||||
paginated_meta = paginated_response.json().get("meta", {})
|
||||
assert "pagination" in paginated_meta
|
||||
assert paginated_meta["pagination"]["count"] == total_providers
|
||||
|
||||
unpaginated_response = authenticated_client.get(
|
||||
reverse("provider-list"), {"page[disable]": "true"}
|
||||
)
|
||||
assert unpaginated_response.status_code == status.HTTP_200_OK
|
||||
unpaginated_data = unpaginated_response.json()["data"]
|
||||
assert len(unpaginated_data) == total_providers
|
||||
unpaginated_meta = unpaginated_response.json().get("meta", {})
|
||||
assert "pagination" not in unpaginated_meta
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"include_values, expected_resources",
|
||||
[
|
||||
@@ -1386,13 +1459,25 @@ class TestProviderViewSet:
|
||||
("provider", "aws", 2),
|
||||
("provider.in", "azure,gcp", 2),
|
||||
("uid", "123456789012", 1),
|
||||
("uid.icontains", "1", 5),
|
||||
(
|
||||
"uid.icontains",
|
||||
"1",
|
||||
6,
|
||||
), # Updated: includes OCI provider with "1" in UID
|
||||
("alias", "aws_testing_1", 1),
|
||||
("alias.icontains", "aws", 2),
|
||||
("inserted_at", TODAY, 6),
|
||||
("inserted_at.gte", "2024-01-01", 6),
|
||||
("inserted_at", TODAY, 7), # Updated: 7 providers now (added OCI)
|
||||
(
|
||||
"inserted_at.gte",
|
||||
"2024-01-01",
|
||||
7,
|
||||
), # Updated: 7 providers now (added OCI)
|
||||
("inserted_at.lte", "2024-01-01", 0),
|
||||
("updated_at.gte", "2024-01-01", 6),
|
||||
(
|
||||
"updated_at.gte",
|
||||
"2024-01-01",
|
||||
7,
|
||||
), # Updated: 7 providers now (added OCI)
|
||||
("updated_at.lte", "2024-01-01", 0),
|
||||
]
|
||||
),
|
||||
@@ -1895,6 +1980,43 @@ class TestProviderSecretViewSet:
|
||||
"password": "supersecret",
|
||||
},
|
||||
),
|
||||
# OCI with API key credentials (with key_content)
|
||||
(
|
||||
Provider.ProviderChoices.OCI.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"user": "ocid1.user.oc1..aaaaaaaakldibrbov4ubh25aqdeiroklxjngwka7u6w7no3glmdq3n5sxtkq",
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "-----BEGIN RSA PRIVATE KEY-----\ntest-key-content\n-----END RSA PRIVATE KEY-----",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
},
|
||||
),
|
||||
# OCI with API key credentials (with key_file)
|
||||
(
|
||||
Provider.ProviderChoices.OCI.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"user": "ocid1.user.oc1..aaaaaaaakldibrbov4ubh25aqdeiroklxjngwka7u6w7no3glmdq3n5sxtkq",
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_file": "/path/to/oci_api_key.pem",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
},
|
||||
),
|
||||
# OCI with API key credentials (with passphrase)
|
||||
(
|
||||
Provider.ProviderChoices.OCI.value,
|
||||
ProviderSecret.TypeChoices.STATIC,
|
||||
{
|
||||
"user": "ocid1.user.oc1..aaaaaaaakldibrbov4ubh25aqdeiroklxjngwka7u6w7no3glmdq3n5sxtkq",
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "-----BEGIN RSA PRIVATE KEY-----\ntest-encrypted-key\n-----END RSA PRIVATE KEY-----",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
"pass_phrase": "my-secure-passphrase",
|
||||
},
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_provider_secrets_create_valid(
|
||||
@@ -5781,8 +5903,96 @@ class TestOverviewViewSet:
|
||||
assert response.json()["data"][0]["attributes"]["findings"]["pass"] == 2
|
||||
assert response.json()["data"][0]["attributes"]["findings"]["fail"] == 1
|
||||
assert response.json()["data"][0]["attributes"]["findings"]["muted"] == 1
|
||||
# Since we rely on completed scans, there are only 2 resources now
|
||||
assert response.json()["data"][0]["attributes"]["resources"]["total"] == 2
|
||||
# Aggregated resources include all AWS providers present in the tenant
|
||||
assert response.json()["data"][0]["attributes"]["resources"]["total"] == 3
|
||||
|
||||
def test_overview_providers_aggregates_same_provider_type(
|
||||
self,
|
||||
authenticated_client,
|
||||
scan_summaries_fixture,
|
||||
resources_fixture,
|
||||
providers_fixture,
|
||||
tenants_fixture,
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
_provider1, provider2, *_ = providers_fixture
|
||||
|
||||
scan = Scan.objects.create(
|
||||
name="overview scan aws account 2",
|
||||
provider=provider2,
|
||||
trigger=Scan.TriggerChoices.MANUAL,
|
||||
state=StateChoices.COMPLETED,
|
||||
tenant=tenant,
|
||||
)
|
||||
|
||||
ScanSummary.objects.create(
|
||||
tenant=tenant,
|
||||
scan=scan,
|
||||
check_id="check-aws-two",
|
||||
service="service-extra",
|
||||
severity="medium",
|
||||
region="region-extra",
|
||||
_pass=3,
|
||||
fail=2,
|
||||
muted=1,
|
||||
total=6,
|
||||
)
|
||||
|
||||
Resource.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
provider=provider2,
|
||||
uid="arn:aws:ec2:us-west-2:123456789013:instance/i-aggregation",
|
||||
name="Aggregated Instance",
|
||||
region="us-west-2",
|
||||
service="ec2",
|
||||
type="prowler-test",
|
||||
)
|
||||
|
||||
response = authenticated_client.get(reverse("overview-providers"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
data = response.json()["data"]
|
||||
assert len(data) == 1
|
||||
attributes = data[0]["attributes"]
|
||||
|
||||
assert attributes["findings"]["total"] == 10
|
||||
assert attributes["findings"]["pass"] == 5
|
||||
assert attributes["findings"]["fail"] == 3
|
||||
assert attributes["findings"]["muted"] == 2
|
||||
assert attributes["resources"]["total"] == 4
|
||||
|
||||
def test_overview_providers_count(
|
||||
self,
|
||||
authenticated_client,
|
||||
scan_summaries_fixture,
|
||||
resources_fixture,
|
||||
providers_fixture,
|
||||
tenants_fixture,
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
default_response = authenticated_client.get(reverse("overview-providers"))
|
||||
assert default_response.status_code == status.HTTP_200_OK
|
||||
default_data = default_response.json()["data"]
|
||||
assert len(default_data) == 1
|
||||
assert all("count" not in item["attributes"] for item in default_data)
|
||||
grouped_response = authenticated_client.get(reverse("overview-providers-count"))
|
||||
assert grouped_response.status_code == status.HTTP_200_OK
|
||||
grouped_data = grouped_response.json()["data"]
|
||||
assert len(grouped_data) >= 1
|
||||
|
||||
aggregated = {
|
||||
entry["id"]: entry["attributes"]["count"] for entry in grouped_data
|
||||
}
|
||||
db_counts = (
|
||||
Provider.objects.filter(tenant_id=tenant.id, is_deleted=False)
|
||||
.values("provider")
|
||||
.annotate(count=Count("id"))
|
||||
)
|
||||
expected = {row["provider"]: row["count"] for row in db_counts}
|
||||
|
||||
assert aggregated == expected
|
||||
for entry in grouped_data:
|
||||
assert "findings" not in entry["attributes"]
|
||||
|
||||
def test_overview_services_list_no_required_filters(
|
||||
self, authenticated_client, scan_summaries_fixture
|
||||
@@ -8703,3 +8913,483 @@ class TestTenantApiKeyViewSet:
|
||||
# Verify error object structure
|
||||
error = response_data["errors"][0]
|
||||
assert "detail" in error or "title" in error
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestLighthouseTenantConfigViewSet:
|
||||
"""Test Lighthouse tenant configuration endpoint (singleton pattern)"""
|
||||
|
||||
def test_lighthouse_tenant_config_create_via_patch(self, authenticated_client):
|
||||
"""Test creating a tenant config successfully via PATCH (upsert)"""
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-configurations",
|
||||
"attributes": {
|
||||
"business_context": "Test business context for security analysis",
|
||||
"default_provider": "",
|
||||
"default_models": {},
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
data = response.json()["data"]
|
||||
assert (
|
||||
data["attributes"]["business_context"]
|
||||
== "Test business context for security analysis"
|
||||
)
|
||||
assert data["attributes"]["default_provider"] == ""
|
||||
assert data["attributes"]["default_models"] == {}
|
||||
|
||||
def test_lighthouse_tenant_config_upsert_behavior(self, authenticated_client):
|
||||
"""Test that PATCH creates config if not exists and updates if exists (upsert)"""
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-configurations",
|
||||
"attributes": {
|
||||
"business_context": "First config",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
# First PATCH creates the config
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
first_data = response.json()["data"]
|
||||
assert first_data["attributes"]["business_context"] == "First config"
|
||||
|
||||
# Second PATCH updates the same config (not creating a duplicate)
|
||||
payload["data"]["attributes"]["business_context"] = "Updated config"
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
second_data = response.json()["data"]
|
||||
assert second_data["attributes"]["business_context"] == "Updated config"
|
||||
# Verify it's the same config (same ID)
|
||||
assert first_data["id"] == second_data["id"]
|
||||
|
||||
@patch("openai.OpenAI")
|
||||
def test_lighthouse_tenant_config_retrieve(
|
||||
self, mock_openai_client, authenticated_client, tenants_fixture
|
||||
):
|
||||
"""Test retrieving the singleton tenant config with proper provider and model validation"""
|
||||
|
||||
# Mock OpenAI client and models response
|
||||
mock_models_response = Mock()
|
||||
mock_models_response.data = [
|
||||
Mock(id="gpt-4o"),
|
||||
Mock(id="gpt-4o-mini"),
|
||||
Mock(id="gpt-5"),
|
||||
]
|
||||
mock_openai_client.return_value.models.list.return_value = mock_models_response
|
||||
|
||||
# Create OpenAI provider configuration
|
||||
provider_config = LighthouseProviderConfiguration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
provider_type="openai",
|
||||
credentials=b'{"api_key": "sk-test1234567890T3BlbkFJtest1234567890"}',
|
||||
is_active=True,
|
||||
)
|
||||
|
||||
# Create provider models (simulating refresh)
|
||||
LighthouseProviderModels.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
provider_configuration=provider_config,
|
||||
model_id="gpt-4o",
|
||||
default_parameters={},
|
||||
)
|
||||
LighthouseProviderModels.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
provider_configuration=provider_config,
|
||||
model_id="gpt-4o-mini",
|
||||
default_parameters={},
|
||||
)
|
||||
|
||||
# Create tenant configuration with valid provider and model
|
||||
config = LighthouseTenantConfiguration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
business_context="Test context",
|
||||
default_provider="openai",
|
||||
default_models={"openai": "gpt-4o"},
|
||||
)
|
||||
|
||||
# Retrieve and verify the configuration
|
||||
response = authenticated_client.get(reverse("lighthouse-configurations"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
data = response.json()["data"]
|
||||
assert data["id"] == str(config.id)
|
||||
assert data["attributes"]["business_context"] == "Test context"
|
||||
assert data["attributes"]["default_provider"] == "openai"
|
||||
assert data["attributes"]["default_models"] == {"openai": "gpt-4o"}
|
||||
|
||||
def test_lighthouse_tenant_config_retrieve_not_found(self, authenticated_client):
|
||||
"""Test GET when config doesn't exist returns 404"""
|
||||
response = authenticated_client.get(reverse("lighthouse-configurations"))
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
assert "not found" in response.json()["errors"][0]["detail"].lower()
|
||||
|
||||
def test_lighthouse_tenant_config_partial_update(
|
||||
self, authenticated_client, tenants_fixture
|
||||
):
|
||||
"""Test updating tenant config fields"""
|
||||
from api.models import LighthouseTenantConfiguration
|
||||
|
||||
# Create config first
|
||||
config = LighthouseTenantConfiguration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
business_context="Original context",
|
||||
default_provider="",
|
||||
default_models={},
|
||||
)
|
||||
|
||||
# Update it
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-configurations",
|
||||
"attributes": {
|
||||
"business_context": "Updated context for cloud security",
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
|
||||
# Verify update
|
||||
config.refresh_from_db()
|
||||
assert config.business_context == "Updated context for cloud security"
|
||||
|
||||
def test_lighthouse_tenant_config_update_invalid_provider(
|
||||
self, authenticated_client, tenants_fixture
|
||||
):
|
||||
"""Test validation fails when default_provider is not configured and active"""
|
||||
from api.models import LighthouseTenantConfiguration
|
||||
|
||||
# Create config first
|
||||
LighthouseTenantConfiguration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
business_context="Test",
|
||||
)
|
||||
|
||||
# Try to set invalid provider
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-configurations",
|
||||
"attributes": {
|
||||
"default_provider": "nonexistent-provider",
|
||||
},
|
||||
}
|
||||
}
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "provider" in response.json()["errors"][0]["detail"].lower()
|
||||
|
||||
def test_lighthouse_tenant_config_update_invalid_json_format(
|
||||
self, authenticated_client, tenants_fixture
|
||||
):
|
||||
"""Test that invalid JSON payload is rejected"""
|
||||
from api.models import LighthouseTenantConfiguration
|
||||
|
||||
# Create config first
|
||||
LighthouseTenantConfiguration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
business_context="Test",
|
||||
)
|
||||
|
||||
# Send invalid JSON
|
||||
response = authenticated_client.patch(
|
||||
reverse("lighthouse-configurations"),
|
||||
data="invalid json",
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestLighthouseProviderConfigViewSet:
|
||||
"""Tests for LighthouseProviderConfiguration create validations"""
|
||||
|
||||
def test_invalid_provider_type(self, authenticated_client):
|
||||
"""Add invalid provider (testprovider) should error"""
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "testprovider",
|
||||
"credentials": {"api_key": "sk-testT3BlbkFJkey"},
|
||||
},
|
||||
}
|
||||
}
|
||||
resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
def test_openai_missing_credentials(self, authenticated_client):
|
||||
"""OpenAI provider without credentials should error"""
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
},
|
||||
}
|
||||
}
|
||||
resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"credentials",
|
||||
[
|
||||
{}, # empty credentials
|
||||
{"token": "sk-testT3BlbkFJkey"}, # wrong key name
|
||||
{"api_key": "ks-invalid-format"}, # wrong format
|
||||
],
|
||||
)
|
||||
def test_openai_invalid_credentials(self, authenticated_client, credentials):
|
||||
"""OpenAI provider with invalid credentials should error"""
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": credentials,
|
||||
},
|
||||
}
|
||||
}
|
||||
resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
def test_openai_valid_credentials_success(self, authenticated_client):
|
||||
"""OpenAI provider with valid sk-xxx format should succeed"""
|
||||
valid_key = "sk-abc123T3BlbkFJxyz456"
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": {"api_key": valid_key},
|
||||
},
|
||||
}
|
||||
}
|
||||
resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp.status_code == status.HTTP_201_CREATED
|
||||
data = resp.json()["data"]
|
||||
|
||||
masked_creds = data["attributes"].get("credentials")
|
||||
assert masked_creds is not None
|
||||
assert "api_key" in masked_creds
|
||||
assert masked_creds["api_key"] == ("*" * len(valid_key))
|
||||
|
||||
def test_openai_provider_duplicate_per_tenant(self, authenticated_client):
|
||||
"""If an OpenAI provider exists for tenant, creating again should error"""
|
||||
valid_key = "sk-dup123T3BlbkFJdup456"
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": {"api_key": valid_key},
|
||||
},
|
||||
}
|
||||
}
|
||||
# First creation succeeds
|
||||
resp1 = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp1.status_code == status.HTTP_201_CREATED
|
||||
|
||||
# Second creation should fail with validation error
|
||||
resp2 = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert resp2.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert "already exists" in str(resp2.json()).lower()
|
||||
|
||||
def test_openai_patch_base_url_and_is_active(self, authenticated_client):
|
||||
"""After creating, should be able to patch base_url and is_active"""
|
||||
valid_key = "sk-patch123T3BlbkFJpatch456"
|
||||
create_payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": {"api_key": valid_key},
|
||||
},
|
||||
}
|
||||
}
|
||||
create_resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=create_payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert create_resp.status_code == status.HTTP_201_CREATED
|
||||
provider_id = create_resp.json()["data"]["id"]
|
||||
|
||||
patch_payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"id": provider_id,
|
||||
"attributes": {
|
||||
"base_url": "https://api.example.com/v1",
|
||||
"is_active": False,
|
||||
},
|
||||
}
|
||||
}
|
||||
patch_resp = authenticated_client.patch(
|
||||
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
|
||||
data=patch_payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert patch_resp.status_code == status.HTTP_200_OK
|
||||
updated = patch_resp.json()["data"]["attributes"]
|
||||
assert updated["base_url"] == "https://api.example.com/v1"
|
||||
assert updated["is_active"] is False
|
||||
|
||||
def test_openai_patch_invalid_credentials(self, authenticated_client):
|
||||
"""PATCH with invalid credentials.api_key should error (400)"""
|
||||
valid_key = "sk-ok123T3BlbkFJok456"
|
||||
create_payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": {"api_key": valid_key},
|
||||
},
|
||||
}
|
||||
}
|
||||
create_resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=create_payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert create_resp.status_code == status.HTTP_201_CREATED
|
||||
provider_id = create_resp.json()["data"]["id"]
|
||||
|
||||
# Try patch with invalid api_key format
|
||||
patch_payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"id": provider_id,
|
||||
"attributes": {
|
||||
"credentials": {"api_key": "ks-invalid-format"},
|
||||
},
|
||||
}
|
||||
}
|
||||
patch_resp = authenticated_client.patch(
|
||||
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id}),
|
||||
data=patch_payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert patch_resp.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
def test_openai_get_masking_and_fields_filter(self, authenticated_client):
|
||||
valid_key = "sk-get123T3BlbkFJget456"
|
||||
create_payload = {
|
||||
"data": {
|
||||
"type": "lighthouse-providers",
|
||||
"attributes": {
|
||||
"provider_type": "openai",
|
||||
"credentials": {"api_key": valid_key},
|
||||
},
|
||||
}
|
||||
}
|
||||
create_resp = authenticated_client.post(
|
||||
reverse("lighthouse-providers-list"),
|
||||
data=create_payload,
|
||||
content_type=API_JSON_CONTENT_TYPE,
|
||||
)
|
||||
assert create_resp.status_code == status.HTTP_201_CREATED
|
||||
provider_id = create_resp.json()["data"]["id"]
|
||||
|
||||
# Default GET should return masked credentials
|
||||
get_resp = authenticated_client.get(
|
||||
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
|
||||
)
|
||||
assert get_resp.status_code == status.HTTP_200_OK
|
||||
masked = get_resp.json()["data"]["attributes"]["credentials"]["api_key"]
|
||||
assert masked == ("*" * len(valid_key))
|
||||
|
||||
# Fields filter should return decrypted credentials structure
|
||||
get_full = authenticated_client.get(
|
||||
reverse("lighthouse-providers-detail", kwargs={"pk": provider_id})
|
||||
+ "?fields[lighthouse-providers]=credentials"
|
||||
)
|
||||
assert get_full.status_code == status.HTTP_200_OK
|
||||
creds = get_full.json()["data"]["attributes"]["credentials"]
|
||||
assert creds["api_key"] == valid_key
|
||||
|
||||
def test_delete_provider_updates_tenant_defaults(
|
||||
self, authenticated_client, tenants_fixture
|
||||
):
|
||||
"""Deleting a provider config should clear tenant default_provider and its default_model entry."""
|
||||
|
||||
tenant = tenants_fixture[0]
|
||||
|
||||
# Create provider configuration to delete
|
||||
provider = LighthouseProviderConfiguration.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
provider_type="openai",
|
||||
credentials=b'{"api_key":"sk-test123T3BlbkFJ"}',
|
||||
is_active=True,
|
||||
)
|
||||
|
||||
# Seed tenant defaults referencing the provider we will delete
|
||||
cfg = LighthouseTenantConfiguration.objects.create(
|
||||
tenant_id=tenant.id,
|
||||
business_context="Test",
|
||||
default_provider="openai",
|
||||
default_models={"openai": "gpt-4o", "other": "model-x"},
|
||||
)
|
||||
|
||||
# Delete via API and validate response
|
||||
url = reverse("lighthouse-providers-detail", kwargs={"pk": str(provider.id)})
|
||||
resp = authenticated_client.delete(url)
|
||||
assert resp.status_code in (
|
||||
status.HTTP_204_NO_CONTENT,
|
||||
status.HTTP_200_OK,
|
||||
)
|
||||
|
||||
# Tenant defaults should be updated
|
||||
cfg.refresh_from_db()
|
||||
assert cfg.default_provider == ""
|
||||
assert "openai" not in cfg.default_models
|
||||
|
||||
# Unrelated entries should remain untouched
|
||||
assert cfg.default_models.get("other") == "model-x"
|
||||
|
||||
@@ -20,6 +20,7 @@ from prowler.providers.gcp.gcp_provider import GcpProvider
|
||||
from prowler.providers.github.github_provider import GithubProvider
|
||||
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
|
||||
from prowler.providers.m365.m365_provider import M365Provider
|
||||
from prowler.providers.oraclecloud.oci_provider import OciProvider
|
||||
|
||||
|
||||
class CustomOAuth2Client(OAuth2Client):
|
||||
@@ -67,6 +68,7 @@ def return_prowler_provider(
|
||||
| GithubProvider
|
||||
| KubernetesProvider
|
||||
| M365Provider
|
||||
| OciProvider
|
||||
]:
|
||||
"""Return the Prowler provider class based on the given provider type.
|
||||
|
||||
@@ -74,7 +76,7 @@ def return_prowler_provider(
|
||||
provider (Provider): The provider object containing the provider type and associated secrets.
|
||||
|
||||
Returns:
|
||||
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider: The corresponding provider class.
|
||||
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider | OciProvider: The corresponding provider class.
|
||||
|
||||
Raises:
|
||||
ValueError: If the provider type specified in `provider.provider` is not supported.
|
||||
@@ -92,6 +94,8 @@ def return_prowler_provider(
|
||||
prowler_provider = M365Provider
|
||||
case Provider.ProviderChoices.GITHUB.value:
|
||||
prowler_provider = GithubProvider
|
||||
case Provider.ProviderChoices.OCI.value:
|
||||
prowler_provider = OciProvider
|
||||
case _:
|
||||
raise ValueError(f"Provider type {provider.provider} not supported")
|
||||
return prowler_provider
|
||||
@@ -147,6 +151,7 @@ def initialize_prowler_provider(
|
||||
| GithubProvider
|
||||
| KubernetesProvider
|
||||
| M365Provider
|
||||
| OciProvider
|
||||
):
|
||||
"""Initialize a Prowler provider instance based on the given provider type.
|
||||
|
||||
@@ -155,8 +160,8 @@ def initialize_prowler_provider(
|
||||
mutelist_processor (Processor): The mutelist processor object containing the mutelist configuration.
|
||||
|
||||
Returns:
|
||||
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider: An instance of the corresponding provider class
|
||||
(`AwsProvider`, `AzureProvider`, `GcpProvider`, `GithubProvider`, `KubernetesProvider` or `M365Provider`) initialized with the
|
||||
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider | OciProvider: An instance of the corresponding provider class
|
||||
(`AwsProvider`, `AzureProvider`, `GcpProvider`, `GithubProvider`, `KubernetesProvider`, `M365Provider` or `OciProvider`) initialized with the
|
||||
provider's secrets.
|
||||
"""
|
||||
prowler_provider = return_prowler_provider(provider)
|
||||
|
||||
@@ -12,6 +12,24 @@ from api.models import StateChoices, Task
|
||||
from api.v1.serializers import TaskSerializer
|
||||
|
||||
|
||||
class DisablePaginationMixin:
|
||||
disable_pagination_query_param = "page[disable]"
|
||||
disable_pagination_truthy_values = {"true"}
|
||||
|
||||
def should_disable_pagination(self) -> bool:
|
||||
if not hasattr(self, "request"):
|
||||
return False
|
||||
value = self.request.query_params.get(self.disable_pagination_query_param)
|
||||
if value is None:
|
||||
return False
|
||||
return str(value).lower() in self.disable_pagination_truthy_values
|
||||
|
||||
def paginate_queryset(self, queryset):
|
||||
if self.should_disable_pagination():
|
||||
return None
|
||||
return super().paginate_queryset(queryset)
|
||||
|
||||
|
||||
class PaginateByPkMixin:
|
||||
"""
|
||||
Mixin to paginate on a list of PKs (cheaper than heavy JOINs),
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
import re
|
||||
|
||||
from rest_framework_json_api import serializers
|
||||
|
||||
|
||||
class OpenAICredentialsSerializer(serializers.Serializer):
|
||||
api_key = serializers.CharField()
|
||||
|
||||
def validate_api_key(self, value: str) -> str:
|
||||
pattern = r"^sk-[\w-]+$"
|
||||
if not re.match(pattern, value or ""):
|
||||
raise serializers.ValidationError("Invalid OpenAI API key format.")
|
||||
return value
|
||||
@@ -239,6 +239,41 @@ from rest_framework_json_api import serializers
|
||||
},
|
||||
"required": ["github_app_id", "github_app_key"],
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
"title": "Oracle Cloud Infrastructure (OCI) API Key Credentials",
|
||||
"properties": {
|
||||
"user": {
|
||||
"type": "string",
|
||||
"description": "The OCID of the user to authenticate with.",
|
||||
},
|
||||
"fingerprint": {
|
||||
"type": "string",
|
||||
"description": "The fingerprint of the API signing key.",
|
||||
},
|
||||
"key_file": {
|
||||
"type": "string",
|
||||
"description": "The path to the private key file for API signing. Either key_file or key_content must be provided.",
|
||||
},
|
||||
"key_content": {
|
||||
"type": "string",
|
||||
"description": "The content of the private key for API signing (base64 encoded). Either key_file or key_content must be provided.",
|
||||
},
|
||||
"tenancy": {
|
||||
"type": "string",
|
||||
"description": "The OCID of the tenancy.",
|
||||
},
|
||||
"region": {
|
||||
"type": "string",
|
||||
"description": "The OCI region identifier (e.g., us-ashburn-1, us-phoenix-1).",
|
||||
},
|
||||
"pass_phrase": {
|
||||
"type": "string",
|
||||
"description": "The passphrase for the private key, if encrypted.",
|
||||
},
|
||||
},
|
||||
"required": ["user", "fingerprint", "tenancy", "region"],
|
||||
},
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
@@ -6,8 +6,10 @@ from django.conf import settings
|
||||
from django.contrib.auth import authenticate
|
||||
from django.contrib.auth.models import update_last_login
|
||||
from django.contrib.auth.password_validation import validate_password
|
||||
from django.db import IntegrityError
|
||||
from drf_spectacular.utils import extend_schema_field
|
||||
from jwt.exceptions import InvalidKeyError
|
||||
from rest_framework.reverse import reverse
|
||||
from rest_framework.validators import UniqueTogetherValidator
|
||||
from rest_framework_json_api import serializers
|
||||
from rest_framework_json_api.relations import SerializerMethodResourceRelatedField
|
||||
@@ -25,6 +27,9 @@ from api.models import (
|
||||
Invitation,
|
||||
InvitationRoleRelationship,
|
||||
LighthouseConfiguration,
|
||||
LighthouseProviderConfiguration,
|
||||
LighthouseProviderModels,
|
||||
LighthouseTenantConfiguration,
|
||||
Membership,
|
||||
Processor,
|
||||
Provider,
|
||||
@@ -54,6 +59,7 @@ from api.v1.serializer_utils.integrations import (
|
||||
S3ConfigSerializer,
|
||||
SecurityHubConfigSerializer,
|
||||
)
|
||||
from api.v1.serializer_utils.lighthouse import OpenAICredentialsSerializer
|
||||
from api.v1.serializer_utils.processors import ProcessorConfigField
|
||||
from api.v1.serializer_utils.providers import ProviderSecretField
|
||||
from prowler.lib.mutelist.mutelist import Mutelist
|
||||
@@ -1353,6 +1359,8 @@ class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
|
||||
serializer = KubernetesProviderSecret(data=secret)
|
||||
elif provider_type == Provider.ProviderChoices.M365.value:
|
||||
serializer = M365ProviderSecret(data=secret)
|
||||
elif provider_type == Provider.ProviderChoices.OCI.value:
|
||||
serializer = OracleCloudProviderSecret(data=secret)
|
||||
else:
|
||||
raise serializers.ValidationError(
|
||||
{"provider": f"Provider type not supported {provider_type}"}
|
||||
@@ -1466,6 +1474,19 @@ class GithubProviderSecret(serializers.Serializer):
|
||||
resource_name = "provider-secrets"
|
||||
|
||||
|
||||
class OracleCloudProviderSecret(serializers.Serializer):
|
||||
user = serializers.CharField()
|
||||
fingerprint = serializers.CharField()
|
||||
key_file = serializers.CharField(required=False)
|
||||
key_content = serializers.CharField(required=False)
|
||||
tenancy = serializers.CharField()
|
||||
region = serializers.CharField()
|
||||
pass_phrase = serializers.CharField(required=False)
|
||||
|
||||
class Meta:
|
||||
resource_name = "provider-secrets"
|
||||
|
||||
|
||||
class AWSRoleAssumptionProviderSecret(serializers.Serializer):
|
||||
role_arn = serializers.CharField()
|
||||
external_id = serializers.CharField()
|
||||
@@ -2099,6 +2120,17 @@ class OverviewProviderSerializer(serializers.Serializer):
|
||||
}
|
||||
|
||||
|
||||
class OverviewProviderCountSerializer(serializers.Serializer):
|
||||
id = serializers.CharField(source="provider")
|
||||
count = serializers.IntegerField()
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "providers-count-overview"
|
||||
|
||||
def get_root_meta(self, _resource, _many):
|
||||
return {"version": "v1"}
|
||||
|
||||
|
||||
class OverviewFindingSerializer(serializers.Serializer):
|
||||
id = serializers.CharField(default="n/a")
|
||||
new = serializers.IntegerField()
|
||||
@@ -2750,6 +2782,16 @@ class LighthouseConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"updated_at": {"read_only": True},
|
||||
}
|
||||
|
||||
def validate_temperature(self, value):
|
||||
if not 0 <= value <= 1:
|
||||
raise ValidationError("Temperature must be between 0 and 1.")
|
||||
return value
|
||||
|
||||
def validate_max_tokens(self, value):
|
||||
if not 500 <= value <= 5000:
|
||||
raise ValidationError("Max tokens must be between 500 and 5000.")
|
||||
return value
|
||||
|
||||
def validate(self, attrs):
|
||||
tenant_id = self.context.get("request").tenant_id
|
||||
if LighthouseConfiguration.objects.filter(tenant_id=tenant_id).exists():
|
||||
@@ -2758,6 +2800,11 @@ class LighthouseConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"tenant_id": "Lighthouse configuration already exists for this tenant."
|
||||
}
|
||||
)
|
||||
api_key = attrs.get("api_key")
|
||||
if api_key is not None:
|
||||
OpenAICredentialsSerializer(data={"api_key": api_key}).is_valid(
|
||||
raise_exception=True
|
||||
)
|
||||
return super().validate(attrs)
|
||||
|
||||
def create(self, validated_data):
|
||||
@@ -2802,6 +2849,24 @@ class LighthouseConfigUpdateSerializer(BaseWriteSerializer):
|
||||
"max_tokens": {"required": False},
|
||||
}
|
||||
|
||||
def validate_temperature(self, value):
|
||||
if not 0 <= value <= 1:
|
||||
raise ValidationError("Temperature must be between 0 and 1.")
|
||||
return value
|
||||
|
||||
def validate_max_tokens(self, value):
|
||||
if not 500 <= value <= 5000:
|
||||
raise ValidationError("Max tokens must be between 500 and 5000.")
|
||||
return value
|
||||
|
||||
def validate(self, attrs):
|
||||
api_key = attrs.get("api_key", None)
|
||||
if api_key is not None:
|
||||
OpenAICredentialsSerializer(data={"api_key": api_key}).is_valid(
|
||||
raise_exception=True
|
||||
)
|
||||
return super().validate(attrs)
|
||||
|
||||
def update(self, instance, validated_data):
|
||||
api_key = validated_data.pop("api_key", None)
|
||||
instance = super().update(instance, validated_data)
|
||||
@@ -2931,3 +2996,352 @@ class TenantApiKeyUpdateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
):
|
||||
raise ValidationError("An API key with this name already exists.")
|
||||
return value
|
||||
|
||||
|
||||
# Lighthouse: Provider configurations
|
||||
|
||||
|
||||
class LighthouseProviderConfigSerializer(RLSSerializer):
|
||||
"""
|
||||
Read serializer for LighthouseProviderConfiguration.
|
||||
"""
|
||||
|
||||
# Decrypted credentials are only returned in to_representation when requested
|
||||
credentials = serializers.JSONField(required=False, read_only=True)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderConfiguration
|
||||
fields = [
|
||||
"id",
|
||||
"inserted_at",
|
||||
"updated_at",
|
||||
"provider_type",
|
||||
"base_url",
|
||||
"is_active",
|
||||
"credentials",
|
||||
"url",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
"inserted_at": {"read_only": True},
|
||||
"updated_at": {"read_only": True},
|
||||
"is_active": {"read_only": True},
|
||||
"url": {"read_only": True, "view_name": "lighthouse-providers-detail"},
|
||||
}
|
||||
|
||||
class JSONAPIMeta:
|
||||
resource_name = "lighthouse-providers"
|
||||
|
||||
def to_representation(self, instance):
|
||||
data = super().to_representation(instance)
|
||||
# Support JSON:API fields filter: fields[lighthouse-providers]=credentials,base_url
|
||||
fields_param = self.context.get("request", None) and self.context[
|
||||
"request"
|
||||
].query_params.get("fields[lighthouse-providers]", "")
|
||||
|
||||
creds = instance.credentials_decoded
|
||||
|
||||
requested_fields = (
|
||||
[f.strip() for f in fields_param.split(",")] if fields_param else []
|
||||
)
|
||||
|
||||
if "credentials" in requested_fields:
|
||||
# Return full decrypted credentials JSON
|
||||
data["credentials"] = creds
|
||||
else:
|
||||
# Return masked credentials by default
|
||||
def mask_value(value):
|
||||
if isinstance(value, str):
|
||||
return "*" * len(value)
|
||||
if isinstance(value, dict):
|
||||
return {k: mask_value(v) for k, v in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [mask_value(v) for v in value]
|
||||
return value
|
||||
|
||||
# Always return masked credentials, even if creds is None
|
||||
if creds is not None:
|
||||
data["credentials"] = mask_value(creds)
|
||||
else:
|
||||
# If credentials_decoded returns None, return None for credentials field
|
||||
data["credentials"] = None
|
||||
|
||||
return data
|
||||
|
||||
|
||||
class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
"""
|
||||
Create serializer for LighthouseProviderConfiguration.
|
||||
Accepts credentials as JSON; stored encrypted via credentials_decoded.
|
||||
"""
|
||||
|
||||
credentials = serializers.JSONField(write_only=True, required=True)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderConfiguration
|
||||
fields = [
|
||||
"provider_type",
|
||||
"base_url",
|
||||
"credentials",
|
||||
"is_active",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"is_active": {"required": False},
|
||||
"base_url": {"required": False, "allow_null": True},
|
||||
}
|
||||
|
||||
def create(self, validated_data):
|
||||
credentials = validated_data.pop("credentials")
|
||||
|
||||
instance = LighthouseProviderConfiguration(**validated_data)
|
||||
instance.tenant_id = self.context.get("tenant_id")
|
||||
instance.credentials_decoded = credentials
|
||||
|
||||
try:
|
||||
instance.save()
|
||||
return instance
|
||||
except IntegrityError:
|
||||
raise ValidationError(
|
||||
{
|
||||
"provider_type": "Configuration for this provider already exists for the tenant."
|
||||
}
|
||||
)
|
||||
|
||||
def validate(self, attrs):
|
||||
provider_type = attrs.get("provider_type")
|
||||
credentials = attrs.get("credentials") or {}
|
||||
|
||||
if provider_type == LighthouseProviderConfiguration.LLMProviderChoices.OPENAI:
|
||||
try:
|
||||
OpenAICredentialsSerializer(data=credentials).is_valid(
|
||||
raise_exception=True
|
||||
)
|
||||
except ValidationError as e:
|
||||
details = e.detail.copy()
|
||||
for key, value in details.items():
|
||||
e.detail[f"credentials/{key}"] = value
|
||||
del e.detail[key]
|
||||
raise e
|
||||
|
||||
return super().validate(attrs)
|
||||
|
||||
|
||||
class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
|
||||
"""
|
||||
Update serializer for LighthouseProviderConfiguration.
|
||||
"""
|
||||
|
||||
credentials = serializers.JSONField(write_only=True, required=False)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderConfiguration
|
||||
fields = [
|
||||
"id",
|
||||
"provider_type",
|
||||
"base_url",
|
||||
"credentials",
|
||||
"is_active",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
"provider_type": {"read_only": True},
|
||||
"base_url": {"required": False, "allow_null": True},
|
||||
"is_active": {"required": False},
|
||||
}
|
||||
|
||||
def update(self, instance, validated_data):
|
||||
credentials = validated_data.pop("credentials", None)
|
||||
|
||||
for attr, value in validated_data.items():
|
||||
setattr(instance, attr, value)
|
||||
|
||||
if credentials is not None:
|
||||
instance.credentials_decoded = credentials
|
||||
|
||||
instance.save()
|
||||
return instance
|
||||
|
||||
def validate(self, attrs):
|
||||
provider_type = getattr(self.instance, "provider_type", None)
|
||||
credentials = attrs.get("credentials", None)
|
||||
|
||||
if (
|
||||
credentials is not None
|
||||
and provider_type
|
||||
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
|
||||
):
|
||||
try:
|
||||
OpenAICredentialsSerializer(data=credentials).is_valid(
|
||||
raise_exception=True
|
||||
)
|
||||
except ValidationError as e:
|
||||
details = e.detail.copy()
|
||||
for key, value in details.items():
|
||||
e.detail[f"credentials/{key}"] = value
|
||||
del e.detail[key]
|
||||
raise e
|
||||
|
||||
return super().validate(attrs)
|
||||
|
||||
|
||||
# Lighthouse: Tenant configuration
|
||||
|
||||
|
||||
class LighthouseTenantConfigSerializer(RLSSerializer):
|
||||
"""
|
||||
Read serializer for LighthouseTenantConfiguration.
|
||||
"""
|
||||
|
||||
# Build singleton URL without pk
|
||||
url = serializers.SerializerMethodField()
|
||||
|
||||
def get_url(self, obj):
|
||||
request = self.context.get("request")
|
||||
return reverse("lighthouse-configurations", request=request)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseTenantConfiguration
|
||||
fields = [
|
||||
"id",
|
||||
"inserted_at",
|
||||
"updated_at",
|
||||
"business_context",
|
||||
"default_provider",
|
||||
"default_models",
|
||||
"url",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
"inserted_at": {"read_only": True},
|
||||
"updated_at": {"read_only": True},
|
||||
"url": {"read_only": True},
|
||||
}
|
||||
|
||||
|
||||
class LighthouseTenantConfigUpdateSerializer(BaseWriteSerializer):
|
||||
class Meta:
|
||||
model = LighthouseTenantConfiguration
|
||||
fields = [
|
||||
"id",
|
||||
"business_context",
|
||||
"default_provider",
|
||||
"default_models",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
}
|
||||
|
||||
def validate(self, attrs):
|
||||
request = self.context.get("request")
|
||||
tenant_id = self.context.get("tenant_id") or (
|
||||
getattr(request, "tenant_id", None) if request else None
|
||||
)
|
||||
|
||||
default_provider = attrs.get(
|
||||
"default_provider", getattr(self.instance, "default_provider", "")
|
||||
)
|
||||
default_models = attrs.get(
|
||||
"default_models", getattr(self.instance, "default_models", {})
|
||||
)
|
||||
|
||||
if default_provider:
|
||||
supported = set(LighthouseProviderConfiguration.LLMProviderChoices.values)
|
||||
if default_provider not in supported:
|
||||
raise ValidationError(
|
||||
{"default_provider": f"Unsupported provider '{default_provider}'."}
|
||||
)
|
||||
if not LighthouseProviderConfiguration.objects.filter(
|
||||
tenant_id=tenant_id, provider_type=default_provider, is_active=True
|
||||
).exists():
|
||||
raise ValidationError(
|
||||
{
|
||||
"default_provider": f"No active configuration found for '{default_provider}'."
|
||||
}
|
||||
)
|
||||
|
||||
if default_models is not None and not isinstance(default_models, dict):
|
||||
raise ValidationError(
|
||||
{"default_models": "Must be an object mapping provider -> model_id."}
|
||||
)
|
||||
|
||||
for provider_type, model_id in (default_models or {}).items():
|
||||
provider_cfg = LighthouseProviderConfiguration.objects.filter(
|
||||
tenant_id=tenant_id, provider_type=provider_type, is_active=True
|
||||
).first()
|
||||
if not provider_cfg:
|
||||
raise ValidationError(
|
||||
{
|
||||
"default_models": f"No active configuration for provider '{provider_type}'."
|
||||
}
|
||||
)
|
||||
if not LighthouseProviderModels.objects.filter(
|
||||
tenant_id=tenant_id,
|
||||
provider_configuration=provider_cfg,
|
||||
model_id=model_id,
|
||||
).exists():
|
||||
raise ValidationError(
|
||||
{
|
||||
"default_models": f"Invalid model '{model_id}' for provider '{provider_type}'."
|
||||
}
|
||||
)
|
||||
|
||||
return super().validate(attrs)
|
||||
|
||||
|
||||
# Lighthouse: Provider models
|
||||
|
||||
|
||||
class LighthouseProviderModelsSerializer(RLSSerializer):
|
||||
"""
|
||||
Read serializer for LighthouseProviderModels.
|
||||
"""
|
||||
|
||||
provider_configuration = serializers.ResourceRelatedField(read_only=True)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderModels
|
||||
fields = [
|
||||
"id",
|
||||
"inserted_at",
|
||||
"updated_at",
|
||||
"provider_configuration",
|
||||
"model_id",
|
||||
"model_name",
|
||||
"default_parameters",
|
||||
"url",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
"inserted_at": {"read_only": True},
|
||||
"updated_at": {"read_only": True},
|
||||
"url": {"read_only": True, "view_name": "lighthouse-models-detail"},
|
||||
}
|
||||
|
||||
|
||||
class LighthouseProviderModelsCreateSerializer(RLSSerializer, BaseWriteSerializer):
|
||||
provider_configuration = serializers.ResourceRelatedField(
|
||||
queryset=LighthouseProviderConfiguration.objects.all()
|
||||
)
|
||||
|
||||
class Meta:
|
||||
model = LighthouseProviderModels
|
||||
fields = [
|
||||
"provider_configuration",
|
||||
"model_id",
|
||||
"default_parameters",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"default_parameters": {"required": False},
|
||||
}
|
||||
|
||||
|
||||
class LighthouseProviderModelsUpdateSerializer(BaseWriteSerializer):
|
||||
class Meta:
|
||||
model = LighthouseProviderModels
|
||||
fields = [
|
||||
"id",
|
||||
"default_parameters",
|
||||
]
|
||||
extra_kwargs = {
|
||||
"id": {"read_only": True},
|
||||
}
|
||||
|
||||
@@ -17,6 +17,9 @@ from api.v1.views import (
|
||||
InvitationAcceptViewSet,
|
||||
InvitationViewSet,
|
||||
LighthouseConfigViewSet,
|
||||
LighthouseProviderConfigViewSet,
|
||||
LighthouseProviderModelsViewSet,
|
||||
LighthouseTenantConfigViewSet,
|
||||
MembershipViewSet,
|
||||
OverviewViewSet,
|
||||
ProcessorViewSet,
|
||||
@@ -34,12 +37,12 @@ from api.v1.views import (
|
||||
ScheduleViewSet,
|
||||
SchemaView,
|
||||
TaskViewSet,
|
||||
TenantApiKeyViewSet,
|
||||
TenantFinishACSView,
|
||||
TenantMembersViewSet,
|
||||
TenantViewSet,
|
||||
UserRoleRelationshipView,
|
||||
UserViewSet,
|
||||
TenantApiKeyViewSet,
|
||||
)
|
||||
|
||||
router = routers.DefaultRouter(trailing_slash=False)
|
||||
@@ -67,6 +70,16 @@ router.register(
|
||||
basename="lighthouseconfiguration",
|
||||
)
|
||||
router.register(r"api-keys", TenantApiKeyViewSet, basename="api-key")
|
||||
router.register(
|
||||
r"lighthouse/providers",
|
||||
LighthouseProviderConfigViewSet,
|
||||
basename="lighthouse-providers",
|
||||
)
|
||||
router.register(
|
||||
r"lighthouse/models",
|
||||
LighthouseProviderModelsViewSet,
|
||||
basename="lighthouse-models",
|
||||
)
|
||||
|
||||
tenants_router = routers.NestedSimpleRouter(router, r"tenants", lookup="tenant")
|
||||
tenants_router.register(
|
||||
@@ -137,6 +150,14 @@ urlpatterns = [
|
||||
),
|
||||
name="provider_group-providers-relationship",
|
||||
),
|
||||
# Lighthouse tenant config as singleton endpoint
|
||||
path(
|
||||
"lighthouse/configuration",
|
||||
LighthouseTenantConfigViewSet.as_view(
|
||||
{"get": "list", "patch": "partial_update"}
|
||||
),
|
||||
name="lighthouse-configurations",
|
||||
),
|
||||
# API endpoint to start SAML SSO flow
|
||||
path(
|
||||
"auth/saml/initiate/", SAMLInitiateAPIView.as_view(), name="api_saml_initiate"
|
||||
|
||||
+340
-14
@@ -1,5 +1,6 @@
|
||||
import fnmatch
|
||||
import glob
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime, timedelta, timezone
|
||||
@@ -60,11 +61,13 @@ from tasks.tasks import (
|
||||
backfill_scan_resource_summaries_task,
|
||||
check_integration_connection_task,
|
||||
check_lighthouse_connection_task,
|
||||
check_lighthouse_provider_connection_task,
|
||||
check_provider_connection_task,
|
||||
delete_provider_task,
|
||||
delete_tenant_task,
|
||||
jira_integration_task,
|
||||
perform_scan_task,
|
||||
refresh_lighthouse_provider_models_task,
|
||||
)
|
||||
|
||||
from api.base_views import BaseRLSViewSet, BaseTenantViewset, BaseUserViewset
|
||||
@@ -84,6 +87,8 @@ from api.filters import (
|
||||
InvitationFilter,
|
||||
LatestFindingFilter,
|
||||
LatestResourceFilter,
|
||||
LighthouseProviderConfigFilter,
|
||||
LighthouseProviderModelsFilter,
|
||||
MembershipFilter,
|
||||
ProcessorFilter,
|
||||
ProviderFilter,
|
||||
@@ -106,6 +111,9 @@ from api.models import (
|
||||
Integration,
|
||||
Invitation,
|
||||
LighthouseConfiguration,
|
||||
LighthouseProviderConfiguration,
|
||||
LighthouseProviderModels,
|
||||
LighthouseTenantConfiguration,
|
||||
Membership,
|
||||
Processor,
|
||||
Provider,
|
||||
@@ -139,7 +147,7 @@ from api.utils import (
|
||||
validate_invitation,
|
||||
)
|
||||
from api.uuid_utils import datetime_to_uuid7, uuid7_start
|
||||
from api.v1.mixins import PaginateByPkMixin, TaskManagementMixin
|
||||
from api.v1.mixins import DisablePaginationMixin, PaginateByPkMixin, TaskManagementMixin
|
||||
from api.v1.serializers import (
|
||||
ComplianceOverviewAttributesSerializer,
|
||||
ComplianceOverviewDetailSerializer,
|
||||
@@ -160,8 +168,15 @@ from api.v1.serializers import (
|
||||
LighthouseConfigCreateSerializer,
|
||||
LighthouseConfigSerializer,
|
||||
LighthouseConfigUpdateSerializer,
|
||||
LighthouseProviderConfigCreateSerializer,
|
||||
LighthouseProviderConfigSerializer,
|
||||
LighthouseProviderConfigUpdateSerializer,
|
||||
LighthouseProviderModelsSerializer,
|
||||
LighthouseTenantConfigSerializer,
|
||||
LighthouseTenantConfigUpdateSerializer,
|
||||
MembershipSerializer,
|
||||
OverviewFindingSerializer,
|
||||
OverviewProviderCountSerializer,
|
||||
OverviewProviderSerializer,
|
||||
OverviewServiceSerializer,
|
||||
OverviewSeveritySerializer,
|
||||
@@ -307,7 +322,7 @@ class SchemaView(SpectacularAPIView):
|
||||
|
||||
def get(self, request, *args, **kwargs):
|
||||
spectacular_settings.TITLE = "Prowler API"
|
||||
spectacular_settings.VERSION = "1.14.0"
|
||||
spectacular_settings.VERSION = "1.15.0"
|
||||
spectacular_settings.DESCRIPTION = (
|
||||
"Prowler API specification.\n\nThis file is auto-generated."
|
||||
)
|
||||
@@ -1417,7 +1432,7 @@ class ProviderGroupProvidersRelationshipView(RelationshipView, BaseRLSViewSet):
|
||||
)
|
||||
@method_decorator(CACHE_DECORATOR, name="list")
|
||||
@method_decorator(CACHE_DECORATOR, name="retrieve")
|
||||
class ProviderViewSet(BaseRLSViewSet):
|
||||
class ProviderViewSet(DisablePaginationMixin, BaseRLSViewSet):
|
||||
queryset = Provider.objects.all()
|
||||
serializer_class = ProviderSerializer
|
||||
http_method_names = ["get", "post", "patch", "delete"]
|
||||
@@ -3677,6 +3692,13 @@ class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
|
||||
"each provider are considered in the aggregation to ensure accurate and up-to-date insights."
|
||||
),
|
||||
),
|
||||
providers_count=extend_schema(
|
||||
summary="Get provider counts grouped by type",
|
||||
description=(
|
||||
"Retrieve the number of providers grouped by provider type. "
|
||||
"This endpoint counts every provider in the tenant, including those without completed scans."
|
||||
),
|
||||
),
|
||||
findings=extend_schema(
|
||||
summary="Get aggregated findings data",
|
||||
description=(
|
||||
@@ -3728,6 +3750,8 @@ class OverviewViewSet(BaseRLSViewSet):
|
||||
def get_serializer_class(self):
|
||||
if self.action == "providers":
|
||||
return OverviewProviderSerializer
|
||||
elif self.action == "providers_count":
|
||||
return OverviewProviderCountSerializer
|
||||
elif self.action == "findings":
|
||||
return OverviewFindingSerializer
|
||||
elif self.action == "findings_severity":
|
||||
@@ -3776,10 +3800,7 @@ class OverviewViewSet(BaseRLSViewSet):
|
||||
|
||||
findings_aggregated = (
|
||||
queryset.filter(scan_id__in=latest_scan_ids)
|
||||
.values(
|
||||
"scan__provider_id",
|
||||
provider=F("scan__provider__provider"),
|
||||
)
|
||||
.values(provider=F("scan__provider__provider"))
|
||||
.annotate(
|
||||
findings_passed=Coalesce(Sum("_pass"), 0),
|
||||
findings_failed=Coalesce(Sum("fail"), 0),
|
||||
@@ -3788,13 +3809,16 @@ class OverviewViewSet(BaseRLSViewSet):
|
||||
)
|
||||
)
|
||||
|
||||
resources_aggregated = (
|
||||
Resource.all_objects.filter(tenant_id=tenant_id)
|
||||
.values("provider_id")
|
||||
.annotate(total_resources=Count("id"))
|
||||
)
|
||||
resources_queryset = Resource.all_objects.filter(tenant_id=tenant_id)
|
||||
if hasattr(self, "allowed_providers"):
|
||||
resources_queryset = resources_queryset.filter(
|
||||
provider__in=self.allowed_providers
|
||||
)
|
||||
resources_aggregated = resources_queryset.values(
|
||||
provider_type=F("provider__provider")
|
||||
).annotate(total_resources=Count("id"))
|
||||
resource_map = {
|
||||
row["provider_id"]: row["total_resources"] for row in resources_aggregated
|
||||
row["provider_type"]: row["total_resources"] for row in resources_aggregated
|
||||
}
|
||||
|
||||
overview = []
|
||||
@@ -3802,7 +3826,7 @@ class OverviewViewSet(BaseRLSViewSet):
|
||||
overview.append(
|
||||
{
|
||||
"provider": row["provider"],
|
||||
"total_resources": resource_map.get(row["scan__provider_id"], 0),
|
||||
"total_resources": resource_map.get(row["provider"], 0),
|
||||
"total_findings": row["total_findings"],
|
||||
"findings_passed": row["findings_passed"],
|
||||
"findings_failed": row["findings_failed"],
|
||||
@@ -3815,6 +3839,36 @@ class OverviewViewSet(BaseRLSViewSet):
|
||||
status=status.HTTP_200_OK,
|
||||
)
|
||||
|
||||
@action(
|
||||
detail=False,
|
||||
methods=["get"],
|
||||
url_path="providers/count",
|
||||
url_name="providers-count",
|
||||
)
|
||||
def providers_count(self, request):
|
||||
tenant_id = self.request.tenant_id
|
||||
providers_qs = Provider.objects.filter(tenant_id=tenant_id)
|
||||
|
||||
if hasattr(self, "allowed_providers"):
|
||||
allowed_ids = list(self.allowed_providers.values_list("id", flat=True))
|
||||
if not allowed_ids:
|
||||
overview = []
|
||||
return Response(
|
||||
self.get_serializer(overview, many=True).data,
|
||||
status=status.HTTP_200_OK,
|
||||
)
|
||||
providers_qs = providers_qs.filter(id__in=allowed_ids)
|
||||
|
||||
overview = (
|
||||
providers_qs.values("provider")
|
||||
.annotate(count=Count("id"))
|
||||
.order_by("provider")
|
||||
)
|
||||
return Response(
|
||||
self.get_serializer(overview, many=True).data,
|
||||
status=status.HTTP_200_OK,
|
||||
)
|
||||
|
||||
@action(detail=False, methods=["get"], url_name="findings")
|
||||
def findings(self, request):
|
||||
tenant_id = self.request.tenant_id
|
||||
@@ -4177,21 +4231,25 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
|
||||
tags=["Lighthouse AI"],
|
||||
summary="List all Lighthouse AI configurations",
|
||||
description="Retrieve a list of all Lighthouse AI configurations.",
|
||||
deprecated=True,
|
||||
),
|
||||
create=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Create a new Lighthouse AI configuration",
|
||||
description="Create a new Lighthouse AI configuration with the specified details.",
|
||||
deprecated=True,
|
||||
),
|
||||
partial_update=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Partially update a Lighthouse AI configuration",
|
||||
description="Update certain fields of an existing Lighthouse AI configuration.",
|
||||
deprecated=True,
|
||||
),
|
||||
destroy=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Delete a Lighthouse AI configuration",
|
||||
description="Remove a Lighthouse AI configuration by its ID.",
|
||||
deprecated=True,
|
||||
),
|
||||
connection=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
@@ -4199,6 +4257,7 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
|
||||
description="Verify the connection to the OpenAI API for a specific Lighthouse AI configuration.",
|
||||
request=None,
|
||||
responses={202: OpenApiResponse(response=TaskSerializer)},
|
||||
deprecated=True,
|
||||
),
|
||||
)
|
||||
class LighthouseConfigViewSet(BaseRLSViewSet):
|
||||
@@ -4249,6 +4308,273 @@ class LighthouseConfigViewSet(BaseRLSViewSet):
|
||||
)
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
list=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="List all LLM provider configs",
|
||||
description="Retrieve all LLM provider configurations for the current tenant",
|
||||
),
|
||||
retrieve=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Retrieve LLM provider config",
|
||||
description="Get details for a specific provider configuration in the current tenant.",
|
||||
),
|
||||
create=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Create LLM provider config",
|
||||
description="Create a per-tenant configuration for an LLM provider. Only one configuration per provider type is allowed per tenant.",
|
||||
),
|
||||
partial_update=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Update LLM provider config",
|
||||
description="Partially update a provider configuration (e.g., base_url, is_active).",
|
||||
),
|
||||
destroy=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Delete LLM provider config",
|
||||
description="Delete a provider configuration. Any tenant defaults that reference this provider are cleared during deletion.",
|
||||
),
|
||||
)
|
||||
class LighthouseProviderConfigViewSet(BaseRLSViewSet):
|
||||
queryset = LighthouseProviderConfiguration.objects.all()
|
||||
serializer_class = LighthouseProviderConfigSerializer
|
||||
http_method_names = ["get", "post", "patch", "delete"]
|
||||
filterset_class = LighthouseProviderConfigFilter
|
||||
|
||||
def get_queryset(self):
|
||||
if getattr(self, "swagger_fake_view", False):
|
||||
return LighthouseProviderConfiguration.objects.none()
|
||||
return LighthouseProviderConfiguration.objects.filter(
|
||||
tenant_id=self.request.tenant_id
|
||||
)
|
||||
|
||||
def get_serializer_class(self):
|
||||
if self.action == "create":
|
||||
return LighthouseProviderConfigCreateSerializer
|
||||
elif self.action == "partial_update":
|
||||
return LighthouseProviderConfigUpdateSerializer
|
||||
elif self.action in ["connection", "refresh_models"]:
|
||||
return TaskSerializer
|
||||
return super().get_serializer_class()
|
||||
|
||||
def create(self, request, *args, **kwargs):
|
||||
serializer = self.get_serializer(data=request.data)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
instance = serializer.save()
|
||||
|
||||
read_serializer = LighthouseProviderConfigSerializer(
|
||||
instance, context=self.get_serializer_context()
|
||||
)
|
||||
headers = self.get_success_headers(read_serializer.data)
|
||||
return Response(
|
||||
data=read_serializer.data,
|
||||
status=status.HTTP_201_CREATED,
|
||||
headers=headers,
|
||||
)
|
||||
|
||||
def partial_update(self, request, *args, **kwargs):
|
||||
instance = self.get_object()
|
||||
serializer = self.get_serializer(
|
||||
instance,
|
||||
data=request.data,
|
||||
partial=True,
|
||||
context=self.get_serializer_context(),
|
||||
)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
serializer.save()
|
||||
read_serializer = LighthouseProviderConfigSerializer(
|
||||
instance, context=self.get_serializer_context()
|
||||
)
|
||||
return Response(data=read_serializer.data, status=status.HTTP_200_OK)
|
||||
|
||||
@extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Check LLM provider connection",
|
||||
description="Validate provider credentials asynchronously and toggle is_active.",
|
||||
request=None,
|
||||
responses={202: OpenApiResponse(response=TaskSerializer)},
|
||||
)
|
||||
@action(detail=True, methods=["post"], url_name="connection")
|
||||
def connection(self, request, pk=None):
|
||||
instance = self.get_object()
|
||||
if (
|
||||
instance.provider_type
|
||||
!= LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
|
||||
):
|
||||
return Response(
|
||||
data={
|
||||
"errors": [{"detail": "Only 'openai' provider supported in MVP"}]
|
||||
},
|
||||
status=status.HTTP_400_BAD_REQUEST,
|
||||
)
|
||||
|
||||
with transaction.atomic():
|
||||
task = check_lighthouse_provider_connection_task.delay(
|
||||
provider_config_id=str(instance.id), tenant_id=self.request.tenant_id
|
||||
)
|
||||
|
||||
prowler_task = Task.objects.get(id=task.id)
|
||||
serializer = TaskSerializer(prowler_task)
|
||||
return Response(
|
||||
data=serializer.data,
|
||||
status=status.HTTP_202_ACCEPTED,
|
||||
headers={
|
||||
"Content-Location": reverse(
|
||||
"task-detail", kwargs={"pk": prowler_task.id}
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
@extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Refresh LLM models catalog",
|
||||
description="Fetch available models for this provider configuration and upsert into catalog.",
|
||||
request=None,
|
||||
responses={202: OpenApiResponse(response=TaskSerializer)},
|
||||
)
|
||||
@action(
|
||||
detail=True,
|
||||
methods=["post"],
|
||||
url_path="refresh-models",
|
||||
url_name="refresh-models",
|
||||
)
|
||||
def refresh_models(self, request, pk=None):
|
||||
instance = self.get_object()
|
||||
if (
|
||||
instance.provider_type
|
||||
!= LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
|
||||
):
|
||||
return Response(
|
||||
data={
|
||||
"errors": [{"detail": "Only 'openai' provider supported in MVP"}]
|
||||
},
|
||||
status=status.HTTP_400_BAD_REQUEST,
|
||||
)
|
||||
|
||||
with transaction.atomic():
|
||||
task = refresh_lighthouse_provider_models_task.delay(
|
||||
provider_config_id=str(instance.id), tenant_id=self.request.tenant_id
|
||||
)
|
||||
|
||||
prowler_task = Task.objects.get(id=task.id)
|
||||
serializer = TaskSerializer(prowler_task)
|
||||
return Response(
|
||||
data=serializer.data,
|
||||
status=status.HTTP_202_ACCEPTED,
|
||||
headers={
|
||||
"Content-Location": reverse(
|
||||
"task-detail", kwargs={"pk": prowler_task.id}
|
||||
)
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
list=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Get Lighthouse AI Tenant config",
|
||||
description="Retrieve current tenant-level Lighthouse AI settings. Returns a single configuration object.",
|
||||
),
|
||||
partial_update=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Update Lighthouse AI Tenant config",
|
||||
description="Update tenant-level settings. Validates that the default provider is configured and active and that default model IDs exist for the chosen providers. Auto-creates configuration if it doesn't exist.",
|
||||
),
|
||||
)
|
||||
class LighthouseTenantConfigViewSet(BaseRLSViewSet):
|
||||
"""
|
||||
Singleton endpoint for tenant-level Lighthouse AI configuration.
|
||||
|
||||
This viewset implements a true singleton pattern:
|
||||
- GET returns the single configuration object (or 404 if not found)
|
||||
- PATCH updates/creates the configuration (upsert semantics)
|
||||
- No ID is required in the URL
|
||||
"""
|
||||
|
||||
queryset = LighthouseTenantConfiguration.objects.all()
|
||||
serializer_class = LighthouseTenantConfigSerializer
|
||||
http_method_names = ["get", "patch"]
|
||||
|
||||
def get_queryset(self):
|
||||
if getattr(self, "swagger_fake_view", False):
|
||||
return LighthouseTenantConfiguration.objects.none()
|
||||
return LighthouseTenantConfiguration.objects.filter(
|
||||
tenant_id=self.request.tenant_id
|
||||
)
|
||||
|
||||
def get_serializer_class(self):
|
||||
if self.action == "partial_update":
|
||||
return LighthouseTenantConfigUpdateSerializer
|
||||
return super().get_serializer_class()
|
||||
|
||||
def get_object(self):
|
||||
"""Retrieve the singleton instance for the current tenant."""
|
||||
obj = LighthouseTenantConfiguration.objects.filter(
|
||||
tenant_id=self.request.tenant_id
|
||||
).first()
|
||||
if obj is None:
|
||||
raise NotFound("Tenant Lighthouse configuration not found")
|
||||
self.check_object_permissions(self.request, obj)
|
||||
return obj
|
||||
|
||||
def list(self, request, *args, **kwargs):
|
||||
"""GET endpoint for singleton - returns single object, not an array."""
|
||||
instance = self.get_object()
|
||||
serializer = self.get_serializer(instance)
|
||||
return Response(serializer.data)
|
||||
|
||||
def partial_update(self, request, *args, **kwargs):
|
||||
"""PATCH endpoint for singleton - no pk required. Auto-creates if not exists."""
|
||||
# Auto-create tenant config if it doesn't exist (upsert semantics)
|
||||
instance, created = LighthouseTenantConfiguration.objects.get_or_create(
|
||||
tenant_id=self.request.tenant_id,
|
||||
defaults={},
|
||||
)
|
||||
|
||||
# Extract attributes from JSON:API payload
|
||||
try:
|
||||
payload = json.loads(request.body)
|
||||
attributes = payload.get("data", {}).get("attributes", {})
|
||||
except (json.JSONDecodeError, AttributeError):
|
||||
raise ValidationError("Invalid JSON:API payload")
|
||||
|
||||
serializer = self.get_serializer(instance, data=attributes, partial=True)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
serializer.save()
|
||||
read_serializer = LighthouseTenantConfigSerializer(
|
||||
instance, context=self.get_serializer_context()
|
||||
)
|
||||
return Response(read_serializer.data, status=status.HTTP_200_OK)
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
list=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="List all LLM models",
|
||||
description="List available LLM models per configured provider for the current tenant.",
|
||||
),
|
||||
retrieve=extend_schema(
|
||||
tags=["Lighthouse AI"],
|
||||
summary="Retrieve LLM model details",
|
||||
description="Get details for a specific LLM model.",
|
||||
),
|
||||
)
|
||||
class LighthouseProviderModelsViewSet(BaseRLSViewSet):
|
||||
queryset = LighthouseProviderModels.objects.all()
|
||||
serializer_class = LighthouseProviderModelsSerializer
|
||||
filterset_class = LighthouseProviderModelsFilter
|
||||
# Expose as read-only catalog collection
|
||||
http_method_names = ["get"]
|
||||
|
||||
def get_queryset(self):
|
||||
if getattr(self, "swagger_fake_view", False):
|
||||
return LighthouseProviderModels.objects.none()
|
||||
return LighthouseProviderModels.objects.filter(tenant_id=self.request.tenant_id)
|
||||
|
||||
def get_serializer_class(self):
|
||||
return super().get_serializer_class()
|
||||
|
||||
|
||||
@extend_schema_view(
|
||||
list=extend_schema(
|
||||
tags=["Processor"],
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
from django.contrib import admin
|
||||
from django.urls import include, path
|
||||
|
||||
urlpatterns = [
|
||||
path("admin/", admin.site.urls),
|
||||
path("api/v1/", include("api.v1.urls")),
|
||||
]
|
||||
|
||||
@@ -499,8 +499,14 @@ def providers_fixture(tenants_fixture):
|
||||
alias="m365_testing",
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
provider7 = Provider.objects.create(
|
||||
provider="oci",
|
||||
uid="ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
alias="oci_testing",
|
||||
tenant_id=tenant.id,
|
||||
)
|
||||
|
||||
return provider1, provider2, provider3, provider4, provider5, provider6
|
||||
return provider1, provider2, provider3, provider4, provider5, provider6, provider7
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -21,6 +21,8 @@ from prowler.lib.outputs.compliance.aws_well_architected.aws_well_architected im
|
||||
AWSWellArchitected,
|
||||
)
|
||||
from prowler.lib.outputs.compliance.c5.c5_aws import AWSC5
|
||||
from prowler.lib.outputs.compliance.c5.c5_azure import AzureC5
|
||||
from prowler.lib.outputs.compliance.c5.c5_gcp import GCPC5
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_aws import CCC_AWS
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_azure import CCC_Azure
|
||||
from prowler.lib.outputs.compliance.ccc.ccc_gcp import CCC_GCP
|
||||
@@ -30,6 +32,7 @@ from prowler.lib.outputs.compliance.cis.cis_gcp import GCPCIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_github import GithubCIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_kubernetes import KubernetesCIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_m365 import M365CIS
|
||||
from prowler.lib.outputs.compliance.cis.cis_oci import OCICIS
|
||||
from prowler.lib.outputs.compliance.ens.ens_aws import AWSENS
|
||||
from prowler.lib.outputs.compliance.ens.ens_azure import AzureENS
|
||||
from prowler.lib.outputs.compliance.ens.ens_gcp import GCPENS
|
||||
@@ -87,6 +90,7 @@ COMPLIANCE_CLASS_MAP = {
|
||||
(lambda name: name.startswith("iso27001_"), AzureISO27001),
|
||||
(lambda name: name == "ccc_azure", CCC_Azure),
|
||||
(lambda name: name == "prowler_threatscore_azure", ProwlerThreatScoreAzure),
|
||||
(lambda name: name == "c5_azure", AzureC5),
|
||||
],
|
||||
"gcp": [
|
||||
(lambda name: name.startswith("cis_"), GCPCIS),
|
||||
@@ -95,6 +99,7 @@ COMPLIANCE_CLASS_MAP = {
|
||||
(lambda name: name.startswith("iso27001_"), GCPISO27001),
|
||||
(lambda name: name == "prowler_threatscore_gcp", ProwlerThreatScoreGCP),
|
||||
(lambda name: name == "ccc_gcp", CCC_GCP),
|
||||
(lambda name: name == "c5_gcp", GCPC5),
|
||||
],
|
||||
"kubernetes": [
|
||||
(lambda name: name.startswith("cis_"), KubernetesCIS),
|
||||
@@ -108,6 +113,9 @@ COMPLIANCE_CLASS_MAP = {
|
||||
"github": [
|
||||
(lambda name: name.startswith("cis_"), GithubCIS),
|
||||
],
|
||||
"oci": [
|
||||
(lambda name: name.startswith("cis_"), OCICIS),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ from celery.utils.log import get_task_logger
|
||||
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
|
||||
from tasks.utils import batched
|
||||
|
||||
from api.db_router import READ_REPLICA_ALIAS
|
||||
from api.db_router import READ_REPLICA_ALIAS, MainRouter
|
||||
from api.db_utils import rls_transaction
|
||||
from api.models import Finding, Integration, Provider
|
||||
from api.utils import initialize_prowler_integration, initialize_prowler_provider
|
||||
@@ -179,7 +179,7 @@ def get_security_hub_client_from_integration(
|
||||
if the connection was successful and the SecurityHub client or connection object.
|
||||
"""
|
||||
# Get the provider associated with this integration
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
provider_relationship = integration.integrationproviderrelationship_set.first()
|
||||
if not provider_relationship:
|
||||
return Connection(
|
||||
@@ -208,7 +208,7 @@ def get_security_hub_client_from_integration(
|
||||
regions_status[region] = region in connection.enabled_regions
|
||||
|
||||
# Save regions information in the integration configuration
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=MainRouter.default_db):
|
||||
integration.configuration["regions"] = regions_status
|
||||
integration.save()
|
||||
|
||||
@@ -223,7 +223,7 @@ def get_security_hub_client_from_integration(
|
||||
return True, security_hub
|
||||
else:
|
||||
# Reset regions information if connection fails
|
||||
with rls_transaction(tenant_id):
|
||||
with rls_transaction(tenant_id, using=MainRouter.default_db):
|
||||
integration.configuration["regions"] = {}
|
||||
integration.save()
|
||||
|
||||
@@ -334,8 +334,11 @@ def upload_security_hub_integration(
|
||||
f"Security Hub connection failed for integration {integration.id}: "
|
||||
f"{security_hub.error}"
|
||||
)
|
||||
integration.connected = False
|
||||
integration.save()
|
||||
with rls_transaction(
|
||||
tenant_id, using=MainRouter.default_db
|
||||
):
|
||||
integration.connected = False
|
||||
integration.save()
|
||||
break # Skip this integration
|
||||
|
||||
security_hub_client = security_hub
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
from typing import Dict, Set
|
||||
|
||||
import openai
|
||||
from celery.utils.log import get_task_logger
|
||||
|
||||
from api.models import LighthouseProviderConfiguration, LighthouseProviderModels
|
||||
|
||||
logger = get_task_logger(__name__)
|
||||
|
||||
|
||||
def _extract_openai_api_key(
|
||||
provider_cfg: LighthouseProviderConfiguration,
|
||||
) -> str | None:
|
||||
"""
|
||||
Safely extract the OpenAI API key from a provider configuration.
|
||||
|
||||
Args:
|
||||
provider_cfg (LighthouseProviderConfiguration): The provider configuration instance
|
||||
containing the credentials.
|
||||
|
||||
Returns:
|
||||
str | None: The API key string if present and valid, otherwise None.
|
||||
"""
|
||||
creds = provider_cfg.credentials_decoded
|
||||
if not isinstance(creds, dict):
|
||||
return None
|
||||
api_key = creds.get("api_key")
|
||||
if not isinstance(api_key, str) or not api_key:
|
||||
return None
|
||||
return api_key
|
||||
|
||||
|
||||
def check_lighthouse_provider_connection(provider_config_id: str) -> Dict:
|
||||
"""
|
||||
Validate a Lighthouse provider configuration by calling the provider API and
|
||||
toggle its active state accordingly.
|
||||
|
||||
Currently supports the OpenAI provider by invoking `models.list` to verify that
|
||||
the provided credentials are valid.
|
||||
|
||||
Args:
|
||||
provider_config_id (str): The primary key of the `LighthouseProviderConfiguration`
|
||||
to validate.
|
||||
|
||||
Returns:
|
||||
dict: A result dictionary with the following keys:
|
||||
- "connected" (bool): Whether the provider credentials are valid.
|
||||
- "error" (str | None): The error message when not connected, otherwise None.
|
||||
|
||||
Side Effects:
|
||||
- Updates and persists `is_active` on the `LighthouseProviderConfiguration`.
|
||||
|
||||
Raises:
|
||||
LighthouseProviderConfiguration.DoesNotExist: If no configuration exists with the given ID.
|
||||
"""
|
||||
provider_cfg = LighthouseProviderConfiguration.objects.get(pk=provider_config_id)
|
||||
|
||||
# TODO: Add support for other providers
|
||||
if (
|
||||
provider_cfg.provider_type
|
||||
!= LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
|
||||
):
|
||||
return {"connected": False, "error": "Unsupported provider type"}
|
||||
|
||||
api_key = _extract_openai_api_key(provider_cfg)
|
||||
if not api_key:
|
||||
provider_cfg.is_active = False
|
||||
provider_cfg.save()
|
||||
return {"connected": False, "error": "API key is invalid or missing"}
|
||||
|
||||
try:
|
||||
client = openai.OpenAI(api_key=api_key)
|
||||
_ = client.models.list()
|
||||
provider_cfg.is_active = True
|
||||
provider_cfg.save()
|
||||
return {"connected": True, "error": None}
|
||||
except Exception as e:
|
||||
logger.warning("OpenAI connection check failed: %s", str(e))
|
||||
provider_cfg.is_active = False
|
||||
provider_cfg.save()
|
||||
return {"connected": False, "error": str(e)}
|
||||
|
||||
|
||||
def refresh_lighthouse_provider_models(provider_config_id: str) -> Dict:
|
||||
"""
|
||||
Refresh the catalog of models for a Lighthouse provider configuration.
|
||||
|
||||
For the OpenAI provider, this fetches the current list of models, upserts entries
|
||||
into `LighthouseProviderModels`, and deletes stale entries no longer returned by
|
||||
the provider.
|
||||
|
||||
Args:
|
||||
provider_config_id (str): The primary key of the `LighthouseProviderConfiguration`
|
||||
whose models should be refreshed.
|
||||
|
||||
Returns:
|
||||
dict: A result dictionary with the following keys on success:
|
||||
- "created" (int): Number of new model rows created.
|
||||
- "updated" (int): Number of existing model rows updated.
|
||||
- "deleted" (int): Number of stale model rows removed.
|
||||
If an error occurs, the dictionary will contain an "error" (str) field instead.
|
||||
|
||||
Raises:
|
||||
LighthouseProviderConfiguration.DoesNotExist: If no configuration exists with the given ID.
|
||||
"""
|
||||
provider_cfg = LighthouseProviderConfiguration.objects.get(pk=provider_config_id)
|
||||
|
||||
if (
|
||||
provider_cfg.provider_type
|
||||
!= LighthouseProviderConfiguration.LLMProviderChoices.OPENAI
|
||||
):
|
||||
return {
|
||||
"created": 0,
|
||||
"updated": 0,
|
||||
"deleted": 0,
|
||||
"error": "Unsupported provider type",
|
||||
}
|
||||
|
||||
api_key = _extract_openai_api_key(provider_cfg)
|
||||
if not api_key:
|
||||
return {
|
||||
"created": 0,
|
||||
"updated": 0,
|
||||
"deleted": 0,
|
||||
"error": "API key is invalid or missing",
|
||||
}
|
||||
|
||||
try:
|
||||
client = openai.OpenAI(api_key=api_key)
|
||||
models = client.models.list()
|
||||
fetched_ids: Set[str] = {m.id for m in getattr(models, "data", [])}
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("OpenAI models refresh failed: %s", str(e))
|
||||
return {"created": 0, "updated": 0, "deleted": 0, "error": str(e)}
|
||||
|
||||
created = 0
|
||||
updated = 0
|
||||
|
||||
for model_id in fetched_ids:
|
||||
obj, was_created = LighthouseProviderModels.objects.update_or_create(
|
||||
tenant_id=provider_cfg.tenant_id,
|
||||
provider_configuration=provider_cfg,
|
||||
model_id=model_id,
|
||||
defaults={
|
||||
"model_name": model_id, # OpenAI doesn't return a separate display name
|
||||
"default_parameters": {},
|
||||
},
|
||||
)
|
||||
if was_created:
|
||||
created += 1
|
||||
else:
|
||||
updated += 1
|
||||
|
||||
# Delete stale models not present anymore
|
||||
deleted, _ = (
|
||||
LighthouseProviderModels.objects.filter(
|
||||
tenant_id=provider_cfg.tenant_id, provider_configuration=provider_cfg
|
||||
)
|
||||
.exclude(model_id__in=fetched_ids)
|
||||
.delete()
|
||||
)
|
||||
|
||||
return {"created": created, "updated": updated, "deleted": deleted}
|
||||
@@ -27,6 +27,10 @@ from tasks.jobs.integrations import (
|
||||
upload_s3_integration,
|
||||
upload_security_hub_integration,
|
||||
)
|
||||
from tasks.jobs.lighthouse_providers import (
|
||||
check_lighthouse_provider_connection,
|
||||
refresh_lighthouse_provider_models,
|
||||
)
|
||||
from tasks.jobs.report import generate_threatscore_report_job
|
||||
from tasks.jobs.scan import (
|
||||
aggregate_findings,
|
||||
@@ -524,6 +528,24 @@ def check_lighthouse_connection_task(lighthouse_config_id: str, tenant_id: str =
|
||||
return check_lighthouse_connection(lighthouse_config_id=lighthouse_config_id)
|
||||
|
||||
|
||||
@shared_task(base=RLSTask, name="lighthouse-provider-connection-check")
|
||||
@set_tenant
|
||||
def check_lighthouse_provider_connection_task(
|
||||
provider_config_id: str, tenant_id: str | None = None
|
||||
) -> dict:
|
||||
"""Task wrapper to validate provider credentials and set is_active."""
|
||||
return check_lighthouse_provider_connection(provider_config_id=provider_config_id)
|
||||
|
||||
|
||||
@shared_task(base=RLSTask, name="lighthouse-provider-models-refresh")
|
||||
@set_tenant
|
||||
def refresh_lighthouse_provider_models_task(
|
||||
provider_config_id: str, tenant_id: str | None = None
|
||||
) -> dict:
|
||||
"""Task wrapper to refresh provider models catalog for the given configuration."""
|
||||
return refresh_lighthouse_provider_models(provider_config_id=provider_config_id)
|
||||
|
||||
|
||||
@shared_task(name="integration-check")
|
||||
def check_integrations_task(tenant_id: str, provider_id: str, scan_id: str = None):
|
||||
"""
|
||||
|
||||
@@ -9,6 +9,7 @@ from tasks.jobs.integrations import (
|
||||
upload_security_hub_integration,
|
||||
)
|
||||
|
||||
from api.db_router import READ_REPLICA_ALIAS, MainRouter
|
||||
from api.models import Integration
|
||||
from api.utils import prowler_integration_connection_test
|
||||
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHubConnection
|
||||
@@ -880,7 +881,8 @@ class TestSecurityHubIntegrationUploads:
|
||||
# Verify RLS transaction was used correctly
|
||||
# Should be called twice: once for getting provider info, once for resetting regions
|
||||
assert mock_rls.call_count == 2
|
||||
mock_rls.assert_any_call(tenant_id)
|
||||
mock_rls.assert_any_call(tenant_id, using=READ_REPLICA_ALIAS)
|
||||
mock_rls.assert_any_call(tenant_id, using=MainRouter.default_db)
|
||||
|
||||
# Verify test_connection was called with integration credentials (not provider's)
|
||||
mock_test_connection.assert_called_once_with(
|
||||
|
||||
@@ -35,7 +35,8 @@ dashboard = dash.Dash(
|
||||
|
||||
# Logo
|
||||
prowler_logo = html.Img(
|
||||
src="https://prowler.com/wp-content/uploads/logo-dashboard.png", alt="Prowler Logo"
|
||||
src="https://cdn.prod.website-files.com/68c4ec3f9fb7b154fbcb6e36/68ffb46d40ed7faa37a592a5_prowler-logo.png",
|
||||
alt="Prowler Logo",
|
||||
)
|
||||
|
||||
menu_icons = {
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import warnings
|
||||
|
||||
from dashboard.common_methods import get_section_containers_3_levels
|
||||
|
||||
warnings.filterwarnings("ignore")
|
||||
|
||||
|
||||
def get_table(data):
|
||||
data["REQUIREMENTS_DESCRIPTION"] = (
|
||||
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
|
||||
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
|
||||
)
|
||||
|
||||
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION"
|
||||
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
|
||||
|
||||
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
|
||||
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
|
||||
|
||||
aux = data[
|
||||
[
|
||||
"REQUIREMENTS_DESCRIPTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"CHECKID",
|
||||
"STATUS",
|
||||
"REGION",
|
||||
"ACCOUNTID",
|
||||
"RESOURCEID",
|
||||
]
|
||||
]
|
||||
|
||||
return get_section_containers_3_levels(
|
||||
aux,
|
||||
"REQUIREMENTS_ATTRIBUTES_SECTION",
|
||||
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
|
||||
"REQUIREMENTS_DESCRIPTION",
|
||||
)
|
||||
@@ -0,0 +1,335 @@
|
||||
# Prowler API Reference Documentation
|
||||
|
||||
This directory contains the API reference documentation for Prowler Cloud, integrated with Mintlify.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
api-reference/
|
||||
├── README.md # This file
|
||||
├── openapi.yaml # OpenAPI specification (auto-synced from api/src/backend/api/specs/v1.yaml)
|
||||
├── introduction.mdx # API introduction and getting started guide
|
||||
│
|
||||
├── tokens/ # Authentication endpoints
|
||||
│ ├── create.mdx # Create JWT token
|
||||
│ ├── refresh.mdx # Refresh JWT token
|
||||
│ └── switch.mdx # Switch tenant context
|
||||
│
|
||||
├── api-keys/ # API Keys endpoints
|
||||
│ ├── list.mdx
|
||||
│ ├── create.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── update.mdx
|
||||
│ └── revoke.mdx
|
||||
│
|
||||
├── users/ # User endpoints
|
||||
│ └── me.mdx # Get current user
|
||||
│
|
||||
├── tenants/ # Tenant management
|
||||
│ ├── list.mdx
|
||||
│ ├── invitations-list.mdx
|
||||
│ └── invitations-create.mdx
|
||||
│
|
||||
├── invitations/ # Invitation endpoints
|
||||
│ └── accept.mdx
|
||||
│
|
||||
├── providers/ # Cloud provider management
|
||||
│ ├── list.mdx
|
||||
│ ├── create.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── update.mdx
|
||||
│ ├── delete.mdx
|
||||
│ └── check-connection.mdx
|
||||
│
|
||||
├── scans/ # Security scan endpoints
|
||||
│ ├── list.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── compliance.mdx
|
||||
│ ├── report.mdx
|
||||
│ └── threatscore.mdx
|
||||
│
|
||||
├── findings/ # Security findings endpoints
|
||||
│ ├── list.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── latest.mdx
|
||||
│ ├── services-regions.mdx
|
||||
│ ├── metadata.mdx
|
||||
│ └── metadata-latest.mdx
|
||||
│
|
||||
├── resources/ # Cloud resource endpoints
|
||||
│ ├── list.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── latest.mdx
|
||||
│ ├── metadata.mdx
|
||||
│ └── metadata-latest.mdx
|
||||
│
|
||||
├── compliance/ # Compliance framework endpoints
|
||||
│ ├── list.mdx
|
||||
│ ├── requirements.mdx
|
||||
│ ├── attributes.mdx
|
||||
│ └── metadata.mdx
|
||||
│
|
||||
├── overviews/ # Dashboard overview endpoints
|
||||
│ ├── findings.mdx
|
||||
│ ├── findings-severity.mdx
|
||||
│ ├── providers.mdx
|
||||
│ ├── providers-count.mdx
|
||||
│ └── services.mdx
|
||||
│
|
||||
├── integrations/ # External integrations
|
||||
│ ├── list.mdx
|
||||
│ ├── create.mdx
|
||||
│ ├── retrieve.mdx
|
||||
│ ├── update.mdx
|
||||
│ ├── delete.mdx
|
||||
│ ├── check-connection.mdx
|
||||
│ └── jira-dispatch.mdx
|
||||
│
|
||||
├── lighthouse/ # Lighthouse AI endpoints
|
||||
│ ├── configuration-get.mdx
|
||||
│ ├── configuration-update.mdx
|
||||
│ ├── providers-list.mdx
|
||||
│ ├── providers-create.mdx
|
||||
│ └── models-list.mdx
|
||||
│
|
||||
├── processors/ # Finding processors (mutelists)
|
||||
│ ├── list.mdx
|
||||
│ └── create.mdx
|
||||
│
|
||||
├── schedules/ # Automated scan scheduling
|
||||
│ └── daily.mdx
|
||||
│
|
||||
└── tasks/ # Asynchronous task management
|
||||
├── list.mdx
|
||||
└── retrieve.mdx
|
||||
```
|
||||
|
||||
**Total: 131 endpoint documentation files organized in 18 groups**
|
||||
|
||||
✅ **100% API Coverage** - All 123 operation IDs documented
|
||||
|
||||
## How It Works
|
||||
|
||||
The API documentation uses Mintlify's native OpenAPI support to automatically generate interactive API documentation from the OpenAPI specification file.
|
||||
|
||||
### Components
|
||||
|
||||
1. **openapi.yaml**: The source of truth for API endpoints, copied from `api/src/backend/api/specs/v1.yaml`
|
||||
2. **MDX files**: Enhanced documentation for each endpoint with examples, tips, and additional context
|
||||
3. **docs.json**: Mintlify configuration that references the OpenAPI spec
|
||||
|
||||
## Updating the Documentation
|
||||
|
||||
### When the API Spec Changes
|
||||
|
||||
When you update the OpenAPI specification in `api/src/backend/api/specs/v1.yaml`, you need to sync it to the docs:
|
||||
|
||||
```bash
|
||||
# From the root of the repository
|
||||
cp api/src/backend/api/specs/v1.yaml docs/api-reference/openapi.yaml
|
||||
```
|
||||
|
||||
Consider automating this with a pre-commit hook or CI/CD pipeline.
|
||||
|
||||
### Adding New Endpoints
|
||||
|
||||
To document a new API endpoint:
|
||||
|
||||
1. **Update the OpenAPI spec** in `api/src/backend/api/specs/v1.yaml`
|
||||
2. **Sync to docs**: Copy the spec to `docs/api-reference/openapi.yaml`
|
||||
3. **Create MDX file**: Create a new `.mdx` file in the appropriate directory
|
||||
4. **Update navigation**: Add the new page to `docs/docs.json` in the API Reference tab
|
||||
|
||||
#### MDX File Template
|
||||
|
||||
```mdx
|
||||
---
|
||||
title: "Endpoint Title"
|
||||
api: "METHOD /api/v1/endpoint"
|
||||
description: "Brief description of what this endpoint does."
|
||||
---
|
||||
|
||||
Detailed description of the endpoint.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `param` (required/optional) - Description
|
||||
|
||||
## Query Parameters
|
||||
|
||||
- `param` (required/optional) - Description
|
||||
|
||||
## Request Body
|
||||
|
||||
```json
|
||||
{
|
||||
"example": "request"
|
||||
}
|
||||
```
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X METHOD "https://api.prowler.com/api/v1/endpoint" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
Description of the response.
|
||||
```
|
||||
|
||||
### Testing Changes Locally
|
||||
|
||||
To preview documentation changes locally:
|
||||
|
||||
```bash
|
||||
cd docs
|
||||
mintlify dev
|
||||
```
|
||||
|
||||
Then open http://localhost:3000 in your browser.
|
||||
|
||||
## Mintlify Configuration
|
||||
|
||||
The API documentation is configured in `docs/docs.json`:
|
||||
|
||||
- **openapi**: Points to the OpenAPI spec file
|
||||
- **api.baseUrl**: The base URL for the API
|
||||
- **api.auth.method**: Authentication method (bearer token)
|
||||
- **api.playground.mode**: Interactive API playground mode
|
||||
|
||||
## API Endpoint Groups
|
||||
|
||||
### Core Endpoints
|
||||
- **Authentication (3)**: JWT token management and tenant switching
|
||||
- **API Keys (5)**: Programmatic API access management
|
||||
- **Users (1)**: User profile and information
|
||||
- **Tenants (3)**: Organization and invitation management
|
||||
|
||||
### Cloud Infrastructure
|
||||
- **Providers (6)**: Cloud provider (AWS, Azure, GCP, etc.) configuration
|
||||
- **Scans (5)**: Security scan execution and results
|
||||
- **Findings (6)**: Security findings and vulnerabilities
|
||||
- **Resources (5)**: Cloud resource inventory and metadata
|
||||
|
||||
### Compliance & Reporting
|
||||
- **Compliance (4)**: Compliance framework assessments (CIS, PCI-DSS, etc.)
|
||||
- **Overviews (5)**: Dashboard aggregated statistics
|
||||
|
||||
### Integrations & Automation
|
||||
- **Integrations (7)**: External service integrations (S3, Security Hub, JIRA, Slack)
|
||||
- **Lighthouse AI (5)**: AI-powered security insights configuration
|
||||
- **Processors (2)**: Finding processors and mutelists
|
||||
- **Schedules (1)**: Automated scan scheduling
|
||||
|
||||
### System
|
||||
- **Tasks (2)**: Asynchronous task monitoring
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep OpenAPI spec up to date**: Always update the source spec first, then sync to docs
|
||||
2. **Add examples**: Include real-world examples in MDX files
|
||||
3. **Use callouts**: Leverage Mintlify components like `<Note>`, `<Tip>`, `<Warning>` for important information
|
||||
4. **Test playground**: Verify that the interactive API playground works for each endpoint
|
||||
5. **Document filters**: For list endpoints, clearly document all available filters
|
||||
6. **Include rate limits**: Document any rate limiting or pagination requirements
|
||||
7. **Group related endpoints**: Keep related endpoints in the same directory
|
||||
8. **Use consistent naming**: Follow the pattern `action.mdx` (e.g., `list.mdx`, `create.mdx`, `retrieve.mdx`)
|
||||
|
||||
## Resources
|
||||
|
||||
- [Mintlify OpenAPI Guide](https://mintlify.com/docs/api-playground/openapi-support)
|
||||
- [Mintlify Components](https://mintlify.com/docs/content/components)
|
||||
- [JSON:API Specification](https://jsonapi.org/)
|
||||
- [Prowler Cloud API](https://api.prowler.com/api/v1/docs)
|
||||
|
||||
## Syncing OpenAPI Spec
|
||||
|
||||
The OpenAPI specification is maintained in the API repository and should be synced regularly:
|
||||
|
||||
```bash
|
||||
# Using the provided sync script
|
||||
cd docs
|
||||
./sync-api-spec.sh
|
||||
|
||||
# Or manually
|
||||
cp ../api/src/backend/api/specs/v1.yaml ./api-reference/openapi.yaml
|
||||
```
|
||||
|
||||
The `sync-api-spec.sh` script automates this process and can be integrated into your CI/CD pipeline.
|
||||
|
||||
## Automation 🤖
|
||||
|
||||
**NEW**: Documentation can now be auto-generated from the OpenAPI specification!
|
||||
|
||||
### Quick Start with Automation
|
||||
|
||||
```bash
|
||||
cd docs
|
||||
|
||||
# 1. Sync latest OpenAPI spec from GitHub
|
||||
./sync-api-spec.sh
|
||||
|
||||
# 2. Generate MDX files automatically
|
||||
python3 generate-api-docs.py
|
||||
|
||||
# 3. Test locally
|
||||
mintlify dev
|
||||
```
|
||||
|
||||
### Available Scripts
|
||||
|
||||
1. **`sync-api-spec.sh`** - Downloads latest OpenAPI spec from GitHub
|
||||
2. **`generate-api-docs.py`** - Generates/updates MDX files from OpenAPI spec
|
||||
|
||||
### Features
|
||||
|
||||
- ✅ Auto-generate MDX files from OpenAPI spec
|
||||
- ✅ Update existing or create new endpoints
|
||||
- ✅ Dry-run mode to preview changes
|
||||
- ✅ Proper directory structure and naming
|
||||
- ✅ Frontmatter generation (title, API path, description)
|
||||
- ✅ Parameter documentation extraction
|
||||
- ✅ Request/response examples
|
||||
|
||||
### Documentation
|
||||
|
||||
See **[AUTOMATION.md](./AUTOMATION.md)** for complete automation guide including:
|
||||
- Detailed usage instructions
|
||||
- CI/CD integration examples
|
||||
- Troubleshooting
|
||||
- Best practices
|
||||
|
||||
## Recent Updates
|
||||
|
||||
### January 2025 - Complete API Documentation & Automation
|
||||
|
||||
#### Documentation Expansion
|
||||
- ✅ **100% Coverage**: All 123 operation IDs documented (131 MDX files)
|
||||
- ✅ **18 Endpoint Groups**: Complete organization
|
||||
- ✅ **New Groups Added**:
|
||||
- Provider Secrets (5 endpoints)
|
||||
- Provider Groups (8 endpoints)
|
||||
- Roles (8 endpoints)
|
||||
- SAML Configuration (5 endpoints)
|
||||
- Lighthouse AI expanded (17 endpoints with nested structure)
|
||||
- Processors (5 endpoints)
|
||||
- Tasks (3 endpoints)
|
||||
- Compliance Overview (4 endpoints)
|
||||
- Overviews (5 endpoints)
|
||||
|
||||
#### Automation Implementation
|
||||
- ✅ **Auto-generation script**: `generate-api-docs.py`
|
||||
- ✅ **Sync script**: `sync-api-spec.sh`
|
||||
- ✅ **Complete automation guide**: AUTOMATION.md
|
||||
- ✅ **CI/CD ready**: GitHub Actions example included
|
||||
|
||||
#### Quality Improvements
|
||||
- ✅ Field name corrections verified against OpenAPI spec
|
||||
- ✅ JSON:API compliance in all examples
|
||||
- ✅ Proper nested structures (Provider Secrets, Tenant Memberships/Invitations)
|
||||
- ✅ Comprehensive documentation files (CORRECTIONS.md, VERIFICATION.md, IMPROVEMENTS.md)
|
||||
|
||||
**Total coverage increased from 14 to 131 documented endpoints (843% increase)**
|
||||
@@ -0,0 +1,182 @@
|
||||
---
|
||||
title: "Create API Key"
|
||||
api: "POST /api/v1/api-keys"
|
||||
description: "Create a new API key for the tenant."
|
||||
---
|
||||
|
||||
Create a new API key for programmatic access to the Prowler API. The API key will be returned in the response and should be stored securely as it cannot be retrieved later.
|
||||
|
||||
## Request Body
|
||||
|
||||
The request must follow the JSON:API specification format:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"attributes": {
|
||||
"name": "string",
|
||||
"expires_at": "2024-12-31T23:59:59Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Attributes
|
||||
|
||||
- `name` (required) - A descriptive name for the API key
|
||||
- `expires_at` (optional) - Expiration date for the API key in ISO 8601 format
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.prowler.com/api/v1/api-keys" \
|
||||
-H "Authorization: Bearer YOUR_JWT_TOKEN" \
|
||||
-H "Content-Type: application/vnd.api+json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"attributes": {
|
||||
"name": "Production API Key",
|
||||
"expires_at": "2025-12-31T23:59:59Z"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (201 Created)
|
||||
|
||||
Returns the created API key **including the full key value** (only shown once):
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "api-key-uuid",
|
||||
"attributes": {
|
||||
"name": "Production API Key",
|
||||
"prefix": "pk_live_abc123",
|
||||
"api_key": "pk_live_abc123def456ghi789jkl012mno345pqr678stu901vwx234yz",
|
||||
"expires_at": "2025-12-31T23:59:59Z",
|
||||
"revoked": false,
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"last_used_at": null
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `id` - Unique UUID for the API key record
|
||||
- `name` - Descriptive name you provided (3-100 characters)
|
||||
- `prefix` - First characters of the key (for identification, read-only)
|
||||
- `api_key` - **Full API key value** (only returned on creation, read-only)
|
||||
- `expires_at` - Expiration date in ISO 8601 format (optional)
|
||||
- `revoked` - Whether key has been revoked (always false on creation, read-only)
|
||||
- `inserted_at` - When key was created (read-only)
|
||||
- `last_used_at` - Last usage timestamp (null initially, read-only)
|
||||
|
||||
### Error Responses
|
||||
|
||||
**400 Bad Request** - Invalid expiration date
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "400",
|
||||
"title": "Invalid Expiration Date",
|
||||
"detail": "Expiration date must be in the future"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**422 Unprocessable Entity** - Missing required fields
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "422",
|
||||
"title": "Validation Error",
|
||||
"detail": "Name is required",
|
||||
"source": {
|
||||
"pointer": "/data/attributes/name"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**429 Too Many Requests** - API key limit reached
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "429",
|
||||
"title": "API Key Limit Exceeded",
|
||||
"detail": "Maximum of 10 active API keys per tenant. Revoke unused keys to create new ones."
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Usage Example
|
||||
|
||||
After creating the API key, use it to authenticate API requests:
|
||||
|
||||
```bash
|
||||
# Save the API key
|
||||
API_KEY="pk_live_abc123def456ghi789jkl012mno345pqr678stu901vwx234yz"
|
||||
|
||||
# Use it in requests
|
||||
curl -X GET "https://api.prowler.com/api/v1/providers" \
|
||||
-H "Authorization: Bearer $API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
1. **Store Securely**: Save the API key in a secure location immediately (password manager, secrets vault, environment variables)
|
||||
2. **Never Commit**: Do not commit API keys to version control systems
|
||||
3. **Use Environment Variables**: Store keys in environment variables, not in code
|
||||
4. **Set Expiration**: Always set an expiration date for API keys (recommended: 90 days)
|
||||
5. **Rotate Regularly**: Rotate API keys every 90 days
|
||||
6. **Limit Scope**: Use separate API keys for different environments (dev, staging, production)
|
||||
7. **Monitor Usage**: Regularly check `last_used_at` to identify unused keys
|
||||
8. **Revoke Unused**: Revoke API keys that are no longer needed
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
Use descriptive names that include:
|
||||
- **Environment**: Production, Staging, Development
|
||||
- **Purpose**: CI/CD, Dashboard, Integration
|
||||
- **Owner/Team**: Security Team, DevOps, Monitoring
|
||||
|
||||
Examples:
|
||||
- "Production CI/CD Pipeline"
|
||||
- "Staging Dashboard API Access"
|
||||
- "Security Team Automation"
|
||||
- "JIRA Integration - Production"
|
||||
|
||||
<Warning>
|
||||
**CRITICAL:** The API key is only shown once in the creation response. It cannot be retrieved later. If you lose the key, you must revoke it and create a new one. Store it securely immediately after creation.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
API keys have the same permissions as the user who created them. For automated systems, consider creating a dedicated service account with limited permissions.
|
||||
</Note>
|
||||
|
||||
<Tip>
|
||||
**Automation Tip:** API keys are ideal for CI/CD pipelines, scheduled scans, and integrations. Unlike JWT tokens, they don't expire hourly and don't require refresh logic.
|
||||
</Tip>
|
||||
@@ -0,0 +1,124 @@
|
||||
---
|
||||
title: "List API Keys"
|
||||
api: "GET /api/v1/api-keys"
|
||||
description: "Retrieve a list of API keys for the tenant, with filtering support."
|
||||
---
|
||||
|
||||
Retrieve a list of all API keys associated with your tenant. This endpoint supports various filtering options to help you find specific keys.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Filtering
|
||||
|
||||
- `filter[name]` - Filter by exact API key name
|
||||
- `filter[name__icontains]` - Filter by API key name (case-insensitive)
|
||||
- `filter[prefix]` - Filter by API key prefix
|
||||
- `filter[revoked]` - Filter by revocation status (boolean)
|
||||
- `filter[expires_at]`, `filter[expires_at__gte]`, `filter[expires_at__lte]` - Filter by expiration date
|
||||
- `filter[inserted_at]`, `filter[inserted_at__gte]`, `filter[inserted_at__lte]` - Filter by creation date
|
||||
- `filter[search]` - General search term
|
||||
|
||||
### Pagination
|
||||
|
||||
- `page[number]` - Page number to retrieve
|
||||
- `page[size]` - Number of results per page
|
||||
|
||||
### Sorting
|
||||
|
||||
- `sort` - Field to sort by. Available options: `name`, `-name`, `prefix`, `-prefix`, `revoked`, `-revoked`, `inserted_at`, `-inserted_at`, `expires_at`, `-expires_at`
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[api-keys]` - Specify which fields to return: `name`, `prefix`, `expires_at`, `revoked`, `inserted_at`, `last_used_at`, `entity`
|
||||
|
||||
### Include Related Resources
|
||||
|
||||
- `include` - Include related resources. Available: `entity`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/api-keys?page[size]=10&sort=-inserted_at" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns a paginated list of API keys with their metadata:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"type": "api-keys",
|
||||
"id": "api-key-uuid-1",
|
||||
"attributes": {
|
||||
"name": "Production CI/CD Pipeline",
|
||||
"prefix": "pk_live_abc123",
|
||||
"expires_at": "2025-12-31T23:59:59Z",
|
||||
"revoked": false,
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"last_used_at": "2024-01-20T14:22:00Z"
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"type": "api-keys",
|
||||
"id": "api-key-uuid-2",
|
||||
"attributes": {
|
||||
"name": "Staging Dashboard API Access",
|
||||
"prefix": "pk_live_def456",
|
||||
"expires_at": null,
|
||||
"revoked": false,
|
||||
"inserted_at": "2024-01-10T08:15:00Z",
|
||||
"last_used_at": null
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"meta": {
|
||||
"page": {
|
||||
"number": 1,
|
||||
"size": 20,
|
||||
"total": 2,
|
||||
"total_pages": 1
|
||||
}
|
||||
},
|
||||
"links": {
|
||||
"self": "https://api.prowler.com/api/v1/api-keys?page[number]=1",
|
||||
"first": "https://api.prowler.com/api/v1/api-keys?page[number]=1",
|
||||
"last": "https://api.prowler.com/api/v1/api-keys?page[number]=1"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `id` (UUID) - Unique identifier for the API key record
|
||||
- `name` (string) - Descriptive name (3-100 characters)
|
||||
- `prefix` (string, read-only) - First characters of the key for identification
|
||||
- `expires_at` (datetime, nullable) - Expiration date in ISO 8601 format
|
||||
- `revoked` (boolean, read-only) - Whether key has been revoked
|
||||
- `inserted_at` (datetime, read-only) - When key was created
|
||||
- `last_used_at` (datetime, nullable, read-only) - Last usage timestamp
|
||||
|
||||
<Note>
|
||||
The full API key value is never returned after creation. Only the prefix is shown for identification purposes.
|
||||
</Note>
|
||||
@@ -0,0 +1,89 @@
|
||||
---
|
||||
title: "Retrieve API Key"
|
||||
api: "GET /api/v1/api-keys/{id}"
|
||||
description: "Fetch detailed information about a specific API key by its ID."
|
||||
---
|
||||
|
||||
Retrieve detailed information about a specific API key. Note that the full API key value is never returned, only metadata about the key.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `id` (required) - The UUID of the API key to retrieve
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[api-keys]` - Specify which fields to return: `name`, `prefix`, `expires_at`, `revoked`, `inserted_at`, `last_used_at`, `entity`
|
||||
|
||||
### Include Related Resources
|
||||
|
||||
- `include` - Include related resources. Available: `entity`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/api-keys/{id}" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns detailed metadata about the API key:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "api-key-uuid",
|
||||
"attributes": {
|
||||
"name": "Production CI/CD Pipeline",
|
||||
"prefix": "pk_live_abc123",
|
||||
"expires_at": "2025-12-31T23:59:59Z",
|
||||
"revoked": false,
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"last_used_at": "2024-01-20T14:22:00Z"
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `id` (UUID) - Unique identifier for the API key record
|
||||
- `name` (string) - Descriptive name (3-100 characters)
|
||||
- `prefix` (string, read-only) - First characters of the key for identification
|
||||
- `expires_at` (datetime, nullable) - Expiration date in ISO 8601 format
|
||||
- `revoked` (boolean, read-only) - Whether key has been revoked
|
||||
- `inserted_at` (datetime, read-only) - When key was created
|
||||
- `last_used_at` (datetime, nullable, read-only) - Last usage timestamp
|
||||
|
||||
### Error Responses
|
||||
|
||||
**404 Not Found** - API key does not exist
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "404",
|
||||
"title": "Not Found",
|
||||
"detail": "API key not found"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
The full API key value is never returned after creation. Only the prefix is shown for identification purposes.
|
||||
</Note>
|
||||
@@ -0,0 +1,86 @@
|
||||
---
|
||||
title: "Revoke API Key"
|
||||
api: "DELETE /api/v1/api-keys/{id}/revoke"
|
||||
description: "Revoke an API key by its ID. This action is irreversible."
|
||||
---
|
||||
|
||||
Revoke an API key, preventing it from being used for further API requests. This action cannot be undone.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `id` (required) - The UUID of the API key to revoke
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X DELETE "https://api.prowler.com/api/v1/api-keys/{id}/revoke" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns the API key with `revoked` status set to `true`:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "api-key-uuid",
|
||||
"attributes": {
|
||||
"name": "Production CI/CD Pipeline",
|
||||
"prefix": "pk_live_abc123",
|
||||
"expires_at": "2025-12-31T23:59:59Z",
|
||||
"revoked": true,
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"last_used_at": "2024-01-20T14:22:00Z"
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Error Responses
|
||||
|
||||
**404 Not Found** - API key does not exist
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "404",
|
||||
"title": "Not Found",
|
||||
"detail": "API key not found"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**409 Conflict** - API key already revoked
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "409",
|
||||
"title": "Already Revoked",
|
||||
"detail": "This API key has already been revoked"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Revoking an API key is permanent and cannot be undone. Any applications using this key will immediately lose access. The `revoked` field will be set to `true` and the key will no longer authenticate API requests.
|
||||
</Warning>
|
||||
|
||||
<Tip>
|
||||
Monitor the `last_used_at` field before revoking to ensure the key is no longer in active use.
|
||||
</Tip>
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
title: "Update API Key"
|
||||
api: "PATCH /api/v1/api-keys/{id}"
|
||||
description: "Modify certain fields of an existing API key without affecting other settings."
|
||||
---
|
||||
|
||||
Partially update an API key's properties. This endpoint allows you to modify the name or expiration date without affecting other settings.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `id` (required) - The UUID of the API key to update
|
||||
|
||||
## Request Body
|
||||
|
||||
The request must follow the JSON:API specification format:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "string",
|
||||
"attributes": {
|
||||
"name": "string",
|
||||
"expires_at": "2024-12-31T23:59:59Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Attributes
|
||||
|
||||
- `name` (optional, string) - Update the API key name (3-100 characters)
|
||||
|
||||
<Note>
|
||||
Only the `name` field can be updated. Other fields like `expires_at`, `prefix`, and `revoked` are read-only and cannot be modified after creation.
|
||||
</Note>
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X PATCH "https://api.prowler.com/api/v1/api-keys/{id}" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
|
||||
"attributes": {
|
||||
"name": "Updated API Key Name"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns the updated API key with modified metadata:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "api-keys",
|
||||
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
|
||||
"attributes": {
|
||||
"name": "Updated API Key Name",
|
||||
"prefix": "pk_live_abc123",
|
||||
"expires_at": "2025-12-31T23:59:59Z",
|
||||
"revoked": false,
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"last_used_at": "2024-01-20T14:22:00Z"
|
||||
},
|
||||
"relationships": {
|
||||
"entity": {
|
||||
"data": {
|
||||
"type": "users",
|
||||
"id": "user-uuid"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Warning>
|
||||
The full API key value is never returned in responses after creation. Only the prefix is shown for identification.
|
||||
</Warning>
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Retrieve Compliance Attributes"
|
||||
api: "GET /api/v1/compliance-overviews/attributes"
|
||||
description: "Get compliance framework attributes."
|
||||
---
|
||||
|
||||
Retrieve detailed attributes for compliance frameworks.
|
||||
@@ -0,0 +1,17 @@
|
||||
---
|
||||
title: "List Compliance Overviews"
|
||||
api: "GET /api/v1/compliance-overviews"
|
||||
description: "Retrieve compliance framework coverage for scans."
|
||||
---
|
||||
|
||||
Get compliance status across frameworks like CIS, PCI-DSS, HIPAA, etc.
|
||||
|
||||
## Query Parameters
|
||||
- `filter[scan]` - Filter by scan UUID (required)
|
||||
- `filter[framework]` - Filter by framework name
|
||||
|
||||
## Example
|
||||
\`\`\`bash
|
||||
curl "https://api.prowler.com/api/v1/compliance-overviews?filter[scan]=scan-uuid" \\
|
||||
-H "Authorization: Bearer YOUR_API_KEY"
|
||||
\`\`\`
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Retrieve Compliance Metadata"
|
||||
api: "GET /api/v1/compliance-overviews/metadata"
|
||||
description: "Get available compliance frameworks and metadata."
|
||||
---
|
||||
|
||||
Fetch list of available compliance frameworks and their details.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Retrieve Compliance Requirements"
|
||||
api: "GET /api/v1/compliance-overviews/requirements"
|
||||
description: "Get compliance requirements details."
|
||||
---
|
||||
|
||||
Fetch specific requirements for compliance frameworks.
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
title: "List Latest Findings"
|
||||
api: "GET /api/v1/findings/latest"
|
||||
description: "Retrieve the latest findings from the most recent scans for each provider."
|
||||
---
|
||||
|
||||
Retrieve the most recent findings from the latest scan for each provider in your account. This endpoint automatically filters to show only the latest scan results, making it ideal for dashboards and monitoring.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
The endpoint supports the same filtering options as the regular findings list, except for the `inserted_at` filter (which is not required for this endpoint).
|
||||
|
||||
### Common Filters
|
||||
|
||||
**By Status:**
|
||||
- `filter[status]`, `filter[status__in]` - Filter by status
|
||||
- `filter[severity]`, `filter[severity__in]` - Filter by severity
|
||||
- `filter[delta]`, `filter[delta__in]` - Filter by change status
|
||||
- `filter[muted]` - Filter by mute status
|
||||
|
||||
**By Provider:**
|
||||
- `filter[provider]`, `filter[provider__in]` - Filter by provider
|
||||
- `filter[provider_type]`, `filter[provider_type__in]` - Filter by provider type
|
||||
- `filter[provider_alias]`, `filter[provider_alias__icontains]` - Filter by provider alias
|
||||
|
||||
**By Resource:**
|
||||
- `filter[resource_name]`, `filter[resource_name__icontains]` - Filter by resource name
|
||||
- `filter[resource_type]`, `filter[resource_type__icontains]` - Filter by resource type
|
||||
- `filter[region]`, `filter[region__in]` - Filter by region
|
||||
- `filter[service]`, `filter[service__icontains]` - Filter by service
|
||||
|
||||
**By Check:**
|
||||
- `filter[check_id]`, `filter[check_id__icontains]` - Filter by check ID
|
||||
|
||||
### Pagination & Sorting
|
||||
|
||||
- `page[number]`, `page[size]` - Pagination options
|
||||
- `sort` - Sort by field
|
||||
- `fields[findings]` - Select specific fields
|
||||
- `include` - Include related resources
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings/latest?filter[severity]=critical&filter[status]=FAIL&filter[provider_type]=aws" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns findings from the most recent scan for each provider:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"type": "findings",
|
||||
"id": "finding-uuid",
|
||||
"attributes": {
|
||||
"uid": "prowler-aws-iam-password-policy-weak",
|
||||
"check_id": "iam_password_policy_uppercase",
|
||||
"status": "FAIL",
|
||||
"status_extended": "IAM password policy does not require uppercase letters",
|
||||
"severity": "medium",
|
||||
"muted": false,
|
||||
"muted_reason": null,
|
||||
"delta": "new",
|
||||
"inserted_at": "2024-01-20T10:30:00Z",
|
||||
"updated_at": "2024-01-20T10:30:00Z",
|
||||
"first_seen_at": "2024-01-20T10:30:00Z",
|
||||
"check_metadata": {},
|
||||
"raw_result": {}
|
||||
},
|
||||
"relationships": {
|
||||
"scan": {
|
||||
"data": {
|
||||
"type": "scans",
|
||||
"id": "scan-uuid"
|
||||
}
|
||||
},
|
||||
"resources": {
|
||||
"data": []
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"meta": {
|
||||
"page": {
|
||||
"number": 1,
|
||||
"size": 20,
|
||||
"total": 45,
|
||||
"total_pages": 3
|
||||
}
|
||||
},
|
||||
"links": {
|
||||
"self": "https://api.prowler.com/api/v1/findings/latest?page[number]=1",
|
||||
"first": "https://api.prowler.com/api/v1/findings/latest?page[number]=1",
|
||||
"next": "https://api.prowler.com/api/v1/findings/latest?page[number]=2",
|
||||
"last": "https://api.prowler.com/api/v1/findings/latest?page[number]=3"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Tip>
|
||||
Use this endpoint for real-time monitoring and dashboards, as it automatically focuses on the latest scan results without requiring date filters.
|
||||
</Tip>
|
||||
@@ -0,0 +1,287 @@
|
||||
---
|
||||
title: "List Findings"
|
||||
api: "GET /api/v1/findings"
|
||||
description: "Retrieve a list of all findings with options for filtering by various criteria."
|
||||
---
|
||||
|
||||
Retrieve security findings (vulnerabilities, misconfigurations, compliance violations) across your cloud infrastructure. This is one of the most frequently used endpoints for analyzing your security posture.
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Security Dashboard**: Display current security findings and trends
|
||||
- **Compliance Reporting**: Filter findings by compliance framework
|
||||
- **Vulnerability Management**: Track and remediate security issues
|
||||
- **Alerting & Notifications**: Query new or critical findings for alerts
|
||||
- **Risk Assessment**: Analyze findings by severity and service
|
||||
- **Audit Trails**: Export findings for compliance audits
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Required Filters
|
||||
|
||||
- `filter[inserted_at]` - At least one variation of this filter is required
|
||||
- `filter[inserted_at__gte]` - Filter by creation date (greater than or equal). Maximum date range is 7 days
|
||||
- `filter[inserted_at__lte]` - Filter by creation date (less than or equal). Maximum date range is 7 days
|
||||
|
||||
### Filtering Options
|
||||
|
||||
**By Check:**
|
||||
- `filter[check_id]`, `filter[check_id__icontains]`, `filter[check_id__in]` - Filter by check ID
|
||||
|
||||
**By Status:**
|
||||
- `filter[status]` - Filter by status: `FAIL`, `PASS`, `MANUAL`
|
||||
- `filter[severity]` - Filter by severity: `critical`, `high`, `medium`, `low`, `informational`
|
||||
- `filter[delta]` - Filter by change status: `new`, `changed`
|
||||
- `filter[muted]` - Filter by mute status (boolean)
|
||||
|
||||
**By Provider:**
|
||||
- `filter[provider]`, `filter[provider__in]` - Filter by provider UUID
|
||||
- `filter[provider_type]`, `filter[provider_type__in]` - Filter by provider type: `aws`, `azure`, `gcp`, `kubernetes`, `m365`, `github`, `oci`
|
||||
- `filter[provider_alias]`, `filter[provider_alias__icontains]` - Filter by provider alias
|
||||
- `filter[provider_uid]`, `filter[provider_uid__icontains]` - Filter by provider UID
|
||||
|
||||
**By Resource:**
|
||||
- `filter[resource_name]`, `filter[resource_name__icontains]` - Filter by resource name
|
||||
- `filter[resource_type]`, `filter[resource_type__icontains]` - Filter by resource type
|
||||
- `filter[resource_uid]`, `filter[resource_uid__icontains]` - Filter by resource UID
|
||||
- `filter[region]`, `filter[region__icontains]`, `filter[region__in]` - Filter by region
|
||||
- `filter[service]`, `filter[service__icontains]` - Filter by cloud service
|
||||
|
||||
**By Scan:**
|
||||
- `filter[scan]`, `filter[scan__in]` - Filter by scan UUID
|
||||
|
||||
### Pagination
|
||||
|
||||
- `page[number]` - Page number to retrieve
|
||||
- `page[size]` - Number of results per page
|
||||
|
||||
### Sorting
|
||||
|
||||
- `sort` - Field to sort by: `status`, `-status`, `severity`, `-severity`, `check_id`, `-check_id`, `inserted_at`, `-inserted_at`, `updated_at`, `-updated_at`
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[findings]` - Specify which fields to return
|
||||
|
||||
### Include Related Resources
|
||||
|
||||
- `include` - Include related resources: `scan`, `resources`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings?filter[inserted_at__gte]=2024-01-01&filter[status]=FAIL&filter[severity]=high&page[size]=20" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns a paginated list of findings with full details:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": [
|
||||
{
|
||||
"type": "findings",
|
||||
"id": "finding-uuid",
|
||||
"attributes": {
|
||||
"uid": "prowler-aws-s3-bucket-unencrypted-123456",
|
||||
"check_id": "s3_bucket_default_encryption",
|
||||
"status": "FAIL",
|
||||
"status_extended": "S3 Bucket 'my-bucket' does not have default encryption enabled",
|
||||
"severity": "high",
|
||||
"muted": false,
|
||||
"muted_reason": null,
|
||||
"delta": "new",
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"updated_at": "2024-01-15T10:30:00Z",
|
||||
"first_seen_at": "2024-01-15T10:30:00Z",
|
||||
"check_metadata": {
|
||||
"Provider": "aws",
|
||||
"CheckID": "s3_bucket_default_encryption",
|
||||
"CheckTitle": "Check if S3 buckets have default encryption enabled",
|
||||
"CheckType": "Software and Configuration Checks",
|
||||
"ServiceName": "s3",
|
||||
"SubServiceName": "",
|
||||
"ResourceIdTemplate": "arn:partition:s3:::bucket_name",
|
||||
"Severity": "high",
|
||||
"ResourceType": "AwsS3Bucket",
|
||||
"Description": "S3 buckets should have encryption at rest enabled...",
|
||||
"Risk": "Unencrypted S3 buckets can expose sensitive data...",
|
||||
"RelatedUrl": "https://docs.aws.amazon.com/...",
|
||||
"Remediation": {
|
||||
"Code": {
|
||||
"CLI": "aws s3api put-bucket-encryption...",
|
||||
"NativeIaC": "...",
|
||||
"Other": "...",
|
||||
"Terraform": "..."
|
||||
},
|
||||
"Recommendation": {
|
||||
"Text": "Enable default encryption on S3 bucket",
|
||||
"Url": "https://docs.aws.amazon.com/..."
|
||||
}
|
||||
},
|
||||
"Categories": ["encryption"],
|
||||
"DependsOn": [],
|
||||
"RelatedTo": [],
|
||||
"Notes": ""
|
||||
},
|
||||
"raw_result": {
|
||||
"finding_unique_id": "...",
|
||||
"finding_uid": "...",
|
||||
"status": "FAIL",
|
||||
"status_extended": "...",
|
||||
"raw": {...}
|
||||
}
|
||||
},
|
||||
"relationships": {
|
||||
"scan": {
|
||||
"data": {
|
||||
"type": "scans",
|
||||
"id": "scan-uuid"
|
||||
}
|
||||
},
|
||||
"resources": {
|
||||
"data": [
|
||||
{
|
||||
"type": "resources",
|
||||
"id": "resource-uuid"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
],
|
||||
"meta": {
|
||||
"page": {
|
||||
"number": 1,
|
||||
"size": 20,
|
||||
"total": 156,
|
||||
"total_pages": 8
|
||||
}
|
||||
},
|
||||
"links": {
|
||||
"self": "https://api.prowler.com/api/v1/findings?page[number]=1&page[size]=20",
|
||||
"first": "https://api.prowler.com/api/v1/findings?page[number]=1&page[size]=20",
|
||||
"next": "https://api.prowler.com/api/v1/findings?page[number]=2&page[size]=20",
|
||||
"last": "https://api.prowler.com/api/v1/findings?page[number]=8&page[size]=20"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
**Core Finding Information:**
|
||||
- `uid` (string, required) - Unique identifier for the finding (max 300 characters)
|
||||
- `check_id` (string, required) - ID of the security check that generated this finding (max 100 characters)
|
||||
- `status` (enum, required) - Finding status: `FAIL`, `PASS`, or `MANUAL`
|
||||
- `status_extended` (string, nullable) - Detailed human-readable status message
|
||||
- `severity` (enum, required) - Severity level: `critical`, `high`, `medium`, `low`, `informational`
|
||||
- `delta` (enum, nullable) - Change status: `new`, `changed`, or `null`
|
||||
|
||||
**Mute Management:**
|
||||
- `muted` (boolean) - Whether finding is muted (suppressed)
|
||||
- `muted_reason` (string, nullable) - Reason for muting (3-500 characters)
|
||||
|
||||
**Timestamps:**
|
||||
- `inserted_at` (datetime, read-only) - When finding was first created
|
||||
- `updated_at` (datetime, read-only) - Last time finding was modified
|
||||
- `first_seen_at` (datetime, nullable, read-only) - First time this finding was detected
|
||||
|
||||
**Check Details:**
|
||||
- `check_metadata` (object) - Complete metadata about the security check including:
|
||||
- Title, description, risk assessment
|
||||
- Remediation steps and code examples
|
||||
- Related documentation URLs
|
||||
- Compliance frameworks and categories
|
||||
- `raw_result` (object) - Complete raw output from the security check
|
||||
|
||||
**Relationships:**
|
||||
- `scan` - Reference to the scan that generated this finding
|
||||
- `resources` - References to affected cloud resources (read-only)
|
||||
|
||||
<Note>
|
||||
Resource information (service, region, resource name, etc.) is available through the `resources` relationship. Use `include=resources` to fetch resource details along with findings.
|
||||
</Note>
|
||||
|
||||
### Common Query Patterns
|
||||
|
||||
**Get all critical findings:**
|
||||
```bash
|
||||
curl "https://api.prowler.com/api/v1/findings?filter[severity]=critical&filter[status]=FAIL&filter[inserted_at__gte]=2024-01-01"
|
||||
```
|
||||
|
||||
**Get new findings from last scan:**
|
||||
```bash
|
||||
curl "https://api.prowler.com/api/v1/findings/latest?filter[delta]=new&filter[status]=FAIL"
|
||||
```
|
||||
|
||||
**Get findings for specific service:**
|
||||
```bash
|
||||
curl "https://api.prowler.com/api/v1/findings?filter[service]=s3&filter[inserted_at__gte]=2024-01-01"
|
||||
```
|
||||
|
||||
**Get findings for specific region:**
|
||||
```bash
|
||||
curl "https://api.prowler.com/api/v1/findings?filter[region]=us-east-1&filter[inserted_at__gte]=2024-01-01"
|
||||
```
|
||||
|
||||
**Get unmuted high/critical findings:**
|
||||
```bash
|
||||
curl "https://api.prowler.com/api/v1/findings?filter[severity__in]=critical,high&filter[muted]=false&filter[inserted_at__gte]=2024-01-01"
|
||||
```
|
||||
|
||||
### Pagination
|
||||
|
||||
Use pagination parameters to navigate large result sets:
|
||||
|
||||
```bash
|
||||
# First page, 50 results
|
||||
curl "https://api.prowler.com/api/v1/findings?page[size]=50&page[number]=1&filter[inserted_at__gte]=2024-01-01"
|
||||
|
||||
# Next page
|
||||
curl "https://api.prowler.com/api/v1/findings?page[size]=50&page[number]=2&filter[inserted_at__gte]=2024-01-01"
|
||||
```
|
||||
|
||||
### Error Responses
|
||||
|
||||
**400 Bad Request** - Invalid filter or parameter
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "400",
|
||||
"title": "Invalid Filter",
|
||||
"detail": "Date range cannot exceed 7 days for inserted_at filters"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**422 Unprocessable Entity** - Missing required filter
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "422",
|
||||
"title": "Validation Error",
|
||||
"detail": "At least one variation of filter[inserted_at] is required"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
**Performance Tip:** Maximum date range for `inserted_at` filters is 7 days. For historical analysis, use multiple requests or export data to external storage.
|
||||
</Note>
|
||||
|
||||
<Warning>
|
||||
**Rate Limiting:** This endpoint is rate-limited to 100 requests per minute. For real-time monitoring, consider using webhooks or the `/findings/latest` endpoint.
|
||||
</Warning>
|
||||
|
||||
<Tip>
|
||||
Use `filter[delta]=new` to only retrieve findings discovered in the most recent scan, perfect for alerting on new security issues.
|
||||
</Tip>
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
title: "Retrieve Latest Findings Metadata"
|
||||
api: "GET /api/v1/findings/metadata/latest"
|
||||
description: "Fetch unique metadata values from the latest scans for each provider. This is useful for dynamic filtering."
|
||||
---
|
||||
|
||||
Retrieve metadata values such as services, regions, and resource types from the latest findings for each provider. This endpoint automatically filters to only include data from the most recent scans.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Filtering Options
|
||||
|
||||
**By Check:**
|
||||
- `filter[check_id]`, `filter[check_id__icontains]`, `filter[check_id__in]` - Filter by check ID
|
||||
|
||||
**By Status:**
|
||||
- `filter[status]` - Filter by status: `FAIL`, `PASS`, `MANUAL`
|
||||
- `filter[severity]` - Filter by severity: `critical`, `high`, `medium`, `low`, `informational`
|
||||
- `filter[delta]` - Filter by change status: `new`, `changed`
|
||||
- `filter[muted]` - Filter by mute status (boolean)
|
||||
|
||||
**By Provider:**
|
||||
- `filter[provider]`, `filter[provider__in]` - Filter by provider UUID
|
||||
- `filter[provider_type]`, `filter[provider_type__in]` - Filter by provider type: `aws`, `azure`, `gcp`, `kubernetes`, `m365`, `github`, `oci`
|
||||
- `filter[provider_alias]`, `filter[provider_alias__icontains]` - Filter by provider alias
|
||||
- `filter[provider_uid]`, `filter[provider_uid__icontains]` - Filter by provider UID
|
||||
|
||||
**By Resource:**
|
||||
- `filter[resource_name]`, `filter[resource_name__icontains]` - Filter by resource name
|
||||
- `filter[resource_type]`, `filter[resource_type__icontains]` - Filter by resource type
|
||||
- `filter[resource_uid]`, `filter[resource_uid__icontains]` - Filter by resource UID
|
||||
- `filter[region]`, `filter[region__icontains]`, `filter[region__in]` - Filter by region
|
||||
- `filter[service]`, `filter[service__icontains]` - Filter by cloud service
|
||||
|
||||
### Sorting
|
||||
|
||||
- `sort` - Field to sort by: `status`, `-status`, `severity`, `-severity`, `check_id`, `-check_id`, `inserted_at`, `-inserted_at`, `updated_at`, `-updated_at`
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[findings-metadata]` - Specify which fields to return: `services`, `regions`, `resource_types`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings/metadata/latest?filter[provider_type]=aws" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns unique metadata values from the latest scan findings:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "findings-metadata",
|
||||
"id": "metadata",
|
||||
"attributes": {
|
||||
"services": ["s3", "ec2", "iam", "vpc", "cloudtrail"],
|
||||
"regions": ["us-east-1", "us-west-2", "eu-west-1"],
|
||||
"resource_types": ["aws_s3_bucket", "aws_ec2_instance", "aws_iam_role"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `services` (array of strings) - List of unique cloud services from latest scans
|
||||
- `regions` (array of strings) - List of unique regions from latest scans
|
||||
- `resource_types` (array of strings) - List of unique resource types from latest scans
|
||||
|
||||
<Note>
|
||||
This endpoint is optimized for dashboard views where you only want to show data from the most recent scans.
|
||||
</Note>
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
title: "Retrieve Findings Metadata"
|
||||
api: "GET /api/v1/findings/metadata"
|
||||
description: "Fetch unique metadata values from a set of findings. This is useful for dynamic filtering."
|
||||
---
|
||||
|
||||
Retrieve metadata values such as services, regions, and resource types from your findings. This endpoint is useful for building dynamic filters in your application.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Required Filters
|
||||
|
||||
- `filter[inserted_at]` - At least one variation of this filter is required
|
||||
- `filter[inserted_at__gte]` - Filter by creation date (greater than or equal). Maximum date range is 7 days
|
||||
- `filter[inserted_at__lte]` - Filter by creation date (less than or equal). Maximum date range is 7 days
|
||||
|
||||
### Filtering Options
|
||||
|
||||
**By Check:**
|
||||
- `filter[check_id]`, `filter[check_id__icontains]`, `filter[check_id__in]` - Filter by check ID
|
||||
|
||||
**By Status:**
|
||||
- `filter[status]` - Filter by status: `FAIL`, `PASS`, `MANUAL`
|
||||
- `filter[severity]` - Filter by severity: `critical`, `high`, `medium`, `low`, `informational`
|
||||
- `filter[delta]` - Filter by change status: `new`, `changed`
|
||||
- `filter[muted]` - Filter by mute status (boolean)
|
||||
|
||||
**By Provider:**
|
||||
- `filter[provider]`, `filter[provider__in]` - Filter by provider UUID
|
||||
- `filter[provider_type]`, `filter[provider_type__in]` - Filter by provider type: `aws`, `azure`, `gcp`, `kubernetes`, `m365`, `github`, `oci`
|
||||
- `filter[provider_alias]`, `filter[provider_alias__icontains]` - Filter by provider alias
|
||||
- `filter[provider_uid]`, `filter[provider_uid__icontains]` - Filter by provider UID
|
||||
|
||||
**By Resource:**
|
||||
- `filter[resource_name]`, `filter[resource_name__icontains]` - Filter by resource name
|
||||
- `filter[resource_type]`, `filter[resource_type__icontains]` - Filter by resource type
|
||||
- `filter[resource_uid]`, `filter[resource_uid__icontains]` - Filter by resource UID
|
||||
- `filter[region]`, `filter[region__icontains]`, `filter[region__in]` - Filter by region
|
||||
- `filter[service]`, `filter[service__icontains]` - Filter by cloud service
|
||||
|
||||
**By Scan:**
|
||||
- `filter[scan]`, `filter[scan__in]` - Filter by scan UUID
|
||||
|
||||
### Sorting
|
||||
|
||||
- `sort` - Field to sort by: `status`, `-status`, `severity`, `-severity`, `check_id`, `-check_id`, `inserted_at`, `-inserted_at`, `updated_at`, `-updated_at`
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[findings-metadata]` - Specify which fields to return: `services`, `regions`, `resource_types`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings/metadata?filter[inserted_at__gte]=2024-01-01" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns unique metadata values from the filtered findings:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "findings-metadata",
|
||||
"id": "metadata",
|
||||
"attributes": {
|
||||
"services": ["s3", "ec2", "iam", "rds", "lambda", "cloudtrail"],
|
||||
"regions": ["us-east-1", "us-west-2", "eu-west-1", "ap-southeast-1"],
|
||||
"resource_types": ["aws_s3_bucket", "aws_ec2_instance", "aws_iam_user", "aws_rds_db_instance"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `services` (array of strings) - List of unique cloud services found in the findings
|
||||
- `regions` (array of strings) - List of unique regions found in the findings
|
||||
- `resource_types` (array of strings) - List of unique resource types found in the findings
|
||||
|
||||
<Note>
|
||||
This endpoint is useful for populating dropdowns and filters in your UI with actual values from your findings data. The maximum date range is 7 days.
|
||||
</Note>
|
||||
@@ -0,0 +1,133 @@
|
||||
---
|
||||
title: "Retrieve Finding"
|
||||
api: "GET /api/v1/findings/{id}"
|
||||
description: "Fetch detailed information about a specific finding by its ID."
|
||||
---
|
||||
|
||||
Retrieve complete details about a specific security finding, including check metadata, raw results, and affected resources.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `id` (required) - The UUID of the finding to retrieve
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[findings]` - Specify which fields to return: `uid`, `delta`, `status`, `status_extended`, `severity`, `check_id`, `check_metadata`, `raw_result`, `inserted_at`, `updated_at`, `first_seen_at`, `muted`, `muted_reason`, `url`, `scan`, `resources`
|
||||
|
||||
### Include Related Resources
|
||||
|
||||
- `include` - Include related resources: `scan`, `resources`
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings/{id}?include=scan,resources" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns detailed information about the finding:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "findings",
|
||||
"id": "finding-uuid",
|
||||
"attributes": {
|
||||
"uid": "prowler-aws-s3-bucket-unencrypted-123456",
|
||||
"check_id": "s3_bucket_default_encryption",
|
||||
"status": "FAIL",
|
||||
"status_extended": "S3 Bucket 'my-bucket' does not have default encryption enabled",
|
||||
"severity": "high",
|
||||
"muted": false,
|
||||
"muted_reason": null,
|
||||
"delta": "new",
|
||||
"inserted_at": "2024-01-15T10:30:00Z",
|
||||
"updated_at": "2024-01-15T10:30:00Z",
|
||||
"first_seen_at": "2024-01-15T10:30:00Z",
|
||||
"check_metadata": {
|
||||
"Provider": "aws",
|
||||
"CheckID": "s3_bucket_default_encryption",
|
||||
"CheckTitle": "Check if S3 buckets have default encryption enabled",
|
||||
"CheckType": "Software and Configuration Checks",
|
||||
"ServiceName": "s3",
|
||||
"Severity": "high",
|
||||
"ResourceType": "AwsS3Bucket",
|
||||
"Description": "S3 buckets should have encryption at rest enabled...",
|
||||
"Risk": "Unencrypted S3 buckets can expose sensitive data...",
|
||||
"RelatedUrl": "https://docs.aws.amazon.com/...",
|
||||
"Remediation": {
|
||||
"Code": {
|
||||
"CLI": "aws s3api put-bucket-encryption...",
|
||||
"Terraform": "..."
|
||||
},
|
||||
"Recommendation": {
|
||||
"Text": "Enable default encryption on S3 bucket",
|
||||
"Url": "https://docs.aws.amazon.com/..."
|
||||
}
|
||||
},
|
||||
"Categories": ["encryption"]
|
||||
},
|
||||
"raw_result": {
|
||||
"finding_unique_id": "...",
|
||||
"status": "FAIL",
|
||||
"status_extended": "...",
|
||||
"raw": {}
|
||||
}
|
||||
},
|
||||
"relationships": {
|
||||
"scan": {
|
||||
"data": {
|
||||
"type": "scans",
|
||||
"id": "scan-uuid"
|
||||
}
|
||||
},
|
||||
"resources": {
|
||||
"data": [
|
||||
{
|
||||
"type": "resources",
|
||||
"id": "resource-uuid"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Response Fields
|
||||
|
||||
- `uid` (string, required) - Unique identifier for the finding (max 300 characters)
|
||||
- `check_id` (string, required) - ID of the security check (max 100 characters)
|
||||
- `status` (enum, required) - Finding status: `FAIL`, `PASS`, or `MANUAL`
|
||||
- `status_extended` (string, nullable) - Detailed status message
|
||||
- `severity` (enum, required) - Severity: `critical`, `high`, `medium`, `low`, `informational`
|
||||
- `delta` (enum, nullable) - Change status: `new`, `changed`, or `null`
|
||||
- `muted` (boolean) - Whether finding is muted
|
||||
- `muted_reason` (string, nullable) - Reason for muting (3-500 characters)
|
||||
- `inserted_at` (datetime, read-only) - Creation timestamp
|
||||
- `updated_at` (datetime, read-only) - Last modification timestamp
|
||||
- `first_seen_at` (datetime, nullable, read-only) - First detection timestamp
|
||||
- `check_metadata` (object) - Complete check metadata
|
||||
- `raw_result` (object) - Raw scan output
|
||||
|
||||
### Error Responses
|
||||
|
||||
**404 Not Found** - Finding does not exist
|
||||
```json
|
||||
{
|
||||
"errors": [
|
||||
{
|
||||
"status": "404",
|
||||
"title": "Not Found",
|
||||
"detail": "Finding not found"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,52 @@
|
||||
---
|
||||
title: "Get Services and Regions"
|
||||
api: "GET /api/v1/findings/findings_services_regions"
|
||||
description: "Fetch services and regions affected in findings."
|
||||
---
|
||||
|
||||
<Warning>
|
||||
This endpoint is deprecated. Please use alternative filtering methods for retrieving services and regions information.
|
||||
</Warning>
|
||||
|
||||
Retrieve a list of unique services and regions that have findings matching the specified criteria. This is useful for populating dynamic filters in user interfaces.
|
||||
|
||||
## Query Parameters
|
||||
|
||||
### Field Selection
|
||||
|
||||
- `fields[finding-dynamic-filters]` - Specify which fields to return: `services`, `regions`
|
||||
|
||||
### Filtering
|
||||
|
||||
The endpoint supports the same filtering options as the regular findings list endpoint to scope the services and regions returned.
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/findings/findings_services_regions?filter[provider_type]=aws&filter[status]=FAIL" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
### Success Response (200 OK)
|
||||
|
||||
Returns unique services and regions from findings:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "finding-dynamic-filters",
|
||||
"id": "filters",
|
||||
"attributes": {
|
||||
"services": ["s3", "ec2", "iam"],
|
||||
"regions": ["us-east-1", "us-west-2"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Warning>
|
||||
This endpoint is deprecated. Use `/api/v1/findings/metadata` or `/api/v1/findings/metadata/latest` instead.
|
||||
</Warning>
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
title: "Check Integration Connection"
|
||||
api: "POST /api/v1/integrations/{id}/connection"
|
||||
description: "Try to verify integration connection."
|
||||
---
|
||||
|
||||
Test the connection to an integration to ensure it is configured correctly and can communicate with the external service.
|
||||
|
||||
## Path Parameters
|
||||
|
||||
- `id` (required) - UUID of the integration to test
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.prowler.com/api/v1/integrations/{id}/connection" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
Returns a task object. The connection test is performed asynchronously.
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "tasks",
|
||||
"id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
|
||||
"attributes": {
|
||||
"inserted_at": "2019-08-24T14:15:22Z",
|
||||
"completed_at": "2019-08-24T14:15:22Z",
|
||||
"name": "string",
|
||||
"state": "available",
|
||||
"result": null,
|
||||
"task_args": null,
|
||||
"metadata": null
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
Use the returned task ID to check the connection test status via the Tasks API.
|
||||
</Note>
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Create Integration"
|
||||
api: "POST /api/v1/integrations"
|
||||
description: "Configure a new integration."
|
||||
---
|
||||
|
||||
Add integration with external systems.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Delete Integration"
|
||||
api: "DELETE /api/v1/integrations/{id}"
|
||||
description: "Remove an integration."
|
||||
---
|
||||
|
||||
Delete integration configuration.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Create Jira Dispatch"
|
||||
api: "POST /api/v1/integrations/{id}/jira/dispatches"
|
||||
description: "Send findings to Jira."
|
||||
---
|
||||
|
||||
Dispatch findings to Jira for tracking.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "List Integrations"
|
||||
api: "GET /api/v1/integrations"
|
||||
description: "Retrieve configured integrations."
|
||||
---
|
||||
|
||||
Get all configured integrations (S3, Security Hub, Jira, etc).
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Retrieve Integration"
|
||||
api: "GET /api/v1/integrations/{id}"
|
||||
description: "Get integration details."
|
||||
---
|
||||
|
||||
Fetch specific integration configuration.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Update Integration"
|
||||
api: "PATCH /api/v1/integrations/{id}"
|
||||
description: "Update integration settings."
|
||||
---
|
||||
|
||||
Modify integration configuration.
|
||||
@@ -0,0 +1,121 @@
|
||||
---
|
||||
title: "API Reference"
|
||||
description: "Comprehensive API documentation for Prowler Cloud"
|
||||
---
|
||||
|
||||
## Welcome to the Prowler API
|
||||
|
||||
The Prowler API provides programmatic access to Prowler Cloud's security compliance scanning and reporting capabilities. This API follows RESTful principles and uses JSON:API specification for request and response formatting.
|
||||
|
||||
### Base URL
|
||||
|
||||
```
|
||||
https://api.prowler.com/api/v1
|
||||
```
|
||||
|
||||
### Authentication
|
||||
|
||||
All API requests require authentication using either:
|
||||
|
||||
- **JWT Token**: Obtained through user authentication
|
||||
- **API Key**: Generated from your Prowler Cloud account
|
||||
|
||||
Include your authentication credentials in the `Authorization` header:
|
||||
|
||||
```bash
|
||||
Authorization: Bearer YOUR_API_KEY_OR_JWT_TOKEN
|
||||
```
|
||||
|
||||
### API Keys Management
|
||||
|
||||
You can manage your API keys through the [API Keys endpoints](/api-reference/api-keys/list). API keys provide:
|
||||
|
||||
- Programmatic access to your Prowler Cloud resources
|
||||
- Fine-grained access control
|
||||
- Expiration and revocation capabilities
|
||||
- Usage tracking
|
||||
|
||||
### Request Format
|
||||
|
||||
All requests should include the following headers:
|
||||
|
||||
```
|
||||
Content-Type: application/vnd.api+json
|
||||
Accept: application/vnd.api+json
|
||||
Authorization: Bearer YOUR_API_KEY_OR_JWT_TOKEN
|
||||
```
|
||||
|
||||
### Response Format
|
||||
|
||||
The API follows the [JSON:API specification](https://jsonapi.org/). Responses include:
|
||||
|
||||
- **data**: The primary data for the response
|
||||
- **included**: Related resources (when using `include` parameter)
|
||||
- **meta**: Metadata about the response
|
||||
- **links**: Pagination links
|
||||
|
||||
### Pagination
|
||||
|
||||
List endpoints support pagination using the following query parameters:
|
||||
|
||||
- `page[number]`: The page number to retrieve
|
||||
- `page[size]`: Number of results per page
|
||||
|
||||
### Filtering
|
||||
|
||||
Most list endpoints support filtering using query parameters in the format:
|
||||
|
||||
```
|
||||
filter[field_name]=value
|
||||
filter[field_name__operator]=value
|
||||
```
|
||||
|
||||
Common operators:
|
||||
- `__gte`: Greater than or equal
|
||||
- `__lte`: Less than or equal
|
||||
- `__icontains`: Case-insensitive contains
|
||||
- `__in`: In a list of values
|
||||
|
||||
### Sorting
|
||||
|
||||
Use the `sort` parameter to order results:
|
||||
|
||||
```
|
||||
sort=field_name # Ascending order
|
||||
sort=-field_name # Descending order
|
||||
```
|
||||
|
||||
### Field Selection
|
||||
|
||||
Optimize responses by requesting only specific fields:
|
||||
|
||||
```
|
||||
fields[resource-type]=field1,field2,field3
|
||||
```
|
||||
|
||||
### Rate Limits
|
||||
|
||||
API requests are subject to rate limiting to ensure service stability. Rate limit information is included in response headers.
|
||||
|
||||
### Support
|
||||
|
||||
For questions or issues with the API:
|
||||
|
||||
- Join our [Slack community](https://goto.prowler.com/slack)
|
||||
- Visit [Prowler Cloud](https://cloud.prowler.com)
|
||||
- Check our [GitHub repository](https://github.com/prowler-cloud/prowler)
|
||||
|
||||
## API Endpoints
|
||||
|
||||
Explore the complete API reference using the navigation menu. Main endpoint categories include:
|
||||
|
||||
- **API Keys**: Manage programmatic access credentials
|
||||
- **Findings**: Security findings and vulnerabilities
|
||||
- **Compliance**: Compliance frameworks and requirements
|
||||
- **Scans**: Security scan operations and results
|
||||
- **Providers**: Cloud provider configurations
|
||||
- **Resources**: Cloud resources inventory
|
||||
|
||||
<Note>
|
||||
The Prowler API is currently at version 1.15.0. We maintain backward compatibility and announce breaking changes in advance.
|
||||
</Note>
|
||||
@@ -0,0 +1,50 @@
|
||||
---
|
||||
title: "Accept Invitation"
|
||||
api: "POST /api/v1/invitations/accept"
|
||||
description: "Accept an invitation to an existing tenant."
|
||||
---
|
||||
|
||||
Accept an invitation to join a tenant. This invitation cannot be expired and the emails must match.
|
||||
|
||||
## Request Body
|
||||
|
||||
The request must follow the JSON:API specification format:
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "invitations",
|
||||
"attributes": {
|
||||
"token": "invitation-token"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Attributes
|
||||
|
||||
- `token` (required) - The invitation token received via email
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X POST "https://api.prowler.com/api/v1/invitations/accept" \
|
||||
-H "Authorization: Bearer YOUR_JWT_TOKEN" \
|
||||
-H "Content-Type: application/vnd.api+json" \
|
||||
-d '{
|
||||
"data": {
|
||||
"type": "invitations",
|
||||
"attributes": {
|
||||
"token": "your-invitation-token"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
Returns a success response indicating you have been added to the tenant.
|
||||
|
||||
<Note>
|
||||
The email address associated with your account must match the email address to which the invitation was sent.
|
||||
</Note>
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "List Lighthouse Config (Legacy)"
|
||||
api: "GET /api/v1/lighthouse/configuration"
|
||||
description: "Legacy configuration endpoint."
|
||||
---
|
||||
|
||||
Legacy endpoint for configuration.
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
title: "Update Lighthouse Config (Legacy)"
|
||||
api: "PATCH /api/v1/lighthouse/configuration"
|
||||
description: "Update legacy configuration."
|
||||
---
|
||||
|
||||
Legacy endpoint for updating configuration.
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
title: "Get Lighthouse AI Tenant Config"
|
||||
api: "GET /api/v1/lighthouse/configuration"
|
||||
description: "Retrieve current tenant-level Lighthouse AI settings."
|
||||
---
|
||||
|
||||
Get your tenant's Lighthouse AI configuration including business context and default models.
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X GET "https://api.prowler.com/api/v1/lighthouse/configuration" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
Returns tenant Lighthouse AI configuration.
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
title: "Update Lighthouse AI Tenant Config"
|
||||
api: "PATCH /api/v1/lighthouse/configuration"
|
||||
description: "Update tenant-level Lighthouse AI settings."
|
||||
---
|
||||
|
||||
Update your tenant's Lighthouse AI configuration including business context and default providers/models.
|
||||
|
||||
## Request Body
|
||||
|
||||
```json
|
||||
{
|
||||
"data": {
|
||||
"type": "lighthouse-config",
|
||||
"attributes": {
|
||||
"business_context": "Financial services company focusing on security and compliance",
|
||||
"default_provider": "openai",
|
||||
"default_models": {
|
||||
"chat": "gpt-4",
|
||||
"analysis": "gpt-4"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Example Request
|
||||
|
||||
```bash
|
||||
curl -X PATCH "https://api.prowler.com/api/v1/lighthouse/configuration" \
|
||||
-H "Authorization: Bearer YOUR_API_KEY" \
|
||||
-H "Content-Type: application/vnd.api+json"
|
||||
```
|
||||
|
||||
## Response
|
||||
|
||||
Returns updated configuration.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user