merge master into grouped Jira UI
@@ -72,8 +72,8 @@ NEO4J_APOC_IMPORT_FILE_ENABLED=false
|
||||
NEO4J_APOC_IMPORT_FILE_USE_NEO4J_CONFIG=true
|
||||
NEO4J_APOC_TRIGGER_ENABLED=false
|
||||
NEO4J_DBMS_CONNECTOR_BOLT_LISTEN_ADDRESS=0.0.0.0:7687
|
||||
# Neo4j Prowler settings
|
||||
ATTACK_PATHS_BATCH_SIZE=1000
|
||||
# Attack Paths graph settings
|
||||
ATTACK_PATHS_GRAPH_MUTATION_BATCH_SIZE=1000
|
||||
ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES=3
|
||||
ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS=30
|
||||
ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES=250
|
||||
@@ -158,7 +158,7 @@ SENTRY_RELEASE=local
|
||||
# REO_DEV_CLIENT_ID=
|
||||
|
||||
#### Prowler release version ####
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.35.0
|
||||
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.36.0
|
||||
|
||||
# Social login credentials
|
||||
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
|
||||
|
||||
@@ -113,6 +113,15 @@ jobs:
|
||||
api/changelog.d/**
|
||||
api/AGENTS.md
|
||||
|
||||
# api-container-build-push.yml resolves the SDK pin to the branch tip
|
||||
# before building, so match it here and scan what ships. Push only: PRs
|
||||
# stay deterministic against the committed lock.
|
||||
- name: Refresh prowler SDK pin to current branch tip
|
||||
if: steps.check-changes.outputs.any_changed == 'true' && github.event_name == 'push'
|
||||
run: |
|
||||
pip install --no-cache-dir "uv==0.11.14"
|
||||
(cd api && uv lock --upgrade-package prowler)
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
if: steps.check-changes.outputs.any_changed == 'true'
|
||||
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
|
||||
|
||||
@@ -35,6 +35,20 @@ CVE-2026-13221 pkg:perl-base exp:2026-08-15
|
||||
CVE-2026-13221 pkg:perl-modules-5.36 exp:2026-08-15
|
||||
CVE-2026-13221 pkg:libperl5.36 exp:2026-08-15
|
||||
|
||||
# CVE-2026-57433 — Perl Storable signed integer overflow when deserializing a
|
||||
# crafted SX_HOOK record (retrieve_hook_common passes a wrapped negative count
|
||||
# to av_extend).
|
||||
# Packages: perl, perl-base, perl-modules-5.36, libperl5.36.
|
||||
# Why ignored: perl-base is part of Debian's "Essential: yes" set; it cannot be
|
||||
# removed without breaking dpkg. Prowler does not invoke perl at runtime and
|
||||
# never calls Storable's thaw/retrieve on attacker-controlled blobs, so the
|
||||
# vulnerable deserialization path is unreachable. Fixed upstream in
|
||||
# Storable 3.41; no Debian bookworm fix is available yet.
|
||||
CVE-2026-57433 pkg:perl exp:2026-08-15
|
||||
CVE-2026-57433 pkg:perl-base exp:2026-08-15
|
||||
CVE-2026-57433 pkg:perl-modules-5.36 exp:2026-08-15
|
||||
CVE-2026-57433 pkg:libperl5.36 exp:2026-08-15
|
||||
|
||||
# CVE-2025-7458 — SQLite integer overflow.
|
||||
# Package: libsqlite3-0.
|
||||
# Why ignored: transitive dependency of CPython's stdlib sqlite3 module. The
|
||||
|
||||
@@ -62,6 +62,7 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
|
||||
| Action | Skill |
|
||||
|--------|-------|
|
||||
| Add changelog entry for a PR or feature | `prowler-changelog` |
|
||||
| Adding ConfigRequirements guardrails to compliance requirements | `prowler-compliance` |
|
||||
| Adding DRF pagination or permissions | `django-drf` |
|
||||
| Adding a compliance output formatter (per-provider class + table dispatcher) | `prowler-compliance` |
|
||||
| Adding indexes or constraints to database tables | `django-migration-psql` |
|
||||
@@ -84,6 +85,7 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
|
||||
| Creating ViewSets, serializers, or filters in api/ | `django-drf` |
|
||||
| Creating Zod schemas | `zod-4` |
|
||||
| Creating a git commit | `prowler-commit` |
|
||||
| Creating a universal (multi-provider) compliance framework | `prowler-compliance` |
|
||||
| Creating new checks | `prowler-sdk-check` |
|
||||
| Creating new skills | `skill-creator` |
|
||||
| Creating or reviewing Django migrations | `django-migration-psql` |
|
||||
|
||||
@@ -123,12 +123,12 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
|
||||
|
||||
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/user-guide/compliance/tutorials/compliance) | [Categories](https://docs.prowler.com/user-guide/cli/tutorials/misc#categories) | Support | Interface |
|
||||
|---|---|---|---|---|---|---|
|
||||
| AWS | 615 | 86 | 47 | 19 | Official | UI, API, CLI |
|
||||
| Azure | 190 | 22 | 21 | 16 | Official | UI, API, CLI |
|
||||
| AWS | 621 | 86 | 47 | 19 | Official | UI, API, CLI |
|
||||
| Azure | 191 | 22 | 21 | 16 | Official | UI, API, CLI |
|
||||
| GCP | 109 | 20 | 19 | 12 | Official | UI, API, CLI |
|
||||
| Kubernetes | 90 | 7 | 8 | 11 | Official | UI, API, CLI |
|
||||
| Kubernetes | 92 | 7 | 8 | 11 | Official | UI, API, CLI |
|
||||
| GitHub | 24 | 3 | 2 | 5 | Official | UI, API, CLI |
|
||||
| M365 | 109 | 10 | 6 | 10 | Official | UI, API, CLI |
|
||||
| M365 | 111 | 10 | 6 | 10 | Official | UI, API, CLI |
|
||||
| OCI | 52 | 14 | 5 | 10 | Official | UI, API, CLI |
|
||||
| Alibaba Cloud | 63 | 9 | 6 | 9 | Official | UI, API, CLI |
|
||||
| Cloudflare | 29 | 3 | 2 | 5 | Official | UI, API, CLI |
|
||||
|
||||
@@ -4,6 +4,20 @@ All notable changes to the **Prowler API** are documented in this file.
|
||||
|
||||
<!-- changelog: release notes start -->
|
||||
|
||||
## [1.36.0] (Prowler v5.35.0)
|
||||
|
||||
### 🐞 Fixed
|
||||
|
||||
- `attack-paths-scan-perform` Celery tasks now use the configurable long-task time limits instead of the six-hour defaults [(#12009)](https://github.com/prowler-cloud/prowler/pull/12009)
|
||||
- Attack Paths scans handle provider deletion races cleanly, detect stale tasks after 16 hours, use backend-specific graph synchronization batches, and report exhausted Neptune write retries with the original database error [(#12019)](https://github.com/prowler-cloud/prowler/pull/12019)
|
||||
|
||||
### 🔐 Security
|
||||
|
||||
- Jira integration credentials only accept bare Atlassian site names containing letters, numbers, and hyphens [(#12012)](https://github.com/prowler-cloud/prowler/pull/12012)
|
||||
- Social account linking requires a verified matching email from both the identity provider and the existing user account without sending account connection notifications [(#12013)](https://github.com/prowler-cloud/prowler/pull/12013)
|
||||
|
||||
---
|
||||
|
||||
## [1.35.0] (Prowler v5.34.0)
|
||||
|
||||
### 🐞 Fixed
|
||||
|
||||
@@ -102,7 +102,9 @@ ENV PATH="/home/prowler/.local/bin:$PATH"
|
||||
RUN uv sync --locked --no-install-project && \
|
||||
rm -rf ~/.cache/uv
|
||||
|
||||
RUN .venv/bin/python .venv/lib/python3.12/site-packages/prowler/providers/m365/lib/powershell/m365_powershell.py
|
||||
# Invoked as a module so the base image's Python minor version is not baked
|
||||
# into a site-packages path.
|
||||
RUN .venv/bin/python -m prowler.providers.m365.lib.powershell.m365_powershell
|
||||
|
||||
USER root
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
`attack-paths-scan-perform` Celery tasks now use the configurable long-task time limits instead of the six-hour defaults
|
||||
@@ -0,0 +1 @@
|
||||
Compliance overview ingest now runs in a single transaction per scan with a configurable `COPY` batch size (`DJANGO_COMPLIANCE_COPY_BATCH_SIZE`, default 2000), reducing write pressure on the database
|
||||
@@ -0,0 +1 @@
|
||||
Integration responses no longer disclose providers outside the visibility of the role, including the resources sideloaded through `?include=providers`
|
||||
@@ -0,0 +1 @@
|
||||
Tenant-wide integrations that are not attached to any provider, such as Jira, are now visible and manageable by roles with `manage_integrations` and without unlimited visibility
|
||||
@@ -0,0 +1 @@
|
||||
Integration connection checks, Jira issue type lookups and Jira dispatches now resolve the integration through the provider visibility of the role instead of the whole tenant
|
||||
@@ -0,0 +1 @@
|
||||
Roles without unlimited visibility can no longer attach an integration to providers they cannot see, nor edit or delete an integration bound to them
|
||||
@@ -1 +0,0 @@
|
||||
Jira integration credentials only accept bare Atlassian site names containing letters, numbers, and hyphens
|
||||
@@ -0,0 +1 @@
|
||||
OCI provider secrets no longer require `region`; legacy `region` input is accepted for backwards compatibility but ignored before storing or scanning
|
||||
@@ -1 +0,0 @@
|
||||
Social account linking requires a verified matching email from both the identity provider and the existing user account without sending account connection notifications
|
||||
@@ -71,7 +71,7 @@ name = "prowler-api"
|
||||
package-mode = false
|
||||
# Needed for the SDK compatibility
|
||||
requires-python = ">=3.11,<3.13"
|
||||
version = "1.36.0"
|
||||
version = "1.37.0"
|
||||
|
||||
# Shared ruff baseline (kept in sync with mcp_server/pyproject.toml).
|
||||
# target-version tracks this project's lowest supported Python.
|
||||
|
||||
@@ -27,6 +27,7 @@ from django.conf import (
|
||||
MAX_CUSTOM_QUERY_NODES = env.int("ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES", default=250)
|
||||
|
||||
TEMP_DB_PREFIX = "db-tmp-scan-"
|
||||
DATABASE_NOT_FOUND_CODE = "Neo.ClientError.Database.DatabaseNotFound"
|
||||
|
||||
|
||||
# Exceptions
|
||||
@@ -44,6 +45,10 @@ class GraphDatabaseQueryException(Exception):
|
||||
return self.message
|
||||
|
||||
|
||||
class NeptuneWriteRetryExhaustedException(GraphDatabaseQueryException):
|
||||
pass
|
||||
|
||||
|
||||
class WriteQueryNotAllowedException(GraphDatabaseQueryException):
|
||||
pass
|
||||
|
||||
|
||||
@@ -10,6 +10,28 @@ import neo4j.exceptions
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class RetryExhaustedError(Exception):
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
retry_context: str,
|
||||
method_name: str,
|
||||
attempts: int,
|
||||
elapsed_seconds: float,
|
||||
last_error: Exception,
|
||||
) -> None:
|
||||
self.retry_context = retry_context
|
||||
self.method_name = method_name
|
||||
self.attempts = attempts
|
||||
self.elapsed_seconds = elapsed_seconds
|
||||
self.last_error = last_error
|
||||
last_message = getattr(last_error, "message", None) or str(last_error)
|
||||
super().__init__(
|
||||
f"{retry_context} {method_name} failed after {attempts} attempts over "
|
||||
f"{elapsed_seconds:.3f}s. Last error: {last_message}"
|
||||
)
|
||||
|
||||
|
||||
class RetryableSession:
|
||||
"""Wrapper around ``neo4j.Session`` with a refreshable retry policy."""
|
||||
|
||||
@@ -19,11 +41,13 @@ class RetryableSession:
|
||||
max_retries: int,
|
||||
retry_if: Callable[[Exception], bool] | None = None,
|
||||
initial_retry_delay_seconds: float = 0,
|
||||
retry_context: str | None = None,
|
||||
) -> None:
|
||||
self._session_factory = session_factory
|
||||
self._max_retries = max(0, max_retries)
|
||||
self._retry_if = retry_if
|
||||
self._initial_retry_delay_seconds = max(0.0, initial_retry_delay_seconds)
|
||||
self._retry_context = retry_context
|
||||
self._session = self._session_factory()
|
||||
|
||||
def close(self) -> None:
|
||||
@@ -54,6 +78,7 @@ class RetryableSession:
|
||||
def _call_with_retry(self, method_name: str, *args: Any, **kwargs: Any) -> Any:
|
||||
attempt = 0
|
||||
last_exc: Exception | None = None
|
||||
started_at = time.monotonic()
|
||||
|
||||
while attempt <= self._max_retries:
|
||||
try:
|
||||
@@ -68,17 +93,38 @@ class RetryableSession:
|
||||
attempt += 1
|
||||
|
||||
if attempt > self._max_retries:
|
||||
if self._retry_context is not None:
|
||||
raise RetryExhaustedError(
|
||||
retry_context=self._retry_context,
|
||||
method_name=method_name,
|
||||
attempts=attempt,
|
||||
elapsed_seconds=time.monotonic() - started_at,
|
||||
last_error=exc,
|
||||
) from exc
|
||||
raise
|
||||
|
||||
delay = self._retry_delay(attempt)
|
||||
logger.warning(
|
||||
"Graph session %s failed with %s; retry %s/%s in %.3fs",
|
||||
method_name,
|
||||
type(exc).__name__,
|
||||
attempt,
|
||||
self._max_retries,
|
||||
delay,
|
||||
)
|
||||
if self._retry_context is not None:
|
||||
error_message = getattr(exc, "message", None) or str(exc)
|
||||
logger.warning(
|
||||
"%s %s failed with %s: %s; retry %s/%s in %.3fs",
|
||||
self._retry_context,
|
||||
method_name,
|
||||
type(exc).__name__,
|
||||
error_message,
|
||||
attempt,
|
||||
self._max_retries,
|
||||
delay,
|
||||
)
|
||||
else:
|
||||
logger.warning(
|
||||
"Graph session %s failed with %s; retry %s/%s in %.3fs",
|
||||
method_name,
|
||||
type(exc).__name__,
|
||||
attempt,
|
||||
self._max_retries,
|
||||
delay,
|
||||
)
|
||||
self._refresh_session()
|
||||
if delay:
|
||||
time.sleep(delay)
|
||||
|
||||
@@ -15,6 +15,8 @@ class SinkDatabase(Protocol):
|
||||
has a single graph, and isolation is label-based).
|
||||
"""
|
||||
|
||||
sync_batch_size: int
|
||||
|
||||
def init(self) -> None: ...
|
||||
|
||||
def close(self) -> None: ...
|
||||
|
||||
@@ -54,6 +54,8 @@ DATABASE_NOT_FOUND_CODE = "Neo.ClientError.Database.DatabaseNotFound"
|
||||
class Neo4jSink(SinkDatabase):
|
||||
"""Neo4j-backed sink. Multi-database cluster; tenant isolation is physical."""
|
||||
|
||||
sync_batch_size = env.int("ATTACK_PATHS_NEO4J_SYNC_BATCH_SIZE", default=1000)
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._driver: neo4j.Driver | None = None
|
||||
self._lock = threading.Lock()
|
||||
@@ -203,7 +205,7 @@ class Neo4jSink(SinkDatabase):
|
||||
"""
|
||||
from api.attack_paths.database import GraphDatabaseQueryException
|
||||
from tasks.jobs.attack_paths.config import (
|
||||
BATCH_SIZE,
|
||||
GRAPH_MUTATION_BATCH_SIZE,
|
||||
PROVIDER_RESOURCE_LABEL,
|
||||
get_provider_label,
|
||||
)
|
||||
@@ -251,7 +253,7 @@ class Neo4jSink(SinkDatabase):
|
||||
total_key="rels",
|
||||
deleted_key="deleted_rels",
|
||||
initial_total=deleted_relationships,
|
||||
batch_size=BATCH_SIZE,
|
||||
batch_size=GRAPH_MUTATION_BATCH_SIZE,
|
||||
drop_t0=drop_t0,
|
||||
)
|
||||
relationship_batches += phase_batches
|
||||
@@ -270,7 +272,7 @@ class Neo4jSink(SinkDatabase):
|
||||
total_key="nodes",
|
||||
deleted_key="deleted_nodes",
|
||||
initial_total=0,
|
||||
batch_size=BATCH_SIZE,
|
||||
batch_size=GRAPH_MUTATION_BATCH_SIZE,
|
||||
drop_t0=drop_t0,
|
||||
)
|
||||
|
||||
|
||||
@@ -25,7 +25,7 @@ from urllib.parse import urlsplit
|
||||
|
||||
import neo4j
|
||||
import neo4j.exceptions
|
||||
from api.attack_paths.retryable_session import RetryableSession
|
||||
from api.attack_paths.retryable_session import RetryableSession, RetryExhaustedError
|
||||
from api.attack_paths.sink.base import SinkDatabase
|
||||
from api.attack_paths.sink.drop import (
|
||||
NODE_DELETE_QUERY_TEMPLATE,
|
||||
@@ -85,6 +85,8 @@ def _is_retryable_write_error(exc: Exception) -> bool:
|
||||
class NeptuneSink(SinkDatabase):
|
||||
"""Neptune-backed sink. Single database; isolation is label-based."""
|
||||
|
||||
sync_batch_size = env.int("ATTACK_PATHS_NEPTUNE_SYNC_BATCH_SIZE", default=500)
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._writer: neo4j.Driver | None = None
|
||||
self._reader: neo4j.Driver | None = None
|
||||
@@ -206,6 +208,7 @@ class NeptuneSink(SinkDatabase):
|
||||
from api.attack_paths.database import (
|
||||
ClientStatementException,
|
||||
GraphDatabaseQueryException,
|
||||
NeptuneWriteRetryExhaustedException,
|
||||
WriteQueryNotAllowedException,
|
||||
)
|
||||
|
||||
@@ -227,9 +230,17 @@ class NeptuneSink(SinkDatabase):
|
||||
initial_retry_delay_seconds=(
|
||||
NEPTUNE_WRITE_RETRY_DELAY_SECONDS if is_write_session else 0
|
||||
),
|
||||
retry_context="Neptune write" if is_write_session else None,
|
||||
)
|
||||
yield session_wrapper
|
||||
|
||||
except RetryExhaustedError as exc:
|
||||
last_error = exc.last_error
|
||||
raise NeptuneWriteRetryExhaustedException(
|
||||
message=str(exc),
|
||||
code=getattr(last_error, "code", None),
|
||||
) from last_error
|
||||
|
||||
except neo4j.exceptions.Neo4jError as exc:
|
||||
if (
|
||||
default_access_mode == neo4j.READ_ACCESS
|
||||
@@ -291,7 +302,7 @@ class NeptuneSink(SinkDatabase):
|
||||
graph's branching factor.
|
||||
"""
|
||||
from tasks.jobs.attack_paths.config import (
|
||||
BATCH_SIZE,
|
||||
GRAPH_MUTATION_BATCH_SIZE,
|
||||
PROVIDER_RESOURCE_LABEL,
|
||||
get_provider_label,
|
||||
)
|
||||
@@ -330,7 +341,7 @@ class NeptuneSink(SinkDatabase):
|
||||
total_key="rels",
|
||||
deleted_key="deleted_rels",
|
||||
initial_total=deleted_relationships,
|
||||
batch_size=BATCH_SIZE,
|
||||
batch_size=GRAPH_MUTATION_BATCH_SIZE,
|
||||
drop_t0=drop_t0,
|
||||
)
|
||||
relationship_batches += phase_batches
|
||||
@@ -349,7 +360,7 @@ class NeptuneSink(SinkDatabase):
|
||||
total_key="nodes",
|
||||
deleted_key="deleted_nodes",
|
||||
initial_total=0,
|
||||
batch_size=BATCH_SIZE,
|
||||
batch_size=GRAPH_MUTATION_BATCH_SIZE,
|
||||
drop_t0=drop_t0,
|
||||
)
|
||||
|
||||
|
||||
@@ -3,9 +3,10 @@ from api.db_router import MainRouter, reset_read_db_alias, set_read_db_alias
|
||||
from api.db_utils import POSTGRES_USER_VAR, rls_transaction
|
||||
from api.filters import CustomDjangoFilterBackend
|
||||
from api.models import Role, UserRoleRelationship
|
||||
from api.rbac.permissions import HasPermissions
|
||||
from api.rbac.permissions import HasPermissions, get_role
|
||||
from django.conf import settings
|
||||
from django.db import transaction
|
||||
from django.utils.functional import cached_property
|
||||
from rest_framework import permissions
|
||||
from rest_framework.exceptions import NotAuthenticated
|
||||
from rest_framework.filters import SearchFilter
|
||||
@@ -100,6 +101,11 @@ class BaseRLSViewSet(BaseViewSet):
|
||||
context["tenant_id"] = self.request.tenant_id
|
||||
return context
|
||||
|
||||
@cached_property
|
||||
def user_role(self):
|
||||
"""Role of the requesting user in the active tenant, resolved once per request."""
|
||||
return get_role(self.request.user, self.request.tenant_id)
|
||||
|
||||
|
||||
class BaseTenantViewset(BaseViewSet):
|
||||
def dispatch(self, request, *args, **kwargs):
|
||||
|
||||
@@ -1,12 +1,13 @@
|
||||
import uuid
|
||||
from functools import wraps
|
||||
|
||||
from api.attack_paths.database import GraphDatabaseQueryException
|
||||
from api.db_router import READ_REPLICA_ALIAS
|
||||
from api.db_utils import POSTGRES_TENANT_VAR, SET_CONFIG_QUERY, rls_transaction
|
||||
from api.exceptions import ProviderDeletedException
|
||||
from api.models import Provider, Scan
|
||||
from api.models import Membership, Provider, Scan, Tenant
|
||||
from django.core.exceptions import ObjectDoesNotExist
|
||||
from django.db import DatabaseError, connection, transaction
|
||||
from django.db import DEFAULT_DB_ALIAS, DatabaseError, connection, transaction
|
||||
from rest_framework_json_api.serializers import ValidationError
|
||||
|
||||
|
||||
@@ -75,9 +76,11 @@ def handle_provider_deletion(func):
|
||||
"""
|
||||
Decorator that raises `ProviderDeletedException` if provider was deleted during execution.
|
||||
|
||||
Catches `ObjectDoesNotExist` and `DatabaseError` (including `IntegrityError`), checks if
|
||||
provider still exists, and raises `ProviderDeletedException` if not. Otherwise,
|
||||
re-raises original exception.
|
||||
Catches `ObjectDoesNotExist`, `DatabaseError` (including `IntegrityError`), and
|
||||
`GraphDatabaseQueryException`, checks if provider still exists, and raises
|
||||
`ProviderDeletedException` if not. Graph database errors also check whether the
|
||||
tenant still exists and has memberships. Otherwise, re-raises the original
|
||||
exception.
|
||||
|
||||
Requires `tenant_id` and `provider_id` in kwargs.
|
||||
|
||||
@@ -92,11 +95,16 @@ def handle_provider_deletion(func):
|
||||
def wrapper(*args, **kwargs):
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
except (ObjectDoesNotExist, DatabaseError):
|
||||
except (ObjectDoesNotExist, DatabaseError, GraphDatabaseQueryException) as exc:
|
||||
tenant_id = kwargs.get("tenant_id")
|
||||
provider_id = kwargs.get("provider_id")
|
||||
database_alias = (
|
||||
DEFAULT_DB_ALIAS
|
||||
if isinstance(exc, GraphDatabaseQueryException)
|
||||
else READ_REPLICA_ALIAS
|
||||
)
|
||||
|
||||
with rls_transaction(tenant_id, using=READ_REPLICA_ALIAS):
|
||||
with rls_transaction(tenant_id, using=database_alias):
|
||||
if provider_id is None:
|
||||
scan_id = kwargs.get("scan_id")
|
||||
if scan_id is None:
|
||||
@@ -113,6 +121,13 @@ def handle_provider_deletion(func):
|
||||
raise ProviderDeletedException(
|
||||
f"Provider '{provider_id}' was deleted during the scan"
|
||||
) from None
|
||||
if isinstance(exc, GraphDatabaseQueryException) and (
|
||||
not Tenant.objects.filter(pk=tenant_id).exists()
|
||||
or not Membership.objects.filter(tenant_id=tenant_id).exists()
|
||||
):
|
||||
raise ProviderDeletedException(
|
||||
f"Tenant '{tenant_id}' was deleted during the scan"
|
||||
) from None
|
||||
raise
|
||||
|
||||
return wrapper
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
from enum import Enum
|
||||
|
||||
from api.db_router import MainRouter
|
||||
from api.models import Provider, Role, User
|
||||
from django.db.models import QuerySet
|
||||
from api.models import Integration, Provider, Role, User
|
||||
from django.db.models import Q, QuerySet
|
||||
from rest_framework.exceptions import PermissionDenied
|
||||
from rest_framework.permissions import BasePermission
|
||||
|
||||
@@ -83,3 +83,32 @@ def get_providers(role: Role) -> QuerySet[Provider]:
|
||||
return Provider.objects.filter(
|
||||
tenant_id=tenant_id, provider_groups__in=provider_groups
|
||||
).distinct()
|
||||
|
||||
|
||||
def get_integrations(
|
||||
role: Role, providers: QuerySet[Provider] | None = None
|
||||
) -> QuerySet[Integration]:
|
||||
"""
|
||||
Return a distinct queryset of Integrations visible to the given role.
|
||||
|
||||
Integrations with no providers attached are tenant-wide, as is always the case for
|
||||
Jira, and stay visible regardless of the provider visibility of the role. Integrations
|
||||
attached to providers are only visible when the role can access at least one of them.
|
||||
|
||||
Args:
|
||||
role: A Role instance.
|
||||
providers: Optional queryset of the providers accessible by the role, to reuse
|
||||
an already resolved `get_providers(role)` result within the same request.
|
||||
|
||||
Returns:
|
||||
A QuerySet of Integration objects visible to the role.
|
||||
"""
|
||||
queryset = Integration.objects.filter(tenant_id=role.tenant_id)
|
||||
if role.unlimited_visibility:
|
||||
return queryset
|
||||
|
||||
if providers is None:
|
||||
providers = get_providers(role)
|
||||
return queryset.filter(
|
||||
Q(providers__isnull=True) | Q(providers__in=providers)
|
||||
).distinct()
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
openapi: 3.0.3
|
||||
info:
|
||||
title: Prowler API
|
||||
version: 1.36.0
|
||||
version: 1.37.0
|
||||
description: |-
|
||||
Prowler API specification.
|
||||
|
||||
@@ -6629,8 +6629,10 @@ paths:
|
||||
/api/v1/integrations:
|
||||
get:
|
||||
operationId: api_v1_integrations_list
|
||||
description: Retrieve a list of all configured integrations with options for
|
||||
filtering by various criteria.
|
||||
description: |-
|
||||
Retrieve a list of all configured integrations with options for filtering by various criteria.
|
||||
|
||||
Integrations attached to one or more providers are only returned when the role can access at least one of those providers, and each integration lists only the providers visible to the role. Integrations not attached to any provider, such as Jira, are tenant-wide and are returned for every role.
|
||||
summary: List all integrations
|
||||
parameters:
|
||||
- in: query
|
||||
@@ -6781,7 +6783,8 @@ paths:
|
||||
post:
|
||||
operationId: api_v1_integrations_create
|
||||
description: Register a new integration with the system, providing necessary
|
||||
configuration details.
|
||||
configuration details. Only providers visible to the role can be attached
|
||||
to the integration.
|
||||
summary: Create a new integration
|
||||
tags:
|
||||
- Integration
|
||||
@@ -6810,7 +6813,7 @@ paths:
|
||||
post:
|
||||
operationId: api_v1_integrations_jira_dispatches_create
|
||||
description: |-
|
||||
Send a set of filtered findings to the given integration. At least one finding filter must be provided.
|
||||
Send a set of filtered findings to the given integration. At least one finding filter must be provided. Jira integrations are tenant-wide and do not require unlimited visibility, while the findings sent are limited to the providers the role can access.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
@@ -6883,7 +6886,8 @@ paths:
|
||||
get:
|
||||
operationId: api_v1_integrations_jira_issue_types_retrieve
|
||||
description: Fetch the available issue types from Jira for a given project key
|
||||
and update the integration configuration.
|
||||
and update the integration configuration. Jira integrations are tenant-wide
|
||||
and do not require unlimited visibility.
|
||||
summary: Get available issue types for a Jira project
|
||||
parameters:
|
||||
- in: query
|
||||
@@ -6924,7 +6928,8 @@ paths:
|
||||
get:
|
||||
operationId: api_v1_integrations_retrieve
|
||||
description: Fetch detailed information about a specific integration by its
|
||||
ID.
|
||||
ID. Integrations outside the provider visibility of the role are reported
|
||||
the same way as one that does not exist.
|
||||
summary: Retrieve integration details
|
||||
parameters:
|
||||
- in: query
|
||||
@@ -6978,7 +6983,8 @@ paths:
|
||||
patch:
|
||||
operationId: api_v1_integrations_partial_update
|
||||
description: Modify certain fields of an existing integration without affecting
|
||||
other settings.
|
||||
other settings. Integrations attached to providers outside the visibility
|
||||
of the role cannot be modified by it.
|
||||
summary: Partially update an integration
|
||||
parameters:
|
||||
- in: path
|
||||
@@ -7013,7 +7019,8 @@ paths:
|
||||
description: ''
|
||||
delete:
|
||||
operationId: api_v1_integrations_destroy
|
||||
description: Remove an integration from the system by its ID.
|
||||
description: Remove an integration from the system by its ID. Integrations attached
|
||||
to providers outside the visibility of the role cannot be deleted by it.
|
||||
summary: Delete an integration
|
||||
parameters:
|
||||
- in: path
|
||||
@@ -7033,7 +7040,9 @@ paths:
|
||||
/api/v1/integrations/{id}/connection:
|
||||
post:
|
||||
operationId: api_v1_integrations_connection_create
|
||||
description: Try to verify integration connection
|
||||
description: Try to verify integration connection. Integrations outside the
|
||||
provider visibility of the role are reported the same way as one that does
|
||||
not exist.
|
||||
summary: Check integration connection
|
||||
parameters:
|
||||
- in: path
|
||||
|
||||
@@ -2,11 +2,12 @@ import uuid
|
||||
from unittest.mock import call, patch
|
||||
|
||||
import pytest
|
||||
from api.attack_paths.database import GraphDatabaseQueryException
|
||||
from api.db_utils import POSTGRES_TENANT_VAR, SET_CONFIG_QUERY
|
||||
from api.decorators import handle_provider_deletion, set_tenant
|
||||
from api.exceptions import ProviderDeletedException
|
||||
from django.core.exceptions import ObjectDoesNotExist
|
||||
from django.db import DatabaseError, IntegrityError
|
||||
from django.db import DEFAULT_DB_ALIAS, DatabaseError, IntegrityError
|
||||
|
||||
|
||||
@pytest.mark.django_db
|
||||
@@ -204,6 +205,106 @@ class TestHandleProviderDeletionDecorator:
|
||||
with pytest.raises(DatabaseError):
|
||||
task_func(tenant_id=str(tenant.id), provider_id=str(provider.id))
|
||||
|
||||
@patch("api.decorators.rls_transaction")
|
||||
@patch("api.decorators.Provider.objects.filter")
|
||||
def test_graph_database_error_provider_missing_or_soft_deleted(
|
||||
self, mock_provider_filter, mock_rls, tenants_fixture
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider_id = str(uuid.uuid4())
|
||||
|
||||
mock_rls.return_value.__enter__ = lambda s: None
|
||||
mock_rls.return_value.__exit__ = lambda s, *args: None
|
||||
mock_provider_filter.return_value.exists.return_value = False
|
||||
|
||||
@handle_provider_deletion
|
||||
def task_func(**kwargs):
|
||||
raise GraphDatabaseQueryException("Temporary database not found")
|
||||
|
||||
with pytest.raises(ProviderDeletedException):
|
||||
task_func(tenant_id=str(tenant.id), provider_id=provider_id)
|
||||
|
||||
@patch("api.decorators.rls_transaction")
|
||||
@patch("api.decorators.Tenant.objects.filter")
|
||||
@patch("api.decorators.Provider.objects.filter")
|
||||
def test_graph_database_error_tenant_missing(
|
||||
self, mock_provider_filter, mock_tenant_filter, mock_rls, tenants_fixture
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider_id = str(uuid.uuid4())
|
||||
|
||||
mock_rls.return_value.__enter__ = lambda s: None
|
||||
mock_rls.return_value.__exit__ = lambda s, *args: None
|
||||
mock_provider_filter.return_value.exists.return_value = True
|
||||
mock_tenant_filter.return_value.exists.return_value = False
|
||||
|
||||
@handle_provider_deletion
|
||||
def task_func(**kwargs):
|
||||
raise GraphDatabaseQueryException("Temporary database not found")
|
||||
|
||||
with pytest.raises(ProviderDeletedException):
|
||||
task_func(tenant_id=str(tenant.id), provider_id=provider_id)
|
||||
|
||||
@patch("api.decorators.rls_transaction")
|
||||
@patch("api.decorators.Membership.objects.filter")
|
||||
@patch("api.decorators.Tenant.objects.filter")
|
||||
@patch("api.decorators.Provider.objects.filter")
|
||||
def test_graph_database_error_tenant_without_memberships(
|
||||
self,
|
||||
mock_provider_filter,
|
||||
mock_tenant_filter,
|
||||
mock_membership_filter,
|
||||
mock_rls,
|
||||
tenants_fixture,
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider_id = str(uuid.uuid4())
|
||||
|
||||
mock_rls.return_value.__enter__ = lambda s: None
|
||||
mock_rls.return_value.__exit__ = lambda s, *args: None
|
||||
mock_provider_filter.return_value.exists.return_value = True
|
||||
mock_tenant_filter.return_value.exists.return_value = True
|
||||
mock_membership_filter.return_value.exists.return_value = False
|
||||
|
||||
@handle_provider_deletion
|
||||
def task_func(**kwargs):
|
||||
raise GraphDatabaseQueryException("Temporary database not found")
|
||||
|
||||
with pytest.raises(ProviderDeletedException):
|
||||
task_func(tenant_id=str(tenant.id), provider_id=provider_id)
|
||||
|
||||
@patch("api.decorators.rls_transaction")
|
||||
@patch("api.decorators.Membership.objects.filter")
|
||||
@patch("api.decorators.Tenant.objects.filter")
|
||||
@patch("api.decorators.Provider.objects.filter")
|
||||
def test_graph_database_error_active_provider_and_tenant_reraises(
|
||||
self,
|
||||
mock_provider_filter,
|
||||
mock_tenant_filter,
|
||||
mock_membership_filter,
|
||||
mock_rls,
|
||||
tenants_fixture,
|
||||
):
|
||||
tenant = tenants_fixture[0]
|
||||
provider_id = str(uuid.uuid4())
|
||||
graph_error = GraphDatabaseQueryException("Temporary database not found")
|
||||
|
||||
mock_rls.return_value.__enter__ = lambda s: None
|
||||
mock_rls.return_value.__exit__ = lambda s, *args: None
|
||||
mock_provider_filter.return_value.exists.return_value = True
|
||||
mock_tenant_filter.return_value.exists.return_value = True
|
||||
mock_membership_filter.return_value.exists.return_value = True
|
||||
|
||||
@handle_provider_deletion
|
||||
def task_func(**kwargs):
|
||||
raise graph_error
|
||||
|
||||
with pytest.raises(GraphDatabaseQueryException) as exc_info:
|
||||
task_func(tenant_id=str(tenant.id), provider_id=provider_id)
|
||||
|
||||
assert exc_info.value is graph_error
|
||||
mock_rls.assert_called_once_with(str(tenant.id), using=DEFAULT_DB_ALIAS)
|
||||
|
||||
def test_missing_provider_and_scan_raises_assertion(self, tenants_fixture):
|
||||
"""Raises AssertionError when neither provider_id nor scan_id in kwargs."""
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@ from unittest.mock import ANY, Mock, patch
|
||||
|
||||
import pytest
|
||||
from api.models import (
|
||||
Integration,
|
||||
IntegrationProviderRelationship,
|
||||
Membership,
|
||||
ProviderGroup,
|
||||
ProviderGroupMembership,
|
||||
@@ -681,6 +683,363 @@ class TestLimitedVisibility:
|
||||
response.json()["data"]["relationships"]["providers"]["meta"]["count"] == 1
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def jira_integration(self, tenants_fixture):
|
||||
# Jira is a tenant-wide integration: it is not attached to any provider
|
||||
return Integration.objects.create(
|
||||
tenant_id=tenants_fixture[0].id,
|
||||
enabled=True,
|
||||
connected=True,
|
||||
integration_type=Integration.IntegrationChoices.JIRA,
|
||||
configuration={"projects": {"TEST": "Test project"}},
|
||||
credentials={
|
||||
"domain": "test",
|
||||
"user_mail": "a@b.com",
|
||||
"api_token": "token",
|
||||
},
|
||||
)
|
||||
|
||||
@pytest.fixture
|
||||
def out_of_scope_integration(self, tenants_fixture, provider_factory):
|
||||
tenant_id = tenants_fixture[0].id
|
||||
integration = Integration.objects.create(
|
||||
tenant_id=tenant_id,
|
||||
enabled=True,
|
||||
connected=True,
|
||||
integration_type=Integration.IntegrationChoices.AMAZON_S3,
|
||||
configuration={
|
||||
"bucket_name": "bucket",
|
||||
"output_directory": "output",
|
||||
},
|
||||
credentials={"aws_access_key_id": "key"},
|
||||
)
|
||||
IntegrationProviderRelationship.objects.create(
|
||||
tenant_id=tenant_id,
|
||||
integration=integration,
|
||||
provider=provider_factory(),
|
||||
)
|
||||
return integration
|
||||
|
||||
def test_integrations_list_includes_tenant_wide_integration(
|
||||
self,
|
||||
authenticated_client_rbac_limited,
|
||||
integrations_fixture,
|
||||
jira_integration,
|
||||
aws_provider_pair,
|
||||
):
|
||||
# Integration 2 is attached to both providers, so make both visible to the role
|
||||
# to assert the provider join does not duplicate it in the listing
|
||||
ProviderGroupMembership.objects.create(
|
||||
tenant_id=aws_provider_pair[1].tenant_id,
|
||||
provider=aws_provider_pair[1],
|
||||
provider_group=ProviderGroup.objects.get(name="limited_visibility_group"),
|
||||
)
|
||||
|
||||
response = authenticated_client_rbac_limited.get(reverse("integration-list"))
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
integration_ids = [item["id"] for item in response.json()["data"]]
|
||||
# The tenant-wide Jira integration is visible without unlimited visibility
|
||||
assert str(jira_integration.id) in integration_ids
|
||||
# Integrations attached to more than one visible provider are not duplicated
|
||||
assert integration_ids.count(str(integrations_fixture[1].id)) == 1
|
||||
assert response.json()["meta"]["pagination"]["count"] == len(integration_ids)
|
||||
|
||||
def test_integrations_list_without_provider_groups_keeps_tenant_wide_integration(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture, jira_integration
|
||||
):
|
||||
# A role with no provider group at all sees no provider, but still needs Jira
|
||||
RoleProviderGroupRelationship.objects.all().delete()
|
||||
|
||||
response = authenticated_client_rbac_limited.get(reverse("integration-list"))
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
integration_ids = [item["id"] for item in response.json()["data"]]
|
||||
assert integration_ids == [str(jira_integration.id)]
|
||||
|
||||
def test_integrations_include_providers_hides_out_of_scope_providers(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture, aws_provider_pair
|
||||
):
|
||||
# Integration 2 is related to provider1 (visible) and provider2 (not visible)
|
||||
hidden_provider = aws_provider_pair[1]
|
||||
|
||||
response = authenticated_client_rbac_limited.get(
|
||||
reverse("integration-list"), {"include": "providers"}
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
included_ids = {item["id"] for item in response.json().get("included", [])}
|
||||
assert str(aws_provider_pair[0].id) in included_ids
|
||||
# Sideloaded resources must not disclose the provider the role cannot see
|
||||
assert str(hidden_provider.id) not in included_ids
|
||||
|
||||
def test_integrations_list_with_sparse_fields(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture
|
||||
):
|
||||
response = authenticated_client_rbac_limited.get(
|
||||
reverse("integration-list"), {"fields[integrations]": "enabled"}
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert all(
|
||||
list(item["attributes"].keys()) == ["enabled"]
|
||||
for item in response.json()["data"]
|
||||
)
|
||||
|
||||
def test_integrations_list_excludes_out_of_scope_integration(
|
||||
self, authenticated_client_rbac_limited, out_of_scope_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.get(reverse("integration-list"))
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
integration_ids = [item["id"] for item in response.json()["data"]]
|
||||
assert str(out_of_scope_integration.id) not in integration_ids
|
||||
|
||||
def test_integration_detail_out_of_scope_returns_404(
|
||||
self, authenticated_client_rbac_limited, out_of_scope_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.get(
|
||||
reverse("integration-detail", kwargs={"pk": out_of_scope_integration.id})
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
def test_integration_connection_out_of_scope_returns_404(
|
||||
self, authenticated_client_rbac_limited, out_of_scope_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.post(
|
||||
reverse(
|
||||
"integration-connection", kwargs={"pk": out_of_scope_integration.id}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
def test_integration_update_allowed_when_fully_visible(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture, jira_integration
|
||||
):
|
||||
# Integration 1 is only related to provider1, which the role can access
|
||||
integration = integrations_fixture[0]
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "integrations",
|
||||
"id": str(integration.id),
|
||||
"attributes": {
|
||||
"enabled": False,
|
||||
# integration_type is `amazon_s3`
|
||||
"credentials": {"aws_access_key_id": "new_value"},
|
||||
"configuration": {
|
||||
"bucket_name": "new_bucket_name",
|
||||
"output_directory": "new_output_directory",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client_rbac_limited.patch(
|
||||
reverse("integration-detail", kwargs={"pk": integration.id}),
|
||||
data=json.dumps(payload),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
integration.refresh_from_db()
|
||||
assert integration.enabled is False
|
||||
|
||||
# Tenant-wide integrations have no provider restricting the role
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "integrations",
|
||||
"id": str(jira_integration.id),
|
||||
"attributes": {"enabled": False},
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client_rbac_limited.patch(
|
||||
reverse("integration-detail", kwargs={"pk": jira_integration.id}),
|
||||
data=json.dumps(payload),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
jira_integration.refresh_from_db()
|
||||
assert jira_integration.enabled is False
|
||||
|
||||
def test_integration_create_rejects_out_of_scope_provider(
|
||||
self, authenticated_client_rbac_limited, aws_provider_pair
|
||||
):
|
||||
# provider2 is not in any provider group assigned to the role
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "integrations",
|
||||
"attributes": {
|
||||
"integration_type": "amazon_s3",
|
||||
"configuration": {
|
||||
"bucket_name": "attacker_bucket",
|
||||
"output_directory": "output",
|
||||
},
|
||||
"credentials": {"aws_access_key_id": "key"},
|
||||
},
|
||||
"relationships": {
|
||||
"providers": {
|
||||
"data": [
|
||||
{"type": "providers", "id": str(aws_provider_pair[1].id)}
|
||||
]
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client_rbac_limited.post(
|
||||
reverse("integration-list"),
|
||||
data=json.dumps(payload),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
assert not Integration.objects.filter(
|
||||
integrationproviderrelationship__provider=aws_provider_pair[1],
|
||||
configuration__bucket_name="attacker_bucket",
|
||||
).exists()
|
||||
|
||||
@pytest.mark.parametrize("submitted_providers", [True, False])
|
||||
def test_integration_update_denied_when_shared_with_hidden_provider(
|
||||
self,
|
||||
authenticated_client_rbac_limited,
|
||||
integrations_fixture,
|
||||
aws_provider_pair,
|
||||
submitted_providers,
|
||||
):
|
||||
# Integration 2 is related to provider1 (visible) and provider2 (not visible).
|
||||
# Editing it would reach beyond the visibility of the role, just like deleting
|
||||
# it, so both are rejected consistently
|
||||
integration = integrations_fixture[1]
|
||||
visible_provider, hidden_provider = aws_provider_pair
|
||||
payload = {
|
||||
"data": {
|
||||
"type": "integrations",
|
||||
"id": str(integration.id),
|
||||
"attributes": {
|
||||
"enabled": False,
|
||||
# integration_type is `amazon_s3`
|
||||
"credentials": {"aws_access_key_id": "new_value"},
|
||||
"configuration": {
|
||||
"bucket_name": "new_bucket_name",
|
||||
"output_directory": "new_output_directory",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
if submitted_providers:
|
||||
payload["data"]["relationships"] = {
|
||||
"providers": {
|
||||
"data": [{"type": "providers", "id": str(visible_provider.id)}]
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client_rbac_limited.patch(
|
||||
reverse("integration-detail", kwargs={"pk": integration.id}),
|
||||
data=json.dumps(payload),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_403_FORBIDDEN
|
||||
integration.refresh_from_db()
|
||||
assert integration.enabled is True
|
||||
assert integration.providers.filter(id=hidden_provider.id).exists()
|
||||
assert integration.providers.filter(id=visible_provider.id).exists()
|
||||
|
||||
def test_integration_delete_denied_when_shared_with_hidden_provider(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture
|
||||
):
|
||||
# Integration 2 is related to provider1 (visible) and provider2 (not visible)
|
||||
integration = integrations_fixture[1]
|
||||
|
||||
response = authenticated_client_rbac_limited.delete(
|
||||
reverse("integration-detail", kwargs={"pk": integration.id})
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_403_FORBIDDEN
|
||||
assert Integration.objects.filter(id=integration.id).exists()
|
||||
|
||||
def test_integration_delete_allowed_when_fully_visible(
|
||||
self, authenticated_client_rbac_limited, integrations_fixture, jira_integration
|
||||
):
|
||||
# Integration 1 is only related to provider1, which the role can access
|
||||
integration = integrations_fixture[0]
|
||||
|
||||
response = authenticated_client_rbac_limited.delete(
|
||||
reverse("integration-detail", kwargs={"pk": integration.id})
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_204_NO_CONTENT
|
||||
assert not Integration.objects.filter(id=integration.id).exists()
|
||||
|
||||
# Tenant-wide integrations have no provider restricting the role
|
||||
response = authenticated_client_rbac_limited.delete(
|
||||
reverse("integration-detail", kwargs={"pk": jira_integration.id})
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_204_NO_CONTENT
|
||||
|
||||
def test_jira_issue_types_allowed_without_unlimited_visibility(
|
||||
self, authenticated_client_rbac_limited, jira_integration
|
||||
):
|
||||
with patch("api.v1.views.initialize_prowler_integration") as mock_jira:
|
||||
mock_jira.return_value.get_available_issue_types.return_value = ["Task"]
|
||||
response = authenticated_client_rbac_limited.get(
|
||||
reverse(
|
||||
"integration-jira-issue-types",
|
||||
kwargs={"integration_pk": jira_integration.id},
|
||||
),
|
||||
{"project_key": "TEST"},
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
assert response.json()["data"]["attributes"]["issue_types"] == ["Task"]
|
||||
|
||||
def test_jira_issue_types_out_of_scope_returns_404(
|
||||
self, authenticated_client_rbac_limited, out_of_scope_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.get(
|
||||
reverse(
|
||||
"integration-jira-issue-types",
|
||||
kwargs={"integration_pk": out_of_scope_integration.id},
|
||||
),
|
||||
{"project_key": "TEST"},
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
def test_jira_dispatches_out_of_scope_returns_404(
|
||||
self, authenticated_client_rbac_limited, out_of_scope_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.post(
|
||||
reverse(
|
||||
"integration-jira-dispatches",
|
||||
kwargs={"integration_pk": out_of_scope_integration.id},
|
||||
),
|
||||
data=json.dumps({}),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_404_NOT_FOUND
|
||||
|
||||
def test_jira_dispatches_allowed_without_unlimited_visibility(
|
||||
self, authenticated_client_rbac_limited, jira_integration
|
||||
):
|
||||
response = authenticated_client_rbac_limited.post(
|
||||
reverse(
|
||||
"integration-jira-dispatches",
|
||||
kwargs={"integration_pk": jira_integration.id},
|
||||
),
|
||||
data=json.dumps({}),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
# The integration is reachable: the request fails on payload validation, not RBAC
|
||||
assert response.status_code == status.HTTP_400_BAD_REQUEST
|
||||
|
||||
@pytest.mark.usefixtures("scan_summaries_fixture")
|
||||
def test_overviews_providers(
|
||||
self,
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from api.attack_paths.retryable_session import RetryableSession
|
||||
from api.attack_paths.retryable_session import RetryableSession, RetryExhaustedError
|
||||
from neo4j.exceptions import ServiceUnavailable
|
||||
|
||||
|
||||
@@ -24,6 +24,7 @@ class TestRetryableSession:
|
||||
max_retries=3,
|
||||
retry_if=lambda exc: exc is retryable_error,
|
||||
initial_retry_delay_seconds=2,
|
||||
retry_context="Neptune write",
|
||||
)
|
||||
|
||||
assert session.execute_write(work) == "success"
|
||||
@@ -54,6 +55,7 @@ class TestRetryableSession:
|
||||
max_retries=3,
|
||||
retry_if=lambda _: False,
|
||||
initial_retry_delay_seconds=2,
|
||||
retry_context="Neptune write",
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError) as exc_info:
|
||||
@@ -83,3 +85,81 @@ class TestRetryableSession:
|
||||
driver_sessions[0].close.assert_called_once_with()
|
||||
driver_sessions[1].close.assert_called_once_with()
|
||||
driver_sessions[2].close.assert_not_called()
|
||||
|
||||
def test_retry_exhaustion_with_context_reports_attempts_and_elapsed_time(self):
|
||||
error = RuntimeError("still retryable")
|
||||
driver_sessions = [MagicMock() for _ in range(3)]
|
||||
for driver_session in driver_sessions:
|
||||
driver_session.execute_write.side_effect = error
|
||||
session = RetryableSession(
|
||||
session_factory=MagicMock(side_effect=driver_sessions),
|
||||
max_retries=2,
|
||||
retry_if=lambda _: True,
|
||||
retry_context="Neptune write",
|
||||
)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"api.attack_paths.retryable_session.time.monotonic",
|
||||
side_effect=[100.0, 127.1234],
|
||||
),
|
||||
pytest.raises(RetryExhaustedError) as exc_info,
|
||||
):
|
||||
session.execute_write(MagicMock())
|
||||
|
||||
assert exc_info.value.method_name == "execute_write"
|
||||
assert exc_info.value.attempts == 3
|
||||
assert exc_info.value.elapsed_seconds == pytest.approx(27.1234)
|
||||
assert exc_info.value.last_error is error
|
||||
assert exc_info.value.__cause__ is error
|
||||
assert str(exc_info.value) == (
|
||||
"Neptune write execute_write failed after 3 attempts over 27.123s. "
|
||||
"Last error: still retryable"
|
||||
)
|
||||
|
||||
def test_retry_exhaustion_with_zero_retries_reports_one_attempt(self):
|
||||
error = ServiceUnavailable("still unavailable")
|
||||
driver_session = MagicMock()
|
||||
driver_session.execute_write.side_effect = error
|
||||
session = RetryableSession(
|
||||
session_factory=MagicMock(return_value=driver_session),
|
||||
max_retries=0,
|
||||
retry_context="Neptune write",
|
||||
)
|
||||
|
||||
with pytest.raises(RetryExhaustedError) as exc_info:
|
||||
session.execute_write(MagicMock())
|
||||
|
||||
assert exc_info.value.attempts == 1
|
||||
|
||||
@patch("api.attack_paths.retryable_session.time.sleep")
|
||||
@patch("api.attack_paths.retryable_session.random.uniform", return_value=3.0)
|
||||
def test_contextual_retry_warning_includes_original_error(
|
||||
self, _mock_uniform, _mock_sleep
|
||||
):
|
||||
error = RuntimeError("retryable detail")
|
||||
first_session = MagicMock()
|
||||
first_session.execute_write.side_effect = error
|
||||
second_session = MagicMock()
|
||||
second_session.execute_write.return_value = "success"
|
||||
session = RetryableSession(
|
||||
session_factory=MagicMock(side_effect=[first_session, second_session]),
|
||||
max_retries=1,
|
||||
retry_if=lambda _: True,
|
||||
initial_retry_delay_seconds=2,
|
||||
retry_context="Neptune write",
|
||||
)
|
||||
|
||||
with patch("api.attack_paths.retryable_session.logger.warning") as mock_warning:
|
||||
assert session.execute_write(MagicMock()) == "success"
|
||||
|
||||
mock_warning.assert_called_once_with(
|
||||
"%s %s failed with %s: %s; retry %s/%s in %.3fs",
|
||||
"Neptune write",
|
||||
"execute_write",
|
||||
"RuntimeError",
|
||||
"retryable detail",
|
||||
1,
|
||||
1,
|
||||
3.0,
|
||||
)
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import logging
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
from config.settings import sentry as sentry_settings
|
||||
from config.settings.sentry import before_send
|
||||
|
||||
@@ -82,6 +83,45 @@ def test_before_send_passes_through_non_ignored_log():
|
||||
assert result == event
|
||||
|
||||
|
||||
def test_before_send_ignores_cartography_missing_temporary_database_log():
|
||||
log_record = _make_log_record(
|
||||
msg="Cartography job failed with %s for database %s",
|
||||
name="cartography.graph.job",
|
||||
args=(
|
||||
"Neo.ClientError.Database.DatabaseNotFound",
|
||||
"db-tmp-scan-12345678",
|
||||
),
|
||||
)
|
||||
|
||||
event = MagicMock()
|
||||
|
||||
assert before_send(event, {"log_record": log_record}) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("logger_name", "message"),
|
||||
[
|
||||
(
|
||||
"cartography.graph.job.worker",
|
||||
"Neo.ClientError.Database.DatabaseNotFound for db-tmp-scan-12345678",
|
||||
),
|
||||
(
|
||||
"cartography.graph.job",
|
||||
"DatabaseNotFound for db-tmp-scan-12345678",
|
||||
),
|
||||
(
|
||||
"cartography.graph.job",
|
||||
"Neo.ClientError.Database.DatabaseNotFound for db-tenant-12345678",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_before_send_passes_through_similar_cartography_logs(logger_name, message):
|
||||
log_record = _make_log_record(msg=message, name=logger_name)
|
||||
event = MagicMock()
|
||||
|
||||
assert before_send(event, {"log_record": log_record}) is event
|
||||
|
||||
|
||||
def test_before_send_passes_through_non_ignored_exception():
|
||||
"""Test that before_send passes through exceptions that don't contain ignored exceptions."""
|
||||
exc_info = (Exception, Exception("Some other error message"), None)
|
||||
|
||||
@@ -3,7 +3,12 @@ from api.v1.serializer_utils.integrations import (
|
||||
JiraCredentialSerializer,
|
||||
S3ConfigSerializer,
|
||||
)
|
||||
from api.v1.serializers import ImageProviderSecret, KubernetesProviderSecret
|
||||
from api.v1.serializer_utils.providers import ProviderSecretField
|
||||
from api.v1.serializers import (
|
||||
ImageProviderSecret,
|
||||
KubernetesProviderSecret,
|
||||
OracleCloudProviderSecret,
|
||||
)
|
||||
from rest_framework.exceptions import ValidationError
|
||||
|
||||
|
||||
@@ -190,6 +195,64 @@ class TestImageProviderSecret:
|
||||
assert "non_field_errors" in serializer.errors
|
||||
|
||||
|
||||
class TestOracleCloudProviderSecret:
|
||||
def valid_secret(self, **overrides):
|
||||
secret = {
|
||||
"user": "ocid1.user.oc1..aaaaaaaexample",
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "fake-base64-key-content",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaexample",
|
||||
}
|
||||
secret.update(overrides)
|
||||
return secret
|
||||
|
||||
def test_accepts_regionless_secret(self):
|
||||
serializer = OracleCloudProviderSecret(data=self.valid_secret())
|
||||
|
||||
assert serializer.is_valid(), serializer.errors
|
||||
assert "region" not in serializer.validated_data
|
||||
|
||||
def test_accepts_and_ignores_region_field(self):
|
||||
secret = self.valid_secret(region="us-phoenix-1")
|
||||
serializer = OracleCloudProviderSecret(data=secret)
|
||||
|
||||
assert serializer.is_valid(), serializer.errors
|
||||
|
||||
assert "region" not in serializer.validated_data
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"legacy_field, legacy_value",
|
||||
[
|
||||
("region", None),
|
||||
("region", ""),
|
||||
("region", {"name": "us-ashburn-1"}),
|
||||
],
|
||||
)
|
||||
def test_accepts_and_ignores_any_legacy_region_value(
|
||||
self, legacy_field, legacy_value
|
||||
):
|
||||
serializer = OracleCloudProviderSecret(
|
||||
data=self.valid_secret(**{legacy_field: legacy_value})
|
||||
)
|
||||
|
||||
assert serializer.is_valid(), serializer.errors
|
||||
|
||||
assert legacy_field not in serializer.validated_data
|
||||
|
||||
|
||||
class TestProviderSecretFieldSchema:
|
||||
def test_oraclecloud_schema_includes_legacy_region_field(self):
|
||||
schema = ProviderSecretField._spectacular_annotation["field"]
|
||||
oraclecloud_schema = next(
|
||||
credential_schema
|
||||
for credential_schema in schema["oneOf"]
|
||||
if credential_schema["title"]
|
||||
== "Oracle Cloud Infrastructure (OCI) API Key Credentials"
|
||||
)
|
||||
|
||||
assert oraclecloud_schema["properties"]["region"]["deprecated"] is True
|
||||
|
||||
|
||||
class TestKubernetesProviderSecret:
|
||||
def test_valid_static_kubeconfig_is_accepted(self):
|
||||
kubeconfig_content = """
|
||||
|
||||
@@ -11,7 +11,11 @@ from unittest.mock import MagicMock, patch
|
||||
import neo4j
|
||||
import pytest
|
||||
from api.attack_paths import sink as sink_module
|
||||
from api.attack_paths.database import GraphDatabaseQueryException
|
||||
from api.attack_paths.database import (
|
||||
GraphDatabaseQueryException,
|
||||
NeptuneWriteRetryExhaustedException,
|
||||
)
|
||||
from api.attack_paths.retryable_session import RetryExhaustedError
|
||||
from api.attack_paths.sink import factory
|
||||
from api.attack_paths.sink.neo4j import DATABASE_NOT_FOUND_CODE, Neo4jSink
|
||||
from api.attack_paths.sink.neptune import (
|
||||
@@ -123,6 +127,14 @@ class TestSinkFactory:
|
||||
assert mock_driver.call_count == 1
|
||||
|
||||
|
||||
def test_neo4j_sync_batch_size_defaults_to_1000():
|
||||
assert Neo4jSink.sync_batch_size == 1000
|
||||
|
||||
|
||||
def test_neptune_sync_batch_size_defaults_to_500():
|
||||
assert NeptuneSink.sync_batch_size == 500
|
||||
|
||||
|
||||
class TestGetBackendForScan:
|
||||
"""``get_backend_for_scan`` routes by the row's recorded sink backend."""
|
||||
|
||||
@@ -372,6 +384,7 @@ class TestNeptuneRetryPolicy:
|
||||
assert (
|
||||
kwargs["initial_retry_delay_seconds"] == NEPTUNE_WRITE_RETRY_DELAY_SECONDS
|
||||
)
|
||||
assert kwargs["retry_context"] == "Neptune write"
|
||||
|
||||
@patch("api.attack_paths.sink.neptune.RetryableSession")
|
||||
def test_reader_session_does_not_enable_write_retry_policy(self, retryable_session):
|
||||
@@ -384,6 +397,48 @@ class TestNeptuneRetryPolicy:
|
||||
kwargs = retryable_session.call_args.kwargs
|
||||
assert kwargs["retry_if"] is None
|
||||
assert kwargs["initial_retry_delay_seconds"] == 0
|
||||
assert kwargs["retry_context"] is None
|
||||
|
||||
def test_writer_retry_exhaustion_preserves_neptune_error_details(self):
|
||||
message = (
|
||||
"Unexpected server exception 'Operation failed due to conflicting "
|
||||
"concurrent operations (please retry), 0 transactions are currently "
|
||||
"rolling back.'"
|
||||
)
|
||||
error = neo4j.exceptions.Neo4jError._hydrate_neo4j(
|
||||
code="BoltProtocol.unexpectedException",
|
||||
message=message,
|
||||
)
|
||||
retry_error = RetryExhaustedError(
|
||||
retry_context="Neptune write",
|
||||
method_name="execute_write",
|
||||
attempts=4,
|
||||
elapsed_seconds=27.1234,
|
||||
last_error=error,
|
||||
)
|
||||
sink = NeptuneSink()
|
||||
driver = MagicMock()
|
||||
retryable_session = MagicMock()
|
||||
retryable_session.execute_write.side_effect = retry_error
|
||||
|
||||
with (
|
||||
patch.object(sink, "_get_writer", return_value=driver),
|
||||
patch(
|
||||
"api.attack_paths.sink.neptune.RetryableSession",
|
||||
return_value=retryable_session,
|
||||
),
|
||||
pytest.raises(NeptuneWriteRetryExhaustedException) as exc_info,
|
||||
):
|
||||
with sink.get_session() as session:
|
||||
session.execute_write(MagicMock())
|
||||
|
||||
assert exc_info.value.code == "BoltProtocol.unexpectedException"
|
||||
assert str(exc_info.value) == (
|
||||
"BoltProtocol.unexpectedException: Neptune write execute_write failed "
|
||||
"after 4 attempts over 27.123s. Last error: "
|
||||
f"{message}"
|
||||
)
|
||||
assert exc_info.value.__cause__ is error
|
||||
|
||||
|
||||
class TestNeptuneSinkDropSubgraph:
|
||||
|
||||
@@ -171,6 +171,53 @@ class TestInitializeProwlerProvider:
|
||||
key="value", mutelist_content={"key": "value"}
|
||||
)
|
||||
|
||||
@patch("api.utils.return_prowler_provider")
|
||||
def test_initialize_oraclecloud_provider_removes_region_string(
|
||||
self, mock_return_prowler_provider
|
||||
):
|
||||
provider = MagicMock()
|
||||
provider.provider = Provider.ProviderChoices.ORACLECLOUD.value
|
||||
provider.secret.secret = {
|
||||
"user": "ocid1.user.oc1..fake",
|
||||
"fingerprint": "00:11:22:33:44:55:66:77",
|
||||
"key_content": "fake-base64-key-content",
|
||||
"tenancy": "ocid1.tenancy.oc1..fake",
|
||||
"region": "us-ashburn-1",
|
||||
}
|
||||
mock_return_prowler_provider.return_value = MagicMock()
|
||||
|
||||
initialize_prowler_provider(provider)
|
||||
|
||||
mock_return_prowler_provider.return_value.assert_called_once_with(
|
||||
user="ocid1.user.oc1..fake",
|
||||
fingerprint="00:11:22:33:44:55:66:77",
|
||||
key_content="fake-base64-key-content",
|
||||
tenancy="ocid1.tenancy.oc1..fake",
|
||||
)
|
||||
|
||||
@patch("api.utils.return_prowler_provider")
|
||||
def test_initialize_oraclecloud_provider_without_region_omits_scan_filter(
|
||||
self, mock_return_prowler_provider
|
||||
):
|
||||
provider = MagicMock()
|
||||
provider.provider = Provider.ProviderChoices.ORACLECLOUD.value
|
||||
provider.secret.secret = {
|
||||
"user": "ocid1.user.oc1..fake",
|
||||
"fingerprint": "00:11:22:33:44:55:66:77",
|
||||
"key_content": "fake-base64-key-content",
|
||||
"tenancy": "ocid1.tenancy.oc1..fake",
|
||||
}
|
||||
mock_return_prowler_provider.return_value = MagicMock()
|
||||
|
||||
initialize_prowler_provider(provider)
|
||||
|
||||
mock_return_prowler_provider.return_value.assert_called_once_with(
|
||||
user="ocid1.user.oc1..fake",
|
||||
fingerprint="00:11:22:33:44:55:66:77",
|
||||
key_content="fake-base64-key-content",
|
||||
tenancy="ocid1.tenancy.oc1..fake",
|
||||
)
|
||||
|
||||
|
||||
class TestProwlerProviderConnectionTest:
|
||||
@patch("api.utils.return_prowler_provider")
|
||||
@@ -185,6 +232,37 @@ class TestProwlerProviderConnectionTest:
|
||||
key="value", provider_id="1234567890", raise_on_exception=False
|
||||
)
|
||||
|
||||
@patch("api.utils.return_prowler_provider")
|
||||
def test_oraclecloud_connection_test_uses_direct_credentials_without_region(
|
||||
self, mock_return_prowler_provider
|
||||
):
|
||||
provider = MagicMock()
|
||||
provider.uid = "ocid1.tenancy.oc1..aaaaaaaexample"
|
||||
provider.provider = Provider.ProviderChoices.ORACLECLOUD.value
|
||||
provider.secret.secret = {
|
||||
"user": "ocid1.user.oc1..aaaaaaaexample",
|
||||
"fingerprint": "00:11:22:33:44:55:66:77",
|
||||
"key_content": "fake-base64-key-content",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaexample",
|
||||
}
|
||||
mock_return_prowler_provider.return_value = MagicMock()
|
||||
|
||||
prowler_provider_connection_test(provider)
|
||||
|
||||
mock_return_prowler_provider.return_value.test_connection.assert_called_once_with(
|
||||
user="ocid1.user.oc1..aaaaaaaexample",
|
||||
fingerprint="00:11:22:33:44:55:66:77",
|
||||
key_content="fake-base64-key-content",
|
||||
tenancy="ocid1.tenancy.oc1..aaaaaaaexample",
|
||||
region=getattr(
|
||||
OraclecloudProvider,
|
||||
"_bootstrap_region",
|
||||
OraclecloudProvider._home_region,
|
||||
),
|
||||
provider_id="ocid1.tenancy.oc1..aaaaaaaexample",
|
||||
raise_on_exception=False,
|
||||
)
|
||||
|
||||
@pytest.mark.django_db
|
||||
@patch("api.utils.return_prowler_provider")
|
||||
def test_prowler_provider_connection_test_without_secret(
|
||||
@@ -356,7 +434,7 @@ class TestGetProwlerProviderKwargs:
|
||||
expected_result = {**secret_dict, **expected_extra_kwargs}
|
||||
assert result == expected_result
|
||||
|
||||
def test_get_prowler_provider_kwargs_oraclecloud_converts_region_string_to_set(
|
||||
def test_get_prowler_provider_kwargs_oraclecloud_removes_region(
|
||||
self,
|
||||
):
|
||||
secret_dict = {
|
||||
@@ -377,8 +455,13 @@ class TestGetProwlerProviderKwargs:
|
||||
|
||||
result = get_prowler_provider_kwargs(provider)
|
||||
|
||||
expected_result = {**secret_dict, "region": {"us-ashburn-1"}}
|
||||
assert result == expected_result
|
||||
assert result == {
|
||||
"user": "ocid1.user.oc1..fake",
|
||||
"fingerprint": "00:11:22:33:44:55:66:77",
|
||||
"key_content": "-----BEGIN PRIVATE KEY-----\nfake\n-----END PRIVATE KEY-----",
|
||||
"tenancy": "ocid1.tenancy.oc1..fake",
|
||||
"pass_phrase": "fake-passphrase",
|
||||
}
|
||||
|
||||
def test_get_prowler_provider_kwargs_with_mutelist(self):
|
||||
provider_uid = "provider_uid"
|
||||
|
||||
@@ -2917,6 +2917,48 @@ class TestProviderGroupViewSet:
|
||||
|
||||
@pytest.mark.django_db
|
||||
class TestProviderSecretViewSet:
|
||||
@staticmethod
|
||||
def _oraclecloud_secret(**overrides):
|
||||
secret = {
|
||||
"user": "ocid1.user.oc1..aaaaaaaakldibrbov4ubh25aqdeiroklxjngwka7u6w7no3glmdq3n5sxtkq",
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "test-key-content",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
}
|
||||
secret.update(overrides)
|
||||
return secret
|
||||
|
||||
def _create_oraclecloud_secret(
|
||||
self,
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
secret,
|
||||
name="OCI Secret",
|
||||
):
|
||||
data = {
|
||||
"data": {
|
||||
"type": "provider-secrets",
|
||||
"attributes": {
|
||||
"name": name,
|
||||
"secret_type": ProviderSecret.TypeChoices.STATIC,
|
||||
"secret": secret,
|
||||
},
|
||||
"relationships": {
|
||||
"provider": {
|
||||
"data": {
|
||||
"type": "providers",
|
||||
"id": str(oraclecloud_provider.id),
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
return authenticated_client.post(
|
||||
reverse("providersecret-list"),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
def test_provider_secrets_list(self, authenticated_client, provider_secret_fixture):
|
||||
response = authenticated_client.get(reverse("providersecret-list"))
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
@@ -3076,7 +3118,6 @@ current-context: test-context
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "-----BEGIN RSA PRIVATE KEY-----\ntest-key-content\n-----END RSA PRIVATE KEY-----",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
},
|
||||
),
|
||||
# OCI with API key credentials (with key_file)
|
||||
@@ -3088,7 +3129,6 @@ current-context: test-context
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_file": "/path/to/oci_api_key.pem",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
},
|
||||
),
|
||||
# OCI with API key credentials (with passphrase)
|
||||
@@ -3100,7 +3140,6 @@ current-context: test-context
|
||||
"fingerprint": "aa:bb:cc:dd:ee:ff:00:11:22:33:44:55:66:77:88:99",
|
||||
"key_content": "-----BEGIN RSA PRIVATE KEY-----\ntest-encrypted-key\n-----END RSA PRIVATE KEY-----",
|
||||
"tenancy": "ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
|
||||
"region": "us-ashburn-1",
|
||||
"pass_phrase": "my-secure-passphrase",
|
||||
},
|
||||
),
|
||||
@@ -3258,6 +3297,103 @@ current-context: test-context
|
||||
== data["data"]["relationships"]["provider"]["data"]["id"]
|
||||
)
|
||||
|
||||
def test_provider_secrets_create_oraclecloud_without_region_stores_no_region(
|
||||
self,
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
):
|
||||
response = self._create_oraclecloud_secret(
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
self._oraclecloud_secret(),
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_201_CREATED
|
||||
provider_secret = ProviderSecret.objects.get()
|
||||
assert "region" not in provider_secret.secret
|
||||
|
||||
def test_provider_secrets_create_oraclecloud_accepts_and_ignores_region(
|
||||
self,
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
):
|
||||
response = self._create_oraclecloud_secret(
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
self._oraclecloud_secret(
|
||||
key_content=" test-key-content ", region=" us-ashburn-1 "
|
||||
),
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_201_CREATED
|
||||
provider_secret = ProviderSecret.objects.get()
|
||||
assert provider_secret.secret["key_content"] == "test-key-content"
|
||||
assert "region" not in provider_secret.secret
|
||||
|
||||
def test_provider_secrets_update_oraclecloud_without_region_stores_no_region(
|
||||
self,
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
):
|
||||
create_response = self._create_oraclecloud_secret(
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
self._oraclecloud_secret(),
|
||||
)
|
||||
provider_secret = ProviderSecret.objects.get(
|
||||
id=create_response.json()["data"]["id"]
|
||||
)
|
||||
data = {
|
||||
"data": {
|
||||
"type": "provider-secrets",
|
||||
"id": str(provider_secret.id),
|
||||
"attributes": {"secret": self._oraclecloud_secret()},
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client.patch(
|
||||
reverse("providersecret-detail", kwargs={"pk": provider_secret.id}),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
provider_secret.refresh_from_db()
|
||||
assert "region" not in provider_secret.secret
|
||||
|
||||
def test_provider_secrets_update_oraclecloud_accepts_and_ignores_region(
|
||||
self,
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
):
|
||||
create_response = self._create_oraclecloud_secret(
|
||||
authenticated_client,
|
||||
oraclecloud_provider,
|
||||
self._oraclecloud_secret(),
|
||||
)
|
||||
provider_secret = ProviderSecret.objects.get(
|
||||
id=create_response.json()["data"]["id"]
|
||||
)
|
||||
data = {
|
||||
"data": {
|
||||
"type": "provider-secrets",
|
||||
"id": str(provider_secret.id),
|
||||
"attributes": {
|
||||
"secret": self._oraclecloud_secret(region=" us-ashburn-1 ")
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
response = authenticated_client.patch(
|
||||
reverse("providersecret-detail", kwargs={"pk": provider_secret.id}),
|
||||
data=json.dumps(data),
|
||||
content_type="application/vnd.api+json",
|
||||
)
|
||||
|
||||
assert response.status_code == status.HTTP_200_OK
|
||||
provider_secret.refresh_from_db()
|
||||
assert "region" not in provider_secret.secret
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"attributes, error_code, error_pointer",
|
||||
(
|
||||
|
||||
@@ -252,12 +252,6 @@ def get_prowler_provider_kwargs(
|
||||
**prowler_provider_kwargs,
|
||||
"filter_accounts": [provider.uid],
|
||||
}
|
||||
elif provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
|
||||
if isinstance(prowler_provider_kwargs.get("region"), str):
|
||||
prowler_provider_kwargs = {
|
||||
**prowler_provider_kwargs,
|
||||
"region": {prowler_provider_kwargs["region"]},
|
||||
}
|
||||
elif provider.provider == Provider.ProviderChoices.OPENSTACK.value:
|
||||
# clouds_yaml_content, clouds_yaml_cloud and provider_id are validated
|
||||
# in the provider itself, so it's not needed here.
|
||||
@@ -288,6 +282,11 @@ def get_prowler_provider_kwargs(
|
||||
**{k: v for k, v in prowler_provider_kwargs.items() if v},
|
||||
}
|
||||
|
||||
elif provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
|
||||
prowler_provider_kwargs = _normalize_oraclecloud_provider_kwargs(
|
||||
prowler_provider_kwargs
|
||||
)
|
||||
|
||||
if mutelist_processor:
|
||||
mutelist_content = mutelist_processor.configuration.get("Mutelist", {})
|
||||
# IaC and Image providers don't support mutelist (both use Trivy's built-in logic)
|
||||
@@ -300,6 +299,40 @@ def get_prowler_provider_kwargs(
|
||||
return prowler_provider_kwargs
|
||||
|
||||
|
||||
def _normalize_oraclecloud_provider_kwargs(secret: dict) -> dict:
|
||||
"""Normalize external OCI secret fields into SDK provider kwargs."""
|
||||
prowler_provider_kwargs = secret.copy()
|
||||
prowler_provider_kwargs.pop("region", None)
|
||||
|
||||
return prowler_provider_kwargs
|
||||
|
||||
|
||||
def _normalize_oraclecloud_connection_test_kwargs(secret: dict) -> dict:
|
||||
"""Normalize external OCI secret fields into test_connection kwargs."""
|
||||
from prowler.providers.oraclecloud.oraclecloud_provider import OraclecloudProvider
|
||||
|
||||
prowler_provider_kwargs = secret.copy()
|
||||
prowler_provider_kwargs.pop("region", None)
|
||||
|
||||
if (
|
||||
prowler_provider_kwargs.get("user")
|
||||
and prowler_provider_kwargs.get("fingerprint")
|
||||
and prowler_provider_kwargs.get("tenancy")
|
||||
and (
|
||||
prowler_provider_kwargs.get("key_content")
|
||||
or prowler_provider_kwargs.get("key_file")
|
||||
)
|
||||
):
|
||||
# Connection validation needs one OCI endpoint, but scans remain unfiltered.
|
||||
prowler_provider_kwargs["region"] = getattr(
|
||||
OraclecloudProvider,
|
||||
"_bootstrap_region",
|
||||
OraclecloudProvider._home_region,
|
||||
)
|
||||
|
||||
return prowler_provider_kwargs
|
||||
|
||||
|
||||
def initialize_prowler_provider(
|
||||
provider: Provider,
|
||||
mutelist_processor: Processor | None = None,
|
||||
@@ -402,6 +435,15 @@ def prowler_provider_connection_test(provider: Provider) -> Connection:
|
||||
if prowler_provider_kwargs.get("registry_token"):
|
||||
image_kwargs["registry_token"] = prowler_provider_kwargs["registry_token"]
|
||||
return prowler_provider.test_connection(**image_kwargs)
|
||||
elif provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
|
||||
oraclecloud_kwargs = _normalize_oraclecloud_connection_test_kwargs(
|
||||
prowler_provider_kwargs
|
||||
)
|
||||
return prowler_provider.test_connection(
|
||||
**oraclecloud_kwargs,
|
||||
provider_id=provider.uid,
|
||||
raise_on_exception=False,
|
||||
)
|
||||
else:
|
||||
return prowler_provider.test_connection(
|
||||
**prowler_provider_kwargs,
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
import os
|
||||
import re
|
||||
|
||||
from api.models import Integration, IntegrationProviderRelationship, Provider
|
||||
from api.v1.serializer_utils.base import BaseValidateSerializer
|
||||
from django.db import transaction
|
||||
from drf_spectacular.utils import extend_schema_field
|
||||
from rest_framework_json_api import serializers
|
||||
|
||||
@@ -10,6 +12,24 @@ ATLASSIAN_SITE_NAME_REGEX = re.compile(
|
||||
)
|
||||
|
||||
|
||||
def replace_integration_providers(
|
||||
integration: Integration, providers: list[Provider], tenant_id: str
|
||||
) -> None:
|
||||
"""Replace the provider relationships of an integration with the given set."""
|
||||
# Atomic on its own, so callers without an ambient transaction cannot leave the
|
||||
# integration with no relationships if the recreation fails halfway
|
||||
with transaction.atomic():
|
||||
IntegrationProviderRelationship.objects.filter(integration=integration).delete()
|
||||
IntegrationProviderRelationship.objects.bulk_create(
|
||||
[
|
||||
IntegrationProviderRelationship(
|
||||
integration=integration, provider=provider, tenant_id=tenant_id
|
||||
)
|
||||
for provider in providers
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
class S3ConfigSerializer(BaseValidateSerializer):
|
||||
bucket_name = serializers.CharField()
|
||||
output_directory = serializers.CharField(allow_blank=True)
|
||||
|
||||
@@ -295,16 +295,21 @@ from rest_framework_json_api import serializers
|
||||
"type": "string",
|
||||
"description": "The OCID of the tenancy.",
|
||||
},
|
||||
"region": {
|
||||
"type": "string",
|
||||
"description": "The OCI region identifier (e.g., us-ashburn-1, us-phoenix-1).",
|
||||
},
|
||||
"pass_phrase": {
|
||||
"type": "string",
|
||||
"description": "The passphrase for the private key, if encrypted.",
|
||||
},
|
||||
"region": {
|
||||
"type": "string",
|
||||
"deprecated": True,
|
||||
"description": "Legacy OCI region field accepted for backwards compatibility but ignored; OCI scans all regions.",
|
||||
},
|
||||
},
|
||||
"required": ["user", "fingerprint", "tenancy", "region"],
|
||||
"required": ["user", "fingerprint", "tenancy"],
|
||||
"anyOf": [
|
||||
{"required": ["key_file"]},
|
||||
{"required": ["key_content"]},
|
||||
],
|
||||
},
|
||||
{
|
||||
"type": "object",
|
||||
|
||||
@@ -47,6 +47,7 @@ from api.v1.serializer_utils.integrations import (
|
||||
JiraCredentialSerializer,
|
||||
S3ConfigSerializer,
|
||||
SecurityHubConfigSerializer,
|
||||
replace_integration_providers,
|
||||
)
|
||||
from api.v1.serializer_utils.lighthouse import (
|
||||
BedrockCredentialsSerializer,
|
||||
@@ -1672,6 +1673,7 @@ class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
|
||||
validation_error.detail[f"secret/{key}"] = value
|
||||
del validation_error.detail[key]
|
||||
raise validation_error
|
||||
return serializer.validated_data
|
||||
|
||||
|
||||
class AwsProviderSecret(serializers.Serializer):
|
||||
@@ -1813,14 +1815,32 @@ class IacProviderSecret(serializers.Serializer):
|
||||
resource_name = "provider-secrets"
|
||||
|
||||
|
||||
class LegacyOCIRegionField(serializers.Field):
|
||||
def to_internal_value(self, data):
|
||||
return data
|
||||
|
||||
def to_representation(self, value):
|
||||
return value
|
||||
|
||||
|
||||
class OracleCloudProviderSecret(serializers.Serializer):
|
||||
user = serializers.CharField()
|
||||
fingerprint = serializers.CharField()
|
||||
key_file = serializers.CharField(required=False)
|
||||
key_content = serializers.CharField(required=False)
|
||||
tenancy = serializers.CharField()
|
||||
region = serializers.CharField()
|
||||
pass_phrase = serializers.CharField(required=False)
|
||||
region = LegacyOCIRegionField(required=False, allow_null=True)
|
||||
|
||||
def validate(self, attrs):
|
||||
attrs.pop("region", None)
|
||||
|
||||
if "key_file" not in attrs and "key_content" not in attrs:
|
||||
raise serializers.ValidationError(
|
||||
{"key_file": "Either key_file or key_content must be provided."}
|
||||
)
|
||||
|
||||
return attrs
|
||||
|
||||
class Meta:
|
||||
resource_name = "provider-secrets"
|
||||
@@ -1965,7 +1985,11 @@ class ProviderSecretCreateSerializer(RLSSerializer, BaseWriteProviderSecretSeria
|
||||
secret = attrs.get("secret")
|
||||
|
||||
validated_attrs = super().validate(attrs)
|
||||
self.validate_secret_based_on_provider(provider.provider, secret_type, secret)
|
||||
validated_secret = self.validate_secret_based_on_provider(
|
||||
provider.provider, secret_type, secret
|
||||
)
|
||||
if provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
|
||||
validated_attrs["secret"] = validated_secret
|
||||
return validated_attrs
|
||||
|
||||
|
||||
@@ -1997,7 +2021,11 @@ class ProviderSecretUpdateSerializer(BaseWriteProviderSecretSerializer):
|
||||
secret = attrs.get("secret")
|
||||
|
||||
validated_attrs = super().validate(attrs)
|
||||
self.validate_secret_based_on_provider(provider.provider, secret_type, secret)
|
||||
validated_secret = self.validate_secret_based_on_provider(
|
||||
provider.provider, secret_type, secret
|
||||
)
|
||||
if provider.provider == Provider.ProviderChoices.ORACLECLOUD.value:
|
||||
validated_attrs["secret"] = validated_secret
|
||||
return validated_attrs
|
||||
|
||||
|
||||
@@ -2716,6 +2744,37 @@ class ScheduleDailyCreateSerializer(BaseSerializerV1):
|
||||
# Integrations
|
||||
|
||||
|
||||
class IntegrationProviderVisibilityMixin:
|
||||
"""
|
||||
Keep the `providers` relationship within the provider visibility of the role.
|
||||
|
||||
The view injects `allowed_providers` in the serializer context: `None` when the role
|
||||
has unlimited visibility, and the queryset of visible providers otherwise. Roles with
|
||||
limited visibility can neither attach providers they cannot see nor discover, through
|
||||
the serialized output, the ones already attached.
|
||||
"""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
allowed_providers = self.context.get("allowed_providers")
|
||||
if allowed_providers is not None:
|
||||
self.fields["providers"].child_relation.queryset = allowed_providers
|
||||
|
||||
def hide_restricted_providers(self, representation: dict) -> dict:
|
||||
allowed_providers = self.context.get("allowed_providers")
|
||||
# `providers` is missing when the request asks for a subset of the fields
|
||||
if allowed_providers is None or "providers" not in representation:
|
||||
return representation
|
||||
|
||||
allowed_provider_ids = {str(provider.id) for provider in allowed_providers}
|
||||
representation["providers"] = [
|
||||
provider
|
||||
for provider in representation["providers"]
|
||||
if provider["id"] in allowed_provider_ids
|
||||
]
|
||||
return representation
|
||||
|
||||
|
||||
class BaseWriteIntegrationSerializer(BaseWriteSerializer):
|
||||
def validate(self, attrs):
|
||||
integration_type = attrs.get("integration_type")
|
||||
@@ -2848,7 +2907,7 @@ class BaseWriteIntegrationSerializer(BaseWriteSerializer):
|
||||
)
|
||||
|
||||
|
||||
class IntegrationSerializer(RLSSerializer):
|
||||
class IntegrationSerializer(IntegrationProviderVisibilityMixin, RLSSerializer):
|
||||
"""
|
||||
Serializer for the Integration model.
|
||||
"""
|
||||
@@ -2877,15 +2936,9 @@ class IntegrationSerializer(RLSSerializer):
|
||||
}
|
||||
|
||||
def to_representation(self, instance):
|
||||
representation = super().to_representation(instance)
|
||||
allowed_providers = self.context.get("allowed_providers")
|
||||
if allowed_providers:
|
||||
allowed_provider_ids = {str(provider.id) for provider in allowed_providers}
|
||||
representation["providers"] = [
|
||||
provider
|
||||
for provider in representation["providers"]
|
||||
if provider["id"] in allowed_provider_ids
|
||||
]
|
||||
representation = self.hide_restricted_providers(
|
||||
super().to_representation(instance)
|
||||
)
|
||||
if instance.integration_type == Integration.IntegrationChoices.JIRA:
|
||||
representation["configuration"].update(
|
||||
{"domain": instance.credentials.get("domain")}
|
||||
@@ -2893,7 +2946,9 @@ class IntegrationSerializer(RLSSerializer):
|
||||
return representation
|
||||
|
||||
|
||||
class IntegrationCreateSerializer(BaseWriteIntegrationSerializer):
|
||||
class IntegrationCreateSerializer(
|
||||
IntegrationProviderVisibilityMixin, BaseWriteIntegrationSerializer
|
||||
):
|
||||
credentials = IntegrationCredentialField(write_only=True)
|
||||
configuration = IntegrationConfigField()
|
||||
providers = serializers.ResourceRelatedField(
|
||||
@@ -2944,22 +2999,18 @@ class IntegrationCreateSerializer(BaseWriteIntegrationSerializer):
|
||||
tenant_id = self.context.get("tenant_id")
|
||||
|
||||
providers = validated_data.pop("providers", [])
|
||||
integration = Integration.objects.create(tenant_id=tenant_id, **validated_data)
|
||||
|
||||
through_model_instances = [
|
||||
IntegrationProviderRelationship(
|
||||
integration=integration,
|
||||
provider=provider,
|
||||
tenant_id=tenant_id,
|
||||
with transaction.atomic():
|
||||
integration = Integration.objects.create(
|
||||
tenant_id=tenant_id, **validated_data
|
||||
)
|
||||
for provider in providers
|
||||
]
|
||||
IntegrationProviderRelationship.objects.bulk_create(through_model_instances)
|
||||
replace_integration_providers(integration, providers, tenant_id)
|
||||
|
||||
return integration
|
||||
|
||||
|
||||
class IntegrationUpdateSerializer(BaseWriteIntegrationSerializer):
|
||||
class IntegrationUpdateSerializer(
|
||||
IntegrationProviderVisibilityMixin, BaseWriteIntegrationSerializer
|
||||
):
|
||||
credentials = IntegrationCredentialField(write_only=True, required=False)
|
||||
configuration = IntegrationConfigField(required=False)
|
||||
providers = serializers.ResourceRelatedField(
|
||||
@@ -3004,15 +3055,13 @@ class IntegrationUpdateSerializer(BaseWriteIntegrationSerializer):
|
||||
|
||||
def update(self, instance, validated_data):
|
||||
tenant_id = self.context.get("tenant_id")
|
||||
if validated_data.get("providers") is not None:
|
||||
instance.providers.clear()
|
||||
new_relationships = [
|
||||
IntegrationProviderRelationship(
|
||||
integration=instance, provider=provider, tenant_id=tenant_id
|
||||
)
|
||||
for provider in validated_data["providers"]
|
||||
]
|
||||
IntegrationProviderRelationship.objects.bulk_create(new_relationships)
|
||||
# Relationships are replaced here, so they are kept out of the default
|
||||
# `ModelSerializer.update()`, which would otherwise reset them all. The view
|
||||
# rejects updates on integrations shared with providers hidden to the role, so
|
||||
# every existing relationship is visible to the requester at this point
|
||||
providers = validated_data.pop("providers", None)
|
||||
if providers is not None:
|
||||
replace_integration_providers(instance, providers, tenant_id)
|
||||
|
||||
# Preserve regions field for Security Hub integrations
|
||||
if instance.integration_type == Integration.IntegrationChoices.AWS_SECURITY_HUB:
|
||||
@@ -3024,7 +3073,9 @@ class IntegrationUpdateSerializer(BaseWriteIntegrationSerializer):
|
||||
return super().update(instance, validated_data)
|
||||
|
||||
def to_representation(self, instance):
|
||||
representation = super().to_representation(instance)
|
||||
representation = self.hide_restricted_providers(
|
||||
super().to_representation(instance)
|
||||
)
|
||||
# Ensure JIRA integrations show updated domain in configuration from credentials
|
||||
if instance.integration_type == Integration.IntegrationChoices.JIRA:
|
||||
representation["configuration"].update(
|
||||
|
||||
@@ -124,7 +124,12 @@ from api.models import (
|
||||
UserRoleRelationship,
|
||||
)
|
||||
from api.pagination import ComplianceOverviewPagination
|
||||
from api.rbac.permissions import Permissions, get_providers, get_role
|
||||
from api.rbac.permissions import (
|
||||
Permissions,
|
||||
get_integrations,
|
||||
get_providers,
|
||||
get_role,
|
||||
)
|
||||
from api.renderers import APIJSONRenderer, PlainTextRenderer
|
||||
from api.rls import Tenant
|
||||
from api.utils import (
|
||||
@@ -281,6 +286,7 @@ from django.shortcuts import redirect
|
||||
from django.urls import reverse
|
||||
from django.utils.dateparse import parse_date
|
||||
from django.utils.decorators import method_decorator
|
||||
from django.utils.functional import cached_property
|
||||
from django.views.decorators.cache import cache_control
|
||||
from django_celery_beat.models import PeriodicTask
|
||||
from drf_spectacular.settings import spectacular_settings
|
||||
@@ -6652,27 +6658,34 @@ class ScheduleViewSet(BaseRLSViewSet):
|
||||
list=extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="List all integrations",
|
||||
description="Retrieve a list of all configured integrations with options for filtering by various criteria.",
|
||||
description="Retrieve a list of all configured integrations with options for filtering by various criteria.\n\n"
|
||||
"Integrations attached to one or more providers are only returned when the role can access at least one of "
|
||||
"those providers, and each integration lists only the providers visible to the role. Integrations not "
|
||||
"attached to any provider, such as Jira, are tenant-wide and are returned for every role.",
|
||||
),
|
||||
retrieve=extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Retrieve integration details",
|
||||
description="Fetch detailed information about a specific integration by its ID.",
|
||||
description="Fetch detailed information about a specific integration by its ID. Integrations outside the "
|
||||
"provider visibility of the role are reported the same way as one that does not exist.",
|
||||
),
|
||||
create=extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Create a new integration",
|
||||
description="Register a new integration with the system, providing necessary configuration details.",
|
||||
description="Register a new integration with the system, providing necessary configuration details. Only "
|
||||
"providers visible to the role can be attached to the integration.",
|
||||
),
|
||||
partial_update=extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Partially update an integration",
|
||||
description="Modify certain fields of an existing integration without affecting other settings.",
|
||||
description="Modify certain fields of an existing integration without affecting other settings. Integrations "
|
||||
"attached to providers outside the visibility of the role cannot be modified by it.",
|
||||
),
|
||||
destroy=extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Delete an integration",
|
||||
description="Remove an integration from the system by its ID.",
|
||||
description="Remove an integration from the system by its ID. Integrations attached to providers outside "
|
||||
"the visibility of the role cannot be deleted by it.",
|
||||
),
|
||||
)
|
||||
@method_decorator(CACHE_DECORATOR, name="list")
|
||||
@@ -6685,18 +6698,27 @@ class IntegrationViewSet(BaseRLSViewSet):
|
||||
ordering = ["integration_type", "-inserted_at"]
|
||||
# RBAC required permissions
|
||||
required_permissions = [Permissions.MANAGE_INTEGRATIONS]
|
||||
allowed_providers = None
|
||||
|
||||
@cached_property
|
||||
def allowed_providers(self):
|
||||
"""
|
||||
Providers the role can access, or None when it has unlimited visibility.
|
||||
|
||||
Resolved per request and independently of the action, so that writes are scoped
|
||||
as tightly as reads.
|
||||
"""
|
||||
if self.user_role.unlimited_visibility:
|
||||
return None
|
||||
return get_providers(self.user_role)
|
||||
|
||||
def get_queryset(self):
|
||||
user_roles = get_role(self.request.user, self.request.tenant_id)
|
||||
if user_roles.unlimited_visibility:
|
||||
# User has unlimited visibility, return all integrations
|
||||
queryset = Integration.objects.filter(tenant_id=self.request.tenant_id)
|
||||
else:
|
||||
# User lacks permission, filter providers based on provider groups associated with the role
|
||||
allowed_providers = get_providers(user_roles)
|
||||
queryset = Integration.objects.filter(providers__in=allowed_providers)
|
||||
self.allowed_providers = allowed_providers
|
||||
queryset = get_integrations(self.user_role, providers=self.allowed_providers)
|
||||
if self.allowed_providers is not None and self.action in ("list", "retrieve"):
|
||||
# Restrict the relationship itself, so that the providers hidden to the role
|
||||
# are left out of the sideloaded resources of `?include=providers` too
|
||||
queryset = queryset.prefetch_related(
|
||||
Prefetch("providers", queryset=self.allowed_providers)
|
||||
)
|
||||
return queryset
|
||||
|
||||
def get_serializer_class(self):
|
||||
@@ -6711,16 +6733,33 @@ class IntegrationViewSet(BaseRLSViewSet):
|
||||
context["allowed_providers"] = self.allowed_providers
|
||||
return context
|
||||
|
||||
def get_object(self):
|
||||
instance = super().get_object()
|
||||
# Writes on an integration shared with providers hidden to the role would reach
|
||||
# beyond its visibility, so both editing and deleting are rejected consistently
|
||||
if (
|
||||
self.action in ("partial_update", "destroy")
|
||||
and self.allowed_providers is not None
|
||||
and instance.providers.exclude(
|
||||
id__in=self.allowed_providers.values("id")
|
||||
).exists()
|
||||
):
|
||||
raise PermissionDenied(
|
||||
"The integration is attached to providers outside the visibility of your role."
|
||||
)
|
||||
return instance
|
||||
|
||||
@extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Check integration connection",
|
||||
description="Try to verify integration connection",
|
||||
description="Try to verify integration connection. Integrations outside the provider visibility of the role "
|
||||
"are reported the same way as one that does not exist.",
|
||||
request=None,
|
||||
responses={202: OpenApiResponse(response=TaskSerializer)},
|
||||
)
|
||||
@action(detail=True, methods=["post"], url_name="connection")
|
||||
def connection(self, request, pk=None):
|
||||
get_object_or_404(Integration, pk=pk)
|
||||
get_object_or_404(self.get_queryset(), pk=pk)
|
||||
with transaction.atomic():
|
||||
task = check_integration_connection_task.delay(
|
||||
integration_id=pk, tenant_id=self.request.tenant_id
|
||||
@@ -6743,7 +6782,8 @@ class IntegrationViewSet(BaseRLSViewSet):
|
||||
tags=["Integration"],
|
||||
summary="Send findings to a Jira integration",
|
||||
description="Send a set of filtered findings to the given integration. At least one finding filter must be "
|
||||
"provided.\n\n"
|
||||
"provided. Jira integrations are tenant-wide and do not require unlimited visibility, while the findings "
|
||||
"sent are limited to the providers the role can access.\n\n"
|
||||
"## Known Limitations\n\n"
|
||||
"### Issue Types with Required Custom Fields\n\n"
|
||||
"Certain Jira issue types (such as Epic) may require mandatory custom fields that Prowler does not "
|
||||
@@ -6787,24 +6827,37 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
|
||||
return []
|
||||
return super().get_filter_backends()
|
||||
|
||||
def get_queryset(self):
|
||||
tenant_id = self.request.tenant_id
|
||||
user_roles = get_role(self.request.user, self.request.tenant_id)
|
||||
if user_roles.unlimited_visibility:
|
||||
# User has unlimited visibility, return all findings
|
||||
queryset = Finding.all_objects.filter(tenant_id=tenant_id)
|
||||
else:
|
||||
# User lacks permission, filter findings based on provider groups associated with the role
|
||||
queryset = Finding.all_objects.filter(
|
||||
scan__provider__in=get_providers(user_roles)
|
||||
)
|
||||
@cached_property
|
||||
def allowed_providers(self):
|
||||
"""
|
||||
Providers the role can access, or None when it has unlimited visibility.
|
||||
|
||||
return queryset
|
||||
Resolved once per request and shared between the findings queryset and the
|
||||
integration lookup.
|
||||
"""
|
||||
if self.user_role.unlimited_visibility:
|
||||
return None
|
||||
return get_providers(self.user_role)
|
||||
|
||||
def get_queryset(self):
|
||||
if self.allowed_providers is None:
|
||||
# User has unlimited visibility, return all findings
|
||||
return Finding.all_objects.filter(tenant_id=self.request.tenant_id)
|
||||
# Findings are limited to the providers the role can access
|
||||
return Finding.all_objects.filter(scan__provider__in=self.allowed_providers)
|
||||
|
||||
def get_integration(self, integration_pk):
|
||||
"""Retrieve the integration, honoring the provider visibility of the user's role."""
|
||||
return get_object_or_404(
|
||||
get_integrations(self.user_role, providers=self.allowed_providers),
|
||||
pk=integration_pk,
|
||||
)
|
||||
|
||||
@extend_schema(
|
||||
tags=["Integration"],
|
||||
summary="Get available issue types for a Jira project",
|
||||
description="Fetch the available issue types from Jira for a given project key and update the integration configuration.",
|
||||
description="Fetch the available issue types from Jira for a given project key and update the integration "
|
||||
"configuration. Jira integrations are tenant-wide and do not require unlimited visibility.",
|
||||
parameters=[
|
||||
OpenApiParameter(
|
||||
name="project_key",
|
||||
@@ -6817,7 +6870,7 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
|
||||
)
|
||||
@action(detail=False, methods=["get"], url_name="issue-types")
|
||||
def issue_types(self, request, integration_pk=None):
|
||||
integration = get_object_or_404(Integration, pk=integration_pk)
|
||||
integration = self.get_integration(integration_pk)
|
||||
|
||||
project_key = request.query_params.get("project_key")
|
||||
if not project_key:
|
||||
@@ -6862,23 +6915,23 @@ class IntegrationJiraViewSet(BaseRLSViewSet):
|
||||
|
||||
@action(detail=False, methods=["post"], url_name="dispatches")
|
||||
def dispatches(self, request, integration_pk=None):
|
||||
get_object_or_404(Integration, pk=integration_pk)
|
||||
self.get_integration(integration_pk)
|
||||
serializer = self.get_serializer(
|
||||
data=request.data, context={"integration_id": integration_pk}
|
||||
)
|
||||
serializer.is_valid(raise_exception=True)
|
||||
|
||||
if self.filter_queryset(self.get_queryset()).count() == 0:
|
||||
raise ValidationError(
|
||||
{"findings": "No findings match the provided filters"}
|
||||
)
|
||||
|
||||
finding_ids = [
|
||||
str(finding_id)
|
||||
for finding_id in self.filter_queryset(self.get_queryset()).values_list(
|
||||
"id", flat=True
|
||||
)
|
||||
]
|
||||
if not finding_ids:
|
||||
raise ValidationError(
|
||||
{"findings": "No findings match the provided filters"}
|
||||
)
|
||||
|
||||
project_key = serializer.validated_data["project_key"]
|
||||
issue_type = serializer.validated_data["issue_type"]
|
||||
|
||||
|
||||
@@ -312,8 +312,8 @@ ATTACK_PATHS_SCAN_INACTIVITY_THRESHOLD_MINUTES = env.int(
|
||||
"ATTACK_PATHS_SCAN_INACTIVITY_THRESHOLD_MINUTES", 30
|
||||
)
|
||||
ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES = env.int(
|
||||
"ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES", 2880
|
||||
) # 48h
|
||||
"ATTACK_PATHS_SCAN_STALE_THRESHOLD_MINUTES", 960
|
||||
) # 16h
|
||||
|
||||
# Selects where the persistent attack-paths graph is stored. The scan
|
||||
# temporary database is always Neo4j; only the sink is configurable.
|
||||
|
||||
@@ -91,6 +91,13 @@ def before_send(event, hint):
|
||||
log_msg = log_record.getMessage()
|
||||
log_lvl = log_record.levelno
|
||||
|
||||
if (
|
||||
getattr(log_record, "name", "") == "cartography.graph.job"
|
||||
and "Neo.ClientError.Database.DatabaseNotFound" in log_msg
|
||||
and "db-tmp-scan-" in log_msg
|
||||
):
|
||||
return None
|
||||
|
||||
# The Neo4j driver logs transient connection errors (defunct
|
||||
# connections, resets) at ERROR level via the `neo4j.io` logger.
|
||||
# `RetryableSession` handles these with retries. If all retries
|
||||
|
||||
@@ -8,6 +8,8 @@ import aioboto3
|
||||
import boto3
|
||||
import botocore
|
||||
import neo4j
|
||||
import neo4j.exceptions
|
||||
from api.attack_paths.database import DATABASE_NOT_FOUND_CODE
|
||||
from api.models import (
|
||||
AttackPathsScan as ProwlerAPIAttackPathsScan,
|
||||
)
|
||||
@@ -347,6 +349,12 @@ def sync_aws_account(
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
if (
|
||||
isinstance(e, neo4j.exceptions.Neo4jError)
|
||||
and e.code == DATABASE_NOT_FOUND_CODE
|
||||
):
|
||||
raise
|
||||
|
||||
logger.info(
|
||||
f"Synced function {func_name} for AWS account {prowler_api_provider.uid} in {time.perf_counter() - func_t0:.3f}s (FAILED)"
|
||||
)
|
||||
|
||||
@@ -10,13 +10,10 @@ NormalizedList = _provider_config.NormalizedList
|
||||
PROVIDER_CONFIGS = _provider_config.PROVIDER_CONFIGS
|
||||
ProviderConfig = _provider_config.ProviderConfig
|
||||
|
||||
# Batch size for Neo4j write operations (resource labeling, cleanup)
|
||||
BATCH_SIZE = env.int("ATTACK_PATHS_BATCH_SIZE", 1000)
|
||||
# Batch size for graph mutation operations (resource labeling and subgraph deletion)
|
||||
GRAPH_MUTATION_BATCH_SIZE = env.int("ATTACK_PATHS_GRAPH_MUTATION_BATCH_SIZE", 1000)
|
||||
# Batch size for Postgres findings fetch (keyset pagination page size)
|
||||
FINDINGS_BATCH_SIZE = env.int("ATTACK_PATHS_FINDINGS_BATCH_SIZE", 1000)
|
||||
# Batch size for temp-to-tenant graph sync (nodes and relationships per cursor page)
|
||||
SYNC_BATCH_SIZE = env.int("ATTACK_PATHS_SYNC_BATCH_SIZE", 1000)
|
||||
|
||||
# Neo4j internal labels (Prowler-specific, not provider-specific)
|
||||
# - `Internet`: Singleton node representing external internet access for exposed-resource queries
|
||||
# - `ProwlerFinding`: Label for finding nodes created by Prowler and linked to cloud resources
|
||||
|
||||
@@ -21,8 +21,8 @@ from cartography.config import Config as CartographyConfig
|
||||
from celery.utils.log import get_task_logger
|
||||
from prowler.config import config as ProwlerConfig
|
||||
from tasks.jobs.attack_paths.config import (
|
||||
BATCH_SIZE,
|
||||
FINDINGS_BATCH_SIZE,
|
||||
GRAPH_MUTATION_BATCH_SIZE,
|
||||
get_node_uid_field,
|
||||
get_provider_resource_label,
|
||||
get_root_node_label,
|
||||
@@ -135,7 +135,7 @@ def add_resource_label(
|
||||
while labeled_count > 0:
|
||||
result = neo4j_session.run(
|
||||
query,
|
||||
{"provider_uid": provider_uid, "batch_size": BATCH_SIZE},
|
||||
{"provider_uid": provider_uid, "batch_size": GRAPH_MUTATION_BATCH_SIZE},
|
||||
)
|
||||
labeled_count = result.single().get("labeled_count", 0)
|
||||
total_labeled += labeled_count
|
||||
|
||||
@@ -372,7 +372,19 @@ def run(tenant_id: str, scan_id: str, task_id: str) -> dict[str, Any]:
|
||||
|
||||
except Exception as e:
|
||||
exception_message = utils.stringify_exception(e, "Attack Paths scan failed")
|
||||
logger.exception(exception_message)
|
||||
temporary_database_missing = (
|
||||
isinstance(e, graph_database.GraphDatabaseQueryException)
|
||||
and e.code == graph_database.DATABASE_NOT_FOUND_CODE
|
||||
and tmp_database_name in str(e)
|
||||
)
|
||||
if temporary_database_missing:
|
||||
logger.warning(exception_message)
|
||||
else:
|
||||
logger.exception(exception_message)
|
||||
cleanup_log_level = (
|
||||
logging.WARNING if temporary_database_missing else logging.ERROR
|
||||
)
|
||||
cleanup_exc_info = not temporary_database_missing
|
||||
ingestion_exceptions["global_error"] = exception_message
|
||||
|
||||
# Recover `graph_data_ready` based on how far the swap got
|
||||
@@ -387,19 +399,24 @@ def run(tenant_id: str, scan_id: str, task_id: str) -> dict[str, Any]:
|
||||
)
|
||||
|
||||
except Exception:
|
||||
logger.error(
|
||||
f"Failed to recover `graph_data_ready` for provider {attack_paths_scan.provider_id}",
|
||||
exc_info=True,
|
||||
logger.log(
|
||||
cleanup_log_level,
|
||||
"Failed to recover `graph_data_ready` for provider "
|
||||
f"{attack_paths_scan.provider_id}",
|
||||
exc_info=cleanup_exc_info,
|
||||
)
|
||||
|
||||
# Dropping the temporary database if it still exists
|
||||
try:
|
||||
graph_database.drop_database(tmp_cartography_config.neo4j_database)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to drop temporary Neo4j database `{tmp_cartography_config.neo4j_database}` during cleanup: {e}",
|
||||
exc_info=True,
|
||||
except Exception as cleanup_error:
|
||||
logger.log(
|
||||
cleanup_log_level,
|
||||
"Failed to drop temporary Neo4j database "
|
||||
f"`{tmp_cartography_config.neo4j_database}` during cleanup: "
|
||||
f"{cleanup_error}",
|
||||
exc_info=cleanup_exc_info,
|
||||
)
|
||||
|
||||
# Set Attack Paths scan state to FAILED
|
||||
@@ -407,10 +424,12 @@ def run(tenant_id: str, scan_id: str, task_id: str) -> dict[str, Any]:
|
||||
db_utils.finish_attack_paths_scan(
|
||||
attack_paths_scan, StateChoices.FAILED, ingestion_exceptions
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Could not mark Attack Paths scan {attack_paths_scan.id} as `FAILED` (row may have been deleted): {e}",
|
||||
exc_info=True,
|
||||
except Exception as cleanup_error:
|
||||
logger.log(
|
||||
cleanup_log_level,
|
||||
f"Could not mark Attack Paths scan {attack_paths_scan.id} as `FAILED` "
|
||||
f"(row may have been deleted): {cleanup_error}",
|
||||
exc_info=cleanup_exc_info,
|
||||
)
|
||||
|
||||
raise
|
||||
|
||||
@@ -30,7 +30,6 @@ from tasks.jobs.attack_paths.config import (
|
||||
PROVIDER_CONFIGS,
|
||||
PROVIDER_ISOLATION_PROPERTIES,
|
||||
PROVIDER_RESOURCE_LABEL,
|
||||
SYNC_BATCH_SIZE,
|
||||
NormalizedList,
|
||||
get_provider_label,
|
||||
get_tenant_label,
|
||||
@@ -116,6 +115,7 @@ def sync_nodes(
|
||||
Source and target sessions are opened sequentially per batch to avoid
|
||||
holding two Bolt connections simultaneously for the entire sync duration.
|
||||
"""
|
||||
batch_size = sink.sync_batch_size
|
||||
t0 = time.perf_counter()
|
||||
last_id = -1
|
||||
parents_synced = 0
|
||||
@@ -137,7 +137,7 @@ def sync_nodes(
|
||||
with graph_database.get_session(source_database) as source_session:
|
||||
result = source_session.run(
|
||||
NODE_FETCH_QUERY,
|
||||
{"last_id": last_id, "batch_size": SYNC_BATCH_SIZE},
|
||||
{"last_id": last_id, "batch_size": batch_size},
|
||||
)
|
||||
for record in result:
|
||||
batch_count += 1
|
||||
@@ -156,17 +156,17 @@ def sync_nodes(
|
||||
|
||||
for labels, batch in parent_groups.items():
|
||||
rendered_labels = _render_labels(labels, extra_labels)
|
||||
for sink_batch in _iter_sink_batches(batch):
|
||||
for sink_batch in _iter_sink_batches(batch, batch_size):
|
||||
sink.write_nodes(target_database, rendered_labels, sink_batch)
|
||||
|
||||
for child_label, batch in child_groups.items():
|
||||
rendered_labels = _render_labels((child_label,), extra_labels)
|
||||
for sink_batch in _iter_sink_batches(batch):
|
||||
for sink_batch in _iter_sink_batches(batch, batch_size):
|
||||
sink.write_nodes(target_database, rendered_labels, sink_batch)
|
||||
children_synced += len(batch)
|
||||
|
||||
for rel_type, batch in rel_groups.items():
|
||||
for sink_batch in _iter_sink_batches(batch):
|
||||
for sink_batch in _iter_sink_batches(batch, batch_size):
|
||||
sink.write_relationships(
|
||||
target_database, rel_type, provider_id, sink_batch
|
||||
)
|
||||
@@ -205,6 +205,7 @@ def sync_relationships(
|
||||
Source and target sessions are opened sequentially per batch to avoid
|
||||
holding two Bolt connections simultaneously for the entire sync duration.
|
||||
"""
|
||||
batch_size = sink.sync_batch_size
|
||||
t0 = time.perf_counter()
|
||||
last_id = -1
|
||||
total_synced = 0
|
||||
@@ -217,7 +218,7 @@ def sync_relationships(
|
||||
with graph_database.get_session(source_database) as source_session:
|
||||
result = source_session.run(
|
||||
RELATIONSHIPS_FETCH_QUERY,
|
||||
{"last_id": last_id, "batch_size": SYNC_BATCH_SIZE},
|
||||
{"last_id": last_id, "batch_size": batch_size},
|
||||
)
|
||||
for record in result:
|
||||
batch_count += 1
|
||||
@@ -229,7 +230,7 @@ def sync_relationships(
|
||||
break
|
||||
|
||||
for rel_type, batch in grouped.items():
|
||||
for sink_batch in _iter_sink_batches(batch):
|
||||
for sink_batch in _iter_sink_batches(batch, batch_size):
|
||||
sink.write_relationships(
|
||||
target_database, rel_type, provider_id, sink_batch
|
||||
)
|
||||
@@ -247,10 +248,9 @@ def sync_relationships(
|
||||
|
||||
def _iter_sink_batches(
|
||||
rows: list[dict[str, Any]],
|
||||
batch_size: int | None = None,
|
||||
batch_size: int,
|
||||
) -> Iterator[list[dict[str, Any]]]:
|
||||
"""Yield final sink write batches after source rows have been transformed."""
|
||||
batch_size = SYNC_BATCH_SIZE if batch_size is None else batch_size
|
||||
if batch_size <= 0:
|
||||
raise ValueError("Sink batch size must be greater than zero")
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ import re
|
||||
import time
|
||||
import uuid
|
||||
from collections import defaultdict
|
||||
from collections.abc import Iterable
|
||||
from collections.abc import Callable, Iterable
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
@@ -49,6 +49,7 @@ from celery.utils.log import get_task_logger
|
||||
from config.django.base import DJANGO_FINDINGS_BATCH_SIZE
|
||||
from config.env import env
|
||||
from config.settings.celery import CELERY_DEADLOCK_ATTEMPTS
|
||||
from django.core.exceptions import ImproperlyConfigured
|
||||
from django.db import DatabaseError, IntegrityError, OperationalError, transaction
|
||||
from django.db.models import (
|
||||
Case,
|
||||
@@ -99,6 +100,16 @@ COMPLIANCE_REQUIREMENT_COPY_COLUMNS = (
|
||||
FINDINGS_MICRO_BATCH_SIZE = env.int("DJANGO_FINDINGS_MICRO_BATCH_SIZE", default=3000)
|
||||
# Controls how many rows each ORM bulk_create/bulk_update call sends to Postgres.
|
||||
SCAN_DB_BATCH_SIZE = env.int("DJANGO_SCAN_DB_BATCH_SIZE", default=1000)
|
||||
# Rows per COPY statement when ingesting compliance requirement overviews. All
|
||||
# batches of a scan share one transaction/commit; the batch size only bounds the
|
||||
# client-side CSV buffer and how long each individual COPY statement runs on the
|
||||
# writer (memory footprint, lock time and slow-statement logging under load).
|
||||
COMPLIANCE_COPY_BATCH_SIZE = env.int("DJANGO_COMPLIANCE_COPY_BATCH_SIZE", default=2000)
|
||||
if COMPLIANCE_COPY_BATCH_SIZE < 1:
|
||||
raise ImproperlyConfigured(
|
||||
"DJANGO_COMPLIANCE_COPY_BATCH_SIZE must be a positive integer, got "
|
||||
f"{COMPLIANCE_COPY_BATCH_SIZE}"
|
||||
)
|
||||
# Throttle scan progress persistence: minimum progress delta (fraction 0-1)
|
||||
# between two persisted progress updates.
|
||||
PROGRESS_THROTTLE_DELTA = env.float("DJANGO_SCAN_PROGRESS_THROTTLE_DELTA", default=0.01)
|
||||
@@ -356,30 +367,36 @@ def _bulk_update_resource_failed_findings_counts(
|
||||
raise
|
||||
|
||||
|
||||
def _copy_compliance_requirement_rows(
|
||||
tenant_id: str, rows: list[dict[str, Any]]
|
||||
) -> None:
|
||||
"""Stream compliance requirement rows into Postgres using COPY.
|
||||
class ComplianceRowScopeError(ValueError):
|
||||
"""A compliance requirement row does not belong to the scan being ingested."""
|
||||
|
||||
We leverage the admin connection (when available) to bypass the COPY + RLS
|
||||
restriction, writing only the fields required by
|
||||
``ComplianceRequirementOverview``.
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
rows: List of row dictionaries prepared by
|
||||
:func:`create_compliance_requirements`.
|
||||
def _compliance_requirement_rows_to_csv(
|
||||
rows: list[dict[str, Any]], tenant_id: str, scan_id: str
|
||||
) -> io.StringIO:
|
||||
"""Serialize compliance requirement rows into a CSV buffer for COPY.
|
||||
|
||||
COPY runs on the admin connection, which bypasses RLS, so every row is
|
||||
checked against the expected tenant/scan before it is written: a mismatched
|
||||
row would otherwise be inserted verbatim into another tenant's data.
|
||||
"""
|
||||
|
||||
csv_buffer = io.StringIO()
|
||||
writer = csv.writer(csv_buffer)
|
||||
|
||||
datetime_now = datetime.now(tz=UTC)
|
||||
for row in rows:
|
||||
row_tenant_id = str(row.get("tenant_id"))
|
||||
row_scan_id = str(row.get("scan_id"))
|
||||
if row_tenant_id != tenant_id or row_scan_id != scan_id:
|
||||
raise ComplianceRowScopeError(
|
||||
"Compliance requirement row does not belong to the scan being "
|
||||
f"ingested (expected tenant {tenant_id} / scan {scan_id}, got "
|
||||
f"tenant {row_tenant_id} / scan {row_scan_id})"
|
||||
)
|
||||
writer.writerow(
|
||||
[
|
||||
str(row.get("id")),
|
||||
str(row.get("tenant_id")),
|
||||
row_tenant_id,
|
||||
(row.get("inserted_at") or datetime_now).isoformat(),
|
||||
row.get("compliance_id") or "",
|
||||
row.get("framework") or "",
|
||||
@@ -393,65 +410,100 @@ def _copy_compliance_requirement_rows(
|
||||
row.get("total_checks", 0),
|
||||
row.get("passed_findings", 0),
|
||||
row.get("total_findings", 0),
|
||||
str(row.get("scan_id")),
|
||||
row_scan_id,
|
||||
]
|
||||
)
|
||||
|
||||
csv_buffer.seek(0)
|
||||
return csv_buffer
|
||||
|
||||
|
||||
def _copy_compliance_requirement_rows(
|
||||
tenant_id: str, scan_id: str, rows: Iterable[dict[str, Any]], batch_size: int
|
||||
) -> int:
|
||||
"""Replace a scan's compliance requirement rows using batched COPY.
|
||||
|
||||
We leverage the admin connection (when available) to bypass the COPY + RLS
|
||||
restriction. The scan's DELETE and every COPY batch run on one connection
|
||||
inside a single transaction with a single commit, so the writer takes one
|
||||
fsync per scan instead of one per batch, and a failed ingest rolls back
|
||||
without committing a partial delete/insert (which a retry would otherwise
|
||||
delete again, feeding dead rows to autovacuum).
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
scan_id: Scan whose previous rows are replaced.
|
||||
rows: Iterable of row dictionaries, consumed lazily batch by batch.
|
||||
batch_size: Number of rows per COPY statement.
|
||||
|
||||
Returns:
|
||||
int: total number of rows staged and committed.
|
||||
|
||||
Raises:
|
||||
ComplianceRowScopeError: A row belongs to another tenant or scan.
|
||||
"""
|
||||
# Normalized once so the per-row scope check compares like with like even if
|
||||
# the caller passes UUID instances instead of strings.
|
||||
tenant_id = str(tenant_id)
|
||||
scan_id = str(scan_id)
|
||||
total_rows = 0
|
||||
batch_num = 0
|
||||
copy_sql = (
|
||||
"COPY compliance_requirements_overviews ("
|
||||
+ ", ".join(COMPLIANCE_REQUIREMENT_COPY_COLUMNS)
|
||||
+ ") FROM STDIN WITH (FORMAT CSV, DELIMITER ',', QUOTE '\"', ESCAPE '\"', NULL '\\N')"
|
||||
)
|
||||
|
||||
try:
|
||||
with psycopg_connection(MainRouter.admin_db) as connection:
|
||||
connection.autocommit = False
|
||||
try:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.execute(SET_CONFIG_QUERY, [POSTGRES_TENANT_VAR, tenant_id])
|
||||
cursor.copy_expert(copy_sql, csv_buffer)
|
||||
connection.commit()
|
||||
except Exception:
|
||||
connection.rollback()
|
||||
raise
|
||||
finally:
|
||||
csv_buffer.close()
|
||||
with psycopg_connection(MainRouter.admin_db) as connection:
|
||||
connection.autocommit = False
|
||||
try:
|
||||
with connection.cursor() as cursor:
|
||||
cursor.execute(SET_CONFIG_QUERY, [POSTGRES_TENANT_VAR, tenant_id])
|
||||
# Idempotent re-run: clearing this scan's rows inside the same
|
||||
# transaction keeps delete + reinsert atomic.
|
||||
cursor.execute(
|
||||
"DELETE FROM compliance_requirements_overviews "
|
||||
"WHERE tenant_id = %s AND scan_id = %s",
|
||||
[tenant_id, scan_id],
|
||||
)
|
||||
for batch, _is_last in batched(rows, batch_size):
|
||||
if not batch:
|
||||
continue
|
||||
batch_num += 1
|
||||
csv_buffer = _compliance_requirement_rows_to_csv(
|
||||
batch, tenant_id, scan_id
|
||||
)
|
||||
try:
|
||||
cursor.copy_expert(copy_sql, csv_buffer)
|
||||
finally:
|
||||
csv_buffer.close()
|
||||
total_rows += len(batch)
|
||||
logger.info(
|
||||
f"Compliance COPY batch {batch_num}: staged {len(batch)} rows "
|
||||
f"({total_rows} total)"
|
||||
)
|
||||
connection.commit()
|
||||
except Exception:
|
||||
connection.rollback()
|
||||
raise
|
||||
|
||||
return total_rows
|
||||
|
||||
|
||||
def _persist_compliance_requirement_rows(
|
||||
tenant_id: str, rows: Iterable[dict[str, Any]], batch_size: int = 10000
|
||||
def _bulk_create_compliance_requirement_rows(
|
||||
tenant_id: str, scan_id: str, rows: Iterable[dict[str, Any]], batch_size: int
|
||||
) -> int:
|
||||
"""Persist compliance requirement rows using batched COPY with ORM fallback.
|
||||
"""Replace a scan's compliance requirement rows via the ORM.
|
||||
|
||||
``rows`` is consumed lazily in batches, so peak memory stays at ~``batch_size``
|
||||
rows instead of the full set. A batch that fails COPY falls back to an ORM
|
||||
``bulk_create`` of just that batch.
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
rows: Iterable of row dictionaries reflecting the compliance overview
|
||||
state for a scan.
|
||||
batch_size: Number of rows per COPY batch (default: 10000).
|
||||
|
||||
Returns:
|
||||
int: total number of rows persisted.
|
||||
Fallback for when COPY is unavailable; the delete and every ``bulk_create``
|
||||
share one RLS transaction so the replacement stays atomic.
|
||||
"""
|
||||
total_rows = 0
|
||||
batch_num = 0
|
||||
|
||||
for batch, _is_last in batched(rows, batch_size):
|
||||
if not batch:
|
||||
continue
|
||||
batch_num += 1
|
||||
try:
|
||||
_copy_compliance_requirement_rows(tenant_id, batch)
|
||||
except Exception as error:
|
||||
logger.exception(
|
||||
f"COPY bulk insert for compliance requirements batch {batch_num} "
|
||||
"failed; falling back to ORM bulk_create for this batch",
|
||||
exc_info=error,
|
||||
)
|
||||
with rls_transaction(tenant_id):
|
||||
ComplianceRequirementOverview.objects.filter(scan_id=scan_id).delete()
|
||||
for batch, _is_last in batched(rows, batch_size):
|
||||
if not batch:
|
||||
continue
|
||||
fallback_objects = [
|
||||
ComplianceRequirementOverview(
|
||||
id=row["id"],
|
||||
@@ -473,20 +525,58 @@ def _persist_compliance_requirement_rows(
|
||||
)
|
||||
for row in batch
|
||||
]
|
||||
with rls_transaction(tenant_id):
|
||||
ComplianceRequirementOverview.objects.bulk_create(
|
||||
fallback_objects, batch_size=500
|
||||
)
|
||||
|
||||
total_rows += len(batch)
|
||||
logger.info(
|
||||
f"Compliance COPY batch {batch_num}: inserted {len(batch)} rows "
|
||||
f"({total_rows} total)"
|
||||
)
|
||||
|
||||
ComplianceRequirementOverview.objects.bulk_create(
|
||||
fallback_objects, batch_size=500
|
||||
)
|
||||
total_rows += len(batch)
|
||||
return total_rows
|
||||
|
||||
|
||||
def _persist_compliance_requirement_rows(
|
||||
tenant_id: str,
|
||||
scan_id: str,
|
||||
rows_factory: Callable[[], Iterable[dict[str, Any]]],
|
||||
batch_size: int | None = None,
|
||||
) -> int:
|
||||
"""Persist a scan's compliance requirement rows, replacing any previous ones.
|
||||
|
||||
``rows_factory`` must return a fresh row iterator on every call: the COPY
|
||||
path consumes it lazily in batches (peak memory ~``batch_size`` rows), and
|
||||
if COPY fails the whole ingest falls back to a single ORM transaction that
|
||||
re-iterates the rows.
|
||||
|
||||
Args:
|
||||
tenant_id: Target tenant UUID.
|
||||
scan_id: Scan whose compliance overview rows are being replaced.
|
||||
rows_factory: Callable returning an iterable of row dictionaries.
|
||||
batch_size: Rows per COPY/bulk_create batch (default:
|
||||
``COMPLIANCE_COPY_BATCH_SIZE``).
|
||||
|
||||
Returns:
|
||||
int: total number of rows persisted.
|
||||
"""
|
||||
if batch_size is None:
|
||||
batch_size = COMPLIANCE_COPY_BATCH_SIZE
|
||||
|
||||
try:
|
||||
return _copy_compliance_requirement_rows(
|
||||
tenant_id, scan_id, rows_factory(), batch_size
|
||||
)
|
||||
except ComplianceRowScopeError:
|
||||
# Cross-tenant/scan rows are a bug in the caller, not a COPY failure:
|
||||
# retrying through the ORM would persist the very rows we rejected.
|
||||
raise
|
||||
except Exception as error:
|
||||
logger.exception(
|
||||
"COPY bulk insert for compliance requirements failed; "
|
||||
"falling back to ORM bulk_create",
|
||||
exc_info=error,
|
||||
)
|
||||
return _bulk_create_compliance_requirement_rows(
|
||||
tenant_id, scan_id, rows_factory(), batch_size
|
||||
)
|
||||
|
||||
|
||||
def _create_compliance_summaries(
|
||||
tenant_id: str, scan_id: str, requirement_statuses: dict
|
||||
) -> None:
|
||||
@@ -885,15 +975,19 @@ def _process_finding_micro_batch(
|
||||
# Denormalized resource arrays populated directly on insert
|
||||
# (was previously a separate bulk_update; saves a CASE WHEN
|
||||
# over thousands of rows per micro-batch).
|
||||
resource_regions=[resource_instance.region]
|
||||
if resource_instance.region
|
||||
else [],
|
||||
resource_services=[resource_instance.service]
|
||||
if resource_instance.service
|
||||
else [],
|
||||
resource_types=[resource_instance.type]
|
||||
if resource_instance.type
|
||||
else [],
|
||||
resource_regions=(
|
||||
[resource_instance.region]
|
||||
if resource_instance.region
|
||||
else []
|
||||
),
|
||||
resource_services=(
|
||||
[resource_instance.service]
|
||||
if resource_instance.service
|
||||
else []
|
||||
),
|
||||
resource_types=(
|
||||
[resource_instance.type] if resource_instance.type else []
|
||||
),
|
||||
)
|
||||
findings_to_create.append(finding_instance)
|
||||
resource_denormalized_data.append(
|
||||
@@ -1708,8 +1802,10 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
)
|
||||
|
||||
# Yield rows lazily (consumed batch-by-batch by COPY) so peak memory
|
||||
# stays bounded; tally requirement_statuses in the same pass.
|
||||
# stays bounded; tally requirement_statuses in the same pass. The
|
||||
# ORM fallback re-iterates from scratch, so the tally resets first.
|
||||
def _iter_compliance_requirement_rows():
|
||||
requirement_statuses.clear()
|
||||
for region in regions:
|
||||
region_stats = region_requirement_stats.get(region, {})
|
||||
region_findings = findings_count_by_compliance.get(region, {})
|
||||
@@ -1773,12 +1869,10 @@ def create_compliance_requirements(tenant_id: str, scan_id: str):
|
||||
"total_findings": total_findings,
|
||||
}
|
||||
|
||||
# Idempotent re-run: clear this scan's rows before re-inserting.
|
||||
with rls_transaction(tenant_id):
|
||||
ComplianceRequirementOverview.objects.filter(scan_id=scan_id).delete()
|
||||
|
||||
# The delete of the scan's previous rows happens inside the same
|
||||
# transaction as the inserts (see _copy_compliance_requirement_rows).
|
||||
requirements_created = _persist_compliance_requirement_rows(
|
||||
tenant_id, _iter_compliance_requirement_rows()
|
||||
tenant_id_str, scan_id_str, _iter_compliance_requirement_rows
|
||||
)
|
||||
|
||||
# Create pre-aggregated summaries for fast compliance overview lookups
|
||||
|
||||
@@ -11,6 +11,7 @@ from api.compliance import (
|
||||
from api.db_router import READ_REPLICA_ALIAS
|
||||
from api.db_utils import delete_related_daily_task, rls_transaction
|
||||
from api.decorators import handle_provider_deletion, set_tenant
|
||||
from api.exceptions import ProviderDeletedException
|
||||
from api.models import (
|
||||
Finding,
|
||||
Integration,
|
||||
@@ -666,7 +667,13 @@ class AttackPathsScanRLSTask(RLSTask):
|
||||
scan_id = kwargs.get("scan_id")
|
||||
|
||||
if tenant_id and scan_id:
|
||||
logger.error(f"Attack paths scan task {task_id} failed: {exc}")
|
||||
if isinstance(exc, ProviderDeletedException):
|
||||
logger.warning(
|
||||
f"Attack paths scan task {task_id} stopped because its provider "
|
||||
f"or tenant was deleted: {exc}"
|
||||
)
|
||||
else:
|
||||
logger.error(f"Attack paths scan task {task_id} failed: {exc}")
|
||||
attack_paths_db_utils.fail_attack_paths_scan(tenant_id, scan_id, str(exc))
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import neo4j.exceptions
|
||||
import pytest
|
||||
from tasks.jobs.attack_paths import aws
|
||||
|
||||
DATABASE_NOT_FOUND_CODE = "Neo.ClientError.Database.DatabaseNotFound"
|
||||
|
||||
|
||||
def _make_neo4j_error(code: str) -> neo4j.exceptions.Neo4jError:
|
||||
return neo4j.exceptions.Neo4jError._hydrate_neo4j(
|
||||
code=code,
|
||||
message="graph query failed",
|
||||
)
|
||||
|
||||
|
||||
def _resource_functions(failing_sync, following_sync):
|
||||
return {
|
||||
"failing_sync": failing_sync,
|
||||
"following_sync": following_sync,
|
||||
"permission_relationships": MagicMock(),
|
||||
"resourcegroupstaggingapi": MagicMock(),
|
||||
}
|
||||
|
||||
|
||||
def test_sync_aws_account_reraises_database_not_found_immediately():
|
||||
error = _make_neo4j_error(DATABASE_NOT_FOUND_CODE)
|
||||
failing_sync = MagicMock(side_effect=error)
|
||||
following_sync = MagicMock()
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
aws.cartography_aws,
|
||||
"RESOURCE_FUNCTIONS",
|
||||
_resource_functions(failing_sync, following_sync),
|
||||
),
|
||||
patch.object(aws.db_utils, "update_attack_paths_scan_progress"),
|
||||
patch.object(aws.utils, "stringify_exception") as stringify_exception,
|
||||
patch.object(aws.logger, "warning") as warning,
|
||||
pytest.raises(neo4j.exceptions.Neo4jError) as exc_info,
|
||||
):
|
||||
aws.sync_aws_account(
|
||||
SimpleNamespace(uid="123456789012"),
|
||||
[
|
||||
"failing_sync",
|
||||
"following_sync",
|
||||
"permission_relationships",
|
||||
"resourcegroupstaggingapi",
|
||||
],
|
||||
{},
|
||||
MagicMock(),
|
||||
)
|
||||
|
||||
assert exc_info.value is error
|
||||
following_sync.assert_not_called()
|
||||
stringify_exception.assert_not_called()
|
||||
warning.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"error",
|
||||
[
|
||||
_make_neo4j_error("Neo.ClientError.Statement.SyntaxError"),
|
||||
RuntimeError("resource sync failed"),
|
||||
],
|
||||
ids=["different-neo4j-error", "non-neo4j-error"],
|
||||
)
|
||||
def test_sync_aws_account_warns_and_continues_for_other_exceptions(error):
|
||||
failing_sync = MagicMock(side_effect=error)
|
||||
following_sync = MagicMock()
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
aws.cartography_aws,
|
||||
"RESOURCE_FUNCTIONS",
|
||||
_resource_functions(failing_sync, following_sync),
|
||||
),
|
||||
patch.object(aws.db_utils, "update_attack_paths_scan_progress"),
|
||||
patch.object(
|
||||
aws.utils,
|
||||
"stringify_exception",
|
||||
return_value="formatted failure",
|
||||
),
|
||||
patch.object(aws.logger, "warning") as warning,
|
||||
):
|
||||
failed_syncs = aws.sync_aws_account(
|
||||
SimpleNamespace(uid="123456789012"),
|
||||
[
|
||||
"failing_sync",
|
||||
"following_sync",
|
||||
"permission_relationships",
|
||||
"resourcegroupstaggingapi",
|
||||
],
|
||||
{},
|
||||
MagicMock(),
|
||||
)
|
||||
|
||||
assert failed_syncs == {"failing_sync": "formatted failure"}
|
||||
following_sync.assert_called_once_with()
|
||||
warning.assert_called_once()
|
||||
assert "Continuing to the next AWS sync function" in warning.call_args.args[0]
|
||||
@@ -1,3 +1,4 @@
|
||||
import logging
|
||||
from contextlib import nullcontext
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from types import SimpleNamespace
|
||||
@@ -5,7 +6,9 @@ from unittest.mock import MagicMock, call, patch
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from api.attack_paths.database import GraphDatabaseQueryException
|
||||
from api.db_utils import rls_transaction
|
||||
from api.exceptions import ProviderDeletedException
|
||||
from api.models import (
|
||||
AttackPathsScan,
|
||||
Finding,
|
||||
@@ -250,6 +253,32 @@ class TestAttackPathsRun:
|
||||
mock_starting.assert_not_called()
|
||||
mock_create_db.assert_not_called()
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("ingestion_error", "temporary_database_missing"),
|
||||
[
|
||||
(RuntimeError("ingestion boom"), False),
|
||||
(
|
||||
GraphDatabaseQueryException(
|
||||
message="Graph not found: db-scan-id",
|
||||
code="Neo.ClientError.Database.DatabaseNotFound",
|
||||
),
|
||||
True,
|
||||
),
|
||||
(
|
||||
GraphDatabaseQueryException(
|
||||
message="Graph not found: db-tenant-id",
|
||||
code="Neo.ClientError.Database.DatabaseNotFound",
|
||||
),
|
||||
False,
|
||||
),
|
||||
],
|
||||
ids=[
|
||||
"regular-error",
|
||||
"temporary-database-missing",
|
||||
"sink-database-missing",
|
||||
],
|
||||
)
|
||||
@patch("tasks.jobs.attack_paths.scan.logger")
|
||||
@patch(
|
||||
"tasks.jobs.attack_paths.scan.utils.stringify_exception",
|
||||
return_value="Cartography failed: ingestion boom",
|
||||
@@ -302,6 +331,9 @@ class TestAttackPathsRun:
|
||||
mock_drop_db,
|
||||
mock_event_loop,
|
||||
mock_stringify,
|
||||
mock_logger,
|
||||
ingestion_error,
|
||||
temporary_database_missing,
|
||||
tenants_fixture,
|
||||
aws_provider,
|
||||
scans_fixture,
|
||||
@@ -321,7 +353,11 @@ class TestAttackPathsRun:
|
||||
session_ctx = MagicMock()
|
||||
session_ctx.__enter__.return_value = mock_session
|
||||
session_ctx.__exit__.return_value = False
|
||||
ingestion_fn = MagicMock(side_effect=RuntimeError("ingestion boom"))
|
||||
ingestion_fn = MagicMock(side_effect=ingestion_error)
|
||||
if temporary_database_missing:
|
||||
mock_finish.side_effect = DatabaseError(
|
||||
"Save with update_fields did not affect any rows"
|
||||
)
|
||||
|
||||
with (
|
||||
patch(
|
||||
@@ -337,13 +373,28 @@ class TestAttackPathsRun:
|
||||
return_value=ingestion_fn,
|
||||
),
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="ingestion boom"):
|
||||
with pytest.raises(type(ingestion_error)):
|
||||
attack_paths_run(str(tenant.id), str(scan.id), "task-456")
|
||||
|
||||
failure_args = mock_finish.call_args[0]
|
||||
assert failure_args[0] is attack_paths_scan
|
||||
assert failure_args[1] == StateChoices.FAILED
|
||||
assert failure_args[2] == {"global_error": "Cartography failed: ingestion boom"}
|
||||
mock_drop_db.assert_called_once_with("db-scan-id")
|
||||
if temporary_database_missing:
|
||||
mock_logger.warning.assert_any_call("Cartography failed: ingestion boom")
|
||||
mock_logger.exception.assert_not_called()
|
||||
mock_logger.log.assert_called_once_with(
|
||||
logging.WARNING,
|
||||
f"Could not mark Attack Paths scan {attack_paths_scan.id} as `FAILED` "
|
||||
"(row may have been deleted): Save with update_fields did not affect "
|
||||
"any rows",
|
||||
exc_info=False,
|
||||
)
|
||||
else:
|
||||
mock_logger.exception.assert_called_once_with(
|
||||
"Cartography failed: ingestion boom"
|
||||
)
|
||||
|
||||
@patch(
|
||||
"tasks.jobs.attack_paths.scan.utils.stringify_exception",
|
||||
@@ -1265,6 +1316,33 @@ class TestAttackPathsScanRLSTaskOnFailure:
|
||||
|
||||
mock_fail.assert_called_once_with("t-1", "s-1", "boom")
|
||||
|
||||
def test_on_failure_logs_provider_deletion_as_warning(self):
|
||||
from tasks.tasks import AttackPathsScanRLSTask
|
||||
|
||||
task = AttackPathsScanRLSTask()
|
||||
error = ProviderDeletedException("provider deleted")
|
||||
|
||||
with (
|
||||
patch("tasks.tasks.logger") as mock_logger,
|
||||
patch(
|
||||
"tasks.tasks.attack_paths_db_utils.fail_attack_paths_scan"
|
||||
) as mock_fail,
|
||||
):
|
||||
task.on_failure(
|
||||
exc=error,
|
||||
task_id="task-abc",
|
||||
args=(),
|
||||
kwargs={"tenant_id": "t-1", "scan_id": "s-1"},
|
||||
_einfo=None,
|
||||
)
|
||||
|
||||
mock_logger.warning.assert_called_once_with(
|
||||
"Attack paths scan task task-abc stopped because its provider or tenant "
|
||||
"was deleted: provider deleted"
|
||||
)
|
||||
mock_logger.error.assert_not_called()
|
||||
mock_fail.assert_called_once_with("t-1", "s-1", "provider deleted")
|
||||
|
||||
def test_on_failure_skips_when_missing_kwargs(self):
|
||||
from tasks.tasks import AttackPathsScanRLSTask
|
||||
|
||||
@@ -1896,7 +1974,7 @@ class TestSyncNodes:
|
||||
mock_source_1.run.return_value = [row]
|
||||
mock_source_2 = MagicMock()
|
||||
mock_source_2.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1000)
|
||||
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
@@ -1933,7 +2011,7 @@ class TestSyncNodes:
|
||||
src_1.run.return_value = [row]
|
||||
src_2 = MagicMock()
|
||||
src_2.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1000)
|
||||
sink.write_nodes.side_effect = lambda *_a, **_kw: call_order.append(
|
||||
"sink:write"
|
||||
)
|
||||
@@ -1969,18 +2047,15 @@ class TestSyncNodes:
|
||||
src_2.run.return_value = [row_b]
|
||||
src_3 = MagicMock()
|
||||
src_3.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
_make_session_ctx(src_3),
|
||||
],
|
||||
),
|
||||
patch("tasks.jobs.attack_paths.sync.SYNC_BATCH_SIZE", 1),
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
_make_session_ctx(src_3),
|
||||
],
|
||||
):
|
||||
result = sync_module.sync_nodes("src", "tgt", "t-1", "p-1", sink, [])
|
||||
|
||||
@@ -2009,17 +2084,14 @@ class TestSyncNodes:
|
||||
src_1.run.return_value = [row]
|
||||
src_2 = MagicMock()
|
||||
src_2.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=2)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
],
|
||||
),
|
||||
patch("tasks.jobs.attack_paths.sync.SYNC_BATCH_SIZE", 2),
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
],
|
||||
):
|
||||
result = sync_module.sync_nodes(
|
||||
"src", "tgt", "t-1", "p-1", sink, normalized_lists
|
||||
@@ -2037,7 +2109,7 @@ class TestSyncNodes:
|
||||
def test_sync_nodes_empty_source_returns_zero(self):
|
||||
src = MagicMock()
|
||||
src.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1000)
|
||||
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
@@ -2066,7 +2138,7 @@ class TestSyncRelationships:
|
||||
src_1.run.return_value = [row]
|
||||
src_2 = MagicMock()
|
||||
src_2.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1000)
|
||||
sink.write_relationships.side_effect = lambda *_a, **_kw: call_order.append(
|
||||
"sink:write"
|
||||
)
|
||||
@@ -2104,18 +2176,15 @@ class TestSyncRelationships:
|
||||
src_2.run.return_value = [row_b]
|
||||
src_3 = MagicMock()
|
||||
src_3.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
_make_session_ctx(src_3),
|
||||
],
|
||||
),
|
||||
patch("tasks.jobs.attack_paths.sync.SYNC_BATCH_SIZE", 1),
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
_make_session_ctx(src_3),
|
||||
],
|
||||
):
|
||||
total = sync_module.sync_relationships("src", "tgt", "p-1", sink)
|
||||
|
||||
@@ -2140,17 +2209,14 @@ class TestSyncRelationships:
|
||||
src_1.run.return_value = rows
|
||||
src_2 = MagicMock()
|
||||
src_2.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=2)
|
||||
|
||||
with (
|
||||
patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
],
|
||||
),
|
||||
patch("tasks.jobs.attack_paths.sync.SYNC_BATCH_SIZE", 2),
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
side_effect=[
|
||||
_make_session_ctx(src_1),
|
||||
_make_session_ctx(src_2),
|
||||
],
|
||||
):
|
||||
total = sync_module.sync_relationships("src", "tgt", "p-1", sink)
|
||||
|
||||
@@ -2163,7 +2229,7 @@ class TestSyncRelationships:
|
||||
def test_sync_relationships_empty_source_returns_zero(self):
|
||||
src = MagicMock()
|
||||
src.run.return_value = []
|
||||
sink = MagicMock()
|
||||
sink = MagicMock(sync_batch_size=1000)
|
||||
|
||||
with patch(
|
||||
"tasks.jobs.attack_paths.sync.graph_database.get_session",
|
||||
@@ -3056,6 +3122,61 @@ class TestCleanupStaleAttackPathsScans:
|
||||
ap_scan.refresh_from_db()
|
||||
assert ap_scan.state == StateChoices.FAILED
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("age_seconds", "should_clean"),
|
||||
[
|
||||
(960 * 60 - 1, False),
|
||||
(960 * 60, False),
|
||||
(960 * 60 + 1, True),
|
||||
],
|
||||
)
|
||||
@patch("tasks.jobs.attack_paths.cleanup.recover_graph_data_ready")
|
||||
@patch("tasks.jobs.attack_paths.cleanup.graph_database.drop_database")
|
||||
@patch(
|
||||
"tasks.jobs.attack_paths.cleanup.rls_transaction",
|
||||
new=lambda *args, **kwargs: nullcontext(),
|
||||
)
|
||||
@patch("tasks.jobs.attack_paths.cleanup._revoke_task")
|
||||
@patch("tasks.jobs.attack_paths.cleanup._ping_workers")
|
||||
def test_stale_threshold_boundary_is_strict(
|
||||
self,
|
||||
mock_ping,
|
||||
mock_revoke,
|
||||
mock_drop_db,
|
||||
mock_recover,
|
||||
age_seconds,
|
||||
should_clean,
|
||||
tenants_fixture,
|
||||
aws_provider,
|
||||
):
|
||||
from tasks.jobs.attack_paths.cleanup import cleanup_stale_attack_paths_scans
|
||||
|
||||
now = datetime.now(tz=UTC)
|
||||
ap_scan, task_result = self._create_executing_scan(
|
||||
tenants_fixture[0],
|
||||
aws_provider,
|
||||
started_at=now - timedelta(seconds=age_seconds),
|
||||
worker="live-worker@host",
|
||||
)
|
||||
mock_ping.return_value = ({"live-worker@host"}, set())
|
||||
|
||||
with patch("tasks.jobs.attack_paths.cleanup.datetime") as mock_datetime:
|
||||
mock_datetime.now.return_value = now
|
||||
result = cleanup_stale_attack_paths_scans()
|
||||
|
||||
assert result["cleaned_up_count"] == int(should_clean)
|
||||
ap_scan.refresh_from_db()
|
||||
expected_state = StateChoices.FAILED if should_clean else StateChoices.EXECUTING
|
||||
assert ap_scan.state == expected_state
|
||||
if should_clean:
|
||||
mock_revoke.assert_called_once_with(task_result, terminate=True)
|
||||
mock_drop_db.assert_called_once()
|
||||
mock_recover.assert_called_once()
|
||||
else:
|
||||
mock_revoke.assert_not_called()
|
||||
mock_drop_db.assert_not_called()
|
||||
mock_recover.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.attack_paths.cleanup.recover_graph_data_ready")
|
||||
@patch("tasks.jobs.attack_paths.cleanup.graph_database.drop_database")
|
||||
@patch(
|
||||
|
||||
@@ -26,6 +26,7 @@ from prowler.lib.check.models import Severity
|
||||
from prowler.lib.outputs.finding import Status
|
||||
from tasks.jobs.scan import (
|
||||
_ATTACK_SURFACE_MAPPING_CACHE,
|
||||
ComplianceRowScopeError,
|
||||
_aggregate_findings_by_region,
|
||||
_bulk_update_resource_failed_findings_counts,
|
||||
_copy_compliance_requirement_rows,
|
||||
@@ -2314,9 +2315,9 @@ class TestCreateComplianceRequirements:
|
||||
create_compliance_requirements(tenant_id, scan_id)
|
||||
|
||||
mock_persist.assert_called_once()
|
||||
persisted_rows = mock_persist.call_args[0][1]
|
||||
rows_factory = mock_persist.call_args[0][2]
|
||||
requirement_row = next(
|
||||
row for row in persisted_rows if row["requirement_id"] == "1.1"
|
||||
row for row in rows_factory() if row["requirement_id"] == "1.1"
|
||||
)
|
||||
assert requirement_row["requirement_status"] == "FAIL"
|
||||
|
||||
@@ -2454,18 +2455,26 @@ class TestComplianceRequirementCopy:
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
mock_psycopg_connection.assert_called_once_with("admin")
|
||||
connection.cursor.assert_called_once()
|
||||
cursor.execute.assert_called_once()
|
||||
# One execute for set_config plus one for the scan's DELETE.
|
||||
assert cursor.execute.call_count == 2
|
||||
delete_sql, delete_params = cursor.execute.call_args_list[1][0]
|
||||
assert "DELETE FROM compliance_requirements_overviews" in delete_sql
|
||||
assert delete_params == [str(row["tenant_id"]), str(row["scan_id"])]
|
||||
cursor.copy_expert.assert_called_once()
|
||||
connection.commit.assert_called_once()
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert csv_rows[0][0] == str(row["id"])
|
||||
assert csv_rows[0][5] == ""
|
||||
assert csv_rows[0][-1] == str(row["scan_id"])
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.filter")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
@@ -2473,7 +2482,7 @@ class TestComplianceRequirementCopy:
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_filter
|
||||
):
|
||||
inserted_at = datetime.now(UTC)
|
||||
row = {
|
||||
@@ -2494,16 +2503,22 @@ class TestComplianceRequirementCopy:
|
||||
}
|
||||
|
||||
tenant_id = row["tenant_id"]
|
||||
scan_id = str(row["scan_id"])
|
||||
|
||||
ctx = MagicMock()
|
||||
ctx.__enter__.return_value = None
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
_persist_compliance_requirement_rows(tenant_id, scan_id, lambda: [row])
|
||||
|
||||
mock_copy.assert_called_once_with(tenant_id, [row])
|
||||
mock_copy.assert_called_once()
|
||||
assert mock_copy.call_args[0][0] == tenant_id
|
||||
assert mock_copy.call_args[0][1] == scan_id
|
||||
mock_rls_transaction.assert_called_once_with(tenant_id)
|
||||
# The fallback replaces the scan's rows: delete + insert atomically.
|
||||
mock_filter.assert_called_once_with(scan_id=scan_id)
|
||||
mock_filter.return_value.delete.assert_called_once()
|
||||
mock_bulk_create.assert_called_once()
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
@@ -2515,13 +2530,18 @@ class TestComplianceRequirementCopy:
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
|
||||
@patch("tasks.jobs.scan._copy_compliance_requirement_rows", return_value=0)
|
||||
def test_persist_compliance_requirement_rows_no_rows(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
):
|
||||
_persist_compliance_requirement_rows(str(uuid.uuid4()), [])
|
||||
# Even with no rows the COPY path runs: it must clear the scan's
|
||||
# previous rows so a re-run with fewer findings drops stale data.
|
||||
total = _persist_compliance_requirement_rows(
|
||||
str(uuid.uuid4()), str(uuid.uuid4()), lambda: []
|
||||
)
|
||||
|
||||
mock_copy.assert_not_called()
|
||||
assert total == 0
|
||||
mock_copy.assert_called_once()
|
||||
mock_rls_transaction.assert_not_called()
|
||||
mock_bulk_create.assert_not_called()
|
||||
|
||||
@@ -2610,11 +2630,12 @@ class TestComplianceRequirementCopy:
|
||||
]
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(tenant_id, rows)
|
||||
_copy_compliance_requirement_rows(tenant_id, str(scan_id), rows, 2000)
|
||||
|
||||
mock_psycopg_connection.assert_called_once_with("admin")
|
||||
connection.cursor.assert_called_once()
|
||||
cursor.execute.assert_called_once()
|
||||
# set_config + DELETE of the scan's previous rows.
|
||||
assert cursor.execute.call_count == 2
|
||||
cursor.copy_expert.assert_called_once()
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
@@ -2644,6 +2665,60 @@ class TestComplianceRequirementCopy:
|
||||
assert csv_rows[2][5] == "2.0"
|
||||
assert csv_rows[2][9] == "MANUAL"
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_batches_share_one_transaction(
|
||||
self, mock_psycopg_connection, settings
|
||||
):
|
||||
"""Every COPY batch runs on the same connection with a single commit."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = str(uuid.uuid4())
|
||||
inserted_at = datetime.now(UTC)
|
||||
rows = [
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"inserted_at": inserted_at,
|
||||
"compliance_id": "cisa_aws",
|
||||
"framework": "CISA",
|
||||
"version": "1.0",
|
||||
"description": f"Requirement {index}",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": f"req-{index}",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": scan_id,
|
||||
}
|
||||
for index in range(3)
|
||||
]
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
total = _copy_compliance_requirement_rows(tenant_id, scan_id, rows, 1)
|
||||
|
||||
assert total == 3
|
||||
# One connection, three COPY statements, one commit for the whole scan.
|
||||
mock_psycopg_connection.assert_called_once_with("admin")
|
||||
assert cursor.copy_expert.call_count == 3
|
||||
connection.commit.assert_called_once()
|
||||
connection.rollback.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_null_values(
|
||||
self, mock_psycopg_connection, settings
|
||||
@@ -2691,7 +2766,9 @@ class TestComplianceRequirementCopy:
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
assert len(csv_rows) == 1
|
||||
@@ -2747,7 +2824,9 @@ class TestComplianceRequirementCopy:
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
# Verify CSV was generated (csv module handles escaping automatically)
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
@@ -2808,7 +2887,9 @@ class TestComplianceRequirementCopy:
|
||||
|
||||
before_call = datetime.now(UTC)
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
after_call = datetime.now(UTC)
|
||||
|
||||
csv_rows = list(csv.reader(StringIO(captured["data"])))
|
||||
@@ -2861,12 +2942,84 @@ class TestComplianceRequirementCopy:
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
with pytest.raises(Exception, match="COPY command failed"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
# Verify rollback was called
|
||||
connection.rollback.assert_called_once()
|
||||
connection.commit.assert_not_called()
|
||||
|
||||
@pytest.mark.parametrize("mismatched_field", ["tenant_id", "scan_id"])
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_rejects_out_of_scope_rows(
|
||||
self, mock_psycopg_connection, mismatched_field, settings
|
||||
):
|
||||
"""COPY bypasses RLS, so rows from another tenant/scan must be rejected."""
|
||||
settings.DATABASES.setdefault("admin", settings.DATABASES["default"])
|
||||
|
||||
connection = MagicMock()
|
||||
cursor = MagicMock()
|
||||
cursor_context = MagicMock()
|
||||
cursor_context.__enter__.return_value = cursor
|
||||
cursor_context.__exit__.return_value = False
|
||||
connection.cursor.return_value = cursor_context
|
||||
connection.__enter__.return_value = connection
|
||||
connection.__exit__.return_value = False
|
||||
|
||||
context_manager = MagicMock()
|
||||
context_manager.__enter__.return_value = connection
|
||||
context_manager.__exit__.return_value = False
|
||||
mock_psycopg_connection.return_value = context_manager
|
||||
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = str(uuid.uuid4())
|
||||
row = {
|
||||
"id": uuid.uuid4(),
|
||||
"tenant_id": tenant_id,
|
||||
"compliance_id": "test",
|
||||
"framework": "Test",
|
||||
"version": "1.0",
|
||||
"description": "desc",
|
||||
"region": "us-east-1",
|
||||
"requirement_id": "req-1",
|
||||
"requirement_status": "PASS",
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": scan_id,
|
||||
}
|
||||
row[mismatched_field] = str(uuid.uuid4())
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
with pytest.raises(ComplianceRowScopeError):
|
||||
_copy_compliance_requirement_rows(tenant_id, scan_id, [row], 2000)
|
||||
|
||||
cursor.copy_expert.assert_not_called()
|
||||
connection.rollback.assert_called_once()
|
||||
connection.commit.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
"tasks.jobs.scan._copy_compliance_requirement_rows",
|
||||
side_effect=ComplianceRowScopeError("out of scope"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_does_not_fall_back_on_scope_error(
|
||||
self, mock_copy, mock_rls_transaction, mock_model
|
||||
):
|
||||
"""A scope violation is a caller bug: the ORM fallback must not persist it."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = str(uuid.uuid4())
|
||||
|
||||
with pytest.raises(ComplianceRowScopeError):
|
||||
_persist_compliance_requirement_rows(tenant_id, scan_id, lambda: [])
|
||||
|
||||
mock_copy.assert_called_once()
|
||||
mock_rls_transaction.assert_not_called()
|
||||
mock_model.objects.filter.assert_not_called()
|
||||
mock_model.objects.bulk_create.assert_not_called()
|
||||
|
||||
@patch("tasks.jobs.scan.psycopg_connection")
|
||||
def test_copy_compliance_requirement_rows_transaction_rollback_on_set_config_error(
|
||||
self, mock_psycopg_connection, settings
|
||||
@@ -2909,7 +3062,9 @@ class TestComplianceRequirementCopy:
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
with pytest.raises(Exception, match="SET prowler.tenant_id failed"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
# Verify rollback was called
|
||||
connection.rollback.assert_called_once()
|
||||
@@ -2955,7 +3110,9 @@ class TestComplianceRequirementCopy:
|
||||
}
|
||||
|
||||
with patch.object(MainRouter, "admin_db", "admin"):
|
||||
_copy_compliance_requirement_rows(str(row["tenant_id"]), [row])
|
||||
_copy_compliance_requirement_rows(
|
||||
str(row["tenant_id"]), str(row["scan_id"]), [row], 2000
|
||||
)
|
||||
|
||||
# Verify commit was called and rollback was not
|
||||
connection.commit.assert_called_once()
|
||||
@@ -2966,9 +3123,10 @@ class TestComplianceRequirementCopy:
|
||||
@patch("tasks.jobs.scan._copy_compliance_requirement_rows")
|
||||
def test_persist_compliance_requirement_rows_success(self, mock_copy):
|
||||
"""Test successful COPY path without fallback to ORM."""
|
||||
mock_copy.return_value = None # Success, no exception
|
||||
mock_copy.return_value = 1 # Success, no exception
|
||||
|
||||
tenant_id = str(uuid.uuid4())
|
||||
scan_id = str(uuid.uuid4())
|
||||
rows = [
|
||||
{
|
||||
"id": uuid.uuid4(),
|
||||
@@ -2984,16 +3142,21 @@ class TestComplianceRequirementCopy:
|
||||
"passed_checks": 1,
|
||||
"failed_checks": 0,
|
||||
"total_checks": 1,
|
||||
"scan_id": uuid.uuid4(),
|
||||
"scan_id": scan_id,
|
||||
}
|
||||
]
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, rows)
|
||||
total = _persist_compliance_requirement_rows(tenant_id, scan_id, lambda: rows)
|
||||
|
||||
# Verify COPY was called
|
||||
mock_copy.assert_called_once_with(tenant_id, rows)
|
||||
assert total == 1
|
||||
mock_copy.assert_called_once()
|
||||
copy_args = mock_copy.call_args[0]
|
||||
assert copy_args[0] == tenant_id
|
||||
assert copy_args[1] == scan_id
|
||||
assert list(copy_args[2]) == rows
|
||||
|
||||
@patch("tasks.jobs.scan.logger")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.filter")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
@@ -3001,7 +3164,12 @@ class TestComplianceRequirementCopy:
|
||||
side_effect=Exception("COPY failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_logging(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_logger
|
||||
self,
|
||||
mock_copy,
|
||||
mock_rls_transaction,
|
||||
mock_bulk_create,
|
||||
mock_filter,
|
||||
mock_logger,
|
||||
):
|
||||
"""Test logger.exception is called when COPY fails and fallback occurs."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
@@ -3027,7 +3195,9 @@ class TestComplianceRequirementCopy:
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
_persist_compliance_requirement_rows(
|
||||
tenant_id, str(row["scan_id"]), lambda: [row]
|
||||
)
|
||||
|
||||
# Verify logger.exception was called
|
||||
mock_logger.exception.assert_called_once()
|
||||
@@ -3036,6 +3206,7 @@ class TestComplianceRequirementCopy:
|
||||
assert "falling back to ORM" in args[0]
|
||||
assert kwargs.get("exc_info") is not None
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.filter")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
@@ -3043,7 +3214,7 @@ class TestComplianceRequirementCopy:
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_multiple_rows(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_filter
|
||||
):
|
||||
"""Test ORM fallback with multiple rows."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
@@ -3090,10 +3261,14 @@ class TestComplianceRequirementCopy:
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, rows)
|
||||
total = _persist_compliance_requirement_rows(
|
||||
tenant_id, str(scan_id), lambda: rows
|
||||
)
|
||||
|
||||
mock_copy.assert_called_once_with(tenant_id, rows)
|
||||
assert total == 2
|
||||
mock_copy.assert_called_once()
|
||||
mock_rls_transaction.assert_called_once_with(tenant_id)
|
||||
mock_filter.assert_called_once_with(scan_id=str(scan_id))
|
||||
mock_bulk_create.assert_called_once()
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
@@ -3117,6 +3292,7 @@ class TestComplianceRequirementCopy:
|
||||
assert objects[1].passed_checks == 2
|
||||
assert objects[1].failed_checks == 3
|
||||
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.filter")
|
||||
@patch("tasks.jobs.scan.ComplianceRequirementOverview.objects.bulk_create")
|
||||
@patch("tasks.jobs.scan.rls_transaction")
|
||||
@patch(
|
||||
@@ -3124,7 +3300,7 @@ class TestComplianceRequirementCopy:
|
||||
side_effect=Exception("copy failed"),
|
||||
)
|
||||
def test_persist_compliance_requirement_rows_fallback_all_fields(
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create
|
||||
self, mock_copy, mock_rls_transaction, mock_bulk_create, mock_filter
|
||||
):
|
||||
"""Test ORM fallback correctly maps all fields from row dict to model."""
|
||||
tenant_id = str(uuid.uuid4())
|
||||
@@ -3154,7 +3330,7 @@ class TestComplianceRequirementCopy:
|
||||
ctx.__exit__.return_value = False
|
||||
mock_rls_transaction.return_value = ctx
|
||||
|
||||
_persist_compliance_requirement_rows(tenant_id, [row])
|
||||
_persist_compliance_requirement_rows(tenant_id, str(scan_id), lambda: [row])
|
||||
|
||||
args, kwargs = mock_bulk_create.call_args
|
||||
objects = args[0]
|
||||
|
||||
@@ -4673,8 +4673,8 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "prowler"
|
||||
version = "5.32.0"
|
||||
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=master#5dac8a0a53272e4db68c476fb969dc03e88beb68" }
|
||||
version = "5.35.0"
|
||||
source = { git = "https://github.com/prowler-cloud/prowler.git?rev=master#f5ea116763aeffede9f399c8934fc280eaccd315" }
|
||||
dependencies = [
|
||||
{ name = "alibabacloud-actiontrail20200706" },
|
||||
{ name = "alibabacloud-credentials" },
|
||||
@@ -4762,7 +4762,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "prowler-api"
|
||||
version = "1.36.0"
|
||||
version = "1.37.0"
|
||||
source = { virtual = "." }
|
||||
dependencies = [
|
||||
{ name = "cartography" },
|
||||
|
||||
@@ -36,6 +36,9 @@ The former build-time variables map to the new runtime variables as follows:
|
||||
| `NEXT_PUBLIC_GOOGLE_TAG_MANAGER_ID` | `UI_GOOGLE_TAG_MANAGER_ID` |
|
||||
| `NEXT_PUBLIC_SENTRY_DSN`, `SENTRY_DSN` | `UI_SENTRY_DSN` |
|
||||
| `NEXT_PUBLIC_SENTRY_ENVIRONMENT`, `SENTRY_ENVIRONMENT` | `UI_SENTRY_ENVIRONMENT` |
|
||||
| `NEXT_PUBLIC_IS_CLOUD_ENV` | `UI_CLOUD_ENABLED` |
|
||||
|
||||
`UI_CLOUD_ENABLED` is a plain runtime boolean flag that enables Prowler Cloud behavior when set to the exact string `"true"` and defaults to off; unlike the other renamed variables it has no legacy fallback, so `NEXT_PUBLIC_IS_CLOUD_ENV` is no longer read.
|
||||
|
||||
The build-time-only Sentry variables used for source-map upload — `SENTRY_ORG`, `SENTRY_PROJECT`, `SENTRY_AUTH_TOKEN`, and `SENTRY_RELEASE` — keep their names, as they are not part of Prowler Local Server's runtime configuration.
|
||||
|
||||
|
||||
@@ -80,7 +80,12 @@
|
||||
{
|
||||
"group": "Prowler for AI Agents",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-claude-code-plugin"
|
||||
"user-guide/ai-agents/index",
|
||||
"user-guide/ai-agents/claude-code",
|
||||
"user-guide/ai-agents/claude-desktop",
|
||||
"user-guide/ai-agents/codex",
|
||||
"user-guide/ai-agents/cursor",
|
||||
"user-guide/ai-agents/vscode"
|
||||
]
|
||||
}
|
||||
]
|
||||
@@ -217,7 +222,12 @@
|
||||
{
|
||||
"group": "Prowler for AI Agents",
|
||||
"pages": [
|
||||
"getting-started/products/prowler-claude-code-plugin"
|
||||
"user-guide/ai-agents/index",
|
||||
"user-guide/ai-agents/claude-code",
|
||||
"user-guide/ai-agents/claude-desktop",
|
||||
"user-guide/ai-agents/codex",
|
||||
"user-guide/ai-agents/cursor",
|
||||
"user-guide/ai-agents/vscode"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -660,6 +670,10 @@
|
||||
{
|
||||
"source": "/user-guide/tutorials/prowler-cloud-public-ips",
|
||||
"destination": "/security/networking"
|
||||
},
|
||||
{
|
||||
"source": "/getting-started/products/prowler-claude-code-plugin",
|
||||
"destination": "/user-guide/ai-agents/claude-code"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -23,6 +23,28 @@ Most users should use the **Cloud MCP Server** — it needs no installation and
|
||||
- **Cloud MCP Server (HTTP)**: the managed server at `https://mcp.prowler.com/mcp` (or your own self-hosted HTTP server).
|
||||
- **Local MCP Server (STDIO)**: local installation only (runs as a subprocess of your MCP client).
|
||||
|
||||
### Step-by-Step Guides Per Agent
|
||||
|
||||
The tabs below are a quick configuration reference. For a walkthrough with screenshots, troubleshooting, and client-specific caveats, follow the dedicated guide for your agent:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude Code" icon="terminal" href="/user-guide/ai-agents/claude-code">
|
||||
Plugin vs. MCP-only, and which Claude surfaces work
|
||||
</Card>
|
||||
<Card title="Claude Desktop App (Chat)" icon="comment" href="/user-guide/ai-agents/claude-desktop">
|
||||
The Chat tab, via a local bridge
|
||||
</Card>
|
||||
<Card title="Codex" icon="code" href="/user-guide/ai-agents/codex">
|
||||
CLI and the VS Code extension
|
||||
</Card>
|
||||
<Card title="Cursor" icon="arrow-pointer" href="/user-guide/ai-agents/cursor">
|
||||
Global and project scopes
|
||||
</Card>
|
||||
<Card title="VS Code / Copilot" icon="microsoft" href="/user-guide/ai-agents/vscode">
|
||||
Agent mode with secure key prompts
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Cloud MCP Server Configuration (Recommended)
|
||||
|
||||
Connect to the **Cloud MCP Server** at `https://mcp.prowler.com/mcp` over HTTP. This is the recommended path — no installation, always up to date. The same configuration works for a self-hosted HTTP server: just swap the URL.
|
||||
@@ -76,67 +98,6 @@ Connect to the **Cloud MCP Server** at `https://mcp.prowler.com/mcp` over HTTP.
|
||||
The `mcp-remote` tool acts as a bridge for clients that don't support HTTP natively. Learn more at [mcp-remote on npm](https://www.npmjs.com/package/mcp-remote).
|
||||
</Info>
|
||||
</Tab>
|
||||
|
||||
<Tab title="Claude Desktop">
|
||||
1. Open Claude Desktop settings
|
||||
2. Go to "Developer" tab
|
||||
3. Click in "Edit Config" button
|
||||
4. Edit the `claude_desktop_config.json` file with your favorite editor
|
||||
5. Install a reviewed version of `mcp-remote` in a dedicated local workspace:
|
||||
```bash
|
||||
mkdir -p ~/.local/share/prowler-mcp-bridge
|
||||
cd ~/.local/share/prowler-mcp-bridge
|
||||
npm init -y
|
||||
npm install --save-exact mcp-remote@0.1.38
|
||||
```
|
||||
6. Add the following configuration:
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"command": "/absolute/path/to/.local/share/prowler-mcp-bridge/node_modules/.bin/mcp-remote",
|
||||
"args": [
|
||||
"https://mcp.prowler.com/mcp",
|
||||
"--header",
|
||||
"Authorization: Bearer ${PROWLER_API_KEY}"
|
||||
],
|
||||
"env": {
|
||||
"PROWLER_API_KEY": "<your-api-key-here>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab title="Claude Code">
|
||||
Run the following command:
|
||||
```bash
|
||||
export PROWLER_API_KEY="<your-api-key-here>"
|
||||
claude mcp add --transport http prowler https://mcp.prowler.com/mcp --header "Authorization: Bearer $PROWLER_API_KEY" --scope user
|
||||
```
|
||||
</Tab>
|
||||
|
||||
<Tab title="Cursor">
|
||||
1. Open Cursor settings
|
||||
2. Go to "Tools & MCP"
|
||||
3. Click in "New MCP Server" button
|
||||
4. Add to the JSON Configuration the following:
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"url": "https://mcp.prowler.com/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer <your-api-key-here>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
</Tab>
|
||||
|
||||
|
||||
</Tabs>
|
||||
|
||||
## Local MCP Server Configuration
|
||||
|
||||
@@ -128,8 +128,8 @@ To update the environment file:
|
||||
Edit the `.env` file and change version values:
|
||||
|
||||
```env
|
||||
PROWLER_UI_VERSION="5.34.0"
|
||||
PROWLER_API_VERSION="5.34.0"
|
||||
PROWLER_UI_VERSION="5.35.0"
|
||||
PROWLER_API_VERSION="5.35.0"
|
||||
```
|
||||
|
||||
<Note>
|
||||
|
||||
@@ -18,7 +18,7 @@ Read the [public announcement of the Prowler product families](https://prowler-w
|
||||
| Prowler Private Cloud | Prowler Cloud deployed in your own environment. Formerly Prowler Enterprise. See [pricing](https://prowler.com/pricing). |
|
||||
| [Prowler Hub](https://hub.prowler.com) | Free public library of versioned checks, cloud service artifacts, and compliance frameworks. |
|
||||
| [Prowler Lighthouse AI](/getting-started/products/prowler-cloud-lighthouse) | AI security analyst capabilities within Prowler Cloud and Prowler Private Cloud. |
|
||||
| [Prowler MCP](/getting-started/products/prowler-mcp) | MCP server that connects AI assistants and agents to Prowler, including IDE plugins such as [Prowler for Claude Code](/getting-started/products/prowler-claude-code-plugin). |
|
||||
| [Prowler MCP](/getting-started/products/prowler-mcp) | MCP server that connects AI assistants and agents to Prowler, including IDE plugins such as [Prowler for Claude Code](/user-guide/ai-agents/claude-code). |
|
||||
|
||||
{/* Unreleased products. Uncomment these rows in the Prowler Products table when announced:
|
||||
| Prowler Registry | Distribution service for Prowler content such as checks and compliance frameworks. Free and paid tiers. |
|
||||
|
||||
@@ -1,102 +0,0 @@
|
||||
---
|
||||
title: 'Prowler for Claude Code'
|
||||
sidebarTitle: 'Claude Code'
|
||||
---
|
||||
|
||||
End-to-end cloud security and compliance from inside [Claude Code](https://www.claude.com/product/claude-code), powered by the [Prowler MCP server](/getting-started/products/prowler-mcp). The plugin lets Claude walk a Prowler Cloud-connected account through a compliance assessment and remediate findings until the chosen security or industry framework is compliant.
|
||||
|
||||
<Warning>
|
||||
**Preview**: this plugin is under active development. Please report issues on [GitHub](https://github.com/prowler-cloud/prowler/issues) or join the [Slack community](https://goto.prowler.com/slack) for feedback.
|
||||
</Warning>
|
||||
|
||||
## Requirements
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="Claude Code" icon="terminal">
|
||||
Installed and signed in. See the [official install guide](https://www.claude.com/product/claude-code).
|
||||
</Card>
|
||||
<Card title="Prowler Cloud account" icon="cloud">
|
||||
The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
</Card>
|
||||
<Card title="Prowler API key" icon="key">
|
||||
Create one at [cloud.prowler.com/profile](https://cloud.prowler.com/profile).
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Installation
|
||||
|
||||
<Tabs>
|
||||
<Tab title="From GitHub (recommended)">
|
||||
Inside a Claude Code session:
|
||||
|
||||
```text
|
||||
/plugin marketplace add prowler-cloud/prowler
|
||||
/plugin install prowler@prowler-plugins
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="From a local clone">
|
||||
If you already have the repository checked out:
|
||||
|
||||
```text
|
||||
/plugin marketplace add /absolute/path/to/prowler
|
||||
/plugin install prowler@prowler-plugins
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Configuration
|
||||
|
||||
On first install, Claude Code prompts for your **Prowler API key**. The value is stored securely (macOS keychain or `~/.claude/.credentials.json`) and used to authenticate against Prowler Cloud.
|
||||
|
||||
<Note>
|
||||
To rotate the key, uninstall and reinstall the plugin — Claude Code will prompt again.
|
||||
</Note>
|
||||
|
||||
## Verify the installation
|
||||
|
||||
In a Claude Code session:
|
||||
|
||||
```text
|
||||
/mcp → "prowler" appears as a connected server
|
||||
/plugin → "prowler" enabled, skill listed as prowler:framework-compliance-triage
|
||||
```
|
||||
|
||||
If `/mcp` reports the `prowler` server as failed, the most common cause is a rejected API key — re-issue one in Prowler Cloud and reinstall the plugin so it re-prompts.
|
||||
|
||||
## Usage
|
||||
|
||||
Open a conversation that mentions the framework you want to comply with. Examples:
|
||||
|
||||
- *"Make my AWS production account compliant with CIS 4.0."*
|
||||
- *"Make my current Terraform project compliant with Prowler ThreatScore Compliance Framework based on the latest scan results."*
|
||||
- *"Help me get to 100% on PCI-DSS for this GCP project."*
|
||||
|
||||
You pick a **primary tool** (Terraform, gh / az / aws CLI, web console, or mixed) and a **mode**:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude-assisted (default)" icon="hand">
|
||||
Claude shows each fix — target resource, exact commands, side effects, reversibility — and waits for your go-ahead before applying.
|
||||
</Card>
|
||||
<Card title="Claude autonomous" icon="robot">
|
||||
Claude presents a single up-front plan grouped by shared fixes, waits for one confirmation, then proceeds. It pauses mid-loop if a fix has wide blast radius or a finding is not applicable.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
Claude tracks progress in a markdown report under `.prowler/` at your project root — one file per framework × account. Open it any time to see exactly where the flow is. When all findings are addressed, Claude proposes a fresh Prowler scan to verify everything end-to-end.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
```text
|
||||
/plugin uninstall prowler@prowler-plugins
|
||||
/plugin marketplace remove prowler-plugins
|
||||
```
|
||||
|
||||
The stored API key is removed automatically.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| `/mcp` shows `prowler` as failed | Rejected API key | Generate a new one in Prowler Cloud and reinstall the plugin to re-prompt. |
|
||||
| Skill not invoked when expected | The skill description didn't match the prompt | Mention the framework name plus "compliance" or "compliant" in your prompt. |
|
||||
| "Framework not supported" | Prowler Hub does not list the framework for that provider | Open an issue or PR at [github.com/prowler-cloud/prowler](https://github.com/prowler-cloud/prowler). |
|
||||
@@ -24,6 +24,9 @@ The Agentic Cloud Defender does more than answer questions, it helps teams **fin
|
||||
<Card title="Normal and Agentic Views" icon="table-columns">
|
||||
Switch between the standard interface and a chat-first agentic view.
|
||||
</Card>
|
||||
<Card title="Side Panel on Every Page" icon="comment-dots">
|
||||
Open Lighthouse AI as a side panel from any page to get help in context.
|
||||
</Card>
|
||||
<Card title="Provider Connection Checks" icon="plug-circle-check">
|
||||
Credentials are validated automatically when a provider is configured.
|
||||
</Card>
|
||||
@@ -37,6 +40,20 @@ Promoting the chat to a top-level view gives Lighthouse AI the room it needs for
|
||||
|
||||
<img src="/images/prowler-app/lighthouse/prowler-cloud/main-chat-page.png" alt="Lighthouse AI chat view in Prowler Cloud" />
|
||||
|
||||
### Side Panel
|
||||
|
||||
You do not have to switch to the full chat view to reach Lighthouse AI. A side panel is available on every page of Prowler Cloud. While collapsed it stays out of the way; open it from any dashboard, findings list, or configuration screen to ask questions without leaving what you are working on. Open it using the Lighthouse AI button, circled in red in the image below.
|
||||
|
||||
<img src="/images/prowler-app/lighthouse/prowler-cloud/side-panel-closed.png" alt="Collapsed Lighthouse AI side panel on a Prowler Cloud page, with the button to open it circled in red" />
|
||||
|
||||
Once open, the panel slides in alongside your current page and shares the same agent, tools, and persistent chat sessions as the full Chat View, so a conversation started in the panel can be reopened and continued later from either place.
|
||||
|
||||
<img src="/images/prowler-app/lighthouse/prowler-cloud/side-panel-open.png" alt="Lighthouse AI side panel open alongside a Prowler Cloud page" />
|
||||
|
||||
- **Available everywhere:** Summon the assistant from any page while you keep working in the normal view.
|
||||
- **Context-aware help:** Ask about the findings, resources, or compliance data you are currently looking at.
|
||||
- **Continuous sessions:** Conversations opened in the side panel are saved alongside the rest of your chat history.
|
||||
|
||||
### Tool Usage
|
||||
|
||||
Lighthouse AI on Prowler Cloud renders the agent's work as it happens, so responses are easier to follow and to trust. Tool calls and reasoning steps appear in the order they occur within the conversation.
|
||||
@@ -63,6 +80,53 @@ At the top of the configuration page, the optional **Business Context** field le
|
||||
|
||||
Lighthouse AI on Prowler Cloud supports OpenAI, Amazon Bedrock, and OpenAI-compatible providers, with GPT-5.5 as the default. For per-provider setup and how to switch the default provider or model, see [Using Multiple LLM Providers](/user-guide/tutorials/prowler-cloud-lighthouse-multi-llm).
|
||||
|
||||
## Capabilities
|
||||
|
||||
Lighthouse AI works through the [Prowler MCP Server](/getting-started/products/prowler-mcp), which gives the agent a growing catalog of tools to explore and act on your security data. These actions run inside Prowler and never modify your cloud resources. Everything the agent can do maps to one of the following capability areas.
|
||||
|
||||
### Findings and Finding Groups
|
||||
|
||||
- Search and filter security findings across every connected provider by severity, status, region, service, check, date range, and muted state.
|
||||
- Retrieve full finding details, including remediation guidance, check metadata, and affected resources.
|
||||
- Summarize findings with aggregate statistics and trends.
|
||||
- Browse finding groups aggregated by check and drill down into the specific resources each group affects.
|
||||
|
||||
### Resources
|
||||
|
||||
- List and filter cloud resources by provider, region, service, resource type, and tags.
|
||||
- Inspect a resource's configuration, metadata, and related findings.
|
||||
- Review the timeline of cloud API actions performed on a resource (AWS CloudTrail), including who did what and when.
|
||||
- Get an aggregate overview of the resources Prowler has discovered.
|
||||
|
||||
### Compliance
|
||||
|
||||
- Review high-level compliance status across all frameworks, with pass/fail statistics per framework.
|
||||
- Get a requirement-level breakdown for a specific framework, including failed requirements and their associated findings.
|
||||
|
||||
### Attack Paths
|
||||
|
||||
- List Attack Paths scans and discover the queries available for each completed scan.
|
||||
- Run graph-based queries to reveal privilege-escalation chains and exploitable misconfigurations.
|
||||
- Retrieve the Cartography graph schema to build accurate custom queries.
|
||||
|
||||
### Scans and Providers
|
||||
|
||||
- List, inspect, and rename security scans across providers.
|
||||
- Trigger manual scans and schedule automated daily scans for continuous monitoring.
|
||||
- Search connected providers and check their connection status, connect new providers, or remove existing ones.
|
||||
|
||||
### Muting
|
||||
|
||||
- Manage the mutelist for pattern-based bulk muting.
|
||||
- Create, update, list, and delete finding-specific mute rules, each with a documented reason and audit trail.
|
||||
|
||||
### Security Check Catalog and Documentation
|
||||
|
||||
- Browse and search the Prowler Hub catalog of security checks and compliance frameworks, including check code and automated fixers.
|
||||
- Search and retrieve official Prowler documentation to answer how-to and product questions.
|
||||
|
||||
For the complete list of underlying tools, see the [Prowler MCP Tools Reference](/getting-started/basic-usage/prowler-mcp-tools).
|
||||
|
||||
## FAQ
|
||||
|
||||
**Which LLM providers are supported?**
|
||||
@@ -71,7 +135,11 @@ OpenAI (GPT models, including the default GPT-5.5), Amazon Bedrock (Claude, Llam
|
||||
|
||||
**Can Lighthouse AI change my cloud environment?**
|
||||
|
||||
No. Lighthouse AI has read-only access to security data and no tools to modify resources, even when the connected cloud credentials would allow changes.
|
||||
No. Lighthouse AI cannot modify the resources in your connected cloud providers (AWS, Azure, GCP, and others). It has read-only access to that environment and no tools to change it, even when the connected cloud credentials would allow it.
|
||||
|
||||
**Can Lighthouse AI change my Prowler Cloud environment?**
|
||||
|
||||
Yes. Lighthouse AI can take action within Prowler Cloud itself, such as connecting or removing providers, triggering and scheduling scans, and managing mute rules and the mutelist. See [Capabilities](#capabilities) for the full list of what it can do. These actions only affect your Prowler Cloud workspace, never the resources in your cloud providers.
|
||||
|
||||
## Looking for the Open Source Version?
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ The fastest way to get started is the **Cloud MCP Server** at `https://mcp.prowl
|
||||
```
|
||||
|
||||
<Card title="Connect Your MCP Client to the Cloud MCP Server" icon="cloud" href="/getting-started/basic-usage/prowler-mcp#cloud-mcp-server-configuration-recommended" horizontal>
|
||||
Step-by-step setup for Claude Desktop, Claude Code, Cursor, and other clients.
|
||||
Step-by-step setup for Claude Code, Codex, Cursor, VS Code, and other agents.
|
||||
</Card>
|
||||
|
||||
<Note>
|
||||
|
||||
|
Before Width: | Height: | Size: 330 KiB After Width: | Height: | Size: 432 KiB |
|
After Width: | Height: | Size: 662 KiB |
|
After Width: | Height: | Size: 878 KiB |
|
After Width: | Height: | Size: 48 KiB |
|
After Width: | Height: | Size: 47 KiB |
|
After Width: | Height: | Size: 334 KiB |
|
After Width: | Height: | Size: 357 KiB |
|
After Width: | Height: | Size: 396 KiB |
|
After Width: | Height: | Size: 248 KiB |
|
After Width: | Height: | Size: 45 KiB |
|
After Width: | Height: | Size: 370 KiB |
|
After Width: | Height: | Size: 84 KiB |
|
After Width: | Height: | Size: 85 KiB |
|
After Width: | Height: | Size: 51 KiB |
|
After Width: | Height: | Size: 334 KiB |
|
After Width: | Height: | Size: 164 KiB |
|
After Width: | Height: | Size: 53 KiB |
|
After Width: | Height: | Size: 13 KiB |
|
After Width: | Height: | Size: 44 KiB |
@@ -84,6 +84,7 @@ li[data-title="Prowler Lighthouse AI"] > button span:first-child::after,
|
||||
li[data-title="Providers"] > button span:first-child::after,
|
||||
li[data-title="Scans"] > button span:first-child::after,
|
||||
li[data-title="Prowler MCP"] > button span:first-child::after,
|
||||
li[data-title="Prowler for AI Agents"] > button span:first-child::after,
|
||||
div:has(+ ul a[href="/security/encryption"]) h3 span::after,
|
||||
li[id="/user-guide/compliance/tutorials/cross-provider-compliance"] a > div > div > span:first-child::after,
|
||||
li[id="/user-guide/tutorials/prowler-alerts"] a > div > div > span:first-child::after,
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
---
|
||||
title: "Connect Claude Code to Prowler MCP Server"
|
||||
sidebarTitle: "Claude Code"
|
||||
---
|
||||
|
||||
Connect [Claude Code](https://www.claude.com/product/claude-code) to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp`.
|
||||
|
||||
## Where Claude Code Runs
|
||||
|
||||
Claude Code runs in two places. Both read the same configuration file, so you set it up **once from a terminal** and it works in both.
|
||||
|
||||
| Surface | How you open it | Reads | Covered by |
|
||||
|---|---|---|---|
|
||||
| **Claude Code CLI** | `claude` in a terminal | `~/.claude.json` | This guide |
|
||||
| **Claude Code in the desktop app** | The **Code** tab inside the Claude app | `~/.claude.json` — the same file | This guide, [set up from a terminal](#claude-code-in-the-desktop-app-code-tab) |
|
||||
| **Claude app Chat** | The **Chat** tab inside the Claude app | `claude_desktop_config.json` | [Claude App Chat](/user-guide/ai-agents/claude-desktop) — a separate setup |
|
||||
|
||||
<Warning>
|
||||
**The Chat tab is not Claude Code.** It is a different product surface with its own configuration file and its own connection method (a local bridge). Nothing on this page applies to it. If you want Prowler in Chat, use the [Claude App Chat](/user-guide/ai-agents/claude-desktop) guide instead.
|
||||
</Warning>
|
||||
|
||||
## Choose Your Setup
|
||||
|
||||
There are two ways to connect. Both end with the same MCP Server connection, the difference is what comes with it.
|
||||
|
||||
| | 🔌 **Prowler Plugin** | ⚙️ **MCP Connection Only** |
|
||||
|---|---|---|
|
||||
| **What you get** | The MCP connection **plus** the official Prowler skills for cloud security tasks | The MCP connection |
|
||||
| **Setup** | Two slash commands, prompts for the API key | One `claude mcp add` command |
|
||||
| **Guided workflows** | ✅ Skills drive multi-step security work end to end | ❌ You drive the conversation |
|
||||
| **Best for** | Structured cloud security work, such as taking an account to compliance | Ad-hoc queries and your own workflows |
|
||||
| **Where to use it** | Claude Code CLI | Claude Code CLI, and the **recommended setup for the desktop app's [Code tab](#claude-code-in-the-desktop-app-code-tab)** |
|
||||
|
||||
<Note>
|
||||
**The plugin already includes the MCP connection.** If you install the plugin, do **not** also run `claude mcp add` — you would end up with the server configured twice.
|
||||
</Note>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Claude Code** installed and signed in. See the [official install guide](https://www.claude.com/product/claude-code).
|
||||
- **A Prowler Cloud account.** The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
|
||||
## Get Your Prowler API Key
|
||||
|
||||
Create an API key in Prowler Cloud and copy it. The key begins with `pk_` and is shown only once. Check the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide for details.
|
||||
|
||||
---
|
||||
|
||||
# Option 1: Install the Prowler Plugin
|
||||
|
||||
<Warning>
|
||||
**Preview**: this plugin is under active development. Please report issues on [GitHub](https://github.com/prowler-cloud/prowler/issues) or join the [Slack community](https://goto.prowler.com/slack) for feedback.
|
||||
</Warning>
|
||||
|
||||
End-to-end cloud security from inside Claude Code, powered by the Prowler MCP server. The plugin bundles the official Prowler skills, task-specific workflows that let Claude carry out multi-step security work against a Prowler Cloud-connected account, rather than answering one question at a time.
|
||||
|
||||
### Included Skills
|
||||
|
||||
| Skill | What it does |
|
||||
| --- | --- |
|
||||
| `prowler:framework-compliance-triage` | Walks an account through a compliance assessment and remediates findings until the chosen security or industry framework is compliant. |
|
||||
|
||||
<Note>
|
||||
More skills are on the way. Installing the plugin keeps you current — new skills arrive with plugin updates, no extra configuration required.
|
||||
</Note>
|
||||
|
||||
## Installation (Claude Code CLI)
|
||||
|
||||
<Tabs>
|
||||
<Tab title="From GitHub (recommended)">
|
||||
Inside a Claude Code session:
|
||||
|
||||
```text
|
||||
/plugin marketplace add prowler-cloud/prowler
|
||||
/plugin install prowler@prowler-plugins
|
||||
```
|
||||
</Tab>
|
||||
<Tab title="From a local clone">
|
||||
If you already have the repository checked out:
|
||||
|
||||
```text
|
||||
/plugin marketplace add /absolute/path/to/prowler
|
||||
/plugin install prowler@prowler-plugins
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
On first install, Claude Code prompts for your **Prowler API key**. The value is stored securely (macOS keychain or `~/.claude/.credentials.json`) and used to authenticate against Prowler Cloud.
|
||||
|
||||
## Verify the Installation
|
||||
|
||||
In a Claude Code session:
|
||||
|
||||
```text
|
||||
/mcp → "prowler" appears as a connected server
|
||||
/plugin → "prowler" enabled, with the bundled Prowler skills listed
|
||||
```
|
||||
|
||||
If `/mcp` reports the `prowler` server as failed, the most common cause is a rejected API key, re-issue one in Prowler Cloud and reinstall the plugin so it re-prompts.
|
||||
|
||||
## Usage
|
||||
|
||||
Describe the security task you want done and Claude selects the matching skill.
|
||||
|
||||
### Framework Compliance Triage
|
||||
|
||||
Mention the framework you want to comply with:
|
||||
|
||||
- *"Make my AWS production account compliant with CIS 4.0."*
|
||||
- *"Make my current Terraform project compliant with Prowler ThreatScore Compliance Framework based on the latest scan results."*
|
||||
- *"Help me get to 100% on PCI-DSS for this GCP project."*
|
||||
|
||||
You pick a **primary tool** (Terraform, gh / az / aws CLI, web console, or mixed) and a **mode**:
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude-assisted (default)" icon="hand">
|
||||
Claude shows each fix — target resource, exact commands, side effects, reversibility — and waits for your go-ahead before applying.
|
||||
</Card>
|
||||
<Card title="Claude autonomous" icon="robot">
|
||||
Claude presents a single up-front plan grouped by shared fixes, waits for one confirmation, then proceeds. It pauses mid-loop if a fix has wide blast radius or a finding is not applicable.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
Claude tracks progress in a markdown report under `.prowler/` at your project root — one file per framework × account. Open it any time to see exactly where the flow is. When all findings are addressed, Claude proposes a fresh Prowler scan to verify everything end-to-end.
|
||||
|
||||
## Uninstalling
|
||||
|
||||
```text
|
||||
/plugin uninstall prowler@prowler-plugins
|
||||
/plugin marketplace remove prowler-plugins
|
||||
```
|
||||
|
||||
The stored API key is removed automatically.
|
||||
|
||||
---
|
||||
|
||||
# Option 2: Connect the MCP Server Only
|
||||
|
||||
Choose this when you want Prowler's tools available without the Prowler skills.
|
||||
|
||||
## Add the Server
|
||||
|
||||
Claude Code connects to remote HTTP MCP servers natively and supports custom headers, so no bridge is required.
|
||||
|
||||
```bash
|
||||
export PROWLER_API_KEY="pk_your_api_key_here"
|
||||
|
||||
claude mcp add --transport http prowler https://mcp.prowler.com/mcp \
|
||||
--header "Authorization: Bearer $PROWLER_API_KEY" \
|
||||
--scope user
|
||||
```
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/claude/claude-code-mcp-add.png" alt="Terminal showing the claude mcp add command and its confirmation output" />
|
||||
</Frame>
|
||||
|
||||
<Warning>
|
||||
**Always pass `--scope user`.** The default scope is `local`, which binds the server to the single directory you ran the command in. A locally-scoped server does not load when you open Claude Code anywhere else — this is the most common reason Prowler tools appear to vanish.
|
||||
</Warning>
|
||||
|
||||
| Scope | Loads in | Shared | Stored in |
|
||||
|-------|----------|--------|-----------|
|
||||
| `user` | All your projects | No | `~/.claude.json`, top-level `mcpServers` |
|
||||
| `project` | Current project only | Yes, via version control | `.mcp.json` in the project root |
|
||||
| `local` (default) | Current project only | No | `~/.claude.json`, under that project's entry |
|
||||
|
||||
When the same server name exists in more than one scope, precedence is **local → project → user**. The winning entry is used whole; fields are not merged.
|
||||
|
||||
<Warning>
|
||||
Avoid `--scope project` for Prowler. That writes `.mcp.json` into your repository, and committing the file would publish your API key.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
**Local server:** Replace the URL with your own HTTP endpoint. Everything else stays the same.
|
||||
</Note>
|
||||
|
||||
## Verify the Connection
|
||||
|
||||
```bash
|
||||
claude mcp get prowler # shows which scope holds the definition
|
||||
claude mcp list # lists all servers and their status
|
||||
```
|
||||
|
||||
Inside a Claude Code session, run `/mcp` to see connected servers and their tools.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/claude/claude-code-mcp-command.png" alt="Claude Code session showing the /mcp command output with the Prowler server connected" />
|
||||
</Frame>
|
||||
|
||||
## Start Using Prowler MCP
|
||||
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"Onboard this new AWS account in my Prowler organization"*
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/claude/claude-code-prowler-query.png" alt="Claude Code answering a question about critical findings using Prowler MCP tools" />
|
||||
</Frame>
|
||||
|
||||
---
|
||||
|
||||
# Claude Code in the Desktop App (Code Tab)
|
||||
|
||||
The **Code** tab in the Claude desktop app runs the same Claude Code as the CLI, and reads the same `~/.claude.json`. There is no separate Prowler setup for it — you configure it **from a terminal** and the Code tab picks it up.
|
||||
|
||||
<Note>
|
||||
**Use [Option 2](#option-2-connect-the-mcp-server-only) with `--scope user` here.** It is the recommended setup for the Code tab. The Prowler plugin ([Option 1](#option-1-install-the-prowler-plugin)) is not the recommended route for the desktop app — install it in the Claude Code CLI instead.
|
||||
</Note>
|
||||
|
||||
<Warning>
|
||||
**You cannot do this from inside the app.** The desktop app has no interface for adding an MCP server to a Claude Code session. **Settings → Connectors** configures the **Chat** tab, not the **Code** tab, so anything added there never reaches Claude Code. Trying to configure it from the app is the main reason this appears not to work.
|
||||
</Warning>
|
||||
|
||||
<Steps>
|
||||
<Step title="Add the server at user scope from a terminal">
|
||||
In a normal terminal — not inside the app:
|
||||
|
||||
```bash
|
||||
export PROWLER_API_KEY="pk_your_api_key_here"
|
||||
|
||||
claude mcp add --transport http prowler https://mcp.prowler.com/mcp \
|
||||
--header "Authorization: Bearer $PROWLER_API_KEY" \
|
||||
--scope user
|
||||
```
|
||||
|
||||
`--scope user` is what makes this work. It writes to `~/.claude.json`, the file the Code tab reads.
|
||||
</Step>
|
||||
|
||||
<Step title="Confirm it landed at user scope">
|
||||
```bash
|
||||
claude mcp get prowler
|
||||
```
|
||||
|
||||
The scope must be `user`. A `local`-scoped server is bound to the directory you ran the command in and will not load in an app session opened elsewhere.
|
||||
</Step>
|
||||
|
||||
<Step title="Restart the Claude app">
|
||||
Quit the app completely and reopen it. Configuration is read at startup.
|
||||
</Step>
|
||||
|
||||
<Step title="Verify in the Code tab">
|
||||
Open a **Code** tab session and ask for a Prowler tool: "Do you have access to the Prowler MCP tools?", it should respond with a list of available tools or confirming that it has access.
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
---
|
||||
|
||||
# Claude App Chat (Chat Tab)
|
||||
|
||||
Not covered by this page. The **Chat** tab is a separate surface: it does not read `~/.claude.json`, so a server added with `claude mcp add` appears in the CLI and in the Code tab but **never** in Chat. That is expected behavior, not a broken setup.
|
||||
|
||||
Chat reads `claude_desktop_config.json` and reaches the Prowler MCP Server through a local bridge.
|
||||
|
||||
<Card title="Connect the Claude App Chat" icon="comment" href="/user-guide/ai-agents/claude-desktop" horizontal>
|
||||
Separate guide: local bridge and its own configuration file
|
||||
</Card>
|
||||
|
||||
---
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
| Symptom | Likely cause | Fix |
|
||||
| --- | --- | --- |
|
||||
| `/mcp` shows `prowler` as failed | Rejected API key | Generate a new one in Prowler Cloud. With the plugin, reinstall it to re-prompt. |
|
||||
| No MCP servers configured | Server added at `local` scope from another directory | Run `claude mcp get prowler`, then re-add with `--scope user`. |
|
||||
| A stale entry overrides a working one | Precedence is local → project → user | `claude mcp remove prowler --scope local` |
|
||||
| Tools appear in the CLI but not in the app's **Code** tab | Server added at `local` scope, or the app was not restarted | Re-add with `--scope user`, then quit and reopen the app. See [Claude Code in the Desktop App](#claude-code-in-the-desktop-app-code-tab). |
|
||||
| Tools appear in the **Code** tab but not the **Chat** tab | Chat is a different surface with its own config file | Expected. Set Chat up separately, see [Claude App Chat](/user-guide/ai-agents/claude-desktop). |
|
||||
| No way to add the server from inside the app | The app has no MCP interface for Claude Code sessions | Configure it from a terminal with `--scope user`, then restart the app. See [Claude Code in the Desktop App](#claude-code-in-the-desktop-app-code-tab). |
|
||||
| Skill not invoked when expected | The prompt didn't match any skill's description | Name the task explicitly. For compliance triage, mention the framework plus "compliance" or "compliant". |
|
||||
| "Framework not supported" | Prowler Hub does not list the framework for that provider | Open an issue or PR at [github.com/prowler-cloud/prowler](https://github.com/prowler-cloud/prowler). |
|
||||
|
||||
### Authentication Fails With 401
|
||||
|
||||
- Confirm the header value includes the `Bearer ` prefix.
|
||||
- Check that `PROWLER_API_KEY` was set when you ran `claude mcp add` — the shell expands it at that moment and stores the resulting literal value. If the variable was empty, the stored header reads `Bearer ` with nothing after it. Verify with `claude mcp get prowler`.
|
||||
- Confirm the key has not been revoked in Prowler Cloud.
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools and capabilities
|
||||
</Card>
|
||||
<Card title="All MCP Clients" icon="plug" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configuration reference for every supported client
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -0,0 +1,142 @@
|
||||
---
|
||||
title: "Connect the Claude App Chat to Prowler MCP Server"
|
||||
sidebarTitle: "Claude App (Chat)"
|
||||
---
|
||||
|
||||
Connect the **Chat** tab of the Claude desktop app to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp`.
|
||||
|
||||
<Warning>
|
||||
**This page covers the Chat tab only.** Looking for **Claude Code** — either the CLI or the app's **Code** tab? Those are a different surface, with a different configuration file and a different connection method. See [Connect Claude Code](/user-guide/ai-agents/claude-code).
|
||||
</Warning>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Claude desktop app** installed and signed in.
|
||||
- **Node.js and npm**, to install the bridge.
|
||||
- **A Prowler Cloud account.** The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
|
||||
## Why "Add Custom Connector" Does Not Work
|
||||
|
||||
The app's **Settings → Connectors → Add custom connector** dialog is the obvious place to paste an MCP URL, but it does not fit the Prowler Cloud MCP Server for two independent reasons:
|
||||
|
||||
1. **Connectors authenticate with OAuth.** Authenticating with a fixed API key sent as a request header is a separate mechanism that Anthropic documents as **beta**, rolled out on request. Without it, the dialog offers a URL and OAuth client credentials, with nowhere to supply `Authorization: Bearer pk_...`.
|
||||
2. **Connectors do not connect from your machine.** Claude reaches your MCP server from Anthropic's cloud infrastructure rather than your local device. A Prowler MCP Server on `localhost`, behind a VPN, or restricted by an IP allowlist is unreachable that way regardless of authentication.
|
||||
|
||||
Use a local bridge instead, as described below.
|
||||
|
||||
## Step 1: Get Your Prowler API Key
|
||||
|
||||
Create an API key in Prowler Cloud and copy it. The key begins with `pk_` and is shown only once. Check the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide for details.
|
||||
|
||||
## Step 2: Install the Bridge
|
||||
|
||||
`mcp-remote` presents the remote HTTP server to Claude as a local STDIO server and injects the `Authorization` header. Install a pinned version into a dedicated directory:
|
||||
|
||||
```bash
|
||||
mkdir -p ~/.local/share/prowler-mcp-bridge
|
||||
cd ~/.local/share/prowler-mcp-bridge
|
||||
npm init -y
|
||||
npm install --save-exact mcp-remote@0.1.38
|
||||
```
|
||||
|
||||
<Warning>
|
||||
Do not configure Claude to run `npx mcp-remote` directly. `npx` can fetch and execute a new version on every launch, which means unreviewed code runs with access to your API key. Install a pinned version and point Claude at the installed binary.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
`mcp-remote` is community-maintained and is not an Anthropic product. Review it before use.
|
||||
</Note>
|
||||
|
||||
## Step 3: Edit the Configuration File
|
||||
|
||||
In the Claude app, go to **Settings → Developer** and click **Edit Config**. This reveals `claude_desktop_config.json`:
|
||||
|
||||
- **macOS:** `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
- **Windows:** `%APPDATA%\Claude\claude_desktop_config.json`
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/claude/claude-desktop-developer-settings.png" alt="Claude app Settings Developer tab showing the Edit Config button" />
|
||||
</Frame>
|
||||
|
||||
Add the following, replacing the `command` path with the absolute path to the installed binary and the placeholder with your API key:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"command": "/absolute/path/to/.local/share/prowler-mcp-bridge/node_modules/.bin/mcp-remote",
|
||||
"args": [
|
||||
"https://mcp.prowler.com/mcp",
|
||||
"--header",
|
||||
"Authorization: Bearer ${PROWLER_API_KEY}"
|
||||
],
|
||||
"env": {
|
||||
"PROWLER_API_KEY": "pk_your_api_key_here"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
<Note>
|
||||
**Local server:** Replace the URL with your own HTTP endpoint. Everything else stays the same.
|
||||
</Note>
|
||||
|
||||
## Step 4: Restart the App
|
||||
|
||||
Quit the Claude app completely and reopen it. Configuration is read at startup.
|
||||
|
||||
## Step 5: Start Using Prowler MCP
|
||||
|
||||
Open a Chat conversation and ask questions that use the Prowler tools:
|
||||
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"Summarize my CIS compliance status by provider"*
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/claude/claude-desktop-prowler-tools.png" alt="Claude app chat showing the Prowler MCP tools available" />
|
||||
</Frame>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Does Not Appear After Editing the Config
|
||||
|
||||
- Quit and reopen the app entirely — closing the window is not enough on macOS.
|
||||
- Confirm `claude_desktop_config.json` is valid JSON.
|
||||
- Confirm the `command` path points at a real executable. A wrong path surfaces as the server failing to start rather than as an auth error.
|
||||
|
||||
### Tools Appear in Claude Code but Not in Chat
|
||||
|
||||
Expected. The Chat tab does not read `~/.claude.json`, so servers added with `claude mcp add` never appear here. The Chat tab needs an entry in `claude_desktop_config.json`, which is what this guide sets up.
|
||||
|
||||
### Authentication Fails With 401
|
||||
|
||||
- Confirm the header value includes the `Bearer ` prefix.
|
||||
- Confirm the key has not been revoked in Prowler Cloud.
|
||||
|
||||
### Checking the Logs
|
||||
|
||||
- **macOS:** `~/Library/Logs/Claude/mcp*.log`
|
||||
- **Windows:** `%APPDATA%\Claude\logs\mcp*.log`
|
||||
|
||||
```bash
|
||||
tail -f ~/Library/Logs/Claude/mcp*.log
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools and capabilities
|
||||
</Card>
|
||||
<Card title="All MCP Clients" icon="plug" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configuration reference for every supported client
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -0,0 +1,188 @@
|
||||
---
|
||||
title: "Connect Codex / ChatGPT Desktop to Prowler MCP Server"
|
||||
sidebarTitle: "Codex / ChatGPT"
|
||||
---
|
||||
|
||||
Connect [OpenAI Codex](https://learn.chatgpt.com/docs/extend/mcp) to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp` so Codex can query findings, inspect checks, and manage your Prowler providers.
|
||||
|
||||
## Which Codex Surfaces Work
|
||||
|
||||
Codex keeps MCP servers in one file, `~/.codex/config.toml`. You can set it up from either the **Codex / ChatGPT desktop app** or the **Codex CLI** — both write to that same file, so pick whichever you already use.
|
||||
|
||||
| Surface | Set it up here | Notes |
|
||||
|---------|----------------|-------|
|
||||
| **[Codex / ChatGPT desktop app](https://learn.chatgpt.com/docs/app)** (macOS, Windows) | ✅ Yes | **Settings → MCP servers** |
|
||||
| **Codex CLI** (terminal) | ✅ Yes | `codex mcp` commands |
|
||||
| **Codex IDE extension** (VS Code) | Inherits | Works automatically once the app or CLI is configured |
|
||||
| **ChatGPT on the web** | ❌ No | Does not read local Codex configuration |
|
||||
|
||||
<Note>
|
||||
**Codex and ChatGPT share one desktop app.** Since July 2026 the standalone Codex app and the ChatGPT desktop app are the same application: Codex is a dedicated coding surface inside it, alongside Chat and Work. If you already had the Codex app, updating turns it into the new ChatGPT desktop app and it still opens in Codex. Either way, this guide applies.
|
||||
|
||||
Not to be confused with **ChatGPT Classic**, the name given to the previous-generation ChatGPT desktop app.
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
**Configure once, use everywhere.** The Codex documentation states that the ChatGPT desktop app, Codex CLI, and IDE extension "share this configuration. Once you configure your MCP servers, you can switch among those clients without redoing setup." Set the server up in the app or the CLI and the IDE extension picks it up with no extra work.
|
||||
</Note>
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **The Codex / ChatGPT desktop app, or Codex CLI 0.46.0 or later.** Remote MCP servers over streamable HTTP were added to the CLI in 0.46.0 — check with `codex --version` and upgrade if needed.
|
||||
- **A Prowler Cloud account.** The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
|
||||
## Step 1: Get Your Prowler API Key
|
||||
|
||||
Create an API key in Prowler Cloud and copy it. The key begins with `pk_` and is shown only once. Check the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide for details.
|
||||
|
||||
## Step 2: Add the Prowler MCP Server
|
||||
|
||||
The Prowler MCP Server needs two request headers: `Authorization` to authenticate you, and `User-Agent` because Codex does not send one by default.
|
||||
|
||||
Each tab below is a complete setup — follow the one that matches the surface you use.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Codex / ChatGPT desktop app">
|
||||
1. Open **Settings** and select **Plugins → MCPs**
|
||||
2. Click **Add server**
|
||||
3. Enter `prowler` as the name and choose type **Streamable HTTP**
|
||||
4. Enter the URL `https://mcp.prowler.com/mcp`
|
||||
5. Add two headers:
|
||||
|
||||
| Header | Value |
|
||||
|--------|-------|
|
||||
| `Authorization` | `Bearer pk_your_api_key_here` |
|
||||
| `User-Agent` | `codex` |
|
||||
|
||||
6. Save the server
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/codex/codex-app-mcp-servers.png" alt="Codex / ChatGPT desktop app Settings showing the MCP servers panel with the Add server dialog and both headers filled in" />
|
||||
</Frame>
|
||||
|
||||
<Note>
|
||||
**Enter the key directly here rather than using an environment variable.** Codex can read credentials from an environment variable, but desktop applications do not reliably inherit variables exported in a shell profile — on macOS an app launched from Finder or the Dock typically sees none of them. Pasting the key into the dialog is the approach that works consistently in the app.
|
||||
</Note>
|
||||
|
||||
<Warning>
|
||||
**This stores your API key in plain text** in `~/.codex/config.toml`. Treat that file accordingly: exclude it from dotfile repositories and config sync, and create the key from an account with the minimum permissions you need so its exposure is limited. Revoke and re-issue the key in Prowler Cloud if the file is ever shared.
|
||||
</Warning>
|
||||
</Tab>
|
||||
|
||||
<Tab title="Codex CLI">
|
||||
Register the server:
|
||||
|
||||
```bash
|
||||
codex mcp add prowler --url https://mcp.prowler.com/mcp
|
||||
```
|
||||
|
||||
Codex confirms with `Added global MCP server 'prowler'.`
|
||||
|
||||
Then add both headers by hand, since `codex mcp add` has no flag for headers. Open `~/.codex/config.toml` and complete the entry:
|
||||
|
||||
```toml
|
||||
[mcp_servers.prowler]
|
||||
url = "https://mcp.prowler.com/mcp"
|
||||
http_headers = { Authorization = "Bearer pk_your_api_key_here", "User-Agent" = "codex" }
|
||||
```
|
||||
|
||||
<Note>
|
||||
**Write the key literally rather than using an environment variable.** This is the form that works across every Codex surface. All of them read this same file, but only the CLI reliably sees variables exported in your shell profile — see the warning below.
|
||||
</Note>
|
||||
|
||||
<Warning>
|
||||
**This stores your API key in plain text** in `~/.codex/config.toml`. Treat that file accordingly: exclude it from dotfile repositories and config sync, and create the key from an account with the minimum permissions you need so its exposure is limited. Revoke and re-issue the key in Prowler Cloud if the file is ever shared.
|
||||
</Warning>
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
Restart Codex once you are done.
|
||||
|
||||
<Note>
|
||||
**Local server:** Replace the URL with your own HTTP endpoint. Everything else stays the same.
|
||||
</Note>
|
||||
|
||||
## Step 3: Verify the Connection
|
||||
|
||||
Run `/mcp` in the app or in a CLI session to list connected servers and their tools.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/codex/codex-mcp-slash-command.png" alt="Codex composer showing the /mcp command output with Prowler tools listed" />
|
||||
</Frame>
|
||||
|
||||
From the CLI you can also inspect the stored entry directly:
|
||||
|
||||
```bash
|
||||
codex mcp list # one row per server, with status and auth
|
||||
codex mcp get prowler # full entry, header values masked
|
||||
```
|
||||
|
||||
<Warning>
|
||||
**Verify rather than assume.** Codex silently ignores unrecognized keys in `config.toml` — a misspelled key name produces no error at all, and the server simply never receives your credentials. Always confirm with `codex mcp get prowler` after editing the file by hand.
|
||||
</Warning>
|
||||
|
||||
## Step 4: Start Using Prowler MCP
|
||||
|
||||
Ask Codex questions that use the Prowler tools:
|
||||
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"List my connected Prowler providers and their last scan date"*
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/codex/codex-prowler-query.png" alt="Codex answering a question about critical findings using Prowler MCP tools" />
|
||||
</Frame>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Startup Fails With HTTP 403 Forbidden
|
||||
|
||||
Codex reports a handshake failure on startup, with an HTML error page rather than a JSON response:
|
||||
|
||||
```
|
||||
⚠ MCP client for `prowler` failed to start: MCP startup failed: handshaking with MCP server
|
||||
failed: ... unexpected server response: HTTP 403: <html>
|
||||
<head><title>403 Forbidden</title></head>
|
||||
```
|
||||
|
||||
The `User-Agent` header is missing. Codex's HTTP client does not send one, and requests without it are rejected before reaching the MCP server. Note this is a **403**, not a 401 — so it is not an API key problem. Add the header as shown in [Step 2](#step-2-add-the-prowler-mcp-server); the value itself does not matter, only that the header is present.
|
||||
|
||||
### Authentication Fails With 401
|
||||
|
||||
- Run `codex mcp get prowler` and confirm the entry has the headers you expect. Values are masked, but a missing header shows as `-`.
|
||||
- If you used a literal header, confirm the value starts with `Bearer ` and contains the full key.
|
||||
- **If it works in the CLI but fails in the desktop app or the VS Code extension, you are almost certainly using an environment variable.** Those surfaces do not inherit your shell profile. Switch that entry to a literal `Authorization` header as shown in [Step 2](#step-2-add-the-prowler-mcp-server).
|
||||
- If you use an environment variable, verify it is set in the environment Codex was launched from: `echo $PROWLER_API_KEY`.
|
||||
- With `env_http_headers` the variable must include the `Bearer ` prefix. With `bearer_token_env_var` it must **not** — Codex adds the prefix itself.
|
||||
- Confirm the key has not been revoked in Prowler Cloud.
|
||||
|
||||
### Server Not Listed
|
||||
|
||||
- Confirm your Codex CLI version is 0.46.0 or later with `codex --version`.
|
||||
- Run `codex mcp get prowler`. If it reports the server is not found, the entry was not written or the TOML table name is misspelled.
|
||||
- Check for a typo in the key names. Codex ignores unknown keys without warning.
|
||||
|
||||
### Project-Scoped Config Is Ignored
|
||||
|
||||
A `.codex/config.toml` inside a project is loaded **only when the project is trusted**. If your entry lives there and does nothing, trust the project or move the entry to `~/.codex/config.toml`.
|
||||
|
||||
### Tools Do Not Appear After Editing the Config
|
||||
|
||||
Restart Codex. Configuration is read at startup. In the app, quit completely and reopen it, sometimes just clous the window is not enough.
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools and capabilities
|
||||
</Card>
|
||||
<Card title="All MCP Clients" icon="plug" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configuration reference for every supported client
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -0,0 +1,171 @@
|
||||
---
|
||||
title: "Connect Cursor to Prowler MCP Server"
|
||||
sidebarTitle: "Cursor"
|
||||
---
|
||||
|
||||
Connect [Cursor](https://cursor.com/docs/mcp) to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp` so the Cursor agent can query findings, inspect security checks, and manage your Prowler providers while you work.
|
||||
|
||||
Cursor supports remote MCP servers over HTTP natively, so no bridge or local installation is required.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Cursor** installed and authenticated. See the [official install guide](https://cursor.com/download).
|
||||
- **A Prowler Cloud account.** The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
|
||||
## Step 1: Get Your Prowler API Key
|
||||
|
||||
Create an API key in Prowler Cloud and copy it. The key begins with `pk_` and is shown only once. Check the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide for details.
|
||||
|
||||
## Step 2: Add the Prowler MCP Server
|
||||
|
||||
Cursor reads MCP servers from an `mcp.json` file. Choose the scope that fits your use case:
|
||||
|
||||
| Scope | File | Applies to |
|
||||
|-------|------|------------|
|
||||
| **Global** | `~/.cursor/mcp.json` | Every project you open in Cursor |
|
||||
| **Project** | `.cursor/mcp.json` in the project root | That project only |
|
||||
|
||||
Both files are merged. If the same server name appears in both, the project-level entry takes priority.
|
||||
|
||||
For Prowler, the **global** scope is usually the right choice — your findings are not tied to a single repository, and it keeps the API key out of any project directory that might be committed.
|
||||
|
||||
<Steps>
|
||||
<Step title="Open the MCP settings">
|
||||
From Agent Window open **Customize** in the Cursor sidebar, then select the MCP section.
|
||||
|
||||
On earlier versions, press `Cmd + Shift + J` (macOS) or `Ctrl + Shift + J` (Windows/Linux) to open Cursor Settings, then click **Tools & MCP** in the sidebar.
|
||||
|
||||
|
||||
</Step>
|
||||
|
||||
<Step title="Add a new MCP server">
|
||||
Click **New MCP Server** (or **Add Custom MCP**). Cursor opens `mcp.json` in the editor.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/cursor/cursor-customize-page.png" alt="Cursor Customize page with the MCP section open" />
|
||||
</Frame>
|
||||
</Step>
|
||||
|
||||
<Step title="Add the Prowler configuration">
|
||||
Paste the following, replacing the placeholder with your API key:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"url": "https://mcp.prowler.com/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer <your-api-key-here>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Save the file. Cursor picks up the change and connects to the server.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/cursor/cursor-mcp-json.png" alt="Cursor editor showing the completed mcp.json with the Prowler server entry" />
|
||||
</Frame>
|
||||
</Step>
|
||||
</Steps>
|
||||
|
||||
<Note>
|
||||
**Local server:** Replace the URL with your own HTTP endpoint. Everything else stays the same.
|
||||
</Note>
|
||||
|
||||
### Keeping the API Key Out of the File
|
||||
|
||||
Cursor resolves variables in the `command`, `args`, `env`, `url`, and `headers` fields, so you can reference an environment variable instead of writing the key into `mcp.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"prowler": {
|
||||
"url": "https://mcp.prowler.com/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer ${env:PROWLER_API_KEY}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Export the variable in your shell profile (`~/.zshrc`, `~/.bashrc`, or equivalent):
|
||||
|
||||
```bash
|
||||
export PROWLER_API_KEY="pk_your_api_key_here"
|
||||
```
|
||||
|
||||
<Note>
|
||||
The syntax is `${env:NAME}`, not a bare `${NAME}`. Restart Cursor after changing your shell profile so it inherits the new value.
|
||||
</Note>
|
||||
|
||||
<Warning>
|
||||
The `envFile` option does **not** work for remote servers — it is STDIO-only. Use `${env:...}` interpolation with variables set in your shell profile instead.
|
||||
</Warning>
|
||||
|
||||
This form is strongly recommended when using a **project-scoped** `.cursor/mcp.json`, since that file may be committed to version control.
|
||||
|
||||
## Step 3: Verify the Connection
|
||||
|
||||
Return to the MCP settings. The `prowler` server should be listed as enabled, with the Prowler tools shown beneath it.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/cursor/cursor-prowler-connected.png" alt="Cursor MCP settings showing the Prowler server connected with its tools listed" />
|
||||
</Frame>
|
||||
|
||||
## Step 4: Start Using Prowler MCP
|
||||
|
||||
Open the chat panel and ask questions that use the Prowler tools:
|
||||
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"Which of my providers failed the most CIS checks in the last scan?"*
|
||||
|
||||
Cursor asks for approval before running an MCP tool the first time.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/cursor/cursor-prowler-query.png" alt="Cursor chat answering a question about critical findings using Prowler MCP tools" />
|
||||
</Frame>
|
||||
|
||||
You can toggle individual tools on or off from the tools list at the top of the chat panel, which is useful for keeping the active tool count down.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Does Not Connect
|
||||
|
||||
- Check that `mcp.json` is valid JSON. A trailing comma or missing brace prevents the whole file from loading.
|
||||
- Open **MCP Logs** in the Output panel for the specific error.
|
||||
- Confirm the URL is exactly `https://mcp.prowler.com/mcp`.
|
||||
|
||||
### Authentication Fails With 401
|
||||
|
||||
- Verify the header value includes the `Bearer ` prefix: `"Bearer pk_..."`, not just the key.
|
||||
- Confirm the key has not been revoked in Prowler Cloud.
|
||||
- If using `${env:PROWLER_API_KEY}`, check the variable is set in the environment Cursor inherits. Restart Cursor after editing your shell profile — a value exported only in an already-open terminal will not reach the app.
|
||||
|
||||
### The Entire `mcp.json` Is Ignored
|
||||
|
||||
Remove any `"type": "streamable-http"` field. One such entry causes the Cursor CLI to drop every server in the file silently.
|
||||
|
||||
### Some Prowler Tools Are Missing
|
||||
|
||||
Cursor limits how many tools it exposes to the agent at once. With several MCP servers enabled you may exceed it, and some tools become unavailable. Disable servers you are not using, or turn off individual tools from the chat panel's tools list.
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools and capabilities
|
||||
</Card>
|
||||
<Card title="All MCP Clients" icon="plug" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configuration reference for every supported client
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -0,0 +1,49 @@
|
||||
---
|
||||
title: "Connect Your AI Agent to Prowler"
|
||||
sidebarTitle: "Overview"
|
||||
description: "Pick your AI agent and follow its guide to connect it to the Prowler Cloud MCP Server."
|
||||
---
|
||||
|
||||
Connect your AI agent to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp` so it can query findings, inspect security checks, and manage your Prowler providers.
|
||||
|
||||
Pick your agent below. Each guide is a full walkthrough with screenshots, verification steps, and the caveats specific to that client.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude Code" icon="terminal" href="/user-guide/ai-agents/claude-code">
|
||||
Plugin and MCP-only choices, and which Claude surfaces work
|
||||
</Card>
|
||||
<Card title="Claude Desktop App (Chat)" icon="comment" href="/user-guide/ai-agents/claude-desktop">
|
||||
The Chat tab, via a local bridge
|
||||
</Card>
|
||||
<Card title="Codex / ChatGPT Desktop App" icon="code" href="/user-guide/ai-agents/codex">
|
||||
ChatGPT Desktop App, Codex CLI, the VS Code extension through same config file
|
||||
</Card>
|
||||
<Card title="Cursor" icon="arrow-pointer" href="/user-guide/ai-agents/cursor">
|
||||
Agentic code editor. Global and project scopes
|
||||
</Card>
|
||||
<Card title="VS Code / Copilot" icon="microsoft" href="/user-guide/ai-agents/vscode">
|
||||
Agent mode with secure key prompts
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Before You Start
|
||||
|
||||
All guides need the same two things:
|
||||
|
||||
- A **Prowler Cloud account** with at least one cloud provider connected. [Sign up](https://cloud.prowler.com) if you do not have one.
|
||||
- A **Prowler API key**, created in Prowler Cloud. The key begins with `pk_` and is shown only once. See the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide.
|
||||
|
||||
<Note>
|
||||
Using an agent that is not listed here? Any MCP-compatible client can connect. See the [generic configuration reference](/getting-started/basic-usage/prowler-mcp#cloud-mcp-server-configuration-recommended) for the raw connection details.
|
||||
</Note>
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="What Is Prowler MCP" icon="cloud" href="/getting-started/products/prowler-mcp">
|
||||
How the MCP Server fits into Prowler
|
||||
</Card>
|
||||
<Card title="Configuration Reference" icon="gear" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Cloud and local server options, all clients
|
||||
</Card>
|
||||
</CardGroup>
|
||||
@@ -0,0 +1,145 @@
|
||||
---
|
||||
title: "Connect VS Code and GitHub Copilot to Prowler MCP Server"
|
||||
sidebarTitle: "VS Code / Copilot"
|
||||
---
|
||||
|
||||
Connect [Visual Studio Code](https://code.visualstudio.com/docs/agents/reference/mcp-configuration) and GitHub Copilot agent mode to the Prowler Cloud MCP Server at `https://mcp.prowler.com/mcp` so Copilot can query findings, inspect security checks, and manage your Prowler providers.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **VS Code 1.102 or later.** MCP support became generally available in 1.102.
|
||||
- **GitHub Copilot** enabled, with access to agent mode.
|
||||
- **A Prowler Cloud account.** The free tier is enough to start. Sign up at [cloud.prowler.com](https://cloud.prowler.com).
|
||||
|
||||
## Step 1: Get Your Prowler API Key
|
||||
|
||||
Create an API key in Prowler Cloud and copy it. The key begins with `pk_` and is shown only once. Check the [API Keys](/user-guide/tutorials/prowler-app-api-keys#creating-api-keys) guide for details.
|
||||
|
||||
## Step 2: Add the Prowler MCP Server
|
||||
|
||||
VS Code stores MCP servers in an `mcp.json` file. Choose the scope that fits your use case:
|
||||
|
||||
| Scope | How to open it | Applies to |
|
||||
|-------|----------------|------------|
|
||||
| **User** | Command palette → **MCP: Open User Configuration** | Every workspace |
|
||||
| **Workspace** | `.vscode/mcp.json` in the project root | That workspace only |
|
||||
|
||||
For Prowler, the **user** scope is usually the right choice — your findings are not tied to a single repository, and it keeps the API key out of any project directory that might be committed.
|
||||
|
||||
<Steps>
|
||||
<Step title="Open the MCP configuration">
|
||||
Open the command palette with `Cmd + Shift + P` (macOS) or `Ctrl + Shift + P` (Windows/Linux), then run **MCP: Open User Configuration**.
|
||||
|
||||
VS Code opens your user-level `mcp.json`. Use this command rather than navigating to the file by hand — the file lives inside your active profile folder, and the path differs per profile.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/vscode/vscode-command-palette.png" alt="VS Code command palette showing the MCP: Open User Configuration command" />
|
||||
</Frame>
|
||||
</Step>
|
||||
|
||||
<Step title="Add the Prowler configuration">
|
||||
Paste the following. This version prompts you for the API key on first use and stores it securely, so the key is never written into the file:
|
||||
|
||||
```json
|
||||
{
|
||||
"inputs": [
|
||||
{
|
||||
"type": "promptString",
|
||||
"id": "prowler-api-key",
|
||||
"description": "Prowler API Key",
|
||||
"password": true
|
||||
}
|
||||
],
|
||||
"servers": {
|
||||
"prowler": {
|
||||
"type": "http",
|
||||
"url": "https://mcp.prowler.com/mcp",
|
||||
"headers": {
|
||||
"Authorization": "Bearer ${input:prowler-api-key}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Save the file.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/vscode/vscode-mcp-json.png" alt="VS Code editor showing the completed mcp.json with the Prowler server entry" />
|
||||
</Frame>
|
||||
</Step>
|
||||
|
||||
<Step title="Enter your API key">
|
||||
Start the server. VS Code prompts for the Prowler API key. Paste it and press Enter — VS Code stores it securely and does not ask again.
|
||||
</Step>
|
||||
|
||||
</Steps>
|
||||
|
||||
<Warning>
|
||||
**The root key is `servers`, not `mcpServers`.** VS Code uses a different schema from Cursor, Claude, and most other clients. Copying a `mcpServers` snippet from elsewhere silently fails to register the server.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
**Local server:** Replace the URL with your own HTTP endpoint. Everything else stays the same.
|
||||
</Note>
|
||||
|
||||
## Step 3: Verify the Connection
|
||||
|
||||
Run **MCP: List Servers** from the command palette. The `prowler` server should appear as running.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/vscode/vscode-list-servers.png" alt="VS Code MCP: List Servers output showing the Prowler server running" />
|
||||
</Frame>
|
||||
|
||||
Select the server to start, stop, or restart it, and to view its output log if the connection fails.
|
||||
|
||||
## Step 4: Start Using Prowler MCP
|
||||
|
||||
Open the Chat view and switch the mode selector to **Agent**. Click the tools icon to confirm the Prowler tools are available, then ask:
|
||||
|
||||
- *"Show me all critical findings from my AWS accounts"*
|
||||
- *"What does the S3 bucket public access check do?"*
|
||||
- *"Summarize my CIS compliance status by provider"*
|
||||
|
||||
<Frame>
|
||||
<img src="/images/prowler-mcp/vscode/vscode-agent-tools.png" alt="VS Code Copilot Chat in agent mode showing the Prowler tools in the tools picker" />
|
||||
</Frame>
|
||||
|
||||
Copilot asks for confirmation before running an MCP tool for the first time.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server Does Not Appear
|
||||
|
||||
- Confirm the root key is `servers`, not `mcpServers`.
|
||||
- Confirm each server entry has `"type": "http"`.
|
||||
- Check that `mcp.json` is valid JSON.
|
||||
- Verify your VS Code version is 1.102 or later.
|
||||
|
||||
### Authentication Fails With 401
|
||||
|
||||
- Verify the header value includes the `Bearer ` prefix.
|
||||
- Confirm the key has not been revoked in Prowler Cloud.
|
||||
- If you mistyped the key at the prompt, run **MCP: List Servers**, select `prowler`, and restart it to be prompted again.
|
||||
|
||||
### Tools Do Not Appear in Chat
|
||||
|
||||
- Make sure the Chat view is in **Agent** mode. MCP tools are not available in Ask mode.
|
||||
- Open the tools picker and confirm the Prowler tools are enabled.
|
||||
|
||||
## Next Steps
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Tools Reference" icon="wrench" href="/getting-started/basic-usage/prowler-mcp-tools">
|
||||
Explore all available tools and capabilities
|
||||
</Card>
|
||||
<Card title="All MCP Clients" icon="plug" href="/getting-started/basic-usage/prowler-mcp">
|
||||
Configuration reference for every supported client
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Getting Help
|
||||
|
||||
- Search for existing [GitHub issues](https://github.com/prowler-cloud/prowler/issues)
|
||||
- Ask for help in our [Slack community](https://goto.prowler.com/slack)
|
||||
- Report a new issue on [GitHub](https://github.com/prowler-cloud/prowler/issues/new)
|
||||
@@ -184,7 +184,7 @@ $combinedCsv | Export-Csv -Path "CombinedCSV.csv" -NoTypeInformation
|
||||
|
||||
## TODO: Additional Improvements
|
||||
|
||||
Some services need to instantiate another service to perform a check. For instance, `cloudwatch` will instantiate Prowler's `iam` service to perform the `cloudwatch_cross_account_sharing_disabled` check. When the `iam` service is instantiated, it will perform the `__init__` function, and pull all the information required for that service. This provides an opportunity for an improvement in the above script to group related services together so that the `iam` services (or any other cross-service references) isn't repeatedily instantiated by grouping dependant services together. A complete mapping between these services still needs to be further investigated, but these are the cross-references that have been noted:
|
||||
Some services need to instantiate another service to perform a check. For instance, `cloudwatch` will instantiate Prowler's `iam` service to perform the `cloudwatch_cross_account_sharing_disabled` check. When the `iam` service is instantiated, it will perform the `__init__` function, and pull all the information required for that service. This provides an opportunity for an improvement in the above script to group related services together so that the `iam` services (or any other cross-service references) aren't repeatedly instantiated by grouping dependent services together. A complete mapping between these services still needs to be further investigated, but these are the cross-references that have been noted:
|
||||
|
||||
* inspector2 needs lambda and ec2
|
||||
* cloudwatch needs iam
|
||||
|
||||
@@ -114,7 +114,7 @@ The CSV format follows a standardized structure across all providers. The follow
|
||||
|
||||
#### CSV Headers Mapping
|
||||
|
||||
The following table shows the mapping between the CSV headers and the the providers fields:
|
||||
The following table shows the mapping between the CSV headers and the providers fields:
|
||||
|
||||
| Open Source Consolidated| AWS| GCP| AZURE| KUBERNETES
|
||||
|----------|----------|----------|----------|----------
|
||||
|
||||
@@ -434,7 +434,7 @@ prowler oci --oci-config-file /path/to/config
|
||||
|
||||
**Cause**: Insufficient IAM permissions
|
||||
|
||||
**Solution**: Add required policies (see [Required Permissions](./getting-started-oci.md#required-permissions))
|
||||
**Solution**: Add required policies (see [Required Permissions](/user-guide/providers/oci/getting-started-oci#required-permissions))
|
||||
|
||||
### Configuration Validation
|
||||
|
||||
|
||||
@@ -58,7 +58,7 @@ Before you begin, ensure you have:
|
||||
|
||||
### Authentication
|
||||
|
||||
Prowler supports multiple authentication methods for OCI. For detailed authentication setup, see the [OCI Authentication Guide](./authentication).
|
||||
Prowler supports multiple authentication methods for OCI. For detailed authentication setup, see the [OCI Authentication Guide](/user-guide/providers/oci/authentication).
|
||||
|
||||
**Note:** OCI Session Authentication and Config File Authentication both use the same `~/.oci/config` file. The difference is how the config file is generated - automatically via browser (session auth) or manually with API keys.
|
||||
|
||||
@@ -107,7 +107,7 @@ The easiest and most secure method is using OCI session authentication, which au
|
||||
|
||||
#### Alternative: Manual API Key Setup
|
||||
|
||||
If you prefer to manually generate API keys instead of using browser-based session authentication, see the detailed instructions in the [Authentication Guide](./authentication#config-file-authentication-manual-api-key-setup).
|
||||
If you prefer to manually generate API keys instead of using browser-based session authentication, see the detailed instructions in the [Authentication Guide](/user-guide/providers/oci/authentication#config-file-authentication-manual-api-key-setup).
|
||||
|
||||
**Note:** Both methods use the same `~/.oci/config` file - the difference is that manual setup uses static API keys while session authentication uses temporary session tokens.
|
||||
|
||||
|
||||
@@ -24,6 +24,12 @@ When enabled and configured:
|
||||
1. Security findings can be manually sent to Jira from the Findings table.
|
||||
2. Each finding creates a Jira work item with all the check's metadata, including guidance on how to remediate it.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
<VersionBadge version="5.36.0" />
|
||||
|
||||
Configuring and using the Jira integration requires the **Manage Integrations** permission. The Jira integration is tenant-wide, so it does not require **Unlimited Visibility** or any specific Provider Group. Findings sent to Jira are still limited to the providers the role can access.
|
||||
|
||||
## Configuration
|
||||
|
||||
To configure Jira integration in Prowler Cloud:
|
||||
|
||||
@@ -128,7 +128,7 @@ To resend the invitation to the user, it is necessary to explicitly **delete the
|
||||
|
||||
## Managing Groups and Roles
|
||||
|
||||
Roles combine administrative permissions with provider visibility. Administrative permissions control the actions a role can perform. Provider Groups and Unlimited Visibility control the providers, resources, findings, scans, and compliance results the role can access.
|
||||
Roles combine administrative permissions with provider visibility. Administrative permissions control the actions a role can perform. Provider Groups and Unlimited Visibility control the providers, resources, findings, scans, compliance results, and integrations the role can access.
|
||||
|
||||
<Note>
|
||||
**Only users that have the _Manage Account_ or _admin_ permission can access this section.**
|
||||
@@ -142,6 +142,12 @@ New roles have no provider visibility by default. Assign at least one Provider G
|
||||
|
||||
**Unlimited Visibility** grants organization-wide visibility across every provider, regardless of the Provider Groups assigned to the role. It does not grant administrative permissions.
|
||||
|
||||
#### Integration Visibility
|
||||
|
||||
<VersionBadge version="5.36.0" />
|
||||
|
||||
Integrations follow the visibility of the providers attached to them: a role can see an integration when it can access at least one of its providers, and only the providers visible to that role are listed on the integration. Editing or deleting an integration attached to providers outside the visibility of the role is not allowed. Integrations that are not attached to any provider, such as Jira, are tenant-wide and remain available to every role with the **Manage Integrations** permission.
|
||||
|
||||
#### Creating a Provider Group
|
||||
|
||||
Follow these steps to create a provider group in your account:
|
||||
|
||||
@@ -8,7 +8,7 @@ import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
|
||||
|
||||
<VersionBadge version="5.32.0" />
|
||||
|
||||
Scan Configuration lets you override, per provider, specific values in the default configuration Prowler's checks use during a scan. Each configuration modifies how specific checks behave, e.g.: thresholds, allowed values, retention windows, and you attach it to the providers that you want to use it on their next scan.
|
||||
Scan Configuration lets you override, per provider, specific values in the default configuration Prowler's checks use during a scan. Each configuration can modify how specific checks behave, such as thresholds, allowed values, and retention windows, or exclude checks and services from the scan scope. Attach it to the providers that should use it on their next scan.
|
||||
|
||||
<SubscriptionBanner />
|
||||
|
||||
@@ -54,6 +54,24 @@ gcp:
|
||||
storage_min_retention_days: 30
|
||||
```
|
||||
|
||||
### Limiting the Scan Scope
|
||||
|
||||
<VersionBadge version="5.35.0" />
|
||||
|
||||
Use `excluded_checks` to skip individual checks and `excluded_services` to skip every check in a service for the matching provider type:
|
||||
|
||||
```yaml
|
||||
aws:
|
||||
excluded_checks:
|
||||
- s3_bucket_public_access
|
||||
excluded_services:
|
||||
- ec2
|
||||
```
|
||||
|
||||
<Warning>
|
||||
When a Scan Configuration excludes checks or services, Prowler calculates overviews, aggregations, and other result-based information from the reduced scan scope. The displayed information reflects only the checks and services that ran, not a complete assessment of the provider. Consider the applied Scan Configuration when interpreting totals and security posture.
|
||||
</Warning>
|
||||
|
||||
## Creating a Scan Configuration
|
||||
|
||||
<Steps>
|
||||
|
||||
@@ -8,12 +8,14 @@ import { SubscriptionBanner } from "/snippets/subscription-banner.mdx"
|
||||
|
||||
<VersionBadge version="5.19.0" />
|
||||
|
||||
Prowler Cloud enables you to onboard all AWS accounts in your Organization through a single guided wizard. Instead of connecting accounts one by one, you can discover every account in your AWS Organization, select the ones you want to monitor, test connectivity, and launch scans — all from the Prowler Cloud UI.
|
||||
Prowler Cloud onboards every AWS account in your Organization through a single guided wizard. Instead of connecting accounts one by one, you can discover every account in your AWS Organization, select the ones you want to monitor, test connectivity, and launch scans — all from the Prowler Cloud UI.
|
||||
|
||||
<SubscriptionBanner>
|
||||
For CLI-based multi-account scanning, see [AWS Organizations in Prowler CLI](/user-guide/providers/aws/organizations).
|
||||
</SubscriptionBanner>
|
||||
|
||||
To follow this guide you need an active [Prowler Cloud](https://cloud.prowler.com) account and access to your AWS Organization [management account](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html) (or a registered delegated administrator account).
|
||||
|
||||
## Overview
|
||||
|
||||
### Individual Accounts vs Organizations
|
||||
@@ -25,225 +27,17 @@ For CLI-based multi-account scanning, see [AWS Organizations in Prowler CLI](/us
|
||||
|
||||
### How It Works
|
||||
|
||||
Onboarding deploys the **ProwlerScan Identity and Access Management (IAM) role** in your management account and in every member account. The wizard can deploy both from a **single CloudFormation stack** ([Step 4](#step-4-authenticate-with-your-management-account)), or you can deploy them yourself beforehand using Steps 1–2. The onboarding follows this sequence:
|
||||
<VersionBadge version="5.35.0" />
|
||||
|
||||
Onboarding deploys the **ProwlerScan Identity and Access Management (IAM) role** in your management account and in every member account. A **single CloudFormation stack** — launched from the wizard's **Create Stack in Management Account** button ([Step 2](#step-2-authenticate-with-your-management-account)) — creates the management account role **and** a service-managed StackSet that rolls the role out to your member accounts in one operation. Prefer to deploy the roles yourself? See [Deploy the Roles Manually](#deploy-the-roles-manually).
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/onboarding-flow.svg" alt="Onboarding flow: 1. Start the Wizard, 2. Deploy the Roles (single CloudFormation stack), 3. Discover and Connect, 4. Launch Scans" />
|
||||
</Frame>
|
||||
|
||||
## Key Concepts
|
||||
## Step 1: Start the Organization Wizard
|
||||
|
||||
### What Is an External ID?
|
||||
|
||||
An **External ID** is a security token that Prowler generates unique to your tenant. When Prowler assumes the IAM role in your AWS account, it presents this External ID to prove its identity.
|
||||
|
||||
This prevents the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html) — a scenario where an unauthorized party could trick AWS into granting access to your account. By requiring the External ID, only your specific Prowler tenant can assume the role.
|
||||
|
||||
You don't need to create the External ID yourself — Prowler generates it automatically and displays it in the wizard for you to copy.
|
||||
|
||||
### Two Roles Architecture
|
||||
|
||||
Prowler requires **two separate IAM roles** deployed in different places, each with a distinct purpose:
|
||||
|
||||
| Role | Where it lives | What it does | How to deploy it |
|
||||
|------|---------------|--------------|------------------|
|
||||
| **ProwlerScan** (management account) | Your management (root) account only | Discovers the Organization structure **and** scans the management account. Has additional Organizations discovery permissions. | By the wizard's **single stack** ([Step 4](#step-4-authenticate-with-your-management-account)), or on its own via **Quick Create** link or **manually** in the IAM Console ([Step 1](#step-1-create-the-management-account-role)). Cannot be deployed via StackSet. |
|
||||
| **ProwlerScan** (member accounts) | Every member account | Scans the account for security findings. | By the wizard's **single stack** ([Step 4](#step-4-authenticate-with-your-management-account)), or on its own via a **CloudFormation StackSet** ([Step 2](#step-2-deploy-the-cloudformation-stackset)). Automated across all accounts. |
|
||||
|
||||
<Frame caption="Both roles share the same name `ProwlerScan`. The management account role includes additional Organization discovery permissions.">
|
||||
<img src="/images/organizations/two-roles-architecture.svg" alt="Two Roles Architecture: ProwlerScan in management account (Quick Create or Manual, discovery + scanning) and ProwlerScan in member accounts (via StackSet, scanning only)" />
|
||||
</Frame>
|
||||
|
||||
<Note>
|
||||
**Same name, different permissions.** Both roles are named `ProwlerScan` — Prowler expects a consistent role name across all accounts. The management account role has the same scanning permissions as member accounts, plus additional Organizations discovery permissions (see [Step 1](#step-1-create-the-management-account-role) for the full list).
|
||||
</Note>
|
||||
|
||||
### What Is a CloudFormation StackSet?
|
||||
|
||||
A [CloudFormation StackSet](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/what-is-cfnstacksets.html) lets you deploy the same CloudFormation template across multiple AWS accounts in a single operation. Prowler uses a StackSet to deploy the **ProwlerScan** IAM role into every member account of your organization, so you don't have to create the role manually in each account.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Prowler Cloud Account
|
||||
|
||||
You need an active [Prowler Cloud](https://cloud.prowler.com) account. Each AWS account you connect will count as a provider in your subscription. See [Billing Impact](#billing-impact) for details.
|
||||
|
||||
### AWS Organization Enabled
|
||||
|
||||
Your AWS environment must have [AWS Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_introduction.html) enabled. You will need access to the **management account** (or a delegated administrator account) to provide the Organization ID and IAM Role ARN.
|
||||
|
||||
## Step 1: Create the Management Account Role
|
||||
|
||||
The first role you need to create is the **management account role**. This role allows Prowler to discover your Organization structure — listing accounts, OUs, and hierarchy.
|
||||
|
||||
<Warning>
|
||||
**StackSets do not deploy to the management account.** Organizational CloudFormation StackSets with service-managed permissions only target member accounts — this is an AWS limitation, not a Prowler one. The Prowler wizard works around this by having the **same** stack create the management account role (`DeployLocalRole=true`) alongside the StackSet, so a single deployment covers both. You can also create the management account role on its own via the Quick Create link ([Option A](#option-a-quick-create-link-fastest)) or manually ([Option B](#option-b-create-the-role-manually)).
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
**The role must be named `ProwlerScan`** — the same name as the role deployed to member accounts via StackSet. Prowler expects a consistent role name across all accounts in the Organization. If you use a different name, connection tests and scans will fail for the management account.
|
||||
</Note>
|
||||
|
||||
### Option A: Quick Create Link (Fastest)
|
||||
|
||||
The Prowler wizard provides a one-click link that opens the AWS Console with the CloudFormation template pre-configured. This creates a **CloudFormation Stack** (not a StackSet) that deploys the ProwlerScan role with Organizations permissions enabled in your management account.
|
||||
|
||||
<Tip>
|
||||
**[Open Quick Create Stack in AWS Console →](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?templateURL=https%3A%2F%2Fprowler-cloud-public.s3.eu-west-1.amazonaws.com%2Fpermissions%2Ftemplates%2Faws%2Fcloudformation%2Fprowler-scan-role.yml&stackName=Prowler¶m_EnableOrganizations=true)**
|
||||
|
||||
Opens the CloudFormation Console with the Prowler scan role template and `EnableOrganizations=true` pre-filled. You will need to enter the **ExternalId** parameter manually — copy it from the Prowler wizard ([Step 4](#step-4-authenticate-with-your-management-account)).
|
||||
</Tip>
|
||||
|
||||
1. Click **[Open Quick Create Stack in AWS Console →](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?templateURL=https%3A%2F%2Fprowler-cloud-public.s3.eu-west-1.amazonaws.com%2Fpermissions%2Ftemplates%2Faws%2Fcloudformation%2Fprowler-scan-role.yml&stackName=Prowler¶m_EnableOrganizations=true)** or use the **Create Stack in Management Account** button in the Prowler wizard (which also pre-fills the ExternalId).
|
||||
2. Enter the **ExternalId** parameter if not pre-filled.
|
||||
3. Check **"I acknowledge that AWS CloudFormation might create IAM resources with custom names"** and click **Create stack**.
|
||||
4. Wait for the stack to reach **CREATE_COMPLETE** status.
|
||||
|
||||
Take note of the **Role ARN** from the stack's **Outputs** tab — you will need it in the wizard.
|
||||
|
||||
### Option B: Create the Role Manually
|
||||
|
||||
1. Sign in to the [AWS IAM Console](https://console.aws.amazon.com/iam/) in your **management account**.
|
||||
|
||||
2. Go to **Roles > Create role** and select **Custom trust policy**.
|
||||
|
||||
3. Paste the following trust policy. This allows Prowler Cloud to assume the role using your tenant's External ID (you will get this from the Prowler wizard in [Step 3](#step-3-start-the-organization-wizard)):
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": {
|
||||
"AWS": "arn:aws:iam::232136659152:root"
|
||||
},
|
||||
"Action": "sts:AssumeRole",
|
||||
"Condition": {
|
||||
"StringEquals": {
|
||||
"sts:ExternalId": "<YOUR_EXTERNAL_ID>"
|
||||
},
|
||||
"StringLike": {
|
||||
"aws:PrincipalArn": "arn:aws:iam::232136659152:role/prowler*"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Replace `<YOUR_EXTERNAL_ID>` with the External ID shown in the Prowler wizard.
|
||||
|
||||
4. Attach the following AWS managed policies:
|
||||
- **SecurityAudit**
|
||||
- **ViewOnlyAccess**
|
||||
|
||||
This allows Prowler to also scan the management account for security findings, just like any other account.
|
||||
|
||||
5. Create an additional inline policy with the following permissions. These are specific to the management account and allow Prowler to discover your Organization structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "ProwlerOrganizationDiscovery",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"organizations:DescribeAccount",
|
||||
"organizations:DescribeOrganization",
|
||||
"organizations:ListAccounts",
|
||||
"organizations:ListAccountsForParent",
|
||||
"organizations:ListOrganizationalUnitsForParent",
|
||||
"organizations:ListRoots",
|
||||
"organizations:ListTagsForResource"
|
||||
],
|
||||
"Resource": "*"
|
||||
},
|
||||
{
|
||||
"Sid": "ProwlerStackSetManagement",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"organizations:RegisterDelegatedAdministrator",
|
||||
"iam:CreateServiceLinkedRole"
|
||||
],
|
||||
"Resource": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
<Tip>
|
||||
You can optionally restrict the `Resource` field to your specific Organization ARN (e.g., `arn:aws:organizations::123456789012:organization/o-abc123def4`) instead of `"*"` to minimize the blast radius.
|
||||
</Tip>
|
||||
|
||||
6. Name the role **`ProwlerScan`** and click **Create role**. Take note of the **Role ARN** — you will need it in the Prowler wizard.
|
||||
|
||||
The ARN follows this format: `arn:aws:iam::<account-id>:role/ProwlerScan`
|
||||
|
||||
<Warning>
|
||||
The role **must** be named `ProwlerScan`. Do not use a different name.
|
||||
</Warning>
|
||||
|
||||
<Note>
|
||||
If you just created the role, it may take up to **60 seconds** for AWS to propagate it. If you get an error in the Prowler wizard, wait a moment and try again.
|
||||
</Note>
|
||||
|
||||
## Step 2: Deploy the CloudFormation StackSet
|
||||
|
||||
This step deploys the **ProwlerScan** role to your member accounts using a CloudFormation StackSet. It is the **manual alternative** to letting the wizard's single stack create the StackSet for you ([Step 4](#step-4-authenticate-with-your-management-account)) — use it if you prefer to create and manage the StackSet yourself.
|
||||
|
||||
The StackSet uses **service-managed permissions**, which means AWS Organizations handles the cross-account deployment automatically — you don't need to create execution roles manually in each account. The StackSet deploys the ProwlerScan IAM role in every target member account, enabling Prowler to assume that role for cross-account scanning.
|
||||
|
||||
<Note>
|
||||
**Trusted access required:** CloudFormation StackSets must have trusted access enabled in your management account. Verify this in the AWS Console under **AWS Organizations > Settings > Trusted access for AWS CloudFormation StackSets**.
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
**The Prowler wizard now deploys this StackSet for you.** The **Create Stack in Management Account** button ([Step 4](#step-4-authenticate-with-your-management-account)) launches a single CloudFormation Stack that creates the management account role **and** a service-managed StackSet for your member accounts (`DeployStackSet=true`). Use the manual console steps below only if you prefer to create the StackSet yourself.
|
||||
</Note>
|
||||
|
||||
<Tip>
|
||||
**[Open StackSets Console →](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacksets/create)**
|
||||
|
||||
Opens the CloudFormation StackSets creation page directly. You will need to paste the template URL and ExternalId manually.
|
||||
</Tip>
|
||||
|
||||
1. Click the link above or navigate to **CloudFormation > StackSets > Create StackSet** in your management account.
|
||||
2. Choose **Service-managed permissions**.
|
||||
3. Select **Amazon S3 URL** as the template source and paste the following URL:
|
||||
```
|
||||
https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml
|
||||
```
|
||||
4. Set the **ExternalId** parameter to the External ID shown in the Prowler wizard.
|
||||
5. Choose your deployment targets (entire organization or specific OUs).
|
||||
6. Select the AWS regions where you want the role deployed.
|
||||
7. Click **Create StackSet**.
|
||||
|
||||
### Verify StackSet Deployment
|
||||
|
||||
After deploying, verify that all stack instances completed successfully:
|
||||
|
||||
1. In the CloudFormation Console, go to **StackSets** and select your Prowler StackSet.
|
||||
2. Click the **Stack instances** tab.
|
||||
3. Confirm that all instances show **Status: CURRENT** and **Stack status: CREATE_COMPLETE**.
|
||||
|
||||
Deployment typically takes **2–5 minutes** for medium-sized organizations. Large organizations (500+ accounts) may take longer.
|
||||
|
||||
<Note>
|
||||
**Prefer Terraform?** You can deploy the ProwlerScan role using Terraform instead. See the [StackSets deployment guide](/user-guide/providers/aws/organizations#deploying-prowler-iam-roles-across-aws-organizations) for the Terraform module.
|
||||
</Note>
|
||||
|
||||
### Key Considerations
|
||||
|
||||
- **Service-managed permissions**: Always select **Service-managed permissions** when creating the StackSet. This lets AWS Organizations manage the deployment automatically across current and future member accounts.
|
||||
- **Least privilege**: The ProwlerScan role deployed by the StackSet uses `SecurityAudit` and `ViewOnlyAccess` — AWS managed policies that grant read-only access — plus a small set of additional read-only permissions for services not covered by those policies. See the [CloudFormation template](https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml) for the full list. Prowler does not make any changes to your accounts.
|
||||
- **New accounts**: When you add new accounts to your AWS Organization, the StackSet automatically deploys the ProwlerScan role to them if you targeted the organization root or the relevant OU. Combined with Prowler's 6-hour automatic sync, new accounts are onboarded end-to-end without manual intervention.
|
||||
- **Management account**: Organizational StackSets **do not deploy to the management account itself**. If you want to scan the management account, you need to create the ProwlerScan role there separately using a regular CloudFormation Stack.
|
||||
|
||||
## Step 3: Start the Organization Wizard
|
||||
|
||||
Start the Prowler wizard. It walks you through deploying both roles — the management account role and the ProwlerScan role in member accounts — from a single CloudFormation stack ([Step 4](#step-4-authenticate-with-your-management-account)). If you already deployed them beforehand via Steps 1–2, the wizard simply picks up where you left off.
|
||||
The Prowler wizard walks you through the entire flow: deploying both roles from a single CloudFormation stack, discovering your accounts, testing connectivity, and launching scans.
|
||||
|
||||
### Open the Wizard
|
||||
|
||||
@@ -280,13 +74,13 @@ Start the Prowler wizard. It walks you through deploying both roles — the mana
|
||||
|
||||
Click **Next** to proceed to the authentication phase.
|
||||
|
||||
## Step 4: Authenticate with Your Management Account
|
||||
## Step 2: Authenticate with Your Management Account
|
||||
|
||||
The wizard's **Authentication Details** page guides you through three actions: deploying the roles in AWS, entering the deployment account Role ARN, and confirming the deployment. The deployment account is either the management account or, when delegated administrator mode is selected, the delegated administrator account.
|
||||
The **Authentication Details** page guides you through three actions: deploying the roles in AWS, entering the deployment account Role ARN, and confirming the deployment. The deployment account is either the management account or, when delegated administrator mode is selected, the delegated administrator account.
|
||||
|
||||
### External ID
|
||||
|
||||
The wizard displays a **Prowler External ID** at the top — auto-generated and unique to your tenant. Click the copy icon to copy it. The External ID is pre-filled into the deployment link, and the single stack applies it to both the management account role and the member-account StackSet.
|
||||
The wizard displays a **Prowler External ID** at the top — auto-generated and unique to your tenant. Click the copy icon to copy it. The External ID is pre-filled into the deployment link, and the single stack applies it to both the management account role and the member-account StackSet. Learn more in [What Is an External ID?](#what-is-an-external-id).
|
||||
|
||||
### Deploy the Roles
|
||||
|
||||
@@ -294,12 +88,20 @@ The wizard displays a **Prowler External ID** at the top — auto-generated and
|
||||
|
||||
The wizard deploys the deployment account role and the member-account StackSet in a **single** CloudFormation Stack:
|
||||
|
||||
<Note>
|
||||
**Prefer to use your own role?** You do not have to use the Quick Create template. Create the ProwlerScan role yourself — through the IAM Console, Terraform, or your own CloudFormation [(Following this guide)](#deploy-the-roles-manually) — and paste its ARN into the Role ARN field below. The role must use the external ID from the earlier step and include the trust policy and permissions described in [Deploy the Roles Manually](#deploy-the-roles-manually).
|
||||
</Note>
|
||||
|
||||
1. **Organizational Unit or Root ID** — enter the AWS OU (`ou-xxxx-yyyyyyyy`) or organization root (`r-xxxx`) you want to onboard. Prowler rolls the ProwlerScan role out to every member account under this target. Find it in the [AWS Organizations Console](https://console.aws.amazon.com/organizations/); use the **root ID** (`r-`) to cover the entire organization or an **OU ID** (`ou-`) to target a specific unit.
|
||||
|
||||
2. *(Optional)* Check **"I'm deploying from a delegated administrator account"** if you launch the stack from a delegated administrator account instead of the management account.
|
||||
|
||||
3. **Create Stack in Management Account** — or **Create Stack in Delegated Administrator Account** when delegated administrator mode is selected — opens a Quick Create link that deploys, in a single stack: the ProwlerScan role in the account where you launch the stack (`DeployLocalRole`, with `EnableOrganizations=true`) **and** a service-managed StackSet (`DeployStackSet`) that rolls the role out to your member accounts. The External ID, OU/Root ID, and deployment options are pre-filled.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/authentication-details.png" alt="Authentication Details form showing External ID, Organizational Unit or Root ID field, delegated administrator checkbox, deployment account stack button, deployment account Role ARN field, and deployment confirmation checkbox" />
|
||||
</Frame>
|
||||
|
||||
<Tip>
|
||||
**Finding your Organizational Unit or Root ID.** In the [AWS Organizations Console](https://console.aws.amazon.com/organizations/) the root (`r-…`) and OU (`ou-…`) IDs appear in the account tree, or run these from your management account:
|
||||
|
||||
@@ -311,17 +113,11 @@ aws organizations list-roots --query 'Roots[0].Id' --output text
|
||||
aws organizations list-organizational-units-for-parent --parent-id r-xxxx \
|
||||
--query 'OrganizationalUnits[].{Name:Name,Id:Id}' --output table
|
||||
```
|
||||
|
||||
If you deploy the CloudFormation template manually (instead of via the wizard link), set **`DeployStackSet=true`**, **`DeployLocalRole=true`**, and **`EnableOrganizations=true`**, then put the root/OU ID above into **`AWSOrganizationalUnitId`** (required whenever `DeployStackSet=true`). Leave **`DeployFromDelegatedAdmin=false`** unless you launch the stack from a delegated administrator account.
|
||||
</Tip>
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/authentication-details.png" alt="Authentication Details form showing External ID, Organizational Unit or Root ID field, delegated administrator checkbox, deployment account stack button, deployment account Role ARN field, and deployment confirmation checkbox" />
|
||||
</Frame>
|
||||
|
||||
### Enter the Deployment Account Role ARN
|
||||
|
||||
Paste the **Role ARN** created by the single stack above into the **Management Account Role ARN** field or, when delegated administrator mode is selected, the **Delegated Administrator Account Role ARN** field. If you deployed the management account role beforehand, use the role created in [Step 1](#step-1-create-the-management-account-role).
|
||||
Paste the **Role ARN** created by the stack above into the **Management Account Role ARN** field or, when delegated administrator mode is selected, the **Delegated Administrator Account Role ARN** field.
|
||||
|
||||
The ARN follows this format:
|
||||
```
|
||||
@@ -334,6 +130,10 @@ For example: `arn:aws:iam::123456789012:role/ProwlerScan`
|
||||
<img src="/images/organizations/role-arn-field.png" alt="Deployment account Role ARN field in the Authentication Details form" />
|
||||
</Frame>
|
||||
|
||||
<Note>
|
||||
It may take up to **60 seconds** for AWS to generate the IAM Role ARN after the stack completes. If the wizard reports an error, wait a moment and try again.
|
||||
</Note>
|
||||
|
||||
### Confirm and Discover
|
||||
|
||||
1. Check the box: **"The Stack has been successfully deployed in AWS"**.
|
||||
@@ -344,7 +144,7 @@ Here's what happens behind the scenes:
|
||||
- An asynchronous discovery is triggered to query your AWS Organization structure.
|
||||
- You will see a **"Gathering AWS Accounts..."** spinner — this typically takes **30 seconds to 2 minutes** depending on your organization size.
|
||||
|
||||
## Step 5: Select Accounts to Scan
|
||||
## Step 3: Select Accounts to Scan
|
||||
|
||||
### Understanding the Tree View
|
||||
|
||||
@@ -355,6 +155,7 @@ Once discovery completes, the wizard displays a **hierarchical tree view** of yo
|
||||
</Frame>
|
||||
|
||||
- The tree supports up to **5 levels of nesting** (Root > OUs > Sub-OUs > Accounts).
|
||||
- If you deployed the stack for just one OU, that OU will be preselected in the tree.
|
||||
- **Selecting an OU** automatically selects all accounts within it.
|
||||
- **Individual overrides**: deselect specific accounts even if the parent OU is selected.
|
||||
- The header shows **"X of Y accounts selected"** to track your selection.
|
||||
@@ -371,14 +172,12 @@ Only **ACTIVE** accounts can be selected for scanning:
|
||||
| **CLOSED** | No | Account has been closed. |
|
||||
|
||||
<Note>
|
||||
**Your existing data is safe.** If an AWS account is already connected to Prowler as an individual provider, it will appear in the tree with a checkmark indicator.
|
||||
**Your existing data is safe.** If an AWS account is already connected to Prowler as an individual provider, it appears in the tree with a checkmark indicator.
|
||||
|
||||
When you proceed:
|
||||
- The existing provider is **linked** to the organization — it is **not** duplicated.
|
||||
- All your **historical scan data and findings are preserved** — nothing is overwritten.
|
||||
- There is **no additional billing** — the existing provider is reused.
|
||||
|
||||
This is completely safe. You are simply associating the account with the organization for easier management.
|
||||
</Note>
|
||||
|
||||
### Custom Aliases
|
||||
@@ -387,14 +186,9 @@ You can edit the display name for each account before connecting. This alias is
|
||||
|
||||
### Blocked Accounts
|
||||
|
||||
Some accounts may appear as **blocked** (grayed out, not selectable). This happens when:
|
||||
- The account is **already linked to a different organization** in Prowler (`linked_to_other_organization`).
|
||||
Some accounts may appear as **blocked** (grayed out, not selectable) when the account is **already linked to a different organization** in Prowler (`linked_to_other_organization`). Hover over the blocked account to see the specific reason.
|
||||
|
||||
Hover over the blocked account to see the specific reason.
|
||||
|
||||
## Step 6: Test Connections
|
||||
|
||||
### How Connection Testing Works
|
||||
## Step 4: Test Connections
|
||||
|
||||
Click **Test Connections** to verify that Prowler can assume the **ProwlerScan** role in each selected member account.
|
||||
|
||||
@@ -402,154 +196,225 @@ Click **Test Connections** to verify that Prowler can assume the **ProwlerScan**
|
||||
<img src="/images/organizations/test-connections.png" alt="Connection testing in progress with spinners on each account" />
|
||||
</Frame>
|
||||
|
||||
- Each account shows a real-time status indicator:
|
||||
- **Spinner** — test in progress
|
||||
- **Green checkmark (✓)** — connection successful
|
||||
- **Red icon (✗)** — connection failed (hover to see the error)
|
||||
|
||||
### All Tests Pass
|
||||
Each account shows a real-time status indicator:
|
||||
- **Spinner** — test in progress
|
||||
- **Green checkmark (✓)** — connection successful
|
||||
- **Red icon (✗)** — connection failed (hover to see the error)
|
||||
|
||||
If every account connects successfully, you automatically advance to the next step.
|
||||
|
||||
### Some Tests Fail
|
||||
### When Some Tests Fail
|
||||
|
||||
An error banner appears: **"There was a problem connecting to some accounts."**
|
||||
|
||||
You have two options:
|
||||
An error banner appears: **"There was a problem connecting to some accounts."** You have two options:
|
||||
|
||||
**a) Fix and retry:**
|
||||
1. Go to the AWS Console and verify the StackSet deployed to the failing accounts.
|
||||
2. Check that the External ID in the StackSet matches the one shown in Prowler.
|
||||
3. Return to Prowler and click **Test Connections** — only the **failed accounts are re-tested** (smart retry). Accounts that already passed are not tested again.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/test-connections-button.png" alt="Test Connections button" />
|
||||
</Frame>
|
||||
|
||||
**b) Skip and continue:**
|
||||
Click **Skip Connection Validation** to proceed with only the accounts that connected successfully. The failed accounts will not be scanned.
|
||||
Click **Skip Connection Validation** to proceed with only the accounts that connected successfully. The failed accounts will not be scanned. This option is only available when at least one account connected successfully.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/connection-failures-skip.png" alt="Connection test results showing failed accounts with error banner and Skip Connection Validation button" />
|
||||
</Frame>
|
||||
|
||||
<Note>
|
||||
**Skip Connection Validation** is only available when at least one account connected successfully.
|
||||
</Note>
|
||||
If **no accounts** connected successfully, you cannot proceed. Fix the underlying connection issues — see [Troubleshooting](#troubleshooting) — and retry before launching scans.
|
||||
|
||||
### All Tests Fail
|
||||
|
||||
If **no accounts** connected successfully, you cannot proceed:
|
||||
|
||||
> *"No accounts connected successfully. Fix the connection errors and retry before launching scans."*
|
||||
|
||||
You must fix the underlying connection issues before continuing. See [Updating Credentials](#updating-credentials) below.
|
||||
|
||||
### Updating Credentials
|
||||
|
||||
If connection tests fail, here's how to fix common issues:
|
||||
|
||||
1. Open the [CloudFormation Console](https://console.aws.amazon.com/cloudformation/) and check that your StackSet instances show **CREATE_COMPLETE** for the failing accounts. If not, update the StackSet to include the missing OUs.
|
||||
2. Compare the **ExternalId** parameter in your StackSet with the External ID displayed in the Prowler wizard. They must match exactly.
|
||||
3. After fixing the issue in AWS, return to Prowler and click **Test Connections**. Only the previously failed accounts will be re-tested.
|
||||
|
||||
## Step 7: Launch Scans
|
||||
|
||||
### Choose Scan Schedule
|
||||
## Step 5: Launch Scans
|
||||
|
||||
The Organizations wizard uses the same schedule controls described in [Scan Scheduling](/user-guide/tutorials/prowler-scan-scheduling#schedule-options).
|
||||
|
||||
### Launch
|
||||
|
||||
Click **Save**, **Save and launch scan**, or **Launch scan**, depending on the selected schedule option. A toast notification confirms whether the schedule was saved, scans were launched, or both. The toast includes a link to the **Scans** page. Prowler redirects to the **Providers** page.
|
||||
|
||||
Scans are only launched for accounts that are accessible (passed connection testing) and were selected.
|
||||
Click **Save**, **Save and launch scan**, or **Launch scan**, depending on the selected schedule option. A toast notification confirms whether the schedule was saved, scans were launched, or both, and includes a link to the **Scans** page. Prowler then redirects to the **Providers** page. Scans launch only for accounts that passed connection testing and were selected.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/organizations/launch-scan.png" alt="Launch Scan step showing Accounts Connected confirmation, scan schedule selector, and Launch scan button" />
|
||||
</Frame>
|
||||
|
||||
### What Happens Next
|
||||
|
||||
After launching:
|
||||
- Scans appear in the **Scans** page as they start and complete.
|
||||
- Results populate the **Overview** and **Findings** pages.
|
||||
- Prowler runs an **automatic sync every 6 hours** to detect new accounts added to your Organization or accounts that have been removed. New accounts are onboarded automatically based on the parent OU configuration.
|
||||
- Prowler runs an **automatic sync every 6 hours** to detect accounts added to or removed from your Organization. New accounts under the targeted OU or root are onboarded automatically.
|
||||
|
||||
## Billing Impact
|
||||
|
||||
Each AWS account you connect through the Organizations wizard counts as one **provider** in your Prowler Cloud subscription.
|
||||
|
||||
- **Already-connected accounts**: if an account was already linked as a provider, adding it to the organization does **not** incur additional billing. The existing provider is reused.
|
||||
- **Large organizations**: connecting a 500-account organization will result in up to 500 providers on your subscription. Review your plan limits before proceeding.
|
||||
- **Large organizations**: connecting a 500-account organization results in up to 500 providers on your subscription. Review your plan limits before proceeding.
|
||||
- **Deleted providers**: if you later remove an account, the deleted provider no longer counts toward your subscription.
|
||||
|
||||
For pricing details, see [Prowler Cloud Pricing](https://prowler.com/pricing).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Invalid AWS Organization ID
|
||||
### Only Some Accounts Connect
|
||||
|
||||
*"Must be a valid AWS Organization ID"*
|
||||
Discovery succeeds and the tree view appears, but only one account — or a handful — passes the connection test. This almost always means the ProwlerScan role reached the deployment account but not every member account.
|
||||
|
||||
- Verify the Organization ID format: `o-` followed by 10–32 lowercase alphanumeric characters (e.g., `o-abc123def4`)
|
||||
- Copy it directly from the [AWS Organizations Console](https://console.aws.amazon.com/organizations/) to avoid typos
|
||||
- **Confirm the StackSet deployed.** Open the [CloudFormation Console](https://console.aws.amazon.com/cloudformation/) in the deployment account, select your Prowler StackSet, open the **Stack instances** tab, and confirm every instance shows **Status: CURRENT** and **Stack status: CREATE_COMPLETE**. Instances still in progress or in a failed state explain the missing accounts.
|
||||
- **Check the targeted OU or root.** The single stack only rolls the role out to accounts under the **Organizational Unit or Root ID** you entered in [Step 2](#step-2-authenticate-with-your-management-account). Accounts in other OUs are not covered — redeploy targeting the organization root (`r-`) or add the missing OUs.
|
||||
- **Verify the deployment account.** The role is created only in the account where you launched the stack. If you deployed from a **delegated administrator account**, confirm that account is a **registered delegated administrator** for CloudFormation StackSets (registered through AWS Organizations), not just a regular member account. A regular member account cannot create a service-managed StackSet, so only its own role is created — leaving every other account without the role.
|
||||
- **Suspended accounts** cannot be scanned. Deselect them and proceed.
|
||||
|
||||
### Invalid IAM Role ARN
|
||||
### No Accounts Connect
|
||||
|
||||
*"Must be a valid IAM Role ARN"*
|
||||
No account passes the connection test.
|
||||
|
||||
- Verify the ARN format: `arn:aws:iam::<12-digit-account-id>:role/<role-name>`
|
||||
- Copy the ARN directly from the [IAM Console](https://console.aws.amazon.com/iam/) in your management account
|
||||
- **External ID mismatch.** Compare the **ExternalId** parameter in your StackSet with the External ID shown in the Prowler wizard. They must match exactly.
|
||||
- **StackSet not deployed.** Confirm the StackSet exists and its instances reached **CREATE_COMPLETE**. If you deployed the roles manually, verify [trusted access for CloudFormation StackSets](#member-account-role-stackset) is enabled.
|
||||
- **IP-based policies.** If your accounts restrict access by IP, allow the [Prowler Cloud egress IPs](/security/networking).
|
||||
|
||||
### Authentication Failed
|
||||
### Authentication Fails or Times Out
|
||||
|
||||
*"Authentication failed. Please verify the StackSet deployment and Role ARN"*
|
||||
*"Authentication failed. Please verify the StackSet deployment and Role ARN"* or *"Authentication timed out"*
|
||||
|
||||
- Verify the management account role exists and was created in [Step 1](#step-1-create-the-management-account-role)
|
||||
- Confirm the trust policy includes the correct External ID from the wizard
|
||||
- Check the role has all Organizations discovery permissions listed in [Step 1](#step-1-create-the-management-account-role)
|
||||
- Double-check the Role ARN format and account ID for typos
|
||||
- Verify the deployment account role exists and is named exactly `ProwlerScan`.
|
||||
- Confirm the trust policy includes the correct External ID from the wizard.
|
||||
- Check the role has the Organizations discovery permissions listed in [Deploy the Roles Manually](#management-account-role).
|
||||
- Double-check the Role ARN format and account ID for typos.
|
||||
- Retry — the role can take up to **60 seconds** to propagate, and a second attempt often succeeds. For very large organizations (500+ accounts), allow extra time for discovery.
|
||||
|
||||
### Authentication Timed Out
|
||||
### Invalid Organization ID or Role ARN
|
||||
|
||||
*"Authentication timed out"*
|
||||
*"Must be a valid AWS Organization ID"* or *"Must be a valid IAM Role ARN"*
|
||||
|
||||
- Retry the authentication step — the second attempt often succeeds
|
||||
- Check for AWS API rate limiting on the Organizations service
|
||||
- For very large organizations (500+ accounts), allow extra time for discovery
|
||||
|
||||
### Connection Test Fails for All Accounts
|
||||
|
||||
No accounts pass the connection test.
|
||||
|
||||
- Verify the CloudFormation StackSet was deployed — complete [Step 2](#step-2-deploy-the-cloudformation-stackset) and wait for stack instances to reach **CREATE_COMPLETE**
|
||||
- Check that the **ExternalId** parameter in the StackSet matches the External ID shown in the Prowler wizard
|
||||
- If your accounts use IP-based IAM policies, allow [Prowler Cloud egress IPs](/security/networking)
|
||||
|
||||
### Connection Test Fails for Some Accounts
|
||||
|
||||
Some accounts show a red icon while others pass.
|
||||
|
||||
- Expand the StackSet deployment to include the OUs containing the failing accounts
|
||||
- Suspended accounts cannot be scanned — deselect them and proceed
|
||||
- Ensure the [STS regional endpoint](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html) is enabled in the account's region
|
||||
- After fixing, click **Test Connections** — only the failed accounts will be re-tested
|
||||
|
||||
### No Accounts Connected Successfully
|
||||
|
||||
*"No accounts connected successfully. Fix the connection errors and retry before launching scans."*
|
||||
|
||||
- Hover over the red icon on each account to see the specific error
|
||||
- Fix the underlying issues using the guidance above
|
||||
- Click **Test Connections** to retry
|
||||
- Organization ID format: `o-` followed by 10–32 lowercase alphanumeric characters (e.g., `o-abc123def4`).
|
||||
- Role ARN format: `arn:aws:iam::<12-digit-account-id>:role/ProwlerScan`.
|
||||
- Copy both directly from the AWS Console to avoid typos.
|
||||
|
||||
### Failed to Apply Discovery
|
||||
|
||||
*"Failed to apply discovery"*
|
||||
|
||||
- Check the `blocked_reasons` field for any blocked accounts
|
||||
- Retry the operation
|
||||
- If the error persists, contact [Prowler Support](mailto:support@prowler.com)
|
||||
- Check the `blocked_reasons` field for any blocked accounts and retry the operation.
|
||||
- If the error persists, contact [Prowler Support](mailto:support@prowler.com).
|
||||
|
||||
## Deploy the Roles Manually
|
||||
|
||||
The wizard's **Create Stack** button is the fastest path, but you can create both roles yourself — for example with Terraform or your own CloudFormation — and paste the management account Role ARN into [Step 2](#step-2-authenticate-with-your-management-account). Both roles must be named `ProwlerScan`, since Prowler expects a consistent role name across all accounts.
|
||||
|
||||
<Note>
|
||||
**Prefer Terraform?** You can deploy the ProwlerScan role across the organization with Terraform instead of CloudFormation. See the [StackSets deployment guide](/user-guide/providers/aws/organizations#deploying-prowler-iam-roles-across-aws-organizations) for the module.
|
||||
</Note>
|
||||
|
||||
### Management Account Role
|
||||
|
||||
The management account role lets Prowler discover your Organization structure — listing accounts, OUs, and hierarchy — and scan the management account itself. StackSets with service-managed permissions do not deploy to the management account, so this role is always created separately from the member-account StackSet.
|
||||
|
||||
1. Sign in to the [AWS IAM Console](https://console.aws.amazon.com/iam/) in your **management account** (or delegated administrator account).
|
||||
2. Go to **Roles > Create role** and select **Custom trust policy**.
|
||||
3. Paste the following trust policy, replacing `<YOUR_EXTERNAL_ID>` with the External ID shown in the Prowler wizard:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": {
|
||||
"AWS": "arn:aws:iam::232136659152:root"
|
||||
},
|
||||
"Action": "sts:AssumeRole",
|
||||
"Condition": {
|
||||
"StringEquals": {
|
||||
"sts:ExternalId": "<YOUR_EXTERNAL_ID>"
|
||||
},
|
||||
"StringLike": {
|
||||
"aws:PrincipalArn": "arn:aws:iam::232136659152:role/prowler*"
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
4. Attach the AWS managed policies **SecurityAudit** and **ViewOnlyAccess** so Prowler can scan the management account for security findings.
|
||||
5. Add an inline policy with the Organizations discovery permissions:
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "ProwlerOrganizationDiscovery",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"organizations:DescribeAccount",
|
||||
"organizations:DescribeOrganization",
|
||||
"organizations:ListAccounts",
|
||||
"organizations:ListAccountsForParent",
|
||||
"organizations:ListOrganizationalUnitsForParent",
|
||||
"organizations:ListRoots",
|
||||
"organizations:ListTagsForResource"
|
||||
],
|
||||
"Resource": "*"
|
||||
},
|
||||
{
|
||||
"Sid": "ProwlerStackSetManagement",
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"organizations:RegisterDelegatedAdministrator",
|
||||
"iam:CreateServiceLinkedRole"
|
||||
],
|
||||
"Resource": "*"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
<Tip>
|
||||
You can restrict the `Resource` field to your specific Organization ARN (e.g., `arn:aws:organizations::123456789012:organization/o-abc123def4`) instead of `"*"` to minimize the blast radius.
|
||||
</Tip>
|
||||
|
||||
6. Name the role **`ProwlerScan`** and click **Create role**. The ARN follows the format `arn:aws:iam::<account-id>:role/ProwlerScan` — paste it into the wizard.
|
||||
|
||||
### Member Account Role (StackSet)
|
||||
|
||||
Deploy the ProwlerScan role to every member account with a [CloudFormation StackSet](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/what-is-cfnstacksets.html), so you don't create the role manually in each account.
|
||||
|
||||
<Note>
|
||||
**Trusted access required.** CloudFormation StackSets must have trusted access enabled in your management account. Verify this under **AWS Organizations > Settings > Trusted access for AWS CloudFormation StackSets**.
|
||||
</Note>
|
||||
|
||||
1. In your management account, navigate to **CloudFormation > StackSets > Create StackSet** ([open directly](https://us-east-1.console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacksets/create)).
|
||||
2. Choose **Service-managed permissions** so AWS Organizations deploys the role automatically across current and future member accounts.
|
||||
3. Select **Amazon S3 URL** as the template source and paste:
|
||||
```
|
||||
https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml
|
||||
```
|
||||
4. Set the **ExternalId** parameter to the External ID shown in the Prowler wizard.
|
||||
5. Choose your deployment targets (entire organization or specific OUs) and regions, then click **Create StackSet**.
|
||||
6. Open the **Stack instances** tab and confirm every instance shows **Status: CURRENT** and **Stack status: CREATE_COMPLETE**. Deployment typically takes **2–5 minutes**; large organizations (500+ accounts) may take longer.
|
||||
|
||||
The StackSet role uses read-only access only (`SecurityAudit`, `ViewOnlyAccess`, plus a small set of additional read-only permissions). Prowler makes no changes to your accounts. See the [CloudFormation template](https://prowler-cloud-public.s3.eu-west-1.amazonaws.com/permissions/templates/aws/cloudformation/prowler-scan-role.yml) for the full list. When you add new accounts under the targeted OU or root, the StackSet deploys the role automatically, and Prowler's 6-hour sync onboards them end-to-end.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### What Is an External ID?
|
||||
|
||||
An **External ID** is a security token that Prowler generates unique to your tenant. When Prowler assumes the IAM role in your AWS account, it presents this External ID to prove its identity.
|
||||
|
||||
This prevents the [confused deputy problem](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html) — a scenario where an unauthorized party could trick AWS into granting access to your account. By requiring the External ID, only your specific Prowler tenant can assume the role. Prowler generates it automatically and displays it in the wizard for you to copy.
|
||||
|
||||
### Two Roles Architecture
|
||||
|
||||
Prowler uses **two IAM roles**, both named `ProwlerScan` but deployed in different places:
|
||||
|
||||
| Role | Where it lives | What it does |
|
||||
|------|---------------|--------------|
|
||||
| **ProwlerScan** (management account) | Your management (or delegated administrator) account | Discovers the Organization structure **and** scans that account. Includes additional Organizations discovery permissions. |
|
||||
| **ProwlerScan** (member accounts) | Every member account | Scans the account for security findings. |
|
||||
|
||||
Both roles share the name `ProwlerScan` because Prowler expects a consistent role name across all accounts. The single CloudFormation stack in [Step 2](#step-2-authenticate-with-your-management-account) deploys both at once.
|
||||
|
||||
<Frame caption="Both roles share the same name `ProwlerScan`. The management account role includes additional Organization discovery permissions.">
|
||||
<img src="/images/organizations/two-roles-architecture.svg" alt="Two Roles Architecture: ProwlerScan in management account (discovery + scanning) and ProwlerScan in member accounts (via StackSet, scanning only)" />
|
||||
</Frame>
|
||||
|
||||
### What Is a CloudFormation StackSet?
|
||||
|
||||
A [CloudFormation StackSet](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/what-is-cfnstacksets.html) deploys the same CloudFormation template across multiple AWS accounts in a single operation. Prowler uses a service-managed StackSet to deploy the **ProwlerScan** IAM role into every member account of your organization, so you don't create the role manually in each account. StackSets do not deploy to the management account, which is why that role is created separately.
|
||||
|
||||
## What's Next
|
||||
|
||||
|
||||
@@ -4,6 +4,14 @@ All notable changes to the **Prowler MCP Server** are documented in this file.
|
||||
|
||||
<!-- changelog: release notes start -->
|
||||
|
||||
## [0.8.0] (Prowler v5.35.0)
|
||||
|
||||
### 🔄 Changed
|
||||
|
||||
- Core Prowler tool namespace from the `prowler_app_*` prefix to `prowler_*` [(#12017)](https://github.com/prowler-cloud/prowler/pull/12017)
|
||||
|
||||
---
|
||||
|
||||
## [0.7.2] (Prowler v5.28.1)
|
||||
|
||||
### 🐞 Fixed
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
Core Prowler tool namespace from the `prowler_app_*` prefix to `prowler_*`
|
||||
@@ -4,6 +4,18 @@ All notable changes to the **Prowler SDK** are documented in this file.
|
||||
|
||||
<!-- changelog: release notes start -->
|
||||
|
||||
## [5.35.0] (Prowler v5.35.0)
|
||||
|
||||
### 🚀 Added
|
||||
|
||||
- `excluded_checks` and `excluded_services` in scan configurations to narrow the execution scope [(#12028)](https://github.com/prowler-cloud/prowler/pull/12028)
|
||||
|
||||
### 🔐 Security
|
||||
|
||||
- Jira tenant information requests validate site names and do not follow redirects [(#12012)](https://github.com/prowler-cloud/prowler/pull/12012)
|
||||
|
||||
---
|
||||
|
||||
## [5.34.0] (Prowler v5.34.0)
|
||||
|
||||
### 🚀 Added
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
Alibaba Cloud SSH and RDP security group checks no longer produce false negatives when allowed rules use capitalized `Policy="Accept"` values
|
||||
@@ -0,0 +1 @@
|
||||
Fix invalid escape sequence `SyntaxWarning` raised on startup by the S3 bucket name validation regex
|
||||
@@ -0,0 +1 @@
|
||||
Jira output rendering supports grouped Finding Group issues with caller-provided links and capped or uncapped finding copy
|
||||