diff --git a/skills/django-drf/SKILL.md b/skills/django-drf/SKILL.md index 93ea6219f1..69eaa267fc 100644 --- a/skills/django-drf/SKILL.md +++ b/skills/django-drf/SKILL.md @@ -1,8 +1,6 @@ --- name: django-drf -description: > - Django REST Framework patterns. - Trigger: When implementing generic DRF APIs (ViewSets, serializers, routers, permissions, filtersets). For Prowler API specifics (RLS/RBAC/Providers), also use prowler-api. +description: "Trigger: When implementing generic DRF APIs such as viewsets, serializers, routers, permissions, pagination, or filtersets, including JSON:API-capable endpoints. Applies the shared DRF execution patterns used in Prowler." license: Apache-2.0 metadata: author: prowler-cloud @@ -15,491 +13,51 @@ metadata: allowed-tools: Read, Edit, Write, Glob, Grep, Bash, WebFetch, WebSearch, Task --- -## Critical Patterns - -- ALWAYS separate serializers by operation: Read / Create / Update / Include -- ALWAYS use `filterset_class` for complex filtering (not `filterset_fields`) -- ALWAYS validate unknown fields in write serializers (inherit `BaseWriteSerializer`) -- ALWAYS use `select_related`/`prefetch_related` in `get_queryset()` to avoid N+1 -- ALWAYS handle `swagger_fake_view` in `get_queryset()` for schema generation -- ALWAYS use `@extend_schema_field` for OpenAPI docs on `SerializerMethodField` -- NEVER put business logic in serializers - use services/utils -- NEVER use auto-increment PKs - use UUIDv4 or UUIDv7 -- NEVER use trailing slashes in URLs (`trailing_slash=False`) - -> **Note:** `swagger_fake_view` is specific to **drf-spectacular** for OpenAPI schema generation. - ---- - -## Implementation Checklist - -When implementing a new endpoint, review these patterns in order: - -| # | Pattern | Reference | Key Points | -|---|---------|-----------|------------| -| 1 | **Models** | `api/models.py` | UUID PK, `inserted_at`/`updated_at`, `JSONAPIMeta.resource_name` | -| 2 | **ViewSets** | `api/base_views.py`, `api/v1/views.py` | Inherit `BaseRLSViewSet`, `get_queryset()` with N+1 prevention | -| 3 | **Serializers** | `api/v1/serializers.py` | Separate Read/Create/Update/Include, inherit `BaseWriteSerializer` | -| 4 | **Filters** | `api/filters.py` | Use `filterset_class`, inherit base filter classes | -| 5 | **Permissions** | `api/base_views.py` | `required_permissions`, `set_required_permissions()` | -| 6 | **Pagination** | `api/pagination.py` | Custom pagination class if needed | -| 7 | **URL Routing** | `api/v1/urls.py` | `trailing_slash=False`, kebab-case paths | -| 8 | **OpenAPI Schema** | `api/v1/views.py` | `@extend_schema_view` with drf-spectacular | -| 9 | **Tests** | `api/tests/test_views.py` | JSON:API content type, fixture patterns | - -> **Full file paths**: See [references/file-locations.md](references/file-locations.md) - ---- - -## Decision Trees - -### Which Serializer? -``` -GET list/retrieve → Serializer -POST create → CreateSerializer -PATCH update → UpdateSerializer -?include=... → IncludeSerializer -``` - -### Which Base Serializer? -``` -Read-only serializer → BaseModelSerializerV1 -Create with tenant_id → RLSSerializer + BaseWriteSerializer (auto-injects tenant_id on create) -Update with validation → BaseWriteSerializer (tenant_id already exists on object) -Non-model data → BaseSerializerV1 -``` - -### Which Filter Base? -``` -Direct FK to Provider → BaseProviderFilter -FK via Scan → BaseScanProviderFilter -No provider relation → FilterSet -``` - -### Which Base ViewSet? -``` -RLS-protected model → BaseRLSViewSet (most common) -Tenant operations → BaseTenantViewset -User operations → BaseUserViewset -No RLS required → BaseViewSet (rare) -``` - -### Resource Name Format? -``` -Single word model → plural lowercase (Provider → providers) -Multi-word model → plural lowercase kebab (ProviderGroup → provider-groups) -Through/join model → parent-child pattern (UserRoleRelationship → user-roles) -Aggregation/overview → descriptive kebab plural (ComplianceOverview → compliance-overviews) -``` - ---- - -## Serializer Patterns - -### Base Class Hierarchy - -```python -# Read serializer (most common) -class ProviderSerializer(RLSSerializer): - class Meta: - model = Provider - fields = ["id", "provider", "uid", "alias", "connected", "inserted_at"] - -# Write serializer (validates unknown fields) -class ProviderCreateSerializer(RLSSerializer, BaseWriteSerializer): - class Meta: - model = Provider - fields = ["provider", "uid", "alias"] - -# Include serializer (sparse fields for ?include=) -class ProviderIncludeSerializer(RLSSerializer): - class Meta: - model = Provider - fields = ["id", "alias"] # Minimal fields -``` - -### SerializerMethodField with OpenAPI - -```python -from drf_spectacular.utils import extend_schema_field - -class ProviderSerializer(RLSSerializer): - connection = serializers.SerializerMethodField(read_only=True) - - @extend_schema_field({ - "type": "object", - "properties": { - "connected": {"type": "boolean"}, - "last_checked_at": {"type": "string", "format": "date-time"}, - }, - }) - def get_connection(self, obj): - return { - "connected": obj.connected, - "last_checked_at": obj.connection_last_checked_at, - } -``` - -### Included Serializers (JSON:API) - -```python -class ScanSerializer(RLSSerializer): - included_serializers = { - "provider": "api.v1.serializers.ProviderIncludeSerializer", - } -``` - -### Sensitive Data Masking - -```python -def to_representation(self, instance): - data = super().to_representation(instance) - # Mask by default, expose only on explicit request - fields_param = self.context.get("request").query_params.get("fields[my-model]", "") - if "api_key" in fields_param: - data["api_key"] = instance.api_key_decoded - else: - data["api_key"] = "****" if instance.api_key else None - return data -``` - ---- - -## ViewSet Patterns - -### get_queryset() with N+1 Prevention - -**Always combine** `swagger_fake_view` check with `select_related`/`prefetch_related`: - -```python -def get_queryset(self): - # REQUIRED: Return empty queryset for OpenAPI schema generation - if getattr(self, "swagger_fake_view", False): - return Provider.objects.none() - - # N+1 prevention: eager load relationships - return Provider.objects.select_related( - "tenant", - ).prefetch_related( - "provider_groups", - Prefetch("tags", queryset=ProviderTag.objects.filter(tenant_id=self.request.tenant_id)), - ) -``` - -> **Why swagger_fake_view?** drf-spectacular introspects ViewSets to generate OpenAPI schemas. Without this check, it executes real queries and can fail without request context. - -### Action-Specific Serializers - -```python -def get_serializer_class(self): - if self.action == "create": - return ProviderCreateSerializer - elif self.action == "partial_update": - return ProviderUpdateSerializer - elif self.action in ["connection", "destroy"]: - return TaskSerializer - return ProviderSerializer -``` - -### Dynamic Permissions per Action - -```python -class ProviderViewSet(BaseRLSViewSet): - required_permissions = [Permissions.MANAGE_PROVIDERS] - - def set_required_permissions(self): - if self.action in ["list", "retrieve"]: - self.required_permissions = [] # Read-only = no permission - else: - self.required_permissions = [Permissions.MANAGE_PROVIDERS] -``` - -### Cache Decorator - -```python -from django.utils.decorators import method_decorator -from django.views.decorators.cache import cache_control - -CACHE_DECORATOR = cache_control( - max_age=django_settings.CACHE_MAX_AGE, - stale_while_revalidate=django_settings.CACHE_STALE_WHILE_REVALIDATE, -) - -@method_decorator(CACHE_DECORATOR, name="list") -@method_decorator(CACHE_DECORATOR, name="retrieve") -class ProviderViewSet(BaseRLSViewSet): - pass -``` - -### Custom Actions - -```python -# Detail action (operates on single object) -@action(detail=True, methods=["post"], url_name="connection") -def connection(self, request, pk=None): - instance = self.get_object() - # Process instance... - -# List action (operates on collection) -@action(detail=False, methods=["get"], url_name="metadata") -def metadata(self, request): - queryset = self.filter_queryset(self.get_queryset()) - # Aggregate over queryset... -``` - ---- - -## Filter Patterns - -### Base Filter Classes - -```python -class BaseProviderFilter(FilterSet): - """For models with direct FK to Provider""" - provider_id = UUIDFilter(field_name="provider__id", lookup_expr="exact") - provider_id__in = UUIDInFilter(field_name="provider__id", lookup_expr="in") - provider_type = ChoiceFilter(field_name="provider__provider", choices=Provider.ProviderChoices.choices) - -class BaseScanProviderFilter(FilterSet): - """For models with FK to Scan (Scan has FK to Provider)""" - provider_id = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact") -``` - -### Custom Multi-Value Filters - -```python -class UUIDInFilter(BaseInFilter, UUIDFilter): - pass - -class CharInFilter(BaseInFilter, CharFilter): - pass - -class ChoiceInFilter(BaseInFilter, ChoiceFilter): - pass -``` - -### ArrayField Filtering - -```python -# Single value contains -region = CharFilter(method="filter_region") - -def filter_region(self, queryset, name, value): - return queryset.filter(resource_regions__contains=[value]) - -# Multi-value overlap -region__in = CharInFilter(field_name="resource_regions", lookup_expr="overlap") -``` - -### Date Range Validation - -```python -def filter_queryset(self, queryset): - # Require date filter for performance - if not (date_filters_provided): - raise ValidationError([{ - "detail": "At least one date filter is required", - "status": 400, - "source": {"pointer": "/data/attributes/inserted_at"}, - "code": "required", - }]) - - # Validate max range - if date_range > settings.FINDINGS_MAX_DAYS_IN_RANGE: - raise ValidationError(...) - - return super().filter_queryset(queryset) -``` - -### Dynamic FilterSet Selection - -```python -def get_filterset_class(self): - if self.action in ["latest", "metadata_latest"]: - return LatestFindingFilter - return FindingFilter -``` - -### Enum Field Override - -```python -class Meta: - model = Finding - filter_overrides = { - FindingDeltaEnumField: {"filter_class": CharFilter}, - StatusEnumField: {"filter_class": CharFilter}, - SeverityEnumField: {"filter_class": CharFilter}, - } -``` - ---- - -## Performance Patterns - -### PaginateByPkMixin - -For large querysets with expensive joins: - -```python -class PaginateByPkMixin: - def paginate_by_pk(self, request, base_queryset, manager, - select_related=None, prefetch_related=None): - # 1. Get PKs only (cheap) - pk_list = base_queryset.values_list("id", flat=True) - page = self.paginate_queryset(pk_list) - - # 2. Fetch full objects for just the page - queryset = manager.filter(id__in=page) - if select_related: - queryset = queryset.select_related(*select_related) - if prefetch_related: - queryset = queryset.prefetch_related(*prefetch_related) - - # 3. Re-sort to preserve DB ordering - queryset = sorted(queryset, key=lambda obj: page.index(obj.id)) - return self.get_paginated_response(self.get_serializer(queryset, many=True).data) -``` - -### Prefetch in Serializers - -```python -def get_tags(self, obj): - # Use prefetched tags if available - if hasattr(obj, "prefetched_tags"): - return {tag.key: tag.value for tag in obj.prefetched_tags} - # Fallback (causes N+1 if not prefetched) - return obj.get_tags(self.context.get("tenant_id")) -``` - ---- - -## Naming Conventions - -| Entity | Pattern | Example | -|--------|---------|---------| -| Serializer (read) | `Serializer` | `ProviderSerializer` | -| Serializer (create) | `CreateSerializer` | `ProviderCreateSerializer` | -| Serializer (update) | `UpdateSerializer` | `ProviderUpdateSerializer` | -| Serializer (include) | `IncludeSerializer` | `ProviderIncludeSerializer` | -| Filter | `Filter` | `ProviderFilter` | -| ViewSet | `ViewSet` | `ProviderViewSet` | - ---- - -## OpenAPI Documentation - -```python -from drf_spectacular.utils import extend_schema, extend_schema_view - -@extend_schema_view( - list=extend_schema(tags=["Provider"], summary="List all providers"), - retrieve=extend_schema(tags=["Provider"], summary="Retrieve provider"), - create=extend_schema(tags=["Provider"], summary="Create provider"), -) -@extend_schema(tags=["Provider"]) -class ProviderViewSet(BaseRLSViewSet): - pass -``` - ---- - -## API Security Patterns - -> **Full examples**: See [assets/security_patterns.py](assets/security_patterns.py) - -| Pattern | Key Points | -|---------|------------| -| **Input Validation** | Use `validate_()` for sanitization, `validate()` for cross-field | -| **Prevent Mass Assignment** | ALWAYS use explicit `fields` list, NEVER `__all__` or `exclude` | -| **Object-Level Permissions** | Implement `has_object_permission()` for ownership checks | -| **Rate Limiting** | Configure `DEFAULT_THROTTLE_RATES`, use per-view throttles for sensitive endpoints | -| **Prevent Info Disclosure** | Generic error messages, return 404 not 403 for unauthorized (prevents enumeration) | -| **SQL Injection** | ALWAYS use ORM parameterization, NEVER string interpolation in raw SQL | - -### Quick Reference - -```python -# Input validation in serializer -def validate_uid(self, value): - value = value.strip().lower() - if not re.match(r'^[a-z0-9-]+$', value): - raise serializers.ValidationError("Invalid format") - return value - -# Explicit fields (prevent mass assignment) -class Meta: - fields = ["name", "email"] # GOOD: whitelist - read_only_fields = ["id", "inserted_at"] # System fields - -# Object permission -class IsOwnerOrReadOnly(BasePermission): - def has_object_permission(self, request, view, obj): - if request.method in SAFE_METHODS: - return True - return obj.owner == request.user - -# Throttling for sensitive endpoints -class BurstRateThrottle(UserRateThrottle): - rate = "10/minute" - -# Safe error messages (prevent enumeration) -def get_object(self): - try: - return super().get_object() - except Http404: - raise NotFound("Resource not found") # Generic, no internal IDs -``` - ---- - -## Commands - -```bash -# Development -cd api && poetry run python src/backend/manage.py runserver -cd api && poetry run python src/backend/manage.py shell - -# Database -cd api && poetry run python src/backend/manage.py makemigrations -cd api && poetry run python src/backend/manage.py migrate - -# Testing -cd api && poetry run pytest -x --tb=short -cd api && poetry run make lint -``` - ---- - -## Resources - -### Local References -- **File Locations**: See [references/file-locations.md](references/file-locations.md) -- **JSON:API Conventions**: See [references/json-api-conventions.md](references/json-api-conventions.md) -- **Security Patterns**: See [assets/security_patterns.py](assets/security_patterns.py) - -### Context7 MCP (Recommended) - -**Prerequisite:** Install Context7 MCP server for up-to-date documentation lookup. - -When implementing or debugging, query these libraries via `mcp_context7_query-docs`: - -| Library | Context7 ID | Use For | -|---------|-------------|---------| -| **Django** | `/websites/djangoproject_en_5_2` | Models, ORM, migrations | -| **DRF** | `/websites/django-rest-framework` | ViewSets, serializers, permissions | -| **drf-spectacular** | `/tfranzel/drf-spectacular` | OpenAPI schema, `@extend_schema` | - -**Example queries:** -``` -mcp_context7_query-docs(libraryId="/websites/django-rest-framework", query="ViewSet get_queryset best practices") -mcp_context7_query-docs(libraryId="/tfranzel/drf-spectacular", query="extend_schema examples for custom actions") -mcp_context7_query-docs(libraryId="/websites/djangoproject_en_5_2", query="model constraints and indexes") -``` - -> **Note:** Use `mcp_context7_resolve-library-id` first if you need to find the correct library ID. - -### External Docs -- **DRF Docs**: https://www.django-rest-framework.org/ -- **DRF JSON:API**: https://django-rest-framework-json-api.readthedocs.io/ -- **drf-spectacular**: https://drf-spectacular.readthedocs.io/ -- **django-filter**: https://django-filter.readthedocs.io/ +## Activation Contract + +Use this skill for generic DRF implementation structure: serializer layering, viewset composition, filtersets, routing, pagination, schema annotations, and query efficiency. Pair it with `jsonapi` for spec compliance and `prowler-api` when tenant isolation, RBAC, providers, or Celery-specific behavior enters the picture. + +## Hard Rules + +- Always separate serializer responsibilities by operation instead of one serializer doing everything. +- Always use `filterset_class` for meaningful filtering logic. +- Always validate unknown write fields through the repo’s write-serializer pattern. +- Always protect `get_queryset()` with `swagger_fake_view` handling and N+1 prevention. +- Always prefer UUID-based identifiers and kebab-case API paths. +- Never hide business logic in serializers when it belongs in services, utilities, or domain code. + +## Decision Gates + +| Question | Action | +|---|---| +| Is this a generic DRF endpoint concern? | Use this skill as the primary implementation guide. | +| Is the task about payload compliance rather than mechanics? | Load `jsonapi` too. | +| Is the endpoint Prowler-specific because of RLS, RBAC, or providers? | Load `prowler-api` too. | +| Do reads and writes have different responsibilities? | Split read, create, update, and include serializers. | +| Could the queryset explode into N+1 queries or schema-generation failures? | Fix `get_queryset()` with eager loading and `swagger_fake_view` handling. | + +## Execution Steps + +1. Identify the endpoint surface: model, serializer set, filterset, router path, permission rule, or schema annotation. +2. Choose the correct base classes for read, write, include, and viewset behavior. +3. Design `get_queryset()` for correctness first, then add eager loading and schema-safety. +4. Add filtersets, pagination, and action-specific serializers instead of overloading one class. +5. Cross-check response shape with `jsonapi` and any tenant/provider behavior with `prowler-api`. +6. Return the concrete DRF patterns that should be applied in code. + +## Output Contract + +- State which DRF layer is being guided: serializer, viewset, filterset, router, schema, or permission. +- Mention the main pattern chosen, such as split serializers, `filterset_class`, or safe `get_queryset()`. +- Name any companion skills required. +- Flag the main correctness risk: N+1, schema-generation failure, weak validation, or over-coupled serializer logic. + +## References + +- [Repository agent rules](../../AGENTS.md) +- [API component guidance](../../api/AGENTS.md) +- [DRF file locations](references/file-locations.md) +- [JSON:API conventions](references/json-api-conventions.md) +- [Security patterns asset](assets/security_patterns.py) +- [JSON:API skill](../jsonapi/SKILL.md) +- [Prowler API skill](../prowler-api/SKILL.md) diff --git a/skills/jsonapi/SKILL.md b/skills/jsonapi/SKILL.md index a8959a2199..bb7ab322c0 100644 --- a/skills/jsonapi/SKILL.md +++ b/skills/jsonapi/SKILL.md @@ -1,8 +1,6 @@ --- name: jsonapi -description: > - Strict JSON:API v1.1 specification compliance. - Trigger: When creating or modifying API endpoints, reviewing API responses, or validating JSON:API compliance. +description: "Trigger: When creating or modifying API endpoints, reviewing API responses, or validating JSON:API behavior in Prowler. Enforces JSON:API v1.1 response, relationship, and media-type compliance." license: Apache-2.0 metadata: author: prowler-cloud @@ -14,258 +12,48 @@ metadata: - "Reviewing JSON:API compliance" --- -## Use With django-drf +## Activation Contract -This skill focuses on **spec compliance**. For **implementation patterns** (ViewSets, Serializers, Filters), use `django-drf` skill together with this one. +Use this skill when the task is about what the JSON:API contract MUST look like: document shape, media types, relationship linkage, sparse fields, includes, errors, and status-code semantics. Pair it with `django-drf` for implementation mechanics and `prowler-api` for Prowler tenant or provider rules. -| Skill | Focus | -|-------|-------| -| `jsonapi` | What the spec requires (MUST/MUST NOT rules) | -| `django-drf` | How to implement it in DRF (code patterns) | +## Hard Rules -**When creating/modifying endpoints, invoke BOTH skills.** +- Never return `data` and `errors` in the same document. +- Always return JSON:API media types and document members consistent with the spec. +- Always model resource identifiers with string `id` values and kebab-case `type` values. +- Always represent relationships with JSON:API linkage objects, not raw foreign keys. +- Always emit error objects as an array and keep `status` as a string. +- Never hide spec violations behind framework defaults; verify the final payload shape. ---- +## Decision Gates -## Before Implementing/Reviewing +| Question | Action | +|---|---| +| Are you designing endpoint structure or reviewing payload correctness? | Use this skill as the compliance authority. | +| Are you implementing DRF serializers/viewsets/filters too? | Load `django-drf` as a companion skill. | +| Does tenant visibility affect whether a resource should appear? | Load `prowler-api` too. | +| Is the change about relationship payloads or compound documents? | Validate linkage, `include`, and deduplication rules explicitly. | +| Is the response async or task-based? | Confirm status codes and response shape still satisfy JSON:API rules. | -**ALWAYS validate against the latest spec** before creating or modifying endpoints: +## Execution Steps -### Option 1: Context7 MCP (Preferred) +1. Identify the document type involved: success, error, relationship update, compound document, or sparse fieldset response. +2. Check media type, top-level members, and status code semantics first. +3. Validate resource object shape: `type`, string `id`, `attributes`, and `relationships`. +4. Verify query parameter behavior for `include`, `fields`, `filter`, `sort`, and pagination. +5. Review error payloads for array shape, string status, and pointers when field-specific. +6. Hand implementation details back to `django-drf` once compliance constraints are clear. -If Context7 MCP is available, query the JSON:API spec directly: +## Output Contract -``` -mcp_context7_resolve-library-id(query="jsonapi specification") -mcp_context7_query-docs(libraryId="", query="[specific topic: relationships, errors, etc.]") -``` +- State the JSON:API rule or family of rules that governs the task. +- Mention the endpoint or payload surface being validated. +- Name companion skills needed for implementation or tenant-aware behavior. +- Call out the concrete violation risk if the current shape is wrong. -### Option 2: WebFetch (Fallback) +## References -If Context7 is not available, fetch from the official spec: - -``` -WebFetch(url="https://jsonapi.org/format/", prompt="Extract rules for [specific topic]") -``` - -This ensures compliance with the latest JSON:API version, even after spec updates. - ---- - -## Critical Rules (NEVER Break) - -### Document Structure -- NEVER include both `data` and `errors` in the same response -- ALWAYS include at least one of: `data`, `errors`, `meta` -- ALWAYS use `type` and `id` (string) in resource objects -- NEVER include `id` when creating resources (server generates it) - -### Content-Type -- ALWAYS use `Content-Type: application/vnd.api+json` -- ALWAYS use `Accept: application/vnd.api+json` -- NEVER add parameters to media type without `ext`/`profile` - -### Resource Objects -- ALWAYS use **string** for `id` (even if UUID) -- ALWAYS use **lowercase kebab-case** for `type` -- NEVER put `id` or `type` inside `attributes` -- NEVER include foreign keys in `attributes` - use `relationships` - -### Relationships -- ALWAYS include at least one of: `links`, `data`, or `meta` -- ALWAYS use resource linkage format: `{"type": "...", "id": "..."}` -- NEVER use raw IDs in relationships - always use linkage objects - -### Error Objects -- ALWAYS return errors as array: `{"errors": [...]}` -- ALWAYS include `status` as **string** (e.g., `"400"`, not `400`) -- ALWAYS include `source.pointer` for field-specific errors - ---- - -## HTTP Status Codes (Mandatory) - -| Operation | Success | Async | Conflict | Not Found | Forbidden | Bad Request | -|-----------|---------|-------|----------|-----------|-----------|-------------| -| **GET** | `200` | - | - | `404` | `403` | `400` | -| **POST** | `201` | `202` | `409` | `404` | `403` | `400` | -| **PATCH** | `200` | `202` | `409` | `404` | `403` | `400` | -| **DELETE** | `200`/`204` | `202` | - | `404` | `403` | - | - -### When to Use Each - -| Code | Use When | -|------|----------| -| `200 OK` | Successful GET, PATCH with response body, DELETE with response | -| `201 Created` | POST created resource (MUST include `Location` header) | -| `202 Accepted` | Async operation started (return task reference) | -| `204 No Content` | Successful DELETE, PATCH with no response body | -| `400 Bad Request` | Invalid query params, malformed request, unknown fields | -| `403 Forbidden` | Authentication ok but no permission, client-generated ID rejected | -| `404 Not Found` | Resource doesn't exist OR RLS hides it (never reveal which) | -| `409 Conflict` | Duplicate ID, type mismatch, relationship conflict | -| `415 Unsupported` | Wrong Content-Type header | - ---- - -## Document Structure - -### Success Response (Single) - -```json -{ - "data": { - "type": "providers", - "id": "550e8400-e29b-41d4-a716-446655440000", - "attributes": { - "alias": "Production", - "connected": true - }, - "relationships": { - "tenant": { - "data": {"type": "tenants", "id": "..."} - } - }, - "links": { - "self": "/api/v1/providers/550e8400-..." - } - }, - "links": { - "self": "/api/v1/providers/550e8400-..." - } -} -``` - -### Success Response (List) - -```json -{ - "data": [ - {"type": "providers", "id": "...", "attributes": {...}}, - {"type": "providers", "id": "...", "attributes": {...}} - ], - "links": { - "self": "/api/v1/providers?page[number]=1", - "first": "/api/v1/providers?page[number]=1", - "last": "/api/v1/providers?page[number]=5", - "prev": null, - "next": "/api/v1/providers?page[number]=2" - }, - "meta": { - "pagination": {"count": 100, "pages": 5} - } -} -``` - -### Error Response - -```json -{ - "errors": [ - { - "status": "400", - "code": "invalid", - "title": "Invalid attribute", - "detail": "UID must be 12 digits for AWS accounts", - "source": {"pointer": "/data/attributes/uid"} - } - ] -} -``` - ---- - -## Query Parameters - -| Family | Format | Example | -|--------|--------|---------| -| `page` | `page[number]`, `page[size]` | `?page[number]=2&page[size]=25` | -| `filter` | `filter[field]`, `filter[field__op]` | `?filter[status]=FAIL` | -| `sort` | Comma-separated, `-` for desc | `?sort=-inserted_at,name` | -| `fields` | `fields[type]` | `?fields[providers]=id,alias` | -| `include` | Comma-separated paths | `?include=provider,scan.task` | - -### Rules - -- MUST return `400` for unsupported query parameters -- MUST return `400` for unsupported `include` paths -- MUST return `400` for unsupported `sort` fields -- MUST NOT include extra fields when `fields[type]` is specified - ---- - -## Common Violations (AVOID) - -| Violation | Wrong | Correct | -|-----------|-------|---------| -| ID as integer | `"id": 123` | `"id": "123"` | -| Type as camelCase | `"type": "providerGroup"` | `"type": "provider-groups"` | -| FK in attributes | `"tenant_id": "..."` | `"relationships": {"tenant": {...}}` | -| Errors not array | `{"error": "..."}` | `{"errors": [{"detail": "..."}]}` | -| Status as number | `"status": 400` | `"status": "400"` | -| Data + errors | `{"data": ..., "errors": ...}` | Only one or the other | -| Missing pointer | `{"detail": "Invalid"}` | `{"detail": "...", "source": {"pointer": "..."}}` | - ---- - -## Relationship Updates - -### To-One Relationship - -```http -PATCH /api/v1/providers/123/relationships/tenant -Content-Type: application/vnd.api+json - -{"data": {"type": "tenants", "id": "456"}} -``` - -To clear: `{"data": null}` - -### To-Many Relationship - -| Operation | Method | Body | -|-----------|--------|------| -| Replace all | PATCH | `{"data": [{...}, {...}]}` | -| Add members | POST | `{"data": [{...}]}` | -| Remove members | DELETE | `{"data": [{...}]}` | - ---- - -## Compound Documents (`include`) - -When using `?include=provider`: - -```json -{ - "data": { - "type": "scans", - "id": "...", - "relationships": { - "provider": { - "data": {"type": "providers", "id": "prov-123"} - } - } - }, - "included": [ - { - "type": "providers", - "id": "prov-123", - "attributes": {"alias": "Production"} - } - ] -} -``` - -### Rules - -- Every included resource MUST be reachable via relationship chain from primary data -- MUST NOT include orphan resources -- MUST NOT duplicate resources (same type+id) - ---- - -## Spec Reference - -- **Full Specification**: https://jsonapi.org/format/ -- **Implementation**: Use `django-drf` skill for DRF-specific patterns -- **Testing**: Use `prowler-test-api` skill for test patterns +- [Repository agent rules](../../AGENTS.md) +- [API component guidance](../../api/AGENTS.md) +- [DRF implementation skill](../django-drf/SKILL.md) +- [Prowler API skill](../prowler-api/SKILL.md) diff --git a/skills/prowler-api/SKILL.md b/skills/prowler-api/SKILL.md index 1e654a3e03..0be71805a0 100644 --- a/skills/prowler-api/SKILL.md +++ b/skills/prowler-api/SKILL.md @@ -1,8 +1,6 @@ --- name: prowler-api -description: > - Prowler API patterns: RLS, RBAC, providers, Celery tasks. - Trigger: When working in api/ on models/serializers/viewsets/filters/tasks involving tenant isolation (RLS), RBAC, or provider lifecycle. +description: "Trigger: When working in `api/` on Prowler-specific models, serializers, viewsets, filters, Celery tasks, provider lifecycle, RBAC, or tenant isolation. Applies the repository’s RLS-first API contract." license: Apache-2.0 metadata: author: prowler-cloud @@ -12,494 +10,52 @@ metadata: allowed-tools: Read, Edit, Write, Glob, Grep, Bash, WebFetch, WebSearch, Task --- -## When to Use - -Use this skill for **Prowler-specific** patterns: -- Row-Level Security (RLS) / tenant isolation -- RBAC permissions and role checks -- Provider lifecycle and validation -- Celery tasks with tenant context -- Multi-database architecture (4-database setup) - -For **generic DRF patterns** (ViewSets, Serializers, Filters, JSON:API), use `django-drf` skill. - ---- - -## Critical Rules - -- ALWAYS use `rls_transaction(tenant_id)` when querying outside ViewSet context -- ALWAYS use `get_role()` before checking permissions (returns FIRST role only) -- ALWAYS use `@set_tenant` then `@handle_provider_deletion` decorator order -- ALWAYS use explicit through models for M2M relationships (required for RLS) -- NEVER access `Provider.objects` without RLS context in Celery tasks -- NEVER bypass RLS by using raw SQL or `connection.cursor()` -- NEVER use Django's default M2M - RLS requires through models with `tenant_id` - -> **Note**: `rls_transaction()` accepts both UUID objects and strings - it converts internally via `str(value)`. - ---- - -## Architecture Overview - -### 4-Database Architecture - -| Database | Alias | Purpose | RLS | -|----------|-------|---------|-----| -| `default` | `prowler_user` | Standard API queries | **Yes** | -| `admin` | `admin` | Migrations, auth bypass | No | -| `replica` | `prowler_user` | Read-only queries | **Yes** | -| `admin_replica` | `admin` | Admin read replica | No | - -```python -# When to use admin (bypasses RLS) -from api.db_router import MainRouter -User.objects.using(MainRouter.admin_db).get(id=user_id) # Auth lookups - -# Standard queries use default (RLS enforced) -Provider.objects.filter(connected=True) # Requires rls_transaction context -``` - -### RLS Transaction Flow - -``` -Request → Authentication → BaseRLSViewSet.initial() - │ - ├─ Extract tenant_id from JWT - ├─ SET api.tenant_id = 'uuid' (PostgreSQL) - └─ All queries now tenant-scoped -``` - ---- - -## Implementation Checklist - -When implementing Prowler-specific API features: - -| # | Pattern | Reference | Key Points | -|---|---------|-----------|------------| -| 1 | **RLS Models** | `api/rls.py` | Inherit `RowLevelSecurityProtectedModel`, add constraint | -| 2 | **RLS Transactions** | `api/db_utils.py` | Use `rls_transaction(tenant_id)` context manager | -| 3 | **RBAC Permissions** | `api/rbac/permissions.py` | `get_role()`, `get_providers()`, `Permissions` enum | -| 4 | **Provider Validation** | `api/models.py` | `validate__uid()` methods on `Provider` model | -| 5 | **Celery Tasks** | `tasks/tasks.py`, `api/decorators.py`, `config/celery.py` | Task definitions, decorators (`@set_tenant`, `@handle_provider_deletion`), `RLSTask` base | -| 6 | **RLS Serializers** | `api/v1/serializers.py` | Inherit `RLSSerializer` to auto-inject `tenant_id` | -| 7 | **Through Models** | `api/models.py` | ALL M2M must use explicit through with `tenant_id` | - -> **Full file paths**: See [references/file-locations.md](references/file-locations.md) - ---- - -## Decision Trees - -### Which Base Model? -``` -Tenant-scoped data → RowLevelSecurityProtectedModel -Global/shared data → models.Model + BaseSecurityConstraint (rare) -Partitioned time-series → PostgresPartitionedModel + RowLevelSecurityProtectedModel -Soft-deletable → Add is_deleted + ActiveProviderManager -``` - -### Which Manager? -``` -Normal queries → Model.objects (excludes deleted) -Include deleted records → Model.all_objects -Celery task context → Must use rls_transaction() first -``` - -### Which Database? -``` -Standard API queries → default (automatic via ViewSet) -Read-only operations → replica (automatic for GET in BaseRLSViewSet) -Auth/admin operations → MainRouter.admin_db -Cross-tenant lookups → MainRouter.admin_db (use sparingly!) -``` - -### Celery Task Decorator Order? -``` -@shared_task(base=RLSTask, name="...", queue="...") -@set_tenant # First: sets tenant context -@handle_provider_deletion # Second: handles deleted providers -def my_task(tenant_id, provider_id): - pass -``` - ---- - -## RLS Model Pattern - -```python -from api.rls import RowLevelSecurityProtectedModel, RowLevelSecurityConstraint - -class MyModel(RowLevelSecurityProtectedModel): - # tenant FK inherited from parent - id = models.UUIDField(primary_key=True, default=uuid4, editable=False) - name = models.CharField(max_length=255) - inserted_at = models.DateTimeField(auto_now_add=True, editable=False) - updated_at = models.DateTimeField(auto_now=True, editable=False) - - class Meta(RowLevelSecurityProtectedModel.Meta): - db_table = "my_models" - constraints = [ - RowLevelSecurityConstraint( - field="tenant_id", - name="rls_on_%(class)s", - statements=["SELECT", "INSERT", "UPDATE", "DELETE"], - ), - ] - - class JSONAPIMeta: - resource_name = "my-models" -``` - -### M2M Relationships (MUST use through models) - -```python -class Resource(RowLevelSecurityProtectedModel): - tags = models.ManyToManyField( - ResourceTag, - through="ResourceTagMapping", # REQUIRED for RLS - ) - -class ResourceTagMapping(RowLevelSecurityProtectedModel): - # Through model MUST have tenant_id for RLS - resource = models.ForeignKey(Resource, on_delete=models.CASCADE) - tag = models.ForeignKey(ResourceTag, on_delete=models.CASCADE) - - class Meta: - constraints = [ - RowLevelSecurityConstraint( - field="tenant_id", - name="rls_on_%(class)s", - statements=["SELECT", "INSERT", "UPDATE", "DELETE"], - ), - ] -``` - ---- - -## Async Task Response Pattern (202 Accepted) - -For long-running operations, return 202 with task reference: - -```python -@action(detail=True, methods=["post"], url_name="connection") -def connection(self, request, pk=None): - with transaction.atomic(): - task = check_provider_connection_task.delay( - provider_id=pk, tenant_id=self.request.tenant_id - ) - prowler_task = Task.objects.get(id=task.id) - serializer = TaskSerializer(prowler_task) - return Response( - data=serializer.data, - status=status.HTTP_202_ACCEPTED, - headers={"Content-Location": reverse("task-detail", kwargs={"pk": prowler_task.id})} - ) -``` - ---- - -## Providers (11 Supported) - -| Provider | UID Format | Example | -|----------|-----------|---------| -| AWS | 12 digits | `123456789012` | -| Azure | UUID v4 | `a1b2c3d4-e5f6-...` | -| GCP | 6-30 chars, lowercase, letter start | `my-gcp-project` | -| M365 | Valid domain | `contoso.onmicrosoft.com` | -| Kubernetes | 2-251 chars | `arn:aws:eks:...` | -| GitHub | 1-39 chars | `my-org` | -| IaC | Git URL | `https://github.com/user/repo.git` | -| Oracle Cloud | OCID format | `ocid1.tenancy.oc1..` | -| MongoDB Atlas | 24-char hex | `507f1f77bcf86cd799439011` | -| Alibaba Cloud | 16 digits | `1234567890123456` | - -**Adding new provider**: Add to `ProviderChoices` enum + create `validate__uid()` staticmethod. - ---- - -## RBAC Permissions - -| Permission | Controls | -|------------|----------| -| `MANAGE_USERS` | User CRUD, role assignments | -| `MANAGE_ACCOUNT` | Tenant settings | -| `MANAGE_BILLING` | Billing/subscription | -| `MANAGE_PROVIDERS` | Provider CRUD | -| `MANAGE_INTEGRATIONS` | Integration config | -| `MANAGE_SCANS` | Scan execution | -| `UNLIMITED_VISIBILITY` | See all providers (bypasses provider_groups) | - -### RBAC Visibility Pattern - -```python -def get_queryset(self): - user_role = get_role(self.request.user) - if user_role.unlimited_visibility: - return Model.objects.filter(tenant_id=self.request.tenant_id) - else: - # Filter by provider_groups assigned to role - return Model.objects.filter(provider__in=get_providers(user_role)) -``` - ---- - -## Celery Queues - -| Queue | Purpose | -|-------|---------| -| `scans` | Prowler scan execution | -| `overview` | Dashboard aggregations (severity, attack surface) | -| `compliance` | Compliance report generation | -| `integrations` | External integrations (Jira, S3, Security Hub) | -| `deletion` | Provider/tenant deletion (async) | -| `backfill` | Historical data backfill operations | -| `scan-reports` | Output generation (CSV, JSON, HTML, PDF) | - ---- - -## Task Composition (Canvas) - -Use Celery's Canvas primitives for complex workflows: - -| Primitive | Use For | -|-----------|---------| -| `chain()` | Sequential execution: A → B → C | -| `group()` | Parallel execution: A, B, C simultaneously | -| Combined | Chain with nested groups for complex workflows | - -> **Note:** Use `.si()` (signature immutable) to prevent result passing. Use `.s()` if you need to pass results. - -> **Examples:** See [assets/celery_patterns.py](assets/celery_patterns.py) for chain, group, and combined patterns. - ---- - -## Beat Scheduling (Periodic Tasks) - -| Operation | Key Points | -|-----------|------------| -| **Create schedule** | `IntervalSchedule.objects.get_or_create(every=24, period=HOURS)` | -| **Create periodic task** | Use task name (not function), `kwargs=json.dumps(...)` | -| **Delete scheduled task** | `PeriodicTask.objects.filter(name=...).delete()` | -| **Avoid race conditions** | Use `countdown=5` to wait for DB commit | - -> **Examples:** See [assets/celery_patterns.py](assets/celery_patterns.py) for schedule_provider_scan pattern. - ---- - -## Advanced Task Patterns - -### `@set_tenant` Behavior - -| Mode | `tenant_id` in kwargs | `tenant_id` passed to function | -|------|----------------------|-------------------------------| -| `@set_tenant` (default) | Popped (removed) | NO - function doesn't receive it | -| `@set_tenant(keep_tenant=True)` | Read but kept | YES - function receives it | - -### Key Patterns - -| Pattern | Description | -|---------|-------------| -| `bind=True` | Access `self.request.id`, `self.request.retries` | -| `get_task_logger(__name__)` | Proper logging in Celery tasks | -| `SoftTimeLimitExceeded` | Catch to save progress before hard kill | -| `countdown=30` | Defer execution by N seconds | -| `eta=datetime(...)` | Execute at specific time | - -> **Examples:** See [assets/celery_patterns.py](assets/celery_patterns.py) for all advanced patterns. - ---- - -## Celery Configuration - -| Setting | Value | Purpose | -|---------|-------|---------| -| `BROKER_VISIBILITY_TIMEOUT` | `86400` (24h) | Prevent re-queue for long tasks | -| `CELERY_RESULT_BACKEND` | `django-db` | Store results in PostgreSQL | -| `CELERY_TASK_TRACK_STARTED` | `True` | Track when tasks start | -| `soft_time_limit` | Task-specific | Raises `SoftTimeLimitExceeded` | -| `time_limit` | Task-specific | Hard kill (SIGKILL) | - -> **Full config:** See [assets/celery_patterns.py](assets/celery_patterns.py) and actual files at `config/celery.py`, `config/settings/celery.py`. - ---- - -## UUIDv7 for Partitioned Tables - -`Finding` and `ResourceFindingMapping` use UUIDv7 for time-based partitioning: - -```python -from uuid6 import uuid7 -from api.uuid_utils import uuid7_start, uuid7_end, datetime_to_uuid7 - -# Partition-aware filtering -start = uuid7_start(datetime_to_uuid7(date_from)) -end = uuid7_end(datetime_to_uuid7(date_to), settings.FINDINGS_TABLE_PARTITION_MONTHS) -queryset.filter(id__gte=start, id__lt=end) -``` - -**Why UUIDv7?** Time-ordered UUIDs enable PostgreSQL to prune partitions during range queries. - ---- - -## Batch Operations with RLS - -```python -from api.db_utils import batch_delete, create_objects_in_batches, update_objects_in_batches - -# Delete in batches (RLS-aware) -batch_delete(tenant_id, queryset, batch_size=1000) - -# Bulk create with RLS -create_objects_in_batches(tenant_id, Finding, objects, batch_size=500) - -# Bulk update with RLS -update_objects_in_batches(tenant_id, Finding, objects, fields=["status"], batch_size=500) -``` - ---- - -## Security Patterns - -> **Full examples**: See [assets/security_patterns.py](assets/security_patterns.py) - -### Tenant Isolation Summary - -| Pattern | Rule | -|---------|------| -| **RLS in ViewSets** | Automatic via `BaseRLSViewSet` - tenant_id from JWT | -| **RLS in Celery** | MUST use `@set_tenant` + `rls_transaction(tenant_id)` | -| **Cross-tenant validation** | Defense-in-depth: verify `obj.tenant_id == request.tenant_id` | -| **Never trust user input** | Use `request.tenant_id` from JWT, never `request.data.get("tenant_id")` | -| **Admin DB bypass** | Only for cross-tenant admin ops - exposes ALL tenants' data | - -### Celery Task Security Summary - -| Pattern | Rule | -|---------|------| -| **Named tasks only** | NEVER use dynamic task names from user input | -| **Validate arguments** | Check UUID format before database queries | -| **Safe queuing** | Use `transaction.on_commit()` to enqueue AFTER commit | -| **Modern retries** | Use `autoretry_for`, `retry_backoff`, `retry_jitter` | -| **Time limits** | Set `soft_time_limit` and `time_limit` to prevent hung tasks | -| **Idempotency** | Use `update_or_create` or idempotency keys | - -### Quick Reference - -```python -# Safe task queuing - task only enqueued after transaction commits -with transaction.atomic(): - provider = Provider.objects.create(**data) - transaction.on_commit( - lambda: verify_provider_connection.delay( - tenant_id=str(request.tenant_id), - provider_id=str(provider.id) - ) - ) - -# Modern retry pattern -@shared_task( - base=RLSTask, - bind=True, - autoretry_for=(ConnectionError, TimeoutError, OperationalError), - retry_backoff=True, - retry_backoff_max=600, - retry_jitter=True, - max_retries=5, - soft_time_limit=300, - time_limit=360, -) -@set_tenant -def sync_provider_data(self, tenant_id, provider_id): - with rls_transaction(tenant_id): - # ... task logic - pass - -# Idempotent task - safe to retry -@shared_task(base=RLSTask, acks_late=True) -@set_tenant -def process_finding(tenant_id, finding_uid, data): - with rls_transaction(tenant_id): - Finding.objects.update_or_create(uid=finding_uid, defaults=data) -``` - ---- - -## Production Deployment Checklist - -> **Full settings**: See [references/production-settings.md](references/production-settings.md) - -Run before every production deployment: - -```bash -cd api && poetry run python src/backend/manage.py check --deploy -``` - -### Critical Settings - -| Setting | Production Value | Risk if Wrong | -|---------|-----------------|---------------| -| `DEBUG` | `False` | Exposes stack traces, settings, SQL queries | -| `SECRET_KEY` | Env var, rotated | Session hijacking, CSRF bypass | -| `ALLOWED_HOSTS` | Explicit list | Host header attacks | -| `SECURE_SSL_REDIRECT` | `True` | Credentials sent over HTTP | -| `SESSION_COOKIE_SECURE` | `True` | Session cookies over HTTP | -| `CSRF_COOKIE_SECURE` | `True` | CSRF tokens over HTTP | -| `SECURE_HSTS_SECONDS` | `31536000` (1 year) | Downgrade attacks | -| `CONN_MAX_AGE` | `60` or higher | Connection pool exhaustion | - ---- - -## Commands - -```bash -# Development -cd api && poetry run python src/backend/manage.py runserver -cd api && poetry run python src/backend/manage.py shell - -# Celery -cd api && poetry run celery -A config.celery worker -l info -Q scans,overview -cd api && poetry run celery -A config.celery beat -l info - -# Testing -cd api && poetry run pytest -x --tb=short - -# Production checks -cd api && poetry run python src/backend/manage.py check --deploy -``` - ---- - -## Resources - -### Local References -- **File Locations**: See [references/file-locations.md](references/file-locations.md) -- **Modeling Decisions**: See [references/modeling-decisions.md](references/modeling-decisions.md) -- **Configuration**: See [references/configuration.md](references/configuration.md) -- **Production Settings**: See [references/production-settings.md](references/production-settings.md) -- **Security Patterns**: See [assets/security_patterns.py](assets/security_patterns.py) - -### Related Skills -- **Generic DRF Patterns**: Use `django-drf` skill -- **API Testing**: Use `prowler-test-api` skill - -### Context7 MCP (Recommended) - -**Prerequisite:** Install Context7 MCP server for up-to-date documentation lookup. - -When implementing or debugging Prowler-specific patterns, query these libraries via `mcp_context7_query-docs`: - -| Library | Context7 ID | Use For | -|---------|-------------|---------| -| **Celery** | `/websites/celeryq_dev_en_stable` | Task patterns, queues, error handling | -| **django-celery-beat** | `/celery/django-celery-beat` | Periodic task scheduling | -| **Django** | `/websites/djangoproject_en_5_2` | Models, ORM, constraints, indexes | - -**Example queries:** -``` -mcp_context7_query-docs(libraryId="/websites/celeryq_dev_en_stable", query="shared_task decorator retry patterns") -mcp_context7_query-docs(libraryId="/celery/django-celery-beat", query="periodic task database scheduler") -mcp_context7_query-docs(libraryId="/websites/djangoproject_en_5_2", query="model constraints CheckConstraint UniqueConstraint") -``` - -> **Note:** Use `mcp_context7_resolve-library-id` first if you need to find the correct library ID. +## Activation Contract + +Use this skill for Prowler API behavior that depends on tenant isolation, RBAC visibility, provider orchestration, or Celery execution semantics. Pair it with `django-drf` for generic DRF patterns and `jsonapi` for response-shape compliance. + +## Hard Rules + +- Always preserve RLS boundaries; queries outside request-scoped viewsets must run inside `rls_transaction(tenant_id)`. +- Always check permissions through the repo’s RBAC helpers before assuming provider visibility. +- Always model tenant-scoped M2M relations with explicit through models carrying `tenant_id`. +- Always keep Celery tenant setup and provider-deletion handling in the established decorator/base-task flow. +- Never bypass RLS with raw SQL, unmanaged cursors, or admin connections unless the design explicitly requires cross-tenant access. +- Never invent generic DRF patterns here when `django-drf` already owns them. + +## Decision Gates + +| Question | Action | +|---|---| +| Is the behavior tenant-scoped data access? | Use RLS-safe models, serializers, and `rls_transaction()` where request context is absent. | +| Is the endpoint mostly generic DRF plumbing? | Load `django-drf` alongside this skill. | +| Is the concern response/media-type compliance? | Load `jsonapi` alongside this skill. | +| Is this async provider or scan orchestration? | Use Celery patterns with tenant-aware task setup. | +| Does the query need admin or cross-tenant access? | Escalate the reason explicitly and use the admin path sparingly. | + +## Execution Steps + +1. Classify the change: RLS model, RBAC/viewset flow, provider lifecycle, serializer boundary, or Celery workflow. +2. Identify where tenant context comes from and where it could be lost. +3. Choose the correct base abstractions for models, serializers, viewsets, and tasks. +4. Validate relationship modeling, provider visibility, and async handoff against existing Prowler patterns. +5. Cross-check the implementation with `django-drf` and `jsonapi` when endpoint behavior is involved. +6. Return only the repo-specific constraints that materially affect the change. + +## Output Contract + +- State the Prowler-specific API constraint that governs the task: RLS, RBAC, provider lifecycle, or Celery tenant handling. +- Name any companion skills required, especially `django-drf` and `jsonapi`. +- Call out the exact files or modules to inspect next. +- Mention any high-risk boundary where tenant isolation or provider visibility could break. + +## References + +- [Repository agent rules](../../AGENTS.md) +- [API component guidance](../../api/AGENTS.md) +- [API file locations](references/file-locations.md) +- [API modeling decisions](references/modeling-decisions.md) +- [API configuration](references/configuration.md) +- [Production settings notes](references/production-settings.md) +- [Celery patterns asset](assets/celery_patterns.py) +- [Security patterns asset](assets/security_patterns.py) diff --git a/skills/prowler-ui/SKILL.md b/skills/prowler-ui/SKILL.md index 3e6407889e..750f399a7f 100644 --- a/skills/prowler-ui/SKILL.md +++ b/skills/prowler-ui/SKILL.md @@ -1,8 +1,6 @@ --- name: prowler-ui -description: > - Prowler UI-specific patterns. For generic patterns, see: typescript, react-19, nextjs-15, tailwind-4. - Trigger: When working inside ui/ on Prowler-specific conventions (shadcn vs HeroUI legacy, folder placement, actions/adapters, shared types/hooks/lib). +description: "Trigger: When working inside `ui/` on Prowler-specific app structure, folder placement, shared UI conventions, shadcn adoption, or display-layer patterns beyond generic React/Next.js guidance. Applies the repo’s UI architecture rules." license: Apache-2.0 metadata: author: prowler-cloud @@ -14,313 +12,51 @@ metadata: allowed-tools: Read, Edit, Write, Glob, Grep, Bash, WebFetch, WebSearch, Task --- -## Related Generic Skills - -- `typescript` - Const types, flat interfaces -- `react-19` - No useMemo/useCallback, compiler -- `nextjs-15` - App Router, Server Actions -- `tailwind-4` - cn() utility, styling rules -- `zod-4` - Schema validation -- `zustand-5` - State management -- `ai-sdk-5` - Chat/AI features -- `playwright` - E2E testing (see also `prowler-test-ui`) - -## Tech Stack (Versions) - -``` -Next.js 15.5.9 | React 19.2.2 | Tailwind 4.1.13 | shadcn/ui -Zod 4.1.11 | React Hook Form 7.62.0 | Zustand 5.0.8 -NextAuth 5.0.0-beta.30 | Recharts 2.15.4 -HeroUI 2.8.4 (LEGACY - do not add new components) -``` - -## CRITICAL: Component Library Rule - -- **ALWAYS**: Use `shadcn/ui` + Tailwind (`components/shadcn/`) -- **NEVER**: Add new HeroUI components (`components/ui/` is legacy only) - -## DECISION TREES - -### Component Placement - -``` -New feature UI? → shadcn/ui + Tailwind -Existing HeroUI feature? → Keep HeroUI (don't mix) -Used 1 feature? → features/{feature}/components/ -Used 2+ features? → components/shared/ -Needs state/hooks? → "use client" -Server component? → No directive needed -``` - -### Code Location - -``` -Server action → actions/{feature}/{feature}.ts -Data transform → actions/{feature}/{feature}.adapter.ts -Types (shared 2+) → types/{domain}.ts -Types (local 1) → {feature}/types.ts -Utils (shared 2+) → lib/ -Utils (local 1) → {feature}/utils/ -Hooks (shared 2+) → hooks/ -Hooks (local 1) → {feature}/hooks.ts -shadcn components → components/shadcn/ -HeroUI components → components/ui/ (LEGACY) -``` - -### Styling Decision - -``` -Tailwind class exists? → className -Dynamic value? → style prop -Conditional styles? → cn() -Static only? → className (no cn()) -Recharts/library? → CHART_COLORS constant + var() -``` - -### Scope Rule (ABSOLUTE) - -- Used 2+ places → `lib/` or `types/` or `hooks/` (components go in `components/{domain}/`) -- Used 1 place → keep local in feature directory -- **This determines ALL folder structure decisions** - -## Project Structure - -``` -ui/ -├── app/ -│ ├── (auth)/ # Auth pages (login, signup) -│ └── (prowler)/ # Main app -│ ├── compliance/ -│ ├── findings/ -│ ├── providers/ -│ ├── scans/ -│ ├── services/ -│ └── integrations/ -├── components/ -│ ├── shadcn/ # shadcn/ui (USE THIS) -│ ├── ui/ # HeroUI (LEGACY) -│ ├── {domain}/ # Domain-specific (compliance, findings, providers, etc.) -│ ├── filters/ # Filter components -│ ├── graphs/ # Chart components -│ └── icons/ # Icon components -├── actions/ # Server actions -├── types/ # Shared types -├── hooks/ # Shared hooks -├── lib/ # Utilities -├── store/ # Zustand state -├── tests/ # Playwright E2E -└── styles/ # Global CSS -``` - -## Recharts (Special Case) - -For Recharts props that don't accept className: - -```typescript -const CHART_COLORS = { - primary: "var(--color-primary)", - secondary: "var(--color-secondary)", - text: "var(--color-text)", - gridLine: "var(--color-border)", -}; - -// Only use var() for library props, NEVER in className - - -``` - -## Form + Validation Pattern - -```typescript -"use client"; -import { useForm } from "react-hook-form"; -import { zodResolver } from "@hookform/resolvers/zod"; -import { z } from "zod"; - -const schema = z.object({ - email: z.email(), // Zod 4 syntax - name: z.string().min(1), -}); - -type FormData = z.infer; - -export function MyForm() { - const { register, handleSubmit, formState: { errors } } = useForm({ - resolver: zodResolver(schema), - }); - - const onSubmit = async (data: FormData) => { - await serverAction(data); - }; - - return ( -
- - {errors.email && {errors.email.message}} - -
- ); -} -``` - -## Commands - -```bash -# Development -cd ui && pnpm install -cd ui && pnpm run dev - -# Code Quality -cd ui && pnpm run typecheck -cd ui && pnpm run lint:fix -cd ui && pnpm run format:write -cd ui && pnpm run healthcheck # typecheck + lint - -# Testing -cd ui && pnpm run test:e2e -cd ui && pnpm run test:e2e:ui -cd ui && pnpm run test:e2e:debug - -# Build -cd ui && pnpm run build -cd ui && pnpm start -``` - -## Batch vs Instant Component API (REQUIRED) - -When a component supports both **batch** (deferred, submit-based) and **instant** (immediate callback) behavior, model the coupling with a discriminated union — never as independent optionals. Coupled props must be all-or-nothing. - -```typescript -// ❌ NEVER: Independent optionals — allows invalid half-states -interface FilterProps { - onBatchApply?: (values: string[]) => void; - onInstantChange?: (value: string) => void; - isBatchMode?: boolean; -} - -// ✅ ALWAYS: Discriminated union — one valid shape per mode -type BatchProps = { - mode: "batch"; - onApply: (values: string[]) => void; - onCancel: () => void; -}; - -type InstantProps = { - mode: "instant"; - onChange: (value: string) => void; - // onApply/onCancel are forbidden here via structural exclusion - onApply?: never; - onCancel?: never; -}; - -type FilterProps = BatchProps | InstantProps; -``` - -This makes invalid prop combinations a compile error, not a runtime surprise. - -## Reuse Shared Display Utilities First (REQUIRED) - -Before adding **local** display maps (labels, provider names, status strings, category formatters), search `ui/types/*` and `ui/lib/*` for existing helpers. - -```typescript -// ✅ CHECK THESE FIRST before creating a new map: -// ui/lib/utils.ts → general formatters -// ui/types/providers.ts → provider display names, icons -// ui/types/findings.ts → severity/status display maps -// ui/types/compliance.ts → category/group formatters - -// ❌ NEVER add a local map that already exists: -const SEVERITY_LABELS: Record = { - critical: "Critical", - high: "High", - // ...duplicating an existing shared map -}; - -// ✅ Import and reuse instead: -import { severityLabel } from "@/types/findings"; -``` - -If a helper doesn't exist and will be used in 2+ places, add it to `ui/lib/` or `ui/types/` and reuse it. Keep local only if used in exactly one place. - -## Derived State Rule (REQUIRED) - -Avoid `useState` + `useEffect` patterns that mirror props or searchParams — they create sync bugs and unnecessary re-renders. Derive values directly from the source of truth. - -```typescript -// ❌ NEVER: Mirror props into state via effect -const [localFilter, setLocalFilter] = useState(filter); -useEffect(() => { setLocalFilter(filter); }, [filter]); - -// ✅ ALWAYS: Derive directly -const localFilter = filter; // or compute inline -``` - -If local state is genuinely needed (e.g., optimistic UI, pending edits before submit), add a short comment: - -```typescript -// Local state needed: user edits are buffered until "Apply" is clicked -const [pending, setPending] = useState(initialValues); -``` - -## Strict Key Typing for Label Maps (REQUIRED) - -Avoid `Record` when the key set is known. Use an explicit union type or a const-key object so typos are caught at compile time. - -```typescript -// ❌ Loose — typos compile silently -const STATUS_LABELS: Record = { - actve: "Active", // typo, no error -}; - -// ✅ Tight — union key -type Status = "active" | "inactive" | "pending"; -const STATUS_LABELS: Record = { - active: "Active", - inactive: "Inactive", - pending: "Pending", - // actve: "Active" ← compile error -}; - -// ✅ Also fine — const satisfies -const STATUS_LABELS = { - active: "Active", - inactive: "Inactive", - pending: "Pending", -} as const satisfies Record; -``` - -## QA Checklist Before Commit - -- [ ] `pnpm run typecheck` passes -- [ ] `pnpm run lint:fix` passes -- [ ] `pnpm run format:write` passes -- [ ] Relevant E2E tests pass -- [ ] All UI states handled (loading, error, empty) -- [ ] No secrets in code (use `.env.local`) -- [ ] Error messages sanitized (no stack traces to users) -- [ ] Server-side validation present (don't trust client) -- [ ] Accessibility: keyboard navigation, ARIA labels -- [ ] Mobile responsive (if applicable) - -## Pre-Re-Review Checklist (Review Thread Hygiene) - -Before requesting re-review from a reviewer: - -- [ ] Every unresolved inline thread has been either fixed or explicitly answered with a rationale -- [ ] If you agreed with a comment: the change is committed and the commit hash is mentioned in the reply -- [ ] If you disagreed: the reply explains why with clear reasoning — do not leave threads silently open -- [ ] Re-request review only after all threads are in a clean state - -## Migrations Reference - -| From | To | Key Changes | -|------|-----|-------------| -| React 18 | 19.1 | Async components, React Compiler (no useMemo/useCallback) | -| Next.js 14 | 15.5 | Improved App Router, better streaming | -| NextUI | HeroUI 2.8.4 | Package rename only, same API | -| Zod 3 | 4 | `z.email()` not `z.string().email()`, `error` not `message` | -| AI SDK 4 | 5 | `@ai-sdk/react`, `sendMessage` not `handleSubmit`, `parts` not `content` | - -## Resources - -- **Documentation**: See [references/](references/) for links to local developer guide +## Activation Contract + +Use this skill when the work depends on Prowler UI structure rather than generic framework syntax: component placement, action/adapter boundaries, shared-vs-local scope decisions, legacy HeroUI avoidance, or shared display utilities. Pair it with `react-19`, `nextjs-15`, `tailwind-4`, `typescript`, `zod-4`, or `zustand-5` when those implementation details matter. + +## Hard Rules + +- Always prefer `components/shadcn/` for new UI; do not introduce new HeroUI usage. +- Always apply the scope rule first: code reused in 2+ places becomes shared, otherwise keep it local. +- Always keep server actions, adapters, types, hooks, and utilities in their intended folders. +- Always derive state directly when possible; do not mirror props or search params into effect-driven local state without a real buffering reason. +- Always reuse shared label, formatter, and display helpers before adding local maps. +- Never encode invalid prop combinations with unrelated optional fields when a discriminated union can model the API correctly. + +## Decision Gates + +| Question | Action | +|---|---| +| Is this a new component? | Build with shadcn + Tailwind conventions. | +| Is logic reused across multiple features? | Promote it to `components/`, `types/`, `hooks/`, or `lib/` as appropriate. | +| Is it only used in one feature? | Keep it inside that feature boundary. | +| Is styling conditional or compositional? | Use `cn()`; use plain `className` for static classes. | +| Does a third-party prop reject Tailwind classes? | Use a constant or `style` value, not `var()` inside `className`. | + +## Execution Steps + +1. Identify whether the change is component structure, state modeling, display formatting, or action/data flow. +2. Apply the scope rule to decide local versus shared placement. +3. Choose shadcn-first component patterns and keep legacy HeroUI isolated. +4. Check shared helpers in `ui/types`, `ui/lib`, and `ui/hooks` before adding duplicates. +5. Validate prop APIs, derived state, and styling decisions against the established UI rules. +6. Pull in generic framework skills only for the parts they specifically own. + +## Output Contract + +- State where the code should live in `ui/` and why. +- Call out the main UI rule applied: shadcn-first, scope rule, derived state, shared helper reuse, or discriminated unions. +- Mention any companion generic skills required. +- Flag any legacy HeroUI or state-sync risk that must be preserved or removed carefully. + +## References + +- [Repository agent rules](../../AGENTS.md) +- [UI component guidance](../../ui/AGENTS.md) +- [UI references](references/ui-docs.md) +- [TypeScript skill](../typescript/SKILL.md) +- [React 19 skill](../react-19/SKILL.md) +- [Next.js 15 skill](../nextjs-15/SKILL.md) +- [Tailwind 4 skill](../tailwind-4/SKILL.md) diff --git a/skills/prowler/SKILL.md b/skills/prowler/SKILL.md index 667ad0cc4b..b3fba13317 100644 --- a/skills/prowler/SKILL.md +++ b/skills/prowler/SKILL.md @@ -1,8 +1,6 @@ --- name: prowler -description: > - Main entry point for Prowler development - quick reference for all components. - Trigger: General Prowler development questions, project overview, component navigation (NOT PR CI gates or GitHub Actions workflows). +description: "Trigger: When the task is general Prowler development, repository navigation, component selection, or project overview work outside PR CI workflow details. Routes the model to the right Prowler surface fast." license: Apache-2.0 metadata: author: prowler-cloud @@ -12,54 +10,47 @@ metadata: allowed-tools: Read, Edit, Write, Glob, Grep, Bash, WebFetch, WebSearch, Task --- -## Components +## Activation Contract -| Component | Stack | Location | -|-----------|-------|----------| -| SDK | Python 3.10+, Poetry | `prowler/` | -| API | Django 5.1, DRF, Celery | `api/` | -| UI | Next.js 15, React 19, Tailwind 4 | `ui/` | -| MCP | FastMCP 2.13.1 | `mcp_server/` | +Use this skill first when the model needs to orient itself in the Prowler monorepo, choose the correct component, or point to the follow-up skill that should own the task. -## Quick Commands +## Hard Rules -```bash -# SDK -poetry install --with dev -poetry run python prowler-cli.py aws --check check_name -poetry run pytest tests/ +- Treat this skill as a router, not the final authority for API, UI, SDK, MCP, CI, or testing implementation details. +- Redirect specialized work to the matching Prowler skill before giving deep guidance. +- Keep component guidance anchored to real repo paths and current stack names. +- Do not use this skill for PR workflow gates or GitHub Actions analysis; those belong to `prowler-pr` or `prowler-ci`. +- Prefer concise orientation over long cookbook explanations. -# API -cd api && poetry run python src/backend/manage.py runserver -cd api && poetry run pytest +## Decision Gates -# UI -cd ui && pnpm run dev -cd ui && pnpm run healthcheck +| Question | Action | +|---|---| +| Is the task about monorepo orientation or “where does this live”? | Use this skill and route to the right component. | +| Is the task inside `api/` with RLS, RBAC, providers, or Celery? | Load `prowler-api`. | +| Is the task inside `ui/` with app structure or component conventions? | Load `prowler-ui`. | +| Is the task about checks, providers, compliance, docs, CI, or PR gates? | Hand off to the corresponding specialized Prowler skill. | +| Is the task only about testing strategy? | Load `tdd` plus the matching test skill. | -# MCP -cd mcp_server && uv run prowler-mcp +## Execution Steps -# Full Stack -docker-compose up -d -``` +1. Identify the affected surface: `prowler/`, `api/`, `ui/`, `mcp_server/`, or cross-cutting docs/CI. +2. Confirm the stack and runtime boundary for that surface. +3. Route to the correct specialized skill before proposing implementation details. +4. If multiple surfaces are involved, call out the primary owner and the supporting skills. +5. Return repo paths, component names, and the next best skill to load. -## Providers +## Output Contract -AWS, Azure, GCP, Kubernetes, GitHub, M365, OCI, AlibabaCloud, Cloudflare, MongoDB Atlas, NHN, LLM, IaC +- State the target component or components. +- Name the follow-up skill or skills that should own the work. +- Mention the canonical repo path(s) to inspect next. +- If the task is out of scope for this router skill, say so explicitly. -## Commit Style +## References -`feat:`, `fix:`, `docs:`, `chore:`, `perf:`, `refactor:`, `test:` - -## Related Skills - -- `prowler-sdk-check` - Create security checks -- `prowler-api` - Django/DRF patterns -- `prowler-ui` - Next.js/React patterns -- `prowler-mcp` - MCP server tools -- `prowler-test` - Testing patterns - -## Resources - -- **Documentation**: See [references/](references/) for links to local developer guide +- [Repository agent rules](../../AGENTS.md) +- [Prowler skill references](references/prowler-docs.md) +- [API component guidance](../../api/AGENTS.md) +- [UI component guidance](../../ui/AGENTS.md) +- [MCP component guidance](../../mcp_server/AGENTS.md)