Compare commits

...
Author SHA1 Message Date
Andoni A. b050c917c6 perf(attack-paths): optimize getPathEdges with O(1) adjacency maps
Pre-build parent and children maps at the start of traversal for O(1)
lookups instead of O(n) array searches per traversal step. This improves
performance for large attack path graphs.
2026-01-14 17:12:13 +01:00
Andoni A. 67933d7d2d Merge branch 'attack-paths-demo' into attack-paths-demo-extras 2026-01-14 17:06:33 +01:00
Andoni Alonso 39280c8b9b feat(attack-paths): add Bedrock and AttachRolePolicy privilege escalation queries (#9793) 2026-01-14 17:01:21 +01:00
Andoni Alonso 4bcaf29b32 feat(attack-paths): improve graph path highlighting (#9769) 2026-01-14 16:59:27 +01:00
Josema Camacho e95be697ef Prowler 511 leaving one database per scan (#9795) 2026-01-14 16:19:02 +01:00
Andoni A. 6fa4565ebd fix(attack-paths): connect virtual nodes from principal instead of effective_principal
When a principal can assume a role (effective_principal), the virtual
relationship was being created from effective_principal but the graph
showed the original principal, causing disconnected nodes.

Now the virtual relationship is created from the original principal,
keeping the graph fully connected while still detecting escalation
paths that require role assumption.
2026-01-14 09:18:18 +01:00
Andoni A. e426c29207 fix(attack-paths): remove duplicate path_target causing disconnected nodes
Removed the extra path_target re-match that was causing role nodes to appear
disconnected in the visualization. The target_role is now only connected via
virtual relationships (PASSES_ROLE → target_role → GRANTS_ACCESS), which
provides a cleaner and more accurate attack path visualization.
2026-01-14 09:11:27 +01:00
Andoni A. 1d8d4f9325 refactor(attack-paths): show target roles in PassRole escalation paths
Updated PassRole queries to display which specific role(s) can be passed
in the visualization, instead of just showing a count. The path now shows:

Principal → New Resource → Target Role → Privilege Escalation

This allows users to see exactly which admin roles a principal can pass
to escalate privileges, which is crucial for security analysis.

Queries updated: Lambda, ECS, Glue, Bedrock, CloudFormation
2026-01-14 09:06:11 +01:00
Andoni A. cad44a3510 fix(attack-paths): fix duplicate virtual nodes in priv escalation queries
Virtual nodes were being created for each result row, causing duplicates
in the graph visualization. Fixed by using aggregation pattern:
1. Deduplicate principals FIRST (before matching target roles)
2. Collect target roles per principal
3. Create ONE virtual node per principal with role count

Queries fixed:
- aws-iam-privesc-passrole-lambda
- aws-glue-privesc-passrole-dev-endpoint
- aws-bedrock-privesc-passrole-code-interpreter
- aws-cloudformation-privesc-passrole-create-stack

The virtual node description now shows "N admin role(s) can be passed"
instead of creating N separate nodes.
2026-01-14 08:58:26 +01:00
Andoni A. ee73e043f9 refactor(attack-paths): apply query improvements to remaining priv escalation queries
Apply the same patterns from PR #9770 to the other privilege escalation
queries that were missing the improvements:

- aws-iam-privesc-create-policy-version
- aws-iam-privesc-attach-role-policy-assume-role
- aws-iam-privesc-passrole-lambda
- aws-iam-privesc-role-chain

Changes applied:
- Add DISTINCT deduplication before creating virtual relationships
- Add re-match paths at the end for proper visualization
- Remove redundant path variables from RETURN statements
- Create unique virtual node IDs per principal->target pair
2026-01-13 16:39:32 +01:00
Andoni A. 815797bc2b fix(attack-paths): hide findings completely in full view
- Change opacity-based hiding to display:none for finding nodes
- Use visibility:hidden for finding edges in full view
- Add isFilteredView to useEffect dependency array
- In filtered view, all nodes/edges remain visible as expected
2026-01-12 17:13:52 +01:00
Andoni A. 9cd249c561 fix(attack-paths): show findings at full opacity in filtered view
When in filtered view, findings are part of the selected path and
should be fully visible, not hidden with reduced opacity.

- Add isFilteredView prop to AttackPathGraph component
- Skip hiding findings when isFilteredView is true
2026-01-12 16:59:47 +01:00
Andoni A. 00fe96a9f7 refactor(attack-paths): redraw graph with filtered data for optimal layout
Reverts the visibility-based approach to use data-changing approach
which redraws the graph with only the selected path nodes, optimizing
the layout for the filtered view.

- Store fullData separately when entering filtered view
- Compute filtered subgraph with only visible nodes and edges
- Graph redraws with new data, auto-fitting to show selected path
- Restore fullData when exiting filtered view
2026-01-12 16:56:35 +01:00
Andoni A. 7c45ee1dbb refactor(attack-paths): use visibility-based filtering with D3 transitions
Replace data-replacement approach with visibleNodeIds Set to fix
animation issues caused by Next.js server component re-renders.

- Changed useGraphState to compute visibleNodeIds without modifying data
- Graph component now animates opacity changes via D3 transitions
- Keeps DOM structure stable while providing smooth visual transitions
- Findings now properly appear when filtering by a resource node
2026-01-12 16:50:25 +01:00
Andoni A. d19a23f829 feat(attack-paths): highlight selected node and show findings in filtered view
- Add orange glow filter and pulsing animation for selected/filtered nodes
- Pass isFilteredView prop to graph component to show findings when filtering
- Update node styling to show thicker border (4px) and orange glow on selection
- Ensure findings are visible in filtered view instead of being hidden by default
2026-01-12 16:41:09 +01:00
Andoni A. b071fffe57 feat(attack-paths): add filtered view when clicking on graph nodes
When clicking a node in the attack path graph:
- Filters the graph to show only upstream (ancestors) and downstream (descendants) paths
- Includes findings directly connected to the selected node
- Shows a "Back to Full View" button to restore the complete graph
- Displays an indicator showing which node is being filtered

Uses atomic Zustand state updates to ensure proper re-rendering of the D3 graph.
2026-01-12 16:33:30 +01:00
Andoni A. 422c55404b refactor(attack-paths): simplify legend by consolidating resource types
Merge all individual resource type items (AWS Account, EC2 Instance,
S3 Bucket, etc.) into a single "Resource" entry to reduce visual
clutter in the graph legend.
2026-01-12 16:26:20 +01:00
Andoni A. 6c307385b0 fix(attack-paths): preserve aws variable in ECS query WITH clause
Add aws to intermediate WITH clause to fix 'Variable aws not defined'
error when deduplicating principals before matching target roles.
2026-01-12 14:47:12 +01:00
Andoni A. 13964ccb1c fix(attack-paths): merge ECS target roles into single virtual node per principal
Instead of creating one virtual ECS task node per target role, merge all
target roles into a single node per principal. The node description shows
how many admin roles can be passed (e.g., '3 admin role(s) can be passed').

This reduces visual clutter when a principal can pass multiple admin roles.
2026-01-12 13:30:53 +01:00
Andoni A. 64ed526e31 refactor(attack-paths): rename virtual nodes from Malicious to New
The virtual nodes represent potential resources that could be created
for privilege escalation, not actual malicious resources. Renamed for
clarity:
- Malicious Task Definition -> New Task Definition
- Malicious Dev Endpoint -> New Dev Endpoint
- Malicious Code Interpreter -> New Code Interpreter
- Malicious Stack -> New Stack
2026-01-09 15:05:12 +01:00
Andoni A. 2388a053ee fix(attack-paths): highlight single path upstream instead of all paths
Changed upstream traversal to follow only one parent at each level
instead of all parents. This prevents the entire graph from lighting
up when selecting a node that has multiple ancestors with many children.

- Upstream: now uses find() to get first parent only
- Downstream: unchanged, still highlights all descendants
2026-01-08 18:18:44 +01:00
Andoni A. 7bb5354275 feat(attack-paths): improve graph visualization and interactions
- Change edge color from orange to white by default
- Highlight entire path in orange on node hover/selection
- Add Ctrl + scroll to zoom functionality with increased speed
- Update node borders to orange on hover/selection
- Add zoom hint to legend
- Remove hover effect from info button
2026-01-08 17:57:24 +01:00
Andoni A. 03cae9895b wip 2026-01-08 15:43:59 +01:00
Andoni A. e398b654d4 merge all privesc nodes into one, change it to look like a finding 2026-01-07 16:45:40 +01:00
Andoni A. d9e978af29 initial version 2026-01-07 10:50:29 +01:00
Josema Camacho 95d9e9a59f feat(attack-paths): Update Cartography dependency and its usage (#9593) 2025-12-18 15:52:15 +01:00
Josema Camacho 48f19d0f11 fix(attack-paths): neo4j.exceptions import (#9356) 2025-12-01 10:31:18 +01:00
Josema Camacho 345033e58a Fix attack paths demo neo4j conneciton (#9352)
Add retryable Neo4j session.
2025-11-29 12:55:49 +01:00
Alan BuscagliaandClaude 15cb87534c feat(attack-paths): apply Scope Rule pattern for feature-local organization (#9270)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-28 17:05:35 +01:00
Josema Camacho 5a85db103d feat(attack-paths): Task and endpoints (#9344)
- Added support to Neo4j
- Added Cartography as Attack Paths Scan
- Added Attack Path Scans endpoints for their management and run queries on those scan
2025-11-28 15:44:15 +01:00
César Arroba 2b86078d06 chore(api): build attack paths demo image (#9349) 2025-11-28 15:33:04 +01:00
lydiavilchezandHugoPBrito b2abdbeb60 feat(gcp-compute): add check to ensure VMs are not preemptible or spot (#9342)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-28 12:49:19 +01:00
lydiavilchez dc852b4595 feat(gcp-compute): add automatic restart check for VM instances (#9271) 2025-11-28 12:21:58 +01:00
Hugo Pereira Brito 1250f582a5 fix(check): custom check folder validation (#9335) 2025-11-28 12:19:47 +01:00
Pedro Martín bb43e924ee fix(report): use pagina for ENS in footer (#9345) 2025-11-28 12:04:30 +01:00
Andoni Alonso 0225627a98 fix(docs): fix image paths (#9341) 2025-11-28 11:20:54 +01:00
Alan Buscaglia 3097513525 fix(ui): filter Risk Pipeline chart by selected providers and show zero-data legends (#9340) 2025-11-27 17:39:01 +01:00
Alan Buscaglia 6af9ff4b4b feat(ui): add interactive charts with filter navigation (#9333) 2025-11-27 16:04:55 +01:00
Hugo Pereira Brito 06fa57a949 fix(docs): info warning format (#9339) 2025-11-27 09:57:05 -05:00
mattkeelerandHugoPBrito dc9e91ac4e fix(m365): Support multiple Exchange mailbox policies (#9241)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-27 14:10:15 +01:00
Shafkat RahmanandAndoni Alonso 59f8dfe5ae feat(github): add immutable releases check (#9162)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-11-27 13:40:15 +01:00
Adrián Jesús Peña Rodríguez 7e0c5540bb feat(api): restore compliance overview endpoint (#9330) 2025-11-27 13:31:15 +01:00
Daniel Barranquero 79ec53bfc5 fix(ui): update changelog (#9334) 2025-11-27 13:16:50 +01:00
Daniel BarranqueroandAlan Buscaglia ed5f6b3af6 feat(ui): add MongoDB Atlas provider support (#9253)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-11-27 12:37:20 +01:00
Andoni Alonso 6e135abaa0 fix(iac): ignore mutelist in IaC scans (#9331) 2025-11-27 11:08:58 +01:00
Hugo Pereira Brito 65b054f798 feat: enhance m365 documentation (#9287) 2025-11-26 16:17:43 +01:00
Alan Buscagliaandalejandrobailo 28d5b2bb6c feat(ui): integrate threat map with regions API endpoint (#9324)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-26 16:12:31 +01:00
Prowler Botandprowler-bot c8d9f37e70 feat(aws): Update regions for AWS services (#9294)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-11-26 09:42:40 -05:00
lydiavilchezandDaniel Barranquero 9d7b9c3327 feat(gcp): Add VPC Service Controls check for Cloud Storage (#9256)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-26 14:45:27 +01:00
Hugo Pereira BritoandPepe Fagoaga 127b8d8e56 fix: typo in pdf report generation (#9322)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-11-26 13:58:40 +01:00
Alan Buscagliaandalejandrobailo 4e9dd46a5e feat(ui): add Risk Pipeline View with Sankey chart to Overview page (#9320)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-26 13:33:58 +01:00
Hugo Pereira Brito 880345bebe fix(sharepoint): false positives on disabled external sharing (#9298) 2025-11-26 12:23:04 +01:00
Andoni Alonso 1259713fd6 docs: remove AMD-only docker images warning (#9315) 2025-11-26 10:26:39 +01:00
Prowler Botandprowler-bot 26088868a2 chore(release): Bump version to v5.15.0 (#9318)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-11-26 10:19:25 +01:00
César Arroba e58574e2a4 chore(github): fix container actions (#9321) 2025-11-26 10:16:26 +01:00
Alan Buscagliaandalejandrobailo a07e599cfc feat(ui): add service watchlist component with real API integration (#9316)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-25 17:03:24 +01:00
Alejandro BailoandAlan Buscaglia e020b3f74b feat: add watchlist component (#9199)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-11-25 16:01:38 +01:00
Alan Buscaglia 8e7e376e4f feat(ui): hide new overview route and filter mongo providers (#9314) 2025-11-25 14:22:03 +01:00
Alan Buscaglia a63a3d3f68 fix: add filters for mongo providers and findings (#9311) 2025-11-25 13:19:49 +01:00
Andoni AlonsoandChandrapal Badshah 10838de636 docs: refactor Lighthouse AI pages (#9310)
Co-authored-by: Chandrapal Badshah <12944530+Chan9390@users.noreply.github.com>
2025-11-25 13:10:29 +01:00
5ebf455e04 docs: Lighthouse multi LLM provider support (#9306)
Co-authored-by: Chandrapal Badshah <12944530+Chan9390@users.noreply.github.com>
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-11-25 13:04:30 +01:00
Daniel Barranquero 0d59441c5f fix(api): add alter to mongodbatlas migration (#9308) 2025-11-25 11:29:07 +01:00
Pepe FagoagaandVíctor Fernández Poyatos 3b05a1430e chore(changelog): reconcile for v5.14 (#9277)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-11-24 19:03:53 +01:00
Alan Buscaglia ea953fb256 fix(ui): UI improvements - buttons, form validations, and chart alignment (#9299) 2025-11-24 17:14:12 +01:00
Andoni Alonso 2198e461c9 feat(iac): use branch as region for IaC findings (#9295) 2025-11-24 17:00:06 +01:00
Adrián Jesús Peña Rodríguez 75abd8f54d fix(threatscore): exclude muted findings from aggregated statistics in threatscore utils (#9296) 2025-11-24 13:25:20 +01:00
Adrián Jesús Peña Rodríguez 2f184a493b feat(threatscore): restore API threatscore snapshots (#9291) 2025-11-24 10:47:03 +01:00
Pepe Fagoaga e2e06a78f9 fix(lock): update poetry lock for prowler (#9290) 2025-11-24 10:05:14 +01:00
Adrián Jesús Peña Rodríguez de5aba6d4d feat(api): add new endpoint for retrieving findings data by region with associated filters and response schema (#9273) 2025-11-21 11:23:31 +01:00
César Arroba 6e7266eacf chore(github): fix sdk build action (#9288) 2025-11-21 11:03:52 +01:00
Alan Buscaglia 58bb66ff27 feat(ui/overview): add click navigation for charts and threat score improvements (#9281) 2025-11-20 18:47:42 +01:00
46bfe02ee8 feat(nis2): support PDF reporting (#9170)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
Co-authored-by: Josema Camacho <josema@prowler.com>
2025-11-20 17:14:54 +01:00
Pepe Fagoaga cee9a9a755 fix(html): logo URI (#9282) 2025-11-20 17:11:51 +01:00
Hugo Pereira Brito b11ba9b5cb feat(docs): add links for sp and cert from getting started to authentication (#9286) 2025-11-20 16:50:18 +01:00
Víctor Fernández Poyatos 789fc84e31 fix(overviews): exclude muted findings from severity overview (#9283) 2025-11-20 16:29:20 +01:00
Alejandro Bailo 6426558b18 fix(ui): pre-release fixes and improvements (#9278) 2025-11-20 16:18:25 +01:00
Hugo Pereira Brito 9a1ddedd94 fix(docs): typo (#9285) 2025-11-20 16:07:22 +01:00
Hugo Pereira Brito 0ae400d2b1 fix(docs): add link from getting started to auth for service accounts (#9284) 2025-11-20 15:55:19 +01:00
Víctor Fernández Poyatos ced122ac0d feat(migrations): add missing remove index operation (#9280) 2025-11-20 15:09:14 +01:00
Hugo Pereira Brito dc7d2d5aeb fix(outputs): refresh scan timestamps per run (#9272) 2025-11-20 13:12:39 +01:00
Alan Buscaglia b6ba6c6e31 feat(hooks): integrate Python pre-commit with Husky for monorepo (#9279) 2025-11-20 12:48:43 +01:00
Hugo Pereira Brito 30312bbc03 fix(docs): remove wrong threatscore warning (#9276) 2025-11-20 09:03:15 +01:00
Pedro Martínandalejandrobailo 94fe87b4a2 feat(ens): support PDF reporting (#9158)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-19 18:57:58 +01:00
Pedro Martín 219bc12365 feat(kubernetes): add Prowler ThreatScore compliance framework (#9235) 2025-11-19 18:31:54 +01:00
Pedro Martín 66394ab061 fix(threatscore): remove typo from 3. Logging and *m*onitoring (#9274) 2025-11-19 17:12:29 +01:00
Rubén De la Torre VicoandDaniel Barranquero 7348ed2179 chore(aws): enhance metadata for kinesis service (#9262)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-19 16:49:31 +01:00
Rubén De la Torre VicoandDaniel Barranquero 0b94f2929d chore(aws): enhance metadata for documentdb service (#8862)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-19 13:49:57 +01:00
Alejandro Bailo c23e2502f3 refactor(ui): redo the whole app with styles (#9234) 2025-11-19 11:37:17 +01:00
Adrián Jesús Peña Rodríguez c418c59b53 feat(compliance): enhance compliance overview filters and documentation (#9244) 2025-11-19 10:35:31 +01:00
Adrián Jesús Peña Rodríguez 3dc4ab5b83 refactor(api): remove ServiceOverviewFilter and update related tests (#9248) 2025-11-19 10:33:31 +01:00
Andoni Alonso 148a6f341b docs(sso): improve okta sso section (#9233) 2025-11-19 08:04:44 +01:00
Daniel Barranquero b5df26452a fix: split file_name not working on Windows (#9268) 2025-11-18 14:45:31 +01:00
Hugo Pereira Brito 45792686aa fix(docs): enhance gcp service account authentication and add missing permissions (#9231) 2025-11-18 14:09:03 +01:00
Rubén De la Torre Vico ee31e82707 fix: make JSON schema simpler to work with more MCP clients (#9257) 2025-11-18 13:35:11 +01:00
lydiavilchezandDaniel Barranquero 0ba1226d88 feat(gcp): implement Cloud Storage Data Access Audit Logs check (#9220)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-18 12:08:54 +01:00
Andoni Alonso 520cc31f73 docs: fix mutelist broken links (#9249) 2025-11-17 18:24:02 +01:00
Andoni Alonso a5a882a975 fix(iac): add trivy installation in CLI image (#9247) 2025-11-17 16:04:01 +01:00
Prowler Botandprowler-bot 84f9309a7c feat(aws): Update regions for AWS services (#9243)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-11-17 09:59:58 -05:00
Rubén De la Torre VicoandHugoPBrito cf3800dbbe chore(aws): enhance metadata for ecs service (#8888)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-17 15:25:30 +01:00
Adrián Jesús Peña Rodríguez d43455971b fix(scan): implement temporary workaround to skip findings with UID exceeding 300 characters (#9246) 2025-11-17 13:15:02 +01:00
Paco Sanchez LopezandAndoni A. 1ea0dabf42 feat(arm): adds support building multiarch prowler containers (#8773)
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-11-17 12:35:33 +01:00
Rubén De la Torre VicoandDaniel Barranquero 0f43789666 chore(kubernetes): enhance metadata for etcd service (#9096)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-17 12:30:21 +01:00
Andoni Alonso 4f8e8ed935 chore(github): replace status/awaiting-response label with status/waiting-for-revision if comment added (#9245) 2025-11-17 12:20:33 +01:00
Rakan Farhouda 518508d5fe feat(api): add metadata attributes to ResourceSerializer and tests (#9098) 2025-11-17 14:10:45 +03:00
Rubén De la Torre VicoandHugoPBrito e715b9fbfb chore(aws): enhance metadata for ecr service (#8872)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-17 11:50:11 +01:00
Marc EspinandAndoni A. 4167de39d2 fix(docs): Fix dead links leading to docs.prowler.cloud (#9240)
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-11-17 09:56:51 +01:00
531ba5c31b feat(azure): new check for Entra ID authentication for Azure PostgreSQL Flexible Server (#8764)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-14 13:54:57 +01:00
031548ca7e feat: Update Lighthouse UI to support multi LLM (#8925)
Co-authored-by: Chandrapal Badshah <12944530+Chan9390@users.noreply.github.com>
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-14 11:46:38 +01:00
Hugo Pereira Brito 866edfb167 chore(outputs): raise an error when using -M asff for a provider other than aws (#9225) 2025-11-13 16:53:22 +01:00
Daniel Barranquero d1380fc19d fix(azure): validation and other errors in cosmosdb, defender, storage and vm (#8915) 2025-11-13 09:17:44 -05:00
Víctor Fernández PoyatosandJosema Camacho 46666d29d3 feat(db): optimize write queries for scan related tasks (#9190)
Co-authored-by: Josema Camacho <josema@prowler.com>
2025-11-13 12:27:57 +01:00
Rubén De la Torre VicoandDaniel Barranquero ce5f2cc5ed chore(aws): enhance metadata for elbv2 service (#9001)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-13 10:45:20 +01:00
Lee Trout c5c7b84afd chore(ec2): prevent test from calling live AWS endpoint (#9228) 2025-11-13 10:12:19 +01:00
Ryan Nolette 3432c8108c chore: updated gitignore file to be more robust for VSCode development environments and AI coding assistants. (#9226) 2025-11-13 09:32:21 +01:00
Andoni Alonso 7c42a61e17 docs(aws): restore STS Ireland endpoint warning (#9229) 2025-11-13 09:30:27 +01:00
Rubén De la Torre VicoandSergio Garcia 575521c025 chore(oraclecloud): enhance metadata for cloudguard service (#9223)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-12 11:58:54 -05:00
Rubén De la Torre VicoandSergio Garcia eab6c23333 chore(oraclecloud): enhance metadata for blockstorage service (#9222)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-12 11:51:29 -05:00
Rubén De la Torre VicoandDaniel Barranquero 8ee9454dbc chore(aws): enhance metadata for elb service (#8935)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-11-12 16:46:12 +01:00
Pedro Martín b46a8fd0ba feat(compliance): change C5 logo (#9224) 2025-11-12 16:01:18 +01:00
Rubén De la Torre VicoandSergio Garcia 77ef4869e3 chore(oraclecloud): enhance metadata for audit service (#9221)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-12 09:24:20 -05:00
Alan BuscagliaandClaude 07ac96661e feat: implement Finding Severity Over Time chart with time range selector (#9106)
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-12 14:33:20 +01:00
Daniel Barranquero 98f8ef1b4b feat(mongodbatlas): add provider_id verification (#9211) 2025-11-12 13:50:00 +01:00
Pepe Fagoaga 5564b4c7ae fix(env): fallback to local (#9215) 2025-11-12 10:14:29 +01:00
Pedro Martín 427dab6810 fix(compliance): handle check_id not in Prowler Checks (#9208) 2025-11-12 09:11:34 +01:00
Andoni Alonso ee62ea384a chore(github): merge labeler actions (#9218) 2025-11-12 08:39:20 +01:00
Andoni Alonso ca4c4c8381 docs: remove Prowler App credentials handling duplicates (#9212) 2025-11-12 08:23:25 +01:00
e246c0cfd7 fix(aws): false negative in iam_role_cross_service_confused_deputy_prevention (#9213)
Co-authored-by: shaun <shaun@snotra.cloud>
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-11-11 17:39:16 -05:00
Rubén De la Torre Vico 74025b2b5e docs: add a architecture schema for MCP Server (#9214) 2025-11-11 11:53:01 -05:00
ccb269caa2 chore(dependencies): add Sentry to /ui (#8730)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-11-11 17:12:42 +01:00
Rubén De la Torre VicoandSergio Garcia 0f22e754f2 chore(mongodbatlas): enhance metadata for projects service (#9093)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-11 11:10:40 -05:00
Josema Camacho 7cb0ed052d chore(security): upgrading django to 5.1.14 (#9176) 2025-11-11 16:51:28 +01:00
Andoni Alonso 1ec36d2285 docs: add Prowler Cloud public IPs (#9209) 2025-11-11 16:11:24 +01:00
lydiavilchez b0ec7daece feat(gcp): add check cloudstorage_bucket_sufficient_retention_period (#9149) 2025-11-11 15:51:57 +01:00
Hugo Pereira Brito 1292abcf91 fix(m365_powershell): restore MSAL.PS (#9210) 2025-11-11 15:35:45 +01:00
Rubén De la Torre VicoandSergio Garcia 136366f4d7 chore(github): enhance metadata for organization service (#9094)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-11 09:34:54 -05:00
StylusFrost 203b46196b fix(test-ui): update authentication method selection in ProvidersPage for AWS Add Provider e2e test (#9161) 2025-11-11 15:11:56 +01:00
Adrián Jesús Peña Rodríguez beec37b0da feat(threatscore): implement ThreatScoreSnapshot model, filter, serializer, and view for ThreatScore metrics retrieval (#9148) 2025-11-11 10:19:48 +01:00
Hugo Pereira Brito 73a277f27b chore(m365_powershell): remove unnecessary test_credentials (#9204) 2025-11-11 10:16:57 +01:00
Andoni Alonso 822d201159 fix(github): hardcode list of prowler-cloud organization members (#9207) 2025-11-11 10:03:12 +01:00
Andoni AlonsoandHugo Pereira Brito 8e07ec8727 docs: refactor contributing docs (#9202)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2025-11-11 09:44:41 +01:00
Sergio Garcia 7c339ed9e4 docs(mutelist): fix misleading docstrings about tag and exception logic (#9205) 2025-11-10 13:39:24 -05:00
Sergio Garcia be0b8bba0d fix(html): rename get_oci_assessment_summary (#9200) 2025-11-10 10:15:54 -05:00
Prowler Botandprowler-bot 521afab4aa feat(aws): Update regions for AWS services (#9194)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-11-10 09:37:18 -05:00
Ethan Troyandpedrooot 789221d901 feat(compliance): add FedRAMP 20x KSI Low compliance frameworks (#9198)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2025-11-10 14:41:18 +01:00
Hugo Pereira Brito ef4e28da03 fix(m365_powershell): teams connection with --sp-env-auth and enhanced timeouts error logging (#9191) 2025-11-10 11:23:56 +01:00
Alejandro Bailo ee2d3ed052 feat: implement new design system variables across new components and add skeletons (#9193) 2025-11-10 09:19:10 +01:00
Pedro Martín 66a04b5547 feat(aws): improve nist_csf_2.0 mapping (#9189) 2025-11-07 10:59:40 -05:00
Hugo Pereira Brito fb9eda208e fix(powershell): depth truncation and parsing error (#9181) 2025-11-07 13:19:37 +01:00
Rakan Farhouda f0b1c4c29e fix(api): update unique constraint for Provider model to exclude soft… (#9054) 2025-11-07 13:16:55 +01:00
Alan Buscagliaandalejandrobailo a73a79f420 fix: exclude docs folder from Tailwind content scanning (#9184)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-07 10:49:27 +01:00
Rubén De la Torre Vico 5d4b7445f8 chore: fix image path in README for Prowler App (#9186) 2025-11-07 10:17:42 +01:00
Rubén De la Torre VicoandSergio Garcia 13e4866507 chore(oraclecloud): enhance metadata for analytics service (#9114)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-11-06 16:58:59 -05:00
UniCodeandMrCloudSec 7d5c4d32ee feat(aws): add compliance NIST CSF 2.0 (#9185)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-11-06 16:55:16 -05:00
Daniel Barranquero 7e03b423dd feat(api): add MongoDBAtlas provider to api (#9167) 2025-11-06 16:37:38 -05:00
0ad5bbf350 feat(github): Add GitHub check ensuring repository creation is limited (#8844)
Signed-off-by: Mauricio Harley <mauricioharley@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
2025-11-06 16:13:10 +01:00
Alejandro Bailo 38f60966e5 fix(ui): improve pre commit (#9180) 2025-11-06 14:32:06 +01:00
Alan Buscagliaandalejandrobailo 7bbc0d8e1b feat: add claude code validation to pre-commit hook (#9177)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-06 13:48:19 +01:00
Pedro Martín edfef51e7a feat(compliance): add naming and visual improvements (#9145) 2025-11-06 13:06:59 +01:00
Hugo Pereira Brito 788113b539 fix: changelog (#9179) 2025-11-06 12:57:51 +01:00
Hugo Pereira Brito 8ab77b7dba fix(gcp): check check_name has no resource_name error (#9169) 2025-11-06 12:37:49 +01:00
Sergio GarciaandAndoni Alonso e038b2fd11 chore(sdk): add validation for invalid checks, services, and categories (#8971)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-11-06 11:46:21 +01:00
dependabot[bot] 2e5f17538d chore(deps): bump agenthunt/conventional-commit-checker-action from 2.0.0 to 2.0.1 (#9127)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:12:17 +01:00
dependabot[bot] 54294c862b chore(deps): bump trufflesecurity/trufflehog from 3.90.11 to 3.90.12 (#9128)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:11:46 +01:00
dependabot[bot] ace2b88c07 chore(deps): bump sorenlouv/backport-github-action from 9.5.1 to 10.2.0 (#9132)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:11:30 +01:00
dependabot[bot] 3de8159de9 chore(deps): bump actions/setup-node from 5.0.0 to 6.0.0 (#9135)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:10:29 +01:00
dependabot[bot] 1a4ae33235 chore(deps): bump softprops/action-gh-release from 2.3.3 to 2.4.1 (#9134)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:09:29 +01:00
dependabot[bot] e0260b91e6 chore(deps): bump peter-evans/create-or-update-comment from 4.0.0 to 5.0.0 (#9133)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:08:38 +01:00
dependabot[bot] 66590f2128 chore(deps): bump github/codeql-action from 3.30.5 to 4.31.2 (#9131)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:07:27 +01:00
dependabot[bot] 33bb2782f0 chore(deps): bump aws-actions/configure-aws-credentials from 5.0.0 to 5.1.0 (#9130)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-11-06 10:05:52 +01:00
César Arroba 2f61c88f74 chore(github): improve container slack notifications (#9144) 2025-11-06 09:33:33 +01:00
Andoni Alonsoandalejandrobailo b25ed9fd27 feat(github): add external resource link (#9153)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-11-05 15:57:41 +01:00
Sergio Garcia 191d51675c chore(ui): rename OCI provider to oraclecloud (#9166) 2025-11-05 08:59:55 -05:00
Andoni Alonso 5b20fd1b3b docs(iac): add IaC getting started in Cloud/App (#9152) 2025-11-05 09:20:18 +01:00
Pepe Fagoaga 02489a5eef docs: get latest version to install Prowler App (#9163) 2025-11-04 18:31:00 +01:00
Sergio Garcia f16f94acf3 chore(oci): rename OCI provider to oraclecloud with oci alias (#9126) 2025-11-04 11:44:56 -05:00
Alejandro Bailo 1e584c5b58 feat: new overview threat score component (#9125) 2025-11-04 15:08:58 +01:00
César Arroba 1bb6bc148e chore(github): fix prepare release action (#9159) 2025-11-04 14:44:25 +01:00
César Arroba 166ab1d2c1 chore(github): fix actions paths (#9154) 2025-11-04 12:27:34 +01:00
StylusFrost dd85ca7c72 test(ui): add M365 provider management E2E tests (#8954) 2025-11-04 11:22:39 +01:00
Andoni Alonso b9aef85aa2 fix(github): user previous command to set labels (#9099) 2025-11-04 11:08:35 +01:00
Andoni Alonso 601495166c feat(iac): add IaC to Prowler App (#8751) 2025-11-04 10:01:58 +01:00
Hugo Pereira Brito 61a66f2bbf fix(aws): firehose_stream_encrypted_at_rest description and logic (#9142) 2025-11-03 11:31:18 -05:00
Rakan Farhouda 8b0b9cad32 fix(aws): update logger import in to use the correct module (#9138) 2025-11-03 18:09:41 +03:00
Alejandro Bailo 000b48b492 feat(ui): add Customer Support link to sidebar (#9143) 2025-11-03 16:01:11 +01:00
JDeepandpedrooot a564d6a04e feat(compliance): Add HIPAA compliance framework for GCP (#8955)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2025-11-03 15:34:08 +01:00
Prowler Botandprowler-bot 82bacef7c7 feat(aws): Update regions for AWS services (#9140)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-11-03 09:10:28 -05:00
Alejandro Bailo a4ac7bb067 feat(ui): move Resource ID field up (#9141) 2025-11-03 11:39:42 +01:00
StylusFrost a41f8dcb18 test(ui): add Azure provider management E2E tests (#8949) 2025-11-03 09:20:24 +01:00
Alejandro Bailo 2bf93c0de6 feat: RSS system (#9109) 2025-11-03 09:17:37 +01:00
Sergio Garcia 39710a6841 fix(api): correct OCI provider compliance directory mapping (#9111) 2025-10-31 10:33:13 -04:00
f330440c54 chore(aws): enhance metadata for codeartifact service (#8850)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2025-10-31 15:21:50 +01:00
c3940c7454 feat: Add Amazon Bedrock & OpenAI Compatible provider to Lighthouse AI (#8957)
Co-authored-by: Chandrapal Badshah <12944530+Chan9390@users.noreply.github.com>
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-10-31 13:54:15 +01:00
Rubén De la Torre Vico df39f332e4 docs: add new definitions for checks serverities (#9123) 2025-10-31 13:22:16 +01:00
lydiavilchezandDaniel Barranquero 4a364d91be feat(gcp): add cloudstorage_bucket_logging_enabled check (#9091)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-31 13:01:55 +01:00
César Arroba 4b99c7b651 chore(github): missed conditional on sdk container action (#9120) 2025-10-31 11:43:09 +01:00
c441423d6a chore(aws): enhance metadata for codebuild service (#8851)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2025-10-31 11:41:34 +01:00
César Arroba 7e7f160b9a chore(sdk): allow sdk checks only on prowler repository (#9116) 2025-10-31 11:31:25 +01:00
César Arroba aaae73cd1c chore(github): rename jobs to know which component they belong (#9117) 2025-10-31 11:31:16 +01:00
Víctor Fernández Poyatos c5e88f4a74 feat(rls-transaction): add retry for read replica connections (#9064) 2025-10-31 11:09:05 +01:00
Víctor Fernández Poyatos 5d4415d090 feat(mute-rules): Support simple muting in API (#9051) 2025-10-31 10:49:17 +01:00
César Arroba 5d840385df chore(github): fix slack messages (#9107) 2025-10-30 17:21:11 +01:00
Pedro Martín f831171a21 feat(compliance): add C5 for GCP provider (#9097) 2025-10-30 15:55:07 +01:00
César Arroba 2740d73fe7 chore(github): improve slack notification action (#9100) 2025-10-30 15:32:14 +01:00
Rubén De la Torre VicoandSergio Garcia 1c906b37cd chore(gcp): enhance metadata for artifacts service (#9088)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-30 10:30:27 -04:00
Sergio Garciaandalejandrobailo 98056b7c85 fix(ui): auto-populate OCI tenancy from provider UID in credentials form (#9074)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2025-10-30 09:47:15 -04:00
f15ef0d16c chore(aws): enhance metadata for elasticbeanstalk service (#8934)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-30 09:38:42 -04:00
Alan Buscaglia c42ce6242f refactor: improve React 19 event typing in select component (#9043) 2025-10-30 14:20:26 +01:00
Alan Buscaglia 702d652de1 feat: add comprehensive CSS theme variables for semantic color system (#9060) 2025-10-30 14:18:47 +01:00
Alan Buscaglia fff02073cf feat(overview): findings visualizations tabs component (#8999) 2025-10-30 14:18:14 +01:00
23e3ea4a41 chore(aws): enhance metadata for cloudwatch service (#8848)
Co-authored-by: HugoPBrito <hugopbrit@gmail.com>
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2025-10-30 14:08:18 +01:00
Chandrapal BadshahandChandrapal Badshah f9afb50ed9 fix(api): standardize JSON:API resource types for Lighthouse endpoints (#9085)
Co-authored-by: Chandrapal Badshah <12944530+Chan9390@users.noreply.github.com>
2025-10-30 13:36:51 +01:00
Andoni Alonso 3b95aad6ce fix(github): use members endpoint to verify author (#9086) 2025-10-30 13:25:00 +01:00
Andoni Alonso ac5737d8c4 docs(threatscore): banner only available in Cloud/App (#9095) 2025-10-30 13:23:48 +01:00
César Arroba a452c8c3eb chore(github): send slack message on container release (#9089) 2025-10-30 13:20:54 +01:00
Adrián Jesús Peña Rodríguez aa8be0b2fe fix(api): update database routing logic in MainRouter (#9080) 2025-10-30 12:30:53 +01:00
Rubén De la Torre VicoandDaniel Barranquero 46bf8e0fef chore(aws): enhance metadata for elasticache service (#8933)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-30 11:39:01 +01:00
Andoni Alonso c0df0cd1a8 chore(github): run community label only in main repo (#9083) 2025-10-30 10:16:55 +01:00
César Arroba 80d58a7b50 chore(github): separate mcp pr jobs in different actions (#9079) 2025-10-30 10:03:05 +01:00
César Arroba 2c28d74598 chore(github): separate api pr jobs in different actions (#9078) 2025-10-30 10:02:53 +01:00
César Arroba 4feab1be55 chore(github): separate ui pr jobs in different actions (#9076) 2025-10-30 10:02:41 +01:00
César Arroba 5bc9b09490 chore(github): separate sdk pr jobs in different actions (#9075) 2025-10-30 10:02:22 +01:00
Pedro MartínandSergio Garcia fcf817618a feat(compliance): add c5 azure base (#9081)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-30 09:54:50 +01:00
Rubén De la Torre VicoandDaniel Barranquero cad97f25ac chore(aws): enhance metadata for eks service (#8890)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-30 09:49:00 +01:00
Rubén De la Torre Vico b854563854 fix(emr): invalid JSON trailing comma (#9082) 2025-10-30 09:38:48 +01:00
Rubén De la Torre VicoandSergio Garcia 573975f3fe chore(aws): enhance metadata for emr service (#9002)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-29 15:37:14 -04:00
Rubén De la Torre VicoandSergio Garcia f4081f92a1 chore(aws): enhance metadata for eventbridge service (#9003)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-29 15:14:36 -04:00
Rubén De la Torre VicoandSergio Garcia 374496e7ff chore(aws): enhance metadata for firehose service (#9004)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-29 14:18:37 -04:00
Rubén De la Torre VicoandSergio Garcia 2a9c2b926d chore(aws): enhance metadata for fms service (#9005)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-29 14:15:00 -04:00
Pedro Martín f2f1e6bce6 feat(dashboard): update logo (#9040) 2025-10-29 14:12:56 -04:00
Rubén De la Torre VicoandSergio Garcia 25c823076f chore(aws): enhance metadata for fsx service (#9006)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-10-29 14:11:53 -04:00
Rubén De la Torre Vico 6ff559c0d4 chore(aws): enhance metadata for glacier service (#9007) 2025-10-29 14:03:14 -04:00
Andoni Alonso 899db55f56 chore(github): refactor community labeler (#9077) 2025-10-29 17:58:48 +01:00
Andoni Alonso 22d801ade2 chore(github): refactor community labeler (#9073) 2025-10-29 16:40:56 +01:00
César Arroba 1dc6d41198 chore: revert files ignore action removal (#9070) 2025-10-29 15:24:34 +01:00
César Arroba 456712a0ef chore(github): fix trivy action (#9066) 2025-10-29 14:51:49 +01:00
Hugo Pereira BritoandCésar Arroba 885ee62062 fix(m365): admincenter service unnecessary msgraph calls and repeated resource_id (#9019)
Co-authored-by: César Arroba <cesar@prowler.com>
2025-10-29 14:50:25 +01:00
César Arroba bbeccaf085 chore(github): improve trivy scan time (#9065) 2025-10-29 14:40:48 +01:00
César Arroba d1aca5641a chore(github): increase sdk tests timeout to 120m (#9062) 2025-10-29 13:47:10 +01:00
Pepe Fagoaga 3b7eba64aa chore: remove not used admin interface (#9059) 2025-10-29 17:37:09 +05:45
César Arroba e9e0797642 chore(github): improve container actions (#9061) 2025-10-29 12:42:53 +01:00
lydiavilchezandDaniel Barranquero aaa5abdead feat(gcp): add cloudstorage_bucket_soft_delete_enabled check (#9028)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-29 12:02:46 +01:00
César Arroba 0a2749b716 chore(github): improve SDK container build and push action (#9034) 2025-10-29 12:00:15 +01:00
César Arroba 8f8bf63086 chore(github): improve UI container build and push action (#9033) 2025-10-29 11:59:54 +01:00
César Arroba ea27817a2c chore(github): improve API container build and push action (#9032) 2025-10-29 11:59:39 +01:00
César Arroba 9068e6bcd0 chore(github): improve sdk pull request action (#9027) 2025-10-29 11:10:08 +01:00
César Arroba a4907d8098 chore(github): improve UI pull request action (#9029) 2025-10-29 10:58:57 +01:00
César Arroba caee7830a5 chore(github): improve SDK refresh AWS regions action (#9031) 2025-10-29 10:35:30 +01:00
César Arroba 65d2989bea chore(github): improve SDK PyPi release action (#9030) 2025-10-29 10:35:20 +01:00
Adrián Jesús Peña Rodríguez 6c34945829 feat(api): enhance overview provider aggregation and resource counting (#9053) 2025-10-29 10:31:40 +01:00
César Arroba ce859ddd1f chore(github): improve bump version action (#9024) 2025-10-29 10:26:31 +01:00
Sergio Garcia 0ca059b45b feat(ui): add Oracle Cloud Infrastructure (OCI) provider support (#8984) 2025-10-28 17:30:12 -04:00
Sergio GarciaandDaniel Barranquero dad100b87a feat(api): add Oracle Cloud Infrastructure (OCI) provider support (#8927)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-28 16:43:24 +01:00
Adrián Jesús Peña Rodríguez 662296aa0e feat(api): enhance provider filtering and pagination capabilities (#8975) 2025-10-28 16:36:35 +01:00
Rubén De la Torre Vico b6d49416f0 docs(mcp): add specific tutorial per famouse MCP Host (#9036) 2025-10-28 16:36:20 +01:00
Pepe Fagoaga 42be77e82e fix(backport): Run ir PR is closed and labeled (#9047) 2025-10-28 19:21:29 +05:45
Daniel Barranquero 63169289b0 fix(ec2): AttributeError in ec2_instance_with_outdated_ami check (#9046) 2025-10-28 09:13:44 -04:00
lydiavilchezandDaniel Barranquero 43d310356d feat(gcp): add cloudstorage_bucket_versioning_enabled check (#9014)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-28 13:20:59 +01:00
Pedro Martín 59ae503681 fix(compliance): handle timestamp when transforming CCC findings (#9042) 2025-10-28 12:45:04 +01:00
Rubén De la Torre VicoandDaniel Barranquero bd62f56df4 chore(aws): enhance metadata for dynamodb service (#8871)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-28 12:08:01 +01:00
Alejandro Bailo 90fbad16b9 feat: add risk severity chart to new overview page (#9041) 2025-10-28 12:07:19 +01:00
Alan BuscagliaandAlejandro Bailo affd0c5ffb chore: upgrade React to 19.2.0 and eslint-plugin-react-hooks to 7.0.1 (#9039)
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
2025-10-28 11:50:07 +01:00
StylusFrost 929bbe3550 test(ui): add AWS provider management E2E tests (#8948) 2025-10-28 11:49:41 +01:00
Andoni Alonso eb7ef4a8b9 chore(github): update dev guide docs link (#9044) 2025-10-28 11:45:30 +01:00
Rubén De la Torre VicoandDaniel Barranquero 017e19ac18 chore(aws): enhance metadata for drs service (#8870)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-10-28 10:23:47 +01:00
Alejandro BailoandAlan Buscaglia be7680786a feat: new overview filters (#9013)
Co-authored-by: Alan Buscaglia <gentlemanprogramming@gmail.com>
2025-10-28 08:44:46 +01:00
SeongYong ChoiandMrCloudSec efba5d2a8d feat(codepipeline): add new check codepipeline_project_repo_private (#5915)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-10-27 18:55:36 -04:00
Alan Buscaglia 44431a56de feat(api-keys): add read docs api key (#8947) 2025-10-27 18:06:44 +01:00
1156 changed files with 97182 additions and 17834 deletions

No files matched your search

+37 -2
View File
@@ -10,13 +10,16 @@ NEXT_PUBLIC_API_BASE_URL=${API_BASE_URL}
NEXT_PUBLIC_API_DOCS_URL=http://prowler-api:8080/api/v1/docs
AUTH_TRUST_HOST=true
UI_PORT=3000
# Temp URL for feeds need to use actual
RSS_FEED_URL=https://prowler.com/blog/rss
# openssl rand -base64 32
AUTH_SECRET="N/c6mnaS5+SWq81+819OrzQZlmx1Vxtp/orjttJSmw8="
# Google Tag Manager ID
NEXT_PUBLIC_GOOGLE_TAG_MANAGER_ID=""
#### Code Review Configuration ####
# Enable Claude Code standards validation on pre-push hook
# Set to 'true' to validate changes against AGENTS.md standards via Claude Code
# Set to 'false' to skip validation
CODE_REVIEW_ENABLED=true
#### Prowler API Configuration ####
PROWLER_API_VERSION="stable"
@@ -35,6 +38,28 @@ POSTGRES_DB=prowler_db
# POSTGRES_REPLICA_USER=prowler
# POSTGRES_REPLICA_PASSWORD=postgres
# POSTGRES_REPLICA_DB=prowler_db
# POSTGRES_REPLICA_MAX_ATTEMPTS=3
# POSTGRES_REPLICA_RETRY_BASE_DELAY=0.5
# Neo4j auth
NEO4J_HOST=neo4j
NEO4J_PORT=7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=neo4j_password
# Neo4j settings
NEO4J_DBMS_MAX__DATABASES=1000000
NEO4J_SERVER_MEMORY_PAGECACHE_SIZE=1G
NEO4J_SERVER_MEMORY_HEAP_INITIAL__SIZE=1G
NEO4J_SERVER_MEMORY_HEAP_MAX__SIZE=1G
NEO4J_POC_EXPORT_FILE_ENABLED=true
NEO4J_APOC_IMPORT_FILE_ENABLED=true
NEO4J_APOC_IMPORT_FILE_USE_NEO4J_CONFIG=true
NEO4J_PLUGINS=["apoc"]
NEO4J_DBMS_SECURITY_PROCEDURES_ALLOWLIST=apoc.*
NEO4J_DBMS_SECURITY_PROCEDURES_UNRESTRICTED=apoc.*
NEO4J_DBMS_CONNECTOR_BOLT_LISTEN_ADDRESS=0.0.0.0:7687
# Neo4j Prowler settings
NEO4J_INSERT_BATCH_SIZE=500
# Celery-Prowler task settings
TASK_RETRY_DELAY_SECONDS=0.1
@@ -103,6 +128,7 @@ DJANGO_THROTTLE_TOKEN_OBTAIN=50/minute
# Sentry settings
SENTRY_ENVIRONMENT=local
SENTRY_RELEASE=local
NEXT_PUBLIC_SENTRY_ENVIRONMENT=${SENTRY_ENVIRONMENT}
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.12.2
@@ -124,3 +150,12 @@ LANGSMITH_TRACING=false
LANGSMITH_ENDPOINT="https://api.smith.langchain.com"
LANGSMITH_API_KEY=""
LANGCHAIN_PROJECT=""
# RSS Feed Configuration
# Multiple feed sources can be configured as a JSON array (must be valid JSON, no trailing commas)
# Each source requires: id, name, type (github_releases|blog|custom), url, and enabled flag
# IMPORTANT: Must be a single line with valid JSON (no newlines, no trailing commas)
# Example with one source:
RSS_FEED_SOURCES='[{"id":"prowler-releases","name":"Prowler Releases","type":"github_releases","url":"https://github.com/prowler-cloud/prowler/releases.atom","enabled":true}]'
# Example with multiple sources (no trailing comma after last item):
# RSS_FEED_SOURCES='[{"id":"prowler-releases","name":"Prowler Releases","type":"github_releases","url":"https://github.com/prowler-cloud/prowler/releases.atom","enabled":true},{"id":"prowler-blog","name":"Prowler Blog","type":"blog","url":"https://prowler.com/blog/rss","enabled":false}]'
+27 -5
View File
@@ -22,8 +22,8 @@ inputs:
runs:
using: 'composite'
steps:
- name: Replace @master with current branch in pyproject.toml
if: github.event_name == 'pull_request' && github.base_ref == 'master'
- name: Replace @master with current branch in pyproject.toml (prowler repo only)
if: github.event_name == 'pull_request' && github.base_ref == 'master' && github.repository == 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
@@ -37,8 +37,8 @@ runs:
python -m pip install --upgrade pip
pipx install poetry==${{ inputs.poetry-version }}
- name: Update SDK resolved_reference to latest commit
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
- name: Update poetry.lock with latest Prowler commit
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
@@ -50,7 +50,21 @@ runs:
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock
- name: Update SDK resolved_reference to latest commit (prowler repo on push)
if: github.event_name == 'push' && github.ref == 'refs/heads/master' && github.repository == 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
echo "Latest commit hash: $LATEST_COMMIT"
sed -i '/url = "https:\/\/github\.com\/prowler-cloud\/prowler\.git"/,/resolved_reference = / {
s/resolved_reference = "[a-f0-9]\{40\}"/resolved_reference = "'"$LATEST_COMMIT"'"/
}' poetry.lock
echo "Updated resolved_reference:"
grep -A2 -B2 "resolved_reference" poetry.lock
- name: Update poetry.lock (prowler repo only)
if: github.repository == 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: poetry lock
@@ -69,3 +83,11 @@ runs:
run: |
poetry install --no-root
poetry run pip list
- name: Update Prowler Cloud API Client
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
poetry remove prowler-cloud-api-client
poetry add ./prowler-cloud-api-client
@@ -0,0 +1,198 @@
# Slack Notification Action
A generic and flexible GitHub composite action for sending Slack notifications using JSON template files. Supports both standalone messages and message updates, with automatic status detection.
## Features
- **Template-based**: All messages use JSON template files for consistency
- **Automatic status detection**: Pass `step-outcome` to auto-calculate success/failure
- **Message updates**: Supports updating existing messages (using `chat.update`)
- **Simple API**: Clean and minimal interface
- **Reusable**: Use across all workflows and scenarios
- **Maintainable**: Centralized message templates
## Use Cases
1. **Container releases**: Track push start and completion with automatic status
2. **Deployments**: Track deployment progress with rich Block Kit formatting
3. **Custom notifications**: Any scenario where you need to notify Slack
## Inputs
| Input | Description | Required | Default |
|-------|-------------|----------|---------|
| `slack-bot-token` | Slack bot token for authentication | Yes | - |
| `payload-file-path` | Path to JSON file with the Slack message payload | Yes | - |
| `update-ts` | Message timestamp to update (leave empty for new messages) | No | `''` |
| `step-outcome` | Step outcome for automatic status detection (sets STATUS_EMOJI and STATUS_TEXT env vars) | No | `''` |
## Outputs
| Output | Description |
|--------|-------------|
| `ts` | Timestamp of the Slack message (use for updates) |
## Usage Examples
### Example 1: Container Release with Automatic Status Detection
Using JSON template files with automatic status detection:
```yaml
# Send start notification
- name: Notify container push started
if: github.event_name == 'release'
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
# Do the work
- name: Build and push container
if: github.event_name == 'release'
id: container-push
uses: docker/build-push-action@...
with:
push: true
tags: ...
# Send completion notification with automatic status detection
- name: Notify container push completed
if: github.event_name == 'release' && always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.container-push.outcome }}
```
**Benefits:**
- No status calculation needed in workflow
- Reusable template files
- Clean and concise
- Automatic `STATUS_EMOJI` and `STATUS_TEXT` env vars set by action
- Consistent message format across all workflows
### Example 2: Deployment with Message Update Pattern
```yaml
# Send initial deployment message
- name: Notify deployment started
id: slack-start
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
ENVIRONMENT: PRODUCTION
COMMIT_HASH: ${{ github.sha }}
VERSION_DEPLOYED: latest
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_WORKFLOW: ${{ github.workflow }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/deployment-started.json"
# Run deployment
- name: Deploy
id: deploy
run: terraform apply -auto-approve
# Determine additional status variables
- name: Determine deployment status
if: always()
id: deploy-status
run: |
if [[ "${{ steps.deploy.outcome }}" == "success" ]]; then
echo "STATUS_COLOR=28a745" >> $GITHUB_ENV
echo "STATUS=Completed" >> $GITHUB_ENV
else
echo "STATUS_COLOR=fc3434" >> $GITHUB_ENV
echo "STATUS=Failed" >> $GITHUB_ENV
fi
# Update the same message with final status
- name: Update deployment notification
if: always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
MESSAGE_TS: ${{ steps.slack-start.outputs.ts }}
COMPONENT: API
ENVIRONMENT: PRODUCTION
COMMIT_HASH: ${{ github.sha }}
VERSION_DEPLOYED: latest
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_WORKFLOW: ${{ github.workflow }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
STATUS: ${{ env.STATUS }}
STATUS_COLOR: ${{ env.STATUS_COLOR }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
update-ts: ${{ steps.slack-start.outputs.ts }}
payload-file-path: "./.github/scripts/slack-messages/deployment-completed.json"
step-outcome: ${{ steps.deploy.outcome }}
```
## Automatic Status Detection
When you provide `step-outcome` input, the action automatically sets these environment variables:
| Outcome | STATUS_EMOJI | STATUS_TEXT |
|---------|--------------|-------------|
| success | `[✓]` | `completed successfully!` |
| failure | `[✗]` | `failed` |
These variables are then available in your payload template files.
## Template File Format
All template files must be valid JSON and support environment variable substitution. Example:
```json
{
"channel": "$SLACK_CHANNEL_ID",
"text": "$STATUS_EMOJI $COMPONENT container release $RELEASE_TAG push $STATUS_TEXT <$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID|View run>"
}
```
See available templates in [`.github/scripts/slack-messages/`](../../scripts/slack-messages/).
## Requirements
- Slack Bot Token with scopes: `chat:write`, `chat:write.public`
- Slack Channel ID where messages will be posted
- JSON template files for your messages
## Benefits
- **Consistency**: All notifications use standardized templates
- **Automatic status handling**: No need to calculate success/failure in workflows
- **Clean workflows**: Minimal boilerplate code
- **Reusable templates**: One template for all components
- **Easy to maintain**: Change template once, applies everywhere
- **Version controlled**: All message formats in git
## Related Resources
- [Slack Block Kit Builder](https://app.slack.com/block-kit-builder)
- [Slack API Method Documentation](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
- [Message templates documentation](../../scripts/slack-messages/README.md)
@@ -0,0 +1,74 @@
name: 'Slack Notification'
description: 'Generic action to send Slack notifications with optional message updates and automatic status detection'
inputs:
slack-bot-token:
description: 'Slack bot token for authentication'
required: true
payload-file-path:
description: 'Path to JSON file with the Slack message payload'
required: true
update-ts:
description: 'Message timestamp to update (only for updates, leave empty for new messages)'
required: false
default: ''
step-outcome:
description: 'Outcome of a step to determine status (success/failure) - automatically sets STATUS_TEXT and STATUS_COLOR env vars'
required: false
default: ''
outputs:
ts:
description: 'Timestamp of the Slack message'
value: ${{ steps.slack-notification.outputs.ts }}
runs:
using: 'composite'
steps:
- name: Determine status
id: status
shell: bash
run: |
if [[ "${{ inputs.step-outcome }}" == "success" ]]; then
echo "STATUS_TEXT=Completed" >> $GITHUB_ENV
echo "STATUS_COLOR=#6aa84f" >> $GITHUB_ENV
elif [[ "${{ inputs.step-outcome }}" == "failure" ]]; then
echo "STATUS_TEXT=Failed" >> $GITHUB_ENV
echo "STATUS_COLOR=#fc3434" >> $GITHUB_ENV
else
# No outcome provided - pending/in progress state
echo "STATUS_COLOR=#dbab09" >> $GITHUB_ENV
fi
- name: Send Slack notification (new message)
if: inputs.update-ts == ''
id: slack-notification-post
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_PAYLOAD_FILE_PATH: ${{ inputs.payload-file-path }}
with:
method: chat.postMessage
token: ${{ inputs.slack-bot-token }}
payload-file-path: ${{ inputs.payload-file-path }}
payload-templated: true
errors: true
- name: Update Slack notification
if: inputs.update-ts != ''
id: slack-notification-update
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_PAYLOAD_FILE_PATH: ${{ inputs.payload-file-path }}
with:
method: chat.update
token: ${{ inputs.slack-bot-token }}
payload-file-path: ${{ inputs.payload-file-path }}
payload-templated: true
errors: true
- name: Set output
id: slack-notification
shell: bash
run: |
if [[ "${{ inputs.update-ts }}" == "" ]]; then
echo "ts=${{ steps.slack-notification-post.outputs.ts }}" >> $GITHUB_OUTPUT
else
echo "ts=${{ inputs.update-ts }}" >> $GITHUB_OUTPUT
fi
+31 -19
View File
@@ -45,22 +45,13 @@ outputs:
runs:
using: 'composite'
steps:
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true'
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
- name: Cache Trivy vulnerability database
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '0'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true'
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
sarif_file: 'trivy-results.sarif'
category: 'trivy-container'
path: ~/.cache/trivy
key: trivy-db-${{ runner.os }}-${{ github.run_id }}
restore-keys: |
trivy-db-${{ runner.os }}-
- name: Run Trivy vulnerability scan (JSON)
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
@@ -70,6 +61,27 @@ runs:
output: 'trivy-report.json'
severity: ${{ inputs.severity }}
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # v0.33.1
with:
image-ref: ${{ inputs.image-name }}:${{ inputs.image-tag }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
uses: github/codeql-action/upload-sarif@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
with:
sarif_file: 'trivy-results.sarif'
category: 'trivy-container'
- name: Upload Trivy report artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
@@ -109,20 +121,20 @@ runs:
with:
script: |
const comment = require('./.github/scripts/trivy-pr-comment.js');
// Unique identifier to find our comment
const marker = '<!-- trivy-scan-comment:${{ inputs.image-name }} -->';
const body = marker + '\n' + comment;
// Find existing comment
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
});
const existingComment = comments.find(c => c.body?.includes(marker));
if (existingComment) {
// Update existing comment
await github.rest.issues.updateComment({
+7
View File
@@ -22,6 +22,13 @@ Please add a detailed description of how to review this PR.
- [ ] Review if is needed to change the [Readme.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/prowler/CHANGELOG.md), if applicable.
#### UI
- [ ] All issue/task requirements work as expected on the UI
- [ ] Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/ui/CHANGELOG.md), if applicable.
#### API
- [ ] Verify if API specs need to be regenerated.
- [ ] Check if version updates are required (e.g., specs, Poetry, etc.).
+462
View File
@@ -0,0 +1,462 @@
# Slack Message Templates
This directory contains reusable message templates for Slack notifications sent from GitHub Actions workflows.
## Usage
These JSON templates are used with the `slackapi/slack-github-action` using the Slack API method (`chat.postMessage` and `chat.update`). All templates support rich Block Kit formatting and message updates.
### Available Templates
**Container Releases**
- `container-release-started.json`: Simple one-line notification when container push starts
- `container-release-completed.json`: Simple one-line notification when container release completes
**Deployments**
- `deployment-started.json`: Deployment start notification with Block Kit formatting
- `deployment-completed.json`: Deployment completion notification (updates the start message)
All templates use the Slack API method and require a Slack Bot Token.
## Setup Requirements
1. Create a Slack App (or use existing)
2. Add Bot Token Scopes: `chat:write`, `chat:write.public`
3. Install the app to your workspace
4. Get the Bot Token from OAuth & Permissions page
5. Add secrets:
- `SLACK_BOT_TOKEN`: Your bot token
- `SLACK_CHANNEL_ID`: The channel ID where messages will be posted
Reference: [Sending data using a Slack API method](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
## Environment Variables
### Required Secrets (GitHub Secrets)
- `SLACK_BOT_TOKEN`: Passed as `token` parameter to the action (not as env variable)
- `SLACK_CHANNEL_ID`: Used in payload as env variable
### Container Release Variables (configured as env)
- `COMPONENT`: Component name (e.g., "API", "SDK", "UI", "MCP")
- `RELEASE_TAG` / `PROWLER_VERSION`: The release tag or version being deployed
- `GITHUB_SERVER_URL`: Provided by GitHub context
- `GITHUB_REPOSITORY`: Provided by GitHub context
- `GITHUB_RUN_ID`: Provided by GitHub context
- `STATUS_EMOJI`: Status symbol (calculated: `[✓]` for success, `[✗]` for failure)
- `STATUS_TEXT`: Status text (calculated: "completed successfully!" or "failed")
### Deployment Variables (configured as env)
- `COMPONENT`: Component name (e.g., "API", "SDK", "UI", "MCP")
- `ENVIRONMENT`: Environment name (e.g., "DEVELOPMENT", "PRODUCTION")
- `COMMIT_HASH`: Commit hash being deployed
- `VERSION_DEPLOYED`: Version being deployed
- `GITHUB_ACTOR`: User who triggered the workflow
- `GITHUB_WORKFLOW`: Workflow name
- `GITHUB_SERVER_URL`: Provided by GitHub context
- `GITHUB_REPOSITORY`: Provided by GitHub context
- `GITHUB_RUN_ID`: Provided by GitHub context
All other variables (MESSAGE_TS, STATUS, STATUS_COLOR, STATUS_EMOJI, etc.) are calculated internally within the workflow and should NOT be configured as environment variables.
## Example Workflow Usage
### Using the Generic Slack Notification Action (Recommended)
**Recommended approach**: Use the generic reusable action `.github/actions/slack-notification` which provides maximum flexibility:
#### Example 1: Container Release (Start + Completion)
```yaml
# Send start notification
- name: Notify container push started
if: github.event_name == 'release'
uses: ./.github/actions/slack-notification
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload: |
{
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
"text": "API container release ${{ env.RELEASE_TAG }} push started... <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
}
# Build and push container
- name: Build and push container
if: github.event_name == 'release'
id: container-push
uses: docker/build-push-action@...
with:
push: true
tags: ...
# Calculate status
- name: Determine push status
if: github.event_name == 'release' && always()
id: push-status
run: |
if [[ "${{ steps.container-push.outcome }}" == "success" ]]; then
echo "emoji=[✓]" >> $GITHUB_OUTPUT
echo "text=completed successfully!" >> $GITHUB_OUTPUT
else
echo "emoji=[✗]" >> $GITHUB_OUTPUT
echo "text=failed" >> $GITHUB_OUTPUT
fi
# Send completion notification
- name: Notify container push completed
if: github.event_name == 'release' && always()
uses: ./.github/actions/slack-notification
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload: |
{
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
"text": "${{ steps.push-status.outputs.emoji }} API container release ${{ env.RELEASE_TAG }} push ${{ steps.push-status.outputs.text }} <${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}|View run>"
}
```
#### Example 2: Simple One-Time Message
```yaml
- name: Send notification
uses: ./.github/actions/slack-notification
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload: |
{
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
"text": "Deployment completed successfully!"
}
```
#### Example 3: Deployment with Message Update Pattern
```yaml
# Send initial deployment message
- name: Notify deployment started
id: slack-start
uses: ./.github/actions/slack-notification
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload: |
{
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
"text": "API deployment to PRODUCTION started",
"attachments": [
{
"color": "dbab09",
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "API | Deployment to PRODUCTION"
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Status:*\nIn Progress"
}
]
}
]
}
]
}
# Run deployment
- name: Deploy
id: deploy
run: terraform apply -auto-approve
# Calculate status
- name: Determine status
if: always()
id: status
run: |
if [[ "${{ steps.deploy.outcome }}" == "success" ]]; then
echo "color=28a745" >> $GITHUB_OUTPUT
echo "emoji=[✓]" >> $GITHUB_OUTPUT
echo "status=Completed" >> $GITHUB_OUTPUT
else
echo "color=fc3434" >> $GITHUB_OUTPUT
echo "emoji=[✗]" >> $GITHUB_OUTPUT
echo "status=Failed" >> $GITHUB_OUTPUT
fi
# Update the same message with final status
- name: Update deployment notification
if: always()
uses: ./.github/actions/slack-notification
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
update-ts: ${{ steps.slack-start.outputs.ts }}
payload: |
{
"channel": "${{ secrets.SLACK_CHANNEL_ID }}",
"ts": "${{ steps.slack-start.outputs.ts }}",
"text": "${{ steps.status.outputs.emoji }} API deployment to PRODUCTION ${{ steps.status.outputs.status }}",
"attachments": [
{
"color": "${{ steps.status.outputs.color }}",
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": "API | Deployment to PRODUCTION"
}
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Status:*\n${{ steps.status.outputs.emoji }} ${{ steps.status.outputs.status }}"
}
]
}
]
}
]
}
```
**Benefits of using the generic action:**
- Maximum flexibility: Build any payload you need directly in the workflow
- No template files needed: Everything inline
- Supports all scenarios: one-time messages, start/update patterns, rich Block Kit
- Easy to customize per use case
- Generic: Works for containers, deployments, or any notification type
For more details, see [Slack Notification Action](../../actions/slack-notification/README.md).
### Using Message Templates (Alternative Approach)
Simple one-line notifications for container releases:
```yaml
# Step 1: Notify when push starts
- name: Notify container push started
if: github.event_name == 'release'
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
# Step 2: Build and push container
- name: Build and push container
id: container-push
uses: docker/build-push-action@...
with:
push: true
tags: ...
# Step 3: Determine push status
- name: Determine push status
if: github.event_name == 'release' && always()
id: push-status
run: |
if [[ "${{ steps.container-push.outcome }}" == "success" ]]; then
echo "status-emoji=[✓]" >> $GITHUB_OUTPUT
echo "status-text=completed successfully!" >> $GITHUB_OUTPUT
else
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
echo "status-text=failed" >> $GITHUB_OUTPUT
fi
# Step 4: Notify when push completes (success or failure)
- name: Notify container push completed
if: github.event_name == 'release' && always()
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
STATUS_EMOJI: ${{ steps.push-status.outputs.status-emoji }}
STATUS_TEXT: ${{ steps.push-status.outputs.status-text }}
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
```
### Deployment with Update Pattern
For deployments that start with one message and update it with the final status:
```yaml
# Step 1: Send deployment start notification
- name: Notify Deployment Start
id: slack-notification-start
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
COMPONENT: API
ENVIRONMENT: PRODUCTION
COMMIT_HASH: ${{ github.sha }}
VERSION_DEPLOYED: latest
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_WORKFLOW: ${{ github.workflow }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
method: chat.postMessage
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/deployment-started.json"
# Step 2: Run your deployment steps
- name: Terraform Plan
id: terraform-plan
run: terraform plan
- name: Terraform Apply
id: terraform-apply
run: terraform apply -auto-approve
# Step 3: Determine status (calculated internally, not configured)
- name: Determine Status
if: always()
id: determine-status
run: |
if [[ "${{ steps.terraform-apply.outcome }}" == "success" ]]; then
echo "status=Completed" >> $GITHUB_OUTPUT
echo "status-color=28a745" >> $GITHUB_OUTPUT
echo "status-emoji=[✓]" >> $GITHUB_OUTPUT
echo "plan-emoji=[✓]" >> $GITHUB_OUTPUT
echo "apply-emoji=[✓]" >> $GITHUB_OUTPUT
elif [[ "${{ steps.terraform-plan.outcome }}" == "failure" || "${{ steps.terraform-apply.outcome }}" == "failure" ]]; then
echo "status=Failed" >> $GITHUB_OUTPUT
echo "status-color=fc3434" >> $GITHUB_OUTPUT
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
if [[ "${{ steps.terraform-plan.outcome }}" == "failure" ]]; then
echo "plan-emoji=[✗]" >> $GITHUB_OUTPUT
else
echo "plan-emoji=[✓]" >> $GITHUB_OUTPUT
fi
if [[ "${{ steps.terraform-apply.outcome }}" == "failure" ]]; then
echo "apply-emoji=[✗]" >> $GITHUB_OUTPUT
else
echo "apply-emoji=[✓]" >> $GITHUB_OUTPUT
fi
else
echo "status=Failed" >> $GITHUB_OUTPUT
echo "status-color=fc3434" >> $GITHUB_OUTPUT
echo "status-emoji=[✗]" >> $GITHUB_OUTPUT
echo "plan-emoji=[?]" >> $GITHUB_OUTPUT
echo "apply-emoji=[?]" >> $GITHUB_OUTPUT
fi
# Step 4: Update the same Slack message (using calculated values)
- name: Notify Deployment Result
if: always()
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_CHANNEL_ID }}
MESSAGE_TS: ${{ steps.slack-notification-start.outputs.ts }}
COMPONENT: API
ENVIRONMENT: PRODUCTION
COMMIT_HASH: ${{ github.sha }}
VERSION_DEPLOYED: latest
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_WORKFLOW: ${{ github.workflow }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
STATUS: ${{ steps.determine-status.outputs.status }}
STATUS_COLOR: ${{ steps.determine-status.outputs.status-color }}
STATUS_EMOJI: ${{ steps.determine-status.outputs.status-emoji }}
PLAN_EMOJI: ${{ steps.determine-status.outputs.plan-emoji }}
APPLY_EMOJI: ${{ steps.determine-status.outputs.apply-emoji }}
TERRAFORM_PLAN_OUTCOME: ${{ steps.terraform-plan.outcome }}
TERRAFORM_APPLY_OUTCOME: ${{ steps.terraform-apply.outcome }}
with:
method: chat.update
token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/deployment-completed.json"
```
**Note**: Variables like `STATUS`, `STATUS_COLOR`, `STATUS_EMOJI`, `PLAN_EMOJI`, `APPLY_EMOJI` are calculated by the `determine-status` step based on the outcomes of previous steps. They should NOT be manually configured.
## Key Features
### Benefits of Using Slack API Method
- **Rich Block Kit Formatting**: Full support for Slack's Block Kit including headers, sections, fields, colors, and attachments
- **Message Updates**: Update the same message instead of posting multiple messages (using `chat.update` with `ts`)
- **Consistent Experience**: Same look and feel as Prowler Cloud notifications
- **Flexible**: Easy to customize message appearance by editing JSON templates
### Differences from Webhook Method
| Feature | webhook-trigger | Slack API (chat.postMessage) |
|---------|-----------------|------------------------------|
| Setup | Workflow Builder webhook | Slack Bot Token + Channel ID |
| Formatting | Plain text/simple | Full Block Kit support |
| Message Update | No | Yes (with chat.update) |
| Authentication | Webhook URL | Bot Token |
| Scopes Required | None | chat:write, chat:write.public |
## Message Appearance
### Container Release (Simple One-Line)
**Start message:**
```
API container release 4.5.0 push started... View run
```
**Completion message (success):**
```
[✓] API container release 4.5.0 push completed successfully! View run
```
**Completion message (failure):**
```
[✗] API container release 4.5.0 push failed View run
```
All messages are simple one-liners with a clickable "View run" link. The completion message adapts to show success `[✓]` or failure `[✗]` based on the outcome of the container push.
### Deployment Start
- Header: Component and environment
- Yellow bar (color: `dbab09`)
- Status: In Progress
- Details: Commit, version, actor, workflow
- Link: Direct link to deployment run
### Deployment Completion
- Header: Component and environment
- Green bar for success (color: `28a745`) / Red bar for failure (color: `fc3434`)
- Status: [✓] Completed or [✗] Failed
- Details: All deployment info plus terraform outcomes
- Link: Direct link to deployment run
## Adding New Templates
1. Create a new JSON file with Block Kit structure
2. Use environment variable placeholders (e.g., `$VAR_NAME`)
3. Include `channel` and `text` fields (required)
4. Add `blocks` or `attachments` for rich formatting
5. For update templates, include `ts` field as `$MESSAGE_TS`
6. Document the template in this README
7. Reference it in your workflow using `payload-file-path`
## Reference
- [Slack Block Kit Builder](https://app.slack.com/block-kit-builder)
- [Slack API Method Documentation](https://docs.slack.dev/tools/slack-github-action/sending-techniques/sending-data-slack-api-method/)
@@ -0,0 +1,17 @@
{
"channel": "${{ env.SLACK_CHANNEL_ID }}",
"attachments": [
{
"color": "${{ env.STATUS_COLOR }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Status:*\n${{ env.STATUS_TEXT }}\n\n${{ env.COMPONENT }} container release ${{ env.RELEASE_TAG }} push ${{ env.STATUS_TEXT }}\n\n<${{ env.GITHUB_SERVER_URL }}/${{ env.GITHUB_REPOSITORY }}/actions/runs/${{ env.GITHUB_RUN_ID }}|View run>"
}
}
]
}
]
}
@@ -0,0 +1,17 @@
{
"channel": "${{ env.SLACK_CHANNEL_ID }}",
"attachments": [
{
"color": "${{ env.STATUS_COLOR }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Status:*\nStarted\n\n${{ env.COMPONENT }} container release ${{ env.RELEASE_TAG }} push started...\n\n<${{ env.GITHUB_SERVER_URL }}/${{ env.GITHUB_REPOSITORY }}/actions/runs/${{ env.GITHUB_RUN_ID }}|View run>"
}
}
]
}
]
}
@@ -1,115 +0,0 @@
name: API - Build and Push containers
on:
push:
branches:
- "master"
paths:
- "api/**"
- "prowler/**"
- ".github/workflows/api-build-lint-push-containers.yml"
# Uncomment the code below to test this action on PRs
# pull_request:
# branches:
# - "master"
# paths:
# - "api/**"
# - ".github/workflows/api-build-lint-push-containers.yml"
release:
types: [published]
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./api
# Container Registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-api
jobs:
repository-check:
name: Repository check
runs-on: ubuntu-latest
outputs:
is_repo: ${{ steps.repository_check.outputs.is_repo }}
steps:
- name: Repository check
id: repository_check
working-directory: /tmp
run: |
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
then
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
else
echo "This action only runs for prowler-cloud/prowler"
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
fi
# Build Prowler OSS container
container-build-push:
needs: repository-check
if: needs.repository-check.outputs.is_repo == 'true'
runs-on: ubuntu-latest
defaults:
run:
working-directory: ${{ env.WORKING_DIRECTORY }}
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set short git commit SHA
id: vars
run: |
shortSha=$(git rev-parse --short ${{ github.sha }})
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push container image (latest)
# Comment the following line for testing
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
# Set push: false for testing
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.SHORT_SHA }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Trigger deployment
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: prowler-api-deploy
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ env.SHORT_SHA }}"}'
+71
View File
@@ -0,0 +1,71 @@
name: 'API: Code Quality'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
API_WORKING_DIR: ./api
jobs:
api-code-quality:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for API changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
.github/workflows/api-code-quality.yml
files_ignore: |
api/docs/**
api/README.md
api/CHANGELOG.md
- name: Setup Python with Poetry
if: steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Poetry check
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry check --lock
- name: Ruff lint
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run ruff check . --exclude contrib
- name: Ruff format
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run ruff format --check . --exclude contrib
- name: Pylint
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
+3 -3
View File
@@ -25,7 +25,7 @@ concurrency:
cancel-in-progress: true
jobs:
analyze:
api-analyze:
name: CodeQL Security Analysis
runs-on: ubuntu-latest
timeout-minutes: 30
@@ -45,12 +45,12 @@ jobs:
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/init@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/analyze@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
category: '/language:${{ matrix.language }}'
@@ -0,0 +1,184 @@
name: 'API: Container Build and Push'
on:
push:
branches:
- 'attack-paths-demo'
paths:
- 'api/**'
- 'prowler/**'
- '.github/workflows/api-container-build-push.yml'
release:
types:
- 'published'
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag (e.g., 5.14.0)'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
env:
# Tags
LATEST_TAG: attack-paths-demo
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.release_tag }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./api
# Container registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-api
jobs:
setup:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
steps:
- name: Calculate short SHA
id: set-short-sha
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
container-build-push:
needs: setup
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Notify container push started
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
- name: Build and push API container for ${{ matrix.arch }}
id: container-push
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
platforms: ${{ matrix.platform }}
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-${{ matrix.arch }}
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
- name: Notify container push completed
if: (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: API
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.container-push.outcome }}
# Create and push multi-architecture manifest
create-manifest:
needs: [setup, container-build-push]
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Create and push manifests for push event
if: github.event_name == 'push'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@f61d18f46c86af724a9c804cb9ff2a6fec741c7c # main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
echo "Cleanup completed"
trigger-deployment:
if: github.event_name == 'push'
needs: [setup, container-build-push]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Trigger API deployment
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: api-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
@@ -0,0 +1,89 @@
name: 'API: Container Checks'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
API_WORKING_DIR: ./api
IMAGE_NAME: prowler-api
jobs:
api-dockerfile-lint:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check if Dockerfile changed
id: dockerfile-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: api/Dockerfile
- name: Lint Dockerfile with Hadolint
if: steps.dockerfile-changed.outputs.any_changed == 'true'
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
dockerfile: api/Dockerfile
ignore: DL3013
api-container-build-and-scan:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for API changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: api/**
files_ignore: |
api/docs/**
api/README.md
api/CHANGELOG.md
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build container
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.API_WORKING_DIR }}
push: false
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan container with Trivy
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
-228
View File
@@ -1,228 +0,0 @@
name: 'API: Pull Request'
on:
push:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
pull_request:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/workflows/api-pull-request.yml'
- 'api/**'
- '!api/docs/**'
- '!api/README.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432
POSTGRES_ADMIN_USER: prowler
POSTGRES_ADMIN_PASSWORD: S3cret
POSTGRES_USER: prowler_user
POSTGRES_PASSWORD: prowler
POSTGRES_DB: postgres-db
VALKEY_HOST: localhost
VALKEY_PORT: 6379
VALKEY_DB: 0
API_WORKING_DIR: ./api
IMAGE_NAME: prowler-api
jobs:
code-quality:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Poetry check
run: poetry check --lock
- name: Ruff lint
run: poetry run ruff check . --exclude contrib
- name: Ruff format
run: poetry run ruff format --check . --exclude contrib
- name: Pylint
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn src/
security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Bandit
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
tests:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
services:
postgres:
image: postgres
env:
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
POSTGRES_USER: ${{ env.POSTGRES_USER }}
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_DB: ${{ env.POSTGRES_DB }}
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
valkey:
image: valkey/valkey:7-alpine3.19
env:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
VALKEY_DB: ${{ env.VALKEY_DB }}
ports:
- 6379:6379
options: >-
--health-cmd "valkey-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python with Poetry
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Run tests with pytest
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: api
dockerfile-lint:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Lint Dockerfile with Hadolint
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
dockerfile: api/Dockerfile
ignore: DL3013
container-build-and-scan:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build container
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.API_WORKING_DIR }}
push: false
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan container with Trivy
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
+69
View File
@@ -0,0 +1,69 @@
name: 'API: Security'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
API_WORKING_DIR: ./api
jobs:
api-security-scans:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for API changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
.github/workflows/api-security.yml
files_ignore: |
api/docs/**
api/README.md
api/CHANGELOG.md
- name: Setup Python with Poetry
if: steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Bandit
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run bandit -q -lll -x '*_test.py,./contrib/' -r .
- name: Safety
if: steps.check-changes.outputs.any_changed == 'true'
# 76352, 76353, 77323 come from SDK, but they cannot upgrade it yet. It does not affect API
# TODO: Botocore needs urllib3 1.X so we need to ignore these vulnerabilities 77744,77745. Remove this once we upgrade to urllib3 2.X
run: poetry run safety check --ignore 70612,66963,74429,76352,76353,77323,77744,77745
- name: Vulture
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run vulture --exclude "contrib,tests,conftest.py" --min-confidence 100 .
+107
View File
@@ -0,0 +1,107 @@
name: 'API: Tests'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
POSTGRES_HOST: localhost
POSTGRES_PORT: 5432
POSTGRES_ADMIN_USER: prowler
POSTGRES_ADMIN_PASSWORD: S3cret
POSTGRES_USER: prowler_user
POSTGRES_PASSWORD: prowler
POSTGRES_DB: postgres-db
VALKEY_HOST: localhost
VALKEY_PORT: 6379
VALKEY_DB: 0
API_WORKING_DIR: ./api
jobs:
api-tests:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.12'
defaults:
run:
working-directory: ./api
services:
postgres:
image: postgres
env:
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
POSTGRES_USER: ${{ env.POSTGRES_USER }}
POSTGRES_PASSWORD: ${{ env.POSTGRES_PASSWORD }}
POSTGRES_DB: ${{ env.POSTGRES_DB }}
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
valkey:
image: valkey/valkey:7-alpine3.19
env:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
VALKEY_DB: ${{ env.VALKEY_DB }}
ports:
- 6379:6379
options: >-
--health-cmd "valkey-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for API changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
api/**
.github/workflows/api-tests.yml
files_ignore: |
api/docs/**
api/README.md
api/CHANGELOG.md
- name: Setup Python with Poetry
if: steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/setup-python-poetry
with:
python-version: ${{ matrix.python-version }}
working-directory: ./api
- name: Run tests with pytest
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run pytest --cov=./src/backend --cov-report=xml src/backend
- name: Upload coverage reports to Codecov
if: steps.check-changes.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: api
+1 -3
View File
@@ -7,8 +7,6 @@ on:
types:
- 'labeled'
- 'closed'
paths:
- '.github/workflows/backport.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
@@ -40,7 +38,7 @@ jobs:
- name: Backport PR
if: steps.label_check.outputs.label_check == 'success'
uses: sorenlouv/backport-github-action@ad888e978060bc1b2798690dd9d03c4036560947 # v9.5.1
uses: sorenlouv/backport-github-action@516854e7c9f962b9939085c9a92ea28411d1ae90 # v10.2.0
with:
github_token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
auto_backport_label_prefix: ${{ env.BACKPORT_LABEL_PREFIX }}
@@ -0,0 +1,39 @@
name: 'Tools: Comment Label Update'
on:
issue_comment:
types:
- 'created'
concurrency:
group: ${{ github.workflow }}-${{ github.event.issue.number }}
cancel-in-progress: false
jobs:
update-labels:
if: contains(github.event.issue.labels.*.name, 'status/awaiting-response')
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
issues: write
pull-requests: write
steps:
- name: Remove 'status/awaiting-response' label
env:
GH_TOKEN: ${{ github.token }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
run: |
echo "Removing 'status/awaiting-response' label from #$ISSUE_NUMBER"
gh api /repos/${{ github.repository }}/issues/$ISSUE_NUMBER/labels/status%2Fawaiting-response \
-X DELETE
- name: Add 'status/waiting-for-revision' label
env:
GH_TOKEN: ${{ github.token }}
ISSUE_NUMBER: ${{ github.event.issue.number }}
run: |
echo "Adding 'status/waiting-for-revision' label to #$ISSUE_NUMBER"
gh api /repos/${{ github.repository }}/issues/$ISSUE_NUMBER/labels \
-X POST \
-f labels[]='status/waiting-for-revision'
+1 -1
View File
@@ -26,6 +26,6 @@ jobs:
steps:
- name: Check PR title format
uses: agenthunt/conventional-commit-checker-action@9e552d650d0e205553ec7792d447929fc78e012b # v2.0.0
uses: agenthunt/conventional-commit-checker-action@f1823f632e95a64547566dcd2c7da920e67117ad # v2.0.1
with:
pr-title-regex: '^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\([^)]+\))?!?: .+'
+1 -1
View File
@@ -28,6 +28,6 @@ jobs:
fetch-depth: 0
- name: Scan for secrets with TruffleHog
uses: trufflesecurity/trufflehog@ad6fc8fb446b8fafbf7ea8193d2d6bfd42f45690 # v3.90.11
uses: trufflesecurity/trufflehog@b84c3d14d189e16da175e2c27fa8136603783ffc # v3.90.12
with:
extra_args: '--results=verified,unknown'
-34
View File
@@ -1,34 +0,0 @@
name: Label Community Contributors PRs
on:
pull_request:
types: [opened]
jobs:
add-community-label:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Label community contributors
env:
GH_TOKEN: ${{ github.token }}
run: |
# Fetch fresh PR data to get current author_association
ASSOCIATION=$(gh api /repos/${{ github.repository }}/pulls/${{ github.event.number }} --jq '.author_association')
AUTHOR=$(gh api /repos/${{ github.repository }}/pulls/${{ github.event.number }} --jq '.user.login')
echo "Author: $AUTHOR, Association: $ASSOCIATION"
# Members have associations like: OWNER, MEMBER, COLLABORATOR
# Non-members have: CONTRIBUTOR, FIRST_TIME_CONTRIBUTOR, FIRST_TIMER, NONE
if [[ "$ASSOCIATION" != "OWNER" && "$ASSOCIATION" != "MEMBER" && "$ASSOCIATION" != "COLLABORATOR" ]]; then
gh api /repos/${{ github.repository }}/issues/${{ github.event.number }}/labels \
-X POST \
-f labels[]='community'
echo "Added 'community' label for $ASSOCIATION contributor"
else
echo "Skipped labeling for $ASSOCIATION"
fi
+63
View File
@@ -27,3 +27,66 @@ jobs:
uses: actions/labeler@634933edcd8ababfe52f92936142cc22ac488b1b # v6.0.1
with:
sync-labels: true
label-community:
name: Add 'community' label if the PR is from a community contributor
needs: labeler
if: github.repository == 'prowler-cloud/prowler' && github.event.action == 'opened'
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Check if author is org member
id: check_membership
env:
AUTHOR: ${{ github.event.pull_request.user.login }}
run: |
# Hardcoded list of prowler-cloud organization members
# This list includes members who have set their organization membership as private
ORG_MEMBERS=(
"AdriiiPRodri"
"Alan-TheGentleman"
"alejandrobailo"
"amitsharm"
"andoniaf"
"cesararroba"
"Chan9390"
"danibarranqueroo"
"HugoPBrito"
"jfagoagas"
"josemazo"
"lydiavilchez"
"mmuller88"
"MrCloudSec"
"pedrooot"
"prowler-bot"
"puchy22"
"rakan-pro"
"RosaRivasProwler"
"StylusFrost"
"toniblyx"
"vicferpoy"
)
echo "Checking if $AUTHOR is a member of prowler-cloud organization"
# Check if author is in the org members list
if printf '%s\n' "${ORG_MEMBERS[@]}" | grep -q "^${AUTHOR}$"; then
echo "is_member=true" >> $GITHUB_OUTPUT
echo "$AUTHOR is an organization member"
else
echo "is_member=false" >> $GITHUB_OUTPUT
echo "$AUTHOR is not an organization member"
fi
- name: Add community label
if: steps.check_membership.outputs.is_member == 'false'
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ github.token }}
run: |
echo "Adding 'community' label to PR #$PR_NUMBER"
gh api /repos/${{ github.repository }}/issues/${{ github.event.number }}/labels \
-X POST \
-f labels[]='community'
+120 -40
View File
@@ -10,18 +10,24 @@ on:
release:
types:
- 'published'
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag (e.g., 5.14.0)'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
cancel-in-progress: false
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name }}
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.release_tag }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./mcp_server
@@ -43,7 +49,16 @@ jobs:
container-build-push:
needs: setup
runs-on: ubuntu-latest
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
timeout-minutes: 30
permissions:
contents: read
@@ -61,47 +76,112 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push MCP container (latest)
- name: Notify container push started
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: MCP
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
- name: Build and push MCP container for ${{ matrix.arch }}
id: container-push
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
platforms: ${{ matrix.platform }}
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-${{ matrix.arch }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event_name == 'release' && github.event.release.published_at || github.event.head_commit.timestamp }}
${{ github.event_name == 'release' && format('org.opencontainers.image.version={0}', env.RELEASE_TAG) || '' }}
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
- name: Notify container push completed
if: (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: MCP
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.container-push.outcome }}
# Create and push multi-architecture manifest
create-manifest:
needs: [setup, container-build-push]
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Create and push manifests for push event
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.head_commit.timestamp }}
cache-from: type=gha
cache-to: type=gha,mode=max
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Build and push MCP container (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
labels: |
org.opencontainers.image.title=Prowler MCP Server
org.opencontainers.image.description=Model Context Protocol server for Prowler
org.opencontainers.image.vendor=ProwlerPro, Inc.
org.opencontainers.image.version=${{ env.RELEASE_TAG }}
org.opencontainers.image.source=https://github.com/${{ github.repository }}
org.opencontainers.image.revision=${{ github.sha }}
org.opencontainers.image.created=${{ github.event.release.published_at }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
echo "Cleanup completed"
trigger-deployment:
if: github.event_name == 'push'
needs: [setup, container-build-push]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Trigger MCP deployment
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
@@ -1,24 +1,14 @@
name: 'MCP: Pull Request'
name: 'MCP: Container Checks'
on:
push:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/workflows/mcp-pull-request.yml'
- 'mcp_server/**'
- '!mcp_server/README.md'
- '!mcp_server/CHANGELOG.md'
pull_request:
branches:
- 'master'
- 'v5.*'
paths:
- '.github/workflows/mcp-pull-request.yml'
- 'mcp_server/**'
- '!mcp_server/README.md'
- '!mcp_server/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -29,8 +19,7 @@ env:
IMAGE_NAME: prowler-mcp
jobs:
dockerfile-lint:
if: github.repository == 'prowler-cloud/prowler'
mcp-dockerfile-lint:
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
@@ -40,13 +29,19 @@ jobs:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check if Dockerfile changed
id: dockerfile-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: mcp_server/Dockerfile
- name: Lint Dockerfile with Hadolint
if: steps.dockerfile-changed.outputs.any_changed == 'true'
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
dockerfile: mcp_server/Dockerfile
container-build-and-scan:
if: github.repository == 'prowler-cloud/prowler'
mcp-container-build-and-scan:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
@@ -58,10 +53,21 @@ jobs:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for MCP changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: mcp_server/**
files_ignore: |
mcp_server/README.md
mcp_server/CHANGELOG.md
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build MCP container
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.MCP_WORKING_DIR }}
@@ -72,6 +78,7 @@ jobs:
cache-to: type=gha,mode=max
- name: Scan MCP container with Trivy
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
+1 -1
View File
@@ -83,7 +83,7 @@ jobs:
- name: Update PR comment with changelog status
if: github.event.pull_request.head.repo.full_name == github.repository
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
issue-number: ${{ github.event.pull_request.number }}
comment-id: ${{ steps.find-comment.outputs.comment-id }}
+1 -1
View File
@@ -97,7 +97,7 @@ jobs:
body-includes: '<!-- conflict-checker-comment -->'
- name: Create or update comment
uses: peter-evans/create-or-update-comment@71345be0265236311c031f5c7866368bd1eff043 # v4.0.0
uses: peter-evans/create-or-update-comment@e8674b075228eee787fea43ef493e45ece1004c9 # v5.0.0
with:
comment-id: ${{ steps.find-comment.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
+71 -81
View File
@@ -47,7 +47,7 @@ jobs:
git config --global user.name 'prowler-bot'
git config --global user.email '179230569+prowler-bot@users.noreply.github.com'
- name: Parse version and read changelogs
- name: Parse version and determine branch
run: |
# Validate version format (reusing pattern from sdk-bump-version.yml)
if [[ $PROWLER_VERSION =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
@@ -64,66 +64,83 @@ jobs:
BRANCH_NAME="v${MAJOR_VERSION}.${MINOR_VERSION}"
echo "BRANCH_NAME=${BRANCH_NAME}" >> "${GITHUB_ENV}"
# Function to extract the latest version from changelog
extract_latest_version() {
local changelog_file="$1"
if [ -f "$changelog_file" ]; then
# Extract the first version entry (most recent) from changelog
# Format: ## [version] (1.2.3) or ## [vversion] (v1.2.3)
local version=$(grep -m 1 '^## \[' "$changelog_file" | sed 's/^## \[\(.*\)\].*/\1/' | sed 's/^v//' | tr -d '[:space:]')
echo "$version"
else
echo ""
fi
}
# Read actual versions from changelogs (source of truth)
UI_VERSION=$(extract_latest_version "ui/CHANGELOG.md")
API_VERSION=$(extract_latest_version "api/CHANGELOG.md")
SDK_VERSION=$(extract_latest_version "prowler/CHANGELOG.md")
MCP_VERSION=$(extract_latest_version "mcp_server/CHANGELOG.md")
echo "UI_VERSION=${UI_VERSION}" >> "${GITHUB_ENV}"
echo "API_VERSION=${API_VERSION}" >> "${GITHUB_ENV}"
echo "SDK_VERSION=${SDK_VERSION}" >> "${GITHUB_ENV}"
echo "MCP_VERSION=${MCP_VERSION}" >> "${GITHUB_ENV}"
if [ -n "$UI_VERSION" ]; then
echo "Read UI version from changelog: $UI_VERSION"
else
echo "Warning: No UI version found in ui/CHANGELOG.md"
fi
if [ -n "$API_VERSION" ]; then
echo "Read API version from changelog: $API_VERSION"
else
echo "Warning: No API version found in api/CHANGELOG.md"
fi
if [ -n "$SDK_VERSION" ]; then
echo "Read SDK version from changelog: $SDK_VERSION"
else
echo "Warning: No SDK version found in prowler/CHANGELOG.md"
fi
if [ -n "$MCP_VERSION" ]; then
echo "Read MCP version from changelog: $MCP_VERSION"
else
echo "Warning: No MCP version found in mcp_server/CHANGELOG.md"
fi
echo "Prowler version: $PROWLER_VERSION"
echo "Branch name: $BRANCH_NAME"
echo "UI version: $UI_VERSION"
echo "API version: $API_VERSION"
echo "SDK version: $SDK_VERSION"
echo "MCP version: $MCP_VERSION"
echo "Is minor release: $([ $PATCH_VERSION -eq 0 ] && echo 'true' || echo 'false')"
else
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
exit 1
fi
- name: Checkout release branch
run: |
echo "Checking out branch $BRANCH_NAME for release $PROWLER_VERSION..."
if git show-ref --verify --quiet "refs/heads/$BRANCH_NAME"; then
echo "Branch $BRANCH_NAME exists locally, checking out..."
git checkout "$BRANCH_NAME"
elif git show-ref --verify --quiet "refs/remotes/origin/$BRANCH_NAME"; then
echo "Branch $BRANCH_NAME exists remotely, checking out..."
git checkout -b "$BRANCH_NAME" "origin/$BRANCH_NAME"
else
echo "ERROR: Branch $BRANCH_NAME does not exist. For minor releases (X.Y.0), create it manually first. For patch releases (X.Y.Z), the branch should already exist."
exit 1
fi
- name: Read changelog versions from release branch
run: |
# Function to extract the latest version from changelog
extract_latest_version() {
local changelog_file="$1"
if [ -f "$changelog_file" ]; then
# Extract the first version entry (most recent) from changelog
# Format: ## [version] (1.2.3) or ## [vversion] (v1.2.3)
local version=$(grep -m 1 '^## \[' "$changelog_file" | sed 's/^## \[\(.*\)\].*/\1/' | sed 's/^v//' | tr -d '[:space:]')
echo "$version"
else
echo ""
fi
}
# Read actual versions from changelogs (source of truth)
UI_VERSION=$(extract_latest_version "ui/CHANGELOG.md")
API_VERSION=$(extract_latest_version "api/CHANGELOG.md")
SDK_VERSION=$(extract_latest_version "prowler/CHANGELOG.md")
MCP_VERSION=$(extract_latest_version "mcp_server/CHANGELOG.md")
echo "UI_VERSION=${UI_VERSION}" >> "${GITHUB_ENV}"
echo "API_VERSION=${API_VERSION}" >> "${GITHUB_ENV}"
echo "SDK_VERSION=${SDK_VERSION}" >> "${GITHUB_ENV}"
echo "MCP_VERSION=${MCP_VERSION}" >> "${GITHUB_ENV}"
if [ -n "$UI_VERSION" ]; then
echo "Read UI version from changelog: $UI_VERSION"
else
echo "Warning: No UI version found in ui/CHANGELOG.md"
fi
if [ -n "$API_VERSION" ]; then
echo "Read API version from changelog: $API_VERSION"
else
echo "Warning: No API version found in api/CHANGELOG.md"
fi
if [ -n "$SDK_VERSION" ]; then
echo "Read SDK version from changelog: $SDK_VERSION"
else
echo "Warning: No SDK version found in prowler/CHANGELOG.md"
fi
if [ -n "$MCP_VERSION" ]; then
echo "Read MCP version from changelog: $MCP_VERSION"
else
echo "Warning: No MCP version found in mcp_server/CHANGELOG.md"
fi
echo "UI version: $UI_VERSION"
echo "API version: $API_VERSION"
echo "SDK version: $SDK_VERSION"
echo "MCP version: $MCP_VERSION"
- name: Extract and combine changelog entries
run: |
set -e
@@ -255,21 +272,6 @@ jobs:
echo "Combined changelog preview:"
cat combined_changelog.md
- name: Checkout release branch for patch release
if: ${{ env.PATCH_VERSION != '0' }}
run: |
echo "Patch release detected, checking out existing branch $BRANCH_NAME..."
if git show-ref --verify --quiet "refs/heads/$BRANCH_NAME"; then
echo "Branch $BRANCH_NAME exists locally, checking out..."
git checkout "$BRANCH_NAME"
elif git show-ref --verify --quiet "refs/remotes/origin/$BRANCH_NAME"; then
echo "Branch $BRANCH_NAME exists remotely, checking out..."
git checkout -b "$BRANCH_NAME" "origin/$BRANCH_NAME"
else
echo "ERROR: Branch $BRANCH_NAME should exist for patch release $PROWLER_VERSION"
exit 1
fi
- name: Verify SDK version in pyproject.toml
run: |
CURRENT_VERSION=$(grep '^version = ' pyproject.toml | sed -E 's/version = "([^"]+)"/\1/' | tr -d '[:space:]')
@@ -323,18 +325,6 @@ jobs:
fi
echo "✓ api/src/backend/api/v1/views.py version: $CURRENT_API_VERSION"
- name: Checkout release branch for minor release
if: ${{ env.PATCH_VERSION == '0' }}
run: |
echo "Minor release detected (patch = 0), checking out existing branch $BRANCH_NAME..."
if git show-ref --verify --quiet "refs/remotes/origin/$BRANCH_NAME"; then
echo "Branch $BRANCH_NAME exists remotely, checking out..."
git checkout -b "$BRANCH_NAME" "origin/$BRANCH_NAME"
else
echo "ERROR: Branch $BRANCH_NAME should exist for minor release $PROWLER_VERSION. Please create it manually first."
exit 1
fi
- name: Update API prowler dependency for minor release
if: ${{ env.PATCH_VERSION == '0' }}
run: |
@@ -392,7 +382,7 @@ jobs:
no-changelog
- name: Create draft release
uses: softprops/action-gh-release@6cbd405e2c4e67a21c47fa9e383d020e4e28b836 # v2.3.3
uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2.4.1
with:
tag_name: ${{ env.PROWLER_VERSION }}
name: Prowler ${{ env.PROWLER_VERSION }}
@@ -1,202 +0,0 @@
name: SDK - Build and Push containers
on:
push:
branches:
# For `v3-latest`
- "v3"
# For `v4-latest`
- "v4.6"
# For `latest`
- "master"
paths-ignore:
- ".github/**"
- "README.md"
- "docs/**"
- "ui/**"
- "api/**"
release:
types: [published]
env:
# AWS Configuration
AWS_REGION_STG: eu-west-1
AWS_REGION_PLATFORM: eu-west-1
AWS_REGION: us-east-1
# Container's configuration
IMAGE_NAME: prowler
DOCKERFILE_PATH: ./Dockerfile
# Tags
LATEST_TAG: latest
STABLE_TAG: stable
# The RELEASE_TAG is set during runtime in releases
RELEASE_TAG: ""
# The PROWLER_VERSION and PROWLER_VERSION_MAJOR are set during runtime in releases
PROWLER_VERSION: ""
PROWLER_VERSION_MAJOR: ""
# TEMPORARY_TAG: temporary
# Python configuration
PYTHON_VERSION: 3.12
# Container Registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler
jobs:
# Build Prowler OSS container
container-build-push:
# needs: dockerfile-linter
runs-on: ubuntu-latest
outputs:
prowler_version_major: ${{ steps.get-prowler-version.outputs.PROWLER_VERSION_MAJOR }}
prowler_version: ${{ steps.get-prowler-version.outputs.PROWLER_VERSION }}
env:
POETRY_VIRTUALENVS_CREATE: "false"
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install Poetry
run: |
pipx install poetry==2.*
pipx inject poetry poetry-bumpversion
- name: Get Prowler version
id: get-prowler-version
run: |
PROWLER_VERSION="$(poetry version -s 2>/dev/null)"
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_ENV}"
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
# Store prowler version major just for the release
PROWLER_VERSION_MAJOR="${PROWLER_VERSION%%.*}"
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_ENV}"
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_OUTPUT}"
case ${PROWLER_VERSION_MAJOR} in
3)
echo "LATEST_TAG=v3-latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=v3-stable" >> "${GITHUB_ENV}"
;;
4)
echo "LATEST_TAG=v4-latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=v4-stable" >> "${GITHUB_ENV}"
;;
5)
echo "LATEST_TAG=latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=stable" >> "${GITHUB_ENV}"
;;
*)
# Fallback if any other version is present
echo "Releasing another Prowler major version, aborting..."
exit 1
;;
esac
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push container image (latest)
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
push: true
tags: |
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
file: ${{ env.DOCKERFILE_PATH }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
# Use local context to get changes
# https://github.com/docker/build-push-action#path-context
context: .
push: true
tags: |
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.PROWLER_VERSION }}
${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ env.STABLE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.PROWLER_VERSION }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
file: ${{ env.DOCKERFILE_PATH }}
cache-from: type=gha
cache-to: type=gha,mode=max
# - name: Push README to Docker Hub (toniblyx)
# uses: peter-evans/dockerhub-description@432a30c9e07499fd01da9f8a49f0faf9e0ca5b77 # v4.0.2
# with:
# username: ${{ secrets.DOCKERHUB_USERNAME }}
# password: ${{ secrets.DOCKERHUB_TOKEN }}
# repository: ${{ env.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}
# readme-filepath: ./README.md
#
# - name: Push README to Docker Hub (prowlercloud)
# uses: peter-evans/dockerhub-description@432a30c9e07499fd01da9f8a49f0faf9e0ca5b77 # v4.0.2
# with:
# username: ${{ secrets.DOCKERHUB_USERNAME }}
# password: ${{ secrets.DOCKERHUB_TOKEN }}
# repository: ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}
# readme-filepath: ./README.md
dispatch-action:
needs: container-build-push
runs-on: ubuntu-latest
steps:
- name: Get latest commit info (latest)
if: github.event_name == 'push'
run: |
LATEST_COMMIT_HASH=$(echo ${{ github.event.after }} | cut -b -7)
echo "LATEST_COMMIT_HASH=${LATEST_COMMIT_HASH}" >> $GITHUB_ENV
- name: Dispatch event (latest)
if: github.event_name == 'push' && needs.container-build-push.outputs.prowler_version_major == '3'
run: |
curl https://api.github.com/repos/${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}/dispatches \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
--data '{"event_type":"dispatch","client_payload":{"version":"v3-latest", "tag": "${{ env.LATEST_COMMIT_HASH }}"}}'
- name: Dispatch event (release)
if: github.event_name == 'release' && needs.container-build-push.outputs.prowler_version_major == '3'
run: |
curl https://api.github.com/repos/${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}/dispatches \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}" \
-H "X-GitHub-Api-Version: 2022-11-28" \
--data '{"event_type":"dispatch","client_payload":{"version":"release", "tag":"${{ needs.container-build-push.outputs.prowler_version }}"}}'
+168 -96
View File
@@ -1,146 +1,218 @@
name: SDK - Bump Version
name: 'SDK: Bump Version'
on:
release:
types: [published]
types:
- 'published'
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
cancel-in-progress: false
env:
PROWLER_VERSION: ${{ github.event.release.tag_name }}
BASE_BRANCH: master
jobs:
bump-version:
name: Bump Version
detect-release-type:
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
is_minor: ${{ steps.detect.outputs.is_minor }}
is_patch: ${{ steps.detect.outputs.is_patch }}
major_version: ${{ steps.detect.outputs.major_version }}
minor_version: ${{ steps.detect.outputs.minor_version }}
patch_version: ${{ steps.detect.outputs.patch_version }}
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Get Prowler version
shell: bash
- name: Detect release type and parse version
id: detect
run: |
if [[ $PROWLER_VERSION =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
MAJOR_VERSION=${BASH_REMATCH[1]}
MINOR_VERSION=${BASH_REMATCH[2]}
FIX_VERSION=${BASH_REMATCH[3]}
PATCH_VERSION=${BASH_REMATCH[3]}
# Export version components to GitHub environment
echo "MAJOR_VERSION=${MAJOR_VERSION}" >> "${GITHUB_ENV}"
echo "MINOR_VERSION=${MINOR_VERSION}" >> "${GITHUB_ENV}"
echo "FIX_VERSION=${FIX_VERSION}" >> "${GITHUB_ENV}"
echo "major_version=${MAJOR_VERSION}" >> "${GITHUB_OUTPUT}"
echo "minor_version=${MINOR_VERSION}" >> "${GITHUB_OUTPUT}"
echo "patch_version=${PATCH_VERSION}" >> "${GITHUB_OUTPUT}"
if (( MAJOR_VERSION == 5 )); then
if (( FIX_VERSION == 0 )); then
echo "Minor Release: $PROWLER_VERSION"
if (( MAJOR_VERSION != 5 )); then
echo "::error::Releasing another Prowler major version, aborting..."
exit 1
fi
# Set up next minor version for master
BUMP_VERSION_TO=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).${FIX_VERSION}
echo "BUMP_VERSION_TO=${BUMP_VERSION_TO}" >> "${GITHUB_ENV}"
TARGET_BRANCH=${BASE_BRANCH}
echo "TARGET_BRANCH=${TARGET_BRANCH}" >> "${GITHUB_ENV}"
# Set up patch version for version branch
PATCH_VERSION_TO=${MAJOR_VERSION}.${MINOR_VERSION}.1
echo "PATCH_VERSION_TO=${PATCH_VERSION_TO}" >> "${GITHUB_ENV}"
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
echo "Bumping to next minor version: ${BUMP_VERSION_TO} in branch ${TARGET_BRANCH}"
echo "Bumping to next patch version: ${PATCH_VERSION_TO} in branch ${VERSION_BRANCH}"
else
echo "Patch Release: $PROWLER_VERSION"
BUMP_VERSION_TO=${MAJOR_VERSION}.${MINOR_VERSION}.$((FIX_VERSION + 1))
echo "BUMP_VERSION_TO=${BUMP_VERSION_TO}" >> "${GITHUB_ENV}"
TARGET_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
echo "TARGET_BRANCH=${TARGET_BRANCH}" >> "${GITHUB_ENV}"
echo "Bumping to next patch version: ${BUMP_VERSION_TO} in branch ${TARGET_BRANCH}"
fi
if (( PATCH_VERSION == 0 )); then
echo "is_minor=true" >> "${GITHUB_OUTPUT}"
echo "is_patch=false" >> "${GITHUB_OUTPUT}"
echo "✓ Minor release detected: $PROWLER_VERSION"
else
echo "Releasing another Prowler major version, aborting..."
exit 1
echo "is_minor=false" >> "${GITHUB_OUTPUT}"
echo "is_patch=true" >> "${GITHUB_OUTPUT}"
echo "✓ Patch release detected: $PROWLER_VERSION"
fi
else
echo "Invalid version syntax: '$PROWLER_VERSION' (must be N.N.N)" >&2
echo "::error::Invalid version syntax: '$PROWLER_VERSION' (must be X.Y.Z)"
exit 1
fi
- name: Bump versions in files
bump-minor-version:
needs: detect-release-type
if: needs.detect-release-type.outputs.is_minor == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Calculate next minor version
run: |
echo "Using PROWLER_VERSION=$PROWLER_VERSION"
echo "Using BUMP_VERSION_TO=$BUMP_VERSION_TO"
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
set -e
NEXT_MINOR_VERSION=${MAJOR_VERSION}.$((MINOR_VERSION + 1)).0
echo "NEXT_MINOR_VERSION=${NEXT_MINOR_VERSION}" >> "${GITHUB_ENV}"
echo "Bumping version in pyproject.toml ..."
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${BUMP_VERSION_TO}\"|" pyproject.toml
echo "Current version: $PROWLER_VERSION"
echo "Next minor version: $NEXT_MINOR_VERSION"
echo "Bumping version in prowler/config/config.py ..."
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${BUMP_VERSION_TO}\"|" prowler/config/config.py
- name: Bump versions in files for master
run: |
set -e
echo "Bumping version in .env ..."
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${BUMP_VERSION_TO}|" .env
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${NEXT_MINOR_VERSION}\"|" pyproject.toml
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${NEXT_MINOR_VERSION}\"|" prowler/config/config.py
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${NEXT_MINOR_VERSION}|" .env
git --no-pager diff
echo "Files modified:"
git --no-pager diff
- name: Create Pull Request
- name: Create PR for next minor version to master
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
base: ${{ env.TARGET_BRANCH }}
commit-message: "chore(release): Bump version to v${{ env.BUMP_VERSION_TO }}"
branch: "version-bump-to-v${{ env.BUMP_VERSION_TO }}"
title: "chore(release): Bump version to v${{ env.BUMP_VERSION_TO }}"
labels: no-changelog
body: |
### Description
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
base: master
commit-message: 'chore(release): Bump version to v${{ env.NEXT_MINOR_VERSION }}'
branch: version-bump-to-v${{ env.NEXT_MINOR_VERSION }}
title: 'chore(release): Bump version to v${{ env.NEXT_MINOR_VERSION }}'
labels: no-changelog
body: |
### Description
Bump Prowler version to v${{ env.BUMP_VERSION_TO }}
Bump Prowler version to v${{ env.NEXT_MINOR_VERSION }} after releasing v${{ env.PROWLER_VERSION }}.
### License
### License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
- name: Handle patch version for minor release
if: env.FIX_VERSION == '0'
- name: Checkout version branch
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: v${{ needs.detect-release-type.outputs.major_version }}.${{ needs.detect-release-type.outputs.minor_version }}
- name: Calculate first patch version
run: |
echo "Using PROWLER_VERSION=$PROWLER_VERSION"
echo "Using PATCH_VERSION_TO=$PATCH_VERSION_TO"
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
set -e
FIRST_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.1
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
echo "Bumping version in pyproject.toml ..."
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${PATCH_VERSION_TO}\"|" pyproject.toml
echo "FIRST_PATCH_VERSION=${FIRST_PATCH_VERSION}" >> "${GITHUB_ENV}"
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
echo "Bumping version in prowler/config/config.py ..."
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${PATCH_VERSION_TO}\"|" prowler/config/config.py
echo "First patch version: $FIRST_PATCH_VERSION"
echo "Version branch: $VERSION_BRANCH"
echo "Bumping version in .env ..."
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PATCH_VERSION_TO}|" .env
- name: Bump versions in files for version branch
run: |
set -e
git --no-pager diff
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${FIRST_PATCH_VERSION}\"|" pyproject.toml
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${FIRST_PATCH_VERSION}\"|" prowler/config/config.py
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${FIRST_PATCH_VERSION}|" .env
- name: Create Pull Request for patch version
if: env.FIX_VERSION == '0'
echo "Files modified:"
git --no-pager diff
- name: Create PR for first patch version to version branch
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
base: ${{ env.VERSION_BRANCH }}
commit-message: "chore(release): Bump version to v${{ env.PATCH_VERSION_TO }}"
branch: "version-bump-to-v${{ env.PATCH_VERSION_TO }}"
title: "chore(release): Bump version to v${{ env.PATCH_VERSION_TO }}"
labels: no-changelog
body: |
### Description
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
base: ${{ env.VERSION_BRANCH }}
commit-message: 'chore(release): Bump version to v${{ env.FIRST_PATCH_VERSION }}'
branch: version-bump-to-v${{ env.FIRST_PATCH_VERSION }}
title: 'chore(release): Bump version to v${{ env.FIRST_PATCH_VERSION }}'
labels: no-changelog
body: |
### Description
Bump Prowler version to v${{ env.PATCH_VERSION_TO }}
Bump Prowler version to v${{ env.FIRST_PATCH_VERSION }} in version branch after releasing v${{ env.PROWLER_VERSION }}.
### License
### License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
bump-patch-version:
needs: detect-release-type
if: needs.detect-release-type.outputs.is_patch == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Calculate next patch version
run: |
MAJOR_VERSION=${{ needs.detect-release-type.outputs.major_version }}
MINOR_VERSION=${{ needs.detect-release-type.outputs.minor_version }}
PATCH_VERSION=${{ needs.detect-release-type.outputs.patch_version }}
NEXT_PATCH_VERSION=${MAJOR_VERSION}.${MINOR_VERSION}.$((PATCH_VERSION + 1))
VERSION_BRANCH=v${MAJOR_VERSION}.${MINOR_VERSION}
echo "NEXT_PATCH_VERSION=${NEXT_PATCH_VERSION}" >> "${GITHUB_ENV}"
echo "VERSION_BRANCH=${VERSION_BRANCH}" >> "${GITHUB_ENV}"
echo "Current version: $PROWLER_VERSION"
echo "Next patch version: $NEXT_PATCH_VERSION"
echo "Target branch: $VERSION_BRANCH"
- name: Bump versions in files for version branch
run: |
set -e
sed -i "s|version = \"${PROWLER_VERSION}\"|version = \"${NEXT_PATCH_VERSION}\"|" pyproject.toml
sed -i "s|prowler_version = \"${PROWLER_VERSION}\"|prowler_version = \"${NEXT_PATCH_VERSION}\"|" prowler/config/config.py
sed -i "s|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${PROWLER_VERSION}|NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${NEXT_PATCH_VERSION}|" .env
echo "Files modified:"
git --no-pager diff
- name: Create PR for next patch version to version branch
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
base: ${{ env.VERSION_BRANCH }}
commit-message: 'chore(release): Bump version to v${{ env.NEXT_PATCH_VERSION }}'
branch: version-bump-to-v${{ env.NEXT_PATCH_VERSION }}
title: 'chore(release): Bump version to v${{ env.NEXT_PATCH_VERSION }}'
labels: no-changelog
body: |
### Description
Bump Prowler version to v${{ env.NEXT_PATCH_VERSION }} after releasing v${{ env.PROWLER_VERSION }}.
### License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
+90
View File
@@ -0,0 +1,90 @@
name: 'SDK: Code Quality'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
sdk-code-quality:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.9'
- '3.10'
- '3.11'
- '3.12'
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for SDK changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ./**
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
- name: Install Poetry
if: steps.check-changes.outputs.any_changed == 'true'
run: pipx install poetry==2.1.1
- name: Set up Python ${{ matrix.python-version }}
if: steps.check-changes.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'
- name: Install dependencies
if: steps.check-changes.outputs.any_changed == 'true'
run: |
poetry install --no-root
poetry run pip list
- name: Check Poetry lock file
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry check --lock
- name: Lint with flake8
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude contrib,ui,api
- name: Check format with black
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run black --exclude api ui --check .
- name: Lint with pylint
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run pylint --disable=W,C,R,E -j 0 -rn -sn prowler/
+4 -3
View File
@@ -31,7 +31,8 @@ concurrency:
cancel-in-progress: true
jobs:
analyze:
sdk-analyze:
if: github.repository == 'prowler-cloud/prowler'
name: CodeQL Security Analysis
runs-on: ubuntu-latest
timeout-minutes: 30
@@ -51,12 +52,12 @@ jobs:
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/init@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/sdk-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/analyze@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
category: '/language:${{ matrix.language }}'
@@ -0,0 +1,282 @@
name: 'SDK: Container Build and Push'
on:
push:
branches:
- 'v3' # For v3-latest
- 'v4.6' # For v4-latest
- 'master' # For latest
paths-ignore:
- '.github/**'
- '!.github/workflows/sdk-container-build-push.yml'
- 'README.md'
- 'docs/**'
- 'ui/**'
- 'api/**'
release:
types:
- 'published'
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag (e.g., 5.14.0)'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
env:
# Container configuration
IMAGE_NAME: prowler
DOCKERFILE_PATH: ./Dockerfile
# Python configuration
PYTHON_VERSION: '3.12'
# Tags (dynamically set based on version)
LATEST_TAG: latest
STABLE_TAG: stable
# Container registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler
# AWS configuration (for ECR)
AWS_REGION: us-east-1
jobs:
container-build-push:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
timeout-minutes: 45
permissions:
contents: read
packages: write
outputs:
prowler_version: ${{ steps.get-prowler-version.outputs.prowler_version }}
prowler_version_major: ${{ steps.get-prowler-version.outputs.prowler_version_major }}
latest_tag: ${{ steps.get-prowler-version.outputs.latest_tag }}
stable_tag: ${{ steps.get-prowler-version.outputs.stable_tag }}
env:
POETRY_VIRTUALENVS_CREATE: 'false'
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install Poetry
run: |
pipx install poetry==2.1.1
pipx inject poetry poetry-bumpversion
- name: Get Prowler version and set tags
id: get-prowler-version
run: |
PROWLER_VERSION="$(poetry version -s 2>/dev/null)"
echo "prowler_version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
echo "PROWLER_VERSION=${PROWLER_VERSION}" >> "${GITHUB_ENV}"
# Extract major version
PROWLER_VERSION_MAJOR="${PROWLER_VERSION%%.*}"
echo "prowler_version_major=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_OUTPUT}"
echo "PROWLER_VERSION_MAJOR=${PROWLER_VERSION_MAJOR}" >> "${GITHUB_ENV}"
# Set version-specific tags
case ${PROWLER_VERSION_MAJOR} in
3)
echo "LATEST_TAG=v3-latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=v3-stable" >> "${GITHUB_ENV}"
echo "latest_tag=v3-latest" >> "${GITHUB_OUTPUT}"
echo "stable_tag=v3-stable" >> "${GITHUB_OUTPUT}"
echo "✓ Prowler v3 detected - tags: v3-latest, v3-stable"
;;
4)
echo "LATEST_TAG=v4-latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=v4-stable" >> "${GITHUB_ENV}"
echo "latest_tag=v4-latest" >> "${GITHUB_OUTPUT}"
echo "stable_tag=v4-stable" >> "${GITHUB_OUTPUT}"
echo "✓ Prowler v4 detected - tags: v4-latest, v4-stable"
;;
5)
echo "LATEST_TAG=latest" >> "${GITHUB_ENV}"
echo "STABLE_TAG=stable" >> "${GITHUB_ENV}"
echo "latest_tag=latest" >> "${GITHUB_OUTPUT}"
echo "stable_tag=stable" >> "${GITHUB_OUTPUT}"
echo "✓ Prowler v5 detected - tags: latest, stable"
;;
*)
echo "::error::Unsupported Prowler major version: ${PROWLER_VERSION_MAJOR}"
exit 1
;;
esac
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Notify container push started
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: SDK
RELEASE_TAG: ${{ env.PROWLER_VERSION }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
- name: Build and push SDK container for ${{ matrix.arch }}
id: container-push
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: ${{ env.DOCKERFILE_PATH }}
push: true
platforms: ${{ matrix.platform }}
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}-${{ matrix.arch }}
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
- name: Notify container push completed
if: (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: SDK
RELEASE_TAG: ${{ env.PROWLER_VERSION }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.container-push.outcome }}
# Create and push multi-architecture manifest
create-manifest:
needs: [container-build-push]
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Create and push manifests for push event
if: github.event_name == 'push'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-arm64
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.container-build-push.outputs.prowler_version }} \
-t ${{ secrets.DOCKER_HUB_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.container-build-push.outputs.stable_tag }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.container-build-push.outputs.prowler_version }} \
-t ${{ secrets.PUBLIC_ECR_REPOSITORY }}/${{ env.IMAGE_NAME }}:${{ needs.container-build-push.outputs.stable_tag }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.prowler_version }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.stable_tag }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-arm64
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.container-build-push.outputs.latest_tag }}-arm64" || true
echo "Cleanup completed"
dispatch-v3-deployment:
if: needs.container-build-push.outputs.prowler_version_major == '3'
needs: container-build-push
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Calculate short SHA
id: short-sha
run: echo "short_sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
- name: Dispatch v3 deployment (latest)
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}
event-type: dispatch
client-payload: '{"version":"v3-latest","tag":"${{ steps.short-sha.outputs.short_sha }}"}'
- name: Dispatch v3 deployment (release)
if: github.event_name == 'release'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.DISPATCH_OWNER }}/${{ secrets.DISPATCH_REPO }}
event-type: dispatch
client-payload: '{"version":"release","tag":"${{ needs.container-build-push.outputs.prowler_version }}"}'
+103
View File
@@ -0,0 +1,103 @@
name: 'SDK: Container Checks'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
IMAGE_NAME: prowler
jobs:
sdk-dockerfile-lint:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check if Dockerfile changed
id: dockerfile-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: Dockerfile
- name: Lint Dockerfile with Hadolint
if: steps.dockerfile-changed.outputs.any_changed == 'true'
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
dockerfile: Dockerfile
ignore: DL3013
sdk-container-build-and-scan:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for SDK changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ./**
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build SDK container
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
push: false
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Scan SDK container with Trivy
if: steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
-286
View File
@@ -1,286 +0,0 @@
name: SDK - Pull Request
on:
push:
branches:
- "master"
- "v3"
- "v4.*"
- "v5.*"
pull_request:
branches:
- "master"
- "v3"
- "v4.*"
- "v5.*"
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12"]
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Test if changes are in not ignored paths
id: are-non-ignored-files-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ./**
files_ignore: |
.github/**
docs/**
permissions/**
api/**
ui/**
prowler/CHANGELOG.md
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
- name: Install poetry
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
python -m pip install --upgrade pip
pipx install poetry==2.1.1
- name: Set up Python ${{ matrix.python-version }}
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"
- name: Install dependencies
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry install --no-root
poetry run pip list
VERSION=$(curl --silent "https://api.github.com/repos/hadolint/hadolint/releases/latest" | \
grep '"tag_name":' | \
sed -E 's/.*"v([^"]+)".*/\1/' \
) && curl -L -o /tmp/hadolint "https://github.com/hadolint/hadolint/releases/download/v${VERSION}/hadolint-Linux-x86_64" \
&& chmod +x /tmp/hadolint
- name: Poetry check
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry check --lock
- name: Lint with flake8
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude contrib,ui,api
- name: Checking format with black
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run black --exclude api ui --check .
- name: Lint with pylint
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pylint --disable=W,C,R,E -j 0 -rn -sn prowler/
- name: Bandit
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run bandit -q -lll -x '*_test.py,./contrib/,./api/,./ui' -r .
- name: Safety
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run safety check --ignore 70612 -r pyproject.toml
- name: Vulture
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run vulture --exclude "contrib,api,ui" --min-confidence 100 .
- name: Dockerfile - Check if Dockerfile has changed
id: dockerfile-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
Dockerfile
- name: Hadolint
if: steps.dockerfile-changed-files.outputs.any_changed == 'true'
run: |
/tmp/hadolint Dockerfile --ignore=DL3013
# Test AWS
- name: AWS - Check if any file has changed
id: aws-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/aws/**
./tests/providers/aws/**
./poetry.lock
- name: AWS - Test
if: steps.aws-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml tests/providers/aws
# Test Azure
- name: Azure - Check if any file has changed
id: azure-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/azure/**
./tests/providers/azure/**
./poetry.lock
- name: Azure - Test
if: steps.azure-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/azure --cov-report=xml:azure_coverage.xml tests/providers/azure
# Test GCP
- name: GCP - Check if any file has changed
id: gcp-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/gcp/**
./tests/providers/gcp/**
./poetry.lock
- name: GCP - Test
if: steps.gcp-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/gcp --cov-report=xml:gcp_coverage.xml tests/providers/gcp
# Test Kubernetes
- name: Kubernetes - Check if any file has changed
id: kubernetes-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/kubernetes/**
./tests/providers/kubernetes/**
./poetry.lock
- name: Kubernetes - Test
if: steps.kubernetes-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/kubernetes --cov-report=xml:kubernetes_coverage.xml tests/providers/kubernetes
# Test GitHub
- name: GitHub - Check if any file has changed
id: github-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/github/**
./tests/providers/github/**
./poetry.lock
- name: GitHub - Test
if: steps.github-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/github --cov-report=xml:github_coverage.xml tests/providers/github
# Test NHN
- name: NHN - Check if any file has changed
id: nhn-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/nhn/**
./tests/providers/nhn/**
./poetry.lock
- name: NHN - Test
if: steps.nhn-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/nhn --cov-report=xml:nhn_coverage.xml tests/providers/nhn
# Test M365
- name: M365 - Check if any file has changed
id: m365-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/m365/**
./tests/providers/m365/**
./poetry.lock
- name: M365 - Test
if: steps.m365-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/m365 --cov-report=xml:m365_coverage.xml tests/providers/m365
# Test IaC
- name: IaC - Check if any file has changed
id: iac-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/iac/**
./tests/providers/iac/**
./poetry.lock
- name: IaC - Test
if: steps.iac-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/iac --cov-report=xml:iac_coverage.xml tests/providers/iac
# Test MongoDB Atlas
- name: MongoDB Atlas - Check if any file has changed
id: mongodb-atlas-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/mongodbatlas/**
./tests/providers/mongodbatlas/**
.poetry.lock
- name: MongoDB Atlas - Test
if: steps.mongodb-atlas-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodb_atlas_coverage.xml tests/providers/mongodbatlas
# Test OCI
- name: OCI - Check if any file has changed
id: oci-changed-files
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/providers/oraclecloud/**
./tests/providers/oraclecloud/**
./poetry.lock
- name: OCI - Test
if: steps.oci-changed-files.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oci_coverage.xml tests/providers/oraclecloud
# Common Tests
- name: Lib - Test
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/lib --cov-report=xml:lib_coverage.xml tests/lib
- name: Config - Test
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pytest -n auto --cov=./prowler/config --cov-report=xml:config_coverage.xml tests/config
# Codecov
- name: Upload coverage reports to Codecov
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler
files: ./aws_coverage.xml,./azure_coverage.xml,./gcp_coverage.xml,./kubernetes_coverage.xml,./github_coverage.xml,./nhn_coverage.xml,./m365_coverage.xml,./oci_coverage.xml,./lib_coverage.xml,./config_coverage.xml
+85 -64
View File
@@ -1,98 +1,119 @@
name: SDK - PyPI release
name: 'SDK: PyPI Release'
on:
release:
types: [published]
types:
- 'published'
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name }}
cancel-in-progress: false
env:
RELEASE_TAG: ${{ github.event.release.tag_name }}
PYTHON_VERSION: 3.11
# CACHE: "poetry"
PYTHON_VERSION: '3.12'
jobs:
repository-check:
name: Repository check
validate-release:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
outputs:
is_repo: ${{ steps.repository_check.outputs.is_repo }}
steps:
- name: Repository check
id: repository_check
working-directory: /tmp
run: |
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
then
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
else
echo "This action only runs for prowler-cloud/prowler"
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
fi
prowler_version: ${{ steps.parse-version.outputs.version }}
major_version: ${{ steps.parse-version.outputs.major }}
release-prowler-job:
runs-on: ubuntu-latest
needs: repository-check
if: needs.repository-check.outputs.is_repo == 'true'
env:
POETRY_VIRTUALENVS_CREATE: "false"
name: Release Prowler to PyPI
steps:
- name: Repository check
working-directory: /tmp
run: |
if [[ "${{ github.repository }}" != "prowler-cloud/prowler" ]]; then
echo "This action only runs for prowler-cloud/prowler"
exit 1
fi
- name: Get Prowler version
- name: Parse and validate version
id: parse-version
run: |
PROWLER_VERSION="${{ env.RELEASE_TAG }}"
echo "version=${PROWLER_VERSION}" >> "${GITHUB_OUTPUT}"
case ${PROWLER_VERSION%%.*} in
3)
echo "Releasing Prowler v3 with tag ${PROWLER_VERSION}"
# Extract major version
MAJOR_VERSION="${PROWLER_VERSION%%.*}"
echo "major=${MAJOR_VERSION}" >> "${GITHUB_OUTPUT}"
# Validate major version
case ${MAJOR_VERSION} in
3|4|5)
echo "✓ Releasing Prowler v${MAJOR_VERSION} with tag ${PROWLER_VERSION}"
;;
4)
echo "Releasing Prowler v4 with tag ${PROWLER_VERSION}"
;;
5)
echo "Releasing Prowler v5 with tag ${PROWLER_VERSION}"
;;
*)
echo "Releasing another Prowler major version, aborting..."
*)
echo "::error::Unsupported Prowler major version: ${MAJOR_VERSION}"
exit 1
;;
esac
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
publish-prowler:
needs: validate-release
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
id-token: write
environment:
name: pypi-prowler
url: https://pypi.org/project/prowler/${{ needs.validate-release.outputs.prowler_version }}/
- name: Install dependencies
run: |
pipx install poetry==2.1.1
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Setup Python
- name: Install Poetry
run: pipx install poetry==2.1.1
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
# cache: ${{ env.CACHE }}
cache: 'poetry'
- name: Build Prowler package
run: |
poetry build
run: poetry build
- name: Publish Prowler package to PyPI
run: |
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
poetry publish
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with:
print-hash: true
- name: Replicate PyPI package
publish-prowler-cloud:
needs: validate-release
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
id-token: write
environment:
name: pypi-prowler-cloud
url: https://pypi.org/project/prowler-cloud/${{ needs.validate-release.outputs.prowler_version }}/
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Install Poetry
run: pipx install poetry==2.1.1
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'poetry'
- name: Install toml package
run: pip install toml
- name: Replicate PyPI package for prowler-cloud
run: |
rm -rf ./dist && rm -rf ./build && rm -rf prowler.egg-info
pip install toml
rm -rf ./dist ./build prowler.egg-info
python util/replicate_pypi_package.py
poetry build
- name: Build prowler-cloud package
run: poetry build
- name: Publish prowler-cloud package to PyPI
run: |
poetry config pypi-token.pypi ${{ secrets.PYPI_API_TOKEN }}
poetry publish
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0
with:
print-hash: true
@@ -1,68 +1,90 @@
# This is a basic workflow to help you get started with Actions
name: SDK - Refresh AWS services' regions
name: 'SDK: Refresh AWS Regions'
on:
schedule:
- cron: "0 9 * * 1" # runs at 09:00 UTC every Monday
- cron: '0 9 * * 1' # Every Monday at 09:00 UTC
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: false
env:
GITHUB_BRANCH: "master"
AWS_REGION_DEV: us-east-1
PYTHON_VERSION: '3.12'
AWS_REGION: 'us-east-1'
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
refresh-aws-regions:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
id-token: write
pull-requests: write
contents: write
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: ${{ env.GITHUB_BRANCH }}
- name: setup python
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
ref: 'master'
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: 3.9 #install the python needed
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install boto3
run: pip install boto3
- name: Configure AWS Credentials -- DEV
uses: aws-actions/configure-aws-credentials@a03048d87541d1d9fcf2ecf528a4a65ba9bd7838 # v5.0.0
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@00943011d9042930efac3dcd3a170e4273319bc8 # v5.1.0
with:
aws-region: ${{ env.AWS_REGION_DEV }}
aws-region: ${{ env.AWS_REGION }}
role-to-assume: ${{ secrets.DEV_IAM_ROLE_ARN }}
role-session-name: refresh-AWS-regions-dev
role-session-name: prowler-refresh-aws-regions
# Runs a single command using the runners shell
- name: Run a one-line script
run: python3 util/update_aws_services_regions.py
- name: Update AWS services regions
run: python util/update_aws_services_regions.py
# Create pull request
- name: Create Pull Request
- name: Create pull request
id: create-pr
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: "feat(regions_update): Update regions for AWS services"
branch: "aws-services-regions-updated-${{ github.sha }}"
labels: "status/waiting-for-revision, severity/low, provider/aws, no-changelog"
title: "chore(regions_update): Changes in regions for AWS services"
author: 'prowler-bot <179230569+prowler-bot@users.noreply.github.com>'
committer: 'github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>'
commit-message: 'feat(aws): update regions for AWS services'
branch: 'aws-regions-update-${{ github.run_number }}'
title: 'feat(aws): Update regions for AWS services'
labels: |
status/waiting-for-revision
severity/low
provider/aws
no-changelog
body: |
### Description
This PR updates the regions for AWS services.
Automated update of AWS service regions from the official AWS IP ranges.
**Trigger:** ${{ github.event_name == 'schedule' && 'Scheduled (weekly)' || github.event_name == 'workflow_dispatch' && 'Manual' || 'Workflow update' }}
**Run:** [#${{ github.run_number }}](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})
### Checklist
- [x] This is an automated update from AWS official sources
- [x] No manual review of region data required
### License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
- name: PR creation result
run: |
if [[ "${{ steps.create-pr.outputs.pull-request-number }}" ]]; then
echo "✓ Pull request #${{ steps.create-pr.outputs.pull-request-number }} created successfully"
echo "URL: ${{ steps.create-pr.outputs.pull-request-url }}"
else
echo "✓ No changes detected - AWS regions are up to date"
fi
+77
View File
@@ -0,0 +1,77 @@
name: 'SDK: Security'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
sdk-security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for SDK changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ./**
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
- name: Install Poetry
if: steps.check-changes.outputs.any_changed == 'true'
run: pipx install poetry==2.1.1
- name: Set up Python 3.12
if: steps.check-changes.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
cache: 'poetry'
- name: Install dependencies
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry install --no-root
- name: Security scan with Bandit
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run bandit -q -lll -x '*_test.py,./contrib/,./api/,./ui' -r .
- name: Security scan with Safety
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run safety check --ignore 70612 -r pyproject.toml
- name: Dead code detection with Vulture
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry run vulture --exclude "contrib,api,ui" --min-confidence 100 .
+360
View File
@@ -0,0 +1,360 @@
name: 'SDK: Tests'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
sdk-tests:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 120
permissions:
contents: read
strategy:
matrix:
python-version:
- '3.9'
- '3.10'
- '3.11'
- '3.12'
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for SDK changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ./**
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
- name: Install Poetry
if: steps.check-changes.outputs.any_changed == 'true'
run: pipx install poetry==2.1.1
- name: Set up Python ${{ matrix.python-version }}
if: steps.check-changes.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: ${{ matrix.python-version }}
cache: 'poetry'
- name: Install dependencies
if: steps.check-changes.outputs.any_changed == 'true'
run: poetry install --no-root
# AWS Provider
- name: Check if AWS files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-aws
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/aws/**
./tests/**/aws/**
./poetry.lock
- name: Run AWS tests
if: steps.changed-aws.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/aws --cov-report=xml:aws_coverage.xml tests/providers/aws
- name: Upload AWS coverage to Codecov
if: steps.changed-aws.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-aws
files: ./aws_coverage.xml
# Azure Provider
- name: Check if Azure files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-azure
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/azure/**
./tests/**/azure/**
./poetry.lock
- name: Run Azure tests
if: steps.changed-azure.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/azure --cov-report=xml:azure_coverage.xml tests/providers/azure
- name: Upload Azure coverage to Codecov
if: steps.changed-azure.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-azure
files: ./azure_coverage.xml
# GCP Provider
- name: Check if GCP files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-gcp
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/gcp/**
./tests/**/gcp/**
./poetry.lock
- name: Run GCP tests
if: steps.changed-gcp.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/gcp --cov-report=xml:gcp_coverage.xml tests/providers/gcp
- name: Upload GCP coverage to Codecov
if: steps.changed-gcp.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-gcp
files: ./gcp_coverage.xml
# Kubernetes Provider
- name: Check if Kubernetes files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-kubernetes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/kubernetes/**
./tests/**/kubernetes/**
./poetry.lock
- name: Run Kubernetes tests
if: steps.changed-kubernetes.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/kubernetes --cov-report=xml:kubernetes_coverage.xml tests/providers/kubernetes
- name: Upload Kubernetes coverage to Codecov
if: steps.changed-kubernetes.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-kubernetes
files: ./kubernetes_coverage.xml
# GitHub Provider
- name: Check if GitHub files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-github
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/github/**
./tests/**/github/**
./poetry.lock
- name: Run GitHub tests
if: steps.changed-github.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/github --cov-report=xml:github_coverage.xml tests/providers/github
- name: Upload GitHub coverage to Codecov
if: steps.changed-github.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-github
files: ./github_coverage.xml
# NHN Provider
- name: Check if NHN files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-nhn
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/nhn/**
./tests/**/nhn/**
./poetry.lock
- name: Run NHN tests
if: steps.changed-nhn.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/nhn --cov-report=xml:nhn_coverage.xml tests/providers/nhn
- name: Upload NHN coverage to Codecov
if: steps.changed-nhn.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-nhn
files: ./nhn_coverage.xml
# M365 Provider
- name: Check if M365 files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-m365
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/m365/**
./tests/**/m365/**
./poetry.lock
- name: Run M365 tests
if: steps.changed-m365.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/m365 --cov-report=xml:m365_coverage.xml tests/providers/m365
- name: Upload M365 coverage to Codecov
if: steps.changed-m365.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-m365
files: ./m365_coverage.xml
# IaC Provider
- name: Check if IaC files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-iac
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/iac/**
./tests/**/iac/**
./poetry.lock
- name: Run IaC tests
if: steps.changed-iac.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/iac --cov-report=xml:iac_coverage.xml tests/providers/iac
- name: Upload IaC coverage to Codecov
if: steps.changed-iac.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-iac
files: ./iac_coverage.xml
# MongoDB Atlas Provider
- name: Check if MongoDB Atlas files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-mongodbatlas
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/mongodbatlas/**
./tests/**/mongodbatlas/**
./poetry.lock
- name: Run MongoDB Atlas tests
if: steps.changed-mongodbatlas.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/mongodbatlas --cov-report=xml:mongodbatlas_coverage.xml tests/providers/mongodbatlas
- name: Upload MongoDB Atlas coverage to Codecov
if: steps.changed-mongodbatlas.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-mongodbatlas
files: ./mongodbatlas_coverage.xml
# OCI Provider
- name: Check if OCI files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-oraclecloud
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/**/oraclecloud/**
./tests/**/oraclecloud/**
./poetry.lock
- name: Run OCI tests
if: steps.changed-oraclecloud.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/providers/oraclecloud --cov-report=xml:oraclecloud_coverage.xml tests/providers/oraclecloud
- name: Upload OCI coverage to Codecov
if: steps.changed-oraclecloud.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-oraclecloud
files: ./oraclecloud_coverage.xml
# Lib
- name: Check if Lib files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-lib
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/lib/**
./tests/lib/**
./poetry.lock
- name: Run Lib tests
if: steps.changed-lib.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/lib --cov-report=xml:lib_coverage.xml tests/lib
- name: Upload Lib coverage to Codecov
if: steps.changed-lib.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-lib
files: ./lib_coverage.xml
# Config
- name: Check if Config files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-config
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
./prowler/config/**
./tests/config/**
./poetry.lock
- name: Run Config tests
if: steps.changed-config.outputs.any_changed == 'true'
run: poetry run pytest -n auto --cov=./prowler/config --cov-report=xml:config_coverage.xml tests/config
- name: Upload Config coverage to Codecov
if: steps.changed-config.outputs.any_changed == 'true'
uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-config
files: ./config_coverage.xml
@@ -1,121 +0,0 @@
name: UI - Build and Push containers
on:
push:
branches:
- "master"
paths:
- "ui/**"
- ".github/workflows/ui-build-lint-push-containers.yml"
# Uncomment the below code to test this action on PRs
# pull_request:
# branches:
# - "master"
# paths:
# - "ui/**"
# - ".github/workflows/ui-build-lint-push-containers.yml"
release:
types: [published]
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./ui
# Container Registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-ui
NEXT_PUBLIC_API_BASE_URL: http://prowler-api:8080/api/v1
jobs:
repository-check:
name: Repository check
runs-on: ubuntu-latest
outputs:
is_repo: ${{ steps.repository_check.outputs.is_repo }}
steps:
- name: Repository check
id: repository_check
working-directory: /tmp
run: |
if [[ ${{ github.repository }} == "prowler-cloud/prowler" ]]
then
echo "is_repo=true" >> "${GITHUB_OUTPUT}"
else
echo "This action only runs for prowler-cloud/prowler"
echo "is_repo=false" >> "${GITHUB_OUTPUT}"
fi
# Build Prowler OSS container
container-build-push:
needs: repository-check
if: needs.repository-check.outputs.is_repo == 'true'
runs-on: ubuntu-latest
defaults:
run:
working-directory: ${{ env.WORKING_DIRECTORY }}
steps:
- name: Checkout
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set short git commit SHA
id: vars
run: |
shortSha=$(git rev-parse --short ${{ github.sha }})
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push container image (latest)
# Comment the following line for testing
if: github.event_name == 'push'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=${{ env.SHORT_SHA }}
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
# Set push: false for testing
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.SHORT_SHA }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v${{ env.RELEASE_TAG }}
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
push: true
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }}
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Trigger deployment
if: github.event_name == 'push'
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: prowler-ui-deploy
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ env.SHORT_SHA }}"}'
+4 -3
View File
@@ -27,7 +27,8 @@ concurrency:
cancel-in-progress: true
jobs:
analyze:
ui-analyze:
if: github.repository == 'prowler-cloud/prowler'
name: CodeQL Security Analysis
runs-on: ubuntu-latest
timeout-minutes: 30
@@ -47,12 +48,12 @@ jobs:
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Initialize CodeQL
uses: github/codeql-action/init@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/init@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/ui-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@3599b3baa15b485a2e49ef411a7a4bb2452e7f93 # v3.30.5
uses: github/codeql-action/analyze@0499de31b99561a6d14a36a5f662c2a54f91beee # v4.31.2
with:
category: '/language:${{ matrix.language }}'
@@ -0,0 +1,189 @@
name: 'UI: Container Build and Push'
on:
push:
branches:
- 'master'
paths:
- 'ui/**'
- '.github/workflows/ui-container-build-push.yml'
release:
types:
- 'published'
workflow_dispatch:
inputs:
release_tag:
description: 'Release tag (e.g., 5.14.0)'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false
env:
# Tags
LATEST_TAG: latest
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.release_tag }}
STABLE_TAG: stable
WORKING_DIRECTORY: ./ui
# Container registries
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-ui
# Build args
NEXT_PUBLIC_API_BASE_URL: http://prowler-api:8080/api/v1
jobs:
setup:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
outputs:
short-sha: ${{ steps.set-short-sha.outputs.short-sha }}
steps:
- name: Calculate short SHA
id: set-short-sha
run: echo "short-sha=${GITHUB_SHA::7}" >> $GITHUB_OUTPUT
container-build-push:
needs: setup
runs-on: ${{ matrix.runner }}
strategy:
matrix:
include:
- platform: linux/amd64
runner: ubuntu-latest
arch: amd64
- platform: linux/arm64
runner: ubuntu-24.04-arm
arch: arm64
timeout-minutes: 30
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Login to DockerHub
uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Notify container push started
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: UI
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-started.json"
- name: Build and push UI container for ${{ matrix.arch }}
id: container-push
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=${{ (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && format('v{0}', env.RELEASE_TAG) || needs.setup.outputs.short-sha }}
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
push: true
platforms: ${{ matrix.platform }}
tags: |
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-${{ matrix.arch }}
cache-from: type=gha,scope=${{ matrix.arch }}
cache-to: type=gha,mode=max,scope=${{ matrix.arch }}
- name: Notify container push completed
if: (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && always()
uses: ./.github/actions/slack-notification
env:
SLACK_CHANNEL_ID: ${{ secrets.SLACK_PLATFORM_DEPLOYMENTS }}
COMPONENT: UI
RELEASE_TAG: ${{ env.RELEASE_TAG }}
GITHUB_SERVER_URL: ${{ github.server_url }}
GITHUB_REPOSITORY: ${{ github.repository }}
GITHUB_RUN_ID: ${{ github.run_id }}
with:
slack-bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.container-push.outcome }}
# Create and push multi-architecture manifest
create-manifest:
needs: [setup, container-build-push]
if: github.event_name == 'push' || github.event_name == 'release' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Login to DockerHub
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Create and push manifests for push event
if: github.event_name == 'push'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.LATEST_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Create and push manifests for release event
if: github.event_name == 'release' || github.event_name == 'workflow_dispatch'
run: |
docker buildx imagetools create \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.RELEASE_TAG }} \
-t ${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ env.STABLE_TAG }} \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64 \
${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@main
- name: Cleanup intermediate architecture tags
if: always()
run: |
echo "Cleaning up intermediate tags..."
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-amd64" || true
regctl tag delete "${{ env.PROWLERCLOUD_DOCKERHUB_REPOSITORY }}/${{ env.PROWLERCLOUD_DOCKERHUB_IMAGE }}:${{ needs.setup.outputs.short-sha }}-arm64" || true
echo "Cleanup completed"
trigger-deployment:
if: github.event_name == 'push'
needs: [setup, container-build-push]
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Trigger UI deployment
uses: peter-evans/repository-dispatch@5fc4efd1a4797ddb68ffd0714a238564e4cc0e6f # v4.0.0
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: ui-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+91
View File
@@ -0,0 +1,91 @@
name: 'UI: Container Checks'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
UI_WORKING_DIR: ./ui
IMAGE_NAME: prowler-ui
jobs:
ui-dockerfile-lint:
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check if Dockerfile changed
id: dockerfile-changed
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ui/Dockerfile
- name: Lint Dockerfile with Hadolint
if: steps.dockerfile-changed.outputs.any_changed == 'true'
uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0
with:
dockerfile: ui/Dockerfile
ignore: DL3018
ui-container-build-and-scan:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
security-events: write
pull-requests: write
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for UI changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: ui/**
files_ignore: |
ui/CHANGELOG.md
ui/README.md
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build UI container
if: steps.check-changes.outputs.any_changed == 'true'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.UI_WORKING_DIR }}
target: prod
push: false
load: true
tags: ${{ env.IMAGE_NAME }}:${{ github.sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_51LwpXXXX
- name: Scan UI container with Trivy
if: github.repository == 'prowler-cloud/prowler' && steps.check-changes.outputs.any_changed == 'true'
uses: ./.github/actions/trivy-scan
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
severity: 'CRITICAL'
+16 -1
View File
@@ -18,6 +18,21 @@ jobs:
AUTH_TRUST_HOST: true
NEXTAUTH_URL: 'http://localhost:3000'
NEXT_PUBLIC_API_BASE_URL: 'http://localhost:8080/api/v1'
E2E_ADMIN_USER: ${{ secrets.E2E_ADMIN_USER }}
E2E_ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD }}
E2E_AWS_PROVIDER_ACCOUNT_ID: ${{ secrets.E2E_AWS_PROVIDER_ACCOUNT_ID }}
E2E_AWS_PROVIDER_ACCESS_KEY: ${{ secrets.E2E_AWS_PROVIDER_ACCESS_KEY }}
E2E_AWS_PROVIDER_SECRET_KEY: ${{ secrets.E2E_AWS_PROVIDER_SECRET_KEY }}
E2E_AWS_PROVIDER_ROLE_ARN: ${{ secrets.E2E_AWS_PROVIDER_ROLE_ARN }}
E2E_AZURE_SUBSCRIPTION_ID: ${{ secrets.E2E_AZURE_SUBSCRIPTION_ID }}
E2E_AZURE_CLIENT_ID: ${{ secrets.E2E_AZURE_CLIENT_ID }}
E2E_AZURE_SECRET_ID: ${{ secrets.E2E_AZURE_SECRET_ID }}
E2E_AZURE_TENANT_ID: ${{ secrets.E2E_AZURE_TENANT_ID }}
E2E_M365_DOMAIN_ID: ${{ secrets.E2E_M365_DOMAIN_ID }}
E2E_M365_CLIENT_ID: ${{ secrets.E2E_M365_CLIENT_ID }}
E2E_M365_SECRET_ID: ${{ secrets.E2E_M365_SECRET_ID }}
E2E_M365_TENANT_ID: ${{ secrets.E2E_M365_TENANT_ID }}
E2E_M365_CERTIFICATE_CONTENT: ${{ secrets.E2E_M365_CERTIFICATE_CONTENT }}
E2E_NEW_PASSWORD: ${{ secrets.E2E_NEW_PASSWORD }}
steps:
- name: Checkout repository
@@ -60,7 +75,7 @@ jobs:
echo "All database fixtures loaded successfully!"
'
- name: Setup Node.js environment
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
with:
node-version: '20.x'
cache: 'npm'
-65
View File
@@ -1,65 +0,0 @@
name: UI - Pull Request
on:
push:
branches:
- "master"
- "v5.*"
paths:
- ".github/workflows/ui-pull-request.yml"
- "ui/**"
pull_request:
branches:
- master
- "v5.*"
paths:
- 'ui/**'
env:
UI_WORKING_DIR: ./ui
IMAGE_NAME: prowler-ui
jobs:
test-and-coverage:
runs-on: ubuntu-latest
strategy:
matrix:
os: [ubuntu-latest]
node-version: [20.x]
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
persist-credentials: false
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@a0853c24544627f65ddf259abe73b1d18a591444 # v5.0.0
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: './ui/package-lock.json'
- name: Install dependencies
working-directory: ./ui
run: npm ci
- name: Run Healthcheck
working-directory: ./ui
run: npm run healthcheck
- name: Build the application
working-directory: ./ui
run: npm run build
test-container-build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build Container
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.UI_WORKING_DIR }}
# Always build using `prod` target
target: prod
push: false
tags: ${{ env.IMAGE_NAME }}:latest
outputs: type=docker
build-args: |
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_51LwpXXXX
+64
View File
@@ -0,0 +1,64 @@
name: 'UI: Tests'
on:
push:
branches:
- 'master'
- 'v5.*'
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
UI_WORKING_DIR: ./ui
NODE_VERSION: '20.x'
jobs:
ui-tests:
runs-on: ubuntu-latest
timeout-minutes: 20
permissions:
contents: read
defaults:
run:
working-directory: ./ui
steps:
- name: Checkout repository
uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
- name: Check for UI changes
id: check-changes
uses: tj-actions/changed-files@24d32ffd492484c1d75e0c0b894501ddb9d30d62 # v47.0.0
with:
files: |
ui/**
.github/workflows/ui-tests.yml
files_ignore: |
ui/CHANGELOG.md
ui/README.md
- name: Setup Node.js ${{ env.NODE_VERSION }}
if: steps.check-changes.outputs.any_changed == 'true'
uses: actions/setup-node@2028fbc5c25fe9cf00d9f06a71cc4710d4507903 # v6.0.0
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
cache-dependency-path: './ui/package-lock.json'
- name: Install dependencies
if: steps.check-changes.outputs.any_changed == 'true'
run: npm ci
- name: Run healthcheck
if: steps.check-changes.outputs.any_changed == 'true'
run: npm run healthcheck
- name: Build application
if: steps.check-changes.outputs.any_changed == 'true'
run: npm run build
+69 -5
View File
@@ -45,21 +45,86 @@ pytest_*.xml
.coverage
htmlcov/
# VSCode files
# VSCode files and settings
.vscode/
*.code-workspace
.vscode-test/
# Cursor files
# VSCode extension settings and workspaces
.history/
.ionide/
# MCP Server Settings (various locations)
**/cline_mcp_settings.json
**/mcp_settings.json
**/mcp-config.json
**/mcpServers.json
.mcp/
# AI Coding Assistants - Cursor
.cursorignore
.cursor/
.cursorrules
# RooCode files
# AI Coding Assistants - RooCode
.roo/
.rooignore
.roomodes
# Cline files
# AI Coding Assistants - Cline (formerly Claude Dev)
.cline/
.clineignore
.clinerules
# AI Coding Assistants - Continue
.continue/
continue.json
.continuerc
.continuerc.json
# AI Coding Assistants - GitHub Copilot
.copilot/
.github/copilot/
# AI Coding Assistants - Amazon Q Developer (formerly CodeWhisperer)
.aws/
.codewhisperer/
.amazonq/
.aws-toolkit/
# AI Coding Assistants - Tabnine
.tabnine/
tabnine_config.json
# AI Coding Assistants - Kiro
.kiro/
.kiroignore
kiro.config.json
# AI Coding Assistants - Aider
.aider/
.aider.chat.history.md
.aider.input.history
.aider.tags.cache.v3/
# AI Coding Assistants - Windsurf
.windsurf/
.windsurfignore
# AI Coding Assistants - Replit Agent
.replit
.replitignore
# AI Coding Assistants - Supermaven
.supermaven/
# AI Coding Assistants - Sourcegraph Cody
.cody/
# AI Coding Assistants - General
.ai/
.aiconfig
ai-config.json
# Terraform
.terraform*
@@ -70,7 +135,6 @@ htmlcov/
ui/.env*
api/.env*
mcp_server/.env*
.env.local
# Coverage
.coverage*
+9
View File
@@ -126,3 +126,12 @@ repos:
entry: bash -c 'vulture --exclude "contrib,.venv,api/src/backend/api/tests/,api/src/backend/conftest.py,api/src/backend/tasks/tests/" --min-confidence 100 .'
language: system
files: '.*\.py'
- id: ui-checks
name: UI - Husky Pre-commit
description: "Run UI pre-commit checks (Claude Code validation + healthcheck)"
entry: bash -c 'cd ui && .husky/pre-commit'
language: system
files: '^ui/.*\.(ts|tsx|js|jsx|json|css)$'
pass_filenames: false
verbose: true
+1 -1
View File
@@ -10,4 +10,4 @@
Want some swag as appreciation for your contribution?
# Prowler Developer Guide
https://docs.prowler.com/projects/prowler-open-source/en/latest/developer-guide/introduction/
https://goto.prowler.com/devguide
+22
View File
@@ -4,6 +4,10 @@ LABEL maintainer="https://github.com/prowler-cloud/prowler"
LABEL org.opencontainers.image.source="https://github.com/prowler-cloud/prowler"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
ARG TRIVY_VERSION=0.66.0
ENV TRIVY_VERSION=${TRIVY_VERSION}
# hadolint ignore=DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
@@ -25,6 +29,24 @@ RUN ARCH=$(uname -m) && \
ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh && \
rm /tmp/powershell.tar.gz
# Install Trivy for IaC scanning
RUN ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
TRIVY_ARCH="Linux-64bit" ; \
elif [ "$ARCH" = "aarch64" ]; then \
TRIVY_ARCH="Linux-ARM64" ; \
else \
echo "Unsupported architecture for Trivy: $ARCH" && exit 1 ; \
fi && \
wget --progress=dot:giga "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_${TRIVY_ARCH}.tar.gz" -O /tmp/trivy.tar.gz && \
tar zxf /tmp/trivy.tar.gz -C /tmp && \
mv /tmp/trivy /usr/local/bin/trivy && \
chmod +x /usr/local/bin/trivy && \
rm /tmp/trivy.tar.gz && \
# Create trivy cache directory with proper permissions
mkdir -p /tmp/.cache/trivy && \
chmod 777 /tmp/.cache/trivy
# Add prowler user
RUN addgroup --gid 1000 prowler && \
adduser --uid 1000 --gid 1000 --disabled-password --gecos "" prowler
+31 -14
View File
@@ -56,7 +56,7 @@ Prowler includes hundreds of built-in controls to ensure compliance with standar
Prowler App is a web-based application that simplifies running Prowler across your cloud provider accounts. It provides a user-friendly interface to visualize the results and streamline your security assessments.
![Prowler App](docs/products/img/overview.png)
![Prowler App](docs/images/products/overview.png)
>For more details, refer to the [Prowler App Documentation](https://docs.prowler.com/projects/prowler-open-source/en/latest/#prowler-app-installation)
@@ -73,26 +73,43 @@ prowler <provider>
```console
prowler dashboard
```
![Prowler Dashboard](docs/products/img/dashboard.png)
![Prowler Dashboard](docs/images/products/dashboard.png)
## Attack Paths
Attack Paths automatically extends every completed AWS scan with a Neo4j graph that combines Cartography's cloud inventory with Prowler findings. The feature runs in the API worker after each scan and therefore requires:
- An accessible Neo4j instance (the Docker Compose files already ships a `neo4j` service).
- The following environment variables so Django and Celery can connect:
| Variable | Description | Default |
| --- | --- | --- |
| `NEO4J_HOST` | Hostname used by the API containers. | `neo4j` |
| `NEO4J_PORT` | Bolt port exposed by Neo4j. | `7687` |
| `NEO4J_USER` / `NEO4J_PASSWORD` | Credentials with rights to create per-tenant databases. | `neo4j` / `neo4j_password` |
Every AWS provider scan will enqueue an Attack Paths ingestion job automatically. Other cloud providers will be added in future iterations.
# Prowler at a Glance
> [!Tip]
> For the most accurate and up-to-date information about checks, services, frameworks, and categories, visit [**Prowler Hub**](https://hub.prowler.com).
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Stage | Interface |
|---|---|---|---|---|---|---|---|
| AWS | 576 | 82 | 38 | 10 | Official | Stable | UI, API, CLI |
| GCP | 79 | 13 | 11 | 3 | Official | Stable | UI, API, CLI |
| Azure | 162 | 19 | 12 | 4 | Official | Stable | UI, API, CLI |
| Kubernetes | 83 | 7 | 5 | 7 | Official | Stable | UI, API, CLI |
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Interface |
|---|---|---|---|---|---|---|
| AWS | 576 | 82 | 39 | 10 | Official | UI, API, CLI |
| GCP | 79 | 13 | 13 | 3 | Official | UI, API, CLI |
| Azure | 162 | 19 | 13 | 4 | Official | UI, API, CLI |
| Kubernetes | 83 | 7 | 5 | 7 | Official | UI, API, CLI |
| GitHub | 17 | 2 | 1 | 0 | Official | Stable | UI, API, CLI |
| M365 | 70 | 7 | 3 | 2 | Official | Stable | UI, API, CLI |
| OCI | 51 | 13 | 1 | 10 | Official | Stable | CLI |
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | Beta | CLI |
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | Beta | CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | Beta | CLI |
| NHN | 6 | 2 | 1 | 0 | Unofficial | Beta | CLI |
| M365 | 70 | 7 | 3 | 2 | Official | UI, API, CLI |
| OCI | 51 | 13 | 1 | 10 | Official | UI, API, CLI |
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | UI, API, CLI |
| MongoDB Atlas | 10 | 3 | 0 | 0 | Official | UI, API, CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | CLI |
| NHN | 6 | 2 | 1 | 0 | Unofficial | CLI |
> [!Note]
> The numbers in the table are updated periodically.
+68 -8
View File
@@ -2,12 +2,72 @@
All notable changes to the **Prowler API** are documented in this file.
## [1.15.0] (Prowler UNRELEASED)
## [1.16.0] (Unreleased)
### Added
- Support for configuring multiple LLM providers [(#8772)](https://github.com/prowler-cloud/prowler/pull/8772)
- Attack Paths backend support [(#9344)](https://github.com/prowler-cloud/prowler/pull/9344)
## [1.14.0] (Prowler 5.13.0)
### Changed
- Restore the compliance overview endpoint's mandatory filters [(#9330)](https://github.com/prowler-cloud/prowler/pull/9330)
---
## [1.15.1] (Prowler v5.14.1)
### Fixed
- Fix typo in PDF reporting [(#9345)](https://github.com/prowler-cloud/prowler/pull/9345)
- Fix IaC provider initialization failure when mutelist processor is configured [(#9331)](https://github.com/prowler-cloud/prowler/pull/9331)
---
## [1.15.0] (Prowler v5.14.0)
### Added
- IaC (Infrastructure as Code) provider support for remote repositories [(#8751)](https://github.com/prowler-cloud/prowler/pull/8751)
- Extend `GET /api/v1/providers` with provider-type filters and optional pagination disable to support the new Overview filters [(#8975)](https://github.com/prowler-cloud/prowler/pull/8975)
- New endpoint to retrieve the number of providers grouped by provider type [(#8975)](https://github.com/prowler-cloud/prowler/pull/8975)
- Support for configuring multiple LLM providers [(#8772)](https://github.com/prowler-cloud/prowler/pull/8772)
- Support C5 compliance framework for Azure provider [(#9081)](https://github.com/prowler-cloud/prowler/pull/9081)
- Support for Oracle Cloud Infrastructure (OCI) provider [(#8927)](https://github.com/prowler-cloud/prowler/pull/8927)
- Support muting findings based on simple rules with custom reason [(#9051)](https://github.com/prowler-cloud/prowler/pull/9051)
- Support C5 compliance framework for the GCP provider [(#9097)](https://github.com/prowler-cloud/prowler/pull/9097)
- Support for Amazon Bedrock and OpenAI compatible providers in Lighthouse AI [(#8957)](https://github.com/prowler-cloud/prowler/pull/8957)
- Support PDF reporting for ENS compliance framework [(#9158)](https://github.com/prowler-cloud/prowler/pull/9158)
- Support PDF reporting for NIS2 compliance framework [(#9170)](https://github.com/prowler-cloud/prowler/pull/9170)
- Tenant-wide ThreatScore overview aggregation and snapshot persistence with backfill support [(#9148)](https://github.com/prowler-cloud/prowler/pull/9148)
- Added `metadata`, `details`, and `partition` attributes to `/resources` endpoint & `details`, and `partition` to `/findings` endpoint [(#9098)](https://github.com/prowler-cloud/prowler/pull/9098)
- Support for MongoDB Atlas provider [(#9167)](https://github.com/prowler-cloud/prowler/pull/9167)
- Support Prowler ThreatScore for the K8S provider [(#9235)](https://github.com/prowler-cloud/prowler/pull/9235)
- Enhanced compliance overview endpoint with provider filtering and latest scan aggregation [(#9244)](https://github.com/prowler-cloud/prowler/pull/9244)
- New endpoint `GET /api/v1/overview/regions` to retrieve aggregated findings data by region [(#9273)](https://github.com/prowler-cloud/prowler/pull/9273)
### Changed
- Optimized database write queries for scan related tasks [(#9190)](https://github.com/prowler-cloud/prowler/pull/9190)
- Date filters are now optional for `GET /api/v1/overviews/services` endpoint; returns latest scan data by default [(#9248)](https://github.com/prowler-cloud/prowler/pull/9248)
### Fixed
- Scans no longer fail when findings have UIDs exceeding 300 characters; such findings are now skipped with detailed logging [(#9246)](https://github.com/prowler-cloud/prowler/pull/9246)
- Updated unique constraint for `Provider` model to exclude soft-deleted entries, resolving duplicate errors when re-deleting providers [(#9054)](https://github.com/prowler-cloud/prowler/pull/9054)
- Removed compliance generation for providers without compliance frameworks [(#9208)](https://github.com/prowler-cloud/prowler/pull/9208)
- Refresh output report timestamps for each scan [(#9272)](https://github.com/prowler-cloud/prowler/pull/9272)
- Severity overview endpoint now ignores muted findings as expected [(#9283)](https://github.com/prowler-cloud/prowler/pull/9283)
- Fixed discrepancy between ThreatScore PDF report values and database calculations [(#9296)](https://github.com/prowler-cloud/prowler/pull/9296)
### Security
- Django updated to the latest 5.1 security release, 5.1.14, due to problems with potential [SQL injection](https://github.com/prowler-cloud/prowler/security/dependabot/113) and [denial-of-service vulnerability](https://github.com/prowler-cloud/prowler/security/dependabot/114) [(#9176)](https://github.com/prowler-cloud/prowler/pull/9176)
---
## [1.14.1] (Prowler v5.13.1)
### Fixed
- `/api/v1/overviews/providers` collapses data by provider type so the UI receives a single aggregated record per cloud family even when multiple accounts exist [(#9053)](https://github.com/prowler-cloud/prowler/pull/9053)
- Added retry logic to database transactions to handle Aurora read replica connection failures during scale-down events [(#9064)](https://github.com/prowler-cloud/prowler/pull/9064)
- Security Hub integrations stop failing when they read relationships via the replica by allowing replica relations and saving updates through the primary [(#9080)](https://github.com/prowler-cloud/prowler/pull/9080)
---
## [1.14.0] (Prowler v5.13.0)
### Added
- Default JWT keys are generated and stored if they are missing from configuration [(#8655)](https://github.com/prowler-cloud/prowler/pull/8655)
@@ -31,14 +91,14 @@ All notable changes to the **Prowler API** are documented in this file.
---
## [1.13.2] (Prowler 5.12.3)
## [1.13.2] (Prowler v5.12.3)
### Fixed
- 500 error when deleting user [(#8731)](https://github.com/prowler-cloud/prowler/pull/8731)
---
## [1.13.1] (Prowler 5.12.2)
## [1.13.1] (Prowler v5.12.2)
### Changed
- Renamed compliance overview task queue to `compliance` [(#8755)](https://github.com/prowler-cloud/prowler/pull/8755)
@@ -48,7 +108,7 @@ All notable changes to the **Prowler API** are documented in this file.
---
## [1.13.0] (Prowler 5.12.0)
## [1.13.0] (Prowler v5.12.0)
### Added
- Integration with JIRA, enabling sending findings to a JIRA project [(#8622)](https://github.com/prowler-cloud/prowler/pull/8622), [(#8637)](https://github.com/prowler-cloud/prowler/pull/8637)
@@ -57,7 +117,7 @@ All notable changes to the **Prowler API** are documented in this file.
---
## [1.12.0] (Prowler 5.11.0)
## [1.12.0] (Prowler v5.11.0)
### Added
- Lighthouse support for OpenAI GPT-5 [(#8527)](https://github.com/prowler-cloud/prowler/pull/8527)
@@ -69,7 +129,7 @@ All notable changes to the **Prowler API** are documented in this file.
---
## [1.11.0] (Prowler 5.10.0)
## [1.11.0] (Prowler v5.10.0)
### Added
- Github provider support [(#8271)](https://github.com/prowler-cloud/prowler/pull/8271)
+21
View File
@@ -5,6 +5,9 @@ LABEL maintainer="https://github.com/prowler-cloud/api"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
ARG TRIVY_VERSION=0.66.0
ENV TRIVY_VERSION=${TRIVY_VERSION}
# hadolint ignore=DL3008
RUN apt-get update && apt-get install -y --no-install-recommends \
wget \
@@ -36,6 +39,24 @@ RUN ARCH=$(uname -m) && \
ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh && \
rm /tmp/powershell.tar.gz
# Install Trivy for IaC scanning
RUN ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
TRIVY_ARCH="Linux-64bit" ; \
elif [ "$ARCH" = "aarch64" ]; then \
TRIVY_ARCH="Linux-ARM64" ; \
else \
echo "Unsupported architecture for Trivy: $ARCH" && exit 1 ; \
fi && \
wget --progress=dot:giga "https://github.com/aquasecurity/trivy/releases/download/v${TRIVY_VERSION}/trivy_${TRIVY_VERSION}_${TRIVY_ARCH}.tar.gz" -O /tmp/trivy.tar.gz && \
tar zxf /tmp/trivy.tar.gz -C /tmp && \
mv /tmp/trivy /usr/local/bin/trivy && \
chmod +x /usr/local/bin/trivy && \
rm /tmp/trivy.tar.gz && \
# Create trivy cache directory with proper permissions
mkdir -p /tmp/.cache/trivy && \
chmod 777 /tmp/.cache/trivy
# Add prowler user
RUN addgroup --gid 1000 prowler && \
adduser --uid 1000 --gid 1000 --disabled-password --gecos "" prowler
+1068 -48
View File
File diff suppressed because it is too large. Load diff
+5 -3
View File
@@ -7,7 +7,7 @@ authors = [{name = "Prowler Engineering", email = "engineering@prowler.com"}]
dependencies = [
"celery[pytest] (>=5.4.0,<6.0.0)",
"dj-rest-auth[with_social,jwt] (==7.0.1)",
"django (==5.1.13)",
"django (==5.1.14)",
"django-allauth[saml] (>=65.8.0,<66.0.0)",
"django-celery-beat (>=2.7.0,<3.0.0)",
"django-celery-results (>=2.5.1,<3.0.0)",
@@ -24,7 +24,7 @@ dependencies = [
"drf-spectacular-jsonapi==0.5.1",
"gunicorn==23.0.0",
"lxml==5.3.2",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@master",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@attack-paths-demo",
"psycopg2-binary==2.9.9",
"pytest-celery[redis] (>=1.0.1,<2.0.0)",
"sentry-sdk[django] (>=2.20.0,<3.0.0)",
@@ -35,7 +35,9 @@ dependencies = [
"markdown (>=3.9,<4.0)",
"drf-simple-apikey (==2.2.1)",
"matplotlib (>=3.10.6,<4.0.0)",
"reportlab (>=4.4.4,<5.0.0)"
"reportlab (>=4.4.4,<5.0.0)",
"neo4j (<6.0.0)",
"cartography @ git+https://github.com/prowler-cloud/cartography@master",
]
description = "Prowler's API (Django/DRF)"
license = "Apache-2.0"
-3
View File
@@ -1,3 +0,0 @@
# from django.contrib import admin
# Register your models here.
+7 -1
View File
@@ -1,4 +1,5 @@
import logging
import atexit
import os
import sys
from pathlib import Path
@@ -30,6 +31,7 @@ class ApiConfig(AppConfig):
def ready(self):
from api import schema_extensions # noqa: F401
from api import signals # noqa: F401
from api.attack_paths import database as graph_database
from api.compliance import load_prowler_compliance
# Generate required cryptographic keys if not present, but only if:
@@ -39,6 +41,10 @@ class ApiConfig(AppConfig):
if "manage.py" not in sys.argv or os.environ.get("RUN_MAIN"):
self._ensure_crypto_keys()
if not getattr(settings, "TESTING", False):
graph_database.init_driver()
atexit.register(graph_database.close_driver)
load_prowler_compliance()
def _ensure_crypto_keys(self):
@@ -54,7 +60,7 @@ class ApiConfig(AppConfig):
global _keys_initialized
# Skip key generation if running tests
if hasattr(settings, "TESTING") and settings.TESTING:
if getattr(settings, "TESTING", False):
return
# Skip if already initialized in this process
@@ -0,0 +1,13 @@
from api.attack_paths.query_definitions import (
AttackPathsQueryDefinition,
AttackPathsQueryParameterDefinition,
get_queries_for_provider,
get_query_by_id,
)
__all__ = [
"AttackPathsQueryDefinition",
"AttackPathsQueryParameterDefinition",
"get_queries_for_provider",
"get_query_by_id",
]
@@ -0,0 +1,144 @@
import logging
import threading
from contextlib import contextmanager
from typing import Iterator
from uuid import UUID
import neo4j
import neo4j.exceptions
from django.conf import settings
from api.attack_paths.retryable_session import RetryableSession
# Without this Celery goes crazy with Neo4j logging
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
SERVICE_UNAVAILABLE_MAX_RETRIES = 3
# Module-level process-wide driver singleton
_driver: neo4j.Driver | None = None
_lock = threading.Lock()
# Base Neo4j functions
def get_uri() -> str:
host = settings.DATABASES["neo4j"]["HOST"]
port = settings.DATABASES["neo4j"]["PORT"]
return f"bolt://{host}:{port}"
def init_driver() -> neo4j.Driver:
global _driver
if _driver is not None:
return _driver
with _lock:
if _driver is None:
uri = get_uri()
config = settings.DATABASES["neo4j"]
_driver = neo4j.GraphDatabase.driver(
uri, auth=(config["USER"], config["PASSWORD"])
)
_driver.verify_connectivity()
return _driver
def get_driver() -> neo4j.Driver:
return init_driver()
def close_driver() -> None: # TODO: Use it
global _driver
with _lock:
if _driver is not None:
try:
_driver.close()
finally:
_driver = None
@contextmanager
def get_session(database: str | None = None) -> Iterator[RetryableSession]:
session_wrapper: RetryableSession | None = None
try:
session_wrapper = RetryableSession(
session_factory=lambda: get_driver().session(database=database),
close_driver=close_driver, # Just to avoid circular imports
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
)
yield session_wrapper
except neo4j.exceptions.Neo4jError as exc:
raise GraphDatabaseQueryException(message=exc.message, code=exc.code)
finally:
if session_wrapper is not None:
session_wrapper.close()
def create_database(database: str) -> None:
query = "CREATE DATABASE $database IF NOT EXISTS"
parameters = {"database": database}
with get_session() as session:
session.run(query, parameters)
def drop_database(database: str) -> None:
query = f"DROP DATABASE `{database}` IF EXISTS DESTROY DATA"
with get_session() as session:
session.run(query)
def drop_subgraph(database: str, root_node_label: str, root_node_id: str) -> int:
query = """
MATCH (a:__ROOT_NODE_LABEL__ {id: $root_node_id})
CALL apoc.path.subgraphNodes(a, {})
YIELD node
DETACH DELETE node
RETURN COUNT(node) AS deleted_nodes_count
""".replace("__ROOT_NODE_LABEL__", root_node_label)
parameters = {"root_node_id": root_node_id}
with get_session(database) as session:
result = session.run(query, parameters)
try:
return result.single()["deleted_nodes_count"]
except neo4j.exceptions.ResultConsumedError:
return 0 # As there are no nodes to delete, the result is empty
# Neo4j functions related to Prowler + Cartography
DATABASE_NAME_TEMPLATE = "db-{attack_paths_scan_id}"
def get_database_name(attack_paths_scan_id: UUID) -> str:
attack_paths_scan_id_str = str(attack_paths_scan_id).lower()
return DATABASE_NAME_TEMPLATE.format(attack_paths_scan_id=attack_paths_scan_id_str)
# Exceptions
class GraphDatabaseQueryException(Exception):
def __init__(self, message: str, code: str | None = None) -> None:
super().__init__(message)
self.message = message
self.code = code
def __str__(self) -> str:
if self.code:
return f"{self.code}: {self.message}"
return self.message
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,87 @@
import logging
from collections.abc import Callable
from typing import Any
import neo4j
import neo4j.exceptions
logger = logging.getLogger(__name__)
class RetryableSession:
"""
Wrapper around `neo4j.Session` that retries `neo4j.exceptions.ServiceUnavailable` errors.
"""
def __init__(
self,
session_factory: Callable[[], neo4j.Session],
close_driver: Callable[[], None], # Just to avoid circular imports
max_retries: int,
) -> None:
self._session_factory = session_factory
self._close_driver = close_driver
self._max_retries = max(0, max_retries)
self._session = self._session_factory()
def close(self) -> None:
if self._session is not None:
self._session.close()
self._session = None
def __enter__(self) -> "RetryableSession":
return self
def __exit__(self, exc_type: Any, exc: Any, exc_tb: Any) -> None:
self.close()
def run(self, *args: Any, **kwargs: Any) -> Any:
return self._call_with_retry("run", *args, **kwargs)
def write_transaction(self, *args: Any, **kwargs: Any) -> Any:
return self._call_with_retry("write_transaction", *args, **kwargs)
def read_transaction(self, *args: Any, **kwargs: Any) -> Any:
return self._call_with_retry("read_transaction", *args, **kwargs)
def execute_write(self, *args: Any, **kwargs: Any) -> Any:
return self._call_with_retry("execute_write", *args, **kwargs)
def execute_read(self, *args: Any, **kwargs: Any) -> Any:
return self._call_with_retry("execute_read", *args, **kwargs)
def __getattr__(self, item: str) -> Any:
return getattr(self._session, item)
def _call_with_retry(self, method_name: str, *args: Any, **kwargs: Any) -> Any:
attempt = 0
last_exc: neo4j.exceptions.ServiceUnavailable | None = None
while attempt <= self._max_retries:
try:
method = getattr(self._session, method_name)
return method(*args, **kwargs)
except (
neo4j.exceptions.ServiceUnavailable
) as exc: # pragma: no cover - depends on infra
last_exc = exc
attempt += 1
if attempt > self._max_retries:
raise
logger.warning(
f"Neo4j session {method_name} failed with ServiceUnavailable ({attempt}/{self._max_retries} attempts). Retrying..."
)
self._refresh_session()
raise last_exc if last_exc else RuntimeError("Unexpected retry loop exit")
def _refresh_session(self) -> None:
if self._session is not None:
self._session.close()
self._close_driver()
self._session = self._session_factory()
@@ -0,0 +1,143 @@
import logging
from typing import Any
from rest_framework.exceptions import APIException, ValidationError
from api.attack_paths import database as graph_database, AttackPathsQueryDefinition
from api.models import AttackPathsScan
from config.custom_logging import BackendLogger
logger = logging.getLogger(BackendLogger.API)
def normalize_run_payload(raw_data):
if not isinstance(raw_data, dict): # Let the serializer handle this
return raw_data
if "data" in raw_data and isinstance(raw_data.get("data"), dict):
data_section = raw_data.get("data") or {}
attributes = data_section.get("attributes") or {}
payload = {
"id": attributes.get("id", data_section.get("id")),
"parameters": attributes.get("parameters"),
}
# Remove `None` parameters to allow defaults downstream
if payload.get("parameters") is None:
payload.pop("parameters")
return payload
return raw_data
def prepare_query_parameters(
definition: AttackPathsQueryDefinition,
provided_parameters: dict[str, Any],
provider_uid: str,
) -> dict[str, Any]:
parameters = dict(provided_parameters or {})
expected_names = {parameter.name for parameter in definition.parameters}
provided_names = set(parameters.keys())
unexpected = provided_names - expected_names
if unexpected:
raise ValidationError(
{"parameters": f"Unknown parameter(s): {', '.join(sorted(unexpected))}"}
)
missing = expected_names - provided_names
if missing:
raise ValidationError(
{
"parameters": f"Missing required parameter(s): {', '.join(sorted(missing))}"
}
)
clean_parameters = {
"provider_uid": str(provider_uid),
}
for definition_parameter in definition.parameters:
raw_value = provided_parameters[definition_parameter.name]
try:
casted_value = definition_parameter.cast(raw_value)
except (ValueError, TypeError) as exc:
raise ValidationError(
{
"parameters": (
f"Invalid value for parameter `{definition_parameter.name}`: {str(exc)}"
)
}
)
clean_parameters[definition_parameter.name] = casted_value
return clean_parameters
def execute_attack_paths_query(
attack_paths_scan: AttackPathsScan,
definition: AttackPathsQueryDefinition,
parameters: dict[str, Any],
) -> dict[str, Any]:
try:
with graph_database.get_session(attack_paths_scan.graph_database) as session:
result = session.run(definition.cypher, parameters)
return _serialize_graph(result.graph())
except graph_database.GraphDatabaseQueryException as exc:
logger.error(f"Query failed for Attack Paths query `{definition.id}`: {exc}")
raise APIException(
"Attack Paths query execution failed due to a database error"
)
def _serialize_graph(graph):
nodes = []
for node in graph.nodes:
nodes.append(
{
"id": node.element_id,
"labels": list(node.labels),
"properties": _serialize_properties(node._properties),
},
)
relationships = []
for relationship in graph.relationships:
relationships.append(
{
"id": relationship.element_id,
"label": relationship.type,
"source": relationship.start_node.element_id,
"target": relationship.end_node.element_id,
"properties": _serialize_properties(relationship._properties),
},
)
return {
"nodes": nodes,
"relationships": relationships,
}
def _serialize_properties(properties: dict[str, Any]) -> dict[str, Any]:
"""Convert Neo4j property values into JSON-serializable primitives."""
def _serialize_value(value: Any) -> Any:
# Neo4j temporal and spatial values expose `to_native` returning Python primitives
if hasattr(value, "to_native") and callable(value.to_native):
return _serialize_value(value.to_native())
if isinstance(value, (list, tuple)):
return [_serialize_value(item) for item in value]
if isinstance(value, dict):
return {key: _serialize_value(val) for key, val in value.items()}
return value
return {key: _serialize_value(val) for key, val in properties.items()}
+1
View File
@@ -144,6 +144,7 @@ def generate_scan_compliance(
Returns:
None: This function modifies the compliance_overview in place.
"""
for compliance_id in PROWLER_CHECKS[provider_type][check_id]:
for requirement in compliance_overview[compliance_id]["requirements"].values():
if check_id in requirement["checks"]:
+3 -2
View File
@@ -48,8 +48,9 @@ class MainRouter:
return db == self.admin_db
def allow_relation(self, obj1, obj2, **hints): # noqa: F841
# Allow relations if both objects are in either "default" or "admin" db connectors
if {obj1._state.db, obj2._state.db} <= {self.default_db, self.admin_db}:
# Allow relations when both objects originate from allowed connectors
allowed_dbs = {self.default_db, self.admin_db, self.replica_db}
if {obj1._state.db, obj2._state.db} <= allowed_dbs:
return True
return None
+66 -19
View File
@@ -1,18 +1,35 @@
import re
import secrets
import time
import uuid
from contextlib import contextmanager
from datetime import datetime, timedelta, timezone
from celery.utils.log import get_task_logger
from config.env import env
from django.conf import settings
from django.contrib.auth.models import BaseUserManager
from django.db import DEFAULT_DB_ALIAS, connection, connections, models, transaction
from django.db import (
DEFAULT_DB_ALIAS,
OperationalError,
connection,
connections,
models,
transaction,
)
from django_celery_beat.models import PeriodicTask
from psycopg2 import connect as psycopg2_connect
from psycopg2.extensions import AsIs, new_type, register_adapter, register_type
from rest_framework_json_api.serializers import ValidationError
from api.db_router import get_read_db_alias, reset_read_db_alias, set_read_db_alias
from api.db_router import (
READ_REPLICA_ALIAS,
get_read_db_alias,
reset_read_db_alias,
set_read_db_alias,
)
logger = get_task_logger(__name__)
DB_USER = settings.DATABASES["default"]["USER"] if not settings.TESTING else "test"
DB_PASSWORD = (
@@ -28,6 +45,9 @@ TASK_RUNNER_DB_TABLE = "django_celery_results_taskresult"
POSTGRES_TENANT_VAR = "api.tenant_id"
POSTGRES_USER_VAR = "api.user_id"
REPLICA_MAX_ATTEMPTS = env.int("POSTGRES_REPLICA_MAX_ATTEMPTS", default=3)
REPLICA_RETRY_BASE_DELAY = env.float("POSTGRES_REPLICA_RETRY_BASE_DELAY", default=0.5)
SET_CONFIG_QUERY = "SELECT set_config(%s, %s::text, TRUE);"
@@ -71,24 +91,51 @@ def rls_transaction(
if db_alias not in connections:
db_alias = DEFAULT_DB_ALIAS
router_token = None
try:
if db_alias != DEFAULT_DB_ALIAS:
router_token = set_read_db_alias(db_alias)
alias = db_alias
is_replica = READ_REPLICA_ALIAS and alias == READ_REPLICA_ALIAS
max_attempts = REPLICA_MAX_ATTEMPTS if is_replica else 1
with transaction.atomic(using=db_alias):
conn = connections[db_alias]
with conn.cursor() as cursor:
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yield cursor
finally:
if router_token is not None:
reset_read_db_alias(router_token)
for attempt in range(1, max_attempts + 1):
router_token = None
# On final attempt, fallback to primary
if attempt == max_attempts and is_replica:
logger.warning(
f"RLS transaction failed after {attempt - 1} attempts on replica, "
f"falling back to primary DB"
)
alias = DEFAULT_DB_ALIAS
conn = connections[alias]
try:
if alias != DEFAULT_DB_ALIAS:
router_token = set_read_db_alias(alias)
with transaction.atomic(using=alias):
with conn.cursor() as cursor:
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yield cursor
return
except OperationalError as e:
# If on primary or max attempts reached, raise
if not is_replica or attempt == max_attempts:
raise
# Retry with exponential backoff
delay = REPLICA_RETRY_BASE_DELAY * (2 ** (attempt - 1))
logger.info(
f"RLS transaction failed on replica (attempt {attempt}/{max_attempts}), "
f"retrying in {delay}s. Error: {e}"
)
time.sleep(delay)
finally:
if router_token is not None:
reset_read_db_alias(router_token)
class CustomUserManager(BaseUserManager):
+83 -24
View File
@@ -27,9 +27,11 @@ from api.models import (
Finding,
Integration,
Invitation,
AttackPathsScan,
LighthouseProviderConfiguration,
LighthouseProviderModels,
Membership,
MuteRule,
OverviewStatusChoices,
PermissionChoices,
Processor,
@@ -46,6 +48,7 @@ from api.models import (
StatusChoices,
Task,
TenantAPIKey,
ThreatScoreSnapshot,
User,
)
from api.rls import Tenant
@@ -247,6 +250,14 @@ class ProviderFilter(FilterSet):
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
provider_type = ChoiceFilter(
choices=Provider.ProviderChoices.choices, field_name="provider"
)
provider_type__in = ChoiceInFilter(
field_name="provider",
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
class Meta:
model = Provider
@@ -320,6 +331,23 @@ class ScanFilter(ProviderRelationshipFilterSet):
}
class AttackPathsScanFilter(ProviderRelationshipFilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
completed_at = DateFilter(field_name="completed_at", lookup_expr="date")
started_at = DateFilter(field_name="started_at", lookup_expr="date")
state = ChoiceFilter(choices=StateChoices.choices)
state__in = ChoiceInFilter(
field_name="state", choices=StateChoices.choices, lookup_expr="in"
)
class Meta:
model = AttackPathsScan
fields = {
"provider": ["exact", "in"],
"scan": ["exact", "in"],
}
class TaskFilter(FilterSet):
name = CharFilter(field_name="task_runner_task__task_name", lookup_expr="exact")
name__icontains = CharFilter(
@@ -802,7 +830,8 @@ class ScanSummarySeverityFilter(ScanSummaryFilter):
elif value == OverviewStatusChoices.PASS:
return queryset.annotate(status_count=F("_pass"))
else:
return queryset.annotate(status_count=F("total"))
# Exclude muted findings by default
return queryset.annotate(status_count=F("_pass") + F("fail"))
def filter_status_in(self, queryset, name, value):
# Validate the status values
@@ -811,7 +840,7 @@ class ScanSummarySeverityFilter(ScanSummaryFilter):
if status_val not in valid_statuses:
raise ValidationError(f"Invalid status value: {status_val}")
# If all statuses or no valid statuses, use total
# If all statuses or no valid statuses, exclude muted findings (pass + fail)
if (
set(value)
>= {
@@ -820,7 +849,7 @@ class ScanSummarySeverityFilter(ScanSummaryFilter):
}
or not value
):
return queryset.annotate(status_count=F("total"))
return queryset.annotate(status_count=F("_pass") + F("fail"))
# Build the sum expression based on status values
sum_expression = None
@@ -838,7 +867,7 @@ class ScanSummarySeverityFilter(ScanSummaryFilter):
sum_expression = sum_expression + field_expr
if sum_expression is None:
return queryset.annotate(status_count=F("total"))
return queryset.annotate(status_count=F("_pass") + F("fail"))
return queryset.annotate(status_count=sum_expression)
@@ -850,26 +879,6 @@ class ScanSummarySeverityFilter(ScanSummaryFilter):
}
class ServiceOverviewFilter(ScanSummaryFilter):
def is_valid(self):
# Check if at least one of the inserted_at filters is present
inserted_at_filters = [
self.data.get("inserted_at"),
self.data.get("inserted_at__gte"),
self.data.get("inserted_at__lte"),
]
if not any(inserted_at_filters):
raise ValidationError(
{
"inserted_at": [
"At least one of filter[inserted_at], filter[inserted_at__gte], or "
"filter[inserted_at__lte] is required."
]
}
)
return super().is_valid()
class IntegrationFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
integration_type = ChoiceFilter(choices=Integration.IntegrationChoices.choices)
@@ -972,3 +981,53 @@ class LighthouseProviderModelsFilter(FilterSet):
fields = {
"model_id": ["exact", "icontains", "in"],
}
class MuteRuleFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
updated_at = DateFilter(field_name="updated_at", lookup_expr="date")
created_by = UUIDFilter(field_name="created_by__id", lookup_expr="exact")
class Meta:
model = MuteRule
fields = {
"id": ["exact", "in"],
"name": ["exact", "icontains"],
"reason": ["icontains"],
"enabled": ["exact"],
"inserted_at": ["gte", "lte"],
"updated_at": ["gte", "lte"],
}
class ThreatScoreSnapshotFilter(FilterSet):
"""
Filter for ThreatScore snapshots.
Allows filtering by scan, provider, compliance_id, and date ranges.
"""
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
scan_id = UUIDFilter(field_name="scan__id", lookup_expr="exact")
scan_id__in = UUIDInFilter(field_name="scan__id", lookup_expr="in")
provider_id = UUIDFilter(field_name="provider__id", lookup_expr="exact")
provider_id__in = UUIDInFilter(field_name="provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
field_name="provider__provider", choices=Provider.ProviderChoices.choices
)
provider_type__in = ChoiceInFilter(
field_name="provider__provider",
choices=Provider.ProviderChoices.choices,
lookup_expr="in",
)
compliance_id = CharFilter(field_name="compliance_id", lookup_expr="exact")
compliance_id__in = CharInFilter(field_name="compliance_id", lookup_expr="in")
class Meta:
model = ThreatScoreSnapshot
fields = {
"scan": ["exact", "in"],
"provider": ["exact", "in"],
"compliance_id": ["exact", "in"],
"inserted_at": ["date", "gte", "lte"],
"overall_score": ["exact", "gte", "lte"],
}
@@ -0,0 +1,41 @@
[
{
"model": "api.attackpathsscan",
"pk": "a7f0f6de-6f8e-4b3a-8cbe-3f6dd9012345",
"fields": {
"tenant": "12646005-9067-4d2a-a098-8bb378604362",
"provider": "b85601a8-4b45-4194-8135-03fb980ef428",
"scan": "01920573-aa9c-73c9-bcda-f2e35c9b19d2",
"state": "completed",
"progress": 100,
"update_tag": 1693586667,
"graph_database": "db-a7f0f6de-6f8e-4b3a-8cbe-3f6dd9012345",
"is_graph_database_deleted": false,
"task": null,
"inserted_at": "2024-09-01T17:24:37Z",
"updated_at": "2024-09-01T17:44:37Z",
"started_at": "2024-09-01T17:34:37Z",
"completed_at": "2024-09-01T17:44:37Z",
"duration": 269,
"ingestion_exceptions": {}
}
},
{
"model": "api.attackpathsscan",
"pk": "4a2fb2af-8a60-4d7d-9cae-4ca65e098765",
"fields": {
"tenant": "12646005-9067-4d2a-a098-8bb378604362",
"provider": "15fce1fa-ecaa-433f-a9dc-62553f3a2555",
"scan": "01929f3b-ed2e-7623-ad63-7c37cd37828f",
"state": "executing",
"progress": 48,
"update_tag": 1697625000,
"graph_database": "db-4a2fb2af-8a60-4d7d-9cae-4ca65e098765",
"is_graph_database_deleted": false,
"task": null,
"inserted_at": "2024-10-18T10:55:57Z",
"updated_at": "2024-10-18T10:56:15Z",
"started_at": "2024-10-18T10:56:05Z"
}
}
]
@@ -0,0 +1,34 @@
# Generated by Django 5.1.7 on 2025-10-14 00:00
from django.db import migrations
import api.db_utils
class Migration(migrations.Migration):
dependencies = [
("api", "0050_lighthouse_multi_llm"),
]
operations = [
migrations.AlterField(
model_name="provider",
name="provider",
field=api.db_utils.ProviderEnumField(
choices=[
("aws", "AWS"),
("azure", "Azure"),
("gcp", "GCP"),
("kubernetes", "Kubernetes"),
("m365", "M365"),
("github", "GitHub"),
("oraclecloud", "Oracle Cloud Infrastructure"),
],
default="aws",
),
),
migrations.RunSQL(
"ALTER TYPE provider ADD VALUE IF NOT EXISTS 'oraclecloud';",
reverse_sql=migrations.RunSQL.noop,
),
]
@@ -0,0 +1,117 @@
# Generated by Django 5.1.13 on 2025-10-22 11:56
import uuid
import django.contrib.postgres.fields
import django.core.validators
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0051_oraclecloud_provider"),
]
operations = [
migrations.CreateModel(
name="MuteRule",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
(
"name",
models.CharField(
help_text="Human-readable name for this rule",
max_length=100,
validators=[django.core.validators.MinLengthValidator(3)],
),
),
(
"reason",
models.TextField(
help_text="Reason for muting",
max_length=500,
validators=[django.core.validators.MinLengthValidator(3)],
),
),
(
"enabled",
models.BooleanField(
default=True, help_text="Whether this rule is currently enabled"
),
),
(
"finding_uids",
django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(max_length=255),
help_text="List of finding UIDs to mute",
size=None,
),
),
],
options={
"db_table": "mute_rules",
"abstract": False,
},
),
migrations.AddField(
model_name="finding",
name="muted_at",
field=models.DateTimeField(
blank=True, help_text="Timestamp when this finding was muted", null=True
),
),
migrations.AlterField(
model_name="tenantapikey",
name="name",
field=models.CharField(
max_length=100,
validators=[django.core.validators.MinLengthValidator(3)],
),
),
migrations.AddField(
model_name="muterule",
name="created_by",
field=models.ForeignKey(
help_text="User who created this rule",
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="created_mute_rules",
to=settings.AUTH_USER_MODEL,
),
),
migrations.AddField(
model_name="muterule",
name="tenant",
field=models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
migrations.AddConstraint(
model_name="muterule",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_muterule",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.AddConstraint(
model_name="muterule",
constraint=models.UniqueConstraint(
fields=("tenant_id", "name"), name="unique_mute_rule_name_per_tenant"
),
),
]
@@ -0,0 +1,25 @@
# Generated by Django 5.1.12 on 2025-10-14 11:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0052_mute_rules"),
]
operations = [
migrations.AlterField(
model_name="lighthouseproviderconfiguration",
name="provider_type",
field=models.CharField(
choices=[
("openai", "OpenAI"),
("bedrock", "AWS Bedrock"),
("openai_compatible", "OpenAI Compatible"),
],
help_text="LLM provider name",
max_length=50,
),
)
]
@@ -0,0 +1,35 @@
# Generated by Django 5.1.10 on 2025-09-09 09:25
from django.db import migrations
import api.db_utils
class Migration(migrations.Migration):
dependencies = [
("api", "0053_lighthouse_bedrock_openai_compatible"),
]
operations = [
migrations.AlterField(
model_name="provider",
name="provider",
field=api.db_utils.ProviderEnumField(
choices=[
("aws", "AWS"),
("azure", "Azure"),
("gcp", "GCP"),
("kubernetes", "Kubernetes"),
("m365", "M365"),
("github", "GitHub"),
("oraclecloud", "Oracle Cloud Infrastructure"),
("iac", "IaC"),
],
default="aws",
),
),
migrations.RunSQL(
"ALTER TYPE provider ADD VALUE IF NOT EXISTS 'iac';",
reverse_sql=migrations.RunSQL.noop,
),
]
@@ -0,0 +1,36 @@
# Generated by Django 5.1.13 on 2025-11-05 08:37
from django.db import migrations
import api.db_utils
class Migration(migrations.Migration):
dependencies = [
("api", "0054_iac_provider"),
]
operations = [
migrations.AlterField(
model_name="provider",
name="provider",
field=api.db_utils.ProviderEnumField(
choices=[
("aws", "AWS"),
("azure", "Azure"),
("gcp", "GCP"),
("kubernetes", "Kubernetes"),
("m365", "M365"),
("github", "GitHub"),
("mongodbatlas", "MongoDB Atlas"),
("iac", "IaC"),
("oraclecloud", "Oracle Cloud Infrastructure"),
],
default="aws",
),
),
migrations.RunSQL(
"ALTER TYPE provider ADD VALUE IF NOT EXISTS 'mongodbatlas';",
reverse_sql=migrations.RunSQL.noop,
),
]
@@ -0,0 +1,24 @@
# Generated by Django 5.1.13 on 2025-11-06 09:20
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0055_mongodbatlas_provider"),
]
operations = [
migrations.RemoveConstraint(
model_name="provider",
name="unique_provider_uids",
),
migrations.AddConstraint(
model_name="provider",
constraint=models.UniqueConstraint(
condition=models.Q(("is_deleted", False)),
fields=("tenant_id", "provider", "uid"),
name="unique_provider_uids",
),
),
]
@@ -0,0 +1,170 @@
# Generated by Django 5.1.13 on 2025-10-31 09:04
import uuid
import django.db.models.deletion
from django.db import migrations, models
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0056_remove_provider_unique_provider_uids_and_more"),
]
operations = [
migrations.CreateModel(
name="ThreatScoreSnapshot",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
(
"compliance_id",
models.CharField(
help_text="Compliance framework ID (e.g., 'prowler_threatscore_aws')",
max_length=100,
),
),
(
"overall_score",
models.DecimalField(
decimal_places=2,
help_text="Overall ThreatScore percentage (0-100)",
max_digits=5,
),
),
(
"score_delta",
models.DecimalField(
blank=True,
decimal_places=2,
help_text="Score change compared to previous snapshot (positive = improvement)",
max_digits=5,
null=True,
),
),
(
"section_scores",
models.JSONField(
blank=True,
default=dict,
help_text="ThreatScore breakdown by section",
),
),
(
"critical_requirements",
models.JSONField(
blank=True,
default=list,
help_text="List of critical failed requirements (risk >= 4)",
),
),
(
"total_requirements",
models.IntegerField(
default=0, help_text="Total number of requirements evaluated"
),
),
(
"passed_requirements",
models.IntegerField(
default=0, help_text="Number of requirements with PASS status"
),
),
(
"failed_requirements",
models.IntegerField(
default=0, help_text="Number of requirements with FAIL status"
),
),
(
"manual_requirements",
models.IntegerField(
default=0, help_text="Number of requirements with MANUAL status"
),
),
(
"total_findings",
models.IntegerField(
default=0,
help_text="Total number of findings across all requirements",
),
),
(
"passed_findings",
models.IntegerField(
default=0, help_text="Number of findings with PASS status"
),
),
(
"failed_findings",
models.IntegerField(
default=0, help_text="Number of findings with FAIL status"
),
),
(
"provider",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="threatscore_snapshots",
related_query_name="threatscore_snapshot",
to="api.provider",
),
),
(
"scan",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="threatscore_snapshots",
related_query_name="threatscore_snapshot",
to="api.scan",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "threatscore_snapshots",
"abstract": False,
},
),
migrations.AddIndex(
model_name="threatscoresnapshot",
index=models.Index(
fields=["tenant_id", "scan_id"], name="threatscore_snap_t_scan_idx"
),
),
migrations.AddIndex(
model_name="threatscoresnapshot",
index=models.Index(
fields=["tenant_id", "provider_id"], name="threatscore_snap_t_prov_idx"
),
),
migrations.AddIndex(
model_name="threatscoresnapshot",
index=models.Index(
fields=["tenant_id", "inserted_at"], name="threatscore_snap_t_time_idx"
),
),
migrations.AddConstraint(
model_name="threatscoresnapshot",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_threatscoresnapshot",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
@@ -0,0 +1,29 @@
from django.contrib.postgres.operations import RemoveIndexConcurrently
from django.db import migrations
class Migration(migrations.Migration):
atomic = False
dependencies = [
("api", "0057_threatscoresnapshot"),
]
operations = [
RemoveIndexConcurrently(
model_name="compliancerequirementoverview",
name="cro_tenant_scan_idx",
),
RemoveIndexConcurrently(
model_name="compliancerequirementoverview",
name="cro_scan_comp_idx",
),
RemoveIndexConcurrently(
model_name="compliancerequirementoverview",
name="cro_scan_comp_req_idx",
),
RemoveIndexConcurrently(
model_name="compliancerequirementoverview",
name="cro_scan_comp_req_reg_idx",
),
]
@@ -0,0 +1,75 @@
# Generated by Django 5.1.13 on 2025-10-30 15:23
import uuid
import django.db.models.deletion
from django.db import migrations, models
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0058_drop_redundant_compliance_requirement_indexes"),
]
operations = [
migrations.CreateModel(
name="ComplianceOverviewSummary",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("compliance_id", models.TextField()),
("requirements_passed", models.IntegerField(default=0)),
("requirements_failed", models.IntegerField(default=0)),
("requirements_manual", models.IntegerField(default=0)),
("total_requirements", models.IntegerField(default=0)),
(
"scan",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="compliance_summaries",
related_query_name="compliance_summary",
to="api.scan",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "compliance_overview_summaries",
"abstract": False,
"indexes": [
models.Index(
fields=["tenant_id", "scan_id"], name="cos_tenant_scan_idx"
)
],
"constraints": [
models.UniqueConstraint(
fields=("tenant_id", "scan_id", "compliance_id"),
name="unique_compliance_summary_per_scan",
)
],
},
),
migrations.AddConstraint(
model_name="complianceoverviewsummary",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_complianceoverviewsummary",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
@@ -0,0 +1,154 @@
# Generated by Django 5.1.13 on 2025-11-06 16:20
import django.db.models.deletion
from django.db import migrations, models
from uuid6 import uuid7
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0059_compliance_overview_summary"),
]
operations = [
migrations.CreateModel(
name="AttackPathsScan",
fields=[
(
"id",
models.UUIDField(
default=uuid7,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
(
"state",
api.db_utils.StateEnumField(
choices=[
("available", "Available"),
("scheduled", "Scheduled"),
("executing", "Executing"),
("completed", "Completed"),
("failed", "Failed"),
("cancelled", "Cancelled"),
],
default="available",
),
),
("progress", models.IntegerField(default=0)),
("started_at", models.DateTimeField(blank=True, null=True)),
("completed_at", models.DateTimeField(blank=True, null=True)),
(
"duration",
models.IntegerField(
blank=True, help_text="Duration in seconds", null=True
),
),
(
"update_tag",
models.BigIntegerField(
blank=True,
help_text="Cartography update tag (epoch)",
null=True,
),
),
(
"graph_database",
models.CharField(blank=True, max_length=63, null=True),
),
(
"is_graph_database_deleted",
models.BooleanField(default=False),
),
(
"ingestion_exceptions",
models.JSONField(blank=True, default=dict, null=True),
),
(
"provider",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
to="api.provider",
),
),
(
"scan",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
to="api.scan",
),
),
(
"task",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
to="api.task",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "attack_paths_scans",
"abstract": False,
"indexes": [
models.Index(
fields=["tenant_id", "provider_id", "-inserted_at"],
name="aps_prov_ins_desc_idx",
),
models.Index(
fields=["tenant_id", "state", "-inserted_at"],
name="aps_state_ins_desc_idx",
),
models.Index(
fields=["tenant_id", "scan_id"],
name="aps_scan_lookup_idx",
),
models.Index(
fields=["tenant_id", "provider_id"],
name="aps_active_graph_idx",
include=["graph_database", "id"],
condition=models.Q(("is_graph_database_deleted", False)),
),
models.Index(
fields=["tenant_id", "provider_id", "-completed_at"],
name="aps_completed_graph_idx",
include=["graph_database", "id"],
condition=models.Q(
("state", "completed"),
("is_graph_database_deleted", False),
),
),
],
},
),
migrations.AddConstraint(
model_name="attackpathsscan",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_attackpathsscan",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
+381 -21
View File
@@ -284,6 +284,9 @@ class Provider(RowLevelSecurityProtectedModel):
KUBERNETES = "kubernetes", _("Kubernetes")
M365 = "m365", _("M365")
GITHUB = "github", _("GitHub")
MONGODBATLAS = "mongodbatlas", _("MongoDB Atlas")
IAC = "iac", _("IaC")
ORACLECLOUD = "oraclecloud", _("Oracle Cloud Infrastructure")
@staticmethod
def validate_aws_uid(value):
@@ -354,6 +357,40 @@ class Provider(RowLevelSecurityProtectedModel):
pointer="/data/attributes/uid",
)
@staticmethod
def validate_iac_uid(value):
# Validate that it's a valid repository URL (git URL format)
if not re.match(
r"^(https?://|git@|ssh://)[^\s/]+[^\s]*\.git$|^(https?://)[^\s/]+[^\s]*$",
value,
):
raise ModelValidationError(
detail="IaC provider ID must be a valid repository URL (e.g., https://github.com/user/repo or https://github.com/user/repo.git).",
code="iac-uid",
pointer="/data/attributes/uid",
)
@staticmethod
def validate_oraclecloud_uid(value):
if not re.match(
r"^ocid1\.([a-z0-9_-]+)\.([a-z0-9_-]+)\.([a-z0-9_-]*)\.([a-z0-9]+)$", value
):
raise ModelValidationError(
detail="Oracle Cloud Infrastructure provider ID must be a valid tenancy OCID in the format: "
"ocid1.<resource_type>.<realm>.<region>.<unique_id>",
code="oraclecloud-uid",
pointer="/data/attributes/uid",
)
@staticmethod
def validate_mongodbatlas_uid(value):
if not re.match(r"^[0-9a-fA-F]{24}$", value):
raise ModelValidationError(
detail="MongoDB Atlas organization ID must be a 24-character hexadecimal string.",
code="mongodbatlas-uid",
pointer="/data/attributes/uid",
)
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
@@ -388,7 +425,8 @@ class Provider(RowLevelSecurityProtectedModel):
constraints = [
models.UniqueConstraint(
fields=("tenant_id", "provider", "uid", "is_deleted"),
fields=("tenant_id", "provider", "uid"),
condition=Q(is_deleted=False),
name="unique_provider_uids",
),
RowLevelSecurityConstraint(
@@ -578,6 +616,101 @@ class Scan(RowLevelSecurityProtectedModel):
resource_name = "scans"
class AttackPathsScan(RowLevelSecurityProtectedModel):
objects = ActiveProviderManager()
all_objects = models.Manager()
id = models.UUIDField(primary_key=True, default=uuid7, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
state = StateEnumField(choices=StateChoices.choices, default=StateChoices.AVAILABLE)
progress = models.IntegerField(default=0)
# Timing
started_at = models.DateTimeField(null=True, blank=True)
completed_at = models.DateTimeField(null=True, blank=True)
duration = models.IntegerField(
null=True, blank=True, help_text="Duration in seconds"
)
# Relationship to the provider and optional prowler Scan and celery Task
provider = models.ForeignKey(
"Provider",
on_delete=models.CASCADE,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
)
scan = models.ForeignKey(
"Scan",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
)
task = models.ForeignKey(
"Task",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="attack_paths_scans",
related_query_name="attack_paths_scan",
)
# Cartography specific metadata
update_tag = models.BigIntegerField(
null=True, blank=True, help_text="Cartography update tag (epoch)"
)
graph_database = models.CharField(max_length=63, null=True, blank=True)
is_graph_database_deleted = models.BooleanField(default=False)
ingestion_exceptions = models.JSONField(default=dict, null=True, blank=True)
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "attack_paths_scans"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
indexes = [
models.Index(
fields=["tenant_id", "provider_id", "-inserted_at"],
name="aps_prov_ins_desc_idx",
),
models.Index(
fields=["tenant_id", "state", "-inserted_at"],
name="aps_state_ins_desc_idx",
),
models.Index(
fields=["tenant_id", "scan_id"],
name="aps_scan_lookup_idx",
),
models.Index(
fields=["tenant_id", "provider_id"],
name="aps_active_graph_idx",
include=["graph_database", "id"],
condition=Q(is_graph_database_deleted=False),
),
models.Index(
fields=["tenant_id", "provider_id", "-completed_at"],
name="aps_completed_graph_idx",
include=["graph_database", "id"],
condition=Q(
state=StateChoices.COMPLETED,
is_graph_database_deleted=False,
),
),
]
class JSONAPIMeta:
resource_name = "attack-paths-scans"
class ResourceTag(RowLevelSecurityProtectedModel):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
@@ -810,6 +943,9 @@ class Finding(PostgresPartitionedModel, RowLevelSecurityProtectedModel):
muted_reason = models.TextField(
blank=True, null=True, validators=[MinLengthValidator(3)], max_length=500
)
muted_at = models.DateTimeField(
null=True, blank=True, help_text="Timestamp when this finding was muted"
)
compliance = models.JSONField(default=dict, null=True, blank=True)
# Denormalize resource data for performance
@@ -1330,35 +1466,70 @@ class ComplianceRequirementOverview(RowLevelSecurityProtectedModel):
),
]
indexes = [
models.Index(fields=["tenant_id", "scan_id"], name="cro_tenant_scan_idx"),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id"],
name="cro_scan_comp_idx",
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id", "region"],
name="cro_scan_comp_reg_idx",
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id", "requirement_id"],
name="cro_scan_comp_req_idx",
),
models.Index(
fields=[
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
"region",
],
name="cro_scan_comp_req_reg_idx",
),
]
class JSONAPIMeta:
resource_name = "compliance-requirements-overviews"
class ComplianceOverviewSummary(RowLevelSecurityProtectedModel):
"""
Pre-aggregated compliance overview aggregated across ALL regions.
One row per (scan_id, compliance_id) combination.
This table optimizes the common case where users view overall compliance
without filtering by region. For region-specific views, the detailed
ComplianceRequirementOverview table is used instead.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
scan = models.ForeignKey(
Scan,
on_delete=models.CASCADE,
related_name="compliance_summaries",
related_query_name="compliance_summary",
)
compliance_id = models.TextField(blank=False)
# Pre-aggregated scores (computed across ALL regions)
requirements_passed = models.IntegerField(default=0)
requirements_failed = models.IntegerField(default=0)
requirements_manual = models.IntegerField(default=0)
total_requirements = models.IntegerField(default=0)
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "compliance_overview_summaries"
constraints = [
models.UniqueConstraint(
fields=("tenant_id", "scan_id", "compliance_id"),
name="unique_compliance_summary_per_scan",
),
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "DELETE"],
),
]
indexes = [
models.Index(
fields=["tenant_id", "scan_id"],
name="cos_tenant_scan_idx",
),
]
class JSONAPIMeta:
resource_name = "compliance-overview-summaries"
class ScanSummary(RowLevelSecurityProtectedModel):
objects = ActiveProviderManager()
all_objects = models.Manager()
@@ -1922,6 +2093,59 @@ class LighthouseConfiguration(RowLevelSecurityProtectedModel):
resource_name = "lighthouse-configurations"
class MuteRule(RowLevelSecurityProtectedModel):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
updated_at = models.DateTimeField(auto_now=True, editable=False)
# Rule metadata
name = models.CharField(
max_length=100,
validators=[MinLengthValidator(3)],
help_text="Human-readable name for this rule",
)
reason = models.TextField(
validators=[MinLengthValidator(3)],
max_length=500,
help_text="Reason for muting",
)
enabled = models.BooleanField(
default=True, help_text="Whether this rule is currently enabled"
)
# Audit fields
created_by = models.ForeignKey(
User,
on_delete=models.SET_NULL,
null=True,
related_name="created_mute_rules",
help_text="User who created this rule",
)
# Rule criteria - array of finding UIDs
finding_uids = ArrayField(
models.CharField(max_length=255), help_text="List of finding UIDs to mute"
)
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "mute_rules"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
models.UniqueConstraint(
fields=("tenant_id", "name"),
name="unique_mute_rule_name_per_tenant",
),
]
class JSONAPIMeta:
resource_name = "mute-rules"
class Processor(RowLevelSecurityProtectedModel):
class ProcessorChoices(models.TextChoices):
MUTELIST = "mutelist", _("Mutelist")
@@ -1970,6 +2194,8 @@ class LighthouseProviderConfiguration(RowLevelSecurityProtectedModel):
class LLMProviderChoices(models.TextChoices):
OPENAI = "openai", _("OpenAI")
BEDROCK = "bedrock", _("AWS Bedrock")
OPENAI_COMPATIBLE = "openai_compatible", _("OpenAI Compatible")
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
@@ -2091,7 +2317,7 @@ class LighthouseTenantConfiguration(RowLevelSecurityProtectedModel):
]
class JSONAPIMeta:
resource_name = "lighthouse-config"
resource_name = "lighthouse-configurations"
class LighthouseProviderModels(RowLevelSecurityProtectedModel):
@@ -2140,3 +2366,137 @@ class LighthouseProviderModels(RowLevelSecurityProtectedModel):
name="lh_prov_models_cfg_idx",
),
]
class JSONAPIMeta:
resource_name = "lighthouse-models"
class ThreatScoreSnapshot(RowLevelSecurityProtectedModel):
"""
Stores historical ThreatScore metrics for a given scan.
Snapshots are created automatically after each ThreatScore report generation.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
scan = models.ForeignKey(
Scan,
on_delete=models.CASCADE,
related_name="threatscore_snapshots",
related_query_name="threatscore_snapshot",
)
provider = models.ForeignKey(
Provider,
on_delete=models.CASCADE,
related_name="threatscore_snapshots",
related_query_name="threatscore_snapshot",
)
compliance_id = models.CharField(
max_length=100,
blank=False,
null=False,
help_text="Compliance framework ID (e.g., 'prowler_threatscore_aws')",
)
# Overall ThreatScore metrics
overall_score = models.DecimalField(
max_digits=5,
decimal_places=2,
help_text="Overall ThreatScore percentage (0-100)",
)
# Score improvement/degradation compared to previous snapshot
score_delta = models.DecimalField(
max_digits=5,
decimal_places=2,
null=True,
blank=True,
help_text="Score change compared to previous snapshot (positive = improvement)",
)
# Section breakdown stored as JSON
# Format: {"1. IAM": 85.5, "2. Attack Surface": 92.3, ...}
section_scores = models.JSONField(
default=dict,
blank=True,
help_text="ThreatScore breakdown by section",
)
# Critical requirements metadata stored as JSON
# Format: [{"requirement_id": "...", "risk_level": 5, "weight": 150, ...}, ...]
critical_requirements = models.JSONField(
default=list,
blank=True,
help_text="List of critical failed requirements (risk >= 4)",
)
# Summary statistics
total_requirements = models.IntegerField(
default=0,
help_text="Total number of requirements evaluated",
)
passed_requirements = models.IntegerField(
default=0,
help_text="Number of requirements with PASS status",
)
failed_requirements = models.IntegerField(
default=0,
help_text="Number of requirements with FAIL status",
)
manual_requirements = models.IntegerField(
default=0,
help_text="Number of requirements with MANUAL status",
)
total_findings = models.IntegerField(
default=0,
help_text="Total number of findings across all requirements",
)
passed_findings = models.IntegerField(
default=0,
help_text="Number of findings with PASS status",
)
failed_findings = models.IntegerField(
default=0,
help_text="Number of findings with FAIL status",
)
def __str__(self):
return f"ThreatScore {self.overall_score}% for scan {self.scan_id} ({self.inserted_at})"
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "threatscore_snapshots"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
indexes = [
models.Index(
fields=["tenant_id", "scan_id"],
name="threatscore_snap_t_scan_idx",
),
models.Index(
fields=["tenant_id", "provider_id"],
name="threatscore_snap_t_prov_idx",
),
models.Index(
fields=["tenant_id", "inserted_at"],
name="threatscore_snap_t_time_idx",
),
]
class JSONAPIMeta:
resource_name = "threatscore-snapshots"
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,39 @@
"""Tests for rls_transaction retry and fallback logic."""
import pytest
from django.db import DEFAULT_DB_ALIAS
from rest_framework_json_api.serializers import ValidationError
from api.db_utils import rls_transaction
@pytest.mark.django_db
class TestRLSTransaction:
"""Simple integration tests for rls_transaction using real DB."""
@pytest.fixture
def tenant(self, tenants_fixture):
return tenants_fixture[0]
def test_success_on_primary(self, tenant):
"""Basic: transaction succeeds on primary database."""
with rls_transaction(str(tenant.id), using=DEFAULT_DB_ALIAS) as cursor:
cursor.execute("SELECT 1")
result = cursor.fetchone()
assert result == (1,)
def test_invalid_uuid_raises_validation_error(self):
"""Invalid UUID raises ValidationError before DB operations."""
with pytest.raises(ValidationError, match="Must be a valid UUID"):
with rls_transaction("not-a-uuid", using=DEFAULT_DB_ALIAS):
pass
def test_custom_parameter_name(self, tenant):
"""Test custom RLS parameter name."""
custom_param = "api.custom_id"
with rls_transaction(
str(tenant.id), parameter=custom_param, using=DEFAULT_DB_ALIAS
) as cursor:
cursor.execute("SELECT current_setting(%s, true)", [custom_param])
result = cursor.fetchone()
assert result == (str(tenant.id),)
@@ -0,0 +1,172 @@
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from rest_framework.exceptions import APIException, ValidationError
from api.attack_paths import database as graph_database
from api.attack_paths import views_helpers
def test_normalize_run_payload_extracts_attributes_section():
payload = {
"data": {
"id": "ignored",
"attributes": {
"id": "aws-rds",
"parameters": {"ip": "192.0.2.0"},
},
}
}
result = views_helpers.normalize_run_payload(payload)
assert result == {"id": "aws-rds", "parameters": {"ip": "192.0.2.0"}}
def test_normalize_run_payload_passthrough_for_non_dict():
sentinel = "not-a-dict"
assert views_helpers.normalize_run_payload(sentinel) is sentinel
def test_prepare_query_parameters_includes_provider_and_casts(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(cast_type=int)
result = views_helpers.prepare_query_parameters(
definition,
{"limit": "5"},
provider_uid="123456789012",
)
assert result["provider_uid"] == "123456789012"
assert result["limit"] == 5
@pytest.mark.parametrize(
"provided,expected_message",
[
({}, "Missing required parameter"),
({"limit": 10, "extra": True}, "Unknown parameter"),
],
)
def test_prepare_query_parameters_validates_names(
attack_paths_query_definition_factory, provided, expected_message
):
definition = attack_paths_query_definition_factory()
with pytest.raises(ValidationError) as exc:
views_helpers.prepare_query_parameters(definition, provided, provider_uid="1")
assert expected_message in str(exc.value)
def test_prepare_query_parameters_validates_cast(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(cast_type=int)
with pytest.raises(ValidationError) as exc:
views_helpers.prepare_query_parameters(
definition,
{"limit": "not-an-int"},
provider_uid="1",
)
assert "Invalid value" in str(exc.value)
def test_execute_attack_paths_query_serializes_graph(
attack_paths_query_definition_factory, attack_paths_graph_stub_classes
):
definition = attack_paths_query_definition_factory(
id="aws-rds",
name="RDS",
description="",
cypher="MATCH (n) RETURN n",
parameters=[],
)
parameters = {"provider_uid": "123"}
attack_paths_scan = SimpleNamespace(graph_database="tenant-db")
node = attack_paths_graph_stub_classes.Node(
element_id="node-1",
labels=["AWSAccount"],
properties={
"name": "account",
"complex": {
"items": [
attack_paths_graph_stub_classes.NativeValue("value"),
{"nested": 1},
]
},
},
)
relationship = attack_paths_graph_stub_classes.Relationship(
element_id="rel-1",
rel_type="OWNS",
start_node=node,
end_node=attack_paths_graph_stub_classes.Node("node-2", ["RDSInstance"], {}),
properties={"weight": 1},
)
graph = SimpleNamespace(nodes=[node], relationships=[relationship])
run_result = MagicMock()
run_result.graph.return_value = graph
session = MagicMock()
session.run.return_value = run_result
session_ctx = MagicMock()
session_ctx.__enter__.return_value = session
session_ctx.__exit__.return_value = False
with patch(
"api.attack_paths.views_helpers.graph_database.get_session",
return_value=session_ctx,
) as mock_get_session:
result = views_helpers.execute_attack_paths_query(
attack_paths_scan, definition, parameters
)
mock_get_session.assert_called_once_with("tenant-db")
session.run.assert_called_once_with(definition.cypher, parameters)
assert result["nodes"][0]["id"] == "node-1"
assert result["nodes"][0]["properties"]["complex"]["items"][0] == "value"
assert result["relationships"][0]["label"] == "OWNS"
def test_execute_attack_paths_query_wraps_graph_errors(
attack_paths_query_definition_factory,
):
definition = attack_paths_query_definition_factory(
id="aws-rds",
name="RDS",
description="",
cypher="MATCH (n) RETURN n",
parameters=[],
)
attack_paths_scan = SimpleNamespace(graph_database="tenant-db")
parameters = {"provider_uid": "123"}
class ExplodingContext:
def __enter__(self):
raise graph_database.GraphDatabaseQueryException("boom")
def __exit__(self, exc_type, exc, tb):
return False
with (
patch(
"api.attack_paths.views_helpers.graph_database.get_session",
return_value=ExplodingContext(),
),
patch("api.attack_paths.views_helpers.logger") as mock_logger,
):
with pytest.raises(APIException):
views_helpers.execute_attack_paths_query(
attack_paths_scan, definition, parameters
)
mock_logger.error.assert_called_once()
+510 -1
View File
@@ -1,12 +1,15 @@
from datetime import datetime, timezone
from enum import Enum
from unittest.mock import patch
from unittest.mock import MagicMock, patch
import pytest
from django.conf import settings
from django.db import DEFAULT_DB_ALIAS, OperationalError
from freezegun import freeze_time
from rest_framework_json_api.serializers import ValidationError
from api.db_utils import (
POSTGRES_TENANT_VAR,
_should_create_index_on_partition,
batch_delete,
create_objects_in_batches,
@@ -14,11 +17,22 @@ from api.db_utils import (
generate_api_key_prefix,
generate_random_token,
one_week_from_now,
rls_transaction,
update_objects_in_batches,
)
from api.models import Provider
@pytest.fixture
def enable_read_replica():
"""
Fixture to enable READ_REPLICA_ALIAS for tests that need replica functionality.
This avoids polluting the global test configuration.
"""
with patch("api.db_utils.READ_REPLICA_ALIAS", "replica"):
yield "replica"
class TestEnumToChoices:
def test_enum_to_choices_simple(self):
class Color(Enum):
@@ -339,3 +353,498 @@ class TestGenerateApiKeyPrefix:
prefix = generate_api_key_prefix()
random_part = prefix[3:] # Strip 'pk_'
assert all(char in allowed_chars for char in random_part)
@pytest.mark.django_db
class TestRlsTransaction:
def test_rls_transaction_valid_uuid_string(self, tenants_fixture):
"""Test rls_transaction with valid UUID string."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with rls_transaction(tenant_id) as cursor:
assert cursor is not None
cursor.execute("SELECT current_setting(%s)", [POSTGRES_TENANT_VAR])
result = cursor.fetchone()
assert result[0] == tenant_id
def test_rls_transaction_valid_uuid_object(self, tenants_fixture):
"""Test rls_transaction with UUID object."""
tenant = tenants_fixture[0]
with rls_transaction(tenant.id) as cursor:
assert cursor is not None
cursor.execute("SELECT current_setting(%s)", [POSTGRES_TENANT_VAR])
result = cursor.fetchone()
assert result[0] == str(tenant.id)
def test_rls_transaction_invalid_uuid_raises_validation_error(self):
"""Test rls_transaction raises ValidationError for invalid UUID."""
invalid_uuid = "not-a-valid-uuid"
with pytest.raises(ValidationError, match="Must be a valid UUID"):
with rls_transaction(invalid_uuid):
pass
def test_rls_transaction_uses_default_database_when_no_alias(self, tenants_fixture):
"""Test rls_transaction uses DEFAULT_DB_ALIAS when no alias specified."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with rls_transaction(tenant_id):
pass
mock_connections.__getitem__.assert_called_with(DEFAULT_DB_ALIAS)
def test_rls_transaction_uses_specified_alias(self, tenants_fixture):
"""Test rls_transaction uses specified database alias via using parameter."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
custom_alias = "custom_db"
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with patch("api.db_utils.set_read_db_alias") as mock_set_alias:
with patch("api.db_utils.reset_read_db_alias") as mock_reset_alias:
mock_set_alias.return_value = "test_token"
with rls_transaction(tenant_id, using=custom_alias):
pass
mock_connections.__getitem__.assert_called_with(custom_alias)
mock_set_alias.assert_called_once_with(custom_alias)
mock_reset_alias.assert_called_once_with("test_token")
def test_rls_transaction_uses_read_replica_from_router(
self, tenants_fixture, enable_read_replica
):
"""Test rls_transaction uses read replica alias from router."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with patch("api.db_utils.set_read_db_alias") as mock_set_alias:
with patch(
"api.db_utils.reset_read_db_alias"
) as mock_reset_alias:
mock_set_alias.return_value = "test_token"
with rls_transaction(tenant_id):
pass
mock_connections.__getitem__.assert_called()
mock_set_alias.assert_called_once()
mock_reset_alias.assert_called_once()
def test_rls_transaction_fallback_to_default_when_alias_not_in_connections(
self, tenants_fixture
):
"""Test rls_transaction falls back to DEFAULT_DB_ALIAS when alias not in connections."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
invalid_alias = "nonexistent_db"
with patch("api.db_utils.get_read_db_alias", return_value=invalid_alias):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
def contains_check(alias):
return alias == DEFAULT_DB_ALIAS
mock_connections.__contains__.side_effect = contains_check
mock_connections.__getitem__.return_value = mock_conn
with patch("api.db_utils.transaction.atomic"):
with rls_transaction(tenant_id):
pass
mock_connections.__getitem__.assert_called_with(DEFAULT_DB_ALIAS)
def test_rls_transaction_successful_execution_on_replica_no_retries(
self, tenants_fixture, enable_read_replica
):
"""Test successful execution on replica without retries."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with patch("api.db_utils.set_read_db_alias", return_value="token"):
with patch("api.db_utils.reset_read_db_alias"):
with rls_transaction(tenant_id):
pass
assert mock_cursor.execute.call_count == 1
def test_rls_transaction_retry_with_exponential_backoff_on_operational_error(
self, tenants_fixture, enable_read_replica
):
"""Test retry with exponential backoff on OperationalError on replica."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
call_count = 0
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
raise OperationalError("Connection error")
return MagicMock(
__enter__=MagicMock(return_value=None),
__exit__=MagicMock(return_value=False),
)
with patch(
"api.db_utils.transaction.atomic", side_effect=atomic_side_effect
):
with patch("api.db_utils.time.sleep") as mock_sleep:
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with patch("api.db_utils.logger") as mock_logger:
with rls_transaction(tenant_id):
pass
assert mock_sleep.call_count == 2
mock_sleep.assert_any_call(0.5)
mock_sleep.assert_any_call(1.0)
assert mock_logger.info.call_count == 2
def test_rls_transaction_max_three_attempts_for_replica(
self, tenants_fixture, enable_read_replica
):
"""Test maximum 3 attempts for replica database."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = OperationalError("Persistent error")
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(OperationalError):
with rls_transaction(tenant_id):
pass
assert mock_atomic.call_count == 3
def test_rls_transaction_only_one_attempt_for_primary(self, tenants_fixture):
"""Test only 1 attempt for primary database."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = OperationalError("Primary error")
with pytest.raises(OperationalError):
with rls_transaction(tenant_id):
pass
assert mock_atomic.call_count == 1
def test_rls_transaction_fallback_to_primary_after_max_attempts(
self, tenants_fixture, enable_read_replica
):
"""Test fallback to primary DB after max attempts on replica."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
call_count = 0
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
raise OperationalError("Replica error")
return MagicMock(
__enter__=MagicMock(return_value=None),
__exit__=MagicMock(return_value=False),
)
with patch(
"api.db_utils.transaction.atomic", side_effect=atomic_side_effect
):
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with patch("api.db_utils.logger") as mock_logger:
with rls_transaction(tenant_id):
pass
mock_logger.warning.assert_called_once()
warning_msg = mock_logger.warning.call_args[0][0]
assert "falling back to primary DB" in warning_msg
def test_rls_transaction_logger_warning_on_fallback(
self, tenants_fixture, enable_read_replica
):
"""Test logger warnings are emitted on fallback to primary."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
call_count = 0
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
raise OperationalError("Replica error")
return MagicMock(
__enter__=MagicMock(return_value=None),
__exit__=MagicMock(return_value=False),
)
with patch(
"api.db_utils.transaction.atomic", side_effect=atomic_side_effect
):
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with patch("api.db_utils.logger") as mock_logger:
with rls_transaction(tenant_id):
pass
assert mock_logger.info.call_count == 2
assert mock_logger.warning.call_count == 1
def test_rls_transaction_operational_error_raised_immediately_on_primary(
self, tenants_fixture
):
"""Test OperationalError raised immediately on primary without retry."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = OperationalError("Primary error")
with patch("api.db_utils.time.sleep") as mock_sleep:
with pytest.raises(OperationalError):
with rls_transaction(tenant_id):
pass
mock_sleep.assert_not_called()
def test_rls_transaction_operational_error_raised_after_max_attempts(
self, tenants_fixture, enable_read_replica
):
"""Test OperationalError raised after max attempts on replica."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = OperationalError(
"Persistent replica error"
)
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(OperationalError):
with rls_transaction(tenant_id):
pass
def test_rls_transaction_router_token_set_for_non_default_alias(
self, tenants_fixture
):
"""Test router token is set when using non-default alias."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
custom_alias = "custom_db"
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with patch("api.db_utils.set_read_db_alias") as mock_set_alias:
with patch("api.db_utils.reset_read_db_alias") as mock_reset_alias:
mock_set_alias.return_value = "test_token"
with rls_transaction(tenant_id, using=custom_alias):
pass
mock_set_alias.assert_called_once_with(custom_alias)
mock_reset_alias.assert_called_once_with("test_token")
def test_rls_transaction_router_token_reset_in_finally_block(self, tenants_fixture):
"""Test router token is reset in finally block even on error."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
custom_alias = "custom_db"
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.side_effect = Exception("Unexpected error")
with patch("api.db_utils.set_read_db_alias", return_value="test_token"):
with patch("api.db_utils.reset_read_db_alias") as mock_reset_alias:
with pytest.raises(Exception):
with rls_transaction(tenant_id, using=custom_alias):
pass
mock_reset_alias.assert_called_once_with("test_token")
def test_rls_transaction_router_token_not_set_for_default_alias(
self, tenants_fixture
):
"""Test router token is not set when using default alias."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
with patch("api.db_utils.set_read_db_alias") as mock_set_alias:
with patch(
"api.db_utils.reset_read_db_alias"
) as mock_reset_alias:
with rls_transaction(tenant_id):
pass
mock_set_alias.assert_not_called()
mock_reset_alias.assert_not_called()
def test_rls_transaction_set_config_query_executed_with_correct_params(
self, tenants_fixture
):
"""Test SET_CONFIG_QUERY executed with correct parameters."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with rls_transaction(tenant_id) as cursor:
cursor.execute("SELECT current_setting(%s)", [POSTGRES_TENANT_VAR])
result = cursor.fetchone()
assert result[0] == tenant_id
def test_rls_transaction_custom_parameter(self, tenants_fixture):
"""Test rls_transaction with custom parameter name."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
custom_param = "api.user_id"
with rls_transaction(tenant_id, parameter=custom_param) as cursor:
cursor.execute("SELECT current_setting(%s)", [custom_param])
result = cursor.fetchone()
assert result[0] == tenant_id
def test_rls_transaction_cursor_yielded_correctly(self, tenants_fixture):
"""Test cursor is yielded correctly."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with rls_transaction(tenant_id) as cursor:
assert cursor is not None
cursor.execute("SELECT 1")
result = cursor.fetchone()
assert result[0] == 1
+82
View File
@@ -20,8 +20,12 @@ from prowler.providers.aws.aws_provider import AwsProvider
from prowler.providers.aws.lib.security_hub.security_hub import SecurityHubConnection
from prowler.providers.azure.azure_provider import AzureProvider
from prowler.providers.gcp.gcp_provider import GcpProvider
from prowler.providers.github.github_provider import GithubProvider
from prowler.providers.iac.iac_provider import IacProvider
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
from prowler.providers.m365.m365_provider import M365Provider
from prowler.providers.mongodbatlas.mongodbatlas_provider import MongodbatlasProvider
from prowler.providers.oraclecloud.oraclecloud_provider import OraclecloudProvider
class TestMergeDicts:
@@ -108,6 +112,10 @@ class TestReturnProwlerProvider:
(Provider.ProviderChoices.AZURE.value, AzureProvider),
(Provider.ProviderChoices.KUBERNETES.value, KubernetesProvider),
(Provider.ProviderChoices.M365.value, M365Provider),
(Provider.ProviderChoices.GITHUB.value, GithubProvider),
(Provider.ProviderChoices.MONGODBATLAS.value, MongodbatlasProvider),
(Provider.ProviderChoices.ORACLECLOUD.value, OraclecloudProvider),
(Provider.ProviderChoices.IAC.value, IacProvider),
],
)
def test_return_prowler_provider(self, provider_type, expected_provider):
@@ -203,6 +211,14 @@ class TestGetProwlerProviderKwargs:
Provider.ProviderChoices.GITHUB.value,
{"organizations": ["provider_uid"]},
),
(
Provider.ProviderChoices.ORACLECLOUD.value,
{},
),
(
Provider.ProviderChoices.MONGODBATLAS.value,
{"atlas_organization_id": "provider_uid"},
),
],
)
def test_get_prowler_provider_kwargs(self, provider_type, expected_extra_kwargs):
@@ -240,6 +256,72 @@ class TestGetProwlerProviderKwargs:
expected_result = {**secret_dict, "mutelist_content": {"key": "value"}}
assert result == expected_result
def test_get_prowler_provider_kwargs_iac_provider(self):
"""Test that IaC provider gets correct kwargs with repository URL."""
provider_uid = "https://github.com/org/repo"
secret_dict = {"access_token": "test_token"}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IAC.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {
"scan_repository_url": provider_uid,
"oauth_app_token": "test_token",
}
assert result == expected_result
def test_get_prowler_provider_kwargs_iac_provider_without_token(self):
"""Test that IaC provider works without access token for public repos."""
provider_uid = "https://github.com/org/public-repo"
secret_dict = {}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IAC.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider)
expected_result = {"scan_repository_url": provider_uid}
assert result == expected_result
def test_get_prowler_provider_kwargs_iac_provider_ignores_mutelist(self):
"""Test that IaC provider does NOT receive mutelist_content.
IaC provider uses Trivy's built-in mutelist logic, so it should not
receive mutelist_content even when a mutelist processor is configured.
"""
provider_uid = "https://github.com/org/repo"
secret_dict = {"access_token": "test_token"}
secret_mock = MagicMock()
secret_mock.secret = secret_dict
mutelist_processor = MagicMock()
mutelist_processor.configuration = {"Mutelist": {"key": "value"}}
provider = MagicMock()
provider.provider = Provider.ProviderChoices.IAC.value
provider.secret = secret_mock
provider.uid = provider_uid
result = get_prowler_provider_kwargs(provider, mutelist_processor)
# IaC provider should NOT have mutelist_content
assert "mutelist_content" not in result
expected_result = {
"scan_repository_url": provider_uid,
"oauth_app_token": "test_token",
}
assert result == expected_result
def test_get_prowler_provider_kwargs_unsupported_provider(self):
# Setup
provider_uid = "provider_uid"
File diff suppressed because it is too large. Load diff
+52 -7
View File
@@ -18,8 +18,11 @@ from prowler.providers.azure.azure_provider import AzureProvider
from prowler.providers.common.models import Connection
from prowler.providers.gcp.gcp_provider import GcpProvider
from prowler.providers.github.github_provider import GithubProvider
from prowler.providers.iac.iac_provider import IacProvider
from prowler.providers.kubernetes.kubernetes_provider import KubernetesProvider
from prowler.providers.m365.m365_provider import M365Provider
from prowler.providers.mongodbatlas.mongodbatlas_provider import MongodbatlasProvider
from prowler.providers.oraclecloud.oraclecloud_provider import OraclecloudProvider
class CustomOAuth2Client(OAuth2Client):
@@ -65,8 +68,11 @@ def return_prowler_provider(
| AzureProvider
| GcpProvider
| GithubProvider
| IacProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
| OraclecloudProvider
]:
"""Return the Prowler provider class based on the given provider type.
@@ -74,7 +80,7 @@ def return_prowler_provider(
provider (Provider): The provider object containing the provider type and associated secrets.
Returns:
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider: The corresponding provider class.
AwsProvider | AzureProvider | GcpProvider | GithubProvider | IacProvider | KubernetesProvider | M365Provider | OraclecloudProvider | MongodbatlasProvider: The corresponding provider class.
Raises:
ValueError: If the provider type specified in `provider.provider` is not supported.
@@ -92,6 +98,12 @@ def return_prowler_provider(
prowler_provider = M365Provider
case Provider.ProviderChoices.GITHUB.value:
prowler_provider = GithubProvider
case Provider.ProviderChoices.MONGODBATLAS.value:
prowler_provider = MongodbatlasProvider
case Provider.ProviderChoices.IAC.value:
prowler_provider = IacProvider
case Provider.ProviderChoices.ORACLECLOUD.value:
prowler_provider = OraclecloudProvider
case _:
raise ValueError(f"Provider type {provider.provider} not supported")
return prowler_provider
@@ -128,10 +140,26 @@ def get_prowler_provider_kwargs(
**prowler_provider_kwargs,
"organizations": [provider.uid],
}
elif provider.provider == Provider.ProviderChoices.IAC.value:
# For IaC provider, uid contains the repository URL
# Extract the access token if present in the secret
prowler_provider_kwargs = {
"scan_repository_url": provider.uid,
}
if "access_token" in provider.secret.secret:
prowler_provider_kwargs["oauth_app_token"] = provider.secret.secret[
"access_token"
]
elif provider.provider == Provider.ProviderChoices.MONGODBATLAS.value:
prowler_provider_kwargs = {
**prowler_provider_kwargs,
"atlas_organization_id": provider.uid,
}
if mutelist_processor:
mutelist_content = mutelist_processor.configuration.get("Mutelist", {})
if mutelist_content:
# IaC provider doesn't support mutelist (uses Trivy's built-in logic)
if mutelist_content and provider.provider != Provider.ProviderChoices.IAC.value:
prowler_provider_kwargs["mutelist_content"] = mutelist_content
return prowler_provider_kwargs
@@ -145,8 +173,11 @@ def initialize_prowler_provider(
| AzureProvider
| GcpProvider
| GithubProvider
| IacProvider
| KubernetesProvider
| M365Provider
| MongodbatlasProvider
| OraclecloudProvider
):
"""Initialize a Prowler provider instance based on the given provider type.
@@ -155,8 +186,8 @@ def initialize_prowler_provider(
mutelist_processor (Processor): The mutelist processor object containing the mutelist configuration.
Returns:
AwsProvider | AzureProvider | GcpProvider | GithubProvider | KubernetesProvider | M365Provider: An instance of the corresponding provider class
(`AwsProvider`, `AzureProvider`, `GcpProvider`, `GithubProvider`, `KubernetesProvider` or `M365Provider`) initialized with the
AwsProvider | AzureProvider | GcpProvider | GithubProvider | IacProvider | KubernetesProvider | M365Provider | OraclecloudProvider | MongodbatlasProvider: An instance of the corresponding provider class
(`AwsProvider`, `AzureProvider`, `GcpProvider`, `GithubProvider`, `IacProvider`, `KubernetesProvider`, `M365Provider`, `OraclecloudProvider` or `MongodbatlasProvider`) initialized with the
provider's secrets.
"""
prowler_provider = return_prowler_provider(provider)
@@ -180,9 +211,23 @@ def prowler_provider_connection_test(provider: Provider) -> Connection:
except Provider.secret.RelatedObjectDoesNotExist as secret_error:
return Connection(is_connected=False, error=secret_error)
return prowler_provider.test_connection(
**prowler_provider_kwargs, provider_id=provider.uid, raise_on_exception=False
)
# For IaC provider, construct the kwargs properly for test_connection
if provider.provider == Provider.ProviderChoices.IAC.value:
# Don't pass repository_url from secret, use scan_repository_url with the UID
iac_test_kwargs = {
"scan_repository_url": provider.uid,
"raise_on_exception": False,
}
# Add access_token if present in the secret
if "access_token" in prowler_provider_kwargs:
iac_test_kwargs["access_token"] = prowler_provider_kwargs["access_token"]
return prowler_provider.test_connection(**iac_test_kwargs)
else:
return prowler_provider.test_connection(
**prowler_provider_kwargs,
provider_id=provider.uid,
raise_on_exception=False,
)
def prowler_integration_connection_test(integration: Integration) -> Connection:
+18
View File
@@ -12,6 +12,24 @@ from api.models import StateChoices, Task
from api.v1.serializers import TaskSerializer
class DisablePaginationMixin:
disable_pagination_query_param = "page[disable]"
disable_pagination_truthy_values = {"true"}
def should_disable_pagination(self) -> bool:
if not hasattr(self, "request"):
return False
value = self.request.query_params.get(self.disable_pagination_query_param)
if value is None:
return False
return str(value).lower() in self.disable_pagination_truthy_values
def paginate_queryset(self, queryset):
if self.should_disable_pagination():
return None
return super().paginate_queryset(queryset)
class PaginateByPkMixin:
"""
Mixin to paginate on a list of PKs (cheaper than heavy JOINs),
@@ -1,5 +1,6 @@
import re
from drf_spectacular.utils import extend_schema_field
from rest_framework_json_api import serializers
@@ -11,3 +12,198 @@ class OpenAICredentialsSerializer(serializers.Serializer):
if not re.match(pattern, value or ""):
raise serializers.ValidationError("Invalid OpenAI API key format.")
return value
def to_internal_value(self, data):
"""Check for unknown fields before DRF filters them out."""
if not isinstance(data, dict):
raise serializers.ValidationError(
{"non_field_errors": ["Credentials must be an object"]}
)
allowed_fields = set(self.fields.keys())
provided_fields = set(data.keys())
extra_fields = provided_fields - allowed_fields
if extra_fields:
raise serializers.ValidationError(
{
"non_field_errors": [
f"Unknown fields in credentials: {', '.join(sorted(extra_fields))}"
]
}
)
return super().to_internal_value(data)
class BedrockCredentialsSerializer(serializers.Serializer):
"""
Serializer for AWS Bedrock credentials validation.
Validates long-term AWS credentials (AKIA) and region format.
"""
access_key_id = serializers.CharField()
secret_access_key = serializers.CharField()
region = serializers.CharField()
def validate_access_key_id(self, value: str) -> str:
"""Validate AWS access key ID format (AKIA for long-term credentials)."""
pattern = r"^AKIA[0-9A-Z]{16}$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS access key ID format. Must be AKIA followed by 16 alphanumeric characters."
)
return value
def validate_secret_access_key(self, value: str) -> str:
"""Validate AWS secret access key format (40 base64 characters)."""
pattern = r"^[A-Za-z0-9/+=]{40}$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS secret access key format. Must be 40 base64 characters."
)
return value
def validate_region(self, value: str) -> str:
"""Validate AWS region format."""
pattern = r"^[a-z]{2}-[a-z]+-\d+$"
if not re.match(pattern, value or ""):
raise serializers.ValidationError(
"Invalid AWS region format. Expected format like 'us-east-1' or 'eu-west-2'."
)
return value
def to_internal_value(self, data):
"""Check for unknown fields before DRF filters them out."""
if not isinstance(data, dict):
raise serializers.ValidationError(
{"non_field_errors": ["Credentials must be an object"]}
)
allowed_fields = set(self.fields.keys())
provided_fields = set(data.keys())
extra_fields = provided_fields - allowed_fields
if extra_fields:
raise serializers.ValidationError(
{
"non_field_errors": [
f"Unknown fields in credentials: {', '.join(sorted(extra_fields))}"
]
}
)
return super().to_internal_value(data)
class BedrockCredentialsUpdateSerializer(BedrockCredentialsSerializer):
"""
Serializer for AWS Bedrock credentials during UPDATE operations.
Inherits all validation logic from BedrockCredentialsSerializer but makes
all fields optional to support partial updates.
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Make all fields optional for updates
for field in self.fields.values():
field.required = False
class OpenAICompatibleCredentialsSerializer(serializers.Serializer):
"""
Minimal serializer for OpenAI-compatible credentials.
Many OpenAI-compatible providers do not use the same key format as OpenAI.
We only require a non-empty API key string. Additional fields can be added later
without breaking existing configurations.
"""
api_key = serializers.CharField()
def validate_api_key(self, value: str) -> str:
if not isinstance(value, str) or not value.strip():
raise serializers.ValidationError("API key is required.")
return value.strip()
def to_internal_value(self, data):
"""Check for unknown fields before DRF filters them out."""
if not isinstance(data, dict):
raise serializers.ValidationError(
{"non_field_errors": ["Credentials must be an object"]}
)
allowed_fields = set(self.fields.keys())
provided_fields = set(data.keys())
extra_fields = provided_fields - allowed_fields
if extra_fields:
raise serializers.ValidationError(
{
"non_field_errors": [
f"Unknown fields in credentials: {', '.join(sorted(extra_fields))}"
]
}
)
return super().to_internal_value(data)
@extend_schema_field(
{
"oneOf": [
{
"type": "object",
"title": "OpenAI Credentials",
"properties": {
"api_key": {
"type": "string",
"description": "OpenAI API key. Must start with 'sk-' followed by alphanumeric characters, "
"hyphens, or underscores.",
"pattern": "^sk-[\\w-]+$",
}
},
"required": ["api_key"],
},
{
"type": "object",
"title": "AWS Bedrock Credentials",
"properties": {
"access_key_id": {
"type": "string",
"description": "AWS access key ID.",
"pattern": "^AKIA[0-9A-Z]{16}$",
},
"secret_access_key": {
"type": "string",
"description": "AWS secret access key.",
"pattern": "^[A-Za-z0-9/+=]{40}$",
},
"region": {
"type": "string",
"description": "AWS region identifier where Bedrock is available. Examples: us-east-1, "
"us-west-2, eu-west-1, ap-northeast-1.",
"pattern": "^[a-z]{2}-[a-z]+-\\d+$",
},
},
"required": ["access_key_id", "secret_access_key", "region"],
},
{
"type": "object",
"title": "OpenAI Compatible Credentials",
"properties": {
"api_key": {
"type": "string",
"description": "API key for OpenAI-compatible provider. The format varies by provider. "
"Note: The 'base_url' field (separate from credentials) is required when using this provider type.",
}
},
"required": ["api_key"],
},
]
}
)
class LighthouseCredentialsField(serializers.JSONField):
pass
@@ -239,6 +239,71 @@ from rest_framework_json_api import serializers
},
"required": ["github_app_id", "github_app_key"],
},
{
"type": "object",
"title": "IaC Repository Credentials",
"properties": {
"repository_url": {
"type": "string",
"description": "Repository URL to scan for IaC files.",
},
"access_token": {
"type": "string",
"description": "Optional access token for private repositories.",
},
},
"required": ["repository_url"],
},
{
"type": "object",
"title": "Oracle Cloud Infrastructure (OCI) API Key Credentials",
"properties": {
"user": {
"type": "string",
"description": "The OCID of the user to authenticate with.",
},
"fingerprint": {
"type": "string",
"description": "The fingerprint of the API signing key.",
},
"key_file": {
"type": "string",
"description": "The path to the private key file for API signing. Either key_file or key_content must be provided.",
},
"key_content": {
"type": "string",
"description": "The content of the private key for API signing (base64 encoded). Either key_file or key_content must be provided.",
},
"tenancy": {
"type": "string",
"description": "The OCID of the tenancy.",
},
"region": {
"type": "string",
"description": "The OCI region identifier (e.g., us-ashburn-1, us-phoenix-1).",
},
"pass_phrase": {
"type": "string",
"description": "The passphrase for the private key, if encrypted.",
},
},
"required": ["user", "fingerprint", "tenancy", "region"],
},
{
"type": "object",
"title": "MongoDB Atlas API Key",
"properties": {
"atlas_public_key": {
"type": "string",
"description": "MongoDB Atlas API public key.",
},
"atlas_private_key": {
"type": "string",
"description": "MongoDB Atlas API private key.",
},
},
"required": ["atlas_public_key", "atlas_private_key"],
},
]
}
)
+514 -7
View File
@@ -21,6 +21,7 @@ from rest_framework_simplejwt.tokens import RefreshToken
from api.db_router import MainRouter
from api.exceptions import ConflictException
from api.models import (
AttackPathsScan,
Finding,
Integration,
IntegrationProviderRelationship,
@@ -31,6 +32,7 @@ from api.models import (
LighthouseProviderModels,
LighthouseTenantConfiguration,
Membership,
MuteRule,
Processor,
Provider,
ProviderGroup,
@@ -46,6 +48,7 @@ from api.models import (
StatusChoices,
Task,
TenantAPIKey,
ThreatScoreSnapshot,
User,
UserRoleRelationship,
)
@@ -59,7 +62,13 @@ from api.v1.serializer_utils.integrations import (
S3ConfigSerializer,
SecurityHubConfigSerializer,
)
from api.v1.serializer_utils.lighthouse import OpenAICredentialsSerializer
from api.v1.serializer_utils.lighthouse import (
BedrockCredentialsSerializer,
BedrockCredentialsUpdateSerializer,
LighthouseCredentialsField,
OpenAICompatibleCredentialsSerializer,
OpenAICredentialsSerializer,
)
from api.v1.serializer_utils.processors import ProcessorConfigField
from api.v1.serializer_utils.providers import ProviderSecretField
from prowler.lib.mutelist.mutelist import Mutelist
@@ -1119,6 +1128,109 @@ class ScanComplianceReportSerializer(serializers.Serializer):
fields = ["id", "name"]
class AttackPathsScanSerializer(RLSSerializer):
state = StateEnumSerializerField(read_only=True)
provider_alias = serializers.SerializerMethodField(read_only=True)
provider_type = serializers.SerializerMethodField(read_only=True)
provider_uid = serializers.SerializerMethodField(read_only=True)
class Meta:
model = AttackPathsScan
fields = [
"id",
"state",
"progress",
"provider",
"provider_alias",
"provider_type",
"provider_uid",
"scan",
"task",
"inserted_at",
"started_at",
"completed_at",
"duration",
]
included_serializers = {
"provider": "api.v1.serializers.ProviderIncludeSerializer",
"scan": "api.v1.serializers.ScanIncludeSerializer",
"task": "api.v1.serializers.TaskSerializer",
}
def get_provider_alias(self, obj):
provider = getattr(obj, "provider", None)
return provider.alias if provider else None
def get_provider_type(self, obj):
provider = getattr(obj, "provider", None)
return provider.provider if provider else None
def get_provider_uid(self, obj):
provider = getattr(obj, "provider", None)
return provider.uid if provider else None
class AttackPathsQueryParameterSerializer(serializers.Serializer):
name = serializers.CharField()
label = serializers.CharField()
data_type = serializers.CharField(default="string")
description = serializers.CharField(allow_null=True, required=False)
placeholder = serializers.CharField(allow_null=True, required=False)
class JSONAPIMeta:
resource_name = "attack-paths-query-parameter"
class AttackPathsQuerySerializer(serializers.Serializer):
id = serializers.CharField()
name = serializers.CharField()
description = serializers.CharField()
provider = serializers.CharField()
parameters = AttackPathsQueryParameterSerializer(many=True)
class JSONAPIMeta:
resource_name = "attack-paths-query"
class AttackPathsQueryRunRequestSerializer(serializers.Serializer):
id = serializers.CharField()
parameters = serializers.DictField(
child=serializers.JSONField(), allow_empty=True, required=False
)
class JSONAPIMeta:
resource_name = "attack-paths-query-run-request"
class AttackPathsNodeSerializer(serializers.Serializer):
id = serializers.CharField()
labels = serializers.ListField(child=serializers.CharField())
properties = serializers.DictField(child=serializers.JSONField())
class JSONAPIMeta:
resource_name = "attack-paths-query-result-node"
class AttackPathsRelationshipSerializer(serializers.Serializer):
id = serializers.CharField()
label = serializers.CharField()
source = serializers.CharField()
target = serializers.CharField()
properties = serializers.DictField(child=serializers.JSONField())
class JSONAPIMeta:
resource_name = "attack-paths-query-result-relationship"
class AttackPathsQueryResultSerializer(serializers.Serializer):
nodes = AttackPathsNodeSerializer(many=True)
relationships = AttackPathsRelationshipSerializer(many=True)
class JSONAPIMeta:
resource_name = "attack-paths-query-result"
class ResourceTagSerializer(RLSSerializer):
"""
Serializer for the ResourceTag model
@@ -1159,11 +1271,17 @@ class ResourceSerializer(RLSSerializer):
"findings",
"failed_findings_count",
"url",
"metadata",
"details",
"partition",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"metadata": {"read_only": True},
"details": {"read_only": True},
"partition": {"read_only": True},
}
included_serializers = {
@@ -1220,11 +1338,15 @@ class ResourceIncludeSerializer(RLSSerializer):
"service",
"type_",
"tags",
"details",
"partition",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"details": {"read_only": True},
"partition": {"read_only": True},
}
@extend_schema_field(
@@ -1355,10 +1477,16 @@ class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
serializer = GCPProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.GITHUB.value:
serializer = GithubProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.IAC.value:
serializer = IacProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.KUBERNETES.value:
serializer = KubernetesProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.M365.value:
serializer = M365ProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.ORACLECLOUD.value:
serializer = OracleCloudProviderSecret(data=secret)
elif provider_type == Provider.ProviderChoices.MONGODBATLAS.value:
serializer = MongoDBAtlasProviderSecret(data=secret)
else:
raise serializers.ValidationError(
{"provider": f"Provider type not supported {provider_type}"}
@@ -1455,6 +1583,14 @@ class GCPServiceAccountProviderSecret(serializers.Serializer):
resource_name = "provider-secrets"
class MongoDBAtlasProviderSecret(serializers.Serializer):
atlas_public_key = serializers.CharField()
atlas_private_key = serializers.CharField()
class Meta:
resource_name = "provider-secrets"
class KubernetesProviderSecret(serializers.Serializer):
kubeconfig_content = serializers.CharField()
@@ -1472,6 +1608,27 @@ class GithubProviderSecret(serializers.Serializer):
resource_name = "provider-secrets"
class IacProviderSecret(serializers.Serializer):
repository_url = serializers.CharField()
access_token = serializers.CharField(required=False)
class Meta:
resource_name = "provider-secrets"
class OracleCloudProviderSecret(serializers.Serializer):
user = serializers.CharField()
fingerprint = serializers.CharField()
key_file = serializers.CharField(required=False)
key_content = serializers.CharField(required=False)
tenancy = serializers.CharField()
region = serializers.CharField()
pass_phrase = serializers.CharField(required=False)
class Meta:
resource_name = "provider-secrets"
class AWSRoleAssumptionProviderSecret(serializers.Serializer):
role_arn = serializers.CharField()
external_id = serializers.CharField()
@@ -2105,6 +2262,17 @@ class OverviewProviderSerializer(serializers.Serializer):
}
class OverviewProviderCountSerializer(serializers.Serializer):
id = serializers.CharField(source="provider")
count = serializers.IntegerField()
class JSONAPIMeta:
resource_name = "providers-count-overview"
def get_root_meta(self, _resource, _many):
return {"version": "v1"}
class OverviewFindingSerializer(serializers.Serializer):
id = serializers.CharField(default="n/a")
new = serializers.IntegerField()
@@ -2165,6 +2333,30 @@ class OverviewServiceSerializer(serializers.Serializer):
return {"version": "v1"}
class OverviewRegionSerializer(serializers.Serializer):
id = serializers.SerializerMethodField()
provider_type = serializers.CharField()
region = serializers.CharField()
total = serializers.IntegerField()
_pass = serializers.IntegerField()
fail = serializers.IntegerField()
muted = serializers.IntegerField()
class JSONAPIMeta:
resource_name = "regions-overview"
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields["pass"] = self.fields.pop("_pass")
def get_id(self, obj):
"""Generate unique ID from provider_type and region."""
return f"{obj['provider_type']}:{obj['region']}"
def get_root_meta(self, _resource, _many):
return {"version": "v1"}
# Schedules
@@ -3049,7 +3241,12 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
Accepts credentials as JSON; stored encrypted via credentials_decoded.
"""
credentials = serializers.JSONField(write_only=True, required=True)
credentials = LighthouseCredentialsField(write_only=True, required=True)
base_url = serializers.URLField(
required=False,
allow_null=True,
help_text="Base URL for the LLM provider API. Required for 'openai_compatible' provider type.",
)
class Meta:
model = LighthouseProviderConfiguration
@@ -3061,7 +3258,10 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
]
extra_kwargs = {
"is_active": {"required": False},
"base_url": {"required": False, "allow_null": True},
"provider_type": {
"help_text": "LLM provider type. Determines which credential format to use. "
"See 'credentials' field documentation for provider-specific requirements."
},
}
def create(self, validated_data):
@@ -3084,6 +3284,7 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
def validate(self, attrs):
provider_type = attrs.get("provider_type")
credentials = attrs.get("credentials") or {}
base_url = attrs.get("base_url")
if provider_type == LighthouseProviderConfiguration.LLMProviderChoices.OPENAI:
try:
@@ -3096,6 +3297,35 @@ class LighthouseProviderConfigCreateSerializer(RLSSerializer, BaseWriteSerialize
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
provider_type == LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
try:
BedrockCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE
):
if not base_url:
raise ValidationError({"base_url": "Base URL is required."})
try:
OpenAICompatibleCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
return super().validate(attrs)
@@ -3105,7 +3335,12 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
Update serializer for LighthouseProviderConfiguration.
"""
credentials = serializers.JSONField(write_only=True, required=False)
credentials = LighthouseCredentialsField(write_only=True, required=False)
base_url = serializers.URLField(
required=False,
allow_null=True,
help_text="Base URL for the LLM provider API. Required for 'openai_compatible' provider type.",
)
class Meta:
model = LighthouseProviderConfiguration
@@ -3119,7 +3354,6 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
extra_kwargs = {
"id": {"read_only": True},
"provider_type": {"read_only": True},
"base_url": {"required": False, "allow_null": True},
"is_active": {"required": False},
}
@@ -3130,7 +3364,11 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
setattr(instance, attr, value)
if credentials is not None:
instance.credentials_decoded = credentials
# Merge partial credentials with existing ones
# New values overwrite existing ones, but unspecified fields are preserved
existing_credentials = instance.credentials_decoded or {}
merged_credentials = {**existing_credentials, **credentials}
instance.credentials_decoded = merged_credentials
instance.save()
return instance
@@ -3138,6 +3376,7 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
def validate(self, attrs):
provider_type = getattr(self.instance, "provider_type", None)
credentials = attrs.get("credentials", None)
base_url = attrs.get("base_url", None)
if (
credentials is not None
@@ -3154,6 +3393,40 @@ class LighthouseProviderConfigUpdateSerializer(BaseWriteSerializer):
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
credentials is not None
and provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.BEDROCK
):
try:
BedrockCredentialsUpdateSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
elif (
credentials is not None
and provider_type
== LighthouseProviderConfiguration.LLMProviderChoices.OPENAI_COMPATIBLE
):
if base_url is None:
pass
elif not base_url:
raise ValidationError({"base_url": "Base URL cannot be empty."})
try:
OpenAICompatibleCredentialsSerializer(data=credentials).is_valid(
raise_exception=True
)
except ValidationError as e:
details = e.detail.copy()
for key, value in details.items():
e.detail[f"credentials/{key}"] = value
del e.detail[key]
raise e
return super().validate(attrs)
@@ -3171,7 +3444,7 @@ class LighthouseTenantConfigSerializer(RLSSerializer):
def get_url(self, obj):
request = self.context.get("request")
return reverse("lighthouse-config", request=request)
return reverse("lighthouse-configurations", request=request)
class Meta:
model = LighthouseTenantConfiguration
@@ -3319,3 +3592,237 @@ class LighthouseProviderModelsUpdateSerializer(BaseWriteSerializer):
extra_kwargs = {
"id": {"read_only": True},
}
# Mute Rules
class MuteRuleSerializer(RLSSerializer):
"""
Serializer for reading MuteRule instances.
"""
finding_uids = serializers.ListField(
child=serializers.CharField(),
read_only=True,
help_text="List of finding UIDs that are muted by this rule",
)
class Meta:
model = MuteRule
fields = [
"id",
"inserted_at",
"updated_at",
"name",
"reason",
"enabled",
"created_by",
"finding_uids",
]
included_serializers = {
"created_by": "api.v1.serializers.UserIncludeSerializer",
}
class MuteRuleCreateSerializer(RLSSerializer, BaseWriteSerializer):
"""
Serializer for creating new MuteRule instances.
Accepts finding_ids in the request, converts them to UIDs, and stores in finding_uids.
"""
finding_ids = serializers.ListField(
child=serializers.UUIDField(),
write_only=True,
required=True,
help_text="List of Finding IDs to mute (will be converted to UIDs)",
)
finding_uids = serializers.ListField(
child=serializers.CharField(),
read_only=True,
help_text="List of finding UIDs that are muted by this rule",
)
class Meta:
model = MuteRule
fields = [
"id",
"inserted_at",
"updated_at",
"name",
"reason",
"enabled",
"created_by",
"finding_ids",
"finding_uids",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"updated_at": {"read_only": True},
"enabled": {"read_only": True},
"created_by": {"read_only": True},
}
def validate_name(self, value):
"""Validate that the name is unique within the tenant."""
tenant_id = self.context.get("tenant_id")
if MuteRule.objects.filter(tenant_id=tenant_id, name=value).exists():
raise ValidationError("A mute rule with this name already exists.")
return value
def validate_finding_ids(self, value):
"""Validate that all finding IDs exist and belong to the tenant."""
if not value:
raise ValidationError("At least one finding_id must be provided.")
tenant_id = self.context.get("tenant_id")
# Check that all findings exist and belong to this tenant
findings = Finding.all_objects.filter(tenant_id=tenant_id, id__in=value)
found_ids = set(findings.values_list("id", flat=True))
provided_ids = set(value)
missing_ids = provided_ids - found_ids
if missing_ids:
raise ValidationError(
f"The following finding IDs do not exist or do not belong to your tenant: {missing_ids}"
)
return value
def validate(self, data):
"""Validate the entire mute rule, including overlap detection."""
data = super().validate(data)
tenant_id = self.context.get("tenant_id")
finding_ids = data.get("finding_ids", [])
if not finding_ids:
return data
# Convert finding IDs to UIDs (deduplicate in case multiple findings have same UID)
findings = Finding.all_objects.filter(id__in=finding_ids, tenant_id=tenant_id)
finding_uids = list(set(findings.values_list("uid", flat=True)))
# Check for overlaps with existing enabled rules
existing_rules = MuteRule.objects.filter(tenant_id=tenant_id, enabled=True)
for rule in existing_rules:
overlap = set(finding_uids) & set(rule.finding_uids)
if overlap:
raise ConflictException(
detail=f"The following finding UIDs are already muted by rule '{rule.name}': {overlap}"
)
# Store finding_uids in validated_data for create
data["finding_uids"] = finding_uids
return data
def create(self, validated_data):
"""Create a new mute rule and set created_by."""
# Remove finding_ids from validated_data (we've already converted to finding_uids)
validated_data.pop("finding_ids", None)
# Set created_by to the current user
request = self.context.get("request")
if request and hasattr(request, "user"):
validated_data["created_by"] = request.user
return super().create(validated_data)
class MuteRuleUpdateSerializer(BaseWriteSerializer):
"""
Serializer for updating MuteRule instances.
"""
class Meta:
model = MuteRule
fields = [
"id",
"name",
"reason",
"enabled",
]
extra_kwargs = {
"id": {"read_only": True},
"name": {"required": False},
"reason": {"required": False},
"enabled": {"required": False},
}
def validate_name(self, value):
"""Validate that the name is unique within the tenant, excluding current instance."""
tenant_id = self.context.get("tenant_id")
if (
MuteRule.objects.filter(tenant_id=tenant_id, name=value)
.exclude(id=self.instance.id)
.exists()
):
raise ValidationError("A mute rule with this name already exists.")
return value
# ThreatScore Snapshots
class ThreatScoreSnapshotSerializer(RLSSerializer):
"""
Serializer for ThreatScore snapshots.
Read-only serializer for retrieving historical ThreatScore metrics.
"""
id = serializers.SerializerMethodField()
class Meta:
model = ThreatScoreSnapshot
fields = [
"id",
"inserted_at",
"scan",
"provider",
"compliance_id",
"overall_score",
"score_delta",
"section_scores",
"critical_requirements",
"total_requirements",
"passed_requirements",
"failed_requirements",
"manual_requirements",
"total_findings",
"passed_findings",
"failed_findings",
]
extra_kwargs = {
"id": {"read_only": True},
"inserted_at": {"read_only": True},
"scan": {"read_only": True},
"provider": {"read_only": True},
"compliance_id": {"read_only": True},
"overall_score": {"read_only": True},
"score_delta": {"read_only": True},
"section_scores": {"read_only": True},
"critical_requirements": {"read_only": True},
"total_requirements": {"read_only": True},
"passed_requirements": {"read_only": True},
"failed_requirements": {"read_only": True},
"manual_requirements": {"read_only": True},
"total_findings": {"read_only": True},
"passed_findings": {"read_only": True},
"failed_findings": {"read_only": True},
}
included_serializers = {
"scan": "api.v1.serializers.ScanIncludeSerializer",
"provider": "api.v1.serializers.ProviderIncludeSerializer",
}
def get_id(self, obj):
if getattr(obj, "_aggregated", False):
return "n/a"
return str(obj.id)
+7 -2
View File
@@ -4,6 +4,7 @@ from drf_spectacular.views import SpectacularRedocView
from rest_framework_nested import routers
from api.v1.views import (
AttackPathsScanViewSet,
ComplianceOverviewViewSet,
CustomSAMLLoginView,
CustomTokenObtainView,
@@ -21,6 +22,7 @@ from api.v1.views import (
LighthouseProviderModelsViewSet,
LighthouseTenantConfigViewSet,
MembershipViewSet,
MuteRuleViewSet,
OverviewViewSet,
ProcessorViewSet,
ProviderGroupProvidersRelationshipView,
@@ -52,6 +54,9 @@ router.register(r"tenants", TenantViewSet, basename="tenant")
router.register(r"providers", ProviderViewSet, basename="provider")
router.register(r"provider-groups", ProviderGroupViewSet, basename="providergroup")
router.register(r"scans", ScanViewSet, basename="scan")
router.register(
r"attack-paths-scans", AttackPathsScanViewSet, basename="attack-paths-scans"
)
router.register(r"tasks", TaskViewSet, basename="task")
router.register(r"resources", ResourceViewSet, basename="resource")
router.register(r"findings", FindingViewSet, basename="finding")
@@ -80,6 +85,7 @@ router.register(
LighthouseProviderModelsViewSet,
basename="lighthouse-models",
)
router.register(r"mute-rules", MuteRuleViewSet, basename="mute-rule")
tenants_router = routers.NestedSimpleRouter(router, r"tenants", lookup="tenant")
tenants_router.register(
@@ -150,13 +156,12 @@ urlpatterns = [
),
name="provider_group-providers-relationship",
),
# Lighthouse tenant config as singleton endpoint
path(
"lighthouse/configuration",
LighthouseTenantConfigViewSet.as_view(
{"get": "list", "patch": "partial_update"}
),
name="lighthouse-config",
name="lighthouse-configurations",
),
# API endpoint to start SAML SSO flow
path(
File diff suppressed because it is too large. Load diff
+1
View File
@@ -1,6 +1,7 @@
import warnings
from celery import Celery, Task
from config.env import env
# Suppress specific warnings from django-rest-auth: https://github.com/iMerica/dj-rest-auth/issues/684
+6
View File
@@ -36,6 +36,12 @@ DATABASES = {
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
},
"neo4j": {
"HOST": env.str("NEO4J_HOST", "neo4j"),
"PORT": env.str("NEO4J_PORT", "7687"),
"USER": env.str("NEO4J_USER", "neo4j"),
"PASSWORD": env.str("NEO4J_PASSWORD", "neo4j_password"),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
@@ -37,6 +37,12 @@ DATABASES = {
"HOST": env("POSTGRES_REPLICA_HOST", default=default_db_host),
"PORT": env("POSTGRES_REPLICA_PORT", default=default_db_port),
},
"neo4j": {
"HOST": env.str("NEO4J_HOST"),
"PORT": env.str("NEO4J_PORT"),
"USER": env.str("NEO4J_USER"),
"PASSWORD": env.str("NEO4J_PASSWORD"),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
-2
View File
@@ -1,7 +1,5 @@
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path("admin/", admin.site.urls),
path("api/v1/", include("api.v1.urls")),
]
+171 -17
View File
@@ -1,8 +1,11 @@
import logging
from types import SimpleNamespace
from datetime import datetime, timedelta, timezone
from unittest.mock import MagicMock, patch
import pytest
from allauth.socialaccount.models import SocialLogin
from django.conf import settings
from django.db import connection as django_connection
@@ -11,10 +14,14 @@ from django.urls import reverse
from django_celery_results.models import TaskResult
from rest_framework import status
from rest_framework.test import APIClient
from tasks.jobs.backfill import backfill_resource_scan_summaries
from api.attack_paths import (
AttackPathsQueryDefinition,
AttackPathsQueryParameterDefinition,
)
from api.db_utils import rls_transaction
from api.models import (
AttackPathsScan,
ComplianceOverview,
ComplianceRequirementOverview,
Finding,
@@ -23,6 +30,7 @@ from api.models import (
Invitation,
LighthouseConfiguration,
Membership,
MuteRule,
Processor,
Provider,
ProviderGroup,
@@ -46,6 +54,7 @@ from api.rls import Tenant
from api.v1.serializers import TokenSerializer
from prowler.lib.check.models import Severity
from prowler.lib.outputs.finding import Status
from tasks.jobs.backfill import backfill_resource_scan_summaries
TODAY = str(datetime.today().date())
API_JSON_CONTENT_TYPE = "application/vnd.api+json"
@@ -158,22 +167,20 @@ def create_test_user_rbac_no_roles(django_db_setup, django_db_blocker, tenants_f
@pytest.fixture(scope="function")
def create_test_user_rbac_limited(django_db_setup, django_db_blocker):
def create_test_user_rbac_limited(django_db_setup, django_db_blocker, tenants_fixture):
with django_db_blocker.unblock():
user = User.objects.create_user(
name="testing_limited",
email="rbac_limited@rbac.com",
password=TEST_PASSWORD,
)
tenant = Tenant.objects.create(
name="Tenant Test",
)
tenant = tenants_fixture[0]
Membership.objects.create(
user=user,
tenant=tenant,
role=Membership.RoleChoices.OWNER,
)
Role.objects.create(
role = Role.objects.create(
name="limited",
tenant_id=tenant.id,
manage_users=False,
@@ -186,7 +193,7 @@ def create_test_user_rbac_limited(django_db_setup, django_db_blocker):
)
UserRoleRelationship.objects.create(
user=user,
role=Role.objects.get(name="limited"),
role=role,
tenant_id=tenant.id,
)
return user
@@ -499,8 +506,29 @@ def providers_fixture(tenants_fixture):
alias="m365_testing",
tenant_id=tenant.id,
)
provider7 = Provider.objects.create(
provider="oraclecloud",
uid="ocid1.tenancy.oc1..aaaaaaaa3dwoazoox4q7wrvriywpokp5grlhgnkwtyt6dmwyou7no6mdmzda",
alias="oci_testing",
tenant_id=tenant.id,
)
provider8 = Provider.objects.create(
provider="mongodbatlas",
uid="64b1d3c0e4b03b1234567890",
alias="mongodbatlas_testing",
tenant_id=tenant.id,
)
return provider1, provider2, provider3, provider4, provider5, provider6
return (
provider1,
provider2,
provider3,
provider4,
provider5,
provider6,
provider7,
provider8,
)
@pytest.fixture
@@ -1086,8 +1114,8 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
region="region1",
_pass=1,
fail=0,
muted=0,
total=1,
muted=2,
total=3,
new=1,
changed=0,
unchanged=0,
@@ -1095,7 +1123,7 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
fail_changed=0,
pass_new=1,
pass_changed=0,
muted_new=0,
muted_new=2,
muted_changed=0,
scan=scan,
)
@@ -1108,8 +1136,8 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
region="region2",
_pass=0,
fail=1,
muted=1,
total=2,
muted=3,
total=4,
new=2,
changed=0,
unchanged=0,
@@ -1117,7 +1145,7 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
fail_changed=0,
pass_new=0,
pass_changed=0,
muted_new=1,
muted_new=3,
muted_changed=0,
scan=scan,
)
@@ -1130,8 +1158,8 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
region="region1",
_pass=1,
fail=0,
muted=0,
total=1,
muted=1,
total=2,
new=1,
changed=0,
unchanged=0,
@@ -1139,7 +1167,7 @@ def scan_summaries_fixture(tenants_fixture, providers_fixture):
fail_changed=0,
pass_new=1,
pass_changed=0,
muted_new=0,
muted_new=1,
muted_changed=0,
scan=scan,
)
@@ -1419,6 +1447,132 @@ def api_keys_fixture(tenants_fixture, create_test_user):
return [api_key1, api_key2, api_key3]
@pytest.fixture
def mute_rules_fixture(tenants_fixture, create_test_user, findings_fixture):
"""Create test mute rules for testing."""
tenant = tenants_fixture[0]
user = create_test_user
# Create two mute rules: one enabled, one disabled
mute_rule1 = MuteRule.objects.create(
tenant_id=tenant.id,
name="Test Rule 1",
reason="Security exception for testing",
enabled=True,
created_by=user,
finding_uids=[findings_fixture[0].uid],
)
mute_rule2 = MuteRule.objects.create(
tenant_id=tenant.id,
name="Test Rule 2",
reason="Compliance exception approved",
enabled=False,
created_by=user,
finding_uids=[findings_fixture[1].uid],
)
return mute_rule1, mute_rule2
@pytest.fixture
def create_attack_paths_scan():
"""Factory fixture to create Attack Paths scans for tests."""
def _create(
provider,
*,
scan=None,
state=StateChoices.COMPLETED,
progress=0,
graph_database="tenant-db",
**extra_fields,
):
scan_instance = scan or Scan.objects.create(
name=extra_fields.pop("scan_name", "Attack Paths Supporting Scan"),
provider=provider,
trigger=Scan.TriggerChoices.MANUAL,
state=extra_fields.pop("scan_state", StateChoices.COMPLETED),
tenant_id=provider.tenant_id,
)
payload = {
"tenant_id": provider.tenant_id,
"provider": provider,
"scan": scan_instance,
"state": state,
"progress": progress,
"graph_database": graph_database,
}
payload.update(extra_fields)
return AttackPathsScan.objects.create(**payload)
return _create
@pytest.fixture
def attack_paths_query_definition_factory():
"""Factory fixture for building Attack Paths query definitions."""
def _create(**overrides):
cast_type = overrides.pop("cast_type", str)
parameters = overrides.pop(
"parameters",
[
AttackPathsQueryParameterDefinition(
name="limit",
label="Limit",
cast=cast_type,
)
],
)
definition_payload = {
"id": "aws-test",
"name": "Attack Paths Test Query",
"description": "Synthetic Attack Paths definition for tests.",
"provider": "aws",
"cypher": "RETURN 1",
"parameters": parameters,
}
definition_payload.update(overrides)
return AttackPathsQueryDefinition(**definition_payload)
return _create
@pytest.fixture
def attack_paths_graph_stub_classes():
"""Provide lightweight graph element stubs for Attack Paths serialization tests."""
class AttackPathsNativeValue:
def __init__(self, value):
self._value = value
def to_native(self):
return self._value
class AttackPathsNode:
def __init__(self, element_id, labels, properties):
self.element_id = element_id
self.labels = labels
self._properties = properties
class AttackPathsRelationship:
def __init__(self, element_id, rel_type, start_node, end_node, properties):
self.element_id = element_id
self.type = rel_type
self.start_node = start_node
self.end_node = end_node
self._properties = properties
return SimpleNamespace(
NativeValue=AttackPathsNativeValue,
Node=AttackPathsNode,
Relationship=AttackPathsRelationship,
)
def get_authorization_header(access_token: str) -> dict:
return {"Authorization": f"Bearer {access_token}"}
Binary file not shown.

After

Width:  |  Height:  |  Size: 95 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

+7
View File
@@ -7,6 +7,7 @@ from tasks.tasks import perform_scheduled_scan_task
from api.db_utils import rls_transaction
from api.exceptions import ConflictException
from api.models import Provider, Scan, StateChoices
from tasks.jobs.attack_paths import db_utils as attack_paths_db_utils
def schedule_provider_scan(provider_instance: Provider):
@@ -39,6 +40,12 @@ def schedule_provider_scan(provider_instance: Provider):
scheduled_at=datetime.now(timezone.utc),
)
attack_paths_db_utils.create_attack_paths_scan(
tenant_id=tenant_id,
scan_id=str(scheduled_scan.id),
provider_id=provider_id,
)
# Schedule the task
periodic_task_instance = PeriodicTask.objects.create(
interval=schedule,
@@ -0,0 +1,5 @@
from tasks.jobs.attack_paths.scan import run as attack_paths_scan
__all__ = [
"attack_paths_scan",
]
Loaded 100 of 1156 files, more files were not shown because too many files have changed in this diff. Show more