Compare commits

...
Author SHA1 Message Date
César Arroba 5be514a991 fix(api): fail migrations fast when blocked on a lock
Set lock_timeout on the migration session so a metadata-only migration
blocked waiting for ACCESS EXCLUSIVE on a busy table fails loudly instead
of queueing every later reader behind it. Configurable via
DJANGO_MIGRATION_LOCK_TIMEOUT (default 5s, 0 disables it).
2026-07-15 18:49:48 +02:00
Prowler Botandprowler-bot 0d899b3076 chore(release): Bump versions to v5.35.0 (#12004)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-15 18:16:32 +02:00
Prowler Botandprowler-bot 3fd9fca32f chore(changelog): v5.34.0 (#11999)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-15 16:21:14 +02:00
Josema Camacho 07d7e9d5bb fix(api): retry wrapped Neptune transient errors (#11996) 2026-07-15 16:01:04 +02:00
Pepe Fagoaga 31b8ab9b4b docs(families): link to public announcement (#11995) 2026-07-15 15:59:08 +02:00
Pepe Fagoaga 60c4e9b257 docs(compliance): unify section pages (#11993) 2026-07-15 13:45:45 +02:00
Hugo Pereira Brito 8e9af708f8 feat(aws): add elbv2_listener_pqc_tls_enabled security check (#11254) 2026-07-15 12:45:23 +01:00
Deep ShahandDaniel Barranquero 24b670ac36 feat(sdk): add AWS Amplify app secret scanning check (#11825)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-15 13:44:20 +02:00
Pepe Fagoaga cc6c731af6 docs: new product family (#11984) 2026-07-15 12:28:50 +02:00
Daniel Barranqueroandalejandrobailo c53acd4184 fix(dashboard): correct UTM tags on local dashboard Prowler Cloud links (#11988)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-15 12:17:02 +02:00
Alejandro Bailo faa74f4c6c feat(ui): add Local Server Cloud upgrade prompts (#11982) 2026-07-15 12:11:28 +02:00
Josema Camacho 077dff7f68 fix(api): resolve critical container scan findings (#11991) 2026-07-15 12:08:52 +02:00
Josema Camacho a4752d6a27 fix(api): prevent false Attack Paths stale cleanup (#11986) 2026-07-15 11:07:21 +02:00
Toriola Opeyemi a9f6e04a84 docs: fix malformed height attribute in ECR badge (#11987) 2026-07-15 08:07:56 +02:00
Andoni Alonso d9224e682f docs(github): fix broken fine-grained PAT anchor in authentication table (#11985) 2026-07-14 17:44:06 +02:00
Daniel Barranquero 9822fd97d7 feat(dashboard): redesign local dashboard sidebar and pages (#11972) 2026-07-14 14:40:21 +02:00
Josema Camacho bd72ec91ea fix(api): combine permissions across assigned roles (#11979) 2026-07-14 13:52:12 +02:00
Haitao ZhengandHugo P.Brito fa9b2c707b feat(kubernetes): add hostPath volume check (#11837)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-14 12:23:59 +01:00
22401a54a0 feat(kubernetes): add core check for readonly root filesystem enabled (#11835)
Co-authored-by: wbro <80239840234@proton.me>
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-14 12:23:17 +01:00
Daniel Barranquero 9c15796c84 fix(ci): extend .trivyignore CVE suppression expiries to 2026-08-15 (#11975) 2026-07-14 12:26:33 +02:00
Prowler Botandprowler-bot db9356ba86 chore(changelog): v5.33.2 forward-sync to master (#11977)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-14 12:16:36 +02:00
Pepe Fagoaga 54237d824a chore(banner): missing feats and highlight component (#11974) 2026-07-14 12:15:13 +02:00
Pepe Fagoaga 7df1054966 chore(step-security): allow endpoints (#11973) 2026-07-14 12:02:35 +02:00
Josema Camacho 53772e2931 fix(api): prevent concurrent scan summary deadlocks (#11970) 2026-07-14 10:40:26 +02:00
Josema Camacho d37d5058bb fix(api): bound attack paths normalized-list child IDs (#11960) 2026-07-14 09:48:12 +02:00
Josema Camacho 8debf70d5c fix(api): retry transient attack paths graph mutations (#11961) 2026-07-14 09:45:52 +02:00
Pepe Fagoaga 1a1e804c00 chore(banner): highlight link to Prowler Cloud (#11964) 2026-07-14 08:46:52 +02:00
stepsecurity-app[bot]andstepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com> c7583a5633 feat(security): security best practices from StepSecurity (#11962)
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>
2026-07-14 08:17:06 +02:00
Adrián Peña 470e218bb8 fix(sdk): preserve regional IMDSv2 account findings (#11959) 2026-07-13 17:32:08 +02:00
Alejandro Bailo 488f3d1bed fix(ui): improve profile and role visibility UX (#11956) 2026-07-13 16:50:43 +02:00
Hugo Pereira Brito 0f6137dd04 fix(aws): target EC2 Amazon AMI loading (#11945) 2026-07-13 15:33:30 +01:00
Alejandro Bailo f5f6e5768d fix(ui): improve Lighthouse overview navigation (#11955) 2026-07-13 16:12:48 +02:00
Hugo Pereira Brito c6dae3a711 chore: remove __init__.py from test directories (#10582) 2026-07-13 15:09:42 +01:00
4242297e72 feat(aws): Update regions for AWS services (#11954)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-13 16:09:17 +02:00
8d4be0f586 feat(aws): Adding CF Template to do full org deployment (#10403)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-07-13 10:42:53 +02:00
Pedro MartínandPepe Fagoaga d78e0189e0 docs(compliance): add cross-provider feature docs (#11944)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-13 10:35:46 +02:00
Hugo Pereira Brito cef131812c fix(ci): correct bot and dependency PR labeling (#11425) 2026-07-13 09:06:12 +01:00
Pepe Fagoaga 3c78a0df43 fix(changelog): correct fragments for the UI (#11952) 2026-07-13 09:31:57 +02:00
mintlify[bot]andmintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> 991e8b8061 docs: brand tone and writing style fixes (#11953)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-07-13 09:14:24 +02:00
Alejandro Bailo dbdbd8a379 feat(ui): add cross-provider compliance view (#11912) 2026-07-10 17:34:53 +02:00
AmanandHugo P.Brito a0a0883578 feat(azure): add check to ensure Function Apps redirect HTTP to HTTPS (#11929)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-10 13:57:58 +01:00
Prowler Botandprowler-bot a4bf70eecf chore(changelog): v5.33.1 forward-sync to master (#11948)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-10 14:50:12 +02:00
Hugo Pereira Brito 5bc41b2609 docs: require duplicate PR check (#11946) 2026-07-10 12:59:49 +01:00
Pablo Fernandez Guerra (PFE)andPablo F.G 58fd4170b5 feat(ui): support dynamic providers (#11869)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-10 13:22:41 +02:00
GOUTHAM HARIGOVINDandDaniel Barranquero 3369e48260 feat(ec2): Implement AMI block public access check (#11794) (#11828)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-10 13:10:15 +02:00
Pepe Fagoaga 106026614d chore(security): allow internal endpoints for Lighthouse AI OpenAI compatible (#11941) 2026-07-10 12:55:16 +02:00
Adrián PeñaandJosema Camacho 3f2e5929d9 fix(api): harden Lighthouse provider base URLs (#11928)
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-07-10 11:24:40 +02:00
Josema Camacho c93ea035fc fix(api): make AWS Attack Paths query aggregation Neo4j compatible (#11931) 2026-07-10 11:19:08 +02:00
César Arroba 847997672d fix(ui): reference renamed UI_POSTHOG_ENABLED flag in metronome billing guard (#11938) 2026-07-10 11:02:34 +02:00
lydiavilchez 08ee83c572 fix(docs): use python3 and add CI check for provider cards hook (#11930) 2026-07-10 09:27:34 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 892cc2bc07 refactor(ui): rename reserved billing flag to CLOUD_BILLING_ENABLED (#11920)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-10 09:26:33 +02:00
stepsecurity-app[bot]stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>Pepe Fagoaga
2a077cae5e feat(security): security best practices from StepSecurity (#11937)
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-10 09:00:40 +02:00
1ee4de18be chore: add trailing newlines to 7 files for POSIX compliance (#11765)
Co-authored-by: Janderik Marins <janderik@email.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 18:19:55 +02:00
Josema CamachoandPepe Fagoaga b44f8ebf5f fix(api): add query-level retry with primary fallback to rls_transaction via execute_wrapper (#10379)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-09 17:47:03 +02:00
Hugo Pereira Brito 6f72785a28 fix(azure): warn on optional function app permission failures (#11922) 2026-07-09 16:23:09 +01:00
Hugo Pereira Brito be78fe8374 fix(jira): surface dispatch failures (#11918) 2026-07-09 16:09:12 +01:00
Pedro Martín 01c004a7af fix(ui): handle level 1 requirements for M365 CIS (#11921) 2026-07-09 16:41:51 +02:00
Yinka MetricsandDaniel Barranquero 0b36d08b92 feat(sdk): add Data Pipeline secret scanning check (#11821)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 16:21:05 +02:00
Josema Camacho bf9c53a4c3 fix(api): keep auth tests order-safe after token revocation (#11919) 2026-07-09 15:41:51 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 80c5363649 refactor(ui): rename integration enable flags to past tense (#11917)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-09 13:09:35 +02:00
César Arroba 9d899ae0e2 ci: rename public ECR push-role secret to PUBLIC_ECR_PUSH_ROLE_ARN (#11916) 2026-07-09 11:27:55 +02:00
Narahari RaghavaandDaniel Barranquero 13bd6fc0bf fix(aws): check statement Effect instead of policy Statement in SCP a… (#11727)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 11:19:33 +02:00
Adrián Peña 18b3c49234 fix(api): invalidate tokens after password updates (#11901) 2026-07-09 10:36:58 +02:00
Adrián Peña 84ea68927f fix(api): scope user role updates to tenant (#11903) 2026-07-09 10:36:49 +02:00
Johannes EnglerandHugo P.Brito 1af9cdd351 feat(stackit): add iaas_server_public_ip_attached check (#11549)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-08 17:13:36 +01:00
Hugo Pereira Brito 25bcbac309 fix(dms): lazy load ec2 for public access check (#11899) 2026-07-08 16:42:23 +01:00
Hugo Pereira Brito 20aad80a78 fix(aws): avoid full ec2 inventory in dlm check (#11850) 2026-07-08 15:59:16 +01:00
Alejandro Bailo 52875b5c7c feat(ui): migrate from HeroUI to shadcn/ui (#11532) 2026-07-08 16:57:02 +02:00
Pepe Fagoaga c665005790 fix(workflow): bump AI issue triage (#11896) 2026-07-08 15:45:52 +02:00
mintlify[bot]andmintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> b2532ebfe5 docs(style): standardize "click" interaction verb (#11893)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-07-08 14:39:29 +02:00
Josema Camacho 3461f9ac49 test(api): reduce report, provider, auth, and sentry test costs (#11888) 2026-07-08 14:31:53 +02:00
UniCodeandDaniel Barranquero d874fc573d feat(e2e): provider for e2e cloud (#11654)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-08 14:21:27 +02:00
Hugo Pereira Brito 991c204a88 fix(sdk): limit ECS task definitions by registration date (#11868) 2026-07-08 13:09:25 +01:00
Hugo Pereira BritoandPablo F.G eee17f0e8b fix(ui): clarify Unlimited Visibility in RBAC forms (#11851)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-08 12:42:18 +01:00
Hugo Pereira Brito 80608bfdbc feat(oraclecloud): support regionless SDK setup (#11853) 2026-07-08 12:16:07 +01:00
Adrián Peña 6c5f54808d feat: add changelog fragments workflow (#11572) 2026-07-08 10:19:12 +02:00
Pepe Fagoaga bb6608c1d7 chore(ui): unify trial expired banner (#11713) 2026-07-08 10:05:36 +02:00
Prowler Botandprowler-bot 18e1bf5195 chore(release): Bump versions to v5.34.0 (#11874)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-07 18:23:01 +02:00
César Arroba 76a2d7bfe6 ci: skip codeql and e2e on changelog-only changes (#11867) 2026-07-07 14:27:41 +02:00
ac3f289de6 feat(ui): gate Sentry, GTM and PostHog behind runtime enable flags (#11682)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:43:34 +02:00
Rubén De la Torre Vico 838f82b255 docs(lighthouse): document Lighthouse AI on Prowler Cloud (#11758) 2026-07-07 13:27:37 +02:00
Josema Camacho 5e00c4bfcf chore(docs): update changelogs for 5.33.0 (#11866) 2026-07-07 13:07:40 +02:00
lydiavilchez f0ae56b8ea feat(docs): auto-generate provider cards in Prowler App tutorial (#11865) 2026-07-07 12:35:52 +02:00
Hugo Pereira BritoandDaniel Barranquero 4484c2f192 fix(azure): refine resource group scoped scan follow-ups (#11796)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 11:20:38 +02:00
Pedro Martín e6bbcb8043 chore(scan-config): improve wording and docs (#11859) 2026-07-07 11:14:19 +02:00
Hugo Pereira Brito ad04e69c35 fix(kubernetes): reject exec auth in cloud kubeconfigs (#11753) 2026-07-07 09:57:46 +01:00
Adrián Peña 6cae37174c fix(api): queue provider scans when one is active (#11848) 2026-07-07 10:28:41 +02:00
Pepe Fagoaga aa6de57430 chore(multi-tenant): delete last tenant from profile page (#11864) 2026-07-07 10:02:36 +02:00
Alejandro Bailo a48aa37f87 fix(ui): Lighthouse chat archive navigation, new chat button, and masked stored credentials (#11860) 2026-07-07 09:57:47 +02:00
Pedro Martín 3cc8f86780 feat(api): remove provider credentials for PDF report (#11845) 2026-07-07 09:45:34 +02:00
AbhinavandDaniel Barranquero 4cb02a0ead fix(azure): read Flexible Server log retention from logfiles.retention_days (fixes #11757) (#11761)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 08:19:44 +02:00
Alejandro Bailo 221c558cee feat(ui): connect Lighthouse v2 to Cloud backend (#11690) 2026-07-06 17:30:31 +02:00
Pedro MartínandCésar Arroba 81c3152ebb chore(changelog): prepare for 5.32.1 (#11856)
Co-authored-by: César Arroba <19954079+cesararroba@users.noreply.github.com>
2026-07-06 16:14:20 +02:00
Pedro Martín 855e9a043e chore(changelog): prepare for 5.32.1 (#11854) 2026-07-06 16:01:31 +02:00
Adrián Peñaandalejandrobailo 7b5d724bb7 fix: handle invitations in social and SAML auth (#11752)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-06 14:49:07 +02:00
Alejandro Bailo 441f2a3c48 fix(ui): enable triage editing in compliance findings table (#11829) 2026-07-06 11:03:37 +02:00
Hugo Pereira Brito efb86bb7ab chore: remove Dependabot config (#11834) 2026-07-03 12:48:17 +01:00
Hugo Pereira Brito 398a0a484f fix(api): restrict user profile updates to self (#11792) 2026-07-03 11:53:01 +01:00
Sanjay Santhanamandpedrooot 55924d8150 fix(compliance): skip MANUAL findings in section tally to avoid KeyError (#11823)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-07-03 12:19:32 +02:00
Josema Camacho 0cf6f2f83e fix(api): add attack paths scan DB defaults (#11826) 2026-07-03 11:30:55 +02:00
Pepe Fagoaga cf18093261 chore(onboarding): rename account -> provider (#11827) 2026-07-03 09:33:37 +02:00
Josema Camacho 1850e209e6 docs(attack-paths): update tutorial for the new UI (#11824) 2026-07-02 18:34:49 +02:00
Prowler Botandprowler-bot 2e37188c9f chore(release): Bump versions to v5.33.0 (#11820)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-02 16:56:03 +02:00
Josema Camacho 4ae7c67d3f docs: add findings triage guide (#11793) 2026-07-02 13:33:30 +02:00
Pepe Fagoaga a76ba156d5 chore(changelog): v5.32.0 (#11805) 2026-07-02 13:17:56 +02:00
Alejandro Bailo cd90a91158 fix(ui): address findings triage QA feedback (#11791) 2026-07-02 12:43:06 +02:00
Son Sulung Suryahatta AsnanandDaniel Barranquero e1b23e2526 feat(s3): new check s3_bucket_object_public to detect public objects (#9517)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-02 11:41:29 +02:00
Legin 537c3ea71e feat(azure): filtering scans at resource group level (#10657)
Signed-off-by: Legin-ML <leginml2004@gmail.com>
2026-07-02 10:27:53 +01:00
Maringanti Vasist AcharyaandHugo P.Brito b6f74c7284 feat(m365): add exchange application access policy check (#11247)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-02 10:16:07 +01:00
César Arroba 8cd008ba91 chore(ci): enable Renovate routine version updates (#11789) 2026-07-02 10:34:54 +02:00
César Arroba 1f13e1d348 chore(ci): disable Dependabot version updates in favor of Renovate (#11788) 2026-07-02 09:30:32 +02:00
Josema Camacho 87a15d7bb8 feat(api): support timestamp precision in findings filters (#11754) 2026-07-02 09:27:22 +02:00
renovate[bot]andrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com> f5bdacd07a chore(docker): update docker (#11601)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-07-02 09:08:50 +02:00
renovate[bot]andrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com> ce80fcd430 chore(ci): update github-actions (#11314)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-07-02 09:08:16 +02:00
Pedro Martín c1c080b072 docs(scan-config): add scan config for CLI as reference (#11786) 2026-07-02 08:59:18 +02:00
Alan Buscagliaandalejandrobailo 587187419f feat(ui): add findings triage (#11704)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-01 17:55:33 +02:00
César Arroba 050a5915ca fix(ci): detect conflict markers in route-group paths and flag unmergeable PRs (#11763) 2026-07-01 17:50:33 +02:00
Alejandro Bailo d4e4d12c5a docs(skills): add design system discipline to prowler-ui (#11764) 2026-07-01 17:31:07 +02:00
Josema Camacho 72cf2a65a6 perf(api): optimize attack paths graph cleanup (#11755) 2026-07-01 16:28:59 +02:00
Pedro MartínandPepe Fagoaga 48db27481d docs(scan-config): add info about attaching scan config (#11751)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-01 16:06:38 +02:00
Adrián Peña 1247c5fb33 docs: clarify SAML userType role mapping (#11759) 2026-07-01 15:48:46 +02:00
Pedro Martínandalejandrobailo 69321418a3 feat(ui): improve scan config ux (#11731)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-01 15:45:38 +02:00
Pepe Fagoaga 301d13a4b9 docs: unify subscription banner (#11756) 2026-07-01 15:12:45 +02:00
Chirag TrivediandDaniel Barranquero 3f8c1e822f feat(apigateway): add check for secrets in REST API stage variables (#11188)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 14:24:18 +02:00
Sergio Garcia 1e1c1c018b feat(iam): add AWS Bedrock AgentCore privilege escalation paths (#11726) 2026-07-01 12:49:34 +02:00
琴焚音落andDaniel Barranquero a212916a49 fix(github): skip archived repos in CODEOWNERS check (#11735)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 12:29:13 +02:00
Samyak ChoudharyandDaniel Barranquero 883ffa1fdb feat(m365): add Entra Conditional Access group management restriction (#11342)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 11:38:54 +02:00
Daniel Barranquero 21d9d6192e feat(okta): add configurable API request throttling and rate-limit retries (#11702) 2026-07-01 10:30:43 +02:00
Hugo Pereira BritoandDaniel Barranquero fd38a0ac03 feat(awslambda): add secrets_ignore_files to skip false-positive files (e.g. *.deps.json) in no-secrets-in-code check (#11222)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 09:03:25 +01:00
César Arroba af6918d57b chore(api): opt out of PowerShell telemetry in the API image (#11746) 2026-07-01 10:02:59 +02:00
César Arroba 9a9cbc997b ci(workflows): push SDK image to Public ECR via OIDC (#11749) 2026-06-30 21:17:57 +02:00
César Arroba aec500ee3b fix(ci): harden pull_request_target workflows (persist-credentials + toJson) (#11747) 2026-06-30 18:29:29 +02:00
César Arroba 8fbc721223 ci(workflows): allowlist nodejs.org and Iconify endpoints in UI tests harden-runner (#11744) 2026-06-30 17:43:02 +02:00
Josema Camacho c3ce3d2b3c fix(api): preflight attack paths graph databases (#11743) 2026-06-30 17:01:48 +02:00
Hugo Pereira Brito c46cbaaa4a feat(sdk): limit selected high-volume AWS resource analysis (#11228) 2026-06-30 15:49:12 +01:00
Daniel Barranquero 34e8e3ca61 chore(api): replace detect-secrets with kingfisher-bin to match the SDK (#11698) 2026-06-30 15:59:18 +02:00
Daniel Barranquero 5dac8a0a53 feat(sdk): replace detect-secrets library with kingfisher (#11694) 2026-06-30 15:36:23 +02:00
ed1fec8866 docs: add scan scheduling guide (#11729)
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-30 15:07:08 +02:00
Daniel Barranquero 2abcb05e22 feat(github): support repository rulesets in default branch protection checks (#11723) 2026-06-30 13:53:32 +01:00
César Arroba aba43440ca chore(ci): remove legacy prowler-api/ui/mcp cloud-deployment dispatch jobs (#11644) 2026-06-30 12:53:51 +02:00
Prowler Botandprowler-bot d47cbb4f8c feat(aws): Update regions for AWS services (#11662)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-30 12:46:29 +02:00
Hugo Pereira Brito 36006de8ce chore(ci): bump Trivy scanner to v0.71.2 (#11728) 2026-06-30 11:45:55 +01:00
Pedro Martín 5ccb044b85 docs(scan-config): add new changes (#11732) 2026-06-30 12:34:26 +02:00
Hugo Pereira Brito e40e9a6483 fix(docker): remove build dependencies from images (#11730) 2026-06-30 11:31:51 +01:00
Pedro Martín 717d48b0e0 chore(config): add missing providers config (#11725) 2026-06-30 11:32:47 +02:00
Josema Camacho 4e7e2f7eab fix(api): cap attack paths sink sync batches (#11724) 2026-06-29 17:48:02 +02:00
Pedro Martín 5404863a3e feat(compliance): add CIS 2.0.1 for K8S (#11722) 2026-06-29 16:10:13 +02:00
Pedro MartínandHugo Pereira Brito 9d4b6c4d16 feat(compliance): add CIS 1.2 for github (#11719)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-06-29 15:54:30 +02:00
Davlet DzhakishevandDaniel Barranquero cd56985480 fix(azure): accept AuditEvent key vault diagnostics (#11660)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-29 14:05:12 +02:00
Hugo Pereira Brito 58eb0fa095 ci: allow Formbucket egress in secrets scan (#11718) 2026-06-29 12:45:52 +01:00
Pedro Martín d850349a1c feat(ui): rename scan configuration endpoint (#11710) 2026-06-29 13:36:38 +02:00
Pedro Martín 36a609f2ee fix(compliance): avoid AttributeError in Okta generic compliance (#11715) 2026-06-29 13:09:51 +02:00
Pedro Martín 0c5ceb7e72 chore(compliance): add missing configrequirements (#11717) 2026-06-29 13:01:52 +02:00
Pedro Martín ed04257e6c feat(compliance): add CIS 5.0 for the gcp provider (#11714) 2026-06-29 12:37:59 +02:00
Pedro Martín e2b2e568a6 feat(compliance): add CIS 7.0 for the AWS provider. (#11707) 2026-06-29 12:12:17 +02:00
Pedro Martín dc432c8c3a feat(compliance): add CIS 6.0 for the azure provider (#11708) 2026-06-29 11:05:34 +02:00
César Arroba 6ffbb8373e ci: retry GitHub API curl in setup-python-uv action (#11711) 2026-06-29 10:39:14 +02:00
Pedro Martín 6bea847232 fix(ci): ignore unfixed libssh2 CVE-2026-55200 (#11709) 2026-06-29 10:32:49 +02:00
StylusFrost 78b94b7043 feat(sdk): add sdk_only provider property to hide providers from the app (#11578) 2026-06-26 16:42:22 +02:00
Vahid GharaviandHugo P.Brito 92634d4261 fix(azure/postgresql): isolate per-server collection failures (#11595)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-26 15:23:44 +01:00
Pedro Martín 007e32c690 chore(readme): update with latests stats (#11703) 2026-06-26 15:45:04 +02:00
Pedro Martín fe7e6675e0 feat(config): add compliance guardrails for the SDK config (#11669) 2026-06-26 15:14:08 +02:00
Pedro MartínandPepe Fagoaga d6f5f060ca feat(compliance): add CIS Controls v8.1 universal framework (#11700)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-26 13:27:02 +02:00
Pedro Martín 9d013910e6 feat(ui): scan configuration management page (#11695) 2026-06-26 12:55:31 +02:00
Pedro Martín 4c281aa464 feat(compliance): add CIS 7.0 for the M365 provider (#11699) 2026-06-26 12:45:12 +02:00
Siddhant JadhavandLydia Vilchez 086805df1d feat(waf): add check for regional web ACL logging enabled (#11539)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-26 12:31:44 +02:00
Josema Camacho 5793cd7e38 feat(api): make Attack Paths sink selectable between Neo4j and Neptune (#11524) 2026-06-26 10:22:29 +02:00
Siddhant JadhavandDaniel Barranquero 9b8b77cec0 feat(stepfunctions): add check for state machine encryption at rest (#11538)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-25 17:14:50 +02:00
5b9824c379 feat(ui): filter by provider group across main views (#11659)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 15:32:00 +02:00
Josema Camacho 2b7db88694 fix(api): handle deleted scans during progress saves (#11696) 2026-06-25 14:12:12 +02:00
Pepe Fagoaga fbddeea254 chore: change trial expired banner when launching scans (#11689) 2026-06-25 12:12:50 +02:00
Pepe Fagoaga 7785829969 chore: changelog v5.31.1 (#11691) 2026-06-25 08:28:34 +02:00
d62abeb407 feat(m365): add entra_conditional_access_policy_no_exclusion_gaps check (#11577)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: Ariel Eli <207917221+arieleli01212@users.noreply.github.com>
2026-06-24 17:03:15 +02:00
Josema Camacho 4e00cfd1b6 fix(api): avoid mutating API key manager during auth (#11686) 2026-06-24 16:50:55 +02:00
Davidm4r 917e5d07ff test(api): speed up API test suite (#11681) 2026-06-24 15:15:29 +02:00
Alejandro Bailo 76286f1186 fix(ui): improve scan scheduling flows (#11684) 2026-06-24 13:35:53 +02:00
93dd696a4f feat(m365): add Azure DevOps Conditional Access check (#11182)
Co-authored-by: Atlas-BountyHunter <atlas-bounty@hermes-agent.local>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-24 13:33:51 +02:00
Daniel Barranquero 36be63af07 fix(alibabacloud): implement ram_password_policy_number and fix cs weekly check loading (#11683) 2026-06-24 12:37:45 +02:00
Adrián Peña dc228e8b36 docs(api): move SAML changelog entry to v5.31.0 (#11677) 2026-06-23 17:18:20 +02:00
Rubén De la Torre Vico 058a1dc8fe chore: unify ruff tooling and route code quality through the Makefile (#11675) 2026-06-23 17:15:05 +02:00
Daniel Barranquero de7da3e960 docs: update provider stats and add Linode to introduction (#11676) 2026-06-23 17:00:35 +02:00
Prowler Botandprowler-bot 3b0124d3fd chore(release): Bump versions to v5.32.0 (#11673)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-23 16:53:29 +02:00
Josema Camacho fb995a79bf chore: modify changelogs for 5.31.0 release (#11671) 2026-06-23 14:09:52 +02:00
9d8c060c49 feat(azure): add recovery_vault_backup_policy_retention_adequate check (#11047)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-23 13:56:52 +02:00
Alejandro BailoandClaude Opus 4.8 0cabceb09c feat(ui): source scheduled scans tab from /schedules endpoint (#11670)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-23 13:44:37 +02:00
s1ns3nz0andDaniel Barranquero 3ee24fba51 feat(azure): add entra_user_with_recent_sign_in check (#11040)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-23 13:13:02 +02:00
s1ns3nz0andLydia Vilchez 48acb3bd2e feat(gcp): add secretmanager_secret_rotation_enabled check (#11026)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-23 11:30:15 +02:00
Alejandro Bailo c6c07957a6 fix(ui): use shared scan launch action errors (#11664) 2026-06-23 09:52:20 +02:00
Pedro MartínandPepe Fagoaga 0610866b73 feat(config): add SDK config's validator (#11518)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-23 09:40:32 +02:00
Hugo Pereira Brito 2afa18d3da fix(changelog): move recovery vault entry to 5.31 (#11666) 2026-06-22 20:37:07 +01:00
s1ns3nz0andHugo P.Brito a0fdc96649 feat(azure): add recovery_vault_has_protected_items check (#11048)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 19:59:25 +01:00
Nikhil KumarandHugo P.Brito b6caaa4268 feat(kubernetes): checks for memory limits, memory requests, and image tag (#11373)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 19:39:39 +01:00
Pedro MartínandHugo P.Brito 04e6e330a7 feat(aws): add rolesanywhere service and pqc trust anchor check (#11319)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 18:39:16 +01:00
s1ns3nz0andDaniel Barranquero 29329f6203 feat(azure): add entra_authentication_methods_policy_strong_auth_enforced check (#11039)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 18:18:22 +02:00
Pedro MartínandHugo P.Brito bdd44a0dce feat(aws): add acmpca service and pqc key algorithm check (#11318)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 17:13:17 +01:00
Pedro MartínandHugo P.Brito 10d9fc35e6 feat(aws): add cloudfront_distributions_pqc_tls_enabled check (#11317)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 16:44:51 +01:00
s1ns3nz0andDaniel Barranquero 6826422a6a feat(azure): add entra_app_registration_credential_not_expired check (#11038)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 17:30:18 +02:00
Pedro MartínandHugo P.Brito ca48fd0719 feat(aws): add apigateway_domain_name_pqc_tls_enabled check (#11316)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 16:12:01 +01:00
s1ns3nz0andDaniel Barranquero b9298b4023 feat(azure): add network_vnet_ddos_protection_enabled check (#11044)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 16:34:52 +02:00
Josema CamachoandCopilot Autofix powered by AI 2375f1d962 fix(api): uvicorn worker keepalive (#11663)
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-22 16:30:33 +02:00
Alejandro Bailo 5ee8b9680d feat(ui): add organization bulk scan scheduling (#11643) 2026-06-22 16:21:35 +02:00
Pedro MartínandHugo P.Brito 45cfe4e411 feat(aws): add transfer_server_pqc_ssh_kex_enabled check (#11315)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 14:30:15 +01:00
abdouandAdrián Jesús Peña Rodríguez 30d737c7d7 fix(api): bound Celery worker concurrency to a configurable default (#11075)
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
2026-06-22 14:05:11 +02:00
s1ns3nz0andDaniel Barranquero 869f0726f5 feat(azure): add network_subnet_nsg_associated check (#11043)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 14:03:54 +02:00
s1ns3nz0andDaniel Barranquero 6dda1ae485 feat(azure): add aks_cluster_local_accounts_disabled check (#11030)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 13:38:17 +02:00
s1ns3nz0andDaniel Barranquero 13f51de5c1 feat(azure): add aks_cluster_azure_monitor_enabled check (#11029)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-22 13:17:23 +02:00
s1ns3nz0andLydia Vilchez 5d5f0676e0 feat(gcp): add secretmanager_secret_not_publicly_accessible check (#11025)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-22 12:55:42 +02:00
8a1d7bcd6b feat(linode): add provider with administration compute and networking services (#11633)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-22 11:19:20 +02:00
s1ns3nz0andLydia Vilchez ccc1f161d2 feat(gcp): add cloudfunction_function_not_publicly_accessible check (#11022)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-22 10:26:03 +02:00
Pepe Fagoaga a7917f779a chore(sdk): changelog for v5.30.3 (#11651) 2026-06-19 15:21:38 +02:00
Pepe Fagoaga 7f96d895bb fix: API changelog from advisory merge (#11649) 2026-06-19 13:48:47 +02:00
Adrián PeñaandPepe Fagoaga bf3b5c2ba7 Merge commit from fork
* fix(saml): cross-tenant account takeover via SAML domain claiming

* chore(changelog): add PR #

* fix(api): bind SAML tokens to validated domain

- Reject SAML assertions with mismatched email domains
- Issue SAML tokens from the validated ACS tenant
- Add regression coverage for cross-tenant SAML token issuance

* fix(api): resolve SAML tenant inside RLS context

- Load the SAML tenant relation before leaving the RLS transaction
- Avoid lazy tenant lookups during the SAML ACS finish flow

---------

Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-19 13:38:51 +02:00
Hugo Pereira Brito 218f64595a fix(metadata): tag IAM policy privilege escalation check (#11648) 2026-06-19 13:09:35 +02:00
Josema Camacho 6d8d553610 fix(api): set gunicorn keep-alive above the load balancer idle timeout to stop 502s (#11647) 2026-06-19 12:49:49 +02:00
Pedro Martín e10cf34ad6 feat(compliance): DORA compliance framework for Alibaba Cloud (#11646) 2026-06-19 12:17:33 +02:00
s1ns3nz0andDaniel Barranquero bbf54011ea feat(azure): add postgresql_flexible_server_high_availability_enabled check (#11046)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-19 11:59:37 +02:00
Pedro Martín 9e173978dc feat(compliance): add DORA compliance framework for Cloudflare (#11645) 2026-06-19 10:37:07 +02:00
s1ns3nz0andDaniel Barranquero d27ec7d62e feat(azure): add postgresql_flexible_server_geo_redundant_backup_enabled check (#11045)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-19 10:21:38 +02:00
Pedro Martín 151dcd2895 feat(compliance): add DORA compliance framework for GCP (#11642) 2026-06-19 09:07:54 +02:00
s1ns3nz0andDaniel Barranquero d961d7efe4 feat(azure): add mysql_flexible_server_high_availability_enabled check (#11042)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-19 09:00:46 +02:00
Josema Camacho 99285d4656 fix(api): close DB connections per request to stop ASGI replica connection leak (#11640) 2026-06-18 17:42:19 +02:00
Adrián Peña 19629e9bb8 fix: simplify local dev launch workflow (#11641) 2026-06-18 16:51:02 +02:00
Adrián Peña b89b427a86 feat: add Makefile local development stack (#11637) 2026-06-18 16:37:42 +02:00
Alejandro Bailo 908d2ce766 feat(ui): per-provider scan schedule management gated by capability (#11521) 2026-06-18 15:47:03 +02:00
853610bbbf feat(ui): resolve public SaaS config at container runtime (#11500)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 15:12:18 +02:00
Pepe Fagoaga 751c7fc29f chore(jira): timeout to 90 seconds (#11638) 2026-06-18 13:03:19 +02:00
s1ns3nz0andDaniel Barranquero 7dd08bc6bf feat(azure): add mysql_flexible_server_geo_redundant_backup_enabled check (#11041)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-18 12:41:04 +02:00
Alejandro Bailo 2111d083df fix(ui): bump transitive dompurify to 3.4.10 to patch XSS advisories (#11636) 2026-06-18 12:00:58 +02:00
s1ns3nz0andHugo P.Brito 82d37c4978 feat(azure): add aks_cluster_defender_enabled check (#11028)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-18 10:22:51 +01:00
Pedro Martín aee3b392a7 fix(compliance): multi-section undercount & leaked provider tab (#11567) 2026-06-18 10:30:27 +02:00
s1ns3nz0andDaniel Barranquero ddbf3405a0 feat(azure): add defender_ensure_defender_cspm_is_on check (#11037)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-18 10:05:02 +02:00
2293cab72c fix(ui): adaptive Attack Paths messages for waiting states (#11512)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 10:03:35 +02:00
César Arroba 5a761f341b ci: remove redundant compliance CODEOWNERS rule (#11634) 2026-06-18 09:07:43 +02:00
s1ns3nz0andDaniel Barranquero 3c68a121e5 feat(azure): add databricks_workspace_no_public_ip_enabled check (#11036)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-18 09:06:25 +02:00
Pedro Martín c0ae8b9739 feat(compliance): add DORA compliance framework for Azure (#11551) 2026-06-18 08:56:04 +02:00
s1ns3nz0andLydia Vilchez 5ec4a1cbba feat(gcp): add cloudfunction_function_inside_vpc check (#11021)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-17 17:35:32 +02:00
Josema Camacho bae74b8181 fix(api): ignore RequestAborted from health-check probe disconnects in Sentry (#11632) 2026-06-17 16:20:17 +02:00
5ecfd6ea20 test(ui): add Okta provider form E2E test (#11600)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
2026-06-17 16:17:35 +02:00
Zeus AlmighteeandDaniel Barranquero e8ffe59ce2 feat(m365/entra): add entra_conditional_access_policy_no_deleted_object_references check (#11236)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-17 16:16:49 +02:00
StylusFrost e2ce41a492 feat(sdk): add Provider.get_class dynamic provider resolver (#11398) 2026-06-17 15:55:21 +02:00
Alejandro Bailo 6546d51a6c fix(ui): show alert permission errors (#11629) 2026-06-17 15:44:52 +02:00
73059ffc7e feat(azure): add databricks_workspace_public_network_access_disabled check (#11035)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: Hugo P.Brito <hugopbrito@Hugos-MacBook-Pro.local>
2026-06-17 14:12:18 +02:00
Josema Camacho f1a30f706a fix(api): raise Gunicorn worker timeout to 120s via GUNICORN_TIMEOUT (#11631) 2026-06-17 14:04:36 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G aa60dc3e17 ci(ui): add missing Google Workspace env vars to E2E workflow (#11599)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-06-17 13:18:39 +02:00
César Arroba 54518bd127 fix(ui): raise node heap to fix arm64 docker build OOM (#11628) 2026-06-17 13:02:20 +02:00
8d4ec561c2 feat(m365): add check for directory sync object takeover protection (#11098)
Co-authored-by: shadyfox <git@twink.energy>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: omobolaji adeyan <omobolaji.adeyan@gmail.com>
2026-06-17 12:15:14 +02:00
Rubén De la Torre Vico 0463cd1559 fix(api): disable ASGI lifespan probe and tune SSE worker loop/connections (#11626) 2026-06-17 11:16:58 +02:00
s1ns3nz0andDaniel Barranquero ca97d7d983 feat(azure): add cosmosdb_account_public_network_access_disabled check (#11034)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-17 11:05:09 +02:00
Pepe Fagoaga 7b8ce51263 chore(changelog): v5.30.2 (#11624) 2026-06-17 09:27:14 +02:00
Alejandro Bailo 262dfda0aa fix(ui): handle alert form errors (#11623) 2026-06-16 17:44:48 +02:00
s1ns3nz0andDaniel Barranquero 8bc42a5ded feat(azure): add cosmosdb_account_minimum_tls_version check (#11033)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-16 16:42:51 +02:00
406712ffa3 chore: scope prek pre-push hook to TruffleHog only (#11609)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:09:24 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G c2d7187a0b test(ui): add Vercel provider E2E tests (#11598)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-06-16 14:40:41 +02:00
lydiavilchez e690e5e86b fix(cli): prevent unrelated built-in provider failures from aborting the CLI (#11618) 2026-06-16 14:25:07 +02:00
Adrián Peña e4d5ca11b3 feat(api): add provider group filters (#11573) 2026-06-16 14:18:34 +02:00
Adrián Peña 181197177c feat(api): only remap SAML user roles when the IdP sends userType (#11520) 2026-06-16 14:18:16 +02:00
renovate[bot]renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>Daniel BarranqueroDaniel Barranquero
f21304c6a8 chore(sdk): update dependency pytest to v9 [security] (#11291)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
Co-authored-by: Daniel Barranquero <74871504+danibarranqueroo@users.noreply.github.com>
2026-06-16 14:17:55 +02:00
Rubén De la Torre VicoandDaniel Barranquero 0cf48a2c35 fix(gcp): surface organization-scan failures instead of silently scanning the home project (#11280)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-16 14:03:13 +02:00
s1ns3nz0andHugo P.Brito 6b4fb934f8 feat(azure): add aks_cluster_auto_upgrade_enabled check (#11027)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-16 13:56:40 +02:00
renovate[bot]renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>Daniel Barranquero
d1ed1eddef chore(sdk): update dependency black to v26 [security] (#11290)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-16 12:42:32 +02:00
César Arroba 0c9f4f6578 ci: run ui/mcp dependency vulnerability scans in prowler-cloud (match api-security) (#11617) 2026-06-16 12:25:57 +02:00
Rubén De la Torre Vico e1f20487ce chore(api): align uv constraints with SDK deps (numpy, py-iam-expand, iamdata; drop awsipranges) (#11594) 2026-06-16 12:00:18 +02:00
26b8c6b663 fix(ui): prevent radio button dot shift when checked (#11608)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 11:54:14 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> 3960827a9c chore(deps): bump starlette from 1.0.0 to 1.3.1 in /mcp_server (#11468)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 11:09:11 +02:00
Pedro Martín e419771b04 perf(api): optimize scan-compliance-overviews task (#11591) 2026-06-16 10:48:55 +02:00
César Arroba 94ce76d679 ci: authenticate GitHub API curl in setup-python-uv action (#11610) 2026-06-16 10:31:58 +02:00
Rubén De la Torre Vico 28c064a9b7 feat(api): add Server-Sent Events (SSE) infrastructure (#11556) 2026-06-16 10:26:20 +02:00
dependabot[bot]anddependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> eeb02453d1 chore(deps): bump pyjwt from 2.12.1 to 2.13.0 in /mcp_server (#11606)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-16 10:14:21 +02:00
Aline AlmeidaandHugo P.Brito cb4b889b20 fix(gcp): credit audit-filtered aggregated sinks in metric-filter checks (#11575)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-06-16 10:11:16 +02:00
Pepe Fagoaga f1e42d1681 chore(api-beat): absolute entrypoint (#11604) 2026-06-16 09:44:18 +02:00
Pepe Fagoaga ca7ce5a8c3 feat(jira): request timeout (#11602) 2026-06-16 09:36:22 +02:00
Pepe Fagoaga 810d8d7686 chore(codepipeline): verify if repo is public with TLS (#11603) 2026-06-16 09:35:11 +02:00
Alejandro Bailo dd1895d2c4 test(ui): remove onboarding e2e suite (#11605) 2026-06-16 09:32:37 +02:00
s1ns3nz0andDaniel Barranquero b5bb85c956 feat(azure): add cosmosdb_account_backup_policy_continuous check (#11032)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-15 19:20:38 +02:00
Davidm4randClaude Opus 4.8 36fe48dbc5 fix(api): patch dependency and container CVEs (#11596)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 18:24:55 +02:00
Alejandro Bailo e5bbffd47c fix(ui): exclude onboarding e2e from oss (#11597) 2026-06-15 17:19:40 +02:00
Daniel Barranquero 566167489b fix(sdk): patch container CVEs and suppress unfixable bookworm criticals (#11592) 2026-06-15 16:59:44 +02:00
renovate[bot]renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>Pepe Fagoaga
3cb360e9ae chore(docker): pin dependencies (#11292)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-15 15:31:24 +02:00
Alejandro Bailo 24e3182329 fix(ui): remove onboarding changelog entry (#11593) 2026-06-15 15:22:47 +02:00
49309b43d3 feat(ui): UI onboarding system (#11430)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
Co-authored-by: Alejandro Bailo <59607668+alejandrobailo@users.noreply.github.com>
2026-06-15 13:53:48 +02:00
Alejandro BailoandPepe Fagoaga 6db8ce672c fix(ui): patch vulnerable dependencies (#11581)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-15 11:50:08 +02:00
Pepe Fagoaga 9465b82747 docs(sdk): reflect Python 3.13 support (#11585) 2026-06-15 11:27:09 +02:00
César Arroba 383d2b218f chore: configure vulture to ignore known false positives (#11583) 2026-06-15 11:15:22 +02:00
Branch VincentandPepe Fagoaga dccd674cf9 chore(sdk): support Python 3.13 (#9293)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-15 10:59:51 +02:00
César Arroba a679865cce ci: always run container and dependency vulnerability scans on PRs (#11582) 2026-06-15 10:38:28 +02:00
César Arroba 15bfa39b23 ci: fail PR checks on critical container image and dependency vulnerabilities (#11580) 2026-06-15 09:57:23 +02:00
Prowler Botandprowler-bot dc3433aaf0 feat(aws): Update regions for AWS services (#11570)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-12 14:15:02 +02:00
Pedro Martín 25fc285966 chore(banner): update info (#11568) 2026-06-12 13:45:34 +02:00
s1ns3nz0andDaniel Barranquero 9022a3a138 feat(azure): add cosmosdb_account_automatic_failover_enabled check (#11031)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-12 13:18:08 +02:00
Josema Camacho ca443b8ff1 chore: prepare API and UI changelogs for 5.30.1 release (#11562) 2026-06-12 12:07:31 +02:00
79e066d3f5 feat(gcp): add cloudsql_instance_high_availability_enabled check (#11024)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
Co-authored-by: lydiavilchez <114735608+lydiavilchez@users.noreply.github.com>
2026-06-12 11:51:13 +02:00
Hugo Pereira Brito 56831a7392 feat(oci): add storage admin delete exclusion check (#11523) 2026-06-12 11:10:46 +02:00
Alejandro Bailo 2e82f1564f fix(ui): show threat map data for okta and google workspace accounts (#11542) 2026-06-12 10:07:56 +02:00
Josema Camacho a394c0fdf6 fix(api): drop_subgraph deletes relationships then nodes to cut Neo4j memory (#11557) 2026-06-11 18:32:35 +02:00
Pedro Martín 20eca78767 fix(compliance): resolve provider from scan in attributes endp (#11546) 2026-06-11 18:00:36 +02:00
Oleksandr_SaninandDaniel Barranquero bba594a1db feat(aws/sagemaker): add sagemaker_clarify_exists check (#11211)
Signed-off-by: Oleksandr Sanin <alexaaander.sanin@gmail.com>
Signed-off-by: Oleksandr Yizchak Sanin <alexaaander.sanin@gmail.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-06-11 17:40:41 +02:00
Hugo Pereira Brito 65f00a197b fix(api): normalize OCI scan region credentials (#11558) 2026-06-11 17:32:28 +02:00
Zeus AlmighteeandLydia Vilchez ce27053c2d feat(aws): add securityhub + config org-wide delegated admin checks (#11259)
Co-authored-by: Lydia Vilchez <lydiavilchezlopez@gmail.com>
2026-06-11 16:53:28 +02:00
Pedro Martín 610febb5d5 fix(api): bump prowler SDK lock to v5.30.0 for okta_idaas_stig (#11553) 2026-06-11 15:53:44 +02:00
Prowler Botandprowler-bot c4378d5992 chore(release): Bump versions to v5.31.0 (#11548)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-11 15:28:25 +02:00
2553 changed files with 177910 additions and 24843 deletions
+18 -5
View File
@@ -6,14 +6,20 @@
PROWLER_UI_VERSION="stable"
AUTH_URL=http://localhost:3000
API_BASE_URL=http://prowler-api:8080/api/v1
# deprecated, use UI_API_BASE_URL
NEXT_PUBLIC_API_BASE_URL=${API_BASE_URL}
UI_API_BASE_URL=${API_BASE_URL}
# deprecated, use UI_API_DOCS_URL
NEXT_PUBLIC_API_DOCS_URL=http://prowler-api:8080/api/v1/docs
UI_API_DOCS_URL=http://prowler-api:8080/api/v1/docs
AUTH_TRUST_HOST=true
UI_PORT=3000
# openssl rand -base64 32
AUTH_SECRET="N/c6mnaS5+SWq81+819OrzQZlmx1Vxtp/orjttJSmw8="
# Google Tag Manager ID
# Google Tag Manager ID (empty/unset ⇒ GTM not loaded, zero egress)
# deprecated, use UI_GOOGLE_TAG_MANAGER_ID
NEXT_PUBLIC_GOOGLE_TAG_MANAGER_ID=""
UI_GOOGLE_TAG_MANAGER_ID=""
#### MCP Server ####
PROWLER_MCP_VERSION=stable
@@ -139,13 +145,20 @@ DJANGO_BROKER_VISIBILITY_TIMEOUT=86400
DJANGO_SENTRY_DSN=
DJANGO_THROTTLE_TOKEN_OBTAIN=50/minute
# Sentry settings
SENTRY_ENVIRONMENT=local
# Sentry for the web app (server + browser). The UI_SENTRY_* values load only
# when UI_SENTRY_ENABLED="true"; without it they are ignored (default off, zero
# egress). The deprecated NEXT_PUBLIC_SENTRY_DSN still activates Sentry without
# the flag. SENTRY_RELEASE (unprefixed) feeds the web app's server/edge SDKs.
UI_SENTRY_DSN=
UI_SENTRY_ENVIRONMENT=local
SENTRY_RELEASE=local
NEXT_PUBLIC_SENTRY_ENVIRONMENT=${SENTRY_ENVIRONMENT}
# Reserved runtime public config (registered now; no UI consumer yet)
# UI_POSTHOG_KEY=
# UI_POSTHOG_HOST=
# REO_DEV_CLIENT_ID=
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.30.0
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.35.0
# Social login credentials
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
-1
View File
@@ -1,7 +1,6 @@
# SDK
/* @prowler-cloud/detection-remediation
/prowler/ @prowler-cloud/detection-remediation
/prowler/compliance/ @prowler-cloud/compliance
/tests/ @prowler-cloud/detection-remediation
/dashboard/ @prowler-cloud/detection-remediation
/docs/ @prowler-cloud/detection-remediation
+3 -3
View File
@@ -1,5 +1,5 @@
name: 'OSV-Scanner'
description: 'Install osv-scanner and scan a lockfile, failing on HIGH/CRITICAL/UNKNOWN severity findings. Posts/updates a PR comment with findings on pull_request events (requires pull-requests: write).'
description: 'Install osv-scanner and scan a lockfile, failing on CRITICAL severity findings. Posts/updates a PR comment with findings on pull_request events (requires pull-requests: write).'
author: 'Prowler'
inputs:
@@ -7,9 +7,9 @@ inputs:
description: 'Path to the lockfile to scan, relative to the repository root (e.g. uv.lock, api/uv.lock, ui/pnpm-lock.yaml).'
required: true
severity-levels:
description: 'Comma-separated severity levels that fail the scan. Default: HIGH,CRITICAL,UNKNOWN.'
description: 'Comma-separated severity levels that fail the scan. Default: CRITICAL.'
required: false
default: 'HIGH,CRITICAL,UNKNOWN'
default: 'CRITICAL'
version:
description: 'osv-scanner release tag to install. When overriding, you MUST also override binary-sha256.'
required: false
+20 -2
View File
@@ -43,8 +43,17 @@ runs:
if: github.repository_owner == 'prowler-cloud' && github.repository != 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
LATEST_COMMIT=$(curl -sf --retry 3 --retry-all-errors --retry-delay 2 --retry-max-time 60 \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/repos/prowler-cloud/prowler/commits/master" \
| jq -er '.sha') || {
echo "::error::Failed to fetch latest prowler/master commit from the GitHub API (HTTP error or missing .sha). Check the GITHUB_TOKEN and API rate limits."
exit 1
}
echo "Latest commit hash: $LATEST_COMMIT"
sed -i "s|\(git = \"https://github\.com/prowler-cloud/prowler\.git?rev=master\)#[a-f0-9]\{40\}\"|\1#${LATEST_COMMIT}\"|g" uv.lock
echo "Updated uv.lock entry:"
@@ -54,8 +63,17 @@ runs:
if: github.event_name == 'push' && github.ref == 'refs/heads/master' && github.repository == 'prowler-cloud/prowler'
shell: bash
working-directory: ${{ inputs.working-directory }}
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
LATEST_COMMIT=$(curl -s "https://api.github.com/repos/prowler-cloud/prowler/commits/master" | jq -r '.sha')
LATEST_COMMIT=$(curl -sf --retry 3 --retry-all-errors --retry-delay 2 --retry-max-time 60 \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/repos/prowler-cloud/prowler/commits/master" \
| jq -er '.sha') || {
echo "::error::Failed to fetch latest prowler/master commit from the GitHub API (HTTP error or missing .sha). Check the GITHUB_TOKEN and API rate limits."
exit 1
}
echo "Latest commit hash: $LATEST_COMMIT"
sed -i "s|\(git = \"https://github\.com/prowler-cloud/prowler\.git?rev=master\)#[a-f0-9]\{40\}\"|\1#${LATEST_COMMIT}\"|g" uv.lock
echo "Updated uv.lock entry:"
+2 -2
View File
@@ -63,7 +63,7 @@ runs:
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
version: 'v0.69.2'
version: 'v0.71.2'
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
@@ -76,7 +76,7 @@ runs:
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
version: 'v0.69.2'
version: 'v0.71.2'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
+10
View File
@@ -5,10 +5,20 @@
"version": "v8",
"sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd"
},
"github/gh-aw-actions/setup@v0.81.6": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.81.6",
"sha": "ba6380cc6e5be5d21677bebe04d52fb48e3abec7"
},
"github/gh-aw/actions/setup@v0.43.23": {
"repo": "github/gh-aw/actions/setup",
"version": "v0.43.23",
"sha": "9382be3ca9ac18917e111a99d4e6bbff58d0dccc"
},
"step-security/harden-runner@v2.20.0": {
"repo": "step-security/harden-runner",
"version": "v2.20.0",
"sha": "bf7454d06d71f1098171f2acdf0cd4708d7b5920"
}
}
}
-138
View File
@@ -1,138 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# v5
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "pip"
# directory: "/api"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# - "component/api"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 25
target-branch: master
labels:
- "dependencies"
- "github_actions"
cooldown:
default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "npm"
# directory: "/ui"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "npm"
# - "component/ui"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 25
target-branch: master
labels:
- "dependencies"
- "docker"
cooldown:
default-days: 7
# - package-ecosystem: "pre-commit"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pre-commit"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/04/15
# v4.6
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "pip"
# - "v4"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "github_actions"
# - "v4"
# - package-ecosystem: "docker"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "docker"
# - "v4"
# Dependabot Updates are temporary disabled - 2025/03/19
# v3
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "pip"
# - "v3"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "github_actions"
# - "v3"
+5
View File
@@ -77,6 +77,11 @@ provider/okta:
- any-glob-to-any-file: "prowler/providers/okta/**"
- any-glob-to-any-file: "tests/providers/okta/**"
provider/linode:
- changed-files:
- any-glob-to-any-file: "prowler/providers/linode/**"
- any-glob-to-any-file: "tests/providers/linode/**"
github_actions:
- changed-files:
- any-glob-to-any-file: ".github/workflows/*"
+11 -6
View File
@@ -18,8 +18,9 @@ Please add a detailed description of how to review this PR.
<summary><b>Community Checklist</b></summary>
- [ ] This feature/issue is listed in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the issue/feature in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](goto.prowler.com/slack)
- [ ] This feature/issue is listed in the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](https://goto.prowler.com/slack)
- [ ] I have reviewed the [open pull requests](https://github.com/prowler-cloud/prowler/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen) and confirmed there is no existing PR that implements the same outcome
</details>
@@ -27,8 +28,8 @@ Please add a detailed description of how to review this PR.
- [ ] Review if the code is being covered by tests.
- [ ] Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
- [ ] Review if backport is needed.
- [ ] Review if is needed to change the [Readme.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/prowler/CHANGELOG.md), if applicable.
- [ ] Review if is needed to change the [README.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure a changelog fragment is added under [prowler/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/prowler/changelog.d), if applicable.
#### SDK/CLI
- Are there new checks included in this PR? Yes / No
@@ -40,7 +41,7 @@ Please add a detailed description of how to review this PR.
- [ ] Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/ui/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [ui/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/ui/changelog.d), if applicable.
#### API
- [ ] All issue/task requirements work as expected on the API
@@ -50,7 +51,11 @@ Please add a detailed description of how to review this PR.
- [ ] Any other relevant evidence of the implementation (if applicable)
- [ ] Verify if API specs need to be regenerated.
- [ ] Check if version updates are required (e.g., specs, uv, etc.).
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/api/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [api/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/api/changelog.d), if applicable.
#### MCP Server
- [ ] All issue/task requirements work as expected on the MCP Server
- [ ] Ensure a changelog fragment is added under [mcp_server/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/mcp_server/changelog.d), if applicable.
### License
+13 -2
View File
@@ -23,6 +23,10 @@
"prConcurrentLimit": 20,
"prHourlyLimit": 10,
"vulnerabilityAlerts": {
"labels": [
"dependencies",
"security"
],
"prHourlyLimit": 0,
"prConcurrentLimit": 0
},
@@ -38,7 +42,7 @@
"schedule": [
"* 22-23,0-5 1 * *"
],
"enabled": false
"enabled": true
},
{
"description": "Minors: 8th of every 3 months, Madrid overnight window (22:00-06:00)",
@@ -48,7 +52,7 @@
"schedule": [
"* 22-23,0-5 8 */3 *"
],
"enabled": false
"enabled": true
},
{
"description": "Majors: 15th of every 3 months, Madrid overnight window",
@@ -58,6 +62,13 @@
"schedule": [
"* 22-23,0-5 15 */3 *"
],
"enabled": true
},
{
"description": "gh-aw compiled lock files - generated by 'gh aw compile', action pins must match the compiler version, never bump directly",
"matchFileNames": [
".github/workflows/*.lock.yml"
],
"enabled": false
},
{
+154
View File
@@ -0,0 +1,154 @@
#!/usr/bin/env python3
"""Rename changelog fragments to their PR number before running towncrier.
For every <slug>.<type>.md in <component_dir>/changelog.d/, find the commit that
added it, resolve its PR via the GitHub API (falling back to the squash-commit
subject), and `git mv` it to <PR>.<type>.md so towncrier renders the PR link.
Unresolvable fragments become +<slug>.<type>.md orphans (rendered without link).
"""
import argparse
import json
import os
import re
import subprocess
import sys
import urllib.error
import urllib.request
FRAGMENT_RE = re.compile(
r"^(?P<slug>[A-Za-z0-9][A-Za-z0-9._-]*?)"
r"\.(?P<type>added|changed|deprecated|removed|fixed|security)"
r"(?:\.(?P<counter>[0-9]+))?\.md$"
)
SUBJECT_PR_RE = re.compile(r" \(#([0-9]+)\)$")
IGNORED_FILES = {".gitkeep", "README.md"}
API_TIMEOUT_SECONDS = 10
def git(*args: str) -> str:
result = subprocess.run(["git", *args], check=True, capture_output=True, text=True)
return result.stdout.strip()
def find_adding_commit(path: str) -> str | None:
"""Find the commit that added a file, following renames.
Falls back to a plain (no --follow) lookup: rename detection can lose the
add event for degenerate content (e.g. files identical to many others).
"""
sha = git("log", "--follow", "--diff-filter=A", "--format=%H", "-1", "--", path)
if not sha:
sha = git("log", "--diff-filter=A", "--format=%H", "-1", "--", path)
return sha or None
def pr_from_api(repo: str, sha: str) -> int | None:
"""Resolve the PR associated with a commit via the GitHub API.
Returns None on any network/API failure so the caller can fall back to
parsing the squash-commit subject.
"""
url = f"https://api.github.com/repos/{repo}/commits/{sha}/pulls"
headers = {
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "prowler-changelog-attribution",
}
token = os.environ.get("GITHUB_TOKEN")
if token:
headers["Authorization"] = f"Bearer {token}"
request = urllib.request.Request(url, headers=headers)
try:
with urllib.request.urlopen(request, timeout=API_TIMEOUT_SECONDS) as response:
pulls = json.load(response)
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError):
return None
if isinstance(pulls, list) and pulls:
return pulls[0].get("number")
return None
def pr_from_subject(sha: str) -> int | None:
subject = git("log", "-1", "--format=%s", sha)
match = SUBJECT_PR_RE.search(subject)
return int(match.group(1)) if match else None
def unique_destination(directory: str, base_name: str, fragment_type: str) -> str:
"""Return a non-colliding fragment path, appending a numeric counter if needed."""
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.md")
counter = 0
while os.path.exists(candidate):
counter += 1
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.{counter}.md")
return candidate
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("component_dir", help="Component directory, e.g. prowler")
parser.add_argument("--repo", default="prowler-cloud/prowler")
parser.add_argument(
"--no-api",
action="store_true",
help="Skip the GitHub API and resolve PRs from commit subjects only",
)
args = parser.parse_args()
fragments_dir = os.path.join(args.component_dir, "changelog.d")
if not os.path.isdir(fragments_dir):
print(f"::error::Fragments directory not found: {fragments_dir}")
return 1
malformed = []
to_process = []
for name in sorted(os.listdir(fragments_dir)):
if name in IGNORED_FILES or name.startswith("+"):
continue
match = FRAGMENT_RE.match(name)
if not match:
malformed.append(name)
continue
if match.group("slug").isdigit():
continue
to_process.append((name, match))
if malformed:
for name in malformed:
print(
f"::error::Malformed fragment filename in {fragments_dir}: {name} "
"(expected <slug>.<type>.md with type one of added|changed|"
"deprecated|removed|fixed|security)"
)
return 1
for name, match in to_process:
slug, fragment_type = match.group("slug"), match.group("type")
path = os.path.join(fragments_dir, name)
sha = find_adding_commit(path)
pr_number = None
if sha:
if not args.no_api:
pr_number = pr_from_api(args.repo, sha)
if pr_number is None:
pr_number = pr_from_subject(sha)
if pr_number is not None:
destination = unique_destination(
fragments_dir, str(pr_number), fragment_type
)
else:
destination = unique_destination(fragments_dir, f"+{slug}", fragment_type)
print(
f"::warning::Could not resolve a PR for {path}; renamed to "
f"{os.path.basename(destination)} (entry will render without a PR link)"
)
git("mv", path, destination)
print(f"{path} -> {destination}")
return 0
if __name__ == "__main__":
sys.exit(main())
+2 -3
View File
@@ -6,8 +6,7 @@
# - .github/workflows/api-security.yml, sdk-security.yml, ui-security.yml
#
# Severity levels (comma-separated) are read from OSV_SEVERITY_LEVELS.
# Default: HIGH,CRITICAL,UNKNOWN — preserves prior .safety-policy.yml policy
# (ignore-cvss-severity-below: 7 + ignore-cvss-unknown-severity: False).
# Default: CRITICAL — only CVSS >= 9.0 findings fail the scan.
# osv-scanner has no native CVSS threshold (google/osv-scanner#1400, closed
# not-planned). Severity is derived from $group.max_severity (numeric CVSS
# score string) which osv-scanner emits per group.
@@ -33,7 +32,7 @@ set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
CONFIG="${ROOT}/osv-scanner.toml"
SEVERITY_LEVELS="${OSV_SEVERITY_LEVELS:-HIGH,CRITICAL,UNKNOWN}"
SEVERITY_LEVELS="${OSV_SEVERITY_LEVELS:-CRITICAL}"
for bin in osv-scanner jq; do
if ! command -v "${bin}" >/dev/null 2>&1; then
+13
View File
@@ -0,0 +1,13 @@
{% set category_order = definitions.keys() %}
{% for section, _ in sections.items() %}
{% for category in category_order if category in sections[section] %}
### {{ definitions[category]['name'] }}
{% for text, values in sections[section][category].items() -%}
- {{ text }}{% if values %} {{ values|join(', ') }}{% endif %}{{ "\n" }}
{%- endfor %}
{% endfor %}
{% endfor %}
---
{{ "\n" }}
+2 -1
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -62,6 +62,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+3 -1
View File
@@ -9,6 +9,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
pull_request:
branches:
- 'master'
@@ -17,6 +18,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
schedule:
- cron: '00 12 * * *'
@@ -44,7 +46,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -30
View File
@@ -46,7 +46,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
@@ -65,7 +65,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -108,7 +108,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -175,7 +175,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -215,7 +215,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -236,7 +236,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -272,27 +272,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger API deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: api-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+6 -6
View File
@@ -12,9 +12,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'api/**'
- '.github/workflows/api-container-checks.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -36,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -72,7 +69,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -95,6 +92,8 @@ jobs:
_http._tcp.deb.debian.org:443
powershellinfraartifacts-gkhedzdeaghdezhr.z01.azurefd.net:443
get.trivy.dev:443
raw.githubusercontent.com:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -111,6 +110,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Set up Docker Buildx
@@ -134,5 +134,5 @@ jobs:
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
fail-on-critical: 'true'
severity: 'CRITICAL'
+2 -8
View File
@@ -16,13 +16,6 @@ on:
branches:
- "master"
- "v5.*"
paths:
- 'api/**'
- '.github/workflows/api-tests.yml'
- '.github/workflows/api-security.yml'
- '.github/actions/setup-python-uv/**'
- '.github/actions/osv-scanner/**'
- '.github/scripts/osv-scan.sh'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -50,7 +43,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -84,6 +77,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+4 -3
View File
@@ -48,7 +48,7 @@ jobs:
services:
postgres:
image: postgres:17@sha256:2cd82735a36356842d5eb1ef80db3ae8f1154172f0f653db48fde079b2a0b7f7
image: postgres:17@sha256:5c855ad7b85e68e48a62f34662853f38b57c1c1d80f3a927ab58034fd6d31c5e
env:
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
@@ -63,7 +63,7 @@ jobs:
--health-timeout 5s
--health-retries 5
valkey:
image: valkey/valkey:7-alpine3.19
image: valkey/valkey:7-alpine3.19@sha256:4054fe7fc607b9326ac7c4691ed26e9670d2ff17a9fb28c2577adecf928acbcc
env:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
@@ -78,7 +78,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -111,6 +111,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+1 -1
View File
@@ -30,7 +30,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+4 -4
View File
@@ -29,7 +29,7 @@ jobs:
patch_version: ${{ steps.detect.outputs.patch_version }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -75,7 +75,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -202,7 +202,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -307,7 +307,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -0,0 +1,35 @@
name: 'Tools: Check Test Init Files'
on:
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: {}
jobs:
check-test-init-files:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@fa2e9d605c4eeb9fcad4c99c224cee0c6c7f3594 # v2.16.0
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Check for __init__.py files in test directories
run: python3 scripts/check_test_init_files.py .
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+527
View File
@@ -0,0 +1,527 @@
name: 'Tools: Compile Changelogs'
run-name: 'Compile changelogs for Prowler ${{ inputs.prowler_version }}'
on:
workflow_dispatch:
inputs:
prowler_version:
description: 'Prowler version being released (e.g., 5.31.0)'
required: true
type: string
target_branch:
description: 'Branch to compile on (master for minor releases, v5.X for patches)'
required: true
type: string
sdk_version:
description: 'SDK version override (empty = mirrors prowler_version; "skip" = hold this component back)'
required: false
type: string
api_version:
description: 'API version override (empty = auto-derive 1.<prowler_minor + 1>.<prowler_patch>; "skip" = hold back)'
required: false
type: string
ui_version:
description: 'UI version override (empty = auto-derive 1.<prowler_minor>.<prowler_patch>; "skip" = hold back)'
required: false
type: string
mcp_version:
description: 'MCP Server version override (empty = auto-derive from pending fragment types; "skip" = hold back)'
required: false
type: string
concurrency:
group: ${{ github.workflow }}-${{ inputs.prowler_version }}
cancel-in-progress: false
env:
PROWLER_VERSION: ${{ inputs.prowler_version }}
TARGET_BRANCH: ${{ inputs.target_branch }}
SDK_VERSION: ${{ inputs.sdk_version }}
API_VERSION: ${{ inputs.api_version }}
UI_VERSION: ${{ inputs.ui_version }}
MCP_VERSION: ${{ inputs.mcp_version }}
permissions: {}
jobs:
compile-changelogs:
if: github.event_name == 'workflow_dispatch' && github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
pull-requests: write
steps:
- name: Harden the runner (Block outbound calls)
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.target_branch }}
fetch-depth: 0 # PR attribution resolves each fragment's adding commit from history
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Install towncrier
run: pip install --no-cache-dir towncrier==25.8.0
- name: Configure Git
run: |
git config --global user.name 'prowler-bot'
git config --global user.email '179230569+prowler-bot@users.noreply.github.com'
- name: Validate version inputs
run: |
if [[ ! "$PROWLER_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid prowler_version syntax: '$PROWLER_VERSION' (must be N.N.N)"
exit 1
fi
if [ "$TARGET_BRANCH" != "master" ] && [[ ! "$TARGET_BRANCH" =~ ^v[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid target_branch syntax: '$TARGET_BRANCH' (must be 'master' or vN.N, e.g. v5.31)"
exit 1
fi
IFS=. read -r prowler_major prowler_minor prowler_patch <<< "$PROWLER_VERSION"
prowler_major=$((10#$prowler_major))
prowler_minor=$((10#$prowler_minor))
prowler_patch=$((10#$prowler_patch))
if [ "$prowler_patch" -eq 0 ]; then
if [ "$TARGET_BRANCH" != "master" ]; then
echo "::error::target_branch must be 'master' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
else
expected_target_branch="v${prowler_major}.${prowler_minor}"
if [ "$TARGET_BRANCH" != "$expected_target_branch" ]; then
echo "::error::target_branch must be '${expected_target_branch}' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
fi
for pair in "sdk_version:$SDK_VERSION" "api_version:$API_VERSION" "ui_version:$UI_VERSION" "mcp_version:$MCP_VERSION"; do
input_name="${pair%%:*}"
input_value="${pair#*:}"
if [ -n "$input_value" ] && [ "$input_value" != "skip" ] && [[ ! "$input_value" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid $input_name syntax: '$input_value' (must be N.N.N, empty for auto-derivation, or 'skip')"
exit 1
fi
done
- name: Compile changelogs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
component_input() {
case "$1" in
prowler) echo "$SDK_VERSION" ;;
api) echo "$API_VERSION" ;;
ui) echo "$UI_VERSION" ;;
mcp_server) echo "$MCP_VERSION" ;;
esac
}
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
pending_fragments() {
find "$1/changelog.d" -maxdepth 1 -type f ! -name '.gitkeep' ! -name 'README.md' | sort
}
# The component's last released version is the first stamped heading
# of its CHANGELOG.md, the same source prepare-release.yml greps.
latest_released_version() {
grep -m1 -E '^## \[v?[0-9]+\.[0-9]+\.[0-9]+\]' "$1/CHANGELOG.md" | sed -E 's/^## \[v?([0-9]+\.[0-9]+\.[0-9]+)\].*/\1/'
}
has_removed_fragments() {
echo "$1" | grep -qE '\.removed(\.[0-9]+)?\.md$'
}
# Resolve every component's effective version before compiling
# anything, so a wrong input cannot leave the tree half-compiled.
# Empty input = auto-derive (latest released version + semver bump
# from the pending fragment types). 'skip' = hold the component back.
errors=0
compiling=""
for component in prowler api ui mcp_server; do
input=$(component_input "$component")
fragments=$(pending_fragments "$component")
if [ "$input" = "skip" ]; then
if [ -n "$fragments" ]; then
echo "::warning::${component}: held back by request; these pending fragments stay for a future release:"
echo "$fragments"
fi
continue
fi
if [ -n "$input" ] && [ -z "$fragments" ]; then
echo "::error::${component}: version input '$input' provided but ${component}/changelog.d/ has no pending fragments (wrong input?)"
errors=1
continue
fi
if [ -z "$fragments" ]; then
continue
fi
removed_fragments=false
if has_removed_fragments "$fragments"; then
removed_fragments=true
fi
current=$(latest_released_version "$component")
if [ -z "$current" ]; then
echo "::error::${component}: could not read the latest released version from ${component}/CHANGELOG.md; restore the released heading before compiling"
errors=1
continue
fi
if [ -n "$input" ]; then
effective="$input"
mode="explicit"
current_key=$(version_key "$current")
effective_key=$(version_key "$effective")
if [[ "$effective_key" < "$current_key" || "$effective_key" == "$current_key" ]]; then
echo "::error::${component}: explicit version '${effective}' must be greater than the latest released version (${current})"
errors=1
continue
fi
else
if [ "$removed_fragments" = "true" ]; then
echo "::error::${component}: pending 'removed' fragments imply a major bump (breaking change); provide its version input explicitly"
errors=1
continue
fi
# SDK, UI, and API versions are deterministic mirrors of the
# Prowler version (the scheme bump-version.yml codifies): the SDK
# mirrors it directly, the UI tracks 1.<minor>.<patch>, and the
# API is the independent 1.<minor + 1>.<patch> stream. Only the
# MCP Server has its own cadence, derived from fragment types.
IFS=. read -r _ prowler_minor prowler_patch <<< "$PROWLER_VERSION"
prowler_minor=$((10#$prowler_minor))
prowler_patch=$((10#$prowler_patch))
case "$component" in
prowler) effective="$PROWLER_VERSION" ;;
ui) effective="1.${prowler_minor}.${prowler_patch}" ;;
api) effective="1.$((prowler_minor + 1)).${prowler_patch}" ;;
mcp_server)
IFS=. read -r major minor patch <<< "$current"
major=$((10#$major))
minor=$((10#$minor))
patch=$((10#$patch))
# Prowler patch releases (vN.N target) are maintenance
# releases, so the MCP Server bumps patch regardless of
# fragment types; a deliberate exception needs the explicit
# version input.
if [ "$TARGET_BRANCH" != "master" ]; then
effective="${major}.${minor}.$((patch + 1))"
if echo "$fragments" | grep -qE '\.(added|deprecated)(\.[0-9]+)?\.md$'; then
echo "::warning::${component}: 'added'/'deprecated' fragments are shipping in a Prowler patch; auto-derived a patch bump (${current} -> ${effective}), pass the version input to override"
fi
elif echo "$fragments" | grep -qE '\.(added|changed|deprecated)(\.[0-9]+)?\.md$'; then
effective="${major}.$((minor + 1)).0"
else
effective="${major}.${minor}.$((patch + 1))"
fi
;;
esac
current_key=$(version_key "$current")
effective_key=$(version_key "$effective")
if [[ "$effective_key" < "$current_key" || "$effective_key" == "$current_key" ]]; then
echo "::error::${component}: auto-derived version '${effective}' is not greater than the latest released version (${current}); check prowler_version or pass the version input explicitly"
errors=1
continue
fi
mode="auto"
echo "::notice::${component}: version auto-derived ${current} -> ${effective}"
fi
if [ "$removed_fragments" = "true" ]; then
IFS=. read -r current_major _ <<< "$current"
current_major=$((10#$current_major))
IFS=. read -r effective_major effective_minor effective_patch <<< "$effective"
effective_major=$((10#$effective_major))
effective_minor=$((10#$effective_minor))
effective_patch=$((10#$effective_patch))
if [ "$effective_major" -le "$current_major" ] || [ "$effective_minor" -ne 0 ] || [ "$effective_patch" -ne 0 ]; then
echo "::error::${component}: removed fragments require a major component release (${current} -> X.0.0 with X > ${current_major}); got ${effective}"
errors=1
continue
fi
fi
# Without the marker the build would insert the new block above the
# file header instead of below it.
if ! grep -q '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md is missing the '<!-- changelog: release notes start -->' marker; restore it after the intro line before compiling"
errors=1
continue
fi
# A hand-written UNRELEASED block means someone followed the old
# convention; its entries would be left out of the compiled block
# and out of the release notes extraction.
if grep -q '(Prowler UNRELEASED)' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md contains a hand-written '(Prowler UNRELEASED)' block; convert its entries to fragments in ${component}/changelog.d/ and delete the block before compiling"
errors=1
continue
fi
echo "${effective} ${mode}" > "${RUNNER_TEMP}/version-${component}.txt"
compiling="${compiling}${component} "
done
if [ "$errors" -ne 0 ]; then
exit 1
fi
if [ -z "$compiling" ]; then
echo "::error::Nothing to compile: no component has pending fragments to release"
exit 1
fi
body_file="${RUNNER_TEMP}/compile-changelogs-pr-body.md"
{
echo "### Description"
echo ""
echo "Compiles the pending changelog fragments into the per-component \`CHANGELOG.md\` files for Prowler v${PROWLER_VERSION}, replacing the manual stamping PR."
echo ""
echo "| Component | Version | Fragments consumed |"
echo "|---|---|---|"
} > "$body_file"
compiled_components=""
for component in prowler api ui mcp_server; do
if [ ! -f "${RUNNER_TEMP}/version-${component}.txt" ]; then
echo "Skipping ${component} (no pending fragments or held back)"
echo "| \`${component}\` | - | 0 |" >> "$body_file"
continue
fi
read -r version mode < "${RUNNER_TEMP}/version-${component}.txt"
version_label="$version"
if [ "$mode" = "auto" ]; then
version_label="${version} (auto)"
fi
count=$(pending_fragments "$component" | wc -l | tr -d ' ')
echo "Compiling ${component} ${version} (${count} fragments, ${mode} version)..."
# Captured before attribution renames them: these original paths are
# what the forward-sync deletes on master (backports copy fragments
# verbatim, so filenames match across branches).
pending_fragments "$component" > "${RUNNER_TEMP}/consumed-${component}.txt"
pre_lines=$(wc -l < "$component/CHANGELOG.md")
# Attribution must run before the build: towncrier renders the
# first dotted segment of each filename as the PR number.
python .github/scripts/changelog_attribution.py "$component"
towncrier build --config "$component/towncrier.toml" --version "$version" --name "Prowler v${PROWLER_VERSION}" --yes
# The build only inserts lines right after the marker, so the new
# stamped block is exactly the added lines following it. Captured
# for the forward-sync to master.
post_lines=$(wc -l < "$component/CHANGELOG.md")
delta=$((post_lines - pre_lines))
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md" | cut -d: -f1)
sed -n "$((marker_line + 1)),$((marker_line + delta))p" "$component/CHANGELOG.md" > "${RUNNER_TEMP}/block-${component}.md"
compiled_components="${compiled_components}${component} "
echo "| \`${component}\` | ${version_label} | ${count} |" >> "$body_file"
done
echo "COMPILED_COMPONENTS=${compiled_components}" >> "$GITHUB_ENV"
{
echo ""
echo "Review that no pending fragment was dropped (the diff must delete every consumed fragment) and that each new version block is correct, then squash-merge."
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$body_file"
echo "PR_BODY_FILE=${body_file}" >> "$GITHUB_ENV"
- name: Create compile PR
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
branch: compile-changelogs-${{ env.PROWLER_VERSION }}
base: ${{ env.TARGET_BRANCH }}
title: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
body-path: ${{ env.PR_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
skip-sync
# Patch compiles (target_branch = v5.X) leave master holding the consumed
# fragments and missing the new version block. This applies the equivalent
# change to master: insert the same stamped blocks under the marker and
# delete the consumed fragments, so the next minor compile cannot
# re-release entries that already shipped in the patch.
- name: Apply forward-sync to master
if: env.TARGET_BRANCH != 'master'
run: |
set -euo pipefail
git checkout -B master origin/master
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
release_from_heading() {
local heading="$1"
echo "$heading" | sed -E 's/^## \[[^]]+\] \(Prowler v?([0-9]+\.[0-9]+\.[0-9]+)\).*/\1/'
}
insert_changelog_block_ordered() {
local component="$1"
local block_file="$2"
local changelog="${component}/CHANGELOG.md"
local incoming_heading incoming_release incoming_key
local marker_line insertion_line duplicate_line total_lines
local line heading existing_release existing_key
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$changelog" | cut -d: -f1)
incoming_heading=$(grep -m1 -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$block_file" || true)
if [ -z "$incoming_heading" ]; then
echo "::error::${block_file} does not contain a stamped Prowler release heading"
exit 1
fi
incoming_release=$(release_from_heading "$incoming_heading")
incoming_key=$(version_key "$incoming_release")
insertion_line=""
duplicate_line=""
while IFS=: read -r line heading; do
existing_release=$(release_from_heading "$heading")
existing_key=$(version_key "$existing_release")
if [[ "$incoming_key" == "$existing_key" ]]; then
duplicate_line="$line"
break
fi
if [[ "$incoming_key" > "$existing_key" ]]; then
insertion_line="$line"
break
fi
done < <(grep -n -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$changelog" || true)
if [ -n "$duplicate_line" ]; then
echo "::error::${changelog} already contains a block for Prowler v${incoming_release} at line ${duplicate_line}; refusing to insert a duplicate"
exit 1
fi
if [ -z "$insertion_line" ]; then
insertion_line=$(($(wc -l < "$changelog") + 1))
fi
if [ "$insertion_line" -le "$marker_line" ]; then
insertion_line=$((marker_line + 1))
fi
# The captured block window can be off by one blank line on either
# end (towncrier re-emits the blank after the marker), so strip the
# outer blank lines and pad exactly one on each side: the block
# must never glue to the marker above or the next heading below.
awk '
/[^[:space:]]/ { for (i = 0; i < pending; i++) print ""; pending = 0; print; started = 1; next }
started { pending++ }
' "$block_file" > "${RUNNER_TEMP}/block-normalized.md"
total_lines=$(wc -l < "$changelog")
{
head -n "$((insertion_line - 1))" "$changelog"
if [ "$insertion_line" -gt 1 ] && [ -n "$(sed -n "$((insertion_line - 1))p" "$changelog")" ]; then
echo ""
fi
cat "${RUNNER_TEMP}/block-normalized.md"
if [ "$insertion_line" -le "$total_lines" ]; then
echo ""
fi
tail -n +"$insertion_line" "$changelog"
} > "${RUNNER_TEMP}/changelog.tmp"
mv "${RUNNER_TEMP}/changelog.tmp" "$changelog"
echo "::notice::Inserted ${component} changelog block for Prowler v${incoming_release} at line ${insertion_line}"
}
sync_body="${RUNNER_TEMP}/forward-sync-pr-body.md"
{
echo "### Description"
echo ""
echo "Forward-syncs the v${PROWLER_VERSION} compiled changelogs from \`${TARGET_BRANCH}\` to \`master\`: inserts the same stamped version blocks under the insertion marker and deletes the consumed fragments, so the next minor compile cannot re-release entries that already shipped in this patch. Opened automatically by the same run that opened the compile PR; review and squash-merge after it."
echo ""
echo "| Component | Fragments deleted on master | Skipped (only on ${TARGET_BRANCH}) |"
echo "|---|---|---|"
} > "$sync_body"
for component in $COMPILED_COMPONENTS; do
block_file="${RUNNER_TEMP}/block-${component}.md"
consumed_file="${RUNNER_TEMP}/consumed-${component}.txt"
if ! grep -qm1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md on master is missing the insertion marker; cannot forward-sync"
exit 1
fi
deleted=0
skipped=0
while IFS= read -r fragment; do
if [ -z "$fragment" ]; then
continue
fi
if [ -f "$fragment" ]; then
git rm -q "$fragment"
deleted=$((deleted + 1))
else
echo "::notice::${fragment} does not exist on master (change landed only on ${TARGET_BRANCH}); skipping its deletion"
skipped=$((skipped + 1))
fi
done < "$consumed_file"
insert_changelog_block_ordered "$component" "$block_file"
echo "| \`${component}\` | ${deleted} | ${skipped} |" >> "$sync_body"
done
{
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$sync_body"
echo "SYNC_BODY_FILE=${sync_body}" >> "$GITHUB_ENV"
- name: Create forward-sync PR
if: env.TARGET_BRANCH != 'master'
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
branch: forward-sync-changelogs-${{ env.PROWLER_VERSION }}
base: master
title: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
body-path: ${{ env.SYNC_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
skip-sync
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -0,0 +1,50 @@
name: 'Docs: Check Provider Cards Snippet'
on:
pull_request:
branches:
- 'master'
- 'v5.*'
paths:
- 'docs/user-guide/providers/**/getting-started-*.mdx'
- 'docs/scripts/generate_provider_cards.py'
- 'docs/snippets/provider-cards.mdx'
- 'api/src/backend/api/models.py'
- '.github/workflows/docs-check-provider-cards.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: {}
jobs:
check-provider-cards:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
github.com:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Verify provider cards snippet is up to date
run: |
if ! python3 docs/scripts/generate_provider_cards.py; then
echo "::error::docs/snippets/provider-cards.mdx is out of sync with the provider getting-started pages or the API ProviderChoices enum."
echo "Run 'python3 docs/scripts/generate_provider_cards.py' locally and commit the regenerated snippet."
echo "--- diff ---"
git diff docs/snippets/provider-cards.mdx
exit 1
fi
+6 -5
View File
@@ -25,14 +25,15 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
# We can't block as Trufflehog needs to verify secrets against vendors
egress-policy: audit
# allowed-endpoints: >
# github.com:443
# ghcr.io:443
# pkg-containers.githubusercontent.com:443
allowed-endpoints: >
github.com:443
ghcr.io:443
pkg-containers.githubusercontent.com:443
www.formbucket.com:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+1301 -703
View File
File diff suppressed because one or more lines are too long
+9 -3
View File
@@ -12,8 +12,8 @@ if: contains(toJson(github.event.issue.labels), 'status/needs-triage')
timeout-minutes: 12
rate-limit:
max: 5
user-rate-limit:
max-runs-per-window: 5
window: 60
concurrency:
@@ -30,6 +30,12 @@ permissions:
engine: copilot
strict: false
pre-steps:
- name: Harden the runner
uses: step-security/harden-runner@v2.20.0
with:
egress-policy: audit
imports:
- ../agents/issue-triage.md
@@ -108,7 +114,7 @@ Triage the following GitHub issue using the Prowler Issue Triage Agent persona.
## Sanitized Issue Content
${{ needs.activation.outputs.text }}
${{ steps.sanitized.outputs.text }}
## Instructions
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -46,7 +46,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -93,7 +93,7 @@ jobs:
fi
- name: Add community label
if: steps.check_membership.outputs.is_member == 'false'
if: steps.check_membership.outputs.is_member == 'false' && github.event.pull_request.user.type != 'Bot'
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ github.token }}
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@fa2e9d605c4eeb9fcad4c99c224cee0c6c7f3594 # v2.16.0
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -30
View File
@@ -45,7 +45,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
@@ -64,7 +64,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -106,7 +106,7 @@ jobs:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -165,7 +165,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -206,7 +206,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -227,7 +227,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -263,27 +263,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger MCP deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: mcp-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+4 -6
View File
@@ -12,9 +12,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'mcp_server/**'
- '.github/workflows/mcp-container-checks.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -36,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -71,7 +68,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -105,6 +102,7 @@ jobs:
files_ignore: |
mcp_server/README.md
mcp_server/CHANGELOG.md
mcp_server/changelog.d/**
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
@@ -127,5 +125,5 @@ jobs:
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
fail-on-critical: 'true'
severity: 'CRITICAL'
+2 -2
View File
@@ -29,7 +29,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -67,7 +67,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -8
View File
@@ -15,12 +15,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'mcp_server/pyproject.toml'
- 'mcp_server/uv.lock'
- '.github/workflows/mcp-security.yml'
- '.github/actions/osv-scanner/**'
- '.github/scripts/osv-scan.sh'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -30,7 +24,6 @@ permissions: {}
jobs:
mcp-security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
@@ -39,7 +32,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -48,7 +48,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -83,7 +83,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+186 -29
View File
@@ -19,6 +19,60 @@ concurrency:
permissions: {}
jobs:
test-changelog-attribution:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 1
persist-credentials: false
- name: Fetch PR base ref for tj-actions/changed-files
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: git fetch --depth=1 origin "${BASE_REF}"
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
.github/scripts/changelog_attribution.py
.github/workflows/pr-check-changelog.yml
.github/workflows/compile-changelogs.yml
.github/towncrier/template.md.jinja
*/towncrier.toml
tests/github/**
- name: Set up Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Test changelog attribution
if: steps.changed-files.outputs.any_changed == 'true'
run: |
python3 -m pip install --user --disable-pip-version-check pytest==9.0.3 towncrier==25.8.0
python3 -m pytest tests/github
check-changelog:
if: contains(github.event.pull_request.labels.*.name, 'no-changelog') == false
runs-on: ubuntu-latest
@@ -31,7 +85,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -62,53 +116,160 @@ jobs:
uv.lock
pyproject.toml
- name: Check for folder changes and changelog presence
- name: Check for folder changes and changelog fragment presence
id: check-folders
run: |
missing_changelogs=""
fragment_name_re='^[A-Za-z0-9][A-Za-z0-9._-]*\.(added|changed|deprecated|removed|fixed|security)(\.[0-9]+)?\.md$'
manual_pr_link_re='(\[\(#[0-9]+\)\]|\[#[0-9]+\]\(|\(#[0-9]+\)|github\.com/[^[:space:]/]+/[^[:space:]/]+/(pull|issues)/[0-9]+)'
folder_alt=$(echo "$MONITORED_FOLDERS" | tr ' ' '|')
missing_fragments=""
invalid_fragments=""
linked_fragments=""
handwritten_changelogs=""
all_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n')
added=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" | tr ' ' '\n')
added_or_renamed=$(printf '%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
added_modified_or_renamed=$(printf '%s\n%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
# Returns success if the folder has a valid fragment added, modified, or renamed.
has_changelog_update() {
local folder="$1"
if echo "$added_modified_or_renamed" | grep "^${folder}/changelog.d/" | sed "s|^${folder}/changelog.d/||" | grep -qE "$fragment_name_re"; then
return 0
fi
return 1
}
if [[ "${STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED}" == "true" ]]; then
# Check monitored folders
for folder in $MONITORED_FOLDERS; do
# Get files changed in this folder
changed_in_folder=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^${folder}/" || true)
if echo "$all_changed" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "Direct CHANGELOG.md edits are not allowed for ${folder}/"
handwritten_changelogs="${handwritten_changelogs}- \`${folder}/CHANGELOG.md\`"$'\n'
fi
changed_in_folder=$(echo "$all_changed" | grep "^${folder}/" | grep -v "^${folder}/CHANGELOG.md$" || true)
if [ -n "$changed_in_folder" ]; then
echo "Detected changes in ${folder}/"
# Check if CHANGELOG.md was updated
if ! echo "$changed_in_folder" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "No changelog update found for ${folder}/"
missing_changelogs="${missing_changelogs}- \`${folder}\`"$'\n'
if ! has_changelog_update "$folder"; then
echo "No changelog fragment found for ${folder}/"
missing_fragments="${missing_fragments}- \`${folder}\`"$'\n'
fi
fi
done
# Check root-level dependency files (uv.lock, pyproject.toml)
# These are associated with the prowler folder changelog
root_deps_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
root_deps_changed=$(echo "$all_changed" | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
if [ -n "$root_deps_changed" ]; then
echo "Detected changes in root dependency files: $root_deps_changed"
# Check if prowler/CHANGELOG.md was already updated (might have been caught above)
prowler_changelog_updated=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^prowler/CHANGELOG.md$" || true)
if [ -z "$prowler_changelog_updated" ]; then
if ! has_changelog_update "prowler"; then
# Only add if prowler wasn't already flagged
if ! echo "$missing_changelogs" | grep -q "prowler"; then
echo "No changelog update found for root dependency changes"
missing_changelogs="${missing_changelogs}- \`prowler\` (root dependency files changed)"$'\n'
if ! echo "$missing_fragments" | grep -q "prowler"; then
echo "No changelog fragment found for root dependency changes"
missing_fragments="${missing_fragments}- \`prowler\` (root dependency files changed)"$'\n'
fi
fi
fi
# Validate the filename of every fragment added by this PR
added_fragments=$(echo "$added_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $added_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ]; then
continue
fi
if ! echo "$name" | grep -qE "$fragment_name_re"; then
echo "Invalid fragment filename: $fragment"
invalid_fragments="${invalid_fragments}- \`${fragment}\`"$'\n'
fi
done
# Lint fragment content: the PR link is attached automatically at
# compile time, so a hand-written PR or issue link would be wrong
touched_fragments=$(echo "$added_modified_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $touched_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ] || [ ! -f "$fragment" ]; then
continue
fi
if grep -qE "$manual_pr_link_re" "$fragment"; then
echo "Fragment contains a hand-written PR or issue link: $fragment"
linked_fragments="${linked_fragments}- \`${fragment}\`"$'\n'
fi
done
fi
{
echo "missing_changelogs<<EOF"
echo -e "${missing_changelogs}"
echo "EOF"
} >> $GITHUB_OUTPUT
# Suggest a slug derived from the branch name for the bot comment
suggested_slug=$(echo "$HEAD_REF" | tr '[:upper:]' '[:lower:]' | sed 's|.*/||; s/[^a-z0-9._-]/-/g; s/^[^a-z0-9]*//')
if [ -z "$suggested_slug" ]; then
suggested_slug="my-change"
fi
fragment_help="A changelog fragment is a small Markdown file named \`<slug>.<type>.md\` under \`<component>/changelog.d/\`, where \`<type>\` is one of \`added\`, \`changed\`, \`deprecated\`, \`removed\`, \`fixed\` or \`security\`. Its content is the changelog entry text, without the PR link (added automatically at release time) and without a trailing period. For example:
\`\`\`
echo 'Entry text describing the change' > <component>/changelog.d/${suggested_slug}.fixed.md
\`\`\`
If this PR does not need a changelog entry, add the \`no-changelog\` label instead."
if [ -n "$missing_fragments" ] || [ -n "$invalid_fragments" ] || [ -n "$linked_fragments" ] || [ -n "$handwritten_changelogs" ]; then
comment_body=""
if [ -n "$missing_fragments" ]; then
comment_body="⚠️ **Changes detected in the following folders without a changelog fragment:**"$'\n\n'"${missing_fragments}"$'\n'
fi
if [ -n "$invalid_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragment filenames that do not follow the naming convention:**"$'\n\n'"${invalid_fragments}"$'\n'
fi
if [ -n "$linked_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragments containing a hand-written PR or issue link (remove it; the PR link is attached automatically at release time):**"$'\n\n'"${linked_fragments}"$'\n'
fi
if [ -n "$handwritten_changelogs" ]; then
comment_body="${comment_body}⚠️ **Direct \`CHANGELOG.md\` edits are not allowed in regular PRs:**"$'\n\n'"${handwritten_changelogs}"$'\n'
fi
comment_body="${comment_body}${fragment_help}"
else
comment_body="✅ All required changelog fragments are present."
fi
write_multiline_output() {
local name="$1"
local value="$2"
local delimiter
while true; do
delimiter="EOF_$(openssl rand -hex 16)"
if ! grep -qxF "$delimiter" <<< "$value"; then
break
fi
done
{
echo "${name}<<${delimiter}"
if [ -n "$value" ]; then
printf '%s\n' "$value"
fi
echo "${delimiter}"
} >> "$GITHUB_OUTPUT"
}
write_multiline_output "missing_fragments" "$missing_fragments"
write_multiline_output "invalid_fragments" "$invalid_fragments"
write_multiline_output "linked_fragments" "$linked_fragments"
write_multiline_output "handwritten_changelogs" "$handwritten_changelogs"
write_multiline_output "comment_body" "$comment_body"
env:
STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED: ${{ steps.changed-files.outputs.any_changed }}
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES: ${{ steps.changed-files.outputs.added_files }}
STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES: ${{ steps.changed-files.outputs.modified_files }}
STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES: ${{ steps.changed-files.outputs.renamed_files }}
HEAD_REF: ${{ github.event.pull_request.head.ref }}
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
@@ -128,14 +289,10 @@ jobs:
edit-mode: replace
body: |
<!-- changelog-check -->
${{ steps.check-folders.outputs.missing_changelogs != '' && format('⚠️ **Changes detected in the following folders without a corresponding update to the `CHANGELOG.md`:**
${{ steps.check-folders.outputs.comment_body }}
{0}
Please add an entry to the corresponding `CHANGELOG.md` file to maintain a clear history of changes.', steps.check-folders.outputs.missing_changelogs) || '✅ All necessary `CHANGELOG.md` files have been updated.' }}
- name: Fail if changelog is missing
if: steps.check-folders.outputs.missing_changelogs != ''
- name: Fail if changelog fragment is missing or invalid
if: steps.check-folders.outputs.missing_fragments != '' || steps.check-folders.outputs.invalid_fragments != '' || steps.check-folders.outputs.linked_fragments != '' || steps.check-folders.outputs.handwritten_changelogs != ''
run: |
echo "::error::Missing changelog updates in some folders"
echo "::error::Missing, invalid, or disallowed changelog updates"
exit 1
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+63 -27
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -37,8 +37,7 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 1
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
persist-credentials: false # No write token in the untrusted PR-head tree; public repo so base fetch/changed-files work unauthenticated
- name: Fetch PR base ref for tj-actions/changed-files
env:
@@ -50,6 +49,8 @@ jobs:
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: '**'
safe_output: false # Raw paths (list read via env var, injection-safe); default escaping backslash-quotes chars like () and breaks the -f test
separator: "\n" # Newline-delimited so the reader tolerates spaces and glob chars in paths
- name: Check for conflict markers
id: conflict-check
@@ -59,19 +60,18 @@ jobs:
CONFLICT_FILES=""
HAS_CONFLICTS=false
# Check each changed file for conflict markers
for file in ${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}; do
if [ -f "$file" ]; then
echo "Checking file: $file"
# Read newline-delimited paths so spaces/globs neither word-split nor glob-expand
while IFS= read -r file; do
[ -n "$file" ] || continue
[ -f "$file" ] || continue
echo "Checking file: $file"
# Look for conflict markers (more precise regex)
if grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$file" 2>/dev/null; then
echo "Conflict markers found in: $file"
CONFLICT_FILES="${CONFLICT_FILES}- \`${file}\`"$'\n'
HAS_CONFLICTS=true
fi
if grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$file" 2>/dev/null; then
echo "Conflict markers found in: $file"
CONFLICT_FILES="${CONFLICT_FILES}- \`${file}\`"$'\n'
HAS_CONFLICTS=true
fi
done
done <<< "$STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES"
if [ "$HAS_CONFLICTS" = true ]; then
echo "has_conflicts=true" >> $GITHUB_OUTPUT
@@ -88,18 +88,49 @@ jobs:
env:
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
- name: Check base-branch mergeability
id: merge-check
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
MERGEABLE=null
# GitHub computes mergeability async, so .mergeable is null until ready; poll until resolved
for attempt in 1 2 3 4 5; do
MERGEABLE=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}" --jq '.mergeable')
if [ "$MERGEABLE" != "null" ]; then
break
fi
echo "Attempt ${attempt}: mergeability not computed yet, retrying..."
sleep 3
done
# Keep 'unknown' distinct from 'clean' so we never assert a clean merge we could not confirm
case "$MERGEABLE" in
false) STATUS=conflict; echo "PR branch cannot be merged cleanly into its base branch" ;;
true) STATUS=clean; echo "PR branch merges cleanly into its base branch" ;;
*) STATUS=unknown; echo "::warning::Mergeability did not resolve after retries; leaving it undetermined" ;;
esac
echo "merge_status=${STATUS}" >> "$GITHUB_OUTPUT"
- name: Manage conflict label
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HAS_CONFLICTS: ${{ steps.conflict-check.outputs.has_conflicts }}
MERGE_STATUS: ${{ steps.merge-check.outputs.merge_status }}
run: |
LABEL_NAME="has-conflicts"
# Add or remove label based on conflict status
if [ "$HAS_CONFLICTS" = "true" ]; then
if [ "$HAS_CONFLICTS" = "true" ] || [ "$MERGE_STATUS" = "conflict" ]; then
echo "Adding conflict label to PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --add-label "$LABEL_NAME" --repo ${{ github.repository }} || true
elif [ "$MERGE_STATUS" = "unknown" ]; then
# Don't drop the label on an undetermined merge state; a later run will settle it
echo "Mergeability undetermined; leaving label unchanged"
else
echo "Removing conflict label from PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --remove-label "$LABEL_NAME" --repo ${{ github.repository }} || true
@@ -121,20 +152,25 @@ jobs:
edit-mode: replace
body: |
<!-- conflict-checker-comment -->
${{ steps.conflict-check.outputs.has_conflicts == 'true' && '⚠️ **Conflict Markers Detected**' || '✅ **Conflict Markers Resolved**' }}
${{ steps.conflict-check.outputs.has_conflicts == 'true' && format('This pull request contains unresolved conflict markers in the following files:
${{ (steps.conflict-check.outputs.has_conflicts == 'true' || steps.merge-check.outputs.merge_status == 'conflict') && '⚠️ **Conflicts Detected**' || (steps.merge-check.outputs.merge_status == 'unknown' && '️ **Conflict Check Incomplete**' || '✅ **No Conflicts**') }}
${{ steps.conflict-check.outputs.has_conflicts == 'true' && format('
**Conflict markers** are present in the following files:
{0}
Please resolve these conflicts by:
1. Locating the conflict markers: `<<<<<<<`, `=======`, and `>>>>>>>`
2. Manually editing the files to resolve the conflicts
3. Removing all conflict markers
4. Committing and pushing the changes', steps.conflict-check.outputs.conflict_files) || 'All conflict markers have been successfully resolved in this pull request.' }}
Resolve them by removing every `<<<<<<<`, `=======`, and `>>>>>>>` marker, then commit and push.', steps.conflict-check.outputs.conflict_files) || '' }}
${{ steps.merge-check.outputs.merge_status == 'conflict' && '
**Merge conflict with the base branch.** This PR cannot be merged cleanly. Update your branch with the latest base (rebase or merge) and resolve the conflicts.' || '' }}
${{ steps.merge-check.outputs.merge_status == 'unknown' && '
GitHub had not finished computing mergeability, so base-branch conflict status could not be verified on this run.' || '' }}
${{ (steps.conflict-check.outputs.has_conflicts != 'true' && steps.merge-check.outputs.merge_status == 'clean') && '
No conflict markers, and the branch merges cleanly into its base.' || '' }}
- name: Fail workflow if conflicts detected
if: steps.conflict-check.outputs.has_conflicts == 'true'
if: steps.conflict-check.outputs.has_conflicts == 'true' || steps.merge-check.outputs.merge_status == 'conflict'
env:
HAS_CONFLICTS: ${{ steps.conflict-check.outputs.has_conflicts }}
MERGE_STATUS: ${{ steps.merge-check.outputs.merge_status }}
run: |
echo "::error::Workflow failed due to conflict markers detected in the PR"
[ "$HAS_CONFLICTS" = "true" ] && echo "::error::Conflict markers detected in changed files"
[ "$MERGE_STATUS" = "conflict" ] && echo "::error::PR branch has merge conflicts with the base branch"
exit 1
+3 -3
View File
@@ -26,7 +26,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -56,6 +56,6 @@ jobs:
"PROWLER_PR_BODY": ${{ toJson(github.event.pull_request.body) }},
"PROWLER_PR_URL": ${{ toJson(github.event.pull_request.html_url) }},
"PROWLER_PR_MERGED_BY": "${{ github.event.pull_request.merged_by.login }}",
"PROWLER_PR_BASE_BRANCH": "${{ github.event.pull_request.base.ref }}",
"PROWLER_PR_HEAD_BRANCH": "${{ github.event.pull_request.head.ref }}"
"PROWLER_PR_BASE_BRANCH": ${{ toJson(github.event.pull_request.base.ref) }},
"PROWLER_PR_HEAD_BRANCH": ${{ toJson(github.event.pull_request.head.ref) }}
}
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -28,12 +28,13 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
raw.githubusercontent.com:443
objects.githubusercontent.com:443
codeload.github.com:443
release-assets.githubusercontent.com:443
@@ -41,6 +42,7 @@ jobs:
files.pythonhosted.org:443
registry.npmjs.org:443
nodejs.org:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+3 -1
View File
@@ -29,10 +29,11 @@ jobs:
- '3.10'
- '3.11'
- '3.12'
- '3.13'
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -54,6 +55,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
+3 -1
View File
@@ -12,6 +12,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -23,6 +24,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -51,7 +53,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+30 -20
View File
@@ -60,7 +60,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -98,7 +98,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -138,15 +138,18 @@ jobs:
permissions:
contents: read
packages: write
id-token: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.ecr-public.us-east-1.amazonaws.com:443
public.ecr.aws:443
sts.amazonaws.com:443
sts.us-east-1.amazonaws.com:443
registry-1.docker.io:443
production.cloudflare.docker.com:443
production.cloudfront.docker.com:443
@@ -173,14 +176,16 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
aws-region: us-east-1
role-to-assume: ${{ secrets.PUBLIC_ECR_PUSH_ROLE_ARN }}
- name: Login to Public ECR
uses: aws-actions/amazon-ecr-login@d539f0932e70871a027e9d5a9d8fc38589180a64 # v2.1.6
with:
registry-type: public
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
@@ -206,10 +211,11 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -221,6 +227,8 @@ jobs:
github.com:443
release-assets.githubusercontent.com:443
api.ecr-public.us-east-1.amazonaws.com:443
sts.amazonaws.com:443
sts.us-east-1.amazonaws.com:443
- name: Login to DockerHub
@@ -229,14 +237,16 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
aws-region: us-east-1
role-to-assume: ${{ secrets.PUBLIC_ECR_PUSH_ROLE_ARN }}
- name: Login to Public ECR
uses: aws-actions/amazon-ecr-login@d539f0932e70871a027e9d5a9d8fc38589180a64 # v2.1.6
with:
registry-type: public
- name: Create and push manifests for push event
if: github.event_name == 'push'
@@ -299,7 +309,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -320,7 +330,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+12 -26
View File
@@ -15,12 +15,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'prowler/**'
- 'Dockerfile*'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/sdk-container-checks.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -41,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -77,7 +71,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -100,6 +94,8 @@ jobs:
_http._tcp.deb.debian.org:443
powershellinfraartifacts-gkhedzdeaghdezhr.z01.azurefd.net:443
get.trivy.dev:443
raw.githubusercontent.com:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -111,25 +107,15 @@ jobs:
id: check-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: ./**
files: |
prowler/**
Dockerfile*
pyproject.toml
uv.lock
.github/workflows/sdk-container-checks.yml
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
skills/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
prowler/changelog.d/**
**/AGENTS.md
- name: Set up Docker Buildx
@@ -153,5 +139,5 @@ jobs:
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
fail-on-critical: 'true'
severity: 'CRITICAL'
+3 -3
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -66,7 +66,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -102,7 +102,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+11 -29
View File
@@ -19,16 +19,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'prowler/**'
- 'tests/**'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/sdk-tests.yml'
- '.github/workflows/sdk-security.yml'
- '.github/actions/setup-python-uv/**'
- '.github/actions/osv-scanner/**'
- '.github/scripts/osv-scan.sh'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -47,7 +37,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -71,27 +61,19 @@ jobs:
id: check-changes
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files:
./**
files: |
prowler/**
tests/**
pyproject.toml
uv.lock
.github/workflows/sdk-tests.yml
.github/workflows/sdk-security.yml
.github/actions/setup-python-uv/**
.github/actions/osv-scanner/**
.github/scripts/osv-scan.sh
files_ignore: |
.github/**
prowler/CHANGELOG.md
docs/**
permissions/**
api/**
ui/**
dashboard/**
mcp_server/**
skills/**
README.md
mkdocs.yml
.backportrc.json
.env
docker-compose*
examples/**
.gitignore
contrib/**
prowler/changelog.d/**
**/AGENTS.md
- name: Setup Python with uv
+55 -5
View File
@@ -29,10 +29,11 @@ jobs:
- '3.10'
- '3.11'
- '3.12'
- '3.13'
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -76,6 +77,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
@@ -540,7 +542,7 @@ jobs:
with:
flags: prowler-py${{ matrix.python-version }}-vercel
files: ./vercel_coverage.xml
# Scaleway Provider
- name: Check if Scaleway files changed
if: steps.check-changes.outputs.any_changed == 'true'
@@ -588,7 +590,55 @@ jobs:
with:
flags: prowler-py${{ matrix.python-version }}-stackit
files: ./stackit_coverage.xml
# Linode Provider
- name: Check if Linode files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-linode
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
./prowler/**/linode/**
./tests/**/linode/**
./uv.lock
- name: Run Linode tests
if: steps.changed-linode.outputs.any_changed == 'true'
run: uv run pytest -n auto --cov=./prowler/providers/linode --cov-report=xml:linode_coverage.xml tests/providers/linode
- name: Upload Linode coverage to Codecov
if: steps.changed-linode.outputs.any_changed == 'true'
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-linode
files: ./linode_coverage.xml
# E2E Networks Provider
- name: Check if E2E Networks files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-e2enetworks
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
./prowler/**/e2enetworks/**
./tests/**/e2enetworks/**
./uv.lock
- name: Run E2E Networks tests
if: steps.changed-e2enetworks.outputs.any_changed == 'true'
run: uv run pytest -n auto --cov=./prowler/providers/e2enetworks --cov-report=xml:e2enetworks_coverage.xml tests/providers/e2enetworks
- name: Upload E2E Networks coverage to Codecov
if: steps.changed-e2enetworks.outputs.any_changed == 'true'
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-e2enetworks
files: ./e2enetworks_coverage.xml
# External Provider (dynamic loading)
- name: Check if External Provider files changed
if: steps.check-changes.outputs.any_changed == 'true'
@@ -608,14 +658,14 @@ jobs:
- name: Upload External Provider coverage to Codecov
if: steps.changed-external.outputs.any_changed == 'true'
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-external
files: ./external_coverage.xml
# Lib
- name: Check if Lib files changed
if: steps.check-changes.outputs.any_changed == 'true'
+2 -2
View File
@@ -52,7 +52,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -73,7 +73,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
python-version: '3.12.13'
- name: Install PyYAML
run: pip install pyyaml
+3 -1
View File
@@ -10,6 +10,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -19,6 +20,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -47,7 +49,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -34
View File
@@ -32,9 +32,6 @@ env:
PROWLERCLOUD_DOCKERHUB_REPOSITORY: prowlercloud
PROWLERCLOUD_DOCKERHUB_IMAGE: prowler-ui
# Build args
NEXT_PUBLIC_API_BASE_URL: http://prowler-api:8080/api/v1
permissions: {}
jobs:
@@ -48,7 +45,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -67,7 +64,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -110,7 +107,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -146,7 +143,6 @@ jobs:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=${{ (github.event_name == 'release' || github.event_name == 'workflow_dispatch') && format('v{0}', env.RELEASE_TAG) || needs.setup.outputs.short-sha }}
NEXT_PUBLIC_API_BASE_URL=${{ env.NEXT_PUBLIC_API_BASE_URL }}
push: true
platforms: ${{ matrix.platform }}
tags: |
@@ -164,7 +160,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -205,7 +201,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -226,7 +222,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -262,27 +258,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger UI deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: ui-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+4 -6
View File
@@ -12,9 +12,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'ui/**'
- '.github/workflows/ui-container-checks.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -36,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -72,7 +69,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -105,6 +102,7 @@ jobs:
files: ui/**
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
@@ -132,5 +130,5 @@ jobs:
with:
image-name: ${{ env.IMAGE_NAME }}
image-tag: ${{ github.sha }}
fail-on-critical: 'false'
fail-on-critical: 'true'
severity: 'CRITICAL'
+15 -4
View File
@@ -14,6 +14,8 @@ on:
- '.github/test-impact.yml'
- 'ui/**'
- 'api/**' # API changes can affect UI E2E
- '!ui/CHANGELOG.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -40,7 +42,8 @@ jobs:
AUTH_SECRET: 'fallback-ci-secret-for-testing'
AUTH_TRUST_HOST: true
NEXTAUTH_URL: 'http://localhost:3000'
NEXT_PUBLIC_API_BASE_URL: 'http://localhost:8080/api/v1'
AUTH_URL: 'http://localhost:3000'
UI_API_BASE_URL: 'http://localhost:8080/api/v1'
E2E_ADMIN_USER: ${{ secrets.E2E_ADMIN_USER }}
E2E_ADMIN_PASSWORD: ${{ secrets.E2E_ADMIN_PASSWORD }}
E2E_AWS_PROVIDER_ACCOUNT_ID: ${{ secrets.E2E_AWS_PROVIDER_ACCOUNT_ID }}
@@ -77,6 +80,14 @@ jobs:
E2E_ALIBABACLOUD_ACCESS_KEY_ID: ${{ secrets.E2E_ALIBABACLOUD_ACCESS_KEY_ID }}
E2E_ALIBABACLOUD_ACCESS_KEY_SECRET: ${{ secrets.E2E_ALIBABACLOUD_ACCESS_KEY_SECRET }}
E2E_ALIBABACLOUD_ROLE_ARN: ${{ secrets.E2E_ALIBABACLOUD_ROLE_ARN }}
E2E_OKTA_DOMAIN: ${{ secrets.E2E_OKTA_DOMAIN }}
E2E_OKTA_CLIENT_ID: ${{ secrets.E2E_OKTA_CLIENT_ID }}
E2E_OKTA_BASE64_PRIVATE_KEY: ${{ secrets.E2E_OKTA_BASE64_PRIVATE_KEY }}
E2E_GOOGLEWORKSPACE_CUSTOMER_ID: ${{ secrets.E2E_GOOGLEWORKSPACE_CUSTOMER_ID }}
E2E_GOOGLEWORKSPACE_SERVICE_ACCOUNT_JSON: ${{ secrets.E2E_GOOGLEWORKSPACE_SERVICE_ACCOUNT_JSON }}
E2E_GOOGLEWORKSPACE_DELEGATED_USER: ${{ secrets.E2E_GOOGLEWORKSPACE_DELEGATED_USER }}
E2E_VERCEL_TEAM_ID: ${{ secrets.E2E_VERCEL_TEAM_ID }}
E2E_VERCEL_API_TOKEN: ${{ secrets.E2E_VERCEL_API_TOKEN }}
# Pass E2E paths from impact analysis
E2E_TEST_PATHS: ${{ needs.impact-analysis.outputs.ui-e2e }}
RUN_ALL_TESTS: ${{ needs.impact-analysis.outputs.run-all }}
@@ -85,7 +96,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -157,7 +168,7 @@ jobs:
timeout=150
elapsed=0
while [ $elapsed -lt $timeout ]; do
if curl -s ${NEXT_PUBLIC_API_BASE_URL}/docs >/dev/null 2>&1; then
if curl -s ${UI_API_BASE_URL}/docs >/dev/null 2>&1; then
echo "Prowler API is ready!"
exit 0
fi
@@ -305,7 +316,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -8
View File
@@ -15,12 +15,6 @@ on:
branches:
- 'master'
- 'v5.*'
paths:
- 'ui/package.json'
- 'ui/pnpm-lock.yaml'
- '.github/workflows/ui-security.yml'
- '.github/actions/osv-scanner/**'
- '.github/scripts/osv-scan.sh'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
@@ -30,7 +24,6 @@ permissions: {}
jobs:
ui-security-scans:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
@@ -39,7 +32,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+10 -1
View File
@@ -31,14 +31,18 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
github.com:443
registry.npmjs.org:443
nodejs.org:443
fonts.googleapis.com:443
fonts.gstatic.com:443
api.iconify.design:443
api.simplesvg.com:443
api.unisvg.com:443
api.github.com:443
release-assets.githubusercontent.com:443
cdn.playwright.dev:443
@@ -60,6 +64,7 @@ jobs:
.github/workflows/ui-tests.yml
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
@@ -131,6 +136,10 @@ jobs:
if: steps.check-changes.outputs.any_changed == 'true'
run: pnpm run healthcheck
- name: Check product-tour alignment
if: steps.check-changes.outputs.any_changed == 'true'
run: pnpm run tour:check
- name: Run pnpm audit
if: steps.check-changes.outputs.any_changed == 'true'
run: pnpm run audit
+4
View File
@@ -169,3 +169,7 @@ GEMINI.md
# Claude Code
.claude/*
# Docker
docker-compose.override.yml
docker-compose-dev.override.yml
+50 -15
View File
@@ -7,6 +7,10 @@
# P50 — dependency validation
default_install_hook_types: [pre-commit]
# Hooks run on commit only by default;
# NOTE: default_stages does NOT override a hook's manifest stages, so fixers shipping pre-push in their
# manifest need an explicit stages: ["pre-commit"] below to stay off push.
default_stages: [pre-commit]
repos:
## GENERAL (prek built-in — no external repo needed)
@@ -21,13 +25,16 @@ repos:
- id: check-json
priority: 10
- id: end-of-file-fixer
stages: ["pre-commit"]
priority: 0
- id: trailing-whitespace
stages: ["pre-commit"]
priority: 0
- id: no-commit-to-branch
priority: 10
- id: pretty-format-json
args: ["--autofix", --no-sort-keys, --no-ensure-ascii]
stages: ["pre-commit"]
priority: 10
## TOML
@@ -65,13 +72,13 @@ repos:
exclude: contrib
priority: 30
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/)
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/, docs/scripts/)
- repo: https://github.com/myint/autoflake
rev: v2.3.3
hooks:
- id: autoflake
name: "SDK - autoflake"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--in-place", "--remove-all-unused-imports", "--remove-unused-variable"]
priority: 20
@@ -80,8 +87,9 @@ repos:
hooks:
- id: isort
name: "SDK - isort"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--profile", "black"]
stages: ["pre-commit"]
priority: 20
- repo: https://github.com/psf/black
@@ -89,7 +97,7 @@ repos:
hooks:
- id: black
name: "SDK - black"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 20
- repo: https://github.com/pycqa/flake8
@@ -97,22 +105,49 @@ repos:
hooks:
- id: flake8
name: "SDK - flake8"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--ignore=E266,W503,E203,E501,W605"]
priority: 30
## PYTHON — API + MCP Server (ruff)
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.11
# Run ruff through `uv run` against each project so prek uses the exact ruff
# version pinned in that project's uv.lock — the same version GitHub Actions
# runs via `uv run ruff`. This removes the drift between the local hooks and
# CI. api/ and mcp_server/ are separate uv projects, so they need separate
# hooks (each `uv run --project` resolves its own pinned ruff + config).
- repo: local
hooks:
- id: ruff
name: "API + MCP - ruff check"
files: { glob: ["{api,mcp_server}/**/*.py"] }
args: ["--fix"]
- id: ruff-check-api
name: "API - ruff check"
entry: uv run --project ./api ruff check --fix
language: system
files: { glob: ["api/**/*.py"] }
priority: 30
- id: ruff-format
name: "API + MCP - ruff format"
files: { glob: ["{api,mcp_server}/**/*.py"] }
- id: ruff-format-api
name: "API - ruff format"
entry: uv run --project ./api ruff format
language: system
files: { glob: ["api/**/*.py"] }
priority: 20
- id: ruff-check-mcp
name: "MCP - ruff check"
entry: uv run --project ./mcp_server ruff check --fix
language: system
files: { glob: ["mcp_server/**/*.py"] }
priority: 30
- id: ruff-format-mcp
name: "MCP - ruff format"
entry: uv run --project ./mcp_server ruff format
language: system
files: { glob: ["mcp_server/**/*.py"] }
priority: 20
- id: generate-provider-cards
name: "Docs - regenerate provider cards snippet"
entry: python3 docs/scripts/generate_provider_cards.py
language: system
files: { glob: ["docs/user-guide/providers/**/getting-started-*.mdx", "docs/scripts/generate_provider_cards.py", "docs/snippets/provider-cards.mdx", "api/src/backend/api/models.py"] }
pass_filenames: false
priority: 20
## PYTHON — uv (API + SDK)
@@ -156,7 +191,7 @@ repos:
entry: pylint --disable=W,C,R,E -j 0 -rn -sn
language: system
types: [python]
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 30
- id: trufflehog
+109
View File
@@ -0,0 +1,109 @@
# Trivy ignore file for prowlercloud/prowler SDK container image.
# Each entry below documents (a) the affected package and why it ships in the
# image, (b) why the CVE is not exploitable in Prowler's runtime, and (c) the
# upstream fix status. Entries carry an expiry so they auto-force re-review.
# Entries are scoped per-package so suppressions cannot drift onto unrelated
# packages that may be assigned the same CVE in the future.
#
# Scanned by: .github/actions/trivy-scan via .github/workflows/sdk-container-checks.yml
# CVE-2026-42496 — perl-archive-tar path traversal via crafted symlinks.
# CVE-2026-8376 — perl heap buffer overflow when compiling regex.
# Packages: perl, perl-base, perl-modules-5.36, libperl5.36.
# Why ignored: perl-base is part of Debian's "Essential: yes" set; it cannot be
# removed without breaking dpkg. The Prowler SDK does not invoke perl at runtime;
# neither vulnerable code path (Archive::Tar parsing or regex compilation of
# attacker-controlled input) is reachable from Prowler. No Debian bookworm fix
# is available yet.
CVE-2026-42496 pkg:perl exp:2026-08-15
CVE-2026-42496 pkg:perl-base exp:2026-08-15
CVE-2026-42496 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-42496 pkg:libperl5.36 exp:2026-08-15
CVE-2026-8376 pkg:perl exp:2026-08-15
CVE-2026-8376 pkg:perl-base exp:2026-08-15
CVE-2026-8376 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-8376 pkg:libperl5.36 exp:2026-08-15
# CVE-2026-13221 - Perl regex trie overflow.
# Packages: perl, perl-base, perl-modules-5.36, libperl5.36.
# Why ignored: upstream confirms Perl 5.36.0 is not affected; the regression
# was introduced after this version. Debian currently marks bookworm as
# vulnerable, which causes Trivy to report a false positive.
# Ref: https://github.com/Perl/perl5/issues/23388
CVE-2026-13221 pkg:perl exp:2026-08-15
CVE-2026-13221 pkg:perl-base exp:2026-08-15
CVE-2026-13221 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-13221 pkg:libperl5.36 exp:2026-08-15
# CVE-2025-7458 — SQLite integer overflow.
# Package: libsqlite3-0.
# Why ignored: transitive dependency of CPython's stdlib sqlite3 module. The
# Prowler SDK does not open user-supplied SQLite databases; SQLite usage is
# internal and bounded. No Debian bookworm fix is available.
CVE-2025-7458 pkg:libsqlite3-0 exp:2026-08-15
# CVE-2026-43185 — Linux kernel ksmbd signedness bug.
# Package: linux-libc-dev.
# Why ignored: linux-libc-dev ships kernel headers for build-time compilation,
# not a running kernel. Containers execute against the host kernel, so these
# headers are inert at runtime. The upstream fix landed in kernel 7.0-rc2 and
# has not been backported to Debian's 6.1 LTS line.
CVE-2026-43185 pkg:linux-libc-dev exp:2026-08-15
# CVE-2023-45853 — zlib MiniZip integer overflow / heap overflow in
# zipOpenNewFileInZip4_64.
# Packages: zlib1g, zlib1g-dev.
# Why ignored: Debian Security Tracker status for bookworm is <ignored>, with
# the published rationale "contrib/minizip not built and src:zlib not producing
# binary packages" — i.e. the vulnerable symbol is not present in the libz.so
# shipped by Debian. Real-not-affected, not unpatched. Upstream fix is in
# zlib 1.3.1, available in Debian trixie (13); migrating the base image would
# clear it fully.
# Ref: https://security-tracker.debian.org/tracker/CVE-2023-45853
CVE-2023-45853 pkg:zlib1g exp:2026-08-15
CVE-2023-45853 pkg:zlib1g-dev exp:2026-08-15
# CVE-2026-55200 — libssh2 out-of-bounds write in ssh2_transport_read() due to
# an unchecked packet_length field in transport.c (heap corruption, possible RCE).
# Package: libssh2-1.
# Why ignored: libssh2-1 is pulled in only as a transitive dependency of libcurl4
# (installed in the SDK Dockerfile for the networking/PowerShell stack). The
# vulnerable path is reached exclusively when libssh2 acts as an SSH/SCP/SFTP
# client parsing transport packets from a server. Prowler never uses libcurl's
# SSH/SCP/SFTP transports; it talks to cloud provider HTTPS endpoints only, so the
# affected code is unreachable at runtime. Fixed upstream in libssh2 commit
# 97acf3df (PR #2052); no Debian bookworm fix is available yet.
# Ref: https://security-tracker.debian.org/tracker/CVE-2026-55200
CVE-2026-55200 pkg:libssh2-1 exp:2026-08-15
# --- API container image (api/Dockerfile) ---
# The entries below are specific to the Prowler API image, which ships
# PowerShell and additional build tooling on top of the same bookworm base.
# CVE-2026-7210 — CPython/Expat hash-flooding denial of service in
# `xml.parsers.expat` and `xml.etree.ElementTree`.
# Packages: the Debian system Python 3.11 (python3.11*, libpython3.11*).
# Why ignored: the API runs under the Python 3.12 interpreter shipped in its
# `.venv`; the system `python3.11` is only present because `python3-dev` is
# pulled in to compile native extensions (xmlsec, lxml) and is never executed
# at runtime. The vulnerable path requires parsing attacker-controlled XML with
# the affected interpreter, which Prowler does not do with the system Python.
# Full mitigation also needs libexpat >= 2.8.0; no Debian bookworm fix yet.
CVE-2026-7210 pkg:python3.11 exp:2026-08-15
CVE-2026-7210 pkg:python3.11-dev exp:2026-08-15
CVE-2026-7210 pkg:python3.11-minimal exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11 exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-dev exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-minimal exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-stdlib exp:2026-08-15
# CVE-2026-33278 — Unbound DNSSEC validator use-after-free (DoS, possible RCE).
# CVE-2026-42960 — Unbound DNS cache poisoning via promiscuous additional records.
# Package: libunbound8.
# Why ignored: libunbound8 is a transitive apt dependency of the TLS/networking
# stack (GnuTLS DANE support); only the shared library ships in the image. Both
# vulnerabilities require operating a live Unbound recursive DNSSEC validator
# that processes attacker-influenced DNS responses. Prowler never starts an
# Unbound resolver, so neither code path is reachable. No Debian bookworm fix yet.
CVE-2026-33278 pkg:libunbound8 exp:2026-08-15
CVE-2026-42960 pkg:libunbound8 exp:2026-08-15
+7
View File
@@ -51,6 +51,7 @@ Use these skills for detailed patterns on-demand:
| `django-migration-psql` | Django migration best practices for PostgreSQL | [SKILL.md](skills/django-migration-psql/SKILL.md) |
| `postgresql-indexing` | PostgreSQL indexing, EXPLAIN, monitoring, maintenance | [SKILL.md](skills/postgresql-indexing/SKILL.md) |
| `prowler-attack-paths-query` | Create Attack Paths openCypher queries | [SKILL.md](skills/prowler-attack-paths-query/SKILL.md) |
| `prowler-tour` | Keep product-tour definitions aligned with the UI | [SKILL.md](skills/prowler-tour/SKILL.md) |
| `gh-aw` | GitHub Agentic Workflows (gh-aw) | [SKILL.md](skills/gh-aw/SKILL.md) |
| `skill-creator` | Create new AI agent skills | [SKILL.md](skills/skill-creator/SKILL.md) |
@@ -67,10 +68,12 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
| Adding new providers | `prowler-provider` |
| Adding privilege escalation detection queries | `prowler-attack-paths-query` |
| Adding services to existing providers | `prowler-provider` |
| Adding, updating, or removing a tour definition (*.tour.ts) | `prowler-tour` |
| After creating/modifying a skill | `skill-sync` |
| App Router / Server Actions | `nextjs-16` |
| Auditing check-to-requirement mappings as a cloud auditor | `prowler-compliance` |
| Building AI chat features | `ai-sdk-5` |
| Changing button labels or section headings on a tour-covered page | `prowler-tour` |
| Committing changes | `prowler-commit` |
| Configuring MCP servers in agentic workflows | `gh-aw` |
| Create PR that requires changelog entry | `prowler-changelog` |
@@ -89,6 +92,7 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
| Creating/updating compliance frameworks | `prowler-compliance` |
| Debug why a GitHub Actions job is failing | `prowler-ci` |
| Debugging gh-aw compilation errors | `gh-aw` |
| Editing a UI file containing data-tour-id attributes | `prowler-tour` |
| Fill .github/pull_request_template.md (Context/Description/Steps to review/Checklist) | `prowler-pr` |
| Fixing bug | `tdd` |
| Fixing compliance JSON bugs (duplicate IDs, empty Section, stale refs) | `prowler-compliance` |
@@ -105,9 +109,12 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
| Modifying gh-aw workflow frontmatter or safe-outputs | `gh-aw` |
| Refactoring code | `tdd` |
| Regenerate AGENTS.md Auto-invoke tables (sync.sh) | `skill-sync` |
| Renaming or removing a data-tour-id attribute value | `prowler-tour` |
| Restructuring routes or layouts covered by a tour | `prowler-tour` |
| Review PR requirements: template, title conventions, changelog gate | `prowler-pr` |
| Review changelog format and conventions | `prowler-changelog` |
| Reviewing JSON:API compliance | `jsonapi` |
| Reviewing Prowler UI components | `prowler-ui` |
| Reviewing compliance framework PRs | `prowler-compliance-review` |
| Running makemigrations or pgmakemigrations | `django-migration-psql` |
| Syncing compliance framework with upstream catalog | `prowler-compliance` |
+16 -2
View File
@@ -1,12 +1,14 @@
FROM python:3.12.11-slim-bookworm@sha256:519591d6871b7bc437060736b9f7456b8731f1499a57e22e6c285135ae657bf7 AS build
FROM python:3.12.13-slim-bookworm@sha256:8a7e7cc04fd3e2bd787f7f24e22d5d119aa590d429b50c95dfe12b3abe52f48b AS build
LABEL maintainer="https://github.com/prowler-cloud/prowler"
LABEL org.opencontainers.image.source="https://github.com/prowler-cloud/prowler"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
# Opt out of PowerShell telemetry (Application Insights -> dc.services.visualstudio.com)
ENV POWERSHELL_TELEMETRY_OPTOUT=1
ARG TRIVY_VERSION=0.70.0
ARG TRIVY_VERSION=0.71.2
ENV TRIVY_VERSION=${TRIVY_VERSION}
ARG ZIZMOR_VERSION=1.24.1
@@ -95,6 +97,18 @@ RUN uv sync --locked --compile-bytecode && \
# Install PowerShell modules
RUN .venv/bin/python prowler/providers/m365/lib/powershell/m365_powershell.py
USER root
# Remove build-only packages from the final image after Python dependencies are installed.
RUN apt-get purge -y --auto-remove \
build-essential \
pkg-config \
libzstd-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
USER prowler
# Remove deprecated dash dependencies
RUN pip uninstall dash-html-components -y && \
pip uninstall dash-core-components -y
+63 -11
View File
@@ -1,5 +1,34 @@
.DEFAULT_GOAL:=help
DEV_LOCAL := ./scripts/development/dev-local.sh
.PHONY: dev dev-setup dev-attach dev-launch dev-stop dev-clean dev-wipe dev-status
##@ Local Development
dev: ## Start local API, worker, and database logs
$(DEV_LOCAL) all
dev-setup: ## Bootstrap local dependencies, migrations, and fixtures
$(DEV_LOCAL) setup
dev-attach: ## Attach to the local tmux development session
$(DEV_LOCAL) attach
dev-launch: ## Start the local stack on fixed ports and attach
$(DEV_LOCAL) launch
dev-stop: ## Stop the local tmux session and containers
$(DEV_LOCAL) kill
dev-clean: ## Remove stopped local development containers
$(DEV_LOCAL) clean
dev-wipe: ## Stop everything and delete local development data
$(DEV_LOCAL) wipe
dev-status: ## Show local development container status
$(DEV_LOCAL) status
##@ Testing
test: ## Test with pytest
rm -rf .coverage && \
@@ -16,18 +45,41 @@ coverage-html: ## Show Test Coverage
coverage html && \
open htmlcov/index.html
##@ Linting
format: ## Format Code
@echo "Running black..."
black .
##@ Code Quality
# `make` is the single entrypoint and mirrors CI exactly (uv run + same flags):
# SDK (prowler/, util/) -> flake8 + black + pylint
# API & MCP server -> ruff (rules live in each project's pyproject.toml)
# `format` applies fixes (incl. ruff's import/upgrade autofixes); `lint` only
# verifies and is what CI gates on.
.PHONY: format format-sdk format-api format-mcp lint lint-sdk lint-api lint-mcp
lint: ## Lint Code
@echo "Running flake8..."
flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude .venv,contrib
@echo "Running black... "
black --check .
@echo "Running pylint..."
pylint --disable=W,C,R,E -j 0 prowler util
format: format-sdk format-api format-mcp ## Format & autofix all components (SDK, API, MCP)
lint: lint-sdk lint-api lint-mcp ## Lint all components (SDK, API, MCP) — mirrors CI
format-sdk: ## Format SDK code (black)
uv run black --exclude "\.venv|api|ui|skills|mcp_server" .
lint-sdk: ## Lint SDK code (flake8, black --check, pylint)
uv run flake8 . --ignore=E266,W503,E203,E501,W605,E128 --exclude .venv,contrib,ui,api,skills,mcp_server
uv run black --exclude "\.venv|api|ui|skills|mcp_server" --check .
uv run pylint --disable=W,C,R,E -j 0 -rn -sn prowler/
format-api: ## Format & autofix API code (ruff)
cd api && uv run ruff check . --exclude contrib --fix
cd api && uv run ruff format . --exclude contrib
lint-api: ## Lint API code (ruff check + format --check)
cd api && uv run ruff check . --exclude contrib
cd api && uv run ruff format --check . --exclude contrib
format-mcp: ## Format & autofix MCP server code (ruff)
cd mcp_server && uv run ruff check . --fix
cd mcp_server && uv run ruff format .
lint-mcp: ## Lint MCP server code (ruff check + format --check)
cd mcp_server && uv run ruff check .
cd mcp_server && uv run ruff format --check .
##@ PyPI
pypi-clean: ## Delete the distribution files
+105 -46
View File
@@ -3,7 +3,7 @@
<img align="center" alt="Prowler logo" src="https://github.com/prowler-cloud/prowler/blob/master/docs/img/prowler-logo-white.png#gh-dark-mode-only" width="50%" height="50%">
</p>
<p align="center">
<b><i>Prowler</b> is the Open Cloud Security Platform trusted by thousands to automate security and compliance in any cloud environment. With hundreds of ready-to-use checks and compliance frameworks, Prowler delivers real-time, customizable monitoring and seamless integrations, making cloud security simple, scalable, and cost-effective for organizations of any size.
<b><i>Prowler</b> is the Open Cloud Security Platform trusted by thousands to automate security and compliance in any cloud environment. With thousands of ready-to-use checks and compliance frameworks, Prowler delivers real-time, customizable monitoring and seamless integrations, making cloud security simple, scalable, and cost-effective for organizations of any size.
</p>
<p align="center">
<b>Secure ANY cloud at AI Speed at <a href="https://prowler.com">prowler.com</i></b>
@@ -21,7 +21,7 @@
<a href="https://pypi.python.org/pypi/prowler/"><img alt="Python Version" src="https://img.shields.io/pypi/pyversions/prowler.svg"></a>
<a href="https://pypistats.org/packages/prowler"><img alt="PyPI Downloads" src="https://img.shields.io/pypi/dw/prowler.svg?label=downloads"></a>
<a href="https://hub.docker.com/r/toniblyx/prowler"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/toniblyx/prowler"></a>
<a href="https://gallery.ecr.aws/prowler-cloud/prowler"><img width="120" height=19" alt="AWS ECR Gallery" src="https://user-images.githubusercontent.com/3985464/151531396-b6535a68-c907-44eb-95a1-a09508178616.png"></a>
<a href="https://gallery.ecr.aws/prowler-cloud/prowler"><img width="120" height="19" alt="AWS ECR Gallery" src="https://user-images.githubusercontent.com/3985464/151531396-b6535a68-c907-44eb-95a1-a09508178616.png"></a>
<a href="https://codecov.io/gh/prowler-cloud/prowler"><img alt="Codecov coverage" src="https://codecov.io/gh/prowler-cloud/prowler/graph/badge.svg?token=OflBGsdpDl"/></a>
<a href="https://insights.linuxfoundation.org/project/prowler-cloud-prowler"><img alt="Linux Foundation insights health score" src="https://insights.linuxfoundation.org/api/badge/health-score?project=prowler-cloud-prowler"/></a>
</p>
@@ -41,7 +41,7 @@
# Description
**Prowler** is the worlds most widely used _Open-Source Cloud Security Platform_ that automates security and compliance across **any cloud environment**. With hundreds of ready-to-use security checks, remediation guidance, and compliance frameworks, Prowler is built to _“Secure ANY Cloud at AI Speed”_. Prowler delivers **AI-driven**, **customizable**, and **easy-to-use** assessments, dashboards, reports, and integrations, making cloud security **simple**, **scalable**, and **cost-effective** for organizations of any size.
**Prowler** is the worlds most widely used _Open-Source Cloud Security Platform_ that automates security and compliance across **any cloud environment**. With thousands of ready-to-use security checks, remediation guidance, and compliance frameworks, Prowler is built to _“Secure ANY Cloud at AI Speed”_. Prowler delivers **AI-driven**, **customizable**, and **easy-to-use** assessments, dashboards, reports, and integrations, making cloud security **simple**, **scalable**, and **cost-effective** for organizations of any size.
Prowler includes hundreds of built-in controls to ensure compliance with standards and frameworks, including:
@@ -54,16 +54,16 @@ Prowler includes hundreds of built-in controls to ensure compliance with standar
- **National Security Standards:** ENS (Spanish National Security Scheme) and KISA ISMS-P (Korean)
- **Custom Security Frameworks:** Tailored to your needs
## Prowler App / Prowler Cloud
## Prowler Cloud & Prowler Local Server
Prowler App / [Prowler Cloud](https://cloud.prowler.com/) is a web-based application that simplifies running Prowler across your cloud provider accounts. It provides a user-friendly interface to visualize the results and streamline your security assessments.
[Prowler Cloud](https://cloud.prowler.com/) and Prowler Local Server, its self-hosted open-source version, are web applications that simplify running Prowler across your cloud provider accounts. They provide a user-friendly interface to visualize the results and streamline your security assessments.
![Prowler App](docs/images/products/overview.png)
![Prowler Cloud](docs/images/products/overview.png)
![Risk Pipeline](docs/images/products/risk-pipeline.png)
![Threat Map](docs/images/products/threat-map.png)
>For more details, refer to the [Prowler App Documentation](https://docs.prowler.com/projects/prowler-open-source/en/latest/#prowler-app-installation)
>For more details, refer to the [Prowler Local Server documentation](https://docs.prowler.com/getting-started/installation/prowler-app)
## Prowler CLI
@@ -73,26 +73,45 @@ prowler <provider>
![Prowler CLI Execution](docs/img/short-display.png)
## Prowler Dashboard
## Prowler Local Dashboard
```console
prowler dashboard
```
![Prowler Dashboard](docs/images/products/dashboard.png)
![Prowler Local Dashboard](docs/images/products/dashboard.png)
## Attack Paths
Attack Paths automatically extends every completed AWS scan with a Neo4j graph that combines Cartography's cloud inventory with Prowler findings. The feature runs in the API worker after each scan and therefore requires:
Attack Paths automatically extends every completed AWS scan with a graph that combines Cartography's cloud inventory with Prowler findings. The feature runs in the API worker after each scan.
- An accessible Neo4j instance (the Docker Compose files already ships a `neo4j` service).
- The following environment variables so Django and Celery can connect:
Two graph backends are supported as the long-lived sink:
| Variable | Description | Default |
| --- | --- | --- |
| `NEO4J_HOST` | Hostname used by the API containers. | `neo4j` |
| `NEO4J_PORT` | Bolt port exposed by Neo4j. | `7687` |
| `NEO4J_USER` / `NEO4J_PASSWORD` | Credentials with rights to create per-tenant databases. | `neo4j` / `neo4j_password` |
- **Neo4j** (default; the Docker Compose files already ship a `neo4j` service).
- **Amazon Neptune** (cloud-managed; opt-in).
Select the sink with `ATTACK_PATHS_SINK_DATABASE` (`neo4j` or `neptune`; default `neo4j`).
> Note: Cartography ingestion always uses a temporary Neo4j database, regardless of the configured sink. The `NEO4J_*` variables below must remain set even when `ATTACK_PATHS_SINK_DATABASE=neptune`.
### Neo4j sink
| Variable | Description | Default |
| --- | --- | --- |
| `NEO4J_HOST` | Hostname used by the API containers. | `neo4j` |
| `NEO4J_PORT` | Bolt port exposed by Neo4j. | `7687` |
| `NEO4J_USER` / `NEO4J_PASSWORD` | Credentials with rights to create per-tenant databases. | `neo4j` / `neo4j_password` |
### Neptune sink
| Variable | Description | Default |
| --- | --- | --- |
| `NEPTUNE_WRITER_ENDPOINT` | Bolt host for the Neptune writer instance. Required when sink is `neptune`. | _empty_ |
| `NEPTUNE_READER_ENDPOINT` | Optional reader endpoint for read-only queries. Falls back to the writer when unset. | _empty_ |
| `NEPTUNE_PORT` | Bolt port exposed by Neptune. | `8182` |
| `AWS_REGION` | Region the Neptune cluster lives in. Required when sink is `neptune`. | _empty_ |
Neptune authenticates with SigV4 using the standard boto3 credential chain. The worker's IAM role (or `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`) supplies the credentials. There is no Neptune password variable.
Every AWS provider scan will enqueue an Attack Paths ingestion job automatically. Other cloud providers will be added in future iterations.
@@ -102,28 +121,30 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
> For the most accurate and up-to-date information about checks, services, frameworks, and categories, visit [**Prowler Hub**](https://hub.prowler.com).
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Interface |
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/user-guide/compliance/tutorials/compliance) | [Categories](https://docs.prowler.com/user-guide/cli/tutorials/misc#categories) | Support | Interface |
|---|---|---|---|---|---|---|
| AWS | 600 | 84 | 44 | 18 | Official | UI, API, CLI |
| Azure | 167 | 22 | 19 | 16 | Official | UI, API, CLI |
| GCP | 102 | 18 | 17 | 12 | Official | UI, API, CLI |
| Kubernetes | 83 | 7 | 7 | 11 | Official | UI, API, CLI |
| GitHub | 24 | 3 | 1 | 5 | Official | UI, API, CLI |
| M365 | 102 | 10 | 4 | 10 | Official | UI, API, CLI |
| OCI | 51 | 14 | 4 | 10 | Official | UI, API, CLI |
| Alibaba Cloud | 63 | 9 | 4 | 9 | Official | UI, API, CLI |
| Cloudflare | 29 | 3 | 0 | 5 | Official | UI, API, CLI |
| AWS | 615 | 86 | 47 | 19 | Official | UI, API, CLI |
| Azure | 190 | 22 | 21 | 16 | Official | UI, API, CLI |
| GCP | 109 | 20 | 19 | 12 | Official | UI, API, CLI |
| Kubernetes | 90 | 7 | 8 | 11 | Official | UI, API, CLI |
| GitHub | 24 | 3 | 2 | 5 | Official | UI, API, CLI |
| M365 | 109 | 10 | 6 | 10 | Official | UI, API, CLI |
| OCI | 52 | 14 | 5 | 10 | Official | UI, API, CLI |
| Alibaba Cloud | 63 | 9 | 6 | 9 | Official | UI, API, CLI |
| Cloudflare | 29 | 3 | 2 | 5 | Official | UI, API, CLI |
| IaC | [See `trivy` docs.](https://trivy.dev/latest/docs/coverage/iac/) | N/A | N/A | N/A | Official | UI, API, CLI |
| MongoDB Atlas | 10 | 3 | 0 | 8 | Official | UI, API, CLI |
| MongoDB Atlas | 10 | 3 | 1 | 8 | Official | UI, API, CLI |
| LLM | [See `promptfoo` docs.](https://www.promptfoo.dev/docs/red-team/plugins/) | N/A | N/A | N/A | Official | CLI |
| Image | N/A | N/A | N/A | N/A | Official | CLI, API |
| Google Workspace | 39 | 5 | 2 | 5 | Official | UI, API, CLI |
| OpenStack | 34 | 5 | 0 | 9 | Official | UI, API, CLI |
| Vercel | 26 | 6 | 0 | 8 | Official | UI, API, CLI |
| Okta | 1 | 1 | 0 | 1 | Official | CLI |
| Scaleway [Contact us](https://prowler.com/contact) | 1 | 1 | 0 | 1 | Unofficial | CLI |
| StackIT [Contact us](https://prowler.com/contact) | 7 | 2 | 0 | 3 | Unofficial | CLI |
| NHN | 6 | 2 | 1 | 0 | Unofficial | CLI |
| Google Workspace | 65 | 11 | 3 | 6 | Official | UI, API, CLI |
| OpenStack | 34 | 5 | 1 | 9 | Official | UI, API, CLI |
| Vercel | 26 | 6 | 1 | 8 | Official | UI, API, CLI |
| Okta | 29 | 8 | 2 | 2 | Official | UI, API, CLI |
| Linode [Contact us](https://prowler.com/contact) | 10 | 3 | 1 | 4 | Unofficial | CLI |
| E2E Networks [Contact us](https://prowler.com/contact) | 27 | 6 | 0 | 2 | Unofficial | CLI |
| Scaleway [Contact us](https://prowler.com/contact) | 1 | 1 | 1 | 1 | Unofficial | CLI |
| StackIT [Contact us](https://prowler.com/contact) | 7 | 2 | 1 | 3 | Unofficial | CLI |
| NHN | 6 | 2 | 2 | 0 | Unofficial | CLI |
> [!Note]
> The numbers in the table are updated periodically.
@@ -139,11 +160,11 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
# 💻 Installation
## Prowler App
## Prowler Local Server
Prowler App offers flexible installation methods tailored to various environments:
Prowler Local Server offers flexible installation methods tailored to various environments:
> For detailed instructions on using Prowler App, refer to the [Prowler App Usage Guide](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/prowler-app/).
> For detailed instructions on using Prowler Local Server, refer to the [usage guide](https://docs.prowler.com/user-guide/tutorials/prowler-app).
### Docker Compose
@@ -176,7 +197,7 @@ docker compose up -d
> [!WARNING]
> 🔒 For a secure setup, the API auto-generates a unique key pair, `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY`, and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate keys, delete the stored key files and restart the API.
Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
Once configured, access Prowler Local Server at http://localhost:3000. Sign up using your email and password to get started.
### Common Issues with Docker Pull Installation
@@ -248,7 +269,7 @@ pnpm run build
pnpm start
```
> Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
> Once configured, access Prowler Local Server at http://localhost:3000. Sign up using your email and password to get started.
#### Pre-commit Hooks Setup
@@ -266,7 +287,7 @@ Prowler CLI is available as a project in [PyPI](https://pypi.org/project/prowler
pip install prowler
prowler -v
```
>For further guidance, refer to [https://docs.prowler.com](https://docs.prowler.com/projects/prowler-open-source/en/latest/#prowler-cli-installation)
>For further guidance, refer to [https://docs.prowler.com](https://docs.prowler.com/getting-started/installation/prowler-cli)
### Containers
@@ -286,7 +307,7 @@ The container images are available here:
- Prowler CLI:
- [DockerHub](https://hub.docker.com/r/prowlercloud/prowler/tags)
- [AWS Public ECR](https://gallery.ecr.aws/prowler-cloud/prowler)
- Prowler App:
- Prowler Local Server:
- [DockerHub - Prowler UI](https://hub.docker.com/r/prowlercloud/prowler-ui/tags)
- [DockerHub - Prowler API](https://hub.docker.com/r/prowlercloud/prowler-api/tags)
@@ -336,17 +357,55 @@ Full configuration, per-provider authentication, and SARIF examples: [Prowler Gi
# ✏️ High level architecture
## Prowler App
**Prowler App** is composed of four key components:
## Prowler Local Server
**Prowler Local Server** is composed of four key components:
- **Prowler UI**: A web-based interface, built with Next.js, providing a user-friendly experience for executing Prowler scans and visualizing results.
- **Prowler API**: A backend service, developed with Django REST Framework, responsible for running Prowler scans and storing the generated results.
- **Prowler SDK**: A Python SDK designed to extend the functionality of the Prowler CLI for advanced capabilities.
- **Prowler MCP Server**: A Model Context Protocol server that provides AI tools for Lighthouse, the AI-powered security assistant. This is a critical dependency for Lighthouse functionality.
![Prowler App Architecture](docs/images/products/prowler-app-architecture.png)
```mermaid
flowchart TB
user([User / Security Team])
cli([Prowler CLI])
<!-- Diagram source: docs/images/products/prowler-app-architecture.mmd — edit there, re-render at https://mermaid.live, and replace the PNG. -->
subgraph APP["Prowler Local Server"]
ui["Prowler UI<br/>(Next.js)"]
api["Prowler API<br/>(Django REST Framework)"]
worker["API Worker<br/>(Celery)"]
beat["API Scheduler<br/>(Celery Beat)"]
mcp["Prowler MCP Server<br/>(Lighthouse AI tools)"]
end
sdk["Prowler SDK<br/>(Python)"]
subgraph DATA["Data Layer"]
pg[("PostgreSQL")]
valkey[("Valkey / Redis")]
neo4j[("Neo4j")]
end
providers["Providers"]
user --> ui
user --> cli
ui -->|REST| api
ui -->|MCP HTTP| mcp
mcp -->|REST| api
api --> pg
api --> valkey
beat -->|enqueue jobs| valkey
valkey -->|dispatch| worker
worker --> pg
worker -->|Attack Paths| neo4j
worker -->|invokes| sdk
cli --> sdk
sdk --> providers
```
<!-- Diagram source: docs/images/products/prowler-app-architecture.mmd — keep this inline block, the docs page getting-started/products/prowler-app.mdx, and the .mmd file in sync. -->
## Prowler CLI
+3
View File
@@ -24,6 +24,9 @@ DJANGO_THROTTLE_TOKEN_OBTAIN=50/minute
# Decide whether to allow Django manage database table partitions
DJANGO_MANAGE_DB_PARTITIONS=[True|False]
DJANGO_CELERY_DEADLOCK_ATTEMPTS=5
# Optional: bound Celery's prefork pool size. Unset → Celery uses os.cpu_count().
# Useful on Kubernetes nodes with many CPUs where unbounded prefork balloons memory.
# DJANGO_CELERY_WORKER_CONCURRENCY=4
DJANGO_BROKER_VISIBILITY_TIMEOUT=86400
DJANGO_SENTRY_DSN=
+151 -1
View File
@@ -2,6 +2,156 @@
All notable changes to the **Prowler API** are documented in this file.
<!-- changelog: release notes start -->
## [1.35.0] (Prowler v5.34.0)
### 🐞 Fixed
- `rls_transaction` now falls back directly to the primary DB for connection-level mid-query read replica failures via `execute_wrapper`, reducing non-streaming read crashes during replica recovery [(#10379)](https://github.com/prowler-cloud/prowler/pull/10379)
- RBAC permission gates now combine permissions from every role assigned to a user in the active tenant [(#11979)](https://github.com/prowler-cloud/prowler/pull/11979)
- `attack-paths-cleanup-stale-scans` now retries worker pings and checks recent scan activity before failing scans and removing temporary databases [(#11986)](https://github.com/prowler-cloud/prowler/pull/11986)
### 🔐 Security
- User role relationship updates are limited to the active tenant to preserve role assignments in other tenants [(#11903)](https://github.com/prowler-cloud/prowler/pull/11903)
- `api` container image removes the unused Debian `libxml2` runtime package and scopes the `CVE-2026-13221` Trivy exception to unaffected Perl 5.36 packages [(#11991)](https://github.com/prowler-cloud/prowler/pull/11991)
---
## [1.34.2] (Prowler v5.33.2)
### 🐞 Fixed
- Attack Paths graph mutations now retry transient Neptune concurrency and deadline failures, while Neo4j mutations use managed transaction retries [(#11968)](https://github.com/prowler-cloud/prowler/pull/11968)
- Attack Paths scans now use bounded child node identifiers for normalized list values in Neo4j and Neptune, preventing Neo4j RANGE index key size failures [(#11969)](https://github.com/prowler-cloud/prowler/pull/11969)
- `scan-summary` aggregation now upserts summaries in deterministic conflict-key order, preventing PostgreSQL deadlocks during concurrent reaggregation [(#11971)](https://github.com/prowler-cloud/prowler/pull/11971)
---
## [1.34.1] (Prowler v5.33.1)
### 🐞 Fixed
- Session tokens are rejected after account password updates [(#11914)](https://github.com/prowler-cloud/prowler/pull/11914)
- Jira dispatch task results now surface user-facing Jira failure messages [(#11925)](https://github.com/prowler-cloud/prowler/pull/11925)
- AWS Attack Paths privilege escalation queries no longer fail on Neo4j with `Aggregation column contains implicit grouping expressions` [(#11939)](https://github.com/prowler-cloud/prowler/pull/11939)
### 🔐 Security
- OpenAI-compatible Lighthouse provider base URLs are restricted before connection checks [(#11940)](https://github.com/prowler-cloud/prowler/pull/11940)
- `LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS` environment variable to allow internal hosts as OpenAI-compatible Lighthouse AI base URLs [(#11942)](https://github.com/prowler-cloud/prowler/pull/11942)
---
## [1.34.0] (Prowler v5.33.0)
### 🚀 Added
- Compliance PDF reports no longer require provider credentials: findings are enriched from the provider metadata stored in the database, so reports generate even after the provider secret is deleted or its credentials become invalid [(#11845)](https://github.com/prowler-cloud/prowler/pull/11845)
### 🐞 Fixed
- Provider scans now queue behind active provider scans instead of dispatching concurrently, and resource failed-finding counters retry database conflicts with stable row locking [(#11848)](https://github.com/prowler-cloud/prowler/pull/11848)
---
## [1.33.1] (Prowler v5.32.1)
### 🐞 Fixed
- Attack Paths: Scan rows now have database defaults for `is_migrated` and `sink_backend` so `scan-perform-scheduled` inserts survive deploy skew [(#11826)](https://github.com/prowler-cloud/prowler/pull/11826)
- Invited users now keep their invitation context when completing authentication with Google, GitHub, or SAML, so the invitation is accepted during login [(#11752)](https://github.com/prowler-cloud/prowler/pull/11752)
### 🔐 Security
- User profile updates now allow users to update their own account while requiring user-management permissions to update other users in the same tenant [(#11792)](https://github.com/prowler-cloud/prowler/pull/11792)
- Kubernetes provider credentials now reject kubeconfigs using `exec` authentication in Prowler Cloud, preventing user-supplied commands from running on Cloud workers [(#11753)](https://github.com/prowler-cloud/prowler/pull/11753)
---
## [1.33.0] (Prowler v5.32.0)
### 🚀 Added
- Timestamp precision support for `/api/v1/findings` `inserted_at` and `updated_at` filters [(#11754)](https://github.com/prowler-cloud/prowler/pull/11754)
### 🔄 Changed
- Attack Paths: AWS Neptune is now supported as a persistent sink database, selectable via `ATTACK_PATHS_SINK_DATABASE=neptune` (default `neo4j`), Cartography's (bumped to 0.138.1) per-scan ingest database stays on Neo4j [(#11524)](https://github.com/prowler-cloud/prowler/pull/11524)
- Attack Paths: Scan task now checks the ingest Neo4j database and configured graph sink before starting graph ingestion [(#11743)](https://github.com/prowler-cloud/prowler/pull/11743)
- Disable PowerShell telemetry in the API container image [(#11746)](https://github.com/prowler-cloud/prowler/pull/11746)
### 🐞 Fixed
- Attack Paths: Provider graph cleanup now deletes Neo4j and Neptune relationships in directed batches before deleting nodes [(#11755)](https://github.com/prowler-cloud/prowler/pull/11755)
- `scan-perform` no longer reports an error when a provider is deleted during a running scan [(#11696)](https://github.com/prowler-cloud/prowler/pull/11696)
---
## [1.32.1] (Prowler v5.31.1)
### 🐞 Fixed
- API key auth no longer mutates `TenantAPIKey.objects` during admin DB lookups [(#11686)](https://github.com/prowler-cloud/prowler/pull/11686)
---
## [1.32.0] (Prowler v5.31.0)
### 🚀 Added
- Provider group filters for API endpoints that support cloud provider filtering, including exact and `__in` variants [(#11573)](https://github.com/prowler-cloud/prowler/pull/11573)
- Provider filters for `GET /api/v1/compliance-overviews`, `/metadata`, and `/requirements`, using latest completed scans per matching provider [(#11587)](https://github.com/prowler-cloud/prowler/pull/11587)
- Server-Sent Events (SSE) infrastructure for the API: a base viewset, a tenant-aware channel manager, and channel-name helpers backed by `django-eventstream` over Valkey Pub/Sub and served through the Gunicorn ASGI worker, so feature endpoints can stream events to clients over a single long-lived connection [(#11556)](https://github.com/prowler-cloud/prowler/pull/11556)
- `DJANGO_CELERY_WORKER_CONCURRENCY` to configure Celery workers concurrency. Unset for default behaviour [(#11075)](https://github.com/prowler-cloud/prowler/pull/11075)
### 🔄 Changed
- Gunicorn worker timeout raised from the 30s default to 120s, so long-running requests are no longer killed prematurely [(#11631)](https://github.com/prowler-cloud/prowler/pull/11631)
- Sentry now drops ASGI's `RequestAborted` errors from health-check probe disconnects on `/health/live` [(#11632)](https://github.com/prowler-cloud/prowler/pull/11632)
- Gunicorn keep-alive timeout now exceeds the load balancer idle timeout, stopping 502s from reused connections [(#11647)](https://github.com/prowler-cloud/prowler/pull/11647)
- API runs under the Uvicorn worker so keep-alive outlives the load balancer idle timeout, fixing Gunicorn's intermittent 502s [(#11663)](https://github.com/prowler-cloud/prowler/pull/11663)
- SAML logins no longer wipe a user's roles when the IdP does not send the `userType` attribute; existing roles are kept, and when `userType` names a role that does not exist it is now created with read-only access (visibility over all providers, no management permissions) instead of no permissions at all [(#11520)](https://github.com/prowler-cloud/prowler/pull/11520)
### 🐞 Fixed
- Database connections no longer leak under the ASGI worker, which previously exhausted the read replica's connection slots and caused 500s on read endpoints [(#11640)](https://github.com/prowler-cloud/prowler/pull/11640)
### 🔐 Security
- `aiohttp` to 3.14.0 and `idna` to 3.15, patching known CVEs [(#11596)](https://github.com/prowler-cloud/prowler/pull/11596)
- Container base image to `python:3.12.13-slim-bookworm` and `trivy` to 0.71.0, patching OS and Go module CVEs [(#11596)](https://github.com/prowler-cloud/prowler/pull/11596)
- `trivy` binary bumped to 0.71.0 patching embedded `golang.org/x/crypto`, `golang.org/x/net`, and Go `stdlib` CVEs [(#11592)](https://github.com/prowler-cloud/prowler/pull/11592)
---
## [1.31.3] (Prowler v5.30.3)
### 🔐 Security
- SAML logins now link to an existing account only when the asserted email domain matches the ACS endpoint and the user is already a member of that domain's tenant, fixing a cross-tenant account takeover [(GHSA-h8m9-jgf8-vwvp)](https://github.com/prowler-cloud/prowler/security/advisories/GHSA-h8m9-jgf8-vwvp)
---
## [1.31.2] (Prowler v5.30.2)
### 🔄 Changed
- `scan-compliance-overviews` task now streams the findings aggregation and the requirement-row writes so it runs faster and its peak memory no longer grows with the number of regions and frameworks [(#11591)](https://github.com/prowler-cloud/prowler/pull/11591)
---
## [1.31.1] (Prowler v5.30.1)
### 🐞 Fixed
- `compliance-overviews/attributes` now resolves the provider from the scan, so multi-provider universal frameworks (e.g. CSA CCM) return the check IDs of the scan's provider and Azure/GCP requirement details show their findings instead of appearing empty [(#11546)](https://github.com/prowler-cloud/prowler/pull/11546)
- Attack Paths: `drop_subgraph` now deletes relationships first and then nodes in batches, using less memory on Neo4j when clearing a dense provider graph [(#11557)](https://github.com/prowler-cloud/prowler/pull/11557)
- OCI scans now use API key credentials with the configured region instead of falling back to `/home/prowler/.oci/config` [(#11558)](https://github.com/prowler-cloud/prowler/pull/11558)
---
## [1.31.0] (Prowler v5.30.0)
### 🚀 Added
@@ -19,7 +169,7 @@ All notable changes to the **Prowler API** are documented in this file.
- Workers now shut down gracefully on deploy or restart, finishing or re-queueing in-flight tasks instead of being force-killed and leaving them stuck [(#11416)](https://github.com/prowler-cloud/prowler/pull/11416)
- Resource `name` is now stored and refreshed on every scan, so resources no longer keep an empty name [(#11476)](https://github.com/prowler-cloud/prowler/pull/11476)
- Compliance catalog now warms in background during startup. `compliance-overviews/attributes` returns `503` while warming, so the first request after a deploy no longer trips the API timeout [(#4554)](https://github.com/prowler-cloud/prowler-cloud/pull/4554)
- Compliance catalog now warms in background during startup. `compliance-overviews/attributes` returns `503` while warming, so the first request after a deploy no longer trips the API timeout [(#11530)](https://github.com/prowler-cloud/prowler/pull/11530)
### 🔐 Security
+22 -2
View File
@@ -1,11 +1,13 @@
FROM python:3.12.10-slim-bookworm@sha256:fd95fa221297a88e1cf49c55ec1828edd7c5a428187e67b5d1805692d11588db AS build
FROM python:3.12.13-slim-bookworm@sha256:8a7e7cc04fd3e2bd787f7f24e22d5d119aa590d429b50c95dfe12b3abe52f48b AS build
LABEL maintainer="https://github.com/prowler-cloud/api"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
# Opt out of PowerShell telemetry (Application Insights -> dc.services.visualstudio.com)
ENV POWERSHELL_TELEMETRY_OPTOUT=1
ARG TRIVY_VERSION=0.70.0
ARG TRIVY_VERSION=0.71.2
ENV TRIVY_VERSION=${TRIVY_VERSION}
ARG ZIZMOR_VERSION=1.24.1
@@ -102,6 +104,24 @@ RUN uv sync --locked --no-install-project && \
RUN .venv/bin/python .venv/lib/python3.12/site-packages/prowler/providers/m365/lib/powershell/m365_powershell.py
USER root
# Remove build-only packages from the final image after Python dependencies are installed.
RUN apt-get purge -y --auto-remove \
gcc \
g++ \
make \
libxml2-dev \
libxmlsec1-dev \
libxmlsec1-openssl \
pkg-config \
libtool \
libxslt1-dev \
python3-dev \
&& rm -rf /var/lib/apt/lists/*
USER prowler
COPY --chown=prowler:prowler src/backend/ ./backend/
COPY --chown=prowler:prowler docker-entrypoint.sh ./docker-entrypoint.sh
+36
View File
@@ -196,6 +196,42 @@ python -m celery -A config.celery worker -l info -E
The Celery worker does not detect and reload changes in the code, so you need to restart it manually when you make changes.
### Makefile-Assisted Local Deployment
This method is an additional local development workflow. It does not replace the manual local deployment or the Docker deployment described in this guide.
PostgreSQL, Valkey, and Neo4j run with Docker Compose, while Django and the Celery worker run natively through `uv`. Additionally, this workflow creates a `tmux` session with panes for the API, worker, and PostgreSQL logs.
Before using this method, ensure `docker compose`, `tmux`, and `uv` are installed.
This workflow is designed for macOS and should also work on Linux when Docker, `tmux`, and `uv` are available. Windows requires script changes before it can be supported.
From the repository root, run:
```console
make dev
```
The API will be available at:
```console
http://localhost:8080/api/v1
```
Use these commands to manage the local stack:
```console
make dev-setup # Bootstrap dependencies, migrations, and fixtures
make dev-attach # Attach to the tmux session
make dev-launch # Start the stack on fixed ports and attach
make dev-stop # Stop the tmux session and containers
make dev-clean # Remove stopped development containers
make dev-wipe # Stop everything and delete local development data
make dev-status # Show development container status
```
This workflow does not start the UI. Start it separately from the `ui/` directory when needed.
### Docker deployment
This method requires `docker` and `docker compose`.
+10
View File
@@ -0,0 +1,10 @@
# Changelog fragments
Each PR adds one small file here instead of editing `CHANGELOG.md` directly, so concurrent PRs never conflict.
- Filename: `<slug>.<type>.md`, e.g. `my-new-check.added.md` (slug is free-form: letters, digits, `.`, `_`, `-`)
- `<type>` is one of: `added`, `changed`, `deprecated`, `removed`, `fixed`, `security`
- Content: one line with the changelog entry text, without the PR link and without a trailing period (the PR link is attached automatically at release time)
- A PR adds as many fragment files as entries it needs, freely mixing types (one file per entry); same-type entries just use different slugs
Fragments are compiled into `CHANGELOG.md` when a release is prepared. Full conventions: `skills/prowler-changelog/SKILL.md`.
@@ -0,0 +1 @@
Database migrations now give up after a few seconds when another query holds the table lock, instead of stalling every request that touches that table
+9 -3
View File
@@ -21,13 +21,19 @@ apply_fixtures() {
}
start_dev_server() {
echo "Starting the development server..."
exec uv run python manage.py runserver 0.0.0.0:"${DJANGO_PORT:-8080}"
echo "Starting the development server (Gunicorn ASGI, debug + reload)..."
# Same server/worker as prod (config.asgi via the native `asgi` worker), so
# SSE streams run on the event loop exactly as they do in production. DEBUG is
# on so guniconf's `reload = DEBUG` hot-reloads edited code (and flips
# `preload_app` off so reload actually takes).
export DJANGO_DEBUG="${DJANGO_DEBUG:-True}"
export DJANGO_BIND_ADDRESS="${DJANGO_BIND_ADDRESS:-0.0.0.0}"
exec uv run gunicorn -c config/guniconf.py config.asgi:application
}
start_prod_server() {
echo "Starting the Gunicorn server..."
exec uv run gunicorn -c config/guniconf.py config.wsgi:application
exec uv run gunicorn -c config/guniconf.py config.asgi:application
}
resolve_worker_hostname() {
+1
View File
@@ -65,6 +65,7 @@ All settings have safe defaults; override via environment variables.
| Env var | Default | Purpose |
| --- | --- | --- |
| `DJANGO_CELERY_WORKER_PREFETCH_MULTIPLIER` | `1` | Tasks reserved per worker process. |
| `DJANGO_CELERY_WORKER_CONCURRENCY` | unset | Optional Celery prefork pool size. When unset, Celery uses its CPU-based default. Set this on worker containers to bound idle memory on hosts with many CPUs. |
| `DJANGO_CELERY_WORKER_SOFT_SHUTDOWN_TIMEOUT` | `60` | Seconds the worker drains/re-queues on `SIGTERM` before force-kill. |
| `DJANGO_CELERY_TASK_TIME_LIMIT` | `21600` (6h) | Hard limit for most tasks; connection checks are capped at 120s. |
| `DJANGO_CELERY_TASK_SOFT_TIME_LIMIT` | hard - 600 | Soft limit; raises `SoftTimeLimitExceeded` for cleanup. |
+47 -19
View File
@@ -14,7 +14,7 @@ dev = [
"pytest-env==1.1.3",
"pytest-randomly==3.15.0",
"pytest-xdist==3.6.1",
"ruff==0.5.0",
"ruff==0.15.11",
"tqdm==4.67.1",
"vulture==2.14",
"prek==0.3.9"
@@ -41,7 +41,9 @@ dependencies = [
"drf-spectacular==0.27.2",
"drf-spectacular-jsonapi==0.5.1",
"defusedxml==0.7.1",
"gunicorn==23.0.0",
"django-eventstream==5.3.3",
"gunicorn==26.0.0",
"uvloop==0.22.1",
"lxml==6.1.0",
"prowler @ git+https://github.com/prowler-cloud/prowler.git@master",
"psycopg2-binary==2.9.9",
@@ -56,11 +58,12 @@ dependencies = [
"matplotlib (==3.10.8)",
"reportlab (==4.4.10)",
"neo4j (==6.1.0)",
"cartography (==0.135.0)",
"cartography (==0.138.1)",
"gevent (==25.9.1)",
"werkzeug (==3.1.7)",
"sqlparse (==0.5.5)",
"fonttools (==4.62.1)"
"fonttools (==4.62.1)",
"uvicorn-worker (==0.4.0)",
]
description = "Prowler's API (Django/DRF)"
license = "Apache-2.0"
@@ -68,7 +71,24 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.31.0"
version = "1.36.0"
# Shared ruff baseline (kept in sync with mcp_server/pyproject.toml).
# target-version tracks this project's lowest supported Python.
[tool.ruff]
src = ["src"]
target-version = "py311"
[tool.ruff.lint]
# Defaults (E4/E7/E9, F) plus import sorting, modern-syntax upgrades, and
# comprehension lints — all mechanically auto-fixable. flake8-bugbear (B) is a
# good next step but needs manual cleanup (e.g. B904 raise-from), so it is left
# out of the shared baseline for now.
extend-select = [
"I", # isort — import ordering (prek's isort hook covers only the SDK)
"UP", # pyupgrade — modern syntax for the min supported Python
"C4" # flake8-comprehensions
]
[tool.uv]
# Transitive pins matching master to avoid silent drift; bump deliberately.
@@ -79,7 +99,7 @@ constraint-dependencies = [
"aiobotocore==2.25.1",
"aiofiles==24.1.0",
"aiohappyeyeballs==2.6.1",
"aiohttp==3.13.5",
"aiohttp==3.14.0",
"aioitertools==0.13.0",
"aiosignal==1.4.0",
"alibabacloud-actiontrail20200706==2.4.1",
@@ -124,9 +144,8 @@ constraint-dependencies = [
"astroid==3.2.4",
"async-timeout==5.0.1",
"attrs==25.4.0",
"authlib==1.6.9",
"authlib==1.6.12",
"autopep8==2.3.2",
"awsipranges==0.3.3",
"azure-cli-core==2.83.0",
"azure-cli-telemetry==1.1.0",
"azure-common==1.1.28",
@@ -174,7 +193,7 @@ constraint-dependencies = [
"blinker==1.9.0",
"boto3==1.40.61",
"botocore==1.40.61",
"cartography==0.135.0",
"cartography==0.138.1",
"celery==5.6.2",
"certifi==2026.1.4",
"cffi==2.0.0",
@@ -199,7 +218,6 @@ constraint-dependencies = [
"debugpy==1.8.20",
"decorator==5.2.1",
"defusedxml==0.7.1",
"detect-secrets==1.5.0",
"dill==0.4.1",
"distro==1.9.0",
"dj-rest-auth==7.0.1",
@@ -209,6 +227,7 @@ constraint-dependencies = [
"django-celery-results==2.6.0",
"django-cors-headers==4.4.0",
"django-environ==0.11.2",
"django-eventstream==5.3.3",
"django-filter==24.3",
"django-guid==3.5.0",
"django-postgres-extra==2.0.9",
@@ -253,7 +272,7 @@ constraint-dependencies = [
"grpc-google-iam-v1==0.14.3",
"grpcio==1.76.0",
"grpcio-status==1.76.0",
"gunicorn==23.0.0",
"gunicorn==26.0.0",
"h11==0.16.0",
"h2==4.3.0",
"hpack==4.1.0",
@@ -262,8 +281,8 @@ constraint-dependencies = [
"httpx==0.28.1",
"humanfriendly==10.0",
"hyperframe==6.1.0",
"iamdata==0.1.202602021",
"idna==3.11",
"iamdata==0.1.202605131",
"idna==3.15",
"importlib-metadata==8.7.1",
"inflection==0.5.1",
"iniconfig==2.3.0",
@@ -281,6 +300,7 @@ constraint-dependencies = [
"jsonschema==4.23.0",
"jsonschema-specifications==2025.9.1",
"keystoneauth1==5.13.0",
"kingfisher-bin==1.104.0",
"kiwisolver==1.4.9",
"knack==0.11.0",
"kombu==5.6.2",
@@ -315,7 +335,7 @@ constraint-dependencies = [
"neo4j==6.1.0",
"nest-asyncio==1.6.0",
"nltk==3.9.4",
"numpy==2.0.2",
"numpy==2.2.6",
"oauthlib==3.3.1",
"oci==2.169.0",
"openai==1.109.1",
@@ -344,7 +364,7 @@ constraint-dependencies = [
"psutil==7.2.2",
"psycopg2-binary==2.9.9",
"py-deviceid==0.1.1",
"py-iam-expand==0.1.0",
"py-iam-expand==0.3.0",
"py-ocsf-models==0.8.1",
"pyasn1==0.6.3",
"pyasn1-modules==0.4.2",
@@ -390,7 +410,7 @@ constraint-dependencies = [
"rpds-py==0.30.0",
"rsa==4.9.1",
"ruamel-yaml==0.19.1",
"ruff==0.5.0",
"ruff==0.15.11",
"s3transfer==0.14.0",
"scaleway==2.10.3",
"scaleway-core==2.10.3",
@@ -420,12 +440,14 @@ constraint-dependencies = [
"uritemplate==4.2.0",
"urllib3==2.7.0",
"uuid6==2024.7.10",
"uvicorn==0.49.0",
"uvloop==0.22.1",
"vine==5.1.0",
"vulture==2.14",
"wcwidth==0.5.3",
"websocket-client==1.9.0",
"werkzeug==3.1.7",
"workos==6.0.4",
"workos==6.0.8",
"wrapt==1.17.3",
"xlsxwriter==3.2.9",
"xmlsec==1.3.17",
@@ -436,8 +458,13 @@ constraint-dependencies = [
"zope-interface==8.2",
"zstd==1.5.7.3"
]
# prowler@master needs okta==3.4.2; cartography 0.135.0 declares okta<1.0.0 for an
# integration prowler does not import.
# prowler@master needs okta==3.4.2, but cartography 0.138.1 requires okta<1.0.0.
# Attack Paths does not ingest Okta today, so override the Cartography
# dependency to the Prowler pin.
#
# prowler@master needs azure-mgmt-containerservice==34.1.0, but cartography
# 0.138.1 requires azure-mgmt-containerservice>=41.0.0. Attack Paths does not
# ingest Azure today, so override the Cartography dependency to the Prowler pin.
#
# prowler@master hard-pins microsoft-kiota-abstractions==1.9.2 in [project.dependencies].
# The microsoft-kiota-http security bump to 1.9.9 (GHSA-7j59-v9qr-6fq9) requires
@@ -453,6 +480,7 @@ constraint-dependencies = [
# that request pyjwt[crypto] and leave cryptography (needed for RS256) only transitive.
override-dependencies = [
"okta==3.4.2",
"azure-mgmt-containerservice==34.1.0",
"microsoft-kiota-abstractions==1.9.9",
"dulwich==1.2.5",
"pyjwt[crypto]==2.13.0"
+91 -24
View File
@@ -1,9 +1,16 @@
from allauth.socialaccount.adapter import DefaultSocialAccountAdapter
from django.db import transaction
from api.db_router import MainRouter
from api.db_utils import rls_transaction
from api.models import Membership, Role, Tenant, User, UserRoleRelationship
from api.models import (
Membership,
Role,
SAMLConfiguration,
Tenant,
User,
UserRoleRelationship,
)
from api.utils import accept_invitation_for_user
from django.db import transaction
class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
@@ -14,11 +21,62 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
except User.DoesNotExist:
return None
@staticmethod
def _get_invitation_token(request):
for source_name in ("data", "POST"):
data = getattr(request, source_name, None) or {}
if not hasattr(data, "get"):
continue
invitation_token = data.get("invitation_token")
if invitation_token:
return invitation_token
wrapped_request = getattr(request, "_request", None)
if wrapped_request and wrapped_request is not request:
return ProwlerSocialAccountAdapter._get_invitation_token(wrapped_request)
return None
def pre_social_login(self, request, sociallogin):
# Link existing accounts with the same email address
email = sociallogin.account.extra_data.get("email")
if sociallogin.provider.id == "saml":
# For SAML, the asserted NameID email cannot be trusted on its own:
# any tenant can claim any email domain in its SAML configuration. To
# prevent cross-tenant account takeover (GHSA-h8m9-jgf8-vwvp), only link
# the incoming SAML session to an existing account when (1) the email
# domain matches the tenant whose ACS endpoint is being used and (2) the
# existing user is already a member of that tenant.
email = sociallogin.user.email
if not email:
return
domain = email.rsplit("@", 1)[-1].lower()
resolver_match = getattr(request, "resolver_match", None)
organization_slug = (
(resolver_match.kwargs or {}).get("organization_slug", "")
if resolver_match
else ""
).lower()
# The ACS endpoint is scoped per email domain; reject mismatches so an
# attacker cannot replay an assertion through another tenant's endpoint.
if organization_slug != domain:
return
try:
saml_config = SAMLConfiguration.objects.using(MainRouter.admin_db).get(
email_domain=domain
)
except SAMLConfiguration.DoesNotExist:
return
existing_user = self.get_user_by_email(email)
if existing_user and existing_user.is_member_of_tenant(
str(saml_config.tenant_id)
):
sociallogin.connect(request, existing_user)
return
if email:
existing_user = self.get_user_by_email(email)
if existing_user:
@@ -42,29 +100,38 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
user.name = social_account_name
user.save(using=MainRouter.admin_db)
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
invitation_token = self._get_invitation_token(request)
if invitation_token:
invitation, _ = accept_invitation_for_user(
user=user,
role=role,
tenant_id=tenant.id,
invitation_token=invitation_token,
)
request.prowler_invitation_token = invitation_token
request.prowler_invitation_tenant_id = str(invitation.tenant_id)
else:
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
else:
request.session["saml_user_created"] = str(user.id)
+6 -9
View File
@@ -1,14 +1,12 @@
import logging
import os
import sys
from pathlib import Path
from django.apps import AppConfig
from django.conf import settings
from config.custom_logging import BackendLogger
from config.env import env
from django.apps import AppConfig
from django.conf import settings
logger = logging.getLogger(BackendLogger.API)
@@ -30,8 +28,10 @@ class ApiConfig(AppConfig):
name = "api"
def ready(self):
from api import schema_extensions # noqa: F401
from api import signals # noqa: F401
from api import (
schema_extensions, # noqa: F401
signals, # noqa: F401
)
# Generate required cryptographic keys if not present, but only if:
# `"manage.py" not in sys.argv[0]`: If an external server (e.g., Gunicorn) is running the app
@@ -42,9 +42,6 @@ class ApiConfig(AppConfig):
):
self._ensure_crypto_keys()
# Neo4j driver is created lazily on first use (see api.attack_paths.database).
# App init never contacts Neo4j, so a Neo4j outage cannot block API startup.
def _ensure_crypto_keys(self):
"""
Orchestrator method that ensures all required cryptographic keys are present.
@@ -5,7 +5,6 @@ from api.attack_paths.queries import (
get_query_by_id,
)
__all__ = [
"AttackPathsQueryDefinition",
"AttackPathsQueryParameterDefinition",
@@ -4,10 +4,10 @@ Cypher sanitizer for custom (user-supplied) Attack Paths queries.
Two responsibilities:
1. **Validation** - reject queries containing SSRF or dangerous procedure
patterns (defense-in-depth; the primary control is ``neo4j.READ_ACCESS``).
patterns (defense-in-depth; the primary control is `neo4j.READ_ACCESS`).
2. **Provider-scoped label injection** - inject a dynamic
``_Provider_{uuid}`` label into every node pattern so the database can
`_Provider_{uuid}` label into every node pattern so the database can
use its native label index for provider isolation.
Label-injection pipeline:
@@ -22,18 +22,16 @@ Label-injection pipeline:
import re
from rest_framework.exceptions import ValidationError
from tasks.jobs.attack_paths.config import get_provider_label
# Step 1 - String / comment protection
# Single combined regex: strings first, then line comments.
# Single combined regex: strings first, then line comments
# The regex engine finds the leftmost match, so a string like 'https://prowler.com'
# is consumed as a string before the // inside it can match as a comment.
# is consumed as a string before the // inside it can match as a comment
_PROTECTED_RE = re.compile(r"'(?:[^'\\]|\\.)*'|\"(?:[^\"\\]|\\.)*\"|//[^\n]*")
# Step 2 - Clause splitting
# OPTIONAL MATCH must come before MATCH to avoid partial matching.
# `OPTIONAL MATCH` must come before `MATCH` to avoid partial matching
_CLAUSE_RE = re.compile(
r"\b(OPTIONAL\s+MATCH|MATCH|WHERE|RETURN|WITH|ORDER\s+BY"
r"|SKIP|LIMIT|UNION|UNWIND|CALL)\b",
@@ -41,10 +39,10 @@ _CLAUSE_RE = re.compile(
)
# Pass A - Labeled node patterns (all segments)
# Matches node patterns that have at least one :Label.
# (?<!\w)\( - open paren NOT preceded by a word char (excludes function calls).
# Group 1: optional variable + one or more :Label
# Group 2: optional {properties} + closing paren
# Matches node patterns that have at least one `:Label`
# `(?<!\w)\(` - open paren NOT preceded by a word char, excludes function calls
# Group 1: optional variable + one or more `:Label`
# Group 2: optional `{`properties`}` + closing paren
_LABELED_NODE_RE = re.compile(
r"(?<!\w)\("
r"("
@@ -57,9 +55,9 @@ _LABELED_NODE_RE = re.compile(
r")"
)
# Pass B - Bare node patterns (MATCH segments only)
# Matches (identifier) or (identifier {properties}) without any :Label.
# Only applied in MATCH/OPTIONAL MATCH segments.
# Pass B - Bare node patterns (`MATCH` segments only)
# Matches (identifier) or (identifier {properties}) without any `:Label`
# Only applied in `MATCH` / `OPTIONAL MATCH` segments
_BARE_NODE_RE = re.compile(
r"(?<!\w)\(" r"(\s*[a-zA-Z_]\w*)" r"(\s*(?:\{[^}]*\})?)" r"\s*\)"
)
@@ -98,6 +96,11 @@ def inject_provider_label(cypher: str, provider_id: str) -> str:
node pattern.
"""
label = get_provider_label(provider_id)
return inject_label(cypher, label)
def inject_label(cypher: str, label: str) -> str:
"""Rewrite a Cypher query to append a label to every node pattern."""
# Step 1: Protect strings and comments (single pass, leftmost-first)
protected: list[str] = []
@@ -136,9 +139,7 @@ def inject_provider_label(cypher: str, provider_id: str) -> str:
return work
# ---------------------------------------------------------------------------
# Validation
# ---------------------------------------------------------------------------
# Patterns that indicate SSRF or dangerous procedure calls
# Defense-in-depth layer - the primary control is `neo4j.READ_ACCESS`
+195 -237
View File
@@ -1,247 +1,32 @@
import atexit
import logging
import threading
"""Backwards-compatible facade over the ingest and sink modules.
from contextlib import contextmanager
from typing import Any, Iterator
Historically this module owned a single Neo4j driver used for both the
cartography temp database and the per-tenant sink database. The port to AWS
Neptune split those roles: the cartography ingest (temp) database is always
Neo4j and lives in `api.attack_paths.ingest`; the sink is configurable
(Neo4j or Neptune) and lives in `api.attack_paths.sink`. This shim preserves
the public API that `tasks/` and `api/v1/views.py` already depend on, and
dispatches to the right module by database-name prefix.
A database name starting with `db-tmp-scan-` is a cartography temp DB and
routes to ingest. Everything else routes to the configured sink.
"""
from contextlib import AbstractContextManager
from typing import Any
from uuid import UUID
import neo4j
import neo4j.exceptions
import neo4j # noqa: F401 - kept for tests that patch api.attack_paths.database.neo4j
from api.attack_paths import ingest
from api.attack_paths import sink as sink_module
from config.env import env
from django.conf import settings
from api.attack_paths.retryable_session import RetryableSession
from tasks.jobs.attack_paths.config import (
BATCH_SIZE,
PROVIDER_RESOURCE_LABEL,
get_provider_label,
from django.conf import (
settings, # noqa: F401 - kept for tests that patch ...database.settings
)
# Without this Celery goes crazy with Neo4j logging
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
SERVICE_UNAVAILABLE_MAX_RETRIES = env.int(
"ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES", default=3
)
READ_QUERY_TIMEOUT_SECONDS = env.int(
"ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30
)
MAX_CUSTOM_QUERY_NODES = env.int("ATTACK_PATHS_MAX_CUSTOM_QUERY_NODES", default=250)
# Shorter than CONN_ACQUISITION_TIMEOUT — the driver requires acquisition to be
# the longer of the two (it may include opening a new connection).
CONNECTION_TIMEOUT = env.int("NEO4J_CONNECTION_TIMEOUT", default=5)
CONN_ACQUISITION_TIMEOUT = env.int("NEO4J_CONN_ACQUISITION_TIMEOUT", default=15)
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
]
CLIENT_STATEMENT_EXCEPTION_PREFIX = "Neo.ClientError.Statement."
# Module-level process-wide driver singleton
_driver: neo4j.Driver | None = None
_lock = threading.Lock()
# Base Neo4j functions
def get_uri() -> str:
host = settings.DATABASES["neo4j"]["HOST"]
port = settings.DATABASES["neo4j"]["PORT"]
return f"bolt://{host}:{port}"
def init_driver() -> neo4j.Driver:
global _driver
if _driver is not None:
return _driver
with _lock:
if _driver is None:
uri = get_uri()
config = settings.DATABASES["neo4j"]
driver = neo4j.GraphDatabase.driver(
uri,
auth=(config["USER"], config["PASSWORD"]),
keep_alive=True,
max_connection_lifetime=7200,
connection_timeout=CONNECTION_TIMEOUT,
connection_acquisition_timeout=CONN_ACQUISITION_TIMEOUT,
max_connection_pool_size=50,
)
# Publish the singleton only after connectivity is verified so a
# failed probe does not leave an unverified driver behind. Close the
# driver on failure so a repeatedly-probed outage cannot leak pools.
try:
driver.verify_connectivity()
except Exception:
driver.close()
raise
_driver = driver
# Register cleanup handler (only runs once since we're inside the _driver is None block)
atexit.register(close_driver)
return _driver
def get_driver() -> neo4j.Driver:
return init_driver()
def close_driver() -> None: # TODO: Use it
global _driver
with _lock:
if _driver is not None:
try:
_driver.close()
finally:
_driver = None
@contextmanager
def get_session(
database: str | None = None, default_access_mode: str | None = None
) -> Iterator[RetryableSession]:
session_wrapper: RetryableSession | None = None
try:
session_wrapper = RetryableSession(
session_factory=lambda: get_driver().session(
database=database, default_access_mode=default_access_mode
),
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
)
yield session_wrapper
except neo4j.exceptions.Neo4jError as exc:
if (
default_access_mode == neo4j.READ_ACCESS
and exc.code
and exc.code in READ_EXCEPTION_CODES
):
message = "Read query not allowed"
code = READ_EXCEPTION_CODES[0]
raise WriteQueryNotAllowedException(message=message, code=code)
message = exc.message if exc.message is not None else str(exc)
if exc.code and exc.code.startswith(CLIENT_STATEMENT_EXCEPTION_PREFIX):
raise ClientStatementException(message=message, code=exc.code)
raise GraphDatabaseQueryException(message=message, code=exc.code)
finally:
if session_wrapper is not None:
session_wrapper.close()
def execute_read_query(
database: str,
cypher: str,
parameters: dict[str, Any] | None = None,
) -> neo4j.graph.Graph:
with get_session(database, default_access_mode=neo4j.READ_ACCESS) as session:
def _run(tx: neo4j.ManagedTransaction) -> neo4j.graph.Graph:
result = tx.run(
cypher, parameters or {}, timeout=READ_QUERY_TIMEOUT_SECONDS
)
return result.graph()
return session.execute_read(_run)
def create_database(database: str) -> None:
query = "CREATE DATABASE $database IF NOT EXISTS"
parameters = {"database": database}
with get_session() as session:
session.run(query, parameters)
def drop_database(database: str) -> None:
query = f"DROP DATABASE `{database}` IF EXISTS DESTROY DATA"
with get_session() as session:
session.run(query)
def drop_subgraph(database: str, provider_id: str) -> int:
"""
Delete all nodes for a provider from the tenant database.
Uses batched deletion to avoid memory issues with large graphs.
Silently returns 0 if the database doesn't exist.
"""
provider_label = get_provider_label(provider_id)
deleted_nodes = 0
try:
with get_session(database) as session:
deleted_count = 1
while deleted_count > 0:
result = session.run(
f"""
MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`)
WITH n LIMIT $batch_size
DETACH DELETE n
RETURN COUNT(n) AS deleted_nodes_count
""",
{"batch_size": BATCH_SIZE},
)
deleted_count = result.single().get("deleted_nodes_count", 0)
deleted_nodes += deleted_count
except GraphDatabaseQueryException as exc:
if exc.code == "Neo.ClientError.Database.DatabaseNotFound":
return 0
raise
return deleted_nodes
def has_provider_data(database: str, provider_id: str) -> bool:
"""
Check if any ProviderResource node exists for this provider.
Returns `False` if the database doesn't exist.
"""
provider_label = get_provider_label(provider_id)
query = f"MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`) RETURN 1 LIMIT 1"
try:
with get_session(database, default_access_mode=neo4j.READ_ACCESS) as session:
result = session.run(query)
return result.single() is not None
except GraphDatabaseQueryException as exc:
if exc.code == "Neo.ClientError.Database.DatabaseNotFound":
return False
raise
def clear_cache(database: str) -> None:
query = "CALL db.clearQueryCaches()"
try:
with get_session(database) as session:
session.run(query)
except GraphDatabaseQueryException as exc:
logging.warning(f"Failed to clear query cache for database `{database}`: {exc}")
# Neo4j functions related to Prowler + Cartography
def get_database_name(entity_id: str | UUID, temporary: bool = False) -> str:
prefix = "tmp-scan" if temporary else "tenant"
return f"db-{prefix}-{str(entity_id).lower()}"
TEMP_DB_PREFIX = "db-tmp-scan-"
# Exceptions
@@ -256,7 +41,6 @@ class GraphDatabaseQueryException(Exception):
def __str__(self) -> str:
if self.code:
return f"{self.code}: {self.message}"
return self.message
@@ -266,3 +50,177 @@ class WriteQueryNotAllowedException(GraphDatabaseQueryException):
class ClientStatementException(GraphDatabaseQueryException):
pass
# Routing
def _is_ingest_database(database: str | None) -> bool:
return bool(database) and database.startswith(TEMP_DB_PREFIX)
# Driver lifecycle
def init_driver() -> Any:
"""Initialize the configured sink backend.
The ingest driver (Neo4j for cartography temp DBs) stays lazy: it is
only initialized when a temp-DB operation actually runs, which never
happens on API pods.
"""
return sink_module.init()
def close_driver() -> None:
"""Close every driver held by this process."""
sink_module.close()
ingest.close_driver()
def get_driver() -> neo4j.Driver:
"""Return the sink backend's underlying driver.
Only meaningful for the Neo4j sink (where the backend has a single Neo4j
driver). On Neptune this returns the writer driver. Kept for tests and
legacy call-sites; prefer `get_session` for new code.
"""
backend = sink_module.get_backend()
# Neo4jSink exposes get_driver(); NeptuneSink exposes get_writer()
if hasattr(backend, "get_driver"):
return backend.get_driver()
if hasattr(backend, "get_writer"):
return backend.get_writer()
raise RuntimeError("Active sink backend does not expose a driver handle")
def verify_connectivity() -> None:
"""Raise if the configured graph database is unreachable on the API read path.
Backend-agnostic entry point for the readiness probe: Neo4j verifies its
driver, Neptune verifies the reader endpoint.
"""
sink_module.get_backend().verify_connectivity()
def verify_scan_databases_available() -> None:
"""Raise if either graph database needed by an Attack Paths scan is unavailable."""
errors: list[str] = []
first_error: Exception | None = None
try:
ingest.get_driver().verify_connectivity()
except Exception as exc:
errors.append(f"ingest Neo4j: {exc}")
first_error = exc
try:
get_driver().verify_connectivity()
except Exception as exc:
errors.append(f"sink {settings.ATTACK_PATHS_SINK_DATABASE}: {exc}")
if first_error is None:
first_error = exc
if errors:
raise RuntimeError(
"Attack Paths graph database unavailable before scan start: "
+ "; ".join(errors)
) from first_error
def get_uri() -> str:
"""Return the sink URI. Retained for backwards compatibility."""
if settings.ATTACK_PATHS_SINK_DATABASE == "neptune":
cfg = settings.DATABASES["neptune"]
return f"bolt+s://{cfg['WRITER_ENDPOINT']}:{cfg['PORT']}"
cfg = settings.DATABASES["neo4j"]
return f"bolt://{cfg['HOST']}:{cfg['PORT']}"
def get_ingest_uri() -> str:
"""Neo4j URI for the cartography temp (ingest) database, which is always
Neo4j regardless of the configured sink."""
return ingest.get_uri()
# Session API
def get_session(
database: str | None = None,
default_access_mode: str | None = None,
) -> AbstractContextManager:
"""Return a session against the right backend.
- `database` names starting with `db-tmp-scan-` always go to ingest.
- No database name → ingest (used for CREATE / DROP DATABASE admin ops).
- Any other name → sink.
"""
if _is_ingest_database(database) or database is None:
return ingest.get_session(
database=database, default_access_mode=default_access_mode
)
return sink_module.get_backend().get_session(
database=database, default_access_mode=default_access_mode
)
def execute_read_query(
database: str,
cypher: str,
parameters: dict[str, Any] | None = None,
) -> neo4j.graph.Graph:
"""Read-only query against the sink."""
return sink_module.get_backend().execute_read_query(database, cypher, parameters)
def create_database(database: str) -> None:
"""Create a database. Temp DBs always land on ingest (Neo4j).
On the Neo4j sink, tenant DBs also route to ingest because both drivers
connect to the same Neo4j cluster. On the Neptune sink, tenant DB creates
are no-ops.
"""
if _is_ingest_database(database):
ingest.create_database(database)
return
sink_module.get_backend().create_database(database)
def drop_database(database: str) -> None:
"""Drop a database. Mirrors `create_database` routing."""
if _is_ingest_database(database):
ingest.drop_database(database)
return
sink_module.get_backend().drop_database(database)
def drop_subgraph(database: str, provider_id: str) -> int:
return sink_module.get_backend().drop_subgraph(database, provider_id)
def has_provider_data(database: str, provider_id: str) -> bool:
return sink_module.get_backend().has_provider_data(database, provider_id)
def clear_cache(database: str) -> None:
if _is_ingest_database(database):
ingest.clear_cache(database)
return
sink_module.get_backend().clear_cache(database)
# Name helper
def get_database_name(entity_id: str | UUID, temporary: bool = False) -> str:
prefix = "tmp-scan" if temporary else "tenant"
return f"db-{prefix}-{str(entity_id).lower()}"
@@ -0,0 +1,29 @@
"""Cartography ingest layer.
Public surface for the per-scan Neo4j temp database driver. Implementation
lives in `api.attack_paths.ingest.driver`.
"""
from api.attack_paths.ingest.driver import (
clear_cache,
close_driver,
create_database,
drop_database,
get_driver,
get_session,
get_uri,
init_driver,
run_cypher,
)
__all__ = [
"clear_cache",
"close_driver",
"create_database",
"drop_database",
"get_driver",
"get_session",
"get_uri",
"init_driver",
"run_cypher",
]
@@ -0,0 +1,187 @@
"""Cartography ingest driver: per-scan throw-away Neo4j database.
Cartography writes each scan's graph into a throw-away Neo4j database named
`db-tmp-scan-{scan_uuid}`. This is always Neo4j, regardless of the configured
sink: Neptune is single-database and cannot host per-scan throw-away
databases. This module owns the Neo4j driver used for those temp DBs and the
admin ops they need (CREATE / DROP DATABASE).
"""
import atexit
import logging
import threading
from collections.abc import Iterator
from contextlib import contextmanager
from typing import Any
import neo4j
import neo4j.exceptions
from api.attack_paths.retryable_session import RetryableSession
from config.env import env
from django.conf import settings
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
SERVICE_UNAVAILABLE_MAX_RETRIES = env.int(
"ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES", default=3
)
CONN_ACQUISITION_TIMEOUT = env.int("NEO4J_CONN_ACQUISITION_TIMEOUT", default=15)
# TCP connect timeout, ordered below the acquisition timeout so an unreachable
# host can't pin a worker on a temp-DB op longer than this.
CONNECTION_TIMEOUT = env.int("NEO4J_CONNECTION_TIMEOUT", default=5)
MAX_CONNECTION_LIFETIME = env.int("NEO4J_MAX_CONNECTION_LIFETIME", default=7200)
MAX_CONNECTION_POOL_SIZE = env.int("NEO4J_MAX_CONNECTION_POOL_SIZE", default=50)
_driver: neo4j.Driver | None = None
_lock = threading.Lock()
def _neo4j_config() -> dict:
return settings.DATABASES["neo4j"]
def get_uri() -> str:
"""Bolt URI for the Neo4j temp (ingest) database. Always Neo4j."""
config = _neo4j_config()
host = config["HOST"]
port = config["PORT"]
if not host or not port:
raise RuntimeError(
"NEO4J_HOST / NEO4J_PORT must be set to use the attack-paths "
"temp database. Workers require Neo4j env even when the sink is Neptune."
)
return f"bolt://{host}:{port}"
def init_driver() -> neo4j.Driver:
"""Initialize the temp-database Neo4j driver. Idempotent."""
global _driver
if _driver is not None:
return _driver
with _lock:
if _driver is None:
config = _neo4j_config()
_driver = neo4j.GraphDatabase.driver(
get_uri(),
auth=(config["USER"], config["PASSWORD"]),
keep_alive=True,
max_connection_lifetime=MAX_CONNECTION_LIFETIME,
connection_timeout=CONNECTION_TIMEOUT,
connection_acquisition_timeout=CONN_ACQUISITION_TIMEOUT,
max_connection_pool_size=MAX_CONNECTION_POOL_SIZE,
)
# Best-effort connectivity check: a Neo4j that is down at boot must
# not crash the worker. The driver reconnects lazily on first use.
try:
_driver.verify_connectivity()
except Exception:
logging.warning(
"Neo4j temp-database unreachable at init; continuing with a "
"lazily-reconnecting driver",
exc_info=True,
)
atexit.register(close_driver)
return _driver
def get_driver() -> neo4j.Driver:
return init_driver()
def close_driver() -> None:
global _driver
with _lock:
if _driver is not None:
try:
_driver.close()
finally:
_driver = None
@contextmanager
def get_session(
database: str | None = None,
default_access_mode: str | None = None,
) -> Iterator[RetryableSession]:
"""Session against the Neo4j temp-database cluster. Used for temp DB sessions
and for admin operations (CREATE / DROP DATABASE) when `database` is None."""
from api.attack_paths.database import (
ClientStatementException,
GraphDatabaseQueryException,
WriteQueryNotAllowedException,
)
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
]
CLIENT_STATEMENT_EXCEPTION_PREFIX = "Neo.ClientError.Statement."
session_wrapper: RetryableSession | None = None
try:
session_wrapper = RetryableSession(
session_factory=lambda: get_driver().session(
database=database, default_access_mode=default_access_mode
),
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
)
yield session_wrapper
except neo4j.exceptions.Neo4jError as exc:
if (
default_access_mode == neo4j.READ_ACCESS
and exc.code
and exc.code in READ_EXCEPTION_CODES
):
raise WriteQueryNotAllowedException(
message="Read query not allowed", code=READ_EXCEPTION_CODES[0]
)
message = exc.message if exc.message is not None else str(exc)
if exc.code and exc.code.startswith(CLIENT_STATEMENT_EXCEPTION_PREFIX):
raise ClientStatementException(message=message, code=exc.code)
raise GraphDatabaseQueryException(message=message, code=exc.code)
finally:
if session_wrapper is not None:
session_wrapper.close()
def create_database(database: str) -> None:
"""Create a database on the Neo4j cluster. Used for temp scan DBs."""
with get_session() as session:
session.run("CREATE DATABASE $database IF NOT EXISTS", {"database": database})
def drop_database(database: str) -> None:
"""Drop a database on the Neo4j cluster. Used for temp scan DBs."""
with get_session() as session:
session.run(f"DROP DATABASE `{database}` IF EXISTS DESTROY DATA")
def clear_cache(database: str) -> None:
"""Best-effort cache clear for a Neo4j database."""
from api.attack_paths.database import GraphDatabaseQueryException
try:
with get_session(database) as session:
session.run("CALL db.clearQueryCaches()")
except GraphDatabaseQueryException as exc:
logging.warning(f"Failed to clear query cache for database `{database}`: {exc}")
def run_cypher(
database: str | None,
cypher: str,
parameters: dict[str, Any] | None = None,
) -> Any:
"""Execute Cypher directly without the context manager. Thin helper."""
with get_session(database) as session:
return session.run(cypher, parameters or {})
@@ -1,12 +1,11 @@
from api.attack_paths.queries.types import (
AttackPathsQueryDefinition,
AttackPathsQueryParameterDefinition,
)
from api.attack_paths.queries.registry import (
get_queries_for_provider,
get_query_by_id,
)
from api.attack_paths.queries.types import (
AttackPathsQueryDefinition,
AttackPathsQueryParameterDefinition,
)
__all__ = [
"AttackPathsQueryDefinition",
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,13 +1,14 @@
from api.attack_paths.queries.types import AttackPathsQueryDefinition
from api.attack_paths.queries.aws import AWS_QUERIES
# TODO: drop after Neptune cutover
from api.attack_paths.queries.aws_deprecated import AWS_DEPRECATED_QUERIES
from api.attack_paths.queries.types import AttackPathsQueryDefinition
# Query definitions organized by provider
# Query definitions for scans synced with the current schema.
_QUERY_DEFINITIONS: dict[str, list[AttackPathsQueryDefinition]] = {
"aws": AWS_QUERIES,
}
# Flat lookup by query ID for O(1) access
_QUERIES_BY_ID: dict[str, AttackPathsQueryDefinition] = {
definition.id: definition
for definitions in _QUERY_DEFINITIONS.values()
@@ -15,11 +16,45 @@ _QUERIES_BY_ID: dict[str, AttackPathsQueryDefinition] = {
}
def get_queries_for_provider(provider: str) -> list[AttackPathsQueryDefinition]:
"""Get all attack path queries for a specific provider."""
return _QUERY_DEFINITIONS.get(provider, [])
# TODO: drop after Neptune cutover
#
# Query definitions for pre-cutover scans (`AttackPathsScan.is_migrated=False`)
# whose graph data was written under the previous schema. Both maps expose the
# same query IDs so the API contract is identical regardless of which set is
# routed to.
_DEPRECATED_QUERY_DEFINITIONS: dict[str, list[AttackPathsQueryDefinition]] = {
"aws": AWS_DEPRECATED_QUERIES,
}
_DEPRECATED_QUERIES_BY_ID: dict[str, AttackPathsQueryDefinition] = {
definition.id: definition
for definitions in _DEPRECATED_QUERY_DEFINITIONS.values()
for definition in definitions
}
def get_query_by_id(query_id: str) -> AttackPathsQueryDefinition | None:
"""Get a specific attack path query by its ID."""
return _QUERIES_BY_ID.get(query_id)
def get_queries_for_provider(
provider: str,
is_migrated: bool = True,
) -> list[AttackPathsQueryDefinition]:
"""Get all attack path queries for a provider.
`is_migrated` selects the catalog: True for scans synced with the current
schema, False for pre-cutover scans still using the legacy graph shape.
# TODO: drop the `is_migrated` parameter after Neptune cutover
"""
catalog = _QUERY_DEFINITIONS if is_migrated else _DEPRECATED_QUERY_DEFINITIONS
return catalog.get(provider, [])
def get_query_by_id(
query_id: str,
is_migrated: bool = True,
) -> AttackPathsQueryDefinition | None:
"""Get a specific attack path query by ID.
`is_migrated` selects the catalog (see `get_queries_for_provider`).
# TODO: drop the `is_migrated` parameter after Neptune cutover
"""
by_id = _QUERIES_BY_ID if is_migrated else _DEPRECATED_QUERIES_BY_ID
return by_id.get(query_id)
@@ -1,5 +1,6 @@
import logging
import random
import time
from collections.abc import Callable
from typing import Any
@@ -10,17 +11,19 @@ logger = logging.getLogger(__name__)
class RetryableSession:
"""
Wrapper around `neo4j.Session` that retries `neo4j.exceptions.ServiceUnavailable` errors.
"""
"""Wrapper around ``neo4j.Session`` with a refreshable retry policy."""
def __init__(
self,
session_factory: Callable[[], neo4j.Session],
max_retries: int,
retry_if: Callable[[Exception], bool] | None = None,
initial_retry_delay_seconds: float = 0,
) -> None:
self._session_factory = session_factory
self._max_retries = max(0, max_retries)
self._retry_if = retry_if
self._initial_retry_delay_seconds = max(0.0, initial_retry_delay_seconds)
self._session = self._session_factory()
def close(self) -> None:
@@ -57,24 +60,47 @@ class RetryableSession:
method = getattr(self._session, method_name)
return method(*args, **kwargs)
except (
BrokenPipeError,
ConnectionResetError,
neo4j.exceptions.ServiceUnavailable,
) as exc: # pragma: no cover - depends on infra
except Exception as exc:
if not self._should_retry(exc):
raise
last_exc = exc
attempt += 1
if attempt > self._max_retries:
raise
delay = self._retry_delay(attempt)
logger.warning(
f"Neo4j session {method_name} failed with {type(exc).__name__} ({attempt}/{self._max_retries} attempts). Retrying..."
"Graph session %s failed with %s; retry %s/%s in %.3fs",
method_name,
type(exc).__name__,
attempt,
self._max_retries,
delay,
)
self._refresh_session()
if delay:
time.sleep(delay)
raise last_exc if last_exc else RuntimeError("Unexpected retry loop exit")
def _should_retry(self, exc: Exception) -> bool:
if isinstance(
exc,
(
BrokenPipeError,
ConnectionResetError,
neo4j.exceptions.ServiceUnavailable,
),
):
return True
return self._retry_if(exc) if self._retry_if else False
def _retry_delay(self, attempt: int) -> float:
max_delay = self._initial_retry_delay_seconds * (2**attempt)
return random.uniform(max_delay / 2, max_delay) if max_delay else 0
def _refresh_session(self) -> None:
if self._session is not None:
try:
@@ -0,0 +1,28 @@
"""Attack-paths sink database layer.
The sink is the persistent store where attack-paths graphs live after a scan
finishes. Currently selectable between Neo4j (OSS / local dev default) and
AWS Neptune (hosted dev/staging/prod). Backend is picked by the
`ATTACK_PATHS_SINK_DATABASE` setting at process init.
This package exposes the public factory API; the implementation lives in
`api.attack_paths.sink.factory`.
"""
from api.attack_paths.sink.factory import (
SinkBackend,
close,
get_backend,
get_backend_for_name,
get_backend_for_scan,
init,
)
__all__ = [
"SinkBackend",
"close",
"get_backend",
"get_backend_for_name",
"get_backend_for_scan",
"init",
]
@@ -0,0 +1,92 @@
"""Protocol every sink backend must implement."""
from contextlib import AbstractContextManager
from typing import Any, Protocol
import neo4j
class SinkDatabase(Protocol):
"""Contract for the persistent attack-paths graph store.
The `database` argument is an opaque identifier passed through from the
legacy `database.py` API surface. On Neo4j it is the per-tenant database
name (e.g. `db-tenant-{uuid}`). On Neptune it is ignored (the cluster
has a single graph, and isolation is label-based).
"""
def init(self) -> None: ...
def close(self) -> None: ...
def verify_connectivity(self) -> None:
"""Raise if the backend the API read path uses is unreachable.
Neo4j verifies its single driver. Neptune verifies the reader
driver (the endpoint the API serves reads from); on single-endpoint
clusters the reader aliases the writer, so that path is covered too.
Used by the readiness probe; must not block longer than the caller's
probe budget.
"""
...
def get_session(
self,
database: str | None = None,
default_access_mode: str | None = None,
) -> AbstractContextManager: ...
def execute_read_query(
self,
database: str,
cypher: str,
parameters: dict[str, Any] | None = None,
) -> neo4j.graph.Graph: ...
def create_database(self, database: str) -> None: ...
def drop_database(self, database: str) -> None: ...
def drop_subgraph(self, database: str, provider_id: str) -> int: ...
def has_provider_data(self, database: str, provider_id: str) -> bool: ...
def clear_cache(self, database: str) -> None: ...
def ensure_sync_indexes(self, database: str) -> None:
"""Create any index needed for the sync write path.
Called once at the start of each provider sync; must be idempotent.
Neo4j creates a `_provider_element_id` index on `_ProviderResource`;
Neptune is a no-op (its `~id` lookup needs no index).
"""
...
def write_nodes(
self,
database: str,
labels: str,
rows: list[dict[str, Any]],
) -> None:
"""Upsert a batch of nodes into the sink.
`labels` is a pre-rendered Cypher label string ready to drop after
the node variable (e.g. `` `AWSUser`:`_ProviderResource`:`_Tenant_x` ``).
Each row carries `provider_element_id` and `props`.
"""
...
def write_relationships(
self,
database: str,
rel_type: str,
provider_id: str,
rows: list[dict[str, Any]],
) -> None:
"""Upsert a batch of relationships into the sink.
Each row carries `start_element_id`, `end_element_id`,
`provider_element_id` and `props`. `rel_type` is the relationship
type (already a valid Cypher identifier).
"""
...
@@ -0,0 +1,81 @@
"""Shared batched deletion helpers for sink backends."""
import logging
import time
from typing import Any
RELATIONSHIP_DELETE_QUERY_TEMPLATES = {
"outgoing relationship": """
MATCH (n:`{provider_label}`)-[r]->()
WITH r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
"incoming relationship": """
MATCH (n:`{provider_label}`)<-[r]-()
WITH r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
}
NODE_DELETE_QUERY_TEMPLATE = """
MATCH (n:{provider_resource_label}:`{provider_label}`)
WITH n LIMIT $batch_size
DELETE n
RETURN COUNT(n) AS deleted_nodes_count
"""
def delete_batches(
*,
session: Any,
logger: logging.Logger,
log_target: str,
provider_id: str,
query: str,
phase: str,
count_key: str,
total_key: str,
deleted_key: str,
initial_total: int,
batch_size: int,
drop_t0: float,
) -> tuple[int, int]:
def delete_batch(tx: Any) -> int:
record = tx.run(query, {"batch_size": batch_size}).single()
return (record[count_key] if record else 0) or 0
deleted_total = initial_total
batches = 0
while True:
logger.info(
"Deleting %s batch from %s "
"(provider=%s, batch=%s, total_%s=%s, elapsed=%.3fs)",
phase,
log_target,
provider_id,
batches + 1,
total_key,
deleted_total,
time.perf_counter() - drop_t0,
)
deleted = session.execute_write(delete_batch)
if deleted == 0:
return deleted_total, batches
batches += 1
deleted_total += deleted
logger.info(
"Deleted %s batch from %s "
"(provider=%s, batch=%s, %s=%s, total_%s=%s, elapsed=%.3fs)",
phase,
log_target,
provider_id,
batches,
deleted_key,
deleted,
total_key,
deleted_total,
time.perf_counter() - drop_t0,
)
@@ -0,0 +1,134 @@
"""Sink backend factory and process-wide handle cache.
Picks the active backend from `settings.ATTACK_PATHS_SINK_DATABASE` at first
use, holds the active backend plus any secondary backends needed to serve
scans written under the previous configuration, and tears them all down on
process shutdown. Imported via `from api.attack_paths import sink as
sink_module`.
"""
import threading
from enum import StrEnum, auto
from api.attack_paths.sink.base import SinkDatabase
from api.models import AttackPathsScan
from django.conf import settings
# Backend names
class SinkBackend(StrEnum):
NEO4J = auto()
NEPTUNE = auto()
# Backend cache
_backend: SinkDatabase | None = None
_secondary_backends: dict[SinkBackend, SinkDatabase] = {}
_lock = threading.Lock()
def _resolve_setting() -> SinkBackend:
raw = settings.ATTACK_PATHS_SINK_DATABASE.lower()
try:
return SinkBackend(raw)
except ValueError:
valid = sorted(b.value for b in SinkBackend)
raise RuntimeError(
f"ATTACK_PATHS_SINK_DATABASE must be one of {valid}; got {raw!r}"
)
def _build_backend(name: SinkBackend) -> SinkDatabase:
if name is SinkBackend.NEO4J:
from api.attack_paths.sink.neo4j import Neo4jSink
return Neo4jSink()
if name is SinkBackend.NEPTUNE:
from api.attack_paths.sink.neptune import NeptuneSink
return NeptuneSink()
raise RuntimeError(f"Unknown sink backend {name!r}")
# Lifecycle
def init(name: SinkBackend | str | None = None) -> SinkDatabase:
"""Initialize the configured sink backend. Idempotent."""
global _backend
if _backend is not None:
return _backend
with _lock:
if _backend is None:
resolved = SinkBackend(name) if name else _resolve_setting()
backend = _build_backend(resolved)
backend.init()
_backend = backend
return _backend
def close() -> None:
"""Close the active backend and every cached secondary backend."""
global _backend
with _lock:
backends = [
b for b in (_backend, *_secondary_backends.values()) if b is not None
]
_backend = None
_secondary_backends.clear()
for backend in backends:
try:
backend.close()
except Exception: # pragma: no cover - best-effort
pass
def get_backend() -> SinkDatabase:
"""Return the active sink. Initializes on first call."""
return init()
# Per-scan routing
def get_backend_for_scan(scan: AttackPathsScan) -> SinkDatabase:
"""Route reads by the sink that stores this scan's graph."""
raw_backend = getattr(scan, "sink_backend", SinkBackend.NEO4J.value)
if not isinstance(raw_backend, str):
raw_backend = SinkBackend.NEO4J.value
return get_backend_for_name(raw_backend)
def get_backend_for_name(name: SinkBackend | str) -> SinkDatabase:
"""Return the backend named by persisted scan metadata."""
resolved = SinkBackend(name)
if resolved is _resolve_setting():
return get_backend()
return _build_backend_cached(resolved)
def _build_backend_cached(name: SinkBackend) -> SinkDatabase:
# TODO: drop after Neptune cutover
# Needed only during cutover to serve Neo4j-written scans from a Neptune-
# configured API pod (and vice versa). Once every scan is on Neptune,
# `get_backend_for_scan` becomes a one-liner returning `get_backend()`.
if name in _secondary_backends:
return _secondary_backends[name]
with _lock:
if name not in _secondary_backends:
backend = _build_backend(name)
backend.init()
_secondary_backends[name] = backend
return _secondary_backends[name]
@@ -0,0 +1,417 @@
"""Neo4j sink implementation.
Owns a Neo4j driver independent from the staging driver. On OSS and local dev
this is the only sink; on hosted deployments it runs only as a legacy read
path while phase-1 drains tenant DBs.
"""
import atexit
import logging
import threading
import time
from collections.abc import Iterator
from contextlib import AbstractContextManager, contextmanager
from typing import Any
import neo4j
import neo4j.exceptions
from api.attack_paths.retryable_session import RetryableSession
from api.attack_paths.sink.base import SinkDatabase
from api.attack_paths.sink.drop import (
NODE_DELETE_QUERY_TEMPLATE,
RELATIONSHIP_DELETE_QUERY_TEMPLATES,
delete_batches,
)
from config.env import env
from django.conf import settings
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
logger = logging.getLogger(__name__)
SERVICE_UNAVAILABLE_MAX_RETRIES = env.int(
"ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES", default=3
)
READ_QUERY_TIMEOUT_SECONDS = env.int(
"ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30
)
CONN_ACQUISITION_TIMEOUT = env.int("NEO4J_CONN_ACQUISITION_TIMEOUT", default=15)
# TCP connect timeout, ordered below the acquisition timeout so an unreachable
# host can't pin a request or the readiness probe longer than this.
CONNECTION_TIMEOUT = env.int("NEO4J_CONNECTION_TIMEOUT", default=5)
MAX_CONNECTION_LIFETIME = env.int("NEO4J_MAX_CONNECTION_LIFETIME", default=7200)
MAX_CONNECTION_POOL_SIZE = env.int("NEO4J_MAX_CONNECTION_POOL_SIZE", default=50)
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
]
CLIENT_STATEMENT_EXCEPTION_PREFIX = "Neo.ClientError.Statement."
DATABASE_NOT_FOUND_CODE = "Neo.ClientError.Database.DatabaseNotFound"
class Neo4jSink(SinkDatabase):
"""Neo4j-backed sink. Multi-database cluster; tenant isolation is physical."""
def __init__(self) -> None:
self._driver: neo4j.Driver | None = None
self._lock = threading.Lock()
self._atexit_registered = False
# Driver
def _config(self) -> dict:
return settings.DATABASES["neo4j"]
def _uri(self) -> str:
cfg = self._config()
host = cfg["HOST"]
port = cfg["PORT"]
if not host or not port:
raise RuntimeError(
"NEO4J_HOST / NEO4J_PORT must be set when ATTACK_PATHS_SINK_DATABASE=neo4j"
)
return f"bolt://{host}:{port}"
def init(self) -> neo4j.Driver:
if self._driver is not None:
return self._driver
with self._lock:
if self._driver is None:
cfg = self._config()
self._driver = neo4j.GraphDatabase.driver(
self._uri(),
auth=(cfg["USER"], cfg["PASSWORD"]),
keep_alive=True,
max_connection_lifetime=MAX_CONNECTION_LIFETIME,
connection_timeout=CONNECTION_TIMEOUT,
connection_acquisition_timeout=CONN_ACQUISITION_TIMEOUT,
max_connection_pool_size=MAX_CONNECTION_POOL_SIZE,
)
# Eager connectivity check is best-effort:
# A Neo4j that is down at boot must not crash the process, same degradation model as Postgres
# The driver reconnects lazily on first use
# /health/ready surfaces the outage until it recovers
try:
self._driver.verify_connectivity()
except Exception:
logger.warning(
"Neo4j sink unreachable at init; continuing with a lazily-reconnecting driver",
exc_info=True,
)
if not self._atexit_registered:
atexit.register(self.close)
self._atexit_registered = True
return self._driver
def _get_driver(self) -> neo4j.Driver:
return self.init()
def verify_connectivity(self) -> None:
self._get_driver().verify_connectivity()
def close(self) -> None:
with self._lock:
if self._driver is not None:
try:
self._driver.close()
finally:
self._driver = None
# Sessions
@contextmanager
def get_session(
self,
database: str | None = None,
default_access_mode: str | None = None,
) -> Iterator[RetryableSession]:
from api.attack_paths.database import (
ClientStatementException,
GraphDatabaseQueryException,
WriteQueryNotAllowedException,
)
session_wrapper: RetryableSession | None = None
try:
session_wrapper = RetryableSession(
session_factory=lambda: self._get_driver().session(
database=database, default_access_mode=default_access_mode
),
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
)
yield session_wrapper
except neo4j.exceptions.Neo4jError as exc:
if (
default_access_mode == neo4j.READ_ACCESS
and exc.code
and exc.code in READ_EXCEPTION_CODES
):
raise WriteQueryNotAllowedException(
message="Read query not allowed", code=READ_EXCEPTION_CODES[0]
)
message = exc.message if exc.message is not None else str(exc)
if exc.code and exc.code.startswith(CLIENT_STATEMENT_EXCEPTION_PREFIX):
raise ClientStatementException(message=message, code=exc.code)
raise GraphDatabaseQueryException(message=message, code=exc.code)
finally:
if session_wrapper is not None:
session_wrapper.close()
# Operations
def execute_read_query(
self,
database: str,
cypher: str,
parameters: dict[str, Any] | None = None,
) -> neo4j.graph.Graph:
with self.get_session(
database, default_access_mode=neo4j.READ_ACCESS
) as session:
def _run(tx: neo4j.ManagedTransaction) -> neo4j.graph.Graph:
result = tx.run(
cypher, parameters or {}, timeout=READ_QUERY_TIMEOUT_SECONDS
)
return result.graph()
return session.execute_read(_run)
def create_database(self, database: str) -> None:
with self.get_session() as session:
session.run(
"CREATE DATABASE $database IF NOT EXISTS", {"database": database}
)
def drop_database(self, database: str) -> None:
with self.get_session() as session:
session.run(f"DROP DATABASE `{database}` IF EXISTS DESTROY DATA")
def drop_subgraph(self, database: str, provider_id: str) -> int:
"""Delete all nodes for a provider from a tenant database, batched.
Deletes relationships then nodes in batches (not `DETACH DELETE`) so a
dense provider's graph cannot exceed Neo4j's transaction memory limit.
Silently returns 0 if the database doesn't exist.
"""
from api.attack_paths.database import GraphDatabaseQueryException
from tasks.jobs.attack_paths.config import (
BATCH_SIZE,
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
provider_label = get_provider_label(provider_id)
deleted_nodes = deleted_relationships = 0
relationship_batches = node_batches = 0
drop_t0 = time.perf_counter()
logger.info(
"Dropping provider graph from Neo4j sink database %s "
"(provider=%s, provider_label=%s)",
database,
provider_id,
provider_label,
)
try:
logger.info(
"Opening Neo4j sink session for provider graph drop "
"(database=%s, provider=%s)",
database,
provider_id,
)
with self.get_session(database) as session:
logger.info(
"Opened Neo4j sink session for provider graph drop "
"(database=%s, provider=%s)",
database,
provider_id,
)
log_target = f"Neo4j sink database {database}"
for (
phase,
query_template,
) in RELATIONSHIP_DELETE_QUERY_TEMPLATES.items():
deleted_relationships, phase_batches = delete_batches(
session=session,
logger=logger,
log_target=log_target,
provider_id=provider_id,
query=query_template.format(provider_label=provider_label),
phase=phase,
count_key="deleted_rels_count",
total_key="rels",
deleted_key="deleted_rels",
initial_total=deleted_relationships,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
relationship_batches += phase_batches
deleted_nodes, node_batches = delete_batches(
session=session,
logger=logger,
log_target=log_target,
provider_id=provider_id,
query=NODE_DELETE_QUERY_TEMPLATE.format(
provider_label=provider_label,
provider_resource_label=PROVIDER_RESOURCE_LABEL,
),
phase="node",
count_key="deleted_nodes_count",
total_key="nodes",
deleted_key="deleted_nodes",
initial_total=0,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
except GraphDatabaseQueryException as exc:
if exc.code == DATABASE_NOT_FOUND_CODE:
logger.info(
"Skipped provider graph drop from Neo4j sink database %s "
"(provider=%s, reason=database_not_found, elapsed=%.3fs)",
database,
provider_id,
time.perf_counter() - drop_t0,
)
return 0
raise
logger.info(
"Finished dropping provider graph from Neo4j sink database %s "
"(provider=%s, relationship_batches=%s, deleted_rels=%s, "
"node_batches=%s, deleted_nodes=%s, elapsed=%.3fs)",
database,
provider_id,
relationship_batches,
deleted_relationships,
node_batches,
deleted_nodes,
time.perf_counter() - drop_t0,
)
return deleted_nodes
def has_provider_data(self, database: str, provider_id: str) -> bool:
from api.attack_paths.database import GraphDatabaseQueryException
from tasks.jobs.attack_paths.config import (
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
provider_label = get_provider_label(provider_id)
query = (
f"MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`) RETURN 1 LIMIT 1"
)
try:
with self.get_session(
database, default_access_mode=neo4j.READ_ACCESS
) as session:
result = session.run(query)
return result.single() is not None
except GraphDatabaseQueryException as exc:
if exc.code == DATABASE_NOT_FOUND_CODE:
return False
raise
def clear_cache(self, database: str) -> None:
from api.attack_paths.database import GraphDatabaseQueryException
try:
with self.get_session(database) as session:
session.run("CALL db.clearQueryCaches()")
except GraphDatabaseQueryException as exc:
logger.warning(
f"Failed to clear query cache for database `{database}`: {exc}"
)
# Sync write path
def ensure_sync_indexes(self, database: str) -> None:
"""Create the `_provider_element_id` lookup index on `_ProviderResource`.
Every synced node carries the `_ProviderResource` label, so a single
index covers both node-upserts and relationship endpoint MATCHes.
Without this index the rel sync degrades to a label scan per row and
large provider syncs become unworkable.
"""
from tasks.jobs.attack_paths.config import (
PROVIDER_ELEMENT_ID_PROPERTY,
PROVIDER_RESOURCE_LABEL,
)
query = (
f"CREATE INDEX provider_element_id_idx IF NOT EXISTS "
f"FOR (n:`{PROVIDER_RESOURCE_LABEL}`) "
f"ON (n.`{PROVIDER_ELEMENT_ID_PROPERTY}`)"
)
with self.get_session(database) as session:
session.execute_write(lambda tx: tx.run(query).consume())
def write_nodes(
self,
database: str,
labels: str,
rows: list[dict[str, Any]],
) -> None:
if not rows:
return
from tasks.jobs.attack_paths.config import (
PROVIDER_ELEMENT_ID_PROPERTY,
PROVIDER_RESOURCE_LABEL,
)
query = f"""
UNWIND $rows AS row
MERGE (n:`{PROVIDER_RESOURCE_LABEL}` {{`{PROVIDER_ELEMENT_ID_PROPERTY}`: row.provider_element_id}})
SET n:{labels}
SET n += row.props
"""
with self.get_session(database) as session:
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
def write_relationships(
self,
database: str,
rel_type: str,
provider_id: str,
rows: list[dict[str, Any]],
) -> None:
if not rows:
return
from tasks.jobs.attack_paths.config import (
PROVIDER_ELEMENT_ID_PROPERTY,
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
provider_label = get_provider_label(provider_id)
query = f"""
UNWIND $rows AS row
MATCH (s:`{PROVIDER_RESOURCE_LABEL}`:`{provider_label}` {{`{PROVIDER_ELEMENT_ID_PROPERTY}`: row.start_element_id}})
MATCH (t:`{PROVIDER_RESOURCE_LABEL}`:`{provider_label}` {{`{PROVIDER_ELEMENT_ID_PROPERTY}`: row.end_element_id}})
MERGE (s)-[r:`{rel_type}` {{`{PROVIDER_ELEMENT_ID_PROPERTY}`: row.provider_element_id}}]->(t)
SET r += row.props
"""
with self.get_session(database) as session:
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
# For compatibility with test harnesses that patch the concrete driver
def get_driver(self) -> neo4j.Driver:
return self._get_driver()
# Helper for tests / external callers that want a writer session specifically
def get_read_session(
sink: Neo4jSink, database: str
) -> AbstractContextManager[RetryableSession]:
return sink.get_session(database, default_access_mode=neo4j.READ_ACCESS)
@@ -0,0 +1,508 @@
"""AWS Neptune sink implementation.
Dual Bolt drivers: one against the writer endpoint for workers, one against
the reader endpoint for the API read path. If `NEPTUNE_READER_ENDPOINT` is
unset the reader falls back to the writer driver so single-node clusters work.
Neptune is single-database. The `database` argument on the SinkDatabase
protocol is ignored; tenant / provider isolation is enforced by labels that
the sync step already writes on every node (see tasks/jobs/attack_paths/sync.py).
SigV4 auth lives at the bottom of this file as `neptune_auth_provider`. The
neo4j driver invokes the returned callable on each token refresh.
"""
import atexit
import datetime
import json
import logging
import threading
import time
from collections.abc import Callable, Iterator
from contextlib import contextmanager
from typing import Any
from urllib.parse import urlsplit
import neo4j
import neo4j.exceptions
from api.attack_paths.retryable_session import RetryableSession
from api.attack_paths.sink.base import SinkDatabase
from api.attack_paths.sink.drop import (
NODE_DELETE_QUERY_TEMPLATE,
RELATIONSHIP_DELETE_QUERY_TEMPLATES,
delete_batches,
)
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
from botocore.session import Session as BotoSession
from config.env import env
from django.conf import settings
from neo4j.auth_management import AuthManagers, ExpiringAuth
logging.getLogger("neo4j").setLevel(logging.ERROR)
logging.getLogger("neo4j").propagate = False
logger = logging.getLogger(__name__)
SERVICE_UNAVAILABLE_MAX_RETRIES = env.int(
"ATTACK_PATHS_SERVICE_UNAVAILABLE_MAX_RETRIES", default=3
)
READ_QUERY_TIMEOUT_SECONDS = env.int(
"ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30
)
# Neptune serverless cold-start can be >30s; give the driver room
CONN_ACQUISITION_TIMEOUT = env.int("NEPTUNE_CONN_ACQUISITION_TIMEOUT", default=60)
# TCP connect timeout, ordered below the acquisition timeout so an unreachable
# endpoint can't pin a request or the readiness probe longer than this. Kept
# generous: cold-start delays query execution, not the socket connect.
CONNECTION_TIMEOUT = env.int("NEPTUNE_CONNECTION_TIMEOUT", default=10)
# Roll connections hourly so SigV4 rotations and cert refreshes don't strand long-lived pool entries
MAX_CONNECTION_LIFETIME = env.int("NEPTUNE_MAX_CONNECTION_LIFETIME", default=3600)
MAX_CONNECTION_POOL_SIZE = env.int("NEPTUNE_MAX_CONNECTION_POOL_SIZE", default=50)
NEPTUNE_WRITE_RETRY_DELAY_SECONDS = 2
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
]
CLIENT_STATEMENT_EXCEPTION_PREFIX = "Neo.ClientError.Statement."
RETRYABLE_WRITE_ERROR_FRAGMENTS = (
"Operation failed due to conflicting concurrent operations",
"Operation terminated (deadline exceeded)",
)
# Refresh 60s before the 5-minute SigV4 window closes
SIGV4_TOKEN_LIFETIME_MINUTES = 4
def _is_retryable_write_error(exc: Exception) -> bool:
if not isinstance(exc, neo4j.exceptions.Neo4jError):
return False
message = exc.message or ""
return any(fragment in message for fragment in RETRYABLE_WRITE_ERROR_FRAGMENTS)
class NeptuneSink(SinkDatabase):
"""Neptune-backed sink. Single database; isolation is label-based."""
def __init__(self) -> None:
self._writer: neo4j.Driver | None = None
self._reader: neo4j.Driver | None = None
self._lock = threading.Lock()
self._atexit_registered = False
# Config
def _config(self) -> dict:
return settings.DATABASES["neptune"]
def _bolt_uri(self, endpoint: str, port: str) -> str:
return f"bolt+s://{endpoint}:{port}"
def _https_url(self, endpoint: str, port: str) -> str:
return f"https://{endpoint}:{port}"
def _build_driver(self, endpoint: str) -> neo4j.Driver:
cfg = self._config()
port = cfg["PORT"]
region = cfg["REGION"]
if not endpoint or not region:
raise RuntimeError(
"NEPTUNE_WRITER_ENDPOINT and AWS_REGION must be set when "
"ATTACK_PATHS_SINK_DATABASE=neptune"
)
return neo4j.GraphDatabase.driver(
self._bolt_uri(endpoint, port),
auth=AuthManagers.bearer(
neptune_auth_provider(region, self._https_url(endpoint, port))
),
keep_alive=True,
max_connection_lifetime=MAX_CONNECTION_LIFETIME,
connection_timeout=CONNECTION_TIMEOUT,
connection_acquisition_timeout=CONN_ACQUISITION_TIMEOUT,
max_connection_pool_size=MAX_CONNECTION_POOL_SIZE,
max_transaction_retry_time=0,
)
# Lifecycle
def init(self) -> None:
if self._writer is not None:
return
with self._lock:
if self._writer is None:
cfg = self._config()
writer_endpoint = cfg["WRITER_ENDPOINT"]
reader_endpoint = cfg["READER_ENDPOINT"] or writer_endpoint
# Eager connectivity checks are best-effort
# A Neptune that is down at boot must not crash the process, same degradation model as Postgres
# Drivers reconnect lazily on first use
# /health/ready surfaces the outage until it recovers
self._writer = self._build_driver(writer_endpoint)
self._verify_best_effort(self._writer, "writer")
if reader_endpoint == writer_endpoint:
self._reader = self._writer
else:
self._reader = self._build_driver(reader_endpoint)
self._verify_best_effort(self._reader, "reader")
if not self._atexit_registered:
atexit.register(self.close)
self._atexit_registered = True
def close(self) -> None:
with self._lock:
# `Driver.close()` is idempotent, so closing the same driver twice
# (when reader aliases writer on single-endpoint configs) is safe
for driver in (self._reader, self._writer):
if driver is None:
continue
try:
driver.close()
except Exception: # pragma: no cover - best-effort
pass
self._writer = None
self._reader = None
# Sessions
def _get_writer(self) -> neo4j.Driver:
self.init()
assert self._writer is not None
return self._writer
def _get_reader(self) -> neo4j.Driver:
self.init()
assert self._reader is not None
return self._reader
@staticmethod
def _verify_best_effort(driver: neo4j.Driver, role: str) -> None:
try:
driver.verify_connectivity()
except Exception:
logger.warning(
"Neptune %s endpoint unreachable at init; continuing with a lazily-reconnecting driver",
role,
exc_info=True,
)
def verify_connectivity(self) -> None:
# The API read path uses the reader driver
# On single-endpoint clusters it aliases the writer, so this also covers the writer
# A writer-only outage is a workers' concern (no HTTP probe there) and deliberately does not fail API readiness
self._get_reader().verify_connectivity()
@contextmanager
def get_session(
self,
database: str | None = None, # noqa: ARG002 - ignored on Neptune
default_access_mode: str | None = None,
) -> Iterator[RetryableSession]:
from api.attack_paths.database import (
ClientStatementException,
GraphDatabaseQueryException,
WriteQueryNotAllowedException,
)
driver = (
self._get_reader()
if default_access_mode == neo4j.READ_ACCESS
else self._get_writer()
)
session_wrapper: RetryableSession | None = None
try:
is_write_session = default_access_mode != neo4j.READ_ACCESS
session_wrapper = RetryableSession(
session_factory=lambda: driver.session(
default_access_mode=default_access_mode
),
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
retry_if=_is_retryable_write_error if is_write_session else None,
initial_retry_delay_seconds=(
NEPTUNE_WRITE_RETRY_DELAY_SECONDS if is_write_session else 0
),
)
yield session_wrapper
except neo4j.exceptions.Neo4jError as exc:
if (
default_access_mode == neo4j.READ_ACCESS
and exc.code
and exc.code in READ_EXCEPTION_CODES
):
raise WriteQueryNotAllowedException(
message="Read query not allowed", code=READ_EXCEPTION_CODES[0]
)
message = exc.message if exc.message is not None else str(exc)
if exc.code and exc.code.startswith(CLIENT_STATEMENT_EXCEPTION_PREFIX):
raise ClientStatementException(message=message, code=exc.code)
raise GraphDatabaseQueryException(message=message, code=exc.code)
finally:
if session_wrapper is not None:
session_wrapper.close()
# Operations
def execute_read_query(
self,
database: str, # noqa: ARG002 - ignored on Neptune
cypher: str,
parameters: dict[str, Any] | None = None,
) -> neo4j.graph.Graph:
with self.get_session(default_access_mode=neo4j.READ_ACCESS) as session:
def _run(tx: neo4j.ManagedTransaction) -> neo4j.graph.Graph:
result = tx.run(
cypher, parameters or {}, timeout=READ_QUERY_TIMEOUT_SECONDS
)
return result.graph()
return session.execute_read(_run)
def create_database(self, database: str) -> None: # noqa: ARG002
# Neptune clusters are single-database; there is nothing to create.
return None
def drop_database(self, database: str) -> None: # noqa: ARG002
# Neptune clusters are single-database; there is nothing to drop.
return None
def drop_subgraph(self, database: str, provider_id: str) -> int: # noqa: ARG002
"""Delete a provider's subgraph in two bounded phases.
Neptune write transactions are capped at ~2 minutes. A naive
`DETACH DELETE` on a label-scanned batch grows unbounded with graph
density (one node can drag thousands of relationships into the same
transaction). Instead:
1. Delete relationships incident to provider nodes, one fixed-size
batch per transaction.
2. Delete the now-orphaned nodes, one fixed-size batch per transaction.
Each transaction does work proportional to `batch_size`, never to the
graph's branching factor.
"""
from tasks.jobs.attack_paths.config import (
BATCH_SIZE,
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
provider_label = get_provider_label(provider_id)
deleted_relationships = 0
relationship_batches = 0
node_batches = 0
drop_t0 = time.perf_counter()
logger.info(
"Dropping provider graph from Neptune sink "
"(provider=%s, provider_label=%s)",
provider_id,
provider_label,
)
logger.info(
"Opening Neptune writer session for provider graph drop (provider=%s)",
provider_id,
)
with self.get_session() as session:
logger.info(
"Opened Neptune writer session for provider graph drop (provider=%s)",
provider_id,
)
for phase, query_template in RELATIONSHIP_DELETE_QUERY_TEMPLATES.items():
deleted_relationships, phase_batches = delete_batches(
session=session,
logger=logger,
log_target="Neptune sink",
provider_id=provider_id,
query=query_template.format(provider_label=provider_label),
phase=phase,
count_key="deleted_rels_count",
total_key="rels",
deleted_key="deleted_rels",
initial_total=deleted_relationships,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
relationship_batches += phase_batches
deleted_nodes, node_batches = delete_batches(
session=session,
logger=logger,
log_target="Neptune sink",
provider_id=provider_id,
query=NODE_DELETE_QUERY_TEMPLATE.format(
provider_label=provider_label,
provider_resource_label=PROVIDER_RESOURCE_LABEL,
),
phase="node",
count_key="deleted_nodes_count",
total_key="nodes",
deleted_key="deleted_nodes",
initial_total=0,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
logger.info(
"Finished dropping provider graph from Neptune sink "
"(provider=%s, relationship_batches=%s, deleted_rels=%s, "
"node_batches=%s, deleted_nodes=%s, elapsed=%.3fs)",
provider_id,
relationship_batches,
deleted_relationships,
node_batches,
deleted_nodes,
time.perf_counter() - drop_t0,
)
return deleted_nodes
def has_provider_data(self, database: str, provider_id: str) -> bool: # noqa: ARG002
from tasks.jobs.attack_paths.config import (
PROVIDER_RESOURCE_LABEL,
get_provider_label,
)
provider_label = get_provider_label(provider_id)
query = (
f"MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`) RETURN 1 LIMIT 1"
)
with self.get_session(default_access_mode=neo4j.READ_ACCESS) as session:
result = session.run(query)
return result.single() is not None
def clear_cache(self, database: str) -> None: # noqa: ARG002
# Neptune has no user-facing cache-clear procedure; no-op.
return None
# Sync write path
def ensure_sync_indexes(self, database: str) -> None: # noqa: ARG002
# Neptune routes node and relationship lookups through `~id`, which is the cluster's primary key
# No additional index is needed or supported
return None
def write_nodes(
self,
database: str, # noqa: ARG002
labels: str,
rows: list[dict[str, Any]],
) -> None:
if not rows:
return
from tasks.jobs.attack_paths.config import (
PROVIDER_ELEMENT_ID_PROPERTY,
PROVIDER_RESOURCE_LABEL,
)
# MERGE on `~id` is the documented and engine-optimized idempotent
# upsert pattern for Neptune openCypher. The label inside the MERGE
# matters: Neptune assigns a default `vertex` label to any node
# created without an explicit one, so we pin `_ProviderResource`
# (which every synced node carries anyway) at MERGE-time. Additional
# labels are added after
#
# We also write `_provider_element_id` as a regular property so
# non-sync code (drop_subgraph, query helpers) keeps a stable contract
# that doesn't know about `~id`
query = f"""
UNWIND $rows AS row
MERGE (n:`{PROVIDER_RESOURCE_LABEL}` {{`~id`: row.provider_element_id}})
SET n:{labels}
SET n += row.props
SET n.`{PROVIDER_ELEMENT_ID_PROPERTY}` = row.provider_element_id
"""
with self.get_session() as session:
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
def write_relationships(
self,
database: str, # noqa: ARG002
rel_type: str,
provider_id: str, # noqa: ARG002 - encoded in start/end `~id` already
rows: list[dict[str, Any]],
) -> None:
if not rows:
return
from tasks.jobs.attack_paths.config import PROVIDER_ELEMENT_ID_PROPERTY
# `id(n) = $value` is Neptune's parameterized fast path; both endpoint
# MATCHes resolve in O(1) via the system `~id`, so per-row work stays
# bounded regardless of batch size
query = f"""
UNWIND $rows AS row
MATCH (s) WHERE id(s) = row.start_element_id
MATCH (e) WHERE id(e) = row.end_element_id
MERGE (s)-[r:`{rel_type}` {{`{PROVIDER_ELEMENT_ID_PROPERTY}`: row.provider_element_id}}]->(e)
SET r += row.props
"""
with self.get_session() as session:
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
# Test helpers
def get_writer(self) -> neo4j.Driver:
return self._get_writer()
def get_reader(self) -> neo4j.Driver:
return self._get_reader()
# SigV4 auth provider
class _NeptuneAuthToken(neo4j.Auth):
"""Neo4j Auth backed by a SigV4-signed GET to `/opencypher`."""
def __init__(self, region: str, url: str) -> None:
session = BotoSession()
credentials = session.get_credentials()
if credentials is None:
raise RuntimeError(
"No AWS credentials available for Neptune SigV4 signing. "
"Ensure the boto3 credential chain can resolve."
)
credentials = credentials.get_frozen_credentials()
request = AWSRequest(method="GET", url=url + "/opencypher")
# SigV4 canonical Host must carry the real `host:port`
# Neptune runs on a non-default port (8182), so `.hostname` would drop it and break signing
request.headers.add_header("Host", urlsplit(url).netloc)
SigV4Auth(credentials, "neptune-db", region).add_auth(request)
auth_obj = {
header: request.headers[header]
for header in (
"Authorization",
"X-Amz-Date",
"X-Amz-Security-Token",
"Host",
)
if header in request.headers
}
auth_obj["HttpMethod"] = "GET"
super().__init__("basic", "username", json.dumps(auth_obj))
def neptune_auth_provider(region: str, https_url: str) -> Callable[[], ExpiringAuth]:
"""Return a callable the neo4j driver can invoke to refresh credentials."""
def _provider() -> ExpiringAuth:
token = _NeptuneAuthToken(region, https_url)
expires_at = (
datetime.datetime.now(datetime.UTC)
+ datetime.timedelta(minutes=SIGV4_TOKEN_LIFETIME_MINUTES)
).timestamp()
return ExpiringAuth(auth=token, expires_at=expires_at)
return _provider
@@ -1,12 +1,11 @@
import logging
from typing import Any, Iterable
from collections.abc import Iterable
from typing import Any
import neo4j
from rest_framework.exceptions import APIException, PermissionDenied, ValidationError
from api.attack_paths import database as graph_database, AttackPathsQueryDefinition
from api.attack_paths import AttackPathsQueryDefinition
from api.attack_paths import database as graph_database
from api.attack_paths import sink as sink_module
from api.attack_paths.cypher_sanitizer import (
inject_provider_label,
validate_custom_query,
@@ -16,7 +15,10 @@ from api.attack_paths.queries.schema import (
RAW_SCHEMA_URL,
get_cartography_schema_query,
)
from api.models import AttackPathsScan
from config.custom_logging import BackendLogger
from config.env import env
from rest_framework.exceptions import APIException, PermissionDenied, ValidationError
from tasks.jobs.attack_paths.config import (
INTERNAL_LABELS,
INTERNAL_PROPERTIES,
@@ -27,6 +29,10 @@ from tasks.jobs.attack_paths.config import (
logger = logging.getLogger(BackendLogger.API)
def _custom_query_timeout_ms() -> int:
return env.int("ATTACK_PATHS_READ_QUERY_TIMEOUT_SECONDS", default=30) * 1000
# Predefined query helpers
@@ -103,13 +109,13 @@ def execute_query(
definition: AttackPathsQueryDefinition,
parameters: dict[str, Any],
provider_id: str,
scan: AttackPathsScan,
) -> dict[str, Any]:
try:
graph = graph_database.execute_read_query(
database=database_name,
cypher=definition.cypher,
parameters=parameters,
)
# TODO: drop after Neptune cutover
# Route reads by the scan row's recorded sink, not by current settings.
backend = sink_module.get_backend_for_scan(scan)
graph = backend.execute_read_query(database_name, definition.cypher, parameters)
return _serialize_graph(graph, provider_id)
except graph_database.WriteQueryNotAllowedException:
@@ -143,22 +149,31 @@ def execute_custom_query(
database_name: str,
cypher: str,
provider_id: str,
scan: AttackPathsScan,
) -> dict[str, Any]:
# Defense-in-depth for custom queries:
# 1. neo4j.READ_ACCESS prevents mutations at the driver level
# 2. inject_provider_label() regex-based label injection scopes node patterns
# 3. _serialize_graph() post-query filter drops nodes without the provider label
# 1. `neo4j.READ_ACCESS` - prevents mutations at the driver level
# 2. `inject_provider_label()` - regex-based label injection scopes node patterns
# 3. `_serialize_graph()` - post-query filter drops nodes without the provider label
# 4. `USING QUERY:TIMEOUTMILLISECONDS` on Neptune - server-side runaway cutoff
#
# Layer 2 is best-effort (regex can't fully parse Cypher);
# layer 3 is the safety net that guarantees provider isolation.
validate_custom_query(cypher)
cypher = inject_provider_label(cypher, provider_id)
# TODO: drop after Neptune cutover
backend = sink_module.get_backend_for_scan(scan)
# Neptune enforces a cluster-level query timeout; prepending the hint
# makes the limit explicit and matches the client-side read timeout.
# Applies only when the scan's graph lives in Neptune.
if getattr(scan, "sink_backend", None) == "neptune":
timeout_ms = _custom_query_timeout_ms()
cypher = f"USING QUERY:TIMEOUTMILLISECONDS {timeout_ms}\n{cypher}"
try:
graph = graph_database.execute_read_query(
database=database_name,
cypher=cypher,
)
graph = backend.execute_read_query(database_name, cypher, None)
serialized = _serialize_graph(graph, provider_id)
return _truncate_graph(serialized)
@@ -181,10 +196,11 @@ def execute_custom_query(
def get_cartography_schema(
database_name: str, provider_id: str
database_name: str, provider_id: str, scan: AttackPathsScan
) -> dict[str, str] | None:
try:
with graph_database.get_session(
backend = sink_module.get_backend_for_scan(scan)
with backend.get_session(
database_name, default_access_mode=neo4j.READ_ACCESS
) as session:
result = session.run(get_cartography_schema_query(provider_id))

Some files were not shown because too many files have changed in this diff Show More