Compare commits

...
Author SHA1 Message Date
Hugo P.Brito ced87cbe44 chore(ui): add auth logo changelog fragment 2026-07-16 15:43:28 +01:00
Hugo P.Brito 73061a8859 fix(ui): restore auth header controls 2026-07-16 15:38:45 +01:00
Hugo P.Brito 0057d867f6 fix(ui): center auth logo 2026-07-16 14:50:14 +01:00
Hugo P.Brito 459902175d fix(ui): remove auth card theme toggle 2026-07-16 13:10:22 +01:00
Adrián Peña 641a11e4a0 fix(api): disable social account connection notifications (#12018) 2026-07-16 14:00:51 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 59b13778e2 fix(ui): show scan in Findings filter from View Findings action (#11997)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-16 12:03:55 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 7d8c64d35b fix(ui): allow rename and delete for dynamic providers (#11957)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-16 11:50:35 +02:00
Adrián Peña a022ad5c7a fix: harden Jira site handling (#12012) 2026-07-16 11:48:12 +02:00
Adrián Peña 0f31f1a3c2 fix(api): strengthen social account linking (#12013) 2026-07-16 11:48:06 +02:00
Alejandro Bailo 1ee71c1f20 feat(ui): redesign app sidebar navigation (#11994) 2026-07-16 11:40:08 +02:00
Josema Camacho 0dc424031b fix(api): apply long task limits to Attack Paths scans (#12009) 2026-07-16 10:54:23 +02:00
Prowler Botandprowler-bot 0d899b3076 chore(release): Bump versions to v5.35.0 (#12004)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-15 18:16:32 +02:00
Prowler Botandprowler-bot 3fd9fca32f chore(changelog): v5.34.0 (#11999)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-15 16:21:14 +02:00
Josema Camacho 07d7e9d5bb fix(api): retry wrapped Neptune transient errors (#11996) 2026-07-15 16:01:04 +02:00
Pepe Fagoaga 31b8ab9b4b docs(families): link to public announcement (#11995) 2026-07-15 15:59:08 +02:00
Pepe Fagoaga 60c4e9b257 docs(compliance): unify section pages (#11993) 2026-07-15 13:45:45 +02:00
Hugo Pereira Brito 8e9af708f8 feat(aws): add elbv2_listener_pqc_tls_enabled security check (#11254) 2026-07-15 12:45:23 +01:00
Deep ShahandDaniel Barranquero 24b670ac36 feat(sdk): add AWS Amplify app secret scanning check (#11825)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-15 13:44:20 +02:00
Pepe Fagoaga cc6c731af6 docs: new product family (#11984) 2026-07-15 12:28:50 +02:00
Daniel Barranqueroandalejandrobailo c53acd4184 fix(dashboard): correct UTM tags on local dashboard Prowler Cloud links (#11988)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-15 12:17:02 +02:00
Alejandro Bailo faa74f4c6c feat(ui): add Local Server Cloud upgrade prompts (#11982) 2026-07-15 12:11:28 +02:00
Josema Camacho 077dff7f68 fix(api): resolve critical container scan findings (#11991) 2026-07-15 12:08:52 +02:00
Josema Camacho a4752d6a27 fix(api): prevent false Attack Paths stale cleanup (#11986) 2026-07-15 11:07:21 +02:00
Toriola Opeyemi a9f6e04a84 docs: fix malformed height attribute in ECR badge (#11987) 2026-07-15 08:07:56 +02:00
Andoni Alonso d9224e682f docs(github): fix broken fine-grained PAT anchor in authentication table (#11985) 2026-07-14 17:44:06 +02:00
Daniel Barranquero 9822fd97d7 feat(dashboard): redesign local dashboard sidebar and pages (#11972) 2026-07-14 14:40:21 +02:00
Josema Camacho bd72ec91ea fix(api): combine permissions across assigned roles (#11979) 2026-07-14 13:52:12 +02:00
Haitao ZhengandHugo P.Brito fa9b2c707b feat(kubernetes): add hostPath volume check (#11837)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-14 12:23:59 +01:00
22401a54a0 feat(kubernetes): add core check for readonly root filesystem enabled (#11835)
Co-authored-by: wbro <80239840234@proton.me>
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-14 12:23:17 +01:00
Daniel Barranquero 9c15796c84 fix(ci): extend .trivyignore CVE suppression expiries to 2026-08-15 (#11975) 2026-07-14 12:26:33 +02:00
Prowler Botandprowler-bot db9356ba86 chore(changelog): v5.33.2 forward-sync to master (#11977)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-14 12:16:36 +02:00
Pepe Fagoaga 54237d824a chore(banner): missing feats and highlight component (#11974) 2026-07-14 12:15:13 +02:00
Pepe Fagoaga 7df1054966 chore(step-security): allow endpoints (#11973) 2026-07-14 12:02:35 +02:00
Josema Camacho 53772e2931 fix(api): prevent concurrent scan summary deadlocks (#11970) 2026-07-14 10:40:26 +02:00
Josema Camacho d37d5058bb fix(api): bound attack paths normalized-list child IDs (#11960) 2026-07-14 09:48:12 +02:00
Josema Camacho 8debf70d5c fix(api): retry transient attack paths graph mutations (#11961) 2026-07-14 09:45:52 +02:00
Pepe Fagoaga 1a1e804c00 chore(banner): highlight link to Prowler Cloud (#11964) 2026-07-14 08:46:52 +02:00
stepsecurity-app[bot]andstepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com> c7583a5633 feat(security): security best practices from StepSecurity (#11962)
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>
2026-07-14 08:17:06 +02:00
Adrián Peña 470e218bb8 fix(sdk): preserve regional IMDSv2 account findings (#11959) 2026-07-13 17:32:08 +02:00
Alejandro Bailo 488f3d1bed fix(ui): improve profile and role visibility UX (#11956) 2026-07-13 16:50:43 +02:00
Hugo Pereira Brito 0f6137dd04 fix(aws): target EC2 Amazon AMI loading (#11945) 2026-07-13 15:33:30 +01:00
Alejandro Bailo f5f6e5768d fix(ui): improve Lighthouse overview navigation (#11955) 2026-07-13 16:12:48 +02:00
Hugo Pereira Brito c6dae3a711 chore: remove __init__.py from test directories (#10582) 2026-07-13 15:09:42 +01:00
4242297e72 feat(aws): Update regions for AWS services (#11954)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-13 16:09:17 +02:00
8d4be0f586 feat(aws): Adding CF Template to do full org deployment (#10403)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-07-13 10:42:53 +02:00
Pedro MartínandPepe Fagoaga d78e0189e0 docs(compliance): add cross-provider feature docs (#11944)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-13 10:35:46 +02:00
Hugo Pereira Brito cef131812c fix(ci): correct bot and dependency PR labeling (#11425) 2026-07-13 09:06:12 +01:00
Pepe Fagoaga 3c78a0df43 fix(changelog): correct fragments for the UI (#11952) 2026-07-13 09:31:57 +02:00
mintlify[bot]andmintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> 991e8b8061 docs: brand tone and writing style fixes (#11953)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-07-13 09:14:24 +02:00
Alejandro Bailo dbdbd8a379 feat(ui): add cross-provider compliance view (#11912) 2026-07-10 17:34:53 +02:00
AmanandHugo P.Brito a0a0883578 feat(azure): add check to ensure Function Apps redirect HTTP to HTTPS (#11929)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-10 13:57:58 +01:00
Prowler Botandprowler-bot a4bf70eecf chore(changelog): v5.33.1 forward-sync to master (#11948)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-10 14:50:12 +02:00
Hugo Pereira Brito 5bc41b2609 docs: require duplicate PR check (#11946) 2026-07-10 12:59:49 +01:00
Pablo Fernandez Guerra (PFE)andPablo F.G 58fd4170b5 feat(ui): support dynamic providers (#11869)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-10 13:22:41 +02:00
GOUTHAM HARIGOVINDandDaniel Barranquero 3369e48260 feat(ec2): Implement AMI block public access check (#11794) (#11828)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-10 13:10:15 +02:00
Pepe Fagoaga 106026614d chore(security): allow internal endpoints for Lighthouse AI OpenAI compatible (#11941) 2026-07-10 12:55:16 +02:00
Adrián PeñaandJosema Camacho 3f2e5929d9 fix(api): harden Lighthouse provider base URLs (#11928)
Co-authored-by: Josema Camacho <josema@prowler.com>
2026-07-10 11:24:40 +02:00
Josema Camacho c93ea035fc fix(api): make AWS Attack Paths query aggregation Neo4j compatible (#11931) 2026-07-10 11:19:08 +02:00
César Arroba 847997672d fix(ui): reference renamed UI_POSTHOG_ENABLED flag in metronome billing guard (#11938) 2026-07-10 11:02:34 +02:00
lydiavilchez 08ee83c572 fix(docs): use python3 and add CI check for provider cards hook (#11930) 2026-07-10 09:27:34 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 892cc2bc07 refactor(ui): rename reserved billing flag to CLOUD_BILLING_ENABLED (#11920)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-10 09:26:33 +02:00
stepsecurity-app[bot]stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>Pepe Fagoaga
2a077cae5e feat(security): security best practices from StepSecurity (#11937)
Signed-off-by: StepSecurity Bot <bot@stepsecurity.io>
Co-authored-by: stepsecurity-app[bot] <188008098+stepsecurity-app[bot]@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-10 09:00:40 +02:00
1ee4de18be chore: add trailing newlines to 7 files for POSIX compliance (#11765)
Co-authored-by: Janderik Marins <janderik@email.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 18:19:55 +02:00
Josema CamachoandPepe Fagoaga b44f8ebf5f fix(api): add query-level retry with primary fallback to rls_transaction via execute_wrapper (#10379)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-09 17:47:03 +02:00
Hugo Pereira Brito 6f72785a28 fix(azure): warn on optional function app permission failures (#11922) 2026-07-09 16:23:09 +01:00
Hugo Pereira Brito be78fe8374 fix(jira): surface dispatch failures (#11918) 2026-07-09 16:09:12 +01:00
Pedro Martín 01c004a7af fix(ui): handle level 1 requirements for M365 CIS (#11921) 2026-07-09 16:41:51 +02:00
Yinka MetricsandDaniel Barranquero 0b36d08b92 feat(sdk): add Data Pipeline secret scanning check (#11821)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 16:21:05 +02:00
Josema Camacho bf9c53a4c3 fix(api): keep auth tests order-safe after token revocation (#11919) 2026-07-09 15:41:51 +02:00
Pablo Fernandez Guerra (PFE)andPablo F.G 80c5363649 refactor(ui): rename integration enable flags to past tense (#11917)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-09 13:09:35 +02:00
César Arroba 9d899ae0e2 ci: rename public ECR push-role secret to PUBLIC_ECR_PUSH_ROLE_ARN (#11916) 2026-07-09 11:27:55 +02:00
Narahari RaghavaandDaniel Barranquero 13bd6fc0bf fix(aws): check statement Effect instead of policy Statement in SCP a… (#11727)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-09 11:19:33 +02:00
Adrián Peña 18b3c49234 fix(api): invalidate tokens after password updates (#11901) 2026-07-09 10:36:58 +02:00
Adrián Peña 84ea68927f fix(api): scope user role updates to tenant (#11903) 2026-07-09 10:36:49 +02:00
Johannes EnglerandHugo P.Brito 1af9cdd351 feat(stackit): add iaas_server_public_ip_attached check (#11549)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-08 17:13:36 +01:00
Hugo Pereira Brito 25bcbac309 fix(dms): lazy load ec2 for public access check (#11899) 2026-07-08 16:42:23 +01:00
Hugo Pereira Brito 20aad80a78 fix(aws): avoid full ec2 inventory in dlm check (#11850) 2026-07-08 15:59:16 +01:00
Alejandro Bailo 52875b5c7c feat(ui): migrate from HeroUI to shadcn/ui (#11532) 2026-07-08 16:57:02 +02:00
Pepe Fagoaga c665005790 fix(workflow): bump AI issue triage (#11896) 2026-07-08 15:45:52 +02:00
mintlify[bot]andmintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com> b2532ebfe5 docs(style): standardize "click" interaction verb (#11893)
Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>
2026-07-08 14:39:29 +02:00
Josema Camacho 3461f9ac49 test(api): reduce report, provider, auth, and sentry test costs (#11888) 2026-07-08 14:31:53 +02:00
UniCodeandDaniel Barranquero d874fc573d feat(e2e): provider for e2e cloud (#11654)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-08 14:21:27 +02:00
Hugo Pereira Brito 991c204a88 fix(sdk): limit ECS task definitions by registration date (#11868) 2026-07-08 13:09:25 +01:00
Hugo Pereira BritoandPablo F.G eee17f0e8b fix(ui): clarify Unlimited Visibility in RBAC forms (#11851)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
2026-07-08 12:42:18 +01:00
Hugo Pereira Brito 80608bfdbc feat(oraclecloud): support regionless SDK setup (#11853) 2026-07-08 12:16:07 +01:00
Adrián Peña 6c5f54808d feat: add changelog fragments workflow (#11572) 2026-07-08 10:19:12 +02:00
Pepe Fagoaga bb6608c1d7 chore(ui): unify trial expired banner (#11713) 2026-07-08 10:05:36 +02:00
Prowler Botandprowler-bot 18e1bf5195 chore(release): Bump versions to v5.34.0 (#11874)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-07 18:23:01 +02:00
César Arroba 76a2d7bfe6 ci: skip codeql and e2e on changelog-only changes (#11867) 2026-07-07 14:27:41 +02:00
ac3f289de6 feat(ui): gate Sentry, GTM and PostHog behind runtime enable flags (#11682)
Co-authored-by: Pablo F.G <pablo.fernandez@prowler.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 13:43:34 +02:00
Rubén De la Torre Vico 838f82b255 docs(lighthouse): document Lighthouse AI on Prowler Cloud (#11758) 2026-07-07 13:27:37 +02:00
Josema Camacho 5e00c4bfcf chore(docs): update changelogs for 5.33.0 (#11866) 2026-07-07 13:07:40 +02:00
lydiavilchez f0ae56b8ea feat(docs): auto-generate provider cards in Prowler App tutorial (#11865) 2026-07-07 12:35:52 +02:00
Hugo Pereira BritoandDaniel Barranquero 4484c2f192 fix(azure): refine resource group scoped scan follow-ups (#11796)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 11:20:38 +02:00
Pedro Martín e6bbcb8043 chore(scan-config): improve wording and docs (#11859) 2026-07-07 11:14:19 +02:00
Hugo Pereira Brito ad04e69c35 fix(kubernetes): reject exec auth in cloud kubeconfigs (#11753) 2026-07-07 09:57:46 +01:00
Adrián Peña 6cae37174c fix(api): queue provider scans when one is active (#11848) 2026-07-07 10:28:41 +02:00
Pepe Fagoaga aa6de57430 chore(multi-tenant): delete last tenant from profile page (#11864) 2026-07-07 10:02:36 +02:00
Alejandro Bailo a48aa37f87 fix(ui): Lighthouse chat archive navigation, new chat button, and masked stored credentials (#11860) 2026-07-07 09:57:47 +02:00
Pedro Martín 3cc8f86780 feat(api): remove provider credentials for PDF report (#11845) 2026-07-07 09:45:34 +02:00
AbhinavandDaniel Barranquero 4cb02a0ead fix(azure): read Flexible Server log retention from logfiles.retention_days (fixes #11757) (#11761)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-07 08:19:44 +02:00
Alejandro Bailo 221c558cee feat(ui): connect Lighthouse v2 to Cloud backend (#11690) 2026-07-06 17:30:31 +02:00
Pedro MartínandCésar Arroba 81c3152ebb chore(changelog): prepare for 5.32.1 (#11856)
Co-authored-by: César Arroba <19954079+cesararroba@users.noreply.github.com>
2026-07-06 16:14:20 +02:00
Pedro Martín 855e9a043e chore(changelog): prepare for 5.32.1 (#11854) 2026-07-06 16:01:31 +02:00
Adrián Peñaandalejandrobailo 7b5d724bb7 fix: handle invitations in social and SAML auth (#11752)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-06 14:49:07 +02:00
Alejandro Bailo 441f2a3c48 fix(ui): enable triage editing in compliance findings table (#11829) 2026-07-06 11:03:37 +02:00
Hugo Pereira Brito efb86bb7ab chore: remove Dependabot config (#11834) 2026-07-03 12:48:17 +01:00
Hugo Pereira Brito 398a0a484f fix(api): restrict user profile updates to self (#11792) 2026-07-03 11:53:01 +01:00
Sanjay Santhanamandpedrooot 55924d8150 fix(compliance): skip MANUAL findings in section tally to avoid KeyError (#11823)
Co-authored-by: pedrooot <pedromarting3@gmail.com>
2026-07-03 12:19:32 +02:00
Josema Camacho 0cf6f2f83e fix(api): add attack paths scan DB defaults (#11826) 2026-07-03 11:30:55 +02:00
Pepe Fagoaga cf18093261 chore(onboarding): rename account -> provider (#11827) 2026-07-03 09:33:37 +02:00
Josema Camacho 1850e209e6 docs(attack-paths): update tutorial for the new UI (#11824) 2026-07-02 18:34:49 +02:00
Prowler Botandprowler-bot 2e37188c9f chore(release): Bump versions to v5.33.0 (#11820)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-07-02 16:56:03 +02:00
Josema Camacho 4ae7c67d3f docs: add findings triage guide (#11793) 2026-07-02 13:33:30 +02:00
Pepe Fagoaga a76ba156d5 chore(changelog): v5.32.0 (#11805) 2026-07-02 13:17:56 +02:00
Alejandro Bailo cd90a91158 fix(ui): address findings triage QA feedback (#11791) 2026-07-02 12:43:06 +02:00
Son Sulung Suryahatta AsnanandDaniel Barranquero e1b23e2526 feat(s3): new check s3_bucket_object_public to detect public objects (#9517)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-02 11:41:29 +02:00
Legin 537c3ea71e feat(azure): filtering scans at resource group level (#10657)
Signed-off-by: Legin-ML <leginml2004@gmail.com>
2026-07-02 10:27:53 +01:00
Maringanti Vasist AcharyaandHugo P.Brito b6f74c7284 feat(m365): add exchange application access policy check (#11247)
Co-authored-by: Hugo P.Brito <hugopbrit@gmail.com>
2026-07-02 10:16:07 +01:00
César Arroba 8cd008ba91 chore(ci): enable Renovate routine version updates (#11789) 2026-07-02 10:34:54 +02:00
César Arroba 1f13e1d348 chore(ci): disable Dependabot version updates in favor of Renovate (#11788) 2026-07-02 09:30:32 +02:00
Josema Camacho 87a15d7bb8 feat(api): support timestamp precision in findings filters (#11754) 2026-07-02 09:27:22 +02:00
renovate[bot]andrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com> f5bdacd07a chore(docker): update docker (#11601)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-07-02 09:08:50 +02:00
renovate[bot]andrenovate[bot] <29139614+renovate[bot]@users.noreply.github.com> ce80fcd430 chore(ci): update github-actions (#11314)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2026-07-02 09:08:16 +02:00
Pedro Martín c1c080b072 docs(scan-config): add scan config for CLI as reference (#11786) 2026-07-02 08:59:18 +02:00
Alan Buscagliaandalejandrobailo 587187419f feat(ui): add findings triage (#11704)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-01 17:55:33 +02:00
César Arroba 050a5915ca fix(ci): detect conflict markers in route-group paths and flag unmergeable PRs (#11763) 2026-07-01 17:50:33 +02:00
Alejandro Bailo d4e4d12c5a docs(skills): add design system discipline to prowler-ui (#11764) 2026-07-01 17:31:07 +02:00
Josema Camacho 72cf2a65a6 perf(api): optimize attack paths graph cleanup (#11755) 2026-07-01 16:28:59 +02:00
Pedro MartínandPepe Fagoaga 48db27481d docs(scan-config): add info about attaching scan config (#11751)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-07-01 16:06:38 +02:00
Adrián Peña 1247c5fb33 docs: clarify SAML userType role mapping (#11759) 2026-07-01 15:48:46 +02:00
Pedro Martínandalejandrobailo 69321418a3 feat(ui): improve scan config ux (#11731)
Co-authored-by: alejandrobailo <alejandrobailo94@gmail.com>
2026-07-01 15:45:38 +02:00
Pepe Fagoaga 301d13a4b9 docs: unify subscription banner (#11756) 2026-07-01 15:12:45 +02:00
Chirag TrivediandDaniel Barranquero 3f8c1e822f feat(apigateway): add check for secrets in REST API stage variables (#11188)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 14:24:18 +02:00
Sergio Garcia 1e1c1c018b feat(iam): add AWS Bedrock AgentCore privilege escalation paths (#11726) 2026-07-01 12:49:34 +02:00
琴焚音落andDaniel Barranquero a212916a49 fix(github): skip archived repos in CODEOWNERS check (#11735)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 12:29:13 +02:00
Samyak ChoudharyandDaniel Barranquero 883ffa1fdb feat(m365): add Entra Conditional Access group management restriction (#11342)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 11:38:54 +02:00
Daniel Barranquero 21d9d6192e feat(okta): add configurable API request throttling and rate-limit retries (#11702) 2026-07-01 10:30:43 +02:00
Hugo Pereira BritoandDaniel Barranquero fd38a0ac03 feat(awslambda): add secrets_ignore_files to skip false-positive files (e.g. *.deps.json) in no-secrets-in-code check (#11222)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2026-07-01 09:03:25 +01:00
César Arroba af6918d57b chore(api): opt out of PowerShell telemetry in the API image (#11746) 2026-07-01 10:02:59 +02:00
César Arroba 9a9cbc997b ci(workflows): push SDK image to Public ECR via OIDC (#11749) 2026-06-30 21:17:57 +02:00
César Arroba aec500ee3b fix(ci): harden pull_request_target workflows (persist-credentials + toJson) (#11747) 2026-06-30 18:29:29 +02:00
César Arroba 8fbc721223 ci(workflows): allowlist nodejs.org and Iconify endpoints in UI tests harden-runner (#11744) 2026-06-30 17:43:02 +02:00
Josema Camacho c3ce3d2b3c fix(api): preflight attack paths graph databases (#11743) 2026-06-30 17:01:48 +02:00
Hugo Pereira Brito c46cbaaa4a feat(sdk): limit selected high-volume AWS resource analysis (#11228) 2026-06-30 15:49:12 +01:00
Daniel Barranquero 34e8e3ca61 chore(api): replace detect-secrets with kingfisher-bin to match the SDK (#11698) 2026-06-30 15:59:18 +02:00
Daniel Barranquero 5dac8a0a53 feat(sdk): replace detect-secrets library with kingfisher (#11694) 2026-06-30 15:36:23 +02:00
ed1fec8866 docs: add scan scheduling guide (#11729)
Co-authored-by: Adrián Jesús Peña Rodríguez <adrianjpr@gmail.com>
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2026-06-30 15:07:08 +02:00
Daniel Barranquero 2abcb05e22 feat(github): support repository rulesets in default branch protection checks (#11723) 2026-06-30 13:53:32 +01:00
César Arroba aba43440ca chore(ci): remove legacy prowler-api/ui/mcp cloud-deployment dispatch jobs (#11644) 2026-06-30 12:53:51 +02:00
Prowler Botandprowler-bot d47cbb4f8c feat(aws): Update regions for AWS services (#11662)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2026-06-30 12:46:29 +02:00
Hugo Pereira Brito 36006de8ce chore(ci): bump Trivy scanner to v0.71.2 (#11728) 2026-06-30 11:45:55 +01:00
Pedro Martín 5ccb044b85 docs(scan-config): add new changes (#11732) 2026-06-30 12:34:26 +02:00
Hugo Pereira Brito e40e9a6483 fix(docker): remove build dependencies from images (#11730) 2026-06-30 11:31:51 +01:00
Pedro Martín 717d48b0e0 chore(config): add missing providers config (#11725) 2026-06-30 11:32:47 +02:00
Josema Camacho 4e7e2f7eab fix(api): cap attack paths sink sync batches (#11724) 2026-06-29 17:48:02 +02:00
Pedro Martín 5404863a3e feat(compliance): add CIS 2.0.1 for K8S (#11722) 2026-06-29 16:10:13 +02:00
Pedro MartínandHugo Pereira Brito 9d4b6c4d16 feat(compliance): add CIS 1.2 for github (#11719)
Co-authored-by: Hugo Pereira Brito <101209179+HugoPBrito@users.noreply.github.com>
2026-06-29 15:54:30 +02:00
1838 changed files with 85982 additions and 17326 deletions
+7 -6
View File
@@ -145,19 +145,20 @@ DJANGO_BROKER_VISIBILITY_TIMEOUT=86400
DJANGO_SENTRY_DSN=
DJANGO_THROTTLE_TOKEN_OBTAIN=50/minute
# Sentry for the web app (server + browser). Empty/unset UI_SENTRY_DSN ⇒
# Sentry disabled, zero egress. SENTRY_RELEASE (unprefixed) feeds the web app's
# server/edge SDKs.
# Sentry for the web app (server + browser). The UI_SENTRY_* values load only
# when UI_SENTRY_ENABLED="true"; without it they are ignored (default off, zero
# egress). The deprecated NEXT_PUBLIC_SENTRY_DSN still activates Sentry without
# the flag. SENTRY_RELEASE (unprefixed) feeds the web app's server/edge SDKs.
UI_SENTRY_DSN=
UI_SENTRY_ENVIRONMENT=local
SENTRY_RELEASE=local
# Reserved runtime public config (registered now; no UI consumer yet)
# POSTHOG_KEY=
# POSTHOG_HOST=
# UI_POSTHOG_KEY=
# UI_POSTHOG_HOST=
# REO_DEV_CLIENT_ID=
#### Prowler release version ####
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.32.0
NEXT_PUBLIC_PROWLER_RELEASE_VERSION=v5.35.0
# Social login credentials
SOCIAL_GOOGLE_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/google"
+2 -2
View File
@@ -63,7 +63,7 @@ runs:
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
version: 'v0.71.0'
version: 'v0.71.2'
- name: Run Trivy vulnerability scan (SARIF)
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
@@ -76,7 +76,7 @@ runs:
exit-code: '0'
scanners: 'vuln'
timeout: '5m'
version: 'v0.71.0'
version: 'v0.71.2'
- name: Upload Trivy results to GitHub Security tab
if: inputs.upload-sarif == 'true' && github.event_name == 'push'
+10
View File
@@ -5,10 +5,20 @@
"version": "v8",
"sha": "ed597411d8f924073f98dfc5c65a23a2325f34cd"
},
"github/gh-aw-actions/setup@v0.81.6": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.81.6",
"sha": "ba6380cc6e5be5d21677bebe04d52fb48e3abec7"
},
"github/gh-aw/actions/setup@v0.43.23": {
"repo": "github/gh-aw/actions/setup",
"version": "v0.43.23",
"sha": "9382be3ca9ac18917e111a99d4e6bbff58d0dccc"
},
"step-security/harden-runner@v2.20.0": {
"repo": "step-security/harden-runner",
"version": "v2.20.0",
"sha": "bf7454d06d71f1098171f2acdf0cd4708d7b5920"
}
}
}
-138
View File
@@ -1,138 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
# v5
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "pip"
# directory: "/api"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "pip"
# - "component/api"
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 25
target-branch: master
labels:
- "dependencies"
- "github_actions"
cooldown:
default-days: 7
# Dependabot Updates are temporary disabled - 2025/03/19
# - package-ecosystem: "npm"
# directory: "/ui"
# schedule:
# interval: "daily"
# open-pull-requests-limit: 10
# target-branch: master
# labels:
# - "dependencies"
# - "npm"
# - "component/ui"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "monthly"
open-pull-requests-limit: 25
target-branch: master
labels:
- "dependencies"
- "docker"
cooldown:
default-days: 7
# - package-ecosystem: "pre-commit"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 25
# target-branch: master
# labels:
# - "dependencies"
# - "pre-commit"
# cooldown:
# default-days: 7
# Dependabot Updates are temporary disabled - 2025/04/15
# v4.6
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "pip"
# - "v4"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "github_actions"
# - "v4"
# - package-ecosystem: "docker"
# directory: "/"
# schedule:
# interval: "weekly"
# open-pull-requests-limit: 10
# target-branch: v4.6
# labels:
# - "dependencies"
# - "docker"
# - "v4"
# Dependabot Updates are temporary disabled - 2025/03/19
# v3
# - package-ecosystem: "pip"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "pip"
# - "v3"
# - package-ecosystem: "github-actions"
# directory: "/"
# schedule:
# interval: "monthly"
# open-pull-requests-limit: 10
# target-branch: v3
# labels:
# - "dependencies"
# - "github_actions"
# - "v3"
+11 -6
View File
@@ -18,8 +18,9 @@ Please add a detailed description of how to review this PR.
<summary><b>Community Checklist</b></summary>
- [ ] This feature/issue is listed in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the issue/feature in [here](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](goto.prowler.com/slack)
- [ ] This feature/issue is listed in the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or roadmap.prowler.com
- [ ] Is it assigned to me, if not, request it via the [open issues](https://github.com/prowler-cloud/prowler/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen) or [Prowler Community Slack](https://goto.prowler.com/slack)
- [ ] I have reviewed the [open pull requests](https://github.com/prowler-cloud/prowler/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen) and confirmed there is no existing PR that implements the same outcome
</details>
@@ -27,8 +28,8 @@ Please add a detailed description of how to review this PR.
- [ ] Review if the code is being covered by tests.
- [ ] Review if code is being documented following this specification https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings
- [ ] Review if backport is needed.
- [ ] Review if is needed to change the [Readme.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/prowler/CHANGELOG.md), if applicable.
- [ ] Review if is needed to change the [README.md](https://github.com/prowler-cloud/prowler/blob/master/README.md)
- [ ] Ensure a changelog fragment is added under [prowler/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/prowler/changelog.d), if applicable.
#### SDK/CLI
- Are there new checks included in this PR? Yes / No
@@ -40,7 +41,7 @@ Please add a detailed description of how to review this PR.
- [ ] Screenshots/Video of the functionality flow (if applicable) - Mobile (X < 640px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Table (640px > X < 1024px)
- [ ] Screenshots/Video of the functionality flow (if applicable) - Desktop (X > 1024px)
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/ui/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [ui/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/ui/changelog.d), if applicable.
#### API
- [ ] All issue/task requirements work as expected on the API
@@ -50,7 +51,11 @@ Please add a detailed description of how to review this PR.
- [ ] Any other relevant evidence of the implementation (if applicable)
- [ ] Verify if API specs need to be regenerated.
- [ ] Check if version updates are required (e.g., specs, uv, etc.).
- [ ] Ensure new entries are added to [CHANGELOG.md](https://github.com/prowler-cloud/prowler/blob/master/api/CHANGELOG.md), if applicable.
- [ ] Ensure a changelog fragment is added under [api/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/api/changelog.d), if applicable.
#### MCP Server
- [ ] All issue/task requirements work as expected on the MCP Server
- [ ] Ensure a changelog fragment is added under [mcp_server/changelog.d/](https://github.com/prowler-cloud/prowler/tree/master/mcp_server/changelog.d), if applicable.
### License
+13 -2
View File
@@ -23,6 +23,10 @@
"prConcurrentLimit": 20,
"prHourlyLimit": 10,
"vulnerabilityAlerts": {
"labels": [
"dependencies",
"security"
],
"prHourlyLimit": 0,
"prConcurrentLimit": 0
},
@@ -38,7 +42,7 @@
"schedule": [
"* 22-23,0-5 1 * *"
],
"enabled": false
"enabled": true
},
{
"description": "Minors: 8th of every 3 months, Madrid overnight window (22:00-06:00)",
@@ -48,7 +52,7 @@
"schedule": [
"* 22-23,0-5 8 */3 *"
],
"enabled": false
"enabled": true
},
{
"description": "Majors: 15th of every 3 months, Madrid overnight window",
@@ -58,6 +62,13 @@
"schedule": [
"* 22-23,0-5 15 */3 *"
],
"enabled": true
},
{
"description": "gh-aw compiled lock files - generated by 'gh aw compile', action pins must match the compiler version, never bump directly",
"matchFileNames": [
".github/workflows/*.lock.yml"
],
"enabled": false
},
{
+154
View File
@@ -0,0 +1,154 @@
#!/usr/bin/env python3
"""Rename changelog fragments to their PR number before running towncrier.
For every <slug>.<type>.md in <component_dir>/changelog.d/, find the commit that
added it, resolve its PR via the GitHub API (falling back to the squash-commit
subject), and `git mv` it to <PR>.<type>.md so towncrier renders the PR link.
Unresolvable fragments become +<slug>.<type>.md orphans (rendered without link).
"""
import argparse
import json
import os
import re
import subprocess
import sys
import urllib.error
import urllib.request
FRAGMENT_RE = re.compile(
r"^(?P<slug>[A-Za-z0-9][A-Za-z0-9._-]*?)"
r"\.(?P<type>added|changed|deprecated|removed|fixed|security)"
r"(?:\.(?P<counter>[0-9]+))?\.md$"
)
SUBJECT_PR_RE = re.compile(r" \(#([0-9]+)\)$")
IGNORED_FILES = {".gitkeep", "README.md"}
API_TIMEOUT_SECONDS = 10
def git(*args: str) -> str:
result = subprocess.run(["git", *args], check=True, capture_output=True, text=True)
return result.stdout.strip()
def find_adding_commit(path: str) -> str | None:
"""Find the commit that added a file, following renames.
Falls back to a plain (no --follow) lookup: rename detection can lose the
add event for degenerate content (e.g. files identical to many others).
"""
sha = git("log", "--follow", "--diff-filter=A", "--format=%H", "-1", "--", path)
if not sha:
sha = git("log", "--diff-filter=A", "--format=%H", "-1", "--", path)
return sha or None
def pr_from_api(repo: str, sha: str) -> int | None:
"""Resolve the PR associated with a commit via the GitHub API.
Returns None on any network/API failure so the caller can fall back to
parsing the squash-commit subject.
"""
url = f"https://api.github.com/repos/{repo}/commits/{sha}/pulls"
headers = {
"Accept": "application/vnd.github+json",
"X-GitHub-Api-Version": "2022-11-28",
"User-Agent": "prowler-changelog-attribution",
}
token = os.environ.get("GITHUB_TOKEN")
if token:
headers["Authorization"] = f"Bearer {token}"
request = urllib.request.Request(url, headers=headers)
try:
with urllib.request.urlopen(request, timeout=API_TIMEOUT_SECONDS) as response:
pulls = json.load(response)
except (urllib.error.URLError, TimeoutError, json.JSONDecodeError):
return None
if isinstance(pulls, list) and pulls:
return pulls[0].get("number")
return None
def pr_from_subject(sha: str) -> int | None:
subject = git("log", "-1", "--format=%s", sha)
match = SUBJECT_PR_RE.search(subject)
return int(match.group(1)) if match else None
def unique_destination(directory: str, base_name: str, fragment_type: str) -> str:
"""Return a non-colliding fragment path, appending a numeric counter if needed."""
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.md")
counter = 0
while os.path.exists(candidate):
counter += 1
candidate = os.path.join(directory, f"{base_name}.{fragment_type}.{counter}.md")
return candidate
def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("component_dir", help="Component directory, e.g. prowler")
parser.add_argument("--repo", default="prowler-cloud/prowler")
parser.add_argument(
"--no-api",
action="store_true",
help="Skip the GitHub API and resolve PRs from commit subjects only",
)
args = parser.parse_args()
fragments_dir = os.path.join(args.component_dir, "changelog.d")
if not os.path.isdir(fragments_dir):
print(f"::error::Fragments directory not found: {fragments_dir}")
return 1
malformed = []
to_process = []
for name in sorted(os.listdir(fragments_dir)):
if name in IGNORED_FILES or name.startswith("+"):
continue
match = FRAGMENT_RE.match(name)
if not match:
malformed.append(name)
continue
if match.group("slug").isdigit():
continue
to_process.append((name, match))
if malformed:
for name in malformed:
print(
f"::error::Malformed fragment filename in {fragments_dir}: {name} "
"(expected <slug>.<type>.md with type one of added|changed|"
"deprecated|removed|fixed|security)"
)
return 1
for name, match in to_process:
slug, fragment_type = match.group("slug"), match.group("type")
path = os.path.join(fragments_dir, name)
sha = find_adding_commit(path)
pr_number = None
if sha:
if not args.no_api:
pr_number = pr_from_api(args.repo, sha)
if pr_number is None:
pr_number = pr_from_subject(sha)
if pr_number is not None:
destination = unique_destination(
fragments_dir, str(pr_number), fragment_type
)
else:
destination = unique_destination(fragments_dir, f"+{slug}", fragment_type)
print(
f"::warning::Could not resolve a PR for {path}; renamed to "
f"{os.path.basename(destination)} (entry will render without a PR link)"
)
git("mv", path, destination)
print(f"{path} -> {destination}")
return 0
if __name__ == "__main__":
sys.exit(main())
+13
View File
@@ -0,0 +1,13 @@
{% set category_order = definitions.keys() %}
{% for section, _ in sections.items() %}
{% for category in category_order if category in sections[section] %}
### {{ definitions[category]['name'] }}
{% for text, values in sections[section][category].items() -%}
- {{ text }}{% if values %} {{ values|join(', ') }}{% endif %}{{ "\n" }}
{%- endfor %}
{% endfor %}
{% endfor %}
---
{{ "\n" }}
+2 -1
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -62,6 +62,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+3 -1
View File
@@ -9,6 +9,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
pull_request:
branches:
- 'master'
@@ -17,6 +18,7 @@ on:
- 'api/**'
- '.github/workflows/api-codeql.yml'
- '.github/codeql/api-codeql-config.yml'
- '!api/CHANGELOG.md'
schedule:
- cron: '00 12 * * *'
@@ -44,7 +46,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -30
View File
@@ -46,7 +46,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
@@ -65,7 +65,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -108,7 +108,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -175,7 +175,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -215,7 +215,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -236,7 +236,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -272,27 +272,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger API deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: api-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+5 -2
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -69,7 +69,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -92,6 +92,8 @@ jobs:
_http._tcp.deb.debian.org:443
powershellinfraartifacts-gkhedzdeaghdezhr.z01.azurefd.net:443
get.trivy.dev:443
raw.githubusercontent.com:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -108,6 +110,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Set up Docker Buildx
+2 -1
View File
@@ -43,7 +43,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -77,6 +77,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+4 -3
View File
@@ -48,7 +48,7 @@ jobs:
services:
postgres:
image: postgres:17@sha256:2cd82735a36356842d5eb1ef80db3ae8f1154172f0f653db48fde079b2a0b7f7
image: postgres:17@sha256:5c855ad7b85e68e48a62f34662853f38b57c1c1d80f3a927ab58034fd6d31c5e
env:
POSTGRES_HOST: ${{ env.POSTGRES_HOST }}
POSTGRES_PORT: ${{ env.POSTGRES_PORT }}
@@ -63,7 +63,7 @@ jobs:
--health-timeout 5s
--health-retries 5
valkey:
image: valkey/valkey:7-alpine3.19
image: valkey/valkey:7-alpine3.19@sha256:4054fe7fc607b9326ac7c4691ed26e9670d2ff17a9fb28c2577adecf928acbcc
env:
VALKEY_HOST: ${{ env.VALKEY_HOST }}
VALKEY_PORT: ${{ env.VALKEY_PORT }}
@@ -78,7 +78,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -111,6 +111,7 @@ jobs:
api/docs/**
api/README.md
api/CHANGELOG.md
api/changelog.d/**
api/AGENTS.md
- name: Setup Python with uv
+1 -1
View File
@@ -30,7 +30,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+4 -4
View File
@@ -29,7 +29,7 @@ jobs:
patch_version: ${{ steps.detect.outputs.patch_version }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -75,7 +75,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -202,7 +202,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -307,7 +307,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -0,0 +1,35 @@
name: 'Tools: Check Test Init Files'
on:
pull_request:
branches:
- 'master'
- 'v5.*'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: {}
jobs:
check-test-init-files:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@fa2e9d605c4eeb9fcad4c99c224cee0c6c7f3594 # v2.16.0
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Check for __init__.py files in test directories
run: python3 scripts/check_test_init_files.py .
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+527
View File
@@ -0,0 +1,527 @@
name: 'Tools: Compile Changelogs'
run-name: 'Compile changelogs for Prowler ${{ inputs.prowler_version }}'
on:
workflow_dispatch:
inputs:
prowler_version:
description: 'Prowler version being released (e.g., 5.31.0)'
required: true
type: string
target_branch:
description: 'Branch to compile on (master for minor releases, v5.X for patches)'
required: true
type: string
sdk_version:
description: 'SDK version override (empty = mirrors prowler_version; "skip" = hold this component back)'
required: false
type: string
api_version:
description: 'API version override (empty = auto-derive 1.<prowler_minor + 1>.<prowler_patch>; "skip" = hold back)'
required: false
type: string
ui_version:
description: 'UI version override (empty = auto-derive 1.<prowler_minor>.<prowler_patch>; "skip" = hold back)'
required: false
type: string
mcp_version:
description: 'MCP Server version override (empty = auto-derive from pending fragment types; "skip" = hold back)'
required: false
type: string
concurrency:
group: ${{ github.workflow }}-${{ inputs.prowler_version }}
cancel-in-progress: false
env:
PROWLER_VERSION: ${{ inputs.prowler_version }}
TARGET_BRANCH: ${{ inputs.target_branch }}
SDK_VERSION: ${{ inputs.sdk_version }}
API_VERSION: ${{ inputs.api_version }}
UI_VERSION: ${{ inputs.ui_version }}
MCP_VERSION: ${{ inputs.mcp_version }}
permissions: {}
jobs:
compile-changelogs:
if: github.event_name == 'workflow_dispatch' && github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: write
pull-requests: write
steps:
- name: Harden the runner (Block outbound calls)
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ inputs.target_branch }}
fetch-depth: 0 # PR attribution resolves each fragment's adding commit from history
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
persist-credentials: false
- name: Set up Python
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Install towncrier
run: pip install --no-cache-dir towncrier==25.8.0
- name: Configure Git
run: |
git config --global user.name 'prowler-bot'
git config --global user.email '179230569+prowler-bot@users.noreply.github.com'
- name: Validate version inputs
run: |
if [[ ! "$PROWLER_VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid prowler_version syntax: '$PROWLER_VERSION' (must be N.N.N)"
exit 1
fi
if [ "$TARGET_BRANCH" != "master" ] && [[ ! "$TARGET_BRANCH" =~ ^v[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid target_branch syntax: '$TARGET_BRANCH' (must be 'master' or vN.N, e.g. v5.31)"
exit 1
fi
IFS=. read -r prowler_major prowler_minor prowler_patch <<< "$PROWLER_VERSION"
prowler_major=$((10#$prowler_major))
prowler_minor=$((10#$prowler_minor))
prowler_patch=$((10#$prowler_patch))
if [ "$prowler_patch" -eq 0 ]; then
if [ "$TARGET_BRANCH" != "master" ]; then
echo "::error::target_branch must be 'master' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
else
expected_target_branch="v${prowler_major}.${prowler_minor}"
if [ "$TARGET_BRANCH" != "$expected_target_branch" ]; then
echo "::error::target_branch must be '${expected_target_branch}' for Prowler ${PROWLER_VERSION}; got '${TARGET_BRANCH}'"
exit 1
fi
fi
for pair in "sdk_version:$SDK_VERSION" "api_version:$API_VERSION" "ui_version:$UI_VERSION" "mcp_version:$MCP_VERSION"; do
input_name="${pair%%:*}"
input_value="${pair#*:}"
if [ -n "$input_value" ] && [ "$input_value" != "skip" ] && [[ ! "$input_value" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "::error::Invalid $input_name syntax: '$input_value' (must be N.N.N, empty for auto-derivation, or 'skip')"
exit 1
fi
done
- name: Compile changelogs
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
component_input() {
case "$1" in
prowler) echo "$SDK_VERSION" ;;
api) echo "$API_VERSION" ;;
ui) echo "$UI_VERSION" ;;
mcp_server) echo "$MCP_VERSION" ;;
esac
}
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
pending_fragments() {
find "$1/changelog.d" -maxdepth 1 -type f ! -name '.gitkeep' ! -name 'README.md' | sort
}
# The component's last released version is the first stamped heading
# of its CHANGELOG.md, the same source prepare-release.yml greps.
latest_released_version() {
grep -m1 -E '^## \[v?[0-9]+\.[0-9]+\.[0-9]+\]' "$1/CHANGELOG.md" | sed -E 's/^## \[v?([0-9]+\.[0-9]+\.[0-9]+)\].*/\1/'
}
has_removed_fragments() {
echo "$1" | grep -qE '\.removed(\.[0-9]+)?\.md$'
}
# Resolve every component's effective version before compiling
# anything, so a wrong input cannot leave the tree half-compiled.
# Empty input = auto-derive (latest released version + semver bump
# from the pending fragment types). 'skip' = hold the component back.
errors=0
compiling=""
for component in prowler api ui mcp_server; do
input=$(component_input "$component")
fragments=$(pending_fragments "$component")
if [ "$input" = "skip" ]; then
if [ -n "$fragments" ]; then
echo "::warning::${component}: held back by request; these pending fragments stay for a future release:"
echo "$fragments"
fi
continue
fi
if [ -n "$input" ] && [ -z "$fragments" ]; then
echo "::error::${component}: version input '$input' provided but ${component}/changelog.d/ has no pending fragments (wrong input?)"
errors=1
continue
fi
if [ -z "$fragments" ]; then
continue
fi
removed_fragments=false
if has_removed_fragments "$fragments"; then
removed_fragments=true
fi
current=$(latest_released_version "$component")
if [ -z "$current" ]; then
echo "::error::${component}: could not read the latest released version from ${component}/CHANGELOG.md; restore the released heading before compiling"
errors=1
continue
fi
if [ -n "$input" ]; then
effective="$input"
mode="explicit"
current_key=$(version_key "$current")
effective_key=$(version_key "$effective")
if [[ "$effective_key" < "$current_key" || "$effective_key" == "$current_key" ]]; then
echo "::error::${component}: explicit version '${effective}' must be greater than the latest released version (${current})"
errors=1
continue
fi
else
if [ "$removed_fragments" = "true" ]; then
echo "::error::${component}: pending 'removed' fragments imply a major bump (breaking change); provide its version input explicitly"
errors=1
continue
fi
# SDK, UI, and API versions are deterministic mirrors of the
# Prowler version (the scheme bump-version.yml codifies): the SDK
# mirrors it directly, the UI tracks 1.<minor>.<patch>, and the
# API is the independent 1.<minor + 1>.<patch> stream. Only the
# MCP Server has its own cadence, derived from fragment types.
IFS=. read -r _ prowler_minor prowler_patch <<< "$PROWLER_VERSION"
prowler_minor=$((10#$prowler_minor))
prowler_patch=$((10#$prowler_patch))
case "$component" in
prowler) effective="$PROWLER_VERSION" ;;
ui) effective="1.${prowler_minor}.${prowler_patch}" ;;
api) effective="1.$((prowler_minor + 1)).${prowler_patch}" ;;
mcp_server)
IFS=. read -r major minor patch <<< "$current"
major=$((10#$major))
minor=$((10#$minor))
patch=$((10#$patch))
# Prowler patch releases (vN.N target) are maintenance
# releases, so the MCP Server bumps patch regardless of
# fragment types; a deliberate exception needs the explicit
# version input.
if [ "$TARGET_BRANCH" != "master" ]; then
effective="${major}.${minor}.$((patch + 1))"
if echo "$fragments" | grep -qE '\.(added|deprecated)(\.[0-9]+)?\.md$'; then
echo "::warning::${component}: 'added'/'deprecated' fragments are shipping in a Prowler patch; auto-derived a patch bump (${current} -> ${effective}), pass the version input to override"
fi
elif echo "$fragments" | grep -qE '\.(added|changed|deprecated)(\.[0-9]+)?\.md$'; then
effective="${major}.$((minor + 1)).0"
else
effective="${major}.${minor}.$((patch + 1))"
fi
;;
esac
current_key=$(version_key "$current")
effective_key=$(version_key "$effective")
if [[ "$effective_key" < "$current_key" || "$effective_key" == "$current_key" ]]; then
echo "::error::${component}: auto-derived version '${effective}' is not greater than the latest released version (${current}); check prowler_version or pass the version input explicitly"
errors=1
continue
fi
mode="auto"
echo "::notice::${component}: version auto-derived ${current} -> ${effective}"
fi
if [ "$removed_fragments" = "true" ]; then
IFS=. read -r current_major _ <<< "$current"
current_major=$((10#$current_major))
IFS=. read -r effective_major effective_minor effective_patch <<< "$effective"
effective_major=$((10#$effective_major))
effective_minor=$((10#$effective_minor))
effective_patch=$((10#$effective_patch))
if [ "$effective_major" -le "$current_major" ] || [ "$effective_minor" -ne 0 ] || [ "$effective_patch" -ne 0 ]; then
echo "::error::${component}: removed fragments require a major component release (${current} -> X.0.0 with X > ${current_major}); got ${effective}"
errors=1
continue
fi
fi
# Without the marker the build would insert the new block above the
# file header instead of below it.
if ! grep -q '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md is missing the '<!-- changelog: release notes start -->' marker; restore it after the intro line before compiling"
errors=1
continue
fi
# A hand-written UNRELEASED block means someone followed the old
# convention; its entries would be left out of the compiled block
# and out of the release notes extraction.
if grep -q '(Prowler UNRELEASED)' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md contains a hand-written '(Prowler UNRELEASED)' block; convert its entries to fragments in ${component}/changelog.d/ and delete the block before compiling"
errors=1
continue
fi
echo "${effective} ${mode}" > "${RUNNER_TEMP}/version-${component}.txt"
compiling="${compiling}${component} "
done
if [ "$errors" -ne 0 ]; then
exit 1
fi
if [ -z "$compiling" ]; then
echo "::error::Nothing to compile: no component has pending fragments to release"
exit 1
fi
body_file="${RUNNER_TEMP}/compile-changelogs-pr-body.md"
{
echo "### Description"
echo ""
echo "Compiles the pending changelog fragments into the per-component \`CHANGELOG.md\` files for Prowler v${PROWLER_VERSION}, replacing the manual stamping PR."
echo ""
echo "| Component | Version | Fragments consumed |"
echo "|---|---|---|"
} > "$body_file"
compiled_components=""
for component in prowler api ui mcp_server; do
if [ ! -f "${RUNNER_TEMP}/version-${component}.txt" ]; then
echo "Skipping ${component} (no pending fragments or held back)"
echo "| \`${component}\` | - | 0 |" >> "$body_file"
continue
fi
read -r version mode < "${RUNNER_TEMP}/version-${component}.txt"
version_label="$version"
if [ "$mode" = "auto" ]; then
version_label="${version} (auto)"
fi
count=$(pending_fragments "$component" | wc -l | tr -d ' ')
echo "Compiling ${component} ${version} (${count} fragments, ${mode} version)..."
# Captured before attribution renames them: these original paths are
# what the forward-sync deletes on master (backports copy fragments
# verbatim, so filenames match across branches).
pending_fragments "$component" > "${RUNNER_TEMP}/consumed-${component}.txt"
pre_lines=$(wc -l < "$component/CHANGELOG.md")
# Attribution must run before the build: towncrier renders the
# first dotted segment of each filename as the PR number.
python .github/scripts/changelog_attribution.py "$component"
towncrier build --config "$component/towncrier.toml" --version "$version" --name "Prowler v${PROWLER_VERSION}" --yes
# The build only inserts lines right after the marker, so the new
# stamped block is exactly the added lines following it. Captured
# for the forward-sync to master.
post_lines=$(wc -l < "$component/CHANGELOG.md")
delta=$((post_lines - pre_lines))
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md" | cut -d: -f1)
sed -n "$((marker_line + 1)),$((marker_line + delta))p" "$component/CHANGELOG.md" > "${RUNNER_TEMP}/block-${component}.md"
compiled_components="${compiled_components}${component} "
echo "| \`${component}\` | ${version_label} | ${count} |" >> "$body_file"
done
echo "COMPILED_COMPONENTS=${compiled_components}" >> "$GITHUB_ENV"
{
echo ""
echo "Review that no pending fragment was dropped (the diff must delete every consumed fragment) and that each new version block is correct, then squash-merge."
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$body_file"
echo "PR_BODY_FILE=${body_file}" >> "$GITHUB_ENV"
- name: Create compile PR
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
branch: compile-changelogs-${{ env.PROWLER_VERSION }}
base: ${{ env.TARGET_BRANCH }}
title: 'chore(changelog): v${{ env.PROWLER_VERSION }}'
body-path: ${{ env.PR_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
skip-sync
# Patch compiles (target_branch = v5.X) leave master holding the consumed
# fragments and missing the new version block. This applies the equivalent
# change to master: insert the same stamped blocks under the marker and
# delete the consumed fragments, so the next minor compile cannot
# re-release entries that already shipped in the patch.
- name: Apply forward-sync to master
if: env.TARGET_BRANCH != 'master'
run: |
set -euo pipefail
git checkout -B master origin/master
version_key() {
local version="$1"
local major minor patch
IFS=. read -r major minor patch <<< "$version"
printf '%06d.%06d.%06d' "$((10#$major))" "$((10#$minor))" "$((10#$patch))"
}
release_from_heading() {
local heading="$1"
echo "$heading" | sed -E 's/^## \[[^]]+\] \(Prowler v?([0-9]+\.[0-9]+\.[0-9]+)\).*/\1/'
}
insert_changelog_block_ordered() {
local component="$1"
local block_file="$2"
local changelog="${component}/CHANGELOG.md"
local incoming_heading incoming_release incoming_key
local marker_line insertion_line duplicate_line total_lines
local line heading existing_release existing_key
marker_line=$(grep -n -m1 '^<!-- changelog: release notes start -->$' "$changelog" | cut -d: -f1)
incoming_heading=$(grep -m1 -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$block_file" || true)
if [ -z "$incoming_heading" ]; then
echo "::error::${block_file} does not contain a stamped Prowler release heading"
exit 1
fi
incoming_release=$(release_from_heading "$incoming_heading")
incoming_key=$(version_key "$incoming_release")
insertion_line=""
duplicate_line=""
while IFS=: read -r line heading; do
existing_release=$(release_from_heading "$heading")
existing_key=$(version_key "$existing_release")
if [[ "$incoming_key" == "$existing_key" ]]; then
duplicate_line="$line"
break
fi
if [[ "$incoming_key" > "$existing_key" ]]; then
insertion_line="$line"
break
fi
done < <(grep -n -E '^## \[[^]]+\] \(Prowler v?[0-9]+\.[0-9]+\.[0-9]+\)' "$changelog" || true)
if [ -n "$duplicate_line" ]; then
echo "::error::${changelog} already contains a block for Prowler v${incoming_release} at line ${duplicate_line}; refusing to insert a duplicate"
exit 1
fi
if [ -z "$insertion_line" ]; then
insertion_line=$(($(wc -l < "$changelog") + 1))
fi
if [ "$insertion_line" -le "$marker_line" ]; then
insertion_line=$((marker_line + 1))
fi
# The captured block window can be off by one blank line on either
# end (towncrier re-emits the blank after the marker), so strip the
# outer blank lines and pad exactly one on each side: the block
# must never glue to the marker above or the next heading below.
awk '
/[^[:space:]]/ { for (i = 0; i < pending; i++) print ""; pending = 0; print; started = 1; next }
started { pending++ }
' "$block_file" > "${RUNNER_TEMP}/block-normalized.md"
total_lines=$(wc -l < "$changelog")
{
head -n "$((insertion_line - 1))" "$changelog"
if [ "$insertion_line" -gt 1 ] && [ -n "$(sed -n "$((insertion_line - 1))p" "$changelog")" ]; then
echo ""
fi
cat "${RUNNER_TEMP}/block-normalized.md"
if [ "$insertion_line" -le "$total_lines" ]; then
echo ""
fi
tail -n +"$insertion_line" "$changelog"
} > "${RUNNER_TEMP}/changelog.tmp"
mv "${RUNNER_TEMP}/changelog.tmp" "$changelog"
echo "::notice::Inserted ${component} changelog block for Prowler v${incoming_release} at line ${insertion_line}"
}
sync_body="${RUNNER_TEMP}/forward-sync-pr-body.md"
{
echo "### Description"
echo ""
echo "Forward-syncs the v${PROWLER_VERSION} compiled changelogs from \`${TARGET_BRANCH}\` to \`master\`: inserts the same stamped version blocks under the insertion marker and deletes the consumed fragments, so the next minor compile cannot re-release entries that already shipped in this patch. Opened automatically by the same run that opened the compile PR; review and squash-merge after it."
echo ""
echo "| Component | Fragments deleted on master | Skipped (only on ${TARGET_BRANCH}) |"
echo "|---|---|---|"
} > "$sync_body"
for component in $COMPILED_COMPONENTS; do
block_file="${RUNNER_TEMP}/block-${component}.md"
consumed_file="${RUNNER_TEMP}/consumed-${component}.txt"
if ! grep -qm1 '^<!-- changelog: release notes start -->$' "$component/CHANGELOG.md"; then
echo "::error::${component}/CHANGELOG.md on master is missing the insertion marker; cannot forward-sync"
exit 1
fi
deleted=0
skipped=0
while IFS= read -r fragment; do
if [ -z "$fragment" ]; then
continue
fi
if [ -f "$fragment" ]; then
git rm -q "$fragment"
deleted=$((deleted + 1))
else
echo "::notice::${fragment} does not exist on master (change landed only on ${TARGET_BRANCH}); skipping its deletion"
skipped=$((skipped + 1))
fi
done < "$consumed_file"
insert_changelog_block_ordered "$component" "$block_file"
echo "| \`${component}\` | ${deleted} | ${skipped} |" >> "$sync_body"
done
{
echo ""
echo "### License"
echo ""
echo "By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license."
} >> "$sync_body"
echo "SYNC_BODY_FILE=${sync_body}" >> "$GITHUB_ENV"
- name: Create forward-sync PR
if: env.TARGET_BRANCH != 'master'
uses: peter-evans/create-pull-request@5f6978faf089d4d20b00c7766989d076bb2fc7f1 # v8.1.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
commit-message: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
branch: forward-sync-changelogs-${{ env.PROWLER_VERSION }}
base: master
title: 'chore(changelog): v${{ env.PROWLER_VERSION }} forward-sync to master'
body-path: ${{ env.SYNC_BODY_FILE }}
author: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
labels: |
no-changelog
skip-sync
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -0,0 +1,50 @@
name: 'Docs: Check Provider Cards Snippet'
on:
pull_request:
branches:
- 'master'
- 'v5.*'
paths:
- 'docs/user-guide/providers/**/getting-started-*.mdx'
- 'docs/scripts/generate_provider_cards.py'
- 'docs/snippets/provider-cards.mdx'
- 'api/src/backend/api/models.py'
- '.github/workflows/docs-check-provider-cards.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions: {}
jobs:
check-provider-cards:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
github.com:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- name: Verify provider cards snippet is up to date
run: |
if ! python3 docs/scripts/generate_provider_cards.py; then
echo "::error::docs/snippets/provider-cards.mdx is out of sync with the provider getting-started pages or the API ProviderChoices enum."
echo "Run 'python3 docs/scripts/generate_provider_cards.py' locally and commit the regenerated snippet."
echo "--- diff ---"
git diff docs/snippets/provider-cards.mdx
exit 1
fi
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
# We can't block as Trufflehog needs to verify secrets against vendors
egress-policy: audit
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+1301 -703
View File
File diff suppressed because one or more lines are too long
+9 -3
View File
@@ -12,8 +12,8 @@ if: contains(toJson(github.event.issue.labels), 'status/needs-triage')
timeout-minutes: 12
rate-limit:
max: 5
user-rate-limit:
max-runs-per-window: 5
window: 60
concurrency:
@@ -30,6 +30,12 @@ permissions:
engine: copilot
strict: false
pre-steps:
- name: Harden the runner
uses: step-security/harden-runner@v2.20.0
with:
egress-policy: audit
imports:
- ../agents/issue-triage.md
@@ -108,7 +114,7 @@ Triage the following GitHub issue using the Prowler Issue Triage Agent persona.
## Sanitized Issue Content
${{ needs.activation.outputs.text }}
${{ steps.sanitized.outputs.text }}
## Instructions
+3 -3
View File
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -46,7 +46,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -93,7 +93,7 @@ jobs:
fi
- name: Add community label
if: steps.check_membership.outputs.is_member == 'false'
if: steps.check_membership.outputs.is_member == 'false' && github.event.pull_request.user.type != 'Bot'
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ github.token }}
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@fa2e9d605c4eeb9fcad4c99c224cee0c6c7f3594 # v2.16.0
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -30
View File
@@ -45,7 +45,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
@@ -64,7 +64,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -106,7 +106,7 @@ jobs:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -165,7 +165,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -206,7 +206,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -227,7 +227,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -263,27 +263,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger MCP deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: mcp-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+3 -2
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -68,7 +68,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -102,6 +102,7 @@ jobs:
files_ignore: |
mcp_server/README.md
mcp_server/CHANGELOG.md
mcp_server/changelog.d/**
- name: Set up Docker Buildx
if: steps.check-changes.outputs.any_changed == 'true'
+2 -2
View File
@@ -29,7 +29,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -67,7 +67,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -48,7 +48,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -83,7 +83,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+186 -29
View File
@@ -19,6 +19,60 @@ concurrency:
permissions: {}
jobs:
test-changelog-attribution:
if: github.repository == 'prowler-cloud/prowler'
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
objects.githubusercontent.com:443
pypi.org:443
files.pythonhosted.org:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 1
persist-credentials: false
- name: Fetch PR base ref for tj-actions/changed-files
env:
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: git fetch --depth=1 origin "${BASE_REF}"
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
.github/scripts/changelog_attribution.py
.github/workflows/pr-check-changelog.yml
.github/workflows/compile-changelogs.yml
.github/towncrier/template.md.jinja
*/towncrier.toml
tests/github/**
- name: Set up Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0
with:
python-version: '3.12'
- name: Test changelog attribution
if: steps.changed-files.outputs.any_changed == 'true'
run: |
python3 -m pip install --user --disable-pip-version-check pytest==9.0.3 towncrier==25.8.0
python3 -m pytest tests/github
check-changelog:
if: contains(github.event.pull_request.labels.*.name, 'no-changelog') == false
runs-on: ubuntu-latest
@@ -31,7 +85,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -62,53 +116,160 @@ jobs:
uv.lock
pyproject.toml
- name: Check for folder changes and changelog presence
- name: Check for folder changes and changelog fragment presence
id: check-folders
run: |
missing_changelogs=""
fragment_name_re='^[A-Za-z0-9][A-Za-z0-9._-]*\.(added|changed|deprecated|removed|fixed|security)(\.[0-9]+)?\.md$'
manual_pr_link_re='(\[\(#[0-9]+\)\]|\[#[0-9]+\]\(|\(#[0-9]+\)|github\.com/[^[:space:]/]+/[^[:space:]/]+/(pull|issues)/[0-9]+)'
folder_alt=$(echo "$MONITORED_FOLDERS" | tr ' ' '|')
missing_fragments=""
invalid_fragments=""
linked_fragments=""
handwritten_changelogs=""
all_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n')
added=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" | tr ' ' '\n')
added_or_renamed=$(printf '%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
added_modified_or_renamed=$(printf '%s\n%s\n%s' "${STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES}" "${STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES}" | tr ' ' '\n')
# Returns success if the folder has a valid fragment added, modified, or renamed.
has_changelog_update() {
local folder="$1"
if echo "$added_modified_or_renamed" | grep "^${folder}/changelog.d/" | sed "s|^${folder}/changelog.d/||" | grep -qE "$fragment_name_re"; then
return 0
fi
return 1
}
if [[ "${STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED}" == "true" ]]; then
# Check monitored folders
for folder in $MONITORED_FOLDERS; do
# Get files changed in this folder
changed_in_folder=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^${folder}/" || true)
if echo "$all_changed" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "Direct CHANGELOG.md edits are not allowed for ${folder}/"
handwritten_changelogs="${handwritten_changelogs}- \`${folder}/CHANGELOG.md\`"$'\n'
fi
changed_in_folder=$(echo "$all_changed" | grep "^${folder}/" | grep -v "^${folder}/CHANGELOG.md$" || true)
if [ -n "$changed_in_folder" ]; then
echo "Detected changes in ${folder}/"
# Check if CHANGELOG.md was updated
if ! echo "$changed_in_folder" | grep -q "^${folder}/CHANGELOG.md$"; then
echo "No changelog update found for ${folder}/"
missing_changelogs="${missing_changelogs}- \`${folder}\`"$'\n'
if ! has_changelog_update "$folder"; then
echo "No changelog fragment found for ${folder}/"
missing_fragments="${missing_fragments}- \`${folder}\`"$'\n'
fi
fi
done
# Check root-level dependency files (uv.lock, pyproject.toml)
# These are associated with the prowler folder changelog
root_deps_changed=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
root_deps_changed=$(echo "$all_changed" | grep -E "^(uv\.lock|pyproject\.toml)$" || true)
if [ -n "$root_deps_changed" ]; then
echo "Detected changes in root dependency files: $root_deps_changed"
# Check if prowler/CHANGELOG.md was already updated (might have been caught above)
prowler_changelog_updated=$(echo "${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}" | tr ' ' '\n' | grep "^prowler/CHANGELOG.md$" || true)
if [ -z "$prowler_changelog_updated" ]; then
if ! has_changelog_update "prowler"; then
# Only add if prowler wasn't already flagged
if ! echo "$missing_changelogs" | grep -q "prowler"; then
echo "No changelog update found for root dependency changes"
missing_changelogs="${missing_changelogs}- \`prowler\` (root dependency files changed)"$'\n'
if ! echo "$missing_fragments" | grep -q "prowler"; then
echo "No changelog fragment found for root dependency changes"
missing_fragments="${missing_fragments}- \`prowler\` (root dependency files changed)"$'\n'
fi
fi
fi
# Validate the filename of every fragment added by this PR
added_fragments=$(echo "$added_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $added_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ]; then
continue
fi
if ! echo "$name" | grep -qE "$fragment_name_re"; then
echo "Invalid fragment filename: $fragment"
invalid_fragments="${invalid_fragments}- \`${fragment}\`"$'\n'
fi
done
# Lint fragment content: the PR link is attached automatically at
# compile time, so a hand-written PR or issue link would be wrong
touched_fragments=$(echo "$added_modified_or_renamed" | grep -E "^(${folder_alt})/changelog\.d/" || true)
for fragment in $touched_fragments; do
name=$(basename "$fragment")
if [ "$name" = ".gitkeep" ] || [ "$name" = "README.md" ] || [ ! -f "$fragment" ]; then
continue
fi
if grep -qE "$manual_pr_link_re" "$fragment"; then
echo "Fragment contains a hand-written PR or issue link: $fragment"
linked_fragments="${linked_fragments}- \`${fragment}\`"$'\n'
fi
done
fi
{
echo "missing_changelogs<<EOF"
echo -e "${missing_changelogs}"
echo "EOF"
} >> $GITHUB_OUTPUT
# Suggest a slug derived from the branch name for the bot comment
suggested_slug=$(echo "$HEAD_REF" | tr '[:upper:]' '[:lower:]' | sed 's|.*/||; s/[^a-z0-9._-]/-/g; s/^[^a-z0-9]*//')
if [ -z "$suggested_slug" ]; then
suggested_slug="my-change"
fi
fragment_help="A changelog fragment is a small Markdown file named \`<slug>.<type>.md\` under \`<component>/changelog.d/\`, where \`<type>\` is one of \`added\`, \`changed\`, \`deprecated\`, \`removed\`, \`fixed\` or \`security\`. Its content is the changelog entry text, without the PR link (added automatically at release time) and without a trailing period. For example:
\`\`\`
echo 'Entry text describing the change' > <component>/changelog.d/${suggested_slug}.fixed.md
\`\`\`
If this PR does not need a changelog entry, add the \`no-changelog\` label instead."
if [ -n "$missing_fragments" ] || [ -n "$invalid_fragments" ] || [ -n "$linked_fragments" ] || [ -n "$handwritten_changelogs" ]; then
comment_body=""
if [ -n "$missing_fragments" ]; then
comment_body="⚠️ **Changes detected in the following folders without a changelog fragment:**"$'\n\n'"${missing_fragments}"$'\n'
fi
if [ -n "$invalid_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragment filenames that do not follow the naming convention:**"$'\n\n'"${invalid_fragments}"$'\n'
fi
if [ -n "$linked_fragments" ]; then
comment_body="${comment_body}⚠️ **Changelog fragments containing a hand-written PR or issue link (remove it; the PR link is attached automatically at release time):**"$'\n\n'"${linked_fragments}"$'\n'
fi
if [ -n "$handwritten_changelogs" ]; then
comment_body="${comment_body}⚠️ **Direct \`CHANGELOG.md\` edits are not allowed in regular PRs:**"$'\n\n'"${handwritten_changelogs}"$'\n'
fi
comment_body="${comment_body}${fragment_help}"
else
comment_body="✅ All required changelog fragments are present."
fi
write_multiline_output() {
local name="$1"
local value="$2"
local delimiter
while true; do
delimiter="EOF_$(openssl rand -hex 16)"
if ! grep -qxF "$delimiter" <<< "$value"; then
break
fi
done
{
echo "${name}<<${delimiter}"
if [ -n "$value" ]; then
printf '%s\n' "$value"
fi
echo "${delimiter}"
} >> "$GITHUB_OUTPUT"
}
write_multiline_output "missing_fragments" "$missing_fragments"
write_multiline_output "invalid_fragments" "$invalid_fragments"
write_multiline_output "linked_fragments" "$linked_fragments"
write_multiline_output "handwritten_changelogs" "$handwritten_changelogs"
write_multiline_output "comment_body" "$comment_body"
env:
STEPS_CHANGED_FILES_OUTPUTS_ANY_CHANGED: ${{ steps.changed-files.outputs.any_changed }}
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
STEPS_CHANGED_FILES_OUTPUTS_ADDED_FILES: ${{ steps.changed-files.outputs.added_files }}
STEPS_CHANGED_FILES_OUTPUTS_MODIFIED_FILES: ${{ steps.changed-files.outputs.modified_files }}
STEPS_CHANGED_FILES_OUTPUTS_RENAMED_FILES: ${{ steps.changed-files.outputs.renamed_files }}
HEAD_REF: ${{ github.event.pull_request.head.ref }}
- name: Find existing changelog comment
if: github.event.pull_request.head.repo.full_name == github.repository
@@ -128,14 +289,10 @@ jobs:
edit-mode: replace
body: |
<!-- changelog-check -->
${{ steps.check-folders.outputs.missing_changelogs != '' && format('⚠️ **Changes detected in the following folders without a corresponding update to the `CHANGELOG.md`:**
${{ steps.check-folders.outputs.comment_body }}
{0}
Please add an entry to the corresponding `CHANGELOG.md` file to maintain a clear history of changes.', steps.check-folders.outputs.missing_changelogs) || '✅ All necessary `CHANGELOG.md` files have been updated.' }}
- name: Fail if changelog is missing
if: steps.check-folders.outputs.missing_changelogs != ''
- name: Fail if changelog fragment is missing or invalid
if: steps.check-folders.outputs.missing_fragments != '' || steps.check-folders.outputs.invalid_fragments != '' || steps.check-folders.outputs.linked_fragments != '' || steps.check-folders.outputs.handwritten_changelogs != ''
run: |
echo "::error::Missing changelog updates in some folders"
echo "::error::Missing, invalid, or disallowed changelog updates"
exit 1
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+63 -27
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -37,8 +37,7 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 1
# zizmor: ignore[artipacked]
persist-credentials: true # Required by tj-actions/changed-files to fetch PR branch
persist-credentials: false # No write token in the untrusted PR-head tree; public repo so base fetch/changed-files work unauthenticated
- name: Fetch PR base ref for tj-actions/changed-files
env:
@@ -50,6 +49,8 @@ jobs:
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: '**'
safe_output: false # Raw paths (list read via env var, injection-safe); default escaping backslash-quotes chars like () and breaks the -f test
separator: "\n" # Newline-delimited so the reader tolerates spaces and glob chars in paths
- name: Check for conflict markers
id: conflict-check
@@ -59,19 +60,18 @@ jobs:
CONFLICT_FILES=""
HAS_CONFLICTS=false
# Check each changed file for conflict markers
for file in ${STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES}; do
if [ -f "$file" ]; then
echo "Checking file: $file"
# Read newline-delimited paths so spaces/globs neither word-split nor glob-expand
while IFS= read -r file; do
[ -n "$file" ] || continue
[ -f "$file" ] || continue
echo "Checking file: $file"
# Look for conflict markers (more precise regex)
if grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$file" 2>/dev/null; then
echo "Conflict markers found in: $file"
CONFLICT_FILES="${CONFLICT_FILES}- \`${file}\`"$'\n'
HAS_CONFLICTS=true
fi
if grep -qE '^(<<<<<<<|=======|>>>>>>>)' "$file" 2>/dev/null; then
echo "Conflict markers found in: $file"
CONFLICT_FILES="${CONFLICT_FILES}- \`${file}\`"$'\n'
HAS_CONFLICTS=true
fi
done
done <<< "$STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES"
if [ "$HAS_CONFLICTS" = true ]; then
echo "has_conflicts=true" >> $GITHUB_OUTPUT
@@ -88,18 +88,49 @@ jobs:
env:
STEPS_CHANGED_FILES_OUTPUTS_ALL_CHANGED_FILES: ${{ steps.changed-files.outputs.all_changed_files }}
- name: Check base-branch mergeability
id: merge-check
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
MERGEABLE=null
# GitHub computes mergeability async, so .mergeable is null until ready; poll until resolved
for attempt in 1 2 3 4 5; do
MERGEABLE=$(gh api "repos/${REPO}/pulls/${PR_NUMBER}" --jq '.mergeable')
if [ "$MERGEABLE" != "null" ]; then
break
fi
echo "Attempt ${attempt}: mergeability not computed yet, retrying..."
sleep 3
done
# Keep 'unknown' distinct from 'clean' so we never assert a clean merge we could not confirm
case "$MERGEABLE" in
false) STATUS=conflict; echo "PR branch cannot be merged cleanly into its base branch" ;;
true) STATUS=clean; echo "PR branch merges cleanly into its base branch" ;;
*) STATUS=unknown; echo "::warning::Mergeability did not resolve after retries; leaving it undetermined" ;;
esac
echo "merge_status=${STATUS}" >> "$GITHUB_OUTPUT"
- name: Manage conflict label
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
HAS_CONFLICTS: ${{ steps.conflict-check.outputs.has_conflicts }}
MERGE_STATUS: ${{ steps.merge-check.outputs.merge_status }}
run: |
LABEL_NAME="has-conflicts"
# Add or remove label based on conflict status
if [ "$HAS_CONFLICTS" = "true" ]; then
if [ "$HAS_CONFLICTS" = "true" ] || [ "$MERGE_STATUS" = "conflict" ]; then
echo "Adding conflict label to PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --add-label "$LABEL_NAME" --repo ${{ github.repository }} || true
elif [ "$MERGE_STATUS" = "unknown" ]; then
# Don't drop the label on an undetermined merge state; a later run will settle it
echo "Mergeability undetermined; leaving label unchanged"
else
echo "Removing conflict label from PR #${PR_NUMBER}..."
gh pr edit "$PR_NUMBER" --remove-label "$LABEL_NAME" --repo ${{ github.repository }} || true
@@ -121,20 +152,25 @@ jobs:
edit-mode: replace
body: |
<!-- conflict-checker-comment -->
${{ steps.conflict-check.outputs.has_conflicts == 'true' && '⚠️ **Conflict Markers Detected**' || '✅ **Conflict Markers Resolved**' }}
${{ steps.conflict-check.outputs.has_conflicts == 'true' && format('This pull request contains unresolved conflict markers in the following files:
${{ (steps.conflict-check.outputs.has_conflicts == 'true' || steps.merge-check.outputs.merge_status == 'conflict') && '⚠️ **Conflicts Detected**' || (steps.merge-check.outputs.merge_status == 'unknown' && '️ **Conflict Check Incomplete**' || '✅ **No Conflicts**') }}
${{ steps.conflict-check.outputs.has_conflicts == 'true' && format('
**Conflict markers** are present in the following files:
{0}
Please resolve these conflicts by:
1. Locating the conflict markers: `<<<<<<<`, `=======`, and `>>>>>>>`
2. Manually editing the files to resolve the conflicts
3. Removing all conflict markers
4. Committing and pushing the changes', steps.conflict-check.outputs.conflict_files) || 'All conflict markers have been successfully resolved in this pull request.' }}
Resolve them by removing every `<<<<<<<`, `=======`, and `>>>>>>>` marker, then commit and push.', steps.conflict-check.outputs.conflict_files) || '' }}
${{ steps.merge-check.outputs.merge_status == 'conflict' && '
**Merge conflict with the base branch.** This PR cannot be merged cleanly. Update your branch with the latest base (rebase or merge) and resolve the conflicts.' || '' }}
${{ steps.merge-check.outputs.merge_status == 'unknown' && '
GitHub had not finished computing mergeability, so base-branch conflict status could not be verified on this run.' || '' }}
${{ (steps.conflict-check.outputs.has_conflicts != 'true' && steps.merge-check.outputs.merge_status == 'clean') && '
No conflict markers, and the branch merges cleanly into its base.' || '' }}
- name: Fail workflow if conflicts detected
if: steps.conflict-check.outputs.has_conflicts == 'true'
if: steps.conflict-check.outputs.has_conflicts == 'true' || steps.merge-check.outputs.merge_status == 'conflict'
env:
HAS_CONFLICTS: ${{ steps.conflict-check.outputs.has_conflicts }}
MERGE_STATUS: ${{ steps.merge-check.outputs.merge_status }}
run: |
echo "::error::Workflow failed due to conflict markers detected in the PR"
[ "$HAS_CONFLICTS" = "true" ] && echo "::error::Conflict markers detected in changed files"
[ "$MERGE_STATUS" = "conflict" ] && echo "::error::PR branch has merge conflicts with the base branch"
exit 1
+3 -3
View File
@@ -26,7 +26,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -56,6 +56,6 @@ jobs:
"PROWLER_PR_BODY": ${{ toJson(github.event.pull_request.body) }},
"PROWLER_PR_URL": ${{ toJson(github.event.pull_request.html_url) }},
"PROWLER_PR_MERGED_BY": "${{ github.event.pull_request.merged_by.login }}",
"PROWLER_PR_BASE_BRANCH": "${{ github.event.pull_request.base.ref }}",
"PROWLER_PR_HEAD_BRANCH": "${{ github.event.pull_request.head.ref }}"
"PROWLER_PR_BASE_BRANCH": ${{ toJson(github.event.pull_request.base.ref) }},
"PROWLER_PR_HEAD_BRANCH": ${{ toJson(github.event.pull_request.head.ref) }}
}
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -28,12 +28,13 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
github.com:443
raw.githubusercontent.com:443
objects.githubusercontent.com:443
codeload.github.com:443
release-assets.githubusercontent.com:443
@@ -41,6 +42,7 @@ jobs:
files.pythonhosted.org:443
registry.npmjs.org:443
nodejs.org:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+2 -1
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -55,6 +55,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
+3 -1
View File
@@ -12,6 +12,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -23,6 +24,7 @@ on:
- '.github/workflows/sdk-codeql.yml'
- '.github/codeql/sdk-codeql-config.yml'
- '!prowler/CHANGELOG.md'
- '!prowler/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -51,7 +53,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+30 -20
View File
@@ -60,7 +60,7 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -98,7 +98,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -138,15 +138,18 @@ jobs:
permissions:
contents: read
packages: write
id-token: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
api.ecr-public.us-east-1.amazonaws.com:443
public.ecr.aws:443
sts.amazonaws.com:443
sts.us-east-1.amazonaws.com:443
registry-1.docker.io:443
production.cloudflare.docker.com:443
production.cloudfront.docker.com:443
@@ -173,14 +176,16 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
aws-region: us-east-1
role-to-assume: ${{ secrets.PUBLIC_ECR_PUSH_ROLE_ARN }}
- name: Login to Public ECR
uses: aws-actions/amazon-ecr-login@d539f0932e70871a027e9d5a9d8fc38589180a64 # v2.1.6
with:
registry-type: public
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
@@ -206,10 +211,11 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read
id-token: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -221,6 +227,8 @@ jobs:
github.com:443
release-assets.githubusercontent.com:443
api.ecr-public.us-east-1.amazonaws.com:443
sts.amazonaws.com:443
sts.us-east-1.amazonaws.com:443
- name: Login to DockerHub
@@ -229,14 +237,16 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Public ECR
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
- name: Configure AWS credentials (OIDC)
uses: aws-actions/configure-aws-credentials@d979d5b3a71173a29b74b5b88418bfda9437d885 # v6.1.1
with:
registry: public.ecr.aws
username: ${{ secrets.PUBLIC_ECR_AWS_ACCESS_KEY_ID }}
password: ${{ secrets.PUBLIC_ECR_AWS_SECRET_ACCESS_KEY }}
env:
AWS_REGION: ${{ env.AWS_REGION }}
aws-region: us-east-1
role-to-assume: ${{ secrets.PUBLIC_ECR_PUSH_ROLE_ARN }}
- name: Login to Public ECR
uses: aws-actions/amazon-ecr-login@d539f0932e70871a027e9d5a9d8fc38589180a64 # v2.1.6
with:
registry-type: public
- name: Create and push manifests for push event
if: github.event_name == 'push'
@@ -299,7 +309,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -320,7 +330,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+5 -2
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -71,7 +71,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -94,6 +94,8 @@ jobs:
_http._tcp.deb.debian.org:443
powershellinfraartifacts-gkhedzdeaghdezhr.z01.azurefd.net:443
get.trivy.dev:443
raw.githubusercontent.com:443
releases.astral.sh:443
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
@@ -113,6 +115,7 @@ jobs:
.github/workflows/sdk-container-checks.yml
files_ignore: |
prowler/CHANGELOG.md
prowler/changelog.d/**
**/AGENTS.md
- name: Set up Docker Buildx
+3 -3
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -66,7 +66,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -102,7 +102,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -27,7 +27,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -25,7 +25,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+2 -1
View File
@@ -37,7 +37,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -73,6 +73,7 @@ jobs:
.github/scripts/osv-scan.sh
files_ignore: |
prowler/CHANGELOG.md
prowler/changelog.d/**
**/AGENTS.md
- name: Setup Python with uv
+26 -1
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -77,6 +77,7 @@ jobs:
files_ignore: |
.github/**
prowler/CHANGELOG.md
prowler/changelog.d/**
docs/**
permissions/**
api/**
@@ -614,6 +615,30 @@ jobs:
flags: prowler-py${{ matrix.python-version }}-linode
files: ./linode_coverage.xml
# E2E Networks Provider
- name: Check if E2E Networks files changed
if: steps.check-changes.outputs.any_changed == 'true'
id: changed-e2enetworks
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
./prowler/**/e2enetworks/**
./tests/**/e2enetworks/**
./uv.lock
- name: Run E2E Networks tests
if: steps.changed-e2enetworks.outputs.any_changed == 'true'
run: uv run pytest -n auto --cov=./prowler/providers/e2enetworks --cov-report=xml:e2enetworks_coverage.xml tests/providers/e2enetworks
- name: Upload E2E Networks coverage to Codecov
if: steps.changed-e2enetworks.outputs.any_changed == 'true'
uses: codecov/codecov-action@671740ac38dd9b0130fbe1cec585b89eea48d3de # v5.5.2
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
flags: prowler-py${{ matrix.python-version }}-e2enetworks
files: ./e2enetworks_coverage.xml
# External Provider (dynamic loading)
- name: Check if External Provider files changed
if: steps.check-changes.outputs.any_changed == 'true'
+2 -2
View File
@@ -52,7 +52,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -73,7 +73,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.12'
python-version: '3.12.13'
- name: Install PyYAML
run: pip install pyyaml
+3 -1
View File
@@ -10,6 +10,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
pull_request:
branches:
- 'master'
@@ -19,6 +20,7 @@ on:
- '.github/workflows/ui-codeql.yml'
- '.github/codeql/ui-codeql-config.yml'
- '!ui/CHANGELOG.md'
- '!ui/changelog.d/**'
schedule:
- cron: '00 12 * * *'
@@ -47,7 +49,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -30
View File
@@ -45,7 +45,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -64,7 +64,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -107,7 +107,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -160,7 +160,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -201,7 +201,7 @@ jobs:
- name: Install regctl
if: always()
uses: regclient/actions/regctl-installer@da9319db8e44e8b062b3a147e1dfb2f574d41a03 # main
uses: regclient/actions/regctl-installer@9a2d4216180dbb3e2dccfa60d2dd4afd98e42ec5 # main
- name: Cleanup intermediate architecture tags
if: always()
@@ -222,7 +222,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -258,27 +258,3 @@ jobs:
payload-file-path: "./.github/scripts/slack-messages/container-release-completed.json"
step-outcome: ${{ steps.outcome.outputs.outcome }}
update-ts: ${{ needs.notify-release-started.outputs.message-ts }}
trigger-deployment:
needs: [setup, container-build-push]
if: always() && github.event_name == 'push' && needs.setup.result == 'success' && needs.container-build-push.result == 'success'
runs-on: ubuntu-latest
timeout-minutes: 5
permissions:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: block
allowed-endpoints: >
api.github.com:443
- name: Trigger UI deployment
uses: peter-evans/repository-dispatch@28959ce8df70de7be546dd1250a005dd32156697 # v4.0.1
with:
token: ${{ secrets.PROWLER_BOT_ACCESS_TOKEN }}
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: ui-prowler-deployment
client-payload: '{"sha": "${{ github.sha }}", "short_sha": "${{ needs.setup.outputs.short-sha }}"}'
+3 -2
View File
@@ -33,7 +33,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -69,7 +69,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
@@ -102,6 +102,7 @@ jobs:
files: ui/**
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
+4 -2
View File
@@ -14,6 +14,8 @@ on:
- '.github/test-impact.yml'
- 'ui/**'
- 'api/**' # API changes can affect UI E2E
- '!ui/CHANGELOG.md'
- '!api/CHANGELOG.md'
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
@@ -94,7 +96,7 @@ jobs:
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
@@ -314,7 +316,7 @@ jobs:
contents: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit
+1 -1
View File
@@ -32,7 +32,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
+6 -1
View File
@@ -31,14 +31,18 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: block
allowed-endpoints: >
github.com:443
registry.npmjs.org:443
nodejs.org:443
fonts.googleapis.com:443
fonts.gstatic.com:443
api.iconify.design:443
api.simplesvg.com:443
api.unisvg.com:443
api.github.com:443
release-assets.githubusercontent.com:443
cdn.playwright.dev:443
@@ -60,6 +64,7 @@ jobs:
.github/workflows/ui-tests.yml
files_ignore: |
ui/CHANGELOG.md
ui/changelog.d/**
ui/README.md
ui/AGENTS.md
+14 -6
View File
@@ -72,13 +72,13 @@ repos:
exclude: contrib
priority: 30
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/)
## PYTHON — SDK (prowler/, tests/, dashboard/, util/, scripts/, docs/scripts/)
- repo: https://github.com/myint/autoflake
rev: v2.3.3
hooks:
- id: autoflake
name: "SDK - autoflake"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--in-place", "--remove-all-unused-imports", "--remove-unused-variable"]
priority: 20
@@ -87,7 +87,7 @@ repos:
hooks:
- id: isort
name: "SDK - isort"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--profile", "black"]
stages: ["pre-commit"]
priority: 20
@@ -97,7 +97,7 @@ repos:
hooks:
- id: black
name: "SDK - black"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 20
- repo: https://github.com/pycqa/flake8
@@ -105,7 +105,7 @@ repos:
hooks:
- id: flake8
name: "SDK - flake8"
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
args: ["--ignore=E266,W503,E203,E501,W605"]
priority: 30
@@ -142,6 +142,14 @@ repos:
files: { glob: ["mcp_server/**/*.py"] }
priority: 20
- id: generate-provider-cards
name: "Docs - regenerate provider cards snippet"
entry: python3 docs/scripts/generate_provider_cards.py
language: system
files: { glob: ["docs/user-guide/providers/**/getting-started-*.mdx", "docs/scripts/generate_provider_cards.py", "docs/snippets/provider-cards.mdx", "api/src/backend/api/models.py"] }
pass_filenames: false
priority: 20
## PYTHON — uv (API + SDK)
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.11.14
@@ -183,7 +191,7 @@ repos:
entry: pylint --disable=W,C,R,E -j 0 -rn -sn
language: system
types: [python]
files: { glob: ["{prowler,tests,dashboard,util,scripts}/**/*.py"] }
files: { glob: ["{prowler,tests,dashboard,util,scripts,docs/scripts}/**/*.py"] }
priority: 30
- id: trufflehog
+33 -22
View File
@@ -15,21 +15,32 @@
# neither vulnerable code path (Archive::Tar parsing or regex compilation of
# attacker-controlled input) is reachable from Prowler. No Debian bookworm fix
# is available yet.
CVE-2026-42496 pkg:perl exp:2026-07-15
CVE-2026-42496 pkg:perl-base exp:2026-07-15
CVE-2026-42496 pkg:perl-modules-5.36 exp:2026-07-15
CVE-2026-42496 pkg:libperl5.36 exp:2026-07-15
CVE-2026-8376 pkg:perl exp:2026-07-15
CVE-2026-8376 pkg:perl-base exp:2026-07-15
CVE-2026-8376 pkg:perl-modules-5.36 exp:2026-07-15
CVE-2026-8376 pkg:libperl5.36 exp:2026-07-15
CVE-2026-42496 pkg:perl exp:2026-08-15
CVE-2026-42496 pkg:perl-base exp:2026-08-15
CVE-2026-42496 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-42496 pkg:libperl5.36 exp:2026-08-15
CVE-2026-8376 pkg:perl exp:2026-08-15
CVE-2026-8376 pkg:perl-base exp:2026-08-15
CVE-2026-8376 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-8376 pkg:libperl5.36 exp:2026-08-15
# CVE-2026-13221 - Perl regex trie overflow.
# Packages: perl, perl-base, perl-modules-5.36, libperl5.36.
# Why ignored: upstream confirms Perl 5.36.0 is not affected; the regression
# was introduced after this version. Debian currently marks bookworm as
# vulnerable, which causes Trivy to report a false positive.
# Ref: https://github.com/Perl/perl5/issues/23388
CVE-2026-13221 pkg:perl exp:2026-08-15
CVE-2026-13221 pkg:perl-base exp:2026-08-15
CVE-2026-13221 pkg:perl-modules-5.36 exp:2026-08-15
CVE-2026-13221 pkg:libperl5.36 exp:2026-08-15
# CVE-2025-7458 — SQLite integer overflow.
# Package: libsqlite3-0.
# Why ignored: transitive dependency of CPython's stdlib sqlite3 module. The
# Prowler SDK does not open user-supplied SQLite databases; SQLite usage is
# internal and bounded. No Debian bookworm fix is available.
CVE-2025-7458 pkg:libsqlite3-0 exp:2026-07-15
CVE-2025-7458 pkg:libsqlite3-0 exp:2026-08-15
# CVE-2026-43185 — Linux kernel ksmbd signedness bug.
# Package: linux-libc-dev.
@@ -37,7 +48,7 @@ CVE-2025-7458 pkg:libsqlite3-0 exp:2026-07-15
# not a running kernel. Containers execute against the host kernel, so these
# headers are inert at runtime. The upstream fix landed in kernel 7.0-rc2 and
# has not been backported to Debian's 6.1 LTS line.
CVE-2026-43185 pkg:linux-libc-dev exp:2026-07-15
CVE-2026-43185 pkg:linux-libc-dev exp:2026-08-15
# CVE-2023-45853 — zlib MiniZip integer overflow / heap overflow in
# zipOpenNewFileInZip4_64.
@@ -49,8 +60,8 @@ CVE-2026-43185 pkg:linux-libc-dev exp:2026-07-15
# zlib 1.3.1, available in Debian trixie (13); migrating the base image would
# clear it fully.
# Ref: https://security-tracker.debian.org/tracker/CVE-2023-45853
CVE-2023-45853 pkg:zlib1g exp:2026-07-15
CVE-2023-45853 pkg:zlib1g-dev exp:2026-07-15
CVE-2023-45853 pkg:zlib1g exp:2026-08-15
CVE-2023-45853 pkg:zlib1g-dev exp:2026-08-15
# CVE-2026-55200 — libssh2 out-of-bounds write in ssh2_transport_read() due to
# an unchecked packet_length field in transport.c (heap corruption, possible RCE).
@@ -63,7 +74,7 @@ CVE-2023-45853 pkg:zlib1g-dev exp:2026-07-15
# affected code is unreachable at runtime. Fixed upstream in libssh2 commit
# 97acf3df (PR #2052); no Debian bookworm fix is available yet.
# Ref: https://security-tracker.debian.org/tracker/CVE-2026-55200
CVE-2026-55200 pkg:libssh2-1 exp:2026-07-15
CVE-2026-55200 pkg:libssh2-1 exp:2026-08-15
# --- API container image (api/Dockerfile) ---
# The entries below are specific to the Prowler API image, which ships
@@ -78,13 +89,13 @@ CVE-2026-55200 pkg:libssh2-1 exp:2026-07-15
# at runtime. The vulnerable path requires parsing attacker-controlled XML with
# the affected interpreter, which Prowler does not do with the system Python.
# Full mitigation also needs libexpat >= 2.8.0; no Debian bookworm fix yet.
CVE-2026-7210 pkg:python3.11 exp:2026-07-15
CVE-2026-7210 pkg:python3.11-dev exp:2026-07-15
CVE-2026-7210 pkg:python3.11-minimal exp:2026-07-15
CVE-2026-7210 pkg:libpython3.11 exp:2026-07-15
CVE-2026-7210 pkg:libpython3.11-dev exp:2026-07-15
CVE-2026-7210 pkg:libpython3.11-minimal exp:2026-07-15
CVE-2026-7210 pkg:libpython3.11-stdlib exp:2026-07-15
CVE-2026-7210 pkg:python3.11 exp:2026-08-15
CVE-2026-7210 pkg:python3.11-dev exp:2026-08-15
CVE-2026-7210 pkg:python3.11-minimal exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11 exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-dev exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-minimal exp:2026-08-15
CVE-2026-7210 pkg:libpython3.11-stdlib exp:2026-08-15
# CVE-2026-33278 — Unbound DNSSEC validator use-after-free (DoS, possible RCE).
# CVE-2026-42960 — Unbound DNS cache poisoning via promiscuous additional records.
@@ -94,5 +105,5 @@ CVE-2026-7210 pkg:libpython3.11-stdlib exp:2026-07-15
# vulnerabilities require operating a live Unbound recursive DNSSEC validator
# that processes attacker-influenced DNS responses. Prowler never starts an
# Unbound resolver, so neither code path is reachable. No Debian bookworm fix yet.
CVE-2026-33278 pkg:libunbound8 exp:2026-07-15
CVE-2026-42960 pkg:libunbound8 exp:2026-07-15
CVE-2026-33278 pkg:libunbound8 exp:2026-08-15
CVE-2026-42960 pkg:libunbound8 exp:2026-08-15
+1
View File
@@ -114,6 +114,7 @@ When performing these actions, ALWAYS invoke the corresponding skill FIRST:
| Review PR requirements: template, title conventions, changelog gate | `prowler-pr` |
| Review changelog format and conventions | `prowler-changelog` |
| Reviewing JSON:API compliance | `jsonapi` |
| Reviewing Prowler UI components | `prowler-ui` |
| Reviewing compliance framework PRs | `prowler-compliance-review` |
| Running makemigrations or pgmakemigrations | `django-migration-psql` |
| Syncing compliance framework with upstream catalog | `prowler-compliance` |
+16 -2
View File
@@ -1,12 +1,14 @@
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf AS build
FROM python:3.12.13-slim-bookworm@sha256:8a7e7cc04fd3e2bd787f7f24e22d5d119aa590d429b50c95dfe12b3abe52f48b AS build
LABEL maintainer="https://github.com/prowler-cloud/prowler"
LABEL org.opencontainers.image.source="https://github.com/prowler-cloud/prowler"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
# Opt out of PowerShell telemetry (Application Insights -> dc.services.visualstudio.com)
ENV POWERSHELL_TELEMETRY_OPTOUT=1
ARG TRIVY_VERSION=0.71.0
ARG TRIVY_VERSION=0.71.2
ENV TRIVY_VERSION=${TRIVY_VERSION}
ARG ZIZMOR_VERSION=1.24.1
@@ -95,6 +97,18 @@ RUN uv sync --locked --compile-bytecode && \
# Install PowerShell modules
RUN .venv/bin/python prowler/providers/m365/lib/powershell/m365_powershell.py
USER root
# Remove build-only packages from the final image after Python dependencies are installed.
RUN apt-get purge -y --auto-remove \
build-essential \
pkg-config \
libzstd-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
USER prowler
# Remove deprecated dash dependencies
RUN pip uninstall dash-html-components -y && \
pip uninstall dash-core-components -y
+60 -21
View File
@@ -3,7 +3,7 @@
<img align="center" alt="Prowler logo" src="https://github.com/prowler-cloud/prowler/blob/master/docs/img/prowler-logo-white.png#gh-dark-mode-only" width="50%" height="50%">
</p>
<p align="center">
<b><i>Prowler</b> is the Open Cloud Security Platform trusted by thousands to automate security and compliance in any cloud environment. With hundreds of ready-to-use checks and compliance frameworks, Prowler delivers real-time, customizable monitoring and seamless integrations, making cloud security simple, scalable, and cost-effective for organizations of any size.
<b><i>Prowler</b> is the Open Cloud Security Platform trusted by thousands to automate security and compliance in any cloud environment. With thousands of ready-to-use checks and compliance frameworks, Prowler delivers real-time, customizable monitoring and seamless integrations, making cloud security simple, scalable, and cost-effective for organizations of any size.
</p>
<p align="center">
<b>Secure ANY cloud at AI Speed at <a href="https://prowler.com">prowler.com</i></b>
@@ -21,7 +21,7 @@
<a href="https://pypi.python.org/pypi/prowler/"><img alt="Python Version" src="https://img.shields.io/pypi/pyversions/prowler.svg"></a>
<a href="https://pypistats.org/packages/prowler"><img alt="PyPI Downloads" src="https://img.shields.io/pypi/dw/prowler.svg?label=downloads"></a>
<a href="https://hub.docker.com/r/toniblyx/prowler"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/toniblyx/prowler"></a>
<a href="https://gallery.ecr.aws/prowler-cloud/prowler"><img width="120" height=19" alt="AWS ECR Gallery" src="https://user-images.githubusercontent.com/3985464/151531396-b6535a68-c907-44eb-95a1-a09508178616.png"></a>
<a href="https://gallery.ecr.aws/prowler-cloud/prowler"><img width="120" height="19" alt="AWS ECR Gallery" src="https://user-images.githubusercontent.com/3985464/151531396-b6535a68-c907-44eb-95a1-a09508178616.png"></a>
<a href="https://codecov.io/gh/prowler-cloud/prowler"><img alt="Codecov coverage" src="https://codecov.io/gh/prowler-cloud/prowler/graph/badge.svg?token=OflBGsdpDl"/></a>
<a href="https://insights.linuxfoundation.org/project/prowler-cloud-prowler"><img alt="Linux Foundation insights health score" src="https://insights.linuxfoundation.org/api/badge/health-score?project=prowler-cloud-prowler"/></a>
</p>
@@ -41,7 +41,7 @@
# Description
**Prowler** is the worlds most widely used _Open-Source Cloud Security Platform_ that automates security and compliance across **any cloud environment**. With hundreds of ready-to-use security checks, remediation guidance, and compliance frameworks, Prowler is built to _“Secure ANY Cloud at AI Speed”_. Prowler delivers **AI-driven**, **customizable**, and **easy-to-use** assessments, dashboards, reports, and integrations, making cloud security **simple**, **scalable**, and **cost-effective** for organizations of any size.
**Prowler** is the worlds most widely used _Open-Source Cloud Security Platform_ that automates security and compliance across **any cloud environment**. With thousands of ready-to-use security checks, remediation guidance, and compliance frameworks, Prowler is built to _“Secure ANY Cloud at AI Speed”_. Prowler delivers **AI-driven**, **customizable**, and **easy-to-use** assessments, dashboards, reports, and integrations, making cloud security **simple**, **scalable**, and **cost-effective** for organizations of any size.
Prowler includes hundreds of built-in controls to ensure compliance with standards and frameworks, including:
@@ -54,16 +54,16 @@ Prowler includes hundreds of built-in controls to ensure compliance with standar
- **National Security Standards:** ENS (Spanish National Security Scheme) and KISA ISMS-P (Korean)
- **Custom Security Frameworks:** Tailored to your needs
## Prowler App / Prowler Cloud
## Prowler Cloud & Prowler Local Server
Prowler App / [Prowler Cloud](https://cloud.prowler.com/) is a web-based application that simplifies running Prowler across your cloud provider accounts. It provides a user-friendly interface to visualize the results and streamline your security assessments.
[Prowler Cloud](https://cloud.prowler.com/) and Prowler Local Server, its self-hosted open-source version, are web applications that simplify running Prowler across your cloud provider accounts. They provide a user-friendly interface to visualize the results and streamline your security assessments.
![Prowler App](docs/images/products/overview.png)
![Prowler Cloud](docs/images/products/overview.png)
![Risk Pipeline](docs/images/products/risk-pipeline.png)
![Threat Map](docs/images/products/threat-map.png)
>For more details, refer to the [Prowler App Documentation](https://docs.prowler.com/projects/prowler-open-source/en/latest/#prowler-app-installation)
>For more details, refer to the [Prowler Local Server documentation](https://docs.prowler.com/getting-started/installation/prowler-app)
## Prowler CLI
@@ -73,12 +73,12 @@ prowler <provider>
![Prowler CLI Execution](docs/img/short-display.png)
## Prowler Dashboard
## Prowler Local Dashboard
```console
prowler dashboard
```
![Prowler Dashboard](docs/images/products/dashboard.png)
![Prowler Local Dashboard](docs/images/products/dashboard.png)
## Attack Paths
@@ -121,7 +121,7 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
> For the most accurate and up-to-date information about checks, services, frameworks, and categories, visit [**Prowler Hub**](https://hub.prowler.com).
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) | Support | Interface |
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/user-guide/compliance/tutorials/compliance) | [Categories](https://docs.prowler.com/user-guide/cli/tutorials/misc#categories) | Support | Interface |
|---|---|---|---|---|---|---|
| AWS | 615 | 86 | 47 | 19 | Official | UI, API, CLI |
| Azure | 190 | 22 | 21 | 16 | Official | UI, API, CLI |
@@ -141,6 +141,7 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
| Vercel | 26 | 6 | 1 | 8 | Official | UI, API, CLI |
| Okta | 29 | 8 | 2 | 2 | Official | UI, API, CLI |
| Linode [Contact us](https://prowler.com/contact) | 10 | 3 | 1 | 4 | Unofficial | CLI |
| E2E Networks [Contact us](https://prowler.com/contact) | 27 | 6 | 0 | 2 | Unofficial | CLI |
| Scaleway [Contact us](https://prowler.com/contact) | 1 | 1 | 1 | 1 | Unofficial | CLI |
| StackIT [Contact us](https://prowler.com/contact) | 7 | 2 | 1 | 3 | Unofficial | CLI |
| NHN | 6 | 2 | 2 | 0 | Unofficial | CLI |
@@ -159,11 +160,11 @@ Every AWS provider scan will enqueue an Attack Paths ingestion job automatically
# 💻 Installation
## Prowler App
## Prowler Local Server
Prowler App offers flexible installation methods tailored to various environments:
Prowler Local Server offers flexible installation methods tailored to various environments:
> For detailed instructions on using Prowler App, refer to the [Prowler App Usage Guide](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/prowler-app/).
> For detailed instructions on using Prowler Local Server, refer to the [usage guide](https://docs.prowler.com/user-guide/tutorials/prowler-app).
### Docker Compose
@@ -196,7 +197,7 @@ docker compose up -d
> [!WARNING]
> 🔒 For a secure setup, the API auto-generates a unique key pair, `DJANGO_TOKEN_SIGNING_KEY` and `DJANGO_TOKEN_VERIFYING_KEY`, and stores it in `~/.config/prowler-api` (non-container) or the bound Docker volume in `_data/api` (container). Never commit or reuse static/default keys. To rotate keys, delete the stored key files and restart the API.
Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
Once configured, access Prowler Local Server at http://localhost:3000. Sign up using your email and password to get started.
### Common Issues with Docker Pull Installation
@@ -268,7 +269,7 @@ pnpm run build
pnpm start
```
> Once configured, access the Prowler App at http://localhost:3000. Sign up using your email and password to get started.
> Once configured, access Prowler Local Server at http://localhost:3000. Sign up using your email and password to get started.
#### Pre-commit Hooks Setup
@@ -286,7 +287,7 @@ Prowler CLI is available as a project in [PyPI](https://pypi.org/project/prowler
pip install prowler
prowler -v
```
>For further guidance, refer to [https://docs.prowler.com](https://docs.prowler.com/projects/prowler-open-source/en/latest/#prowler-cli-installation)
>For further guidance, refer to [https://docs.prowler.com](https://docs.prowler.com/getting-started/installation/prowler-cli)
### Containers
@@ -306,7 +307,7 @@ The container images are available here:
- Prowler CLI:
- [DockerHub](https://hub.docker.com/r/prowlercloud/prowler/tags)
- [AWS Public ECR](https://gallery.ecr.aws/prowler-cloud/prowler)
- Prowler App:
- Prowler Local Server:
- [DockerHub - Prowler UI](https://hub.docker.com/r/prowlercloud/prowler-ui/tags)
- [DockerHub - Prowler API](https://hub.docker.com/r/prowlercloud/prowler-api/tags)
@@ -356,17 +357,55 @@ Full configuration, per-provider authentication, and SARIF examples: [Prowler Gi
# ✏️ High level architecture
## Prowler App
**Prowler App** is composed of four key components:
## Prowler Local Server
**Prowler Local Server** is composed of four key components:
- **Prowler UI**: A web-based interface, built with Next.js, providing a user-friendly experience for executing Prowler scans and visualizing results.
- **Prowler API**: A backend service, developed with Django REST Framework, responsible for running Prowler scans and storing the generated results.
- **Prowler SDK**: A Python SDK designed to extend the functionality of the Prowler CLI for advanced capabilities.
- **Prowler MCP Server**: A Model Context Protocol server that provides AI tools for Lighthouse, the AI-powered security assistant. This is a critical dependency for Lighthouse functionality.
![Prowler App Architecture](docs/images/products/prowler-app-architecture.png)
```mermaid
flowchart TB
user([User / Security Team])
cli([Prowler CLI])
<!-- Diagram source: docs/images/products/prowler-app-architecture.mmd — edit there, re-render at https://mermaid.live, and replace the PNG. -->
subgraph APP["Prowler Local Server"]
ui["Prowler UI<br/>(Next.js)"]
api["Prowler API<br/>(Django REST Framework)"]
worker["API Worker<br/>(Celery)"]
beat["API Scheduler<br/>(Celery Beat)"]
mcp["Prowler MCP Server<br/>(Lighthouse AI tools)"]
end
sdk["Prowler SDK<br/>(Python)"]
subgraph DATA["Data Layer"]
pg[("PostgreSQL")]
valkey[("Valkey / Redis")]
neo4j[("Neo4j")]
end
providers["Providers"]
user --> ui
user --> cli
ui -->|REST| api
ui -->|MCP HTTP| mcp
mcp -->|REST| api
api --> pg
api --> valkey
beat -->|enqueue jobs| valkey
valkey -->|dispatch| worker
worker --> pg
worker -->|Attack Paths| neo4j
worker -->|invokes| sdk
cli --> sdk
sdk --> providers
```
<!-- Diagram source: docs/images/products/prowler-app-architecture.mmd — keep this inline block, the docs page getting-started/products/prowler-app.mdx, and the .mmd file in sync. -->
## Prowler CLI
+76 -5
View File
@@ -2,18 +2,89 @@
All notable changes to the **Prowler API** are documented in this file.
## [1.33.0] (Prowler UNRELEASED)
<!-- changelog: release notes start -->
## [1.35.0] (Prowler v5.34.0)
### 🐞 Fixed
- `rls_transaction` now falls back directly to the primary DB for connection-level mid-query read replica failures via `execute_wrapper`, reducing non-streaming read crashes during replica recovery [(#10379)](https://github.com/prowler-cloud/prowler/pull/10379)
- RBAC permission gates now combine permissions from every role assigned to a user in the active tenant [(#11979)](https://github.com/prowler-cloud/prowler/pull/11979)
- `attack-paths-cleanup-stale-scans` now retries worker pings and checks recent scan activity before failing scans and removing temporary databases [(#11986)](https://github.com/prowler-cloud/prowler/pull/11986)
### 🔐 Security
- User role relationship updates are limited to the active tenant to preserve role assignments in other tenants [(#11903)](https://github.com/prowler-cloud/prowler/pull/11903)
- `api` container image removes the unused Debian `libxml2` runtime package and scopes the `CVE-2026-13221` Trivy exception to unaffected Perl 5.36 packages [(#11991)](https://github.com/prowler-cloud/prowler/pull/11991)
---
## [1.34.2] (Prowler v5.33.2)
### 🐞 Fixed
- Attack Paths graph mutations now retry transient Neptune concurrency and deadline failures, while Neo4j mutations use managed transaction retries [(#11968)](https://github.com/prowler-cloud/prowler/pull/11968)
- Attack Paths scans now use bounded child node identifiers for normalized list values in Neo4j and Neptune, preventing Neo4j RANGE index key size failures [(#11969)](https://github.com/prowler-cloud/prowler/pull/11969)
- `scan-summary` aggregation now upserts summaries in deterministic conflict-key order, preventing PostgreSQL deadlocks during concurrent reaggregation [(#11971)](https://github.com/prowler-cloud/prowler/pull/11971)
---
## [1.34.1] (Prowler v5.33.1)
### 🐞 Fixed
- Session tokens are rejected after account password updates [(#11914)](https://github.com/prowler-cloud/prowler/pull/11914)
- Jira dispatch task results now surface user-facing Jira failure messages [(#11925)](https://github.com/prowler-cloud/prowler/pull/11925)
- AWS Attack Paths privilege escalation queries no longer fail on Neo4j with `Aggregation column contains implicit grouping expressions` [(#11939)](https://github.com/prowler-cloud/prowler/pull/11939)
### 🔐 Security
- OpenAI-compatible Lighthouse provider base URLs are restricted before connection checks [(#11940)](https://github.com/prowler-cloud/prowler/pull/11940)
- `LIGHTHOUSE_AI_OPENAI_COMPATIBLE_ALLOWED_HOSTS` environment variable to allow internal hosts as OpenAI-compatible Lighthouse AI base URLs [(#11942)](https://github.com/prowler-cloud/prowler/pull/11942)
---
## [1.34.0] (Prowler v5.33.0)
### 🚀 Added
- Compliance PDF reports no longer require provider credentials: findings are enriched from the provider metadata stored in the database, so reports generate even after the provider secret is deleted or its credentials become invalid [(#11845)](https://github.com/prowler-cloud/prowler/pull/11845)
### 🐞 Fixed
- Provider scans now queue behind active provider scans instead of dispatching concurrently, and resource failed-finding counters retry database conflicts with stable row locking [(#11848)](https://github.com/prowler-cloud/prowler/pull/11848)
---
## [1.33.1] (Prowler v5.32.1)
### 🐞 Fixed
- Attack Paths: Scan rows now have database defaults for `is_migrated` and `sink_backend` so `scan-perform-scheduled` inserts survive deploy skew [(#11826)](https://github.com/prowler-cloud/prowler/pull/11826)
- Invited users now keep their invitation context when completing authentication with Google, GitHub, or SAML, so the invitation is accepted during login [(#11752)](https://github.com/prowler-cloud/prowler/pull/11752)
### 🔐 Security
- User profile updates now allow users to update their own account while requiring user-management permissions to update other users in the same tenant [(#11792)](https://github.com/prowler-cloud/prowler/pull/11792)
- Kubernetes provider credentials now reject kubeconfigs using `exec` authentication in Prowler Cloud, preventing user-supplied commands from running on Cloud workers [(#11753)](https://github.com/prowler-cloud/prowler/pull/11753)
---
## [1.33.0] (Prowler v5.32.0)
### 🚀 Added
- Timestamp precision support for `/api/v1/findings` `inserted_at` and `updated_at` filters [(#11754)](https://github.com/prowler-cloud/prowler/pull/11754)
### 🔄 Changed
- Attack Paths: AWS Neptune is now supported as a persistent sink database, selectable via `ATTACK_PATHS_SINK_DATABASE=neptune` (default `neo4j`), Cartography's (bumped to 0.138.1) per-scan ingest database stays on Neo4j [(#11524)](https://github.com/prowler-cloud/prowler/pull/11524)
---
## [1.32.2] (Prowler UNRELEASED)
- Attack Paths: Scan task now checks the ingest Neo4j database and configured graph sink before starting graph ingestion [(#11743)](https://github.com/prowler-cloud/prowler/pull/11743)
- Disable PowerShell telemetry in the API container image [(#11746)](https://github.com/prowler-cloud/prowler/pull/11746)
### 🐞 Fixed
- Attack Paths: Provider graph cleanup now deletes Neo4j and Neptune relationships in directed batches before deleting nodes [(#11755)](https://github.com/prowler-cloud/prowler/pull/11755)
- `scan-perform` no longer reports an error when a provider is deleted during a running scan [(#11696)](https://github.com/prowler-cloud/prowler/pull/11696)
---
+22 -2
View File
@@ -1,11 +1,13 @@
FROM python:3.12.13-slim-bookworm@sha256:76d4b7b6305788c6b4c6a19d6a22a3921bf802e9af4d5e1e5bd771208dba74bf AS build
FROM python:3.12.13-slim-bookworm@sha256:8a7e7cc04fd3e2bd787f7f24e22d5d119aa590d429b50c95dfe12b3abe52f48b AS build
LABEL maintainer="https://github.com/prowler-cloud/api"
ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
# Opt out of PowerShell telemetry (Application Insights -> dc.services.visualstudio.com)
ENV POWERSHELL_TELEMETRY_OPTOUT=1
ARG TRIVY_VERSION=0.71.0
ARG TRIVY_VERSION=0.71.2
ENV TRIVY_VERSION=${TRIVY_VERSION}
ARG ZIZMOR_VERSION=1.24.1
@@ -102,6 +104,24 @@ RUN uv sync --locked --no-install-project && \
RUN .venv/bin/python .venv/lib/python3.12/site-packages/prowler/providers/m365/lib/powershell/m365_powershell.py
USER root
# Remove build-only packages from the final image after Python dependencies are installed.
RUN apt-get purge -y --auto-remove \
gcc \
g++ \
make \
libxml2-dev \
libxmlsec1-dev \
libxmlsec1-openssl \
pkg-config \
libtool \
libxslt1-dev \
python3-dev \
&& rm -rf /var/lib/apt/lists/*
USER prowler
COPY --chown=prowler:prowler src/backend/ ./backend/
COPY --chown=prowler:prowler docker-entrypoint.sh ./docker-entrypoint.sh
+10
View File
@@ -0,0 +1,10 @@
# Changelog fragments
Each PR adds one small file here instead of editing `CHANGELOG.md` directly, so concurrent PRs never conflict.
- Filename: `<slug>.<type>.md`, e.g. `my-new-check.added.md` (slug is free-form: letters, digits, `.`, `_`, `-`)
- `<type>` is one of: `added`, `changed`, `deprecated`, `removed`, `fixed`, `security`
- Content: one line with the changelog entry text, without the PR link and without a trailing period (the PR link is attached automatically at release time)
- A PR adds as many fragment files as entries it needs, freely mixing types (one file per entry); same-type entries just use different slugs
Fragments are compiled into `CHANGELOG.md` when a release is prepared. Full conventions: `skills/prowler-changelog/SKILL.md`.
@@ -0,0 +1 @@
`attack-paths-scan-perform` Celery tasks now use the configurable long-task time limits instead of the six-hour defaults
@@ -0,0 +1 @@
Jira integration credentials only accept bare Atlassian site names containing letters, numbers, and hyphens
@@ -0,0 +1 @@
Social account linking requires a verified matching email from both the identity provider and the existing user account without sending account connection notifications
+2 -2
View File
@@ -71,7 +71,7 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.33.0"
version = "1.36.0"
# Shared ruff baseline (kept in sync with mcp_server/pyproject.toml).
# target-version tracks this project's lowest supported Python.
@@ -218,7 +218,6 @@ constraint-dependencies = [
"debugpy==1.8.20",
"decorator==5.2.1",
"defusedxml==0.7.1",
"detect-secrets==1.5.0",
"dill==0.4.1",
"distro==1.9.0",
"dj-rest-auth==7.0.1",
@@ -301,6 +300,7 @@ constraint-dependencies = [
"jsonschema==4.23.0",
"jsonschema-specifications==2025.9.1",
"keystoneauth1==5.13.0",
"kingfisher-bin==1.104.0",
"kiwisolver==1.4.9",
"knack==0.11.0",
"kombu==5.6.2",
+68 -23
View File
@@ -1,3 +1,5 @@
from allauth.account.models import EmailAddress
from allauth.core.exceptions import ImmediateHttpResponse
from allauth.socialaccount.adapter import DefaultSocialAccountAdapter
from api.db_router import MainRouter
from api.db_utils import rls_transaction
@@ -9,7 +11,9 @@ from api.models import (
User,
UserRoleRelationship,
)
from api.utils import accept_invitation_for_user
from django.db import transaction
from django.http import HttpResponseForbidden
class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
@@ -20,9 +24,30 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
except User.DoesNotExist:
return None
@staticmethod
def _get_invitation_token(request):
for source_name in ("data", "POST"):
data = getattr(request, source_name, None) or {}
if not hasattr(data, "get"):
continue
invitation_token = data.get("invitation_token")
if invitation_token:
return invitation_token
wrapped_request = getattr(request, "_request", None)
if wrapped_request and wrapped_request is not request:
return ProwlerSocialAccountAdapter._get_invitation_token(wrapped_request)
return None
def pre_social_login(self, request, sociallogin):
# Link existing accounts with the same email address
email = sociallogin.account.extra_data.get("email")
# The provider account is already bound, so no email-based linking is needed.
if sociallogin.account.pk:
return
# Prefer the normalized email populated by allauth. GitHub can return the
# primary email separately from the profile stored in extra_data.
email = sociallogin.user.email or sociallogin.account.extra_data.get("email")
if sociallogin.provider.id == "saml":
# For SAML, the asserted NameID email cannot be trusted on its own:
# any tenant can claim any email domain in its SAML configuration. To
@@ -63,6 +88,17 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
if email:
existing_user = self.get_user_by_email(email)
if existing_user:
email_is_verified = EmailAddress.objects.filter(
user=existing_user,
email__iexact=email,
verified=True,
).exists()
provider_verified_email = any(
address.verified and address.email.casefold() == email.casefold()
for address in sociallogin.email_addresses
)
if not email_is_verified or not provider_verified_email:
raise ImmediateHttpResponse(HttpResponseForbidden())
sociallogin.connect(request, existing_user)
def save_user(self, request, sociallogin, form=None):
@@ -83,29 +119,38 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
user.name = social_account_name
user.save(using=MainRouter.admin_db)
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
invitation_token = self._get_invitation_token(request)
if invitation_token:
invitation, _ = accept_invitation_for_user(
user=user,
role=role,
tenant_id=tenant.id,
invitation_token=invitation_token,
)
request.prowler_invitation_token = invitation_token
request.prowler_invitation_tenant_id = str(invitation.tenant_id)
else:
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
else:
request.session["saml_user_created"] = str(user.id)
@@ -96,6 +96,11 @@ def inject_provider_label(cypher: str, provider_id: str) -> str:
node pattern.
"""
label = get_provider_label(provider_id)
return inject_label(cypher, label)
def inject_label(cypher: str, label: str) -> str:
"""Rewrite a Cypher query to append a label to every node pattern."""
# Step 1: Protect strings and comments (single pass, leftmost-first)
protected: list[str] = []
@@ -106,6 +106,31 @@ def verify_connectivity() -> None:
sink_module.get_backend().verify_connectivity()
def verify_scan_databases_available() -> None:
"""Raise if either graph database needed by an Attack Paths scan is unavailable."""
errors: list[str] = []
first_error: Exception | None = None
try:
ingest.get_driver().verify_connectivity()
except Exception as exc:
errors.append(f"ingest Neo4j: {exc}")
first_error = exc
try:
get_driver().verify_connectivity()
except Exception as exc:
errors.append(f"sink {settings.ATTACK_PATHS_SINK_DATABASE}: {exc}")
if first_error is None:
first_error = exc
if errors:
raise RuntimeError(
"Attack Paths graph database unavailable before scan start: "
+ "; ".join(errors)
) from first_error
def get_uri() -> str:
"""Return the sink URI. Retained for backwards compatibility."""
if settings.ATTACK_PATHS_SINK_DATABASE == "neptune":
+26 -13
View File
@@ -418,7 +418,8 @@ AWS_APPRUNNER_PRIVESC_UPDATE_SERVICE = AttackPathsQueryDefinition(
// Find existing App Runner services with roles attached (potential targets)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'tasks.apprunner.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -523,7 +524,8 @@ AWS_BEDROCK_PRIVESC_INVOKE_CODE_INTERPRETER = AttackPathsQueryDefinition(
// Find roles that trust the Bedrock AgentCore service (already attached to existing code interpreters)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'bedrock-agentcore.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -607,7 +609,8 @@ AWS_CLOUDFORMATION_PRIVESC_UPDATE_STACK = AttackPathsQueryDefinition(
// Find roles that trust CloudFormation service (already attached to existing stacks)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'cloudformation.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -753,7 +756,8 @@ AWS_CLOUDFORMATION_PRIVESC_CHANGESET = AttackPathsQueryDefinition(
// Find roles that trust CloudFormation service (already attached to existing stacks)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'cloudformation.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -844,7 +848,8 @@ AWS_CODEBUILD_PRIVESC_START_BUILD = AttackPathsQueryDefinition(
// Find roles that trust CodeBuild service (already attached to existing projects)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'codebuild.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -880,7 +885,8 @@ AWS_CODEBUILD_PRIVESC_START_BUILD_BATCH = AttackPathsQueryDefinition(
// Find roles that trust CodeBuild service (already attached to existing projects)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'codebuild.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1096,7 +1102,8 @@ AWS_EC2_PRIVESC_MODIFY_INSTANCE_ATTRIBUTE = AttackPathsQueryDefinition(
// Find EC2 instances with instance profiles (potential targets)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1187,7 +1194,8 @@ AWS_EC2_PRIVESC_LAUNCH_TEMPLATE = AttackPathsQueryDefinition(
// Find launch templates in the account (potential targets)
MATCH path_target = (aws)--(template:LaunchTemplate)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1223,7 +1231,8 @@ AWS_EC2INSTANCECONNECT_PRIVESC_SEND_SSH_PUBLIC_KEY = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1539,7 +1548,8 @@ AWS_ECS_PRIVESC_EXECUTE_COMMAND = AttackPathsQueryDefinition(
// Target: roles already attached to running tasks (trust ECS tasks service)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'ecs-tasks.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1622,7 +1632,8 @@ AWS_GLUE_PRIVESC_UPDATE_DEV_ENDPOINT = AttackPathsQueryDefinition(
// Find roles that trust Glue service (already attached to existing dev endpoints)
MATCH path_target = (aws)--(target_role:AWSRole)-[:TRUSTS_AWS_PRINCIPAL]->(:AWSPrincipal {{arn: 'glue.amazonaws.com'}})
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -3337,7 +3348,8 @@ AWS_SSM_PRIVESC_START_SESSION = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -3373,7 +3385,8 @@ AWS_SSM_PRIVESC_SEND_COMMAND = AttackPathsQueryDefinition(
// Find EC2 instances with attached roles (targets for credential theft via IMDS)
MATCH path_target = (aws)--(ec2:EC2Instance)-[:STS_ASSUMEROLE_ALLOW]->(target_role:AWSRole)
WITH principal_paths + collect(DISTINCT path_target) AS paths
WITH principal_paths, collect(DISTINCT path_target) AS target_paths
WITH principal_paths + target_paths AS paths
UNWIND paths AS p
UNWIND nodes(p) AS n
@@ -1,4 +1,6 @@
import logging
import random
import time
from collections.abc import Callable
from typing import Any
@@ -9,17 +11,19 @@ logger = logging.getLogger(__name__)
class RetryableSession:
"""
Wrapper around `neo4j.Session` that retries `neo4j.exceptions.ServiceUnavailable` errors.
"""
"""Wrapper around ``neo4j.Session`` with a refreshable retry policy."""
def __init__(
self,
session_factory: Callable[[], neo4j.Session],
max_retries: int,
retry_if: Callable[[Exception], bool] | None = None,
initial_retry_delay_seconds: float = 0,
) -> None:
self._session_factory = session_factory
self._max_retries = max(0, max_retries)
self._retry_if = retry_if
self._initial_retry_delay_seconds = max(0.0, initial_retry_delay_seconds)
self._session = self._session_factory()
def close(self) -> None:
@@ -56,24 +60,47 @@ class RetryableSession:
method = getattr(self._session, method_name)
return method(*args, **kwargs)
except (
BrokenPipeError,
ConnectionResetError,
neo4j.exceptions.ServiceUnavailable,
) as exc: # pragma: no cover - depends on infra
except Exception as exc:
if not self._should_retry(exc):
raise
last_exc = exc
attempt += 1
if attempt > self._max_retries:
raise
delay = self._retry_delay(attempt)
logger.warning(
f"Neo4j session {method_name} failed with {type(exc).__name__} ({attempt}/{self._max_retries} attempts). Retrying..."
"Graph session %s failed with %s; retry %s/%s in %.3fs",
method_name,
type(exc).__name__,
attempt,
self._max_retries,
delay,
)
self._refresh_session()
if delay:
time.sleep(delay)
raise last_exc if last_exc else RuntimeError("Unexpected retry loop exit")
def _should_retry(self, exc: Exception) -> bool:
if isinstance(
exc,
(
BrokenPipeError,
ConnectionResetError,
neo4j.exceptions.ServiceUnavailable,
),
):
return True
return self._retry_if(exc) if self._retry_if else False
def _retry_delay(self, attempt: int) -> float:
max_delay = self._initial_retry_delay_seconds * (2**attempt)
return random.uniform(max_delay / 2, max_delay) if max_delay else 0
def _refresh_session(self) -> None:
if self._session is not None:
try:
@@ -0,0 +1,81 @@
"""Shared batched deletion helpers for sink backends."""
import logging
import time
from typing import Any
RELATIONSHIP_DELETE_QUERY_TEMPLATES = {
"outgoing relationship": """
MATCH (n:`{provider_label}`)-[r]->()
WITH r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
"incoming relationship": """
MATCH (n:`{provider_label}`)<-[r]-()
WITH r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
}
NODE_DELETE_QUERY_TEMPLATE = """
MATCH (n:{provider_resource_label}:`{provider_label}`)
WITH n LIMIT $batch_size
DELETE n
RETURN COUNT(n) AS deleted_nodes_count
"""
def delete_batches(
*,
session: Any,
logger: logging.Logger,
log_target: str,
provider_id: str,
query: str,
phase: str,
count_key: str,
total_key: str,
deleted_key: str,
initial_total: int,
batch_size: int,
drop_t0: float,
) -> tuple[int, int]:
def delete_batch(tx: Any) -> int:
record = tx.run(query, {"batch_size": batch_size}).single()
return (record[count_key] if record else 0) or 0
deleted_total = initial_total
batches = 0
while True:
logger.info(
"Deleting %s batch from %s "
"(provider=%s, batch=%s, total_%s=%s, elapsed=%.3fs)",
phase,
log_target,
provider_id,
batches + 1,
total_key,
deleted_total,
time.perf_counter() - drop_t0,
)
deleted = session.execute_write(delete_batch)
if deleted == 0:
return deleted_total, batches
batches += 1
deleted_total += deleted
logger.info(
"Deleted %s batch from %s "
"(provider=%s, batch=%s, %s=%s, total_%s=%s, elapsed=%.3fs)",
phase,
log_target,
provider_id,
batches,
deleted_key,
deleted,
total_key,
deleted_total,
time.perf_counter() - drop_t0,
)
+46 -83
View File
@@ -17,6 +17,11 @@ import neo4j
import neo4j.exceptions
from api.attack_paths.retryable_session import RetryableSession
from api.attack_paths.sink.base import SinkDatabase
from api.attack_paths.sink.drop import (
NODE_DELETE_QUERY_TEMPLATE,
RELATIONSHIP_DELETE_QUERY_TEMPLATES,
delete_batches,
)
from config.env import env
from django.conf import settings
@@ -204,10 +209,8 @@ class Neo4jSink(SinkDatabase):
)
provider_label = get_provider_label(provider_id)
deleted_nodes = 0
deleted_relationships = 0
relationship_batches = 0
node_batches = 0
deleted_nodes = deleted_relationships = 0
relationship_batches = node_batches = 0
drop_t0 = time.perf_counter()
logger.info(
@@ -232,84 +235,44 @@ class Neo4jSink(SinkDatabase):
database,
provider_id,
)
# Phase 1: delete relationships incident to provider nodes in
# batches. The undirected pattern matches an edge between two
# provider nodes from both ends, so `DISTINCT r` dedupes it to
# delete a full batch of unique relationships each round.
deleted_count = 1
while deleted_count > 0:
next_batch = relationship_batches + 1
logger.info(
"Deleting relationship batch from Neo4j sink database %s "
"(provider=%s, batch=%s, total_rels=%s, elapsed=%.3fs)",
database,
provider_id,
next_batch,
deleted_relationships,
time.perf_counter() - drop_t0,
log_target = f"Neo4j sink database {database}"
for (
phase,
query_template,
) in RELATIONSHIP_DELETE_QUERY_TEMPLATES.items():
deleted_relationships, phase_batches = delete_batches(
session=session,
logger=logger,
log_target=log_target,
provider_id=provider_id,
query=query_template.format(provider_label=provider_label),
phase=phase,
count_key="deleted_rels_count",
total_key="rels",
deleted_key="deleted_rels",
initial_total=deleted_relationships,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
result = session.run(
f"""
MATCH (:`{provider_label}`)-[r]-()
WITH DISTINCT r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
{"batch_size": BATCH_SIZE},
)
deleted_count = result.single().get("deleted_rels_count", 0)
if deleted_count > 0:
relationship_batches += 1
deleted_relationships += deleted_count
logger.info(
"Deleted relationship batch from Neo4j sink database %s "
"(provider=%s, batch=%s, deleted_rels=%s, "
"total_rels=%s, elapsed=%.3fs)",
database,
provider_id,
relationship_batches,
deleted_count,
deleted_relationships,
time.perf_counter() - drop_t0,
)
relationship_batches += phase_batches
# Phase 2: delete the now relationship-free nodes in batches.
deleted_count = 1
while deleted_count > 0:
next_batch = node_batches + 1
logger.info(
"Deleting node batch from Neo4j sink database %s "
"(provider=%s, batch=%s, total_nodes=%s, elapsed=%.3fs)",
database,
provider_id,
next_batch,
deleted_nodes,
time.perf_counter() - drop_t0,
)
result = session.run(
f"""
MATCH (n:{PROVIDER_RESOURCE_LABEL}:`{provider_label}`)
WITH n LIMIT $batch_size
DELETE n
RETURN COUNT(n) AS deleted_nodes_count
""",
{"batch_size": BATCH_SIZE},
)
deleted_count = result.single().get("deleted_nodes_count", 0)
if deleted_count > 0:
node_batches += 1
deleted_nodes += deleted_count
logger.info(
"Deleted node batch from Neo4j sink database %s "
"(provider=%s, batch=%s, deleted_nodes=%s, "
"total_nodes=%s, elapsed=%.3fs)",
database,
provider_id,
node_batches,
deleted_count,
deleted_nodes,
time.perf_counter() - drop_t0,
)
deleted_nodes, node_batches = delete_batches(
session=session,
logger=logger,
log_target=log_target,
provider_id=provider_id,
query=NODE_DELETE_QUERY_TEMPLATE.format(
provider_label=provider_label,
provider_resource_label=PROVIDER_RESOURCE_LABEL,
),
phase="node",
count_key="deleted_nodes_count",
total_key="nodes",
deleted_key="deleted_nodes",
initial_total=0,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
except GraphDatabaseQueryException as exc:
if exc.code == DATABASE_NOT_FOUND_CODE:
@@ -392,7 +355,7 @@ class Neo4jSink(SinkDatabase):
f"ON (n.`{PROVIDER_ELEMENT_ID_PROPERTY}`)"
)
with self.get_session(database) as session:
session.run(query).consume()
session.execute_write(lambda tx: tx.run(query).consume())
def write_nodes(
self,
@@ -414,7 +377,7 @@ class Neo4jSink(SinkDatabase):
SET n += row.props
"""
with self.get_session(database) as session:
session.run(query, {"rows": rows}).consume()
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
def write_relationships(
self,
@@ -440,7 +403,7 @@ class Neo4jSink(SinkDatabase):
SET r += row.props
"""
with self.get_session(database) as session:
session.run(query, {"rows": rows}).consume()
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
# For compatibility with test harnesses that patch the concrete driver
def get_driver(self) -> neo4j.Driver:
@@ -27,6 +27,11 @@ import neo4j
import neo4j.exceptions
from api.attack_paths.retryable_session import RetryableSession
from api.attack_paths.sink.base import SinkDatabase
from api.attack_paths.sink.drop import (
NODE_DELETE_QUERY_TEMPLATE,
RELATIONSHIP_DELETE_QUERY_TEMPLATES,
delete_batches,
)
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
from botocore.session import Session as BotoSession
@@ -54,17 +59,29 @@ CONNECTION_TIMEOUT = env.int("NEPTUNE_CONNECTION_TIMEOUT", default=10)
# Roll connections hourly so SigV4 rotations and cert refreshes don't strand long-lived pool entries
MAX_CONNECTION_LIFETIME = env.int("NEPTUNE_MAX_CONNECTION_LIFETIME", default=3600)
MAX_CONNECTION_POOL_SIZE = env.int("NEPTUNE_MAX_CONNECTION_POOL_SIZE", default=50)
NEPTUNE_WRITE_RETRY_DELAY_SECONDS = 2
READ_EXCEPTION_CODES = [
"Neo.ClientError.Statement.AccessMode",
"Neo.ClientError.Procedure.ProcedureNotFound",
]
CLIENT_STATEMENT_EXCEPTION_PREFIX = "Neo.ClientError.Statement."
RETRYABLE_WRITE_ERROR_FRAGMENTS = (
"Operation failed due to conflicting concurrent operations",
"Operation terminated (deadline exceeded)",
)
# Refresh 60s before the 5-minute SigV4 window closes
SIGV4_TOKEN_LIFETIME_MINUTES = 4
def _is_retryable_write_error(exc: Exception) -> bool:
if not isinstance(exc, neo4j.exceptions.Neo4jError):
return False
message = exc.message or ""
return any(fragment in message for fragment in RETRYABLE_WRITE_ERROR_FRAGMENTS)
class NeptuneSink(SinkDatabase):
"""Neptune-backed sink. Single database; isolation is label-based."""
@@ -200,11 +217,16 @@ class NeptuneSink(SinkDatabase):
session_wrapper: RetryableSession | None = None
try:
is_write_session = default_access_mode != neo4j.READ_ACCESS
session_wrapper = RetryableSession(
session_factory=lambda: driver.session(
default_access_mode=default_access_mode
),
max_retries=SERVICE_UNAVAILABLE_MAX_RETRIES,
retry_if=_is_retryable_write_error if is_write_session else None,
initial_retry_delay_seconds=(
NEPTUNE_WRITE_RETRY_DELAY_SECONDS if is_write_session else 0
),
)
yield session_wrapper
@@ -296,78 +318,40 @@ class NeptuneSink(SinkDatabase):
"Opened Neptune writer session for provider graph drop (provider=%s)",
provider_id,
)
while True:
next_batch = relationship_batches + 1
logger.info(
"Deleting relationship batch from Neptune sink "
"(provider=%s, batch=%s, total_rels=%s, elapsed=%.3fs)",
provider_id,
next_batch,
deleted_relationships,
time.perf_counter() - drop_t0,
)
result = session.run(
f"""
MATCH (:`{provider_label}`)-[r]-()
WITH DISTINCT r LIMIT $batch_size
DELETE r
RETURN COUNT(r) AS deleted_rels_count
""",
{"batch_size": BATCH_SIZE},
)
record = result.single()
deleted_rels = (record["deleted_rels_count"] if record else 0) or 0
if deleted_rels == 0:
break
relationship_batches += 1
deleted_relationships += deleted_rels
logger.info(
"Deleted relationship batch from Neptune sink "
"(provider=%s, batch=%s, deleted_rels=%s, total_rels=%s, "
"elapsed=%.3fs)",
provider_id,
relationship_batches,
deleted_rels,
deleted_relationships,
time.perf_counter() - drop_t0,
for phase, query_template in RELATIONSHIP_DELETE_QUERY_TEMPLATES.items():
deleted_relationships, phase_batches = delete_batches(
session=session,
logger=logger,
log_target="Neptune sink",
provider_id=provider_id,
query=query_template.format(provider_label=provider_label),
phase=phase,
count_key="deleted_rels_count",
total_key="rels",
deleted_key="deleted_rels",
initial_total=deleted_relationships,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
relationship_batches += phase_batches
deleted_nodes = 0
while True:
next_batch = node_batches + 1
logger.info(
"Deleting node batch from Neptune sink "
"(provider=%s, batch=%s, total_nodes=%s, elapsed=%.3fs)",
provider_id,
next_batch,
deleted_nodes,
time.perf_counter() - drop_t0,
)
result = session.run(
f"""
MATCH (n:`{PROVIDER_RESOURCE_LABEL}`:`{provider_label}`)
WITH n LIMIT $batch_size
DELETE n
RETURN COUNT(n) AS deleted_nodes_count
""",
{"batch_size": BATCH_SIZE},
)
record = result.single()
deleted = (record["deleted_nodes_count"] if record else 0) or 0
if deleted == 0:
break
node_batches += 1
deleted_nodes += deleted
logger.info(
"Deleted node batch from Neptune sink "
"(provider=%s, batch=%s, deleted_nodes=%s, total_nodes=%s, "
"elapsed=%.3fs)",
provider_id,
node_batches,
deleted,
deleted_nodes,
time.perf_counter() - drop_t0,
)
deleted_nodes, node_batches = delete_batches(
session=session,
logger=logger,
log_target="Neptune sink",
provider_id=provider_id,
query=NODE_DELETE_QUERY_TEMPLATE.format(
provider_label=provider_label,
provider_resource_label=PROVIDER_RESOURCE_LABEL,
),
phase="node",
count_key="deleted_nodes_count",
total_key="nodes",
deleted_key="deleted_nodes",
initial_total=0,
batch_size=BATCH_SIZE,
drop_t0=drop_t0,
)
logger.info(
"Finished dropping provider graph from Neptune sink "
@@ -438,7 +422,7 @@ class NeptuneSink(SinkDatabase):
SET n.`{PROVIDER_ELEMENT_ID_PROPERTY}` = row.provider_element_id
"""
with self.get_session() as session:
session.run(query, {"rows": rows}).consume()
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
def write_relationships(
self,
@@ -462,7 +446,7 @@ class NeptuneSink(SinkDatabase):
SET r += row.props
"""
with self.get_session() as session:
session.run(query, {"rows": rows}).consume()
session.execute_write(lambda tx: tx.run(query, {"rows": rows}).consume())
# Test helpers
@@ -152,10 +152,10 @@ def execute_custom_query(
scan: AttackPathsScan,
) -> dict[str, Any]:
# Defense-in-depth for custom queries:
# 1. `neo4j.READ_ACCESS` prevents mutations at the driver level
# 2. `inject_provider_label()` regex-based label injection scopes node patterns
# 3. `_serialize_graph()` post-query filter drops nodes without the provider label
# 4. `USING QUERY:TIMEOUTMILLISECONDS` on Neptune server-side runaway cutoff
# 1. `neo4j.READ_ACCESS` - prevents mutations at the driver level
# 2. `inject_provider_label()` - regex-based label injection scopes node patterns
# 3. `_serialize_graph()` - post-query filter drops nodes without the provider label
# 4. `USING QUERY:TIMEOUTMILLISECONDS` on Neptune - server-side runaway cutoff
#
# Layer 2 is best-effort (regex can't fully parse Cypher);
# layer 3 is the safety net that guarantees provider isolation.
+273 -50
View File
@@ -2,7 +2,7 @@ import re
import secrets
import time
import uuid
from contextlib import contextmanager
from contextlib import ExitStack, contextmanager, nullcontext
from datetime import UTC, datetime, timedelta
from api.db_router import (
@@ -48,6 +48,140 @@ REPLICA_MAX_ATTEMPTS = env.int("POSTGRES_REPLICA_MAX_ATTEMPTS", default=3)
REPLICA_RETRY_BASE_DELAY = env.float("POSTGRES_REPLICA_RETRY_BASE_DELAY", default=0.5)
SET_CONFIG_QUERY = "SELECT set_config(%s, %s::text, TRUE);"
SET_TRANSACTION_READ_ONLY_QUERY = "SET TRANSACTION READ ONLY;"
REPLICA_CONNECTION_SQLSTATE_PREFIXES = ("08",)
REPLICA_CONNECTION_SQLSTATES = {"57P01", "57P02", "57P03"}
REPLICA_NON_FAILOVER_SQLSTATES = {"57014", "40001", "40P01"}
REPLICA_CONNECTION_ERROR_MESSAGES = (
"ssl syscall",
"eof detected",
"server closed the connection",
"connection already closed",
"connection not open",
"could not connect to server",
"connection refused",
"connection reset",
"connection timed out",
"lost synchronization",
"terminating connection",
"database system is starting up",
"database system is shutting down",
"database system is in recovery mode",
)
REPLICA_NON_FAILOVER_ERROR_MESSAGES = (
"canceling statement due to user request",
"deadlock detected",
"could not serialize access",
)
def _iter_exception_chain(error: BaseException):
seen = set()
pending = [error]
while pending:
current = pending.pop(0)
if current is None or id(current) in seen:
continue
seen.add(id(current))
yield current
cause = getattr(current, "__cause__", None)
context = getattr(current, "__context__", None)
if cause is not None:
pending.append(cause)
if context is not None:
pending.append(context)
for arg in getattr(current, "args", ()):
if isinstance(arg, BaseException):
pending.append(arg)
def _get_exception_sqlstate(error: BaseException) -> str | None:
for attr in ("pgcode", "sqlstate"):
sqlstate = getattr(error, attr, None)
if sqlstate:
return sqlstate
diag = getattr(error, "diag", None)
if diag is not None:
sqlstate = getattr(diag, "sqlstate", None)
if sqlstate:
return sqlstate
return None
def _is_replica_connection_failure(error: BaseException) -> bool:
"""
Return True only for replica failures where retrying on primary is safe.
Query cancellations, serialization failures, and deadlocks should surface to
callers because replaying them can hide real query or concurrency problems.
"""
messages = []
sqlstates = set()
for chained_error in _iter_exception_chain(error):
sqlstate = _get_exception_sqlstate(chained_error)
if sqlstate:
sqlstates.add(sqlstate)
messages.append(str(chained_error).lower())
if sqlstates & REPLICA_NON_FAILOVER_SQLSTATES:
return False
if any(
sqlstate.startswith(REPLICA_CONNECTION_SQLSTATE_PREFIXES)
or sqlstate in REPLICA_CONNECTION_SQLSTATES
for sqlstate in sqlstates
):
return True
message = " ".join(messages)
if any(marker in message for marker in REPLICA_NON_FAILOVER_ERROR_MESSAGES):
return False
return any(marker in message for marker in REPLICA_CONNECTION_ERROR_MESSAGES)
def _strip_leading_sql_comments(sql: str) -> str:
if not isinstance(sql, str):
return ""
sql_text = sql.lstrip()
while True:
if sql_text.startswith("--"):
newline_index = sql_text.find("\n")
if newline_index == -1:
return ""
sql_text = sql_text[newline_index + 1 :].lstrip()
continue
if sql_text.startswith("/*"):
comment_end_index = sql_text.find("*/", 2)
if comment_end_index == -1:
return ""
sql_text = sql_text[comment_end_index + 2 :].lstrip()
continue
return sql_text
def _is_safe_primary_replay(sql: str, many: bool) -> bool:
if many:
return False
sql_text = _strip_leading_sql_comments(sql)
if not re.match(r"(?is)^SELECT\b", sql_text):
return False
return not any(
re.search(pattern, sql_text, re.IGNORECASE | re.DOTALL)
for pattern in (
r"\bINTO\b",
r"\bFOR\s+(?:NO\s+KEY\s+)?UPDATE\b",
r"\bFOR\s+(?:KEY\s+)?SHARE\b",
)
)
@contextmanager
@@ -77,14 +211,36 @@ def rls_transaction(
retry_on_replica: bool = True,
):
"""
Creates a new database transaction setting the given configuration value for Postgres RLS. It validates the
if the value is a valid UUID.
Context manager that opens an RLS-scoped database transaction.
Sets a Postgres configuration variable (``set_config``) so that Row-Level
Security policies can filter by tenant. When *using* points to a read
replica and *retry_on_replica* is True, replica failures are handled in two
places:
1. **Pre-yield** (connection-setup failures): the function retries
up to ``REPLICA_MAX_ATTEMPTS`` times on the replica, then falls
back to the primary DB.
2. **Post-yield** (mid-query failures): an ``execute_wrapper``
intercepts connection-level ``OperationalError`` during
``cursor.execute()`` calls and falls back directly to the primary DB
for single ``SELECT`` statements. The primary fallback transaction is
read-only, and unsafe statements keep raising the original error.
The wrapper swaps the inner cursor so ``fetchall()`` / ``fetchone()``
read from the new connection transparently.
Limitation: server-side cursors (``.iterator()``) fetch rows via
``fetchmany()``, which the wrapper does not intercept. Call sites
that iterate large result sets with ``.iterator()`` on the replica
should add their own retry logic.
Args:
value (str): Database configuration parameter value.
parameter (str): Database configuration parameter name, by default is 'api.tenant_id'.
using (str | None): Optional database alias to run the transaction against. Defaults to the
active read alias (if any) or Django's default connection.
value: Database configuration parameter value (must be a valid UUID).
parameter: Database configuration parameter name.
using: Optional database alias. Defaults to the active read
alias or Django's default connection.
retry_on_replica: Whether replica setup failures can retry and
connection-level mid-query failures can fall back to primary.
"""
requested_alias = using or get_read_db_alias()
db_alias = requested_alias or DEFAULT_DB_ALIAS
@@ -92,54 +248,121 @@ def rls_transaction(
db_alias = DEFAULT_DB_ALIAS
alias = db_alias
is_replica = READ_REPLICA_ALIAS and alias == READ_REPLICA_ALIAS
max_attempts = REPLICA_MAX_ATTEMPTS if is_replica and retry_on_replica else 1
is_replica = bool(READ_REPLICA_ALIAS and alias == READ_REPLICA_ALIAS)
can_failover = is_replica and retry_on_replica
replica_alias = alias # captured before the loop mutates alias
max_attempts = (REPLICA_MAX_ATTEMPTS + 1) if can_failover else 1
for attempt in range(1, max_attempts + 1):
router_token = None
yielded_cursor = False
# State shared between the generator and the _query_failover closure.
# The fallback transaction.atomic() is registered into fallback_stack
# via enter_context so its __exit__ runs when the outer with-ExitStack
# block exits, with the right exc_info. No manual __enter__/__exit__.
_fallback = {"succeeded": False, "token": None, "caller_exited_cleanly": False}
# On final attempt, fallback to primary
if attempt == max_attempts and is_replica:
logger.warning(
f"RLS transaction failed after {attempt - 1} attempts on replica, "
f"falling back to primary DB"
)
alias = DEFAULT_DB_ALIAS
with ExitStack() as fallback_stack:
conn = connections[alias]
try:
if alias != DEFAULT_DB_ALIAS:
router_token = set_read_db_alias(alias)
def _query_failover(execute, sql, params, many, context):
"""execute_wrapper: replay failed replica queries on the primary DB."""
try:
return execute(sql, params, many, context)
except OperationalError as err:
if not _is_replica_connection_failure(err):
raise
if not _is_safe_primary_replay(sql, many):
raise
with transaction.atomic(using=alias):
with conn.cursor() as cursor:
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yielded_cursor = True
yield cursor
return
except OperationalError as e:
if yielded_cursor:
raise
# If on primary or max attempts reached, raise
if not is_replica or attempt == max_attempts:
raise
try:
connections[replica_alias].close()
except Exception:
pass # Best-effort; connection may already be dead
# Retry with exponential backoff
delay = REPLICA_RETRY_BASE_DELAY * (2 ** (attempt - 1))
logger.info(
f"RLS transaction failed on replica (attempt {attempt}/{max_attempts}), "
f"retrying in {delay}s. Error: {e}"
)
time.sleep(delay)
finally:
if router_token is not None:
reset_read_db_alias(router_token)
logger.warning(
"Mid-query replica connection failure, falling back to primary DB"
)
primary = connections[DEFAULT_DB_ALIAS]
primary.ensure_connection()
fallback_stack.enter_context(transaction.atomic(using=DEFAULT_DB_ALIAS))
fallback_cursor = primary.cursor()
fallback_stack.callback(fallback_cursor.close)
fallback_cursor.execute(SET_TRANSACTION_READ_ONLY_QUERY)
fallback_cursor.execute(SET_CONFIG_QUERY, [parameter, value])
_fallback["token"] = set_read_db_alias(DEFAULT_DB_ALIAS)
fallback_cursor.execute(sql, params)
context["cursor"].db = primary
context["cursor"].cursor = fallback_cursor.cursor
_fallback["succeeded"] = True
return None
for attempt in range(1, max_attempts + 1):
router_token = None
yielded_cursor = False
# On final attempt, fall back to primary
if attempt == max_attempts and can_failover:
if attempt > 1:
logger.warning(
f"RLS transaction failed after {attempt - 1} attempts on replica, "
f"falling back to primary DB"
)
alias = DEFAULT_DB_ALIAS
conn = connections[alias]
try:
if alias != DEFAULT_DB_ALIAS:
router_token = set_read_db_alias(alias)
with transaction.atomic(using=alias):
with conn.cursor() as cursor:
try:
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
wrapper_cm = (
conn.execute_wrapper(_query_failover)
if can_failover and alias == replica_alias
else nullcontext()
)
with wrapper_cm:
yielded_cursor = True
yield cursor
_fallback["caller_exited_cleanly"] = True
return
except OperationalError as e:
if yielded_cursor:
if _fallback["succeeded"] and _fallback["caller_exited_cleanly"]:
# Caller's queries succeeded on primary via failover.
# This error is transaction.atomic() cleanup on the
# dead replica connection, suppress it.
return
raise
if not can_failover or attempt == max_attempts:
raise
try:
connections[alias].close()
except Exception:
pass # Best-effort; connection may already be dead
# Retry with exponential backoff
delay = REPLICA_RETRY_BASE_DELAY * (2 ** (attempt - 1))
logger.info(
f"RLS transaction failed on replica (attempt {attempt}/{max_attempts}), "
f"retrying in {delay}s. Error: {e}"
)
time.sleep(delay)
finally:
if _fallback["token"] is not None:
reset_read_db_alias(_fallback["token"])
_fallback["token"] = None
if router_token is not None:
reset_read_db_alias(router_token)
class CustomUserManager(BaseUserManager):
+206 -59
View File
@@ -67,6 +67,7 @@ from django_filters.rest_framework import (
)
from rest_framework_json_api.django_filters.backends import DjangoFilterBackend
from rest_framework_json_api.serializers import ValidationError
from uuid6 import UUID
class CustomDjangoFilterBackend(DjangoFilterBackend):
@@ -672,35 +673,32 @@ class LatestResourceFilter(ProviderRelationshipFilterSet):
return queryset.filter(tags__text_search=value)
class FindingFilter(CommonFindingFilters):
FINDING_BASE_FILTER_FIELDS = {
"id": ["exact", "in"],
"uid": ["exact", "in"],
"scan": ["exact", "in"],
"delta": ["exact", "in"],
"status": ["exact", "in"],
"severity": ["exact", "in"],
"impact": ["exact", "in"],
"check_id": ["exact", "in", "icontains"],
}
class BaseFindingFilter(CommonFindingFilters):
DATE_FILTER_FIELDS = ()
DATE_FILTER_NAMES = ()
DATE_RANGE_HELP_TEXT = (
f"Maximum date range is {settings.FINDINGS_MAX_DAYS_IN_RANGE} days."
)
DATE_FILTER_REQUIRED_DETAIL = "At least one date filter is required."
scan = UUIDFilter(method="filter_scan_id")
scan__in = UUIDInFilter(method="filter_scan_id_in")
inserted_at = DateFilter(method="filter_inserted_at", lookup_expr="date")
inserted_at__date = DateFilter(method="filter_inserted_at", lookup_expr="date")
inserted_at__gte = DateFilter(
method="filter_inserted_at_gte",
help_text=f"Maximum date range is {settings.FINDINGS_MAX_DAYS_IN_RANGE} days.",
)
inserted_at__lte = DateFilter(
method="filter_inserted_at_lte",
help_text=f"Maximum date range is {settings.FINDINGS_MAX_DAYS_IN_RANGE} days.",
)
class Meta:
model = Finding
fields = {
"id": ["exact", "in"],
"uid": ["exact", "in"],
"scan": ["exact", "in"],
"delta": ["exact", "in"],
"status": ["exact", "in"],
"severity": ["exact", "in"],
"impact": ["exact", "in"],
"check_id": ["exact", "in", "icontains"],
"inserted_at": ["date", "gte", "lte"],
"updated_at": ["gte", "lte"],
}
fields = FINDING_BASE_FILTER_FIELDS
filter_overrides = {
FindingDeltaEnumField: {
"filter_class": CharFilter,
@@ -723,17 +721,13 @@ class FindingFilter(CommonFindingFilters):
return queryset.filter(resource_services__contains=[value])
def filter_queryset(self, queryset):
if not (self.data.get("scan") or self.data.get("scan__in")) and not (
self.data.get("inserted_at")
or self.data.get("inserted_at__date")
or self.data.get("inserted_at__gte")
or self.data.get("inserted_at__lte")
if not (self.data.get("scan") or self.data.get("scan__in")) and not any(
self.data.get(filter_name) for filter_name in self.DATE_FILTER_NAMES
):
raise ValidationError(
[
{
"detail": "At least one date filter is required: filter[inserted_at], filter[inserted_at.gte], "
"or filter[inserted_at.lte].",
"detail": self.DATE_FILTER_REQUIRED_DETAIL,
"status": 400,
"source": {"pointer": "/data/attributes/inserted_at"},
"code": "required",
@@ -742,31 +736,42 @@ class FindingFilter(CommonFindingFilters):
)
cleaned = self.form.cleaned_data
exact_date = cleaned.get("inserted_at") or cleaned.get("inserted_at__date")
gte_date = cleaned.get("inserted_at__gte") or exact_date
lte_date = cleaned.get("inserted_at__lte") or exact_date
if gte_date is None:
gte_date = datetime.now(UTC).date()
if lte_date is None:
lte_date = datetime.now(UTC).date()
if abs(lte_date - gte_date) > timedelta(
days=settings.FINDINGS_MAX_DAYS_IN_RANGE
):
raise ValidationError(
[
{
"detail": f"The date range cannot exceed {settings.FINDINGS_MAX_DAYS_IN_RANGE} days.",
"status": 400,
"source": {"pointer": "/data/attributes/inserted_at"},
"code": "invalid",
}
]
)
for field_name in self.DATE_FILTER_FIELDS:
self.validate_datetime_filter_range(cleaned, field_name)
return super().filter_queryset(queryset)
def validate_datetime_filter_range(self, cleaned, field_name):
exact_value = cleaned.get(field_name) or cleaned.get(f"{field_name}__date")
gte_value = cleaned.get(f"{field_name}__gte") or exact_value
lte_value = cleaned.get(f"{field_name}__lte") or exact_value
if not (exact_value or gte_value or lte_value):
return
default_value = datetime.now(UTC).date()
gte_value = gte_value or default_value
lte_value = lte_value or default_value
gte_datetime = self.filter_value_to_datetime(gte_value, field_name)
lte_datetime = self.filter_value_to_datetime(lte_value, field_name)
if abs(lte_datetime - gte_datetime) <= timedelta(
days=settings.FINDINGS_MAX_DAYS_IN_RANGE
):
return
raise ValidationError(
[
{
"detail": f"The date range cannot exceed {settings.FINDINGS_MAX_DAYS_IN_RANGE} days.",
"status": 400,
"source": {"pointer": f"/data/attributes/{field_name}"},
"code": "invalid",
}
]
)
# Convert filter values to UUIDv7 values for use with partitioning
def filter_scan_id(self, queryset, name, value):
try:
@@ -824,27 +829,169 @@ class FindingFilter(CommonFindingFilters):
datetime_value = self.maybe_date_to_datetime(value)
start = uuid7_start(datetime_to_uuid7(datetime_value))
end = uuid7_start(datetime_to_uuid7(datetime_value + timedelta(days=1)))
return queryset.filter(id__gte=start, id__lt=end)
def filter_inserted_at_gte(self, queryset, name, value):
datetime_value = self.maybe_date_to_datetime(value)
start = uuid7_start(datetime_to_uuid7(datetime_value))
return queryset.filter(id__gte=start)
def filter_inserted_at_lte(self, queryset, name, value):
datetime_value = self.maybe_date_to_datetime(value)
end = uuid7_start(datetime_to_uuid7(datetime_value + timedelta(days=1)))
return queryset.filter(id__lt=end)
@staticmethod
def maybe_date_to_datetime(value):
dt = value
if isinstance(value, datetime):
return value
if isinstance(value, date):
dt = datetime.combine(value, datetime.min.time(), tzinfo=UTC)
return dt
return datetime.combine(value, datetime.min.time(), tzinfo=UTC)
if isinstance(value, str):
return parse(value)
return value
@classmethod
def filter_value_to_datetime(cls, value, field_name):
try:
datetime_value = cls.maybe_date_to_datetime(value)
except (TypeError, ValueError, OverflowError):
raise ValidationError(
[
{
"detail": "Enter a valid date or datetime.",
"status": 400,
"source": {"pointer": f"/data/attributes/{field_name}"},
"code": "invalid",
}
]
)
if datetime_value.tzinfo is None:
return datetime_value.replace(tzinfo=UTC)
return datetime_value.astimezone(UTC)
class FindingFilter(BaseFindingFilter):
DATE_FILTER_FIELDS = ("inserted_at", "updated_at")
DATE_FILTER_NAMES = (
"inserted_at",
"inserted_at__date",
"inserted_at__gte",
"inserted_at__lte",
"updated_at",
"updated_at__date",
"updated_at__gte",
"updated_at__lte",
)
DATE_FILTER_REQUIRED_DETAIL = (
"At least one date filter is required: filter[inserted_at], filter[updated_at], "
"filter[inserted_at.gte], filter[updated_at.gte], filter[inserted_at.lte], "
"or filter[updated_at.lte]."
)
inserted_at = CharFilter(method="filter_inserted_at")
inserted_at__date = DateFilter(method="filter_inserted_at", lookup_expr="date")
inserted_at__gte = CharFilter(
method="filter_inserted_at",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
inserted_at__lte = CharFilter(
method="filter_inserted_at",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
updated_at = CharFilter(method="filter_updated_at")
updated_at__date = DateFilter(method="filter_updated_at", lookup_expr="date")
updated_at__gte = CharFilter(
method="filter_updated_at",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
updated_at__lte = CharFilter(
method="filter_updated_at",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
class Meta(BaseFindingFilter.Meta):
fields = FINDING_BASE_FILTER_FIELDS | {
"inserted_at": ["date", "gte", "lte"],
"updated_at": ["date", "gte", "lte"],
}
def filter_inserted_at(self, queryset, name, value):
start, end = self.filter_value_to_datetime_bounds(value, "inserted_at")
if name.endswith("__gte"):
return queryset.filter(id__gte=self.datetime_to_uuid7_boundary(start))
if name.endswith("__lte"):
return queryset.filter(id__lt=self.datetime_to_uuid7_boundary(end))
return queryset.filter(
id__gte=self.datetime_to_uuid7_boundary(start),
id__lt=self.datetime_to_uuid7_boundary(end),
)
def filter_updated_at(self, queryset, name, value):
start, end = self.filter_value_to_datetime_bounds(value, "updated_at")
if name.endswith("__gte"):
return queryset.filter(updated_at__gte=start)
if name.endswith("__lte"):
return queryset.filter(updated_at__lt=end)
return queryset.filter(updated_at__gte=start, updated_at__lt=end)
@classmethod
def filter_value_to_datetime_bounds(cls, value, field_name):
start = cls.filter_value_to_datetime(value, field_name)
if cls.is_date_filter_value(value):
return start, start + timedelta(days=1)
return start, start + timedelta(milliseconds=1)
@staticmethod
def datetime_to_uuid7_boundary(datetime_value):
timestamp_ms = int(datetime_value.timestamp() * 1000) & 0xFFFFFFFFFFFF
uuid_int = timestamp_ms << 80
uuid_int |= 0x7 << 76
uuid_int |= 0x2 << 62
return UUID(int=uuid_int)
@staticmethod
def is_date_filter_value(value):
if isinstance(value, datetime):
return False
if isinstance(value, date):
return True
return isinstance(value, str) and len(value.strip()) == 10
class FindingMetadataFilter(BaseFindingFilter):
DATE_FILTER_FIELDS = ("inserted_at",)
DATE_FILTER_NAMES = (
"inserted_at",
"inserted_at__date",
"inserted_at__gte",
"inserted_at__lte",
)
DATE_FILTER_REQUIRED_DETAIL = (
"At least one date filter is required: filter[inserted_at], filter[inserted_at.gte], "
"or filter[inserted_at.lte]."
)
inserted_at = DateFilter(method="filter_inserted_at", lookup_expr="date")
inserted_at__date = DateFilter(method="filter_inserted_at", lookup_expr="date")
inserted_at__gte = DateFilter(
method="filter_inserted_at_gte",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
inserted_at__lte = DateFilter(
method="filter_inserted_at_lte",
help_text=BaseFindingFilter.DATE_RANGE_HELP_TEXT,
)
class Meta(BaseFindingFilter.Meta):
fields = FINDING_BASE_FILTER_FIELDS | {
"inserted_at": ["date", "gte", "lte"],
}
class LatestFindingFilter(CommonFindingFilters):
@@ -0,0 +1,25 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0096_attack_paths_scan_is_migrated"),
]
operations = [
migrations.AlterField(
model_name="attackpathsscan",
name="is_migrated",
field=models.BooleanField(db_default=False, default=False),
),
migrations.AlterField(
model_name="attackpathsscan",
name="sink_backend",
field=models.CharField(
choices=[("neo4j", "Neo4j"), ("neptune", "Neptune")],
db_default="neo4j",
default="neo4j",
max_length=16,
),
),
]
+2 -1
View File
@@ -814,9 +814,10 @@ class AttackPathsScan(RowLevelSecurityProtectedModel):
# still using the previous graph shape. Query catalog selection uses this
# flag; physical read routing uses sink_backend below.
# TODO: drop after Neptune cutover
is_migrated = models.BooleanField(default=False)
is_migrated = models.BooleanField(default=False, db_default=False)
sink_backend = models.CharField(
choices=SinkBackendChoices.choices,
db_default=SinkBackendChoices.NEO4J,
default=SinkBackendChoices.NEO4J,
max_length=16,
)
+5 -6
View File
@@ -34,7 +34,7 @@ class HasPermissions(BasePermission):
if not tenant_id:
return False
user_roles = (
user_roles = list(
User.objects.using(MainRouter.admin_db)
.get(id=request.user.id)
.roles.using(MainRouter.admin_db)
@@ -43,11 +43,10 @@ class HasPermissions(BasePermission):
if not user_roles:
return False
for perm in required_permissions:
if not getattr(user_roles[0], perm.value, False):
return False
return True
return all(
any(getattr(role, permission.value, False) for role in user_roles)
for permission in required_permissions
)
def get_role(user: User, tenant_id: str) -> Role:
File diff suppressed because it is too large Load Diff
+2 -1
View File
@@ -16,7 +16,7 @@ if TYPE_CHECKING:
class SSEChannelManager(DefaultChannelManager):
"""Connect `django-eventstream` to the platform's SSE viewsets."""
def get_channels_for_request(self, request: Request, view_kwargs: dict) -> set[str]: # noqa: vulture
def get_channels_for_request(self, request: Request, view_kwargs: dict) -> set[str]:
"""Return the request's channels scoped to the active JWT tenant.
Args:
@@ -30,6 +30,7 @@ class SSEChannelManager(DefaultChannelManager):
The subset of `request.sse_channels` whose embedded tenant
matches the active request tenant.
"""
_ = view_kwargs
try:
request_tenant_id = UUID(str(getattr(request, "tenant_id", None)))
except (TypeError, ValueError):
@@ -1,3 +1,4 @@
import json
import time
from datetime import UTC, datetime, timedelta
from uuid import uuid4
@@ -8,6 +9,23 @@ from conftest import TEST_PASSWORD, get_api_tokens, get_authorization_header
from django.urls import reverse
from drf_simple_apikey.crypto import get_crypto
from rest_framework.test import APIClient
from rest_framework_simplejwt.token_blacklist.models import (
BlacklistedToken,
OutstandingToken,
)
PASSWORD_CHANGE_PASSWORD = "InitialSecret123@"
@pytest.fixture
def password_change_user(tenants_fixture):
user = User.objects.create_user(
name="password_change_user",
email=f"password-change-{uuid4()}@prowler.com",
password=PASSWORD_CHANGE_PASSWORD,
)
Membership.objects.create(user=user, tenant=tenants_fixture[0])
return user
@pytest.mark.django_db
@@ -103,6 +121,120 @@ def test_refresh_token(create_test_user, tenants_fixture):
assert new_refresh_response.status_code == 200
@pytest.mark.django_db
def test_password_change_invalidates_existing_tokens(password_change_user):
client = APIClient()
new_password = "ChangedSecret123@"
access_token, refresh_token = get_api_tokens(
client, password_change_user.email, PASSWORD_CHANGE_PASSWORD
)
auth_headers = get_authorization_header(access_token)
outstanding_token_ids = list(
OutstandingToken.objects.filter(user=password_change_user).values_list(
"id", flat=True
)
)
assert outstanding_token_ids
assert not BlacklistedToken.objects.filter(
token_id__in=outstanding_token_ids
).exists()
password_change_payload = {
"data": {
"type": "users",
"id": str(password_change_user.id),
"attributes": {"password": new_password},
}
}
password_change_response = client.patch(
reverse("user-detail", kwargs={"pk": password_change_user.id}),
data=json.dumps(password_change_payload),
headers=auth_headers,
content_type="application/vnd.api+json",
)
assert password_change_response.status_code == 200, password_change_response.json()
assert BlacklistedToken.objects.filter(
token_id__in=outstanding_token_ids
).count() == len(outstanding_token_ids)
old_access_response = client.get(reverse("user-me"), headers=auth_headers)
assert old_access_response.status_code == 401
old_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": refresh_token},
}
},
format="vnd.api+json",
)
assert old_refresh_response.status_code == 400
new_access_token, _ = get_api_tokens(
client, password_change_user.email, new_password
)
new_access_response = client.get(
reverse("user-me"), headers=get_authorization_header(new_access_token)
)
assert new_access_response.status_code == 200
@pytest.mark.django_db
def test_password_change_invalidates_rotated_refresh_token(
password_change_user,
):
client = APIClient()
new_password = "ChangedSecret123@"
access_token, refresh_token = get_api_tokens(
client, password_change_user.email, PASSWORD_CHANGE_PASSWORD
)
rotated_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": refresh_token},
}
},
format="vnd.api+json",
)
assert rotated_refresh_response.status_code == 200
rotated_refresh_token = rotated_refresh_response.json()["data"]["attributes"][
"refresh"
]
password_change_payload = {
"data": {
"type": "users",
"id": str(password_change_user.id),
"attributes": {"password": new_password},
}
}
password_change_response = client.patch(
reverse("user-detail", kwargs={"pk": password_change_user.id}),
data=json.dumps(password_change_payload),
headers=get_authorization_header(access_token),
content_type="application/vnd.api+json",
)
assert password_change_response.status_code == 200, password_change_response.json()
old_rotated_refresh_response = client.post(
reverse("token-refresh"),
data={
"data": {
"type": "tokens-refresh",
"attributes": {"refresh": rotated_refresh_token},
}
},
format="vnd.api+json",
)
assert old_rotated_refresh_response.status_code == 400
@pytest.mark.django_db
def test_user_me_when_inviting_users(create_test_user, tenants_fixture, roles_fixture):
client = APIClient()
@@ -187,8 +319,9 @@ def test_user_me_when_inviting_users(create_test_user, tenants_fixture, roles_fi
@pytest.mark.django_db
class TestTokenSwitchTenant:
def test_switch_tenant_with_valid_token(self, tenants_fixture, providers_fixture):
def test_switch_tenant_with_valid_token(self, tenants_fixture, aws_provider):
client = APIClient()
assert aws_provider
test_user = "test_email@prowler.com"
test_password = "Test_password1@"
@@ -1396,13 +1529,14 @@ class TestAPIKeyMultiTenantWorkflows:
assert me_response2.json()["data"]["id"] == str(user.id)
def test_api_key_cannot_access_different_tenant_resources(
self, tenants_fixture, providers_fixture
self, tenants_fixture, aws_provider
):
"""API key from one tenant cannot access resources from another tenant.
Verifies RLS enforcement after authentication ensures tenant isolation.
"""
client = APIClient()
assert aws_provider
user1 = User.objects.create_user(
name="tenant1_user",
@@ -1,8 +1,12 @@
"""Tests for rls_transaction retry and fallback logic."""
from unittest.mock import patch
import pytest
from api.db_utils import rls_transaction
from django.db import DEFAULT_DB_ALIAS
from api.db_utils import POSTGRES_TENANT_VAR, rls_transaction
from conftest import TEST_REPLICA_ALIAS
from django.db import DEFAULT_DB_ALIAS, OperationalError, connections
from psycopg2 import OperationalError as Psycopg2OperationalError
from rest_framework_json_api.serializers import ValidationError
@@ -36,3 +40,35 @@ class TestRLSTransaction:
cursor.execute("SELECT current_setting(%s, true)", [custom_param])
result = cursor.fetchone()
assert result == (str(tenant.id),)
@pytest.mark.requires_test_replica_alias
@pytest.mark.django_db(
transaction=True, databases=[DEFAULT_DB_ALIAS, TEST_REPLICA_ALIAS]
)
def test_mid_query_replica_connection_loss_falls_back_to_primary(self, tenant):
"""Real Django connection state: closed replica atomic falls back to primary."""
replica = connections[TEST_REPLICA_ALIAS]
sql = "SELECT current_setting(%s, true), %s"
params = [POSTGRES_TENANT_VAR, 42]
failed_once = {"value": False}
def close_replica_and_raise(execute, sql_arg, params_arg, many, context):
if not failed_once["value"] and sql_arg == sql:
failed_once["value"] = True
replica.close()
try:
raise Psycopg2OperationalError("SSL SYSCALL error: EOF detected")
except Psycopg2OperationalError as psycopg_error:
raise OperationalError(
"SSL SYSCALL error: EOF detected"
) from psycopg_error
return execute(sql_arg, params_arg, many, context)
with patch("api.db_utils.READ_REPLICA_ALIAS", TEST_REPLICA_ALIAS):
with rls_transaction(str(tenant.id), using=TEST_REPLICA_ALIAS) as cursor:
with replica.execute_wrapper(close_replica_and_raise):
cursor.execute(sql, params)
result = cursor.fetchone()
assert failed_once["value"]
assert result == (str(tenant.id), 42)
+210 -16
View File
@@ -2,11 +2,18 @@ from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
from allauth.socialaccount.models import SocialLogin
from allauth.account import app_settings as account_app_settings
from allauth.account.models import EmailAddress
from allauth.core import context
from allauth.core.exceptions import ImmediateHttpResponse
from allauth.socialaccount import app_settings as socialaccount_app_settings
from allauth.socialaccount.internal.flows.login import complete_login
from allauth.socialaccount.models import SocialAccount, SocialLogin
from api.adapters import ProwlerSocialAccountAdapter
from api.db_router import MainRouter
from api.models import SAMLConfiguration
from api.models import Invitation, Membership, SAMLConfiguration, Tenant
from django.contrib.auth import get_user_model
from django.core import mail
User = get_user_model()
@@ -40,6 +47,7 @@ def _saml_request(rf, organization_slug):
def _saml_sociallogin(user):
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.account.pk = None
sociallogin.provider = MagicMock()
sociallogin.provider.id = "saml"
sociallogin.account.extra_data = {}
@@ -48,6 +56,59 @@ def _saml_sociallogin(user):
return sociallogin
def _oauth_sociallogin(
user,
*,
provider="google",
provider_email_verified=True,
include_extra_email=True,
):
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.account.pk = None
sociallogin.provider = MagicMock()
sociallogin.provider.id = provider
sociallogin.account.extra_data = (
{"email": user.email} if include_extra_email else {}
)
sociallogin.email_addresses = [
EmailAddress(
email=user.email,
verified=provider_email_verified,
primary=True,
)
]
sociallogin.user = user
sociallogin.connect = MagicMock()
return sociallogin
def _real_oauth_sociallogin(user, uid):
provider = MagicMock()
provider.id = "google"
provider.app = None
provider.get_settings.return_value = {}
return SocialLogin(
user=user,
account=SocialAccount(
provider="google",
uid=uid,
extra_data={"email": user.email},
),
email_addresses=[EmailAddress(email=user.email, verified=True, primary=True)],
provider=provider,
)
def _verify_local_email(user):
return EmailAddress.objects.create(
user=user,
email=user.email,
verified=True,
primary=True,
)
@pytest.mark.django_db
class TestProwlerSocialAccountAdapter:
def test_get_user_by_email_returns_user(self, create_test_user):
@@ -157,6 +218,7 @@ class TestProwlerSocialAccountAdapter:
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.account.pk = None
sociallogin.provider = MagicMock()
sociallogin.user = MagicMock()
sociallogin.user.email = ""
@@ -168,25 +230,157 @@ class TestProwlerSocialAccountAdapter:
sociallogin.connect.assert_not_called()
def test_pre_social_login_non_saml_links_by_email(self, create_test_user, rf):
"""Non-SAML providers (e.g. Google/GitHub) still link to an existing
local account by email; the tenant binding only applies to SAML."""
def test_pre_social_login_blocks_unverified_local_email(self, create_test_user, rf):
"""A verified OAuth email must not claim an unverified local account."""
adapter = ProwlerSocialAccountAdapter()
sociallogin = _oauth_sociallogin(create_test_user)
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.provider = MagicMock()
sociallogin.provider.id = "google"
sociallogin.account.extra_data = {"email": create_test_user.email}
sociallogin.user = create_test_user
sociallogin.connect = MagicMock()
with pytest.raises(ImmediateHttpResponse) as exc_info:
adapter.pre_social_login(rf.get("/"), sociallogin)
assert exc_info.value.response.status_code == 403
sociallogin.connect.assert_not_called()
def test_complete_oauth_login_does_not_link_unverified_local_email(
self, create_test_user, rf
):
"""Regression test for the complete pre-hijack account-linking flow."""
incoming_user = User(email=create_test_user.email)
incoming_user.set_unusable_password()
sociallogin = _real_oauth_sociallogin(
incoming_user,
uid="victim-google-account",
)
request = rf.get("/")
request.session = {}
with pytest.raises(ImmediateHttpResponse) as exc_info:
complete_login(request, sociallogin, raises=True)
assert exc_info.value.response.status_code == 403
assert not SocialAccount.objects.filter(
provider="google", uid="victim-google-account"
).exists()
def test_pre_social_login_allows_already_connected_account(
self, create_test_user, rf
):
"""Existing provider bindings do not need to relink on every login."""
adapter = ProwlerSocialAccountAdapter()
sociallogin = _oauth_sociallogin(create_test_user)
sociallogin.account.pk = "existing-social-account"
adapter.pre_social_login(rf.get("/"), sociallogin)
call_args = sociallogin.connect.call_args
assert call_args is not None
_, called_user = call_args[0]
assert called_user.email == create_test_user.email
sociallogin.connect.assert_not_called()
def test_pre_social_login_blocks_unverified_provider_email(
self, create_test_user, rf
):
"""An OAuth provider must prove ownership of the matching email."""
_verify_local_email(create_test_user)
adapter = ProwlerSocialAccountAdapter()
sociallogin = _oauth_sociallogin(
create_test_user,
provider="github",
provider_email_verified=False,
)
with pytest.raises(ImmediateHttpResponse) as exc_info:
adapter.pre_social_login(rf.get("/"), sociallogin)
assert exc_info.value.response.status_code == 403
sociallogin.connect.assert_not_called()
def test_pre_social_login_links_verified_emails(self, create_test_user, rf):
_verify_local_email(create_test_user)
adapter = ProwlerSocialAccountAdapter()
sociallogin = _oauth_sociallogin(create_test_user)
request = rf.get("/")
adapter.pre_social_login(request, sociallogin)
sociallogin.connect.assert_called_once_with(request, create_test_user)
def test_verified_social_account_link_does_not_send_notification(
self, create_test_user, rf
):
_verify_local_email(create_test_user)
sociallogin = _real_oauth_sociallogin(
create_test_user,
uid="verified-google-account",
)
request = rf.get("/")
with context.request_context(request):
ProwlerSocialAccountAdapter().pre_social_login(request, sociallogin)
assert SocialAccount.objects.filter(
provider="google",
uid="verified-google-account",
user=create_test_user,
).exists()
assert mail.outbox == []
def test_pre_social_login_uses_verified_email_missing_from_extra_data(
self, create_test_user, rf
):
"""GitHub can return its verified primary email outside extra_data."""
_verify_local_email(create_test_user)
adapter = ProwlerSocialAccountAdapter()
sociallogin = _oauth_sociallogin(
create_test_user,
provider="github",
include_extra_email=False,
)
request = rf.get("/")
adapter.pre_social_login(request, sociallogin)
sociallogin.connect.assert_called_once_with(request, create_test_user)
def test_social_account_linking_settings_are_fail_closed(self):
assert not socialaccount_app_settings.EMAIL_AUTHENTICATION
assert not socialaccount_app_settings.EMAIL_AUTHENTICATION_AUTO_CONNECT
assert not account_app_settings.EMAIL_NOTIFICATIONS
def test_save_user_social_with_invitation_joins_invited_tenant(
self, rf, create_test_user, tenants_fixture
):
adapter = ProwlerSocialAccountAdapter()
invited_tenant = tenants_fixture[2]
invited_email = "frank-invited@example.com"
invitation = Invitation.objects.create(
tenant=invited_tenant,
email=invited_email,
inviter=create_test_user,
)
request = rf.post("/", data={"invitation_token": invitation.token})
request.session = {}
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.provider = MagicMock()
sociallogin.provider.id = "google"
sociallogin.account = MagicMock()
sociallogin.account.extra_data = {"name": "Frank"}
real_user = User.objects.create_user(
name="Frank", email=invited_email, password="Secret123!"
)
tenants_before = Tenant.objects.count()
with patch("api.adapters.super") as mock_super:
mock_super.return_value.save_user.return_value = real_user
adapter.save_user(request, sociallogin)
invitation.refresh_from_db()
assert invitation.state == Invitation.State.ACCEPTED
assert Tenant.objects.count() == tenants_before
assert Membership.objects.filter(
user=real_user,
tenant=invited_tenant,
role=Membership.RoleChoices.MEMBER,
).exists()
def test_save_user_saml_sets_session_flag(self, rf):
adapter = ProwlerSocialAccountAdapter()
@@ -10,6 +10,7 @@ hierarchy; sink-internal behavior is exercised in `test_sink.py`.
from unittest.mock import MagicMock, patch
import api.attack_paths.database as db_module
import pytest
class TestDatabaseNameHelper:
@@ -72,6 +73,71 @@ class TestExecuteReadQueryRoutes:
)
class TestScanDatabaseAvailability:
def test_verify_scan_databases_available_checks_ingest_and_sink(self):
with (
patch("api.attack_paths.database.ingest") as mock_ingest,
patch("api.attack_paths.database.get_driver") as mock_get_driver,
):
db_module.verify_scan_databases_available()
mock_ingest.get_driver.return_value.verify_connectivity.assert_called_once_with()
mock_get_driver.return_value.verify_connectivity.assert_called_once_with()
def test_verify_scan_databases_available_raises_when_ingest_is_down(self):
with (
patch("api.attack_paths.database.ingest") as mock_ingest,
patch("api.attack_paths.database.get_driver"),
):
mock_ingest.get_driver.return_value.verify_connectivity.side_effect = (
RuntimeError("ingest down")
)
with pytest.raises(RuntimeError) as exc:
db_module.verify_scan_databases_available()
assert "Attack Paths graph database unavailable before scan start" in str(
exc.value
)
assert "ingest Neo4j: ingest down" in str(exc.value)
def test_verify_scan_databases_available_raises_when_sink_is_down(self, settings):
settings.ATTACK_PATHS_SINK_DATABASE = "neptune"
with (
patch("api.attack_paths.database.ingest"),
patch("api.attack_paths.database.get_driver") as mock_get_driver,
):
mock_get_driver.return_value.verify_connectivity.side_effect = RuntimeError(
"writer down"
)
with pytest.raises(RuntimeError) as exc:
db_module.verify_scan_databases_available()
assert "sink neptune: writer down" in str(exc.value)
def test_verify_scan_databases_available_reports_both_failures(self, settings):
settings.ATTACK_PATHS_SINK_DATABASE = "neo4j"
with (
patch("api.attack_paths.database.ingest") as mock_ingest,
patch("api.attack_paths.database.get_driver") as mock_get_driver,
):
mock_ingest.get_driver.return_value.verify_connectivity.side_effect = (
RuntimeError("ingest down")
)
mock_get_driver.return_value.verify_connectivity.side_effect = RuntimeError(
"sink down"
)
with pytest.raises(RuntimeError) as exc:
db_module.verify_scan_databases_available()
assert "ingest Neo4j: ingest down" in str(exc.value)
assert "sink neo4j: sink down" in str(exc.value)
class TestSinkOperationsDelegation:
def test_has_provider_data_delegates_to_sink(self, sink_backend_stub):
sink_backend_stub.has_provider_data.return_value = True
@@ -4,6 +4,7 @@ from unittest.mock import patch
import pytest
from api.attack_paths.cypher_sanitizer import (
inject_label,
inject_provider_label,
validate_custom_query,
)
@@ -21,6 +22,13 @@ def _inject(cypher: str) -> str:
return inject_provider_label(cypher, PROVIDER_ID)
def test_generic_inject_label_reuses_provider_injection_pipeline():
result = inject_label("MATCH (n:AWSRole)--(m) RETURN n, m", "_Tenant_test")
assert "(n:AWSRole:_Tenant_test)" in result
assert "(m:_Tenant_test)" in result
# ---------------------------------------------------------------------------
# Pass A - Labeled node patterns (all clauses)
# ---------------------------------------------------------------------------
+573 -19
View File
@@ -1,11 +1,16 @@
from contextlib import contextmanager
from datetime import UTC, datetime
from enum import Enum
from unittest.mock import MagicMock, patch
from unittest.mock import MagicMock, call, patch
import pytest
from api.db_utils import (
POSTGRES_TENANT_VAR,
SET_CONFIG_QUERY,
SET_TRANSACTION_READ_ONLY_QUERY,
PostgresEnumMigration,
_is_replica_connection_failure,
_is_safe_primary_replay,
_should_create_index_on_partition,
batch_delete,
create_objects_in_batches,
@@ -392,10 +397,23 @@ class TestRlsTransaction:
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_replica_conn = MagicMock()
mock_replica_cursor = MagicMock()
mock_replica_conn.cursor.return_value.__enter__.return_value = (
mock_replica_cursor
)
mock_primary_conn = MagicMock()
mock_primary_cursor = MagicMock()
mock_primary_conn.cursor.return_value.__enter__.return_value = (
mock_primary_cursor
)
def connections_getitem(alias):
if alias == "replica":
return mock_replica_conn
return mock_primary_conn
mock_connections.__getitem__.side_effect = connections_getitem
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic"):
@@ -525,7 +543,7 @@ class TestRlsTransaction:
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
if call_count < 4:
raise OperationalError("Connection error")
return MagicMock(
__enter__=MagicMock(return_value=None),
@@ -544,10 +562,11 @@ class TestRlsTransaction:
with rls_transaction(tenant_id):
pass
assert mock_sleep.call_count == 2
assert mock_sleep.call_count == 3
mock_sleep.assert_any_call(0.5)
mock_sleep.assert_any_call(1.0)
assert mock_logger.info.call_count == 2
mock_sleep.assert_any_call(2.0)
assert mock_logger.info.call_count == 3
def test_rls_transaction_operational_error_inside_context_no_retry(
self, tenants_fixture, enable_read_replica
@@ -578,11 +597,12 @@ class TestRlsTransaction:
raise OperationalError("Conflict with recovery")
mock_sleep.assert_not_called()
mock_conn.close.assert_not_called()
def test_rls_transaction_max_three_attempts_for_replica(
def test_rls_transaction_max_attempts_for_replica(
self, tenants_fixture, enable_read_replica
):
"""Test maximum 3 attempts for replica database."""
"""Test REPLICA_MAX_ATTEMPTS replica tries + 1 primary fallback."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
@@ -606,7 +626,11 @@ class TestRlsTransaction:
with rls_transaction(tenant_id):
pass
assert mock_atomic.call_count == 3
assert mock_atomic.call_args_list[-1] == call(
using=DEFAULT_DB_ALIAS
)
# 3 replica + 1 primary = 4 total
assert mock_atomic.call_count == 4
def test_rls_transaction_replica_no_retry_when_disabled(
self, tenants_fixture, enable_read_replica
@@ -617,10 +641,23 @@ class TestRlsTransaction:
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_replica_conn = MagicMock()
mock_replica_cursor = MagicMock()
mock_replica_conn.cursor.return_value.__enter__.return_value = (
mock_replica_cursor
)
mock_primary_conn = MagicMock()
mock_primary_cursor = MagicMock()
mock_primary_conn.cursor.return_value.__enter__.return_value = (
mock_primary_cursor
)
def connections_getitem(alias):
if alias == "replica":
return mock_replica_conn
return mock_primary_conn
mock_connections.__getitem__.side_effect = connections_getitem
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
@@ -682,7 +719,7 @@ class TestRlsTransaction:
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
if call_count < 4:
raise OperationalError("Replica error")
return MagicMock(
__enter__=MagicMock(return_value=None),
@@ -691,7 +728,7 @@ class TestRlsTransaction:
with patch(
"api.db_utils.transaction.atomic", side_effect=atomic_side_effect
):
) as mock_atomic:
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
@@ -701,6 +738,9 @@ class TestRlsTransaction:
with rls_transaction(tenant_id):
pass
assert mock_atomic.call_args_list[-1] == call(
using=DEFAULT_DB_ALIAS
)
mock_logger.warning.assert_called_once()
warning_msg = mock_logger.warning.call_args[0][0]
assert "falling back to primary DB" in warning_msg
@@ -725,7 +765,7 @@ class TestRlsTransaction:
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
if call_count < 4:
raise OperationalError("Replica error")
return MagicMock(
__enter__=MagicMock(return_value=None),
@@ -744,7 +784,7 @@ class TestRlsTransaction:
with rls_transaction(tenant_id):
pass
assert mock_logger.info.call_count == 2
assert mock_logger.info.call_count == 3
assert mock_logger.warning.call_count == 1
def test_rls_transaction_operational_error_raised_immediately_on_primary(
@@ -910,6 +950,520 @@ class TestRlsTransaction:
result = cursor.fetchone()
assert result[0] == 1
# --- Mid-query failover tests ---
class _FakeDatabaseError(Exception):
def __init__(self, message, pgcode=None):
super().__init__(message)
self.pgcode = pgcode
def _install_execute_wrapper(self, connection):
connection.execute_wrappers = []
@contextmanager
def _execute_wrapper(fn):
connection.execute_wrappers.append(fn)
try:
yield
finally:
connection.execute_wrappers.remove(fn)
connection.execute_wrapper = _execute_wrapper
def _mock_replica_and_primary_connections(self, mock_connections):
mock_replica_conn = MagicMock()
self._install_execute_wrapper(mock_replica_conn)
mock_replica_cursor = MagicMock()
mock_replica_conn.cursor.return_value.__enter__.return_value = (
mock_replica_cursor
)
mock_primary_conn = MagicMock()
mock_primary_cursor = MagicMock()
mock_primary_raw_cursor = MagicMock()
mock_primary_cursor.cursor = mock_primary_raw_cursor
mock_primary_conn.cursor.return_value = mock_primary_cursor
def connections_getitem(alias):
if alias == "replica":
return mock_replica_conn
return mock_primary_conn
mock_connections.__getitem__.side_effect = connections_getitem
mock_connections.__contains__.return_value = True
return mock_replica_conn, mock_primary_conn, mock_primary_cursor
@pytest.mark.parametrize(
"error",
[
_FakeDatabaseError("connection lost", pgcode="08006"),
_FakeDatabaseError("terminating connection", pgcode="57P01"),
OperationalError("SSL SYSCALL error: EOF detected"),
OperationalError("server closed the connection unexpectedly"),
OperationalError("database system is starting up"),
],
)
def test_replica_connection_failure_detection_allows_failover(self, error):
assert _is_replica_connection_failure(error)
@pytest.mark.parametrize(
"error",
[
_FakeDatabaseError("canceling statement", pgcode="57014"),
_FakeDatabaseError("could not serialize access", pgcode="40001"),
_FakeDatabaseError("deadlock detected", pgcode="40P01"),
OperationalError("deadlock detected"),
],
)
def test_replica_connection_failure_detection_rejects_query_errors(self, error):
assert not _is_replica_connection_failure(error)
@pytest.mark.parametrize(
("sql", "many", "expected"),
[
("SELECT 1", False, True),
(" -- leading comment\nSELECT 1", False, True),
("/* leading comment */ SELECT 1", False, True),
("SELECT 1", True, False),
("SELECTING 1", False, False),
("INSERT INTO fake_table (name) VALUES (%s)", False, False),
("WITH rows AS (SELECT 1) SELECT * FROM rows", False, False),
("SELECT * INTO fake_table_copy FROM fake_table", False, False),
("SELECT * FROM fake_table FOR UPDATE", False, False),
("SELECT * FROM fake_table FOR SHARE", False, False),
],
)
def test_primary_replay_safety_detection(self, sql, many, expected):
assert _is_safe_primary_replay(sql, many) is expected
def test_mid_query_failure_falls_directly_back_to_primary(
self, tenants_fixture, enable_read_replica
):
"""Mid-query replica connection loss is replayed once on primary."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
mock_primary_conn,
mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
outer_atomic = MagicMock()
outer_atomic.__enter__ = MagicMock(return_value=None)
outer_atomic.__exit__ = MagicMock(return_value=False)
fallback_atomic = MagicMock()
fallback_atomic.__enter__ = MagicMock(return_value=None)
fallback_atomic.__exit__ = MagicMock(return_value=False)
with patch(
"api.db_utils.transaction.atomic",
side_effect=[outer_atomic, fallback_atomic],
) as mock_atomic:
with patch("api.db_utils.time.sleep") as mock_sleep:
with patch(
"api.db_utils.set_read_db_alias",
side_effect=["replica-token", "primary-token"],
) as mock_set_alias:
with patch(
"api.db_utils.reset_read_db_alias"
) as mock_reset_alias:
with rls_transaction(tenant_id):
wrapper = mock_replica_conn.execute_wrappers[0]
context_cursor = MagicMock()
mock_execute = MagicMock(
side_effect=OperationalError(
"SSL SYSCALL error: EOF detected"
)
)
wrapper(
mock_execute,
"SELECT %s",
["value"],
False,
{"cursor": context_cursor},
)
mock_sleep.assert_not_called()
(
mock_replica_conn.ensure_connection.assert_not_called()
)
mock_replica_conn.close.assert_called_once()
(
mock_primary_conn.ensure_connection.assert_called_once()
)
mock_primary_conn.cursor.assert_called_once_with()
mock_primary_cursor.execute.assert_has_calls(
[
call(SET_TRANSACTION_READ_ONLY_QUERY),
call(
SET_CONFIG_QUERY,
[POSTGRES_TENANT_VAR, tenant_id],
),
call("SELECT %s", ["value"]),
]
)
assert context_cursor.db == mock_primary_conn
assert (
context_cursor.cursor
== mock_primary_cursor.cursor
)
mock_set_alias.assert_has_calls(
[
call(enable_read_replica),
call(DEFAULT_DB_ALIAS),
]
)
mock_reset_alias.assert_has_calls(
[call("primary-token"), call("replica-token")]
)
assert mock_atomic.call_args_list == [
call(using=enable_read_replica),
call(using=DEFAULT_DB_ALIAS),
]
assert mock_replica_conn.execute_wrappers == []
@pytest.mark.parametrize(
("sql", "params", "many"),
[
("INSERT INTO fake_table (name) VALUES (%s)", [("one",), ("two",)], True),
("INSERT INTO fake_table (name) VALUES (%s)", ["one"], False),
("UPDATE fake_table SET name = %s", ["one"], False),
("DELETE FROM fake_table WHERE id = %s", [1], False),
(
"WITH deleted AS (DELETE FROM fake_table RETURNING *) "
"SELECT * FROM deleted",
None,
False,
),
("SELECT * INTO fake_table_copy FROM fake_table", None, False),
],
)
def test_mid_query_fallback_rejects_unsafe_replay(
self, tenants_fixture, enable_read_replica, sql, params, many
):
"""Only single SELECT statements are replayed on primary."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
mock_primary_conn,
mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.return_value.__enter__ = MagicMock(return_value=None)
mock_atomic.return_value.__exit__ = MagicMock(return_value=False)
with patch(
"api.db_utils.set_read_db_alias",
side_effect=["replica-token", "primary-token"],
):
with patch("api.db_utils.reset_read_db_alias"):
with rls_transaction(tenant_id):
wrapper = mock_replica_conn.execute_wrappers[0]
mock_execute = MagicMock(
side_effect=OperationalError(
"server closed the connection"
)
)
with pytest.raises(OperationalError):
wrapper(
mock_execute,
sql,
params,
many,
{"cursor": MagicMock()},
)
mock_primary_conn.ensure_connection.assert_not_called()
mock_primary_conn.cursor.assert_not_called()
mock_primary_cursor.execute.assert_not_called()
mock_primary_cursor.executemany.assert_not_called()
def test_mid_query_non_connection_error_does_not_fall_back(
self, tenants_fixture, enable_read_replica
):
"""Query/concurrency errors are not replayed on primary."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
mock_primary_conn,
_mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.return_value.__enter__ = MagicMock(return_value=None)
mock_atomic.return_value.__exit__ = MagicMock(return_value=False)
with patch(
"api.db_utils.set_read_db_alias", return_value="replica-token"
):
with patch("api.db_utils.reset_read_db_alias"):
with rls_transaction(tenant_id):
wrapper = mock_replica_conn.execute_wrappers[0]
mock_execute = MagicMock(
side_effect=OperationalError("deadlock detected")
)
with pytest.raises(OperationalError):
wrapper(
mock_execute,
"SELECT 1",
None,
False,
{"cursor": MagicMock()},
)
mock_replica_conn.close.assert_not_called()
(
mock_primary_conn.ensure_connection.assert_not_called()
)
def test_mid_query_primary_replay_failure_propagates(
self, tenants_fixture, enable_read_replica
):
"""Primary fallback errors propagate as Django OperationalError."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
_mock_primary_conn,
mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
mock_primary_cursor.execute.side_effect = [
None,
None,
OperationalError("primary down"),
]
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.return_value.__enter__ = MagicMock(return_value=None)
mock_atomic.return_value.__exit__ = MagicMock(return_value=False)
with patch(
"api.db_utils.set_read_db_alias",
side_effect=["replica-token", "primary-token"],
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(OperationalError, match="primary down"):
with rls_transaction(tenant_id):
wrapper = mock_replica_conn.execute_wrappers[0]
mock_execute = MagicMock(
side_effect=OperationalError(
"server closed the connection"
)
)
wrapper(
mock_execute,
"SELECT 1",
None,
False,
{"cursor": MagicMock()},
)
mock_primary_cursor.close.assert_called_once()
def test_mid_query_fallback_suppresses_cleanup_error(
self, tenants_fixture, enable_read_replica
):
"""After successful primary fallback, replica cleanup error is suppressed."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
_mock_primary_conn,
mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
# Replica's atomic.__exit__ raises on dead replica cleanup;
# primary's atomic.__exit__ returns False (healthy commit).
mock_outer_atomic = MagicMock()
mock_outer_atomic.__enter__ = MagicMock(return_value=None)
mock_outer_atomic.__exit__ = MagicMock(
side_effect=OperationalError("cleanup failed on dead replica")
)
mock_fallback_atomic = MagicMock()
mock_fallback_atomic.__enter__ = MagicMock(return_value=None)
mock_fallback_atomic.__exit__ = MagicMock(return_value=False)
atomic_call_count = 0
def atomic_side_effect(*args, **kwargs):
nonlocal atomic_call_count
atomic_call_count += 1
if atomic_call_count == 1:
return mock_outer_atomic
return mock_fallback_atomic
with patch(
"api.db_utils.transaction.atomic",
side_effect=atomic_side_effect,
):
with patch(
"api.db_utils.set_read_db_alias",
side_effect=["replica-token", "primary-token"],
):
with patch("api.db_utils.reset_read_db_alias"):
with rls_transaction(tenant_id):
wrapper = mock_replica_conn.execute_wrappers[0]
mock_execute = MagicMock(
side_effect=OperationalError(
"server closed the connection"
)
)
mock_context = {"cursor": MagicMock()}
wrapper(
mock_execute,
"SELECT 1",
None,
False,
mock_context,
)
mock_primary_cursor.execute.assert_has_calls(
[
call(SET_TRANSACTION_READ_ONLY_QUERY),
call(
SET_CONFIG_QUERY,
[POSTGRES_TENANT_VAR, tenant_id],
),
call("SELECT 1", None),
]
)
def test_wrapper_not_installed_on_primary(self, tenants_fixture):
"""execute_wrapper is not installed when targeting primary DB."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=None):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_conn.execute_wrappers = []
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
with patch("api.db_utils.transaction.atomic") as mock_atomic:
mock_atomic.return_value.__enter__ = MagicMock(return_value=None)
mock_atomic.return_value.__exit__ = MagicMock(return_value=False)
with rls_transaction(tenant_id):
# No wrapper installed on primary
assert len(mock_conn.execute_wrappers) == 0
def test_stale_connection_closed_on_pre_yield_retry(
self, tenants_fixture, enable_read_replica
):
"""Stale connection is closed before each pre-yield retry."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
mock_conn = MagicMock()
mock_conn.execute_wrappers = []
mock_cursor = MagicMock()
mock_conn.cursor.return_value.__enter__.return_value = mock_cursor
mock_connections.__getitem__.return_value = mock_conn
mock_connections.__contains__.return_value = True
call_count = 0
def atomic_side_effect(*args, **kwargs):
nonlocal call_count
call_count += 1
if call_count < 3:
raise OperationalError("Connection error")
return MagicMock(
__enter__=MagicMock(return_value=None),
__exit__=MagicMock(return_value=False),
)
with patch(
"api.db_utils.transaction.atomic", side_effect=atomic_side_effect
):
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with rls_transaction(tenant_id):
pass
# close() called for each failed pre-yield attempt
assert mock_conn.close.call_count == 2
def test_caller_error_propagates_after_successful_failover(
self, tenants_fixture, enable_read_replica
):
"""OperationalError raised by caller after failover is NOT suppressed."""
tenant = tenants_fixture[0]
tenant_id = str(tenant.id)
with patch("api.db_utils.get_read_db_alias", return_value=enable_read_replica):
with patch("api.db_utils.connections") as mock_connections:
(
mock_replica_conn,
_mock_primary_conn,
_mock_primary_cursor,
) = self._mock_replica_and_primary_connections(mock_connections)
# Transaction cleanup succeeds so the caller error should surface.
mock_atomic_cm = MagicMock()
mock_atomic_cm.__enter__ = MagicMock(return_value=None)
mock_atomic_cm.__exit__ = MagicMock(return_value=False)
with patch(
"api.db_utils.transaction.atomic", return_value=mock_atomic_cm
):
with patch("api.db_utils.time.sleep"):
with patch(
"api.db_utils.set_read_db_alias", return_value="token"
):
with patch("api.db_utils.reset_read_db_alias"):
with pytest.raises(
OperationalError, match="caller error"
):
with rls_transaction(tenant_id):
# Trigger failover (succeeds on primary)
wrapper = mock_replica_conn.execute_wrappers[0]
mock_execute = MagicMock(
side_effect=OperationalError(
"server closed the connection"
)
)
mock_context = {"cursor": MagicMock()}
wrapper(
mock_execute,
"SELECT 1",
None,
False,
mock_context,
)
# Caller errors after successful failover
# should still propagate.
raise OperationalError("caller error")
class TestPostgresEnumMigration:
"""
+6 -6
View File
@@ -40,10 +40,10 @@ class TestSetTenantDecorator:
@pytest.mark.django_db
class TestHandleProviderDeletionDecorator:
def test_success_no_exception(self, tenants_fixture, providers_fixture):
def test_success_no_exception(self, tenants_fixture, aws_provider):
"""Decorated function runs normally when no exception is raised."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
provider = aws_provider
@handle_provider_deletion
def task_func(**kwargs):
@@ -127,11 +127,11 @@ class TestHandleProviderDeletionDecorator:
@patch("api.decorators.rls_transaction")
@patch("api.decorators.Provider.objects.filter")
def test_provider_exists_reraises_original(
self, mock_filter, mock_rls, tenants_fixture, providers_fixture
self, mock_filter, mock_rls, tenants_fixture, aws_provider
):
"""Re-raises original exception when provider still exists."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
provider = aws_provider
mock_rls.return_value.__enter__ = lambda s: None
mock_rls.return_value.__exit__ = lambda s, *args: None
@@ -187,11 +187,11 @@ class TestHandleProviderDeletionDecorator:
@patch("api.decorators.rls_transaction")
@patch("api.decorators.Provider.objects.filter")
def test_database_error_provider_exists_reraises(
self, mock_filter, mock_rls, tenants_fixture, providers_fixture
self, mock_filter, mock_rls, tenants_fixture, aws_provider
):
"""Re-raises original DatabaseError when provider still exists."""
tenant = tenants_fixture[0]
provider = providers_fixture[0]
provider = aws_provider
mock_rls.return_value.__enter__ = lambda s: None
mock_rls.return_value.__exit__ = lambda s, *args: None
+10 -10
View File
@@ -19,8 +19,8 @@ from django.db import IntegrityError
@pytest.mark.django_db
class TestResourceModel:
def test_setting_tags(self, providers_fixture):
provider, *_ = providers_fixture
def test_setting_tags(self, aws_provider):
provider = aws_provider
tenant_id = provider.tenant_id
resource = Resource.objects.create(
@@ -111,9 +111,9 @@ class TestResourceModel:
# @pytest.mark.django_db
# class TestFindingModel:
# def test_add_finding_with_long_uid(
# self, providers_fixture, scans_fixture, resources_fixture
# self, aws_provider, scans_fixture, resources_fixture
# ):
# provider, *_ = providers_fixture
# provider = aws_provider
# tenant_id = provider.tenant_id
# long_uid = "1" * 500
@@ -372,8 +372,8 @@ class TestSAMLConfigurationModel:
@pytest.mark.django_db
class TestProviderComplianceScoreModel:
def test_create_provider_compliance_score(self, providers_fixture, scans_fixture):
provider = providers_fixture[0]
def test_create_provider_compliance_score(self, aws_provider, scans_fixture):
provider = aws_provider
scan = scans_fixture[0]
scan.completed_at = datetime.now(UTC)
scan.save()
@@ -393,9 +393,9 @@ class TestProviderComplianceScoreModel:
assert score.requirement_status == StatusChoices.PASS
def test_unique_constraint_per_provider_compliance_requirement(
self, providers_fixture, scans_fixture
self, aws_provider, scans_fixture
):
provider = providers_fixture[0]
provider = aws_provider
scan = scans_fixture[0]
scan.completed_at = datetime.now(UTC)
scan.save()
@@ -422,9 +422,9 @@ class TestProviderComplianceScoreModel:
)
def test_different_providers_same_requirement_allowed(
self, providers_fixture, scans_fixture
self, aws_provider_pair, scans_fixture
):
provider1, provider2, *_ = providers_fixture
provider1, provider2 = aws_provider_pair
scan1 = scans_fixture[0]
scan1.completed_at = datetime.now(UTC)
scan1.save()
+147 -50
View File
@@ -11,6 +11,7 @@ from api.models import (
User,
UserRoleRelationship,
)
from api.rbac.permissions import HasPermissions, Permissions
from api.v1.serializers import TokenSerializer
from conftest import TEST_PASSWORD, TODAY
from django.urls import reverse
@@ -103,20 +104,84 @@ class TestUserViewSet:
assert response.json()["data"]["attributes"]["name"] == "Updated Name"
def test_partial_update_user_with_no_permissions(
self, authenticated_client_no_permissions_rbac, create_test_user
self, authenticated_client_no_permissions_rbac, create_test_user_rbac_limited
):
updated_data = {
"data": {
"type": "users",
"id": str(create_test_user_rbac_limited.id),
"attributes": {"name": "Updated Name"},
}
}
response = authenticated_client_no_permissions_rbac.patch(
reverse("user-detail", kwargs={"pk": create_test_user.id}),
reverse("user-detail", kwargs={"pk": create_test_user_rbac_limited.id}),
data=updated_data,
format="vnd.api+json",
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_403_FORBIDDEN
assert response.status_code == status.HTTP_200_OK
assert response.json()["data"]["attributes"]["name"] == "Updated Name"
def test_partial_update_other_user_with_no_permissions_denied(
self, authenticated_client_no_permissions_rbac, tenants_fixture
):
original_email = "target-rbac-update@example.com"
original_password = "OriginalPassword123@"
target_user = User.objects.create_user(
name="target_rbac_update",
email=original_email,
password=original_password,
)
Membership.objects.create(user=target_user, tenant=tenants_fixture[0])
updated_data = {
"data": {
"type": "users",
"id": str(target_user.id),
"attributes": {
"email": "updated-target-rbac@example.com",
"password": "UpdatedPassword123@",
},
}
}
response = authenticated_client_no_permissions_rbac.patch(
reverse("user-detail", kwargs={"pk": target_user.id}),
data=updated_data,
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
target_user.refresh_from_db()
assert target_user.email == original_email
assert target_user.check_password(original_password)
def test_partial_update_other_user_with_manage_users_allowed(
self, authenticated_client_rbac_manage_users_only
):
user = authenticated_client_rbac_manage_users_only.user
tenant = Membership.objects.filter(user=user).first().tenant
target_user = User.objects.create_user(
name="target_manage_users_update",
email="target-manage-users-update@example.com",
password="Password123@",
)
Membership.objects.create(user=target_user, tenant=tenant)
updated_data = {
"data": {
"type": "users",
"id": str(target_user.id),
"attributes": {"name": "Updated Target Name"},
}
}
response = authenticated_client_rbac_manage_users_only.patch(
reverse("user-detail", kwargs={"pk": target_user.id}),
data=updated_data,
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_200_OK
target_user.refresh_from_db()
assert target_user.name == "Updated Target Name"
def test_delete_user_with_all_permissions(
self, authenticated_client_rbac, create_test_user_rbac
@@ -370,11 +435,11 @@ class TestUserViewSet:
@pytest.mark.django_db
class TestProviderViewSet:
def test_list_providers_with_all_permissions(
self, authenticated_client_rbac, providers_fixture
self, authenticated_client_rbac, aws_provider
):
response = authenticated_client_rbac.get(reverse("provider-list"))
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == len(providers_fixture)
assert len(response.json()["data"]) == 1
def test_list_providers_with_no_permissions(
self, authenticated_client_no_permissions_rbac
@@ -386,9 +451,9 @@ class TestProviderViewSet:
assert len(response.json()["data"]) == 0
def test_retrieve_provider_with_all_permissions(
self, authenticated_client_rbac, providers_fixture
self, authenticated_client_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
response = authenticated_client_rbac.get(
reverse("provider-detail", kwargs={"pk": provider.id})
)
@@ -396,9 +461,9 @@ class TestProviderViewSet:
assert response.json()["data"]["attributes"]["alias"] == provider.alias
def test_retrieve_provider_with_no_permissions(
self, authenticated_client_no_permissions_rbac, providers_fixture
self, authenticated_client_no_permissions_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
response = authenticated_client_no_permissions_rbac.get(
reverse("provider-detail", kwargs={"pk": provider.id})
)
@@ -422,9 +487,9 @@ class TestProviderViewSet:
assert response.status_code == status.HTTP_403_FORBIDDEN
def test_partial_update_provider_with_all_permissions(
self, authenticated_client_rbac, providers_fixture
self, authenticated_client_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
payload = {
"data": {
"type": "providers",
@@ -441,9 +506,9 @@ class TestProviderViewSet:
assert response.json()["data"]["attributes"]["alias"] == "updated_alias"
def test_partial_update_provider_with_no_permissions(
self, authenticated_client_no_permissions_rbac, providers_fixture
self, authenticated_client_no_permissions_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
update_payload = {
"data": {
"type": "providers",
@@ -464,7 +529,7 @@ class TestProviderViewSet:
mock_delete_task,
mock_task_get,
authenticated_client_rbac,
providers_fixture,
aws_provider,
tasks_fixture,
):
prowler_task = tasks_fixture[0]
@@ -473,7 +538,7 @@ class TestProviderViewSet:
mock_delete_task.return_value = task_mock
mock_task_get.return_value = prowler_task
provider1, *_ = providers_fixture
provider1 = aws_provider
response = authenticated_client_rbac.delete(
reverse("provider-detail", kwargs={"pk": provider1.id})
)
@@ -485,9 +550,9 @@ class TestProviderViewSet:
assert response.headers["Content-Location"] == f"/api/v1/tasks/{task_mock.id}"
def test_delete_provider_with_no_permissions(
self, authenticated_client_no_permissions_rbac, providers_fixture
self, authenticated_client_no_permissions_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
response = authenticated_client_no_permissions_rbac.delete(
reverse("provider-detail", kwargs={"pk": provider.id})
)
@@ -500,7 +565,7 @@ class TestProviderViewSet:
mock_provider_connection,
mock_task_get,
authenticated_client_rbac,
providers_fixture,
aws_provider,
tasks_fixture,
):
prowler_task = tasks_fixture[0]
@@ -510,7 +575,7 @@ class TestProviderViewSet:
mock_provider_connection.return_value = task_mock
mock_task_get.return_value = prowler_task
provider1, *_ = providers_fixture
provider1 = aws_provider
assert provider1.connected is None
assert provider1.connection_last_checked_at is None
@@ -525,9 +590,9 @@ class TestProviderViewSet:
assert response.headers["Content-Location"] == f"/api/v1/tasks/{task_mock.id}"
def test_connection_with_no_permissions(
self, authenticated_client_no_permissions_rbac, providers_fixture
self, authenticated_client_no_permissions_rbac, aws_provider
):
provider = providers_fixture[0]
provider = aws_provider
response = authenticated_client_no_permissions_rbac.post(
reverse("provider-connection", kwargs={"pk": provider.id})
)
@@ -540,12 +605,10 @@ class TestLimitedVisibility:
TEST_PASSWORD = "Thisisapassword123@"
@pytest.fixture
def limited_admin_user(
self, django_db_setup, django_db_blocker, tenants_fixture, providers_fixture
):
def limited_admin_user(self, django_db_blocker, tenants_fixture, aws_provider):
with django_db_blocker.unblock():
tenant = tenants_fixture[0]
provider = providers_fixture[0]
provider = aws_provider
user = User.objects.create_user(
name="testing",
email=self.TEST_EMAIL,
@@ -592,25 +655,17 @@ class TestLimitedVisibility:
@pytest.fixture
def authenticated_client_rbac_limited(
self, limited_admin_user, tenants_fixture, client
self,
limited_admin_user,
tenants_fixture,
authenticated_client_for_tenant_factory,
):
client.user = limited_admin_user
tenant_id = tenants_fixture[0].id
serializer = TokenSerializer(
data={
"type": "tokens",
"email": self.TEST_EMAIL,
"password": self.TEST_PASSWORD,
"tenant_id": tenant_id,
}
return authenticated_client_for_tenant_factory(
limited_admin_user, tenants_fixture[0]
)
serializer.is_valid(raise_exception=True)
access_token = serializer.validated_data["access"]
client.defaults["HTTP_AUTHORIZATION"] = f"Bearer {access_token}"
return client
def test_integrations(
self, authenticated_client_rbac_limited, integrations_fixture, providers_fixture
self, authenticated_client_rbac_limited, integrations_fixture
):
# Integration 2 is related to provider1 and provider 2
# This user cannot see provider 2
@@ -626,11 +681,11 @@ class TestLimitedVisibility:
response.json()["data"]["relationships"]["providers"]["meta"]["count"] == 1
)
@pytest.mark.usefixtures("scan_summaries_fixture")
def test_overviews_providers(
self,
authenticated_client_rbac_limited,
scan_summaries_fixture,
providers_fixture,
provider_factory,
):
# By default, the associated provider is the one which has the overview data
response = authenticated_client_rbac_limited.get(reverse("overview-providers"))
@@ -640,7 +695,7 @@ class TestLimitedVisibility:
# Changing the provider visibility, no data should be returned
# Only the associated provider to that group is changed
new_provider = providers_fixture[1]
new_provider = provider_factory()
ProviderGroupMembership.objects.all().update(provider=new_provider)
response = authenticated_client_rbac_limited.get(reverse("overview-providers"))
@@ -648,6 +703,7 @@ class TestLimitedVisibility:
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == 0
@pytest.mark.usefixtures("scan_summaries_fixture")
@pytest.mark.parametrize(
"endpoint_name",
[
@@ -659,8 +715,7 @@ class TestLimitedVisibility:
self,
endpoint_name,
authenticated_client_rbac_limited,
scan_summaries_fixture,
providers_fixture,
provider_factory,
):
# By default, the associated provider is the one which has the overview data
response = authenticated_client_rbac_limited.get(
@@ -673,7 +728,7 @@ class TestLimitedVisibility:
# Changing the provider visibility, no data should be returned
# Only the associated provider to that group is changed
new_provider = providers_fixture[1]
new_provider = provider_factory()
ProviderGroupMembership.objects.all().update(provider=new_provider)
response = authenticated_client_rbac_limited.get(
@@ -684,11 +739,11 @@ class TestLimitedVisibility:
data = response.json()["data"]["attributes"].values()
assert all(value == 0 for value in data)
@pytest.mark.usefixtures("scan_summaries_fixture")
def test_overviews_services(
self,
authenticated_client_rbac_limited,
scan_summaries_fixture,
providers_fixture,
provider_factory,
):
# By default, the associated provider is the one which has the overview data
response = authenticated_client_rbac_limited.get(
@@ -700,7 +755,7 @@ class TestLimitedVisibility:
# Changing the provider visibility, no data should be returned
# Only the associated provider to that group is changed
new_provider = providers_fixture[1]
new_provider = provider_factory()
ProviderGroupMembership.objects.all().update(provider=new_provider)
response = authenticated_client_rbac_limited.get(
@@ -762,6 +817,48 @@ class TestRolePermissions:
assert response.status_code == status.HTTP_403_FORBIDDEN
@pytest.mark.django_db
class TestHasPermissions:
def test_permissions_are_combined_across_roles(
self, create_test_user_rbac_no_roles
):
user = create_test_user_rbac_no_roles
tenant = Membership.objects.get(user=user).tenant
manage_users_role = Role.objects.create(
name="manage_users_only",
tenant=tenant,
manage_users=True,
)
UserRoleRelationship.objects.create(
user=user,
role=manage_users_role,
tenant=tenant,
)
request = Mock(user=user, tenant_id=tenant.id)
view = Mock(
required_permissions=[
Permissions.MANAGE_USERS,
Permissions.MANAGE_ACCOUNT,
]
)
permission = HasPermissions()
assert not permission.has_permission(request, view)
manage_account_role = Role.objects.create(
name="manage_account_only",
tenant=tenant,
manage_account=True,
)
UserRoleRelationship.objects.create(
user=user,
role=manage_account_role,
tenant=tenant,
)
assert permission.has_permission(request, view)
@pytest.mark.django_db
class TestUserRoleLinkPermissions:
def test_link_user_roles_with_manage_account_only_allowed(
@@ -0,0 +1,85 @@
from unittest.mock import MagicMock, patch
import pytest
from api.attack_paths.retryable_session import RetryableSession
from neo4j.exceptions import ServiceUnavailable
class TestRetryableSession:
@patch("api.attack_paths.retryable_session.time.sleep")
@patch("api.attack_paths.retryable_session.random.uniform", return_value=3.0)
def test_custom_retry_uses_backoff_and_a_fresh_session(
self, mock_uniform, mock_sleep
):
retryable_error = RuntimeError("retryable")
first_session = MagicMock()
first_session.execute_write.side_effect = retryable_error
second_session = MagicMock()
second_session.execute_write.return_value = "success"
session_factory = MagicMock(side_effect=[first_session, second_session])
work = MagicMock()
session = RetryableSession(
session_factory=session_factory,
max_retries=3,
retry_if=lambda exc: exc is retryable_error,
initial_retry_delay_seconds=2,
)
assert session.execute_write(work) == "success"
assert session_factory.call_count == 2
first_session.close.assert_called_once_with()
mock_uniform.assert_called_once_with(2.0, 4.0)
mock_sleep.assert_called_once_with(3.0)
def test_connection_errors_remain_retryable(self):
first_session = MagicMock()
first_session.run.side_effect = ServiceUnavailable("unavailable")
second_session = MagicMock()
second_session.run.return_value = "success"
session_factory = MagicMock(side_effect=[first_session, second_session])
session = RetryableSession(session_factory=session_factory, max_retries=1)
assert session.run("RETURN 1") == "success"
first_session.close.assert_called_once_with()
def test_non_retryable_error_is_raised_without_refreshing_session(self):
error = RuntimeError("do not retry")
driver_session = MagicMock()
driver_session.execute_write.side_effect = error
session_factory = MagicMock(return_value=driver_session)
session = RetryableSession(
session_factory=session_factory,
max_retries=3,
retry_if=lambda _: False,
initial_retry_delay_seconds=2,
)
with pytest.raises(RuntimeError) as exc_info:
session.execute_write(MagicMock())
assert exc_info.value is error
session_factory.assert_called_once_with()
driver_session.close.assert_not_called()
def test_retry_exhaustion_raises_the_last_error(self):
error = RuntimeError("still retryable")
driver_sessions = [MagicMock() for _ in range(3)]
for driver_session in driver_sessions:
driver_session.execute_write.side_effect = error
session_factory = MagicMock(side_effect=driver_sessions)
session = RetryableSession(
session_factory=session_factory,
max_retries=2,
retry_if=lambda _: True,
)
with pytest.raises(RuntimeError) as exc_info:
session.execute_write(MagicMock())
assert exc_info.value is error
assert session_factory.call_count == 3
driver_sessions[0].close.assert_called_once_with()
driver_sessions[1].close.assert_called_once_with()
driver_sessions[2].close.assert_not_called()
+25 -1
View File
@@ -1,9 +1,33 @@
import logging
from unittest.mock import MagicMock
from unittest.mock import MagicMock, patch
from config.settings import sentry as sentry_settings
from config.settings.sentry import before_send
def test_initialize_sentry_skips_without_dsn():
with (
patch.object(sentry_settings.env, "str", return_value=""),
patch.object(sentry_settings.sentry_sdk, "init") as mock_init,
):
sentry_settings.initialize_sentry()
mock_init.assert_not_called()
def test_initialize_sentry_uses_configured_dsn():
sentry_dsn = "https://fake-public-key@sentry.example.invalid/1"
with (
patch.object(sentry_settings.env, "str", return_value=sentry_dsn),
patch.object(sentry_settings.sentry_sdk, "init") as mock_init,
):
sentry_settings.initialize_sentry()
assert mock_init.call_args.kwargs["dsn"] == sentry_dsn
assert mock_init.call_args.kwargs["before_send"] is sentry_settings.before_send
def _make_log_record(msg, level=logging.ERROR, name="test", args=None):
"""Build a real LogRecord so getMessage() works like in production."""
record = logging.LogRecord(
+129 -2
View File
@@ -1,6 +1,9 @@
import pytest
from api.v1.serializer_utils.integrations import S3ConfigSerializer
from api.v1.serializers import ImageProviderSecret
from api.v1.serializer_utils.integrations import (
JiraCredentialSerializer,
S3ConfigSerializer,
)
from api.v1.serializers import ImageProviderSecret, KubernetesProviderSecret
from rest_framework.exceptions import ValidationError
@@ -100,6 +103,59 @@ class TestS3ConfigSerializer:
assert "output_directory" in serializer.errors
class TestJiraCredentialSerializer:
@pytest.mark.parametrize(
"domain",
(
"a",
"prowler",
"prowler-domain",
"A1-b2-C3",
"a" * 63,
),
)
def test_valid_site_name(self, domain):
serializer = JiraCredentialSerializer(
data={
"user_mail": "testing@prowler.com",
"api_token": "fake-api-token",
"domain": domain,
}
)
assert serializer.is_valid(), serializer.errors
@pytest.mark.parametrize(
"domain",
(
"169.254.169.254#",
"internal/service",
"internal?target",
"internal\\target",
"internal:8000",
"user@internal",
"example.atlassian.net",
"-prowler",
"prowler-",
"a" * 64,
" prowler",
"prowler ",
"prowler\n",
),
)
def test_invalid_site_name(self, domain):
serializer = JiraCredentialSerializer(
data={
"user_mail": "testing@prowler.com",
"api_token": "fake-api-token",
"domain": domain,
}
)
assert not serializer.is_valid()
assert "domain" in serializer.errors
class TestImageProviderSecret:
"""Test cases for ImageProviderSecret validation."""
@@ -132,3 +188,74 @@ class TestImageProviderSecret:
serializer = ImageProviderSecret(data={"registry_password": "pass"})
assert not serializer.is_valid()
assert "non_field_errors" in serializer.errors
class TestKubernetesProviderSecret:
def test_valid_static_kubeconfig_is_accepted(self):
kubeconfig_content = """
apiVersion: v1
kind: Config
clusters:
- name: test-cluster
cluster:
server: https://kubernetes.example.test
users:
- name: test-user
user:
token: test-token
contexts:
- name: test-context
context:
cluster: test-cluster
user: test-user
current-context: test-context
"""
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": kubeconfig_content}
)
assert serializer.is_valid()
def test_kubeconfig_with_exec_authentication_is_rejected(self):
kubeconfig_content = """
apiVersion: v1
kind: Config
clusters:
- name: test-cluster
cluster:
server: https://kubernetes.example.test
users:
- name: test-user
user:
exec:
apiVersion: client.authentication.k8s.io/v1
command: kubectl
contexts:
- name: test-context
context:
cluster: test-cluster
user: test-user
current-context: test-context
"""
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": kubeconfig_content}
)
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors
def test_malformed_kubeconfig_is_rejected(self):
serializer = KubernetesProviderSecret(
data={"kubeconfig_content": "apiVersion: ["}
)
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors
def test_non_mapping_kubeconfig_is_rejected(self):
serializer = KubernetesProviderSecret(data={"kubeconfig_content": "[]"})
assert not serializer.is_valid()
assert "kubeconfig_content" in serializer.errors

Some files were not shown because too many files have changed in this diff Show More