Compare commits

...
Author SHA1 Message Date
Víctor Fernández Poyatos bcbf68736c fix(routing): apply correct routing and RLS settings to every connection 2025-06-17 12:28:18 +02:00
Víctor Fernández Poyatos 04056813eb Merge branch 'master' into PRWLR-4758-django-must-support-read-only-replica-database-config 2025-06-12 13:26:13 +02:00
Pablo Lara e218435b2f fix: improve error handling in UpdateViaCredentialsForm with early re… (#7988) 2025-06-12 11:39:49 +02:00
5ec34ad5e7 chore(regions_update): Changes in regions for AWS services (#7973)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-06-12 17:24:15 +08:00
Pedro MartínandSergio Garcia c4b0859efd fix(dashboard): handle account uids with 0 at start and end (#7955)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-06-12 17:21:52 +08:00
Pedro Martín 1241a490f9 fix(kubernetes): change object type to set for apiserver check (#7952) 2025-06-12 17:02:48 +08:00
Pedro Martín 4ec498a612 fix(k8s): remove typo for PCI 4.0 compliance framework (#7971) 2025-06-12 16:41:58 +08:00
Pedro MartínandSergio Garcia 119c5e80a9 feat(gcp): add NIS 2 compliance framework (#7912)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-06-12 16:40:33 +08:00
sumit-tft d393bc48a2 fix(PRWLR-7380): button nesting hydration error (#7998) 2025-06-12 10:02:20 +02:00
Daniel Barranquero e09e3855b1 fix(gcp): remove azure video from gcp docs (#8001) 2025-06-12 09:54:25 +02:00
Alejandro Bailo 8751615faa feat: MittreAtack compliance detailed view (#8002) 2025-06-12 09:27:47 +02:00
Prowler Botandprowler-bot e7c17ab0b3 chore(regions_update): Changes in regions for AWS services (#7898)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-06-12 15:14:28 +08:00
dependabot[bot] f05d3eb334 chore(deps): bump trufflesecurity/trufflehog from 3.88.26 to 3.88.35 (#7896)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-12 15:14:02 +08:00
dependabot[bot] cf449d4607 chore(deps): bump aws-actions/configure-aws-credentials from 4.1.0 to 4.2.1 (#7895)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-12 15:13:35 +08:00
dependabot[bot] b338ac9add chore(deps): bump codecov/codecov-action from 5.4.2 to 5.4.3 (#7894)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-12 15:13:12 +08:00
dependabot[bot] 366d2b392a chore(deps): bump docker/build-push-action from 6.16.0 to 6.18.0 (#7893)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-12 15:12:52 +08:00
dependabot[bot] 41fc536b44 chore(deps): bump github/codeql-action from 3.28.16 to 3.28.18 (#7892)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-12 15:12:28 +08:00
Adrián Jesús Peña Rodríguez e042445ecf fix(migration): create site stuff before socialaccount (#7999) 2025-06-11 13:34:21 +02:00
Víctor Fernández Poyatos c17129afe3 revert: RLS transactions handling and DB custom backend (#7994) 2025-06-11 14:47:10 +05:45
Alejandro Bailo 4876d8435c feat: generic compliance detailed view (#7990) 2025-06-11 09:40:53 +02:00
Pedro Martín 1bd0d774e5 feat(mutelist): make validate_mutelist method static (#7811) 2025-06-11 11:33:49 +05:45
Alejandro Bailo c119cece89 feat: ThreatScore compliance detailed view (#7979) 2025-06-10 10:43:27 +02:00
Adrián Jesús Peña Rodríguez e24b211d22 feat(sso): add sso with saml to API (#7822) 2025-06-10 10:17:54 +02:00
Hugo Pereira BritoandAndoni Alonso c589c95727 feat(storage): add new check storage_account_key_access_disabled (#7974)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-06-10 08:23:09 +02:00
Hugo Pereira Brito 7e4f1a73bf feat(storage): add new check storage_ensure_file_shares_soft_delete_is_enabled (#7966) 2025-06-10 08:09:11 +02:00
Pepe Fagoaga 4d00aece45 chore(changelog): move entry for their version (#7969) 2025-06-09 21:50:13 +05:45
Hugo Pereira Brito 49aaf011aa fix(parser): add GitHub provider to prowler -h usage section (#7906) 2025-06-09 17:47:29 +02:00
Adrián Jesús Peña Rodríguez 898934c7f8 chore: update django version (#7984) 2025-06-09 17:33:16 +02:00
Pepe Fagoaga 81c4b5a9c1 chore(api): Delete old docker compose file (#7982) 2025-06-09 21:01:52 +05:45
Pepe Fagoaga fe31656ffe fix(k8s): return a session if using kubeconfig_content (#7953) 2025-06-09 19:11:59 +05:45
Hugo Pereira Brito 359059dee6 fix(docs): add Organization.Read.All to M365 provider requirements (#7961) 2025-06-09 12:11:14 +02:00
Alejandro Bailo 2eaa37921d feat: KISA detailed view (#7965) 2025-06-09 09:29:34 +02:00
Pablo Lara 3a99909b75 chore: align Next.js version to 14.2.29 across Prowler and Cloud (#7962) 2025-06-06 13:54:42 +02:00
Pablo Lara 2ecd9ad2c5 docs: update changelog (#7960) 2025-06-06 13:17:38 +02:00
Alejandro BailoandPablo Lara 50dc396aa3 feat: scan id filter drowpdown (#7949)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-06-06 12:38:14 +02:00
Andoni Alonso acf333493a chore(api): reorder docker layers to speed up build times (#7957) 2025-06-06 10:42:14 +02:00
Pedro Martín bd6272f5a7 feat(docs): add information about tenants and read-only roles (#7956) 2025-06-06 10:14:33 +02:00
Pepe Fagoaga 8c95e1efaf chore: update API changelog for v5.7.3 (#7948) 2025-06-05 15:54:36 +02:00
Hugo Pereira BritoandVíctor Fernández Poyatos 845a0aa0d5 fix(changelog): add entries for password encryption in v5.7.3 (#7939)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-06-05 14:23:12 +02:00
Hugo Pereira Brito 75a11be9e6 fix(docs): add final permission assignments example (#7943) 2025-06-05 18:07:43 +05:45
Hugo Pereira BritoandPepe Fagoaga a778d005b6 fix(docs): add mfa warning for users (#7924)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-06-05 17:55:27 +05:45
Pedro Martín 1281f4ec5e chore(changelog): update following the correct format (#7908) 2025-06-05 17:52:36 +05:45
Víctor Fernández Poyatos 6332427e5e fix(compliance): add manual status to requirements (#7938) 2025-06-05 10:54:51 +02:00
Alejandro Bailo d89df83904 fix: Improve the perfomance removing regions heatmap (#7934) 2025-06-05 08:13:47 +02:00
Víctor Fernández Poyatos be420afebc feat(database): handle already closed connections (#7935) 2025-06-04 16:09:36 +02:00
Adrián Jesús Peña Rodríguez fb914a2c90 revert: remove get_with_retry (#7932) 2025-06-04 15:01:47 +02:00
Pablo Lara 4ac3cfc33d docs: update changelog (#7931) 2025-06-04 13:54:25 +02:00
Alejandro Bailo c74360ab63 fix: clear filters sync (#7928) 2025-06-04 13:32:52 +02:00
Alejandro Bailo 4dc4d82d42 feat: aws-well-architected compliance detailed view (#7925) 2025-06-04 12:26:27 +02:00
Víctor Fernández Poyatos 6e7a32cb51 revert(views): calling order to initial view method (#7921) 2025-06-03 16:38:00 +02:00
Alejandro BailoandVíctor Fernández Poyatos 49e501c4be feat: CIS compliance detail view (#7913)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-06-03 15:47:46 +02:00
Víctor Fernández Poyatos 9ee78fe65f fix(views): calling order to initial view method (#7918) 2025-06-03 13:34:44 +02:00
Víctor Fernández Poyatos 7a0549d39c fix(rls): Apply persistent RLS transactions (#7916) 2025-06-03 13:10:41 +02:00
3e8c86d880 feat: ISO compliance detail view (#7897)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-06-03 09:20:52 +02:00
Pablo Lara e34c18757d fix: Fix named export for addCredentialsServiceAccountFormSchema (#7909) 2025-06-03 08:33:24 +02:00
Alejandro BailoandVíctor Fernández Poyatos 5c1a47d108 feat: compliance detail view + ENS (#7853)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-06-02 18:20:22 +02:00
Víctor Fernández Poyatos 59c51d5a4a feat(compliance): Rework compliance overviews (#7877) 2025-06-02 17:06:24 +02:00
Pedro MartínandRubén De la Torre Vico 66aa67f636 feat(changelog): update version with fixes (#7904)
Co-authored-by: Rubén De la Torre Vico <ruben@prowler.com>
2025-06-02 12:32:45 +02:00
Pablo Lara bdda377482 docs: update the changelog (#7901) 2025-06-02 11:49:04 +02:00
Hugo Pereira Brito aa11ed70bd chore(docs): replace old permission images (#7900) 2025-06-02 11:47:11 +02:00
Adrián Jesús Peña RodríguezandPepe Fagoaga 0580dca6cf fix: set user_id for tenant operations (#7890)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-06-02 11:06:49 +02:00
Pablo Lara 678ef0ab5a feat(providers): setup workflow to support new GCP credential method (#7872) 2025-06-02 10:23:39 +02:00
César Arroba 4888c27713 chore: fix commit sha when a pr is merged (#7889) 2025-05-30 17:40:57 +05:45
Hugo Pereira Brito b256c10622 chore: replace Directory.Read.All permission to Domain.Read.All for Azure (#7888) 2025-05-30 10:24:49 +02:00
Adrián Jesús Peña RodríguezandVíctor Fernández Poyatos 878e4e0bbc fix: add new get method to avoid race conditions when creating async tasks (#7876)
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-05-30 10:07:32 +02:00
Hugo Pereira Brito 6c3653c483 fix(docs): remove warning of encrypted password for cloud (#7886) 2025-05-30 12:01:32 +04:00
Daniel BarranqueroandPepe Fagoaga 71ac703e6f fix(api): connection correctly reflected (#7831)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-05-29 20:23:15 +05:45
Sergio Garcia a89e3598f2 fix(gcp): test connection by verifying token (#7882) 2025-05-29 13:20:53 +02:00
Alison Vilela 5d043cc929 fix(awslambda): aws service awslambda not working (#7869) 2025-05-29 12:50:23 +05:45
Pepe Fagoaga 921f94ebbf fix(k8s): UID validation for valid context names (#7871) 2025-05-29 12:32:57 +05:45
sumit-tft 48c9ed8a79 fix(ui): increase limit to retrieve more than 10 scan list (#7865) 2025-05-29 07:52:36 +02:00
Hugo Pereira Brito 12987ec9f9 fix(admincenter): service and group visibility (#7870) 2025-05-28 16:48:49 +02:00
Hugo Pereira Brito 40b90ed063 fix(tests): typo in m365 domain test (#7866) 2025-05-28 16:43:58 +02:00
Alejandro Bailo 60314e781f feat: enhance CustomDropdownFilter (#7868) 2025-05-28 16:30:28 +02:00
Harsh KumarandHarsh Kumar bc56d48595 feat(dashboard): add client-side search functionality to findings table (#7804)
Co-authored-by: Harsh Kumar <harsh.k@cybersecurist.com>
2025-05-28 11:44:01 +02:00
Pedro Martín 2d71cef3d5 feat(azure): add NIS 2 compliance framework (#7857) 2025-05-28 11:35:40 +02:00
Daniel BarranqueroandMrCloudSec 41f6637497 fix(defender): update defender_ensure_notify_alerts_severity_is_high logic (#7862)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-28 10:32:44 +02:00
Pedro MartínandSergio Garcia c2e54bbbcc fix(threatscore): remove compliance name in tests to remove dummy files (#7859)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-28 10:03:20 +02:00
sumit-tft df8aacd09d fix(ui): Added missing icons (kisa, prowlerThreat) on compliance page (#7860) 2025-05-28 09:51:28 +02:00
Matt Keeler 2dd6be59b9 fix(m365): add compliantDevice grant control support (#7844) 2025-05-28 09:05:00 +02:00
9e8e3eb0e6 fix(m365): update documentation (#7823)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-05-28 08:52:03 +02:00
Sergio GarciaandPedro Martín 3728430f8c chore: update README (#7842)
Co-authored-by: Pedro Martín <pedromarting3@gmail.com>
2025-05-27 14:25:37 +02:00
sumit-tftandPablo Lara ea97de7f43 fix(ui): updated to use the correct message when download report clicked (#7758)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-05-27 10:51:08 +02:00
Rubén De la Torre Vico f254a4bc0d feat(app): split SDK App service calls (#7778) 2025-05-27 09:52:50 +02:00
Pedro Martín 66acfd8691 feat(aws): add NIS2 compliance framework (#7839) 2025-05-27 09:35:57 +02:00
Matt Keeler 02ca82004f fix(typo): minor language updates (#7843) 2025-05-27 09:26:51 +02:00
Rubén De la Torre Vico 60b5a79b27 fix(vpc): change the ServiceName from EC2 to VPC (#7840) 2025-05-26 17:52:59 +02:00
be1e3e942b feat(api): support GCP Service Account key (#7824)
Co-authored-by: Sergio Garcia <38561120+garcitm@users.noreply.github.com>
Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2025-05-26 15:42:39 +02:00
Sergio Garcia 3658e85cfc chore(github): add Branch class (#7838) 2025-05-26 14:34:44 +02:00
Adrián Jesús Peña Rodríguez 15e4d1acce refactor(reports): change API response message when tasks are running (#7837) 2025-05-26 12:20:05 +02:00
Andoni Alonso 44afd9ed31 fix: repository repository_dependency_scanning_enabled check logic (#7834) 2025-05-26 10:44:19 +02:00
Andoni Alonso 4f099c5663 refactor(github): use owner instead of repository in findings attributes (#7833) 2025-05-26 10:40:41 +02:00
Andoni AlonsoandSergio Garcia eaec683eb9 feat(repositoy): add new check repository_inactive_not_archived (#7786)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-26 10:39:09 +02:00
Adrián Jesús Peña Rodríguez 50bcd828e9 fix(reports): change invalid search term for tasks (#7830) 2025-05-26 10:24:11 +02:00
Alejandro BailoandPablo Lara 91545e409e feat: change tenant name in /profile page (#7829)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-05-23 14:45:28 +02:00
Alejandro BailoandPablo Lara 33031d2c96 feat: implement provider UID extraction and mapping in scans pages (#7820)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-05-23 13:42:35 +02:00
Hugo Pereira Brito 1b42dda817 fix(formSchemas): encrypted password typo (#7828) 2025-05-23 12:52:17 +02:00
Hugo Pereira Brito f726d964a8 fix(m365): remove last encrypted password appearances (#7825) 2025-05-23 12:27:57 +02:00
Hugo Pereira BritoandDaniel Barranquero 36aaec8a55 chore(m365powershell): manage encryption from plaintext password (#7784)
Co-authored-by: Daniel Barranquero <danielbo2001@gmail.com>
2025-05-22 17:36:58 +02:00
Andoni AlonsoandSergio Garcia 99164ce93e feat(repository): add new check repository_default_branch_requires_signed_commits (#7777)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-22 12:45:13 +02:00
Andoni AlonsoandSergio Garcia 7ebc5d3c31 feat(repository): add new check repository_dependency_scanning_enabled (#7771)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-22 12:22:59 +02:00
Andoni AlonsoandSergio Garcia 06ff3db8af feat(repository): add new check repository_secret_scanning_enabled (#7759)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-22 11:23:42 +02:00
Alejandro Bailo c44ea3943e feat: resources in finding tables (#7813) 2025-05-22 08:58:25 +02:00
Andoni AlonsoandSergio Garcia d036e0054b feat(repository): add new check repository_default_branch_requires_codeowners_review (#7753)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-21 16:18:55 +02:00
Pedro Martín f72eb7e212 fix(files): remove empty files (#7819) 2025-05-21 16:15:04 +02:00
Andoni Alonso 62dcbc2961 feat(repository): add new check repository_has_codeowners_file (#7752) 2025-05-21 15:28:30 +02:00
Hugo Pereira Brito dddec4c688 fix(m365): add powershell.close() to msgraph services (#7816) 2025-05-21 15:13:03 +02:00
Sergio GarciaandPepe Fagoaga 6d00554082 chore(readme): add Prowler Hub link (#7814)
Co-authored-by: Pepe Fagoaga <pepe@prowler.com>
2025-05-21 17:46:54 +05:45
Pedro MartínandSergio Garcia 65d3fcee4c feat(prowler-threatscore): add Weight field inside req (#7795)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-21 12:57:10 +02:00
Pedro MartínandSergio Garcia 16cd0e4661 feat(prowler_threatscore): add a level for accordion in dashboard (#7739)
Co-authored-by: Sergio Garcia <hello@mistercloudsec.com>
2025-05-21 12:46:47 +02:00
Hugo Pereira BritoandAndoni A. 6e184dae93 fix(admincenter): admincenter_users_admins_reduced_license_footprint logic (#7779)
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-05-21 12:46:35 +02:00
Pablo Lara 118f3d163d docs: update changelog UI (#7808) 2025-05-21 12:39:48 +02:00
Pedro Martín 7d84d67935 feat(gcp): add CIS 4.0 compliance framework (#7785) 2025-05-21 12:38:34 +02:00
Víctor Fernández Poyatos 6ab6099afc fix(db-router): allow relation between prowler_user and the rest 2025-05-21 12:36:32 +02:00
Víctor Fernández Poyatos 75db2ace1b Merge branch 'master' into PRWLR-4758-django-must-support-read-only-replica-database-config 2025-05-21 12:26:29 +02:00
Víctor Fernández Poyatos 9b45bec015 chore: update .gitignore 2025-05-21 12:21:45 +02:00
Víctor Fernández Poyatos 1c1c58c975 feat(findings): Add new index for finding UID lookup (#7800) 2025-05-21 11:56:54 +02:00
Andoni Alonso 31ea672c61 fix: move changes to release 5.8 (#7801) 2025-05-21 11:45:54 +02:00
Toni de la Fuente 7016779b8e chore(README): update README.md (#7799) 2025-05-21 11:31:23 +02:00
Pedro MartínandMrCloudSec 4e958fdf39 feat(kubernetes): add CIS 1.11 compliance framework (#7790)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-21 11:09:47 +02:00
Pedro Martín c6259b6c75 fix(dashboard): remove typo from subscribe cards (#7792) 2025-05-21 11:08:52 +02:00
Sergio Garcia 021e243ada feat(kubernetes): support HTTPS_PROXY and K8S_SKIP_TLS_VERIFY (#7720) 2025-05-21 10:49:18 +02:00
Alejandro BailoandPablo Lara acdf420941 feat: profile page (#7780)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-05-21 10:47:32 +02:00
Hugo Pereira BritoandAndoni Alonso 4e84507130 feat(entra): add new check entra_users_mfa_capable (#7734)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-05-21 10:31:56 +02:00
Prowler Botandprowler-bot 2a61610fec chore(regions_update): Changes in regions for AWS services (#7774)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-05-21 10:29:08 +02:00
Daniel BarranqueroandAndoni Alonso 9b127eba93 feat(admincenter): add new check admincenter_external_calendar_sharing_disabled (#7733)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-05-21 09:14:45 +02:00
Hugo Pereira BritoandAndoni Alonso 1a89d65516 fix(m365powershell): add sanitize to test_credentials (#7761)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-05-21 08:49:04 +02:00
Daniel BarranqueroandAndoni Alonso 84749df708 feat(admincenter): add new check admincenter_organization_customer_lockbox_enabled (#7732)
Co-authored-by: Andoni Alonso <14891798+andoniaf@users.noreply.github.com>
2025-05-21 08:48:36 +02:00
Pepe Fagoaga 6f7cd85a18 chore(backport): create label on minor release (#7791) 2025-05-21 12:14:30 +05:45
Alejandro Bailo ad39061e1a fix: retrieve more than 10 providers (#7793) 2025-05-21 08:07:43 +02:00
Pablo Lara 615bacccaf chore: tweak some wording for consistency (#7794) 2025-05-21 07:59:53 +02:00
Prowler Botandprowler-bot b3a2479fab chore(release): Bump version to v5.8.0 (#7788)
Co-authored-by: prowler-bot <179230569+prowler-bot@users.noreply.github.com>
2025-05-20 22:27:21 +05:45
sumit-tftandPablo Lara 871c877a33 fix: AWS I AM role validation when field is empty (#7787)
Co-authored-by: Pablo Lara <larabjj@gmail.com>
2025-05-20 11:25:40 +02:00
Pedro Martín 7fd58de3bf feat(export): support m365 - prowler threatscore (#7783) 2025-05-19 15:59:42 +02:00
Víctor Fernández Poyatos 40f24b4d70 fix(providers): Fix m365 UID validation (#7781) 2025-05-19 13:34:46 +02:00
Adrián Jesús Peña Rodríguez d8f80699d4 chore: update api changelog (#7775) 2025-05-19 14:52:32 +05:45
Pablo Lara f24d0efc77 docs: update changelog (#7773) 2025-05-19 14:34:28 +05:45
Hugo Pereira Brito a18dd76a5a chore(m365): accept all tenant domains in authentication (#7746) 2025-05-19 13:53:54 +05:45
Pedro Martín a2362b4bbc fix(cis): rename and add sections and subsections (#7738) 2025-05-19 09:42:04 +02:00
Pedro Martín e5f1c2b19c feat(aws): add CIS 5.0 compliance framework (#7766) 2025-05-19 09:41:56 +02:00
Pedro Martín 0490ab6944 docs(checks): improve docs related with checks (#7768) 2025-05-19 09:17:14 +02:00
Sergio Garcia 97baa8a1e6 chore(ec2): improve severity logic in SG all ports open check (#7764) 2025-05-16 15:09:48 +02:00
Hugo Pereira BritoandMrCloudSec 637ebdc3db feat(repository): add new check repository_branch_delete_on_merge_enabled (#6209)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-16 15:03:37 +02:00
Hugo Pereira BritoandMrCloudSec 451b36093f feat(repository): add new check repository_default_branch_requires_conversation_resolution (#6208)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-16 14:57:15 +02:00
Víctor Fernández Poyatos beb0457aff fix(findings): Fix latest metadata backfill condition and optimization (#7765) 2025-05-16 14:50:40 +02:00
Víctor Fernández Poyatos 0335ea4e0b fix(findings): Fix latest metadata backfill condition (#7762) 2025-05-16 12:41:12 +02:00
sumit-tft 355abca5a3 fix(ui): Removed the alias if not available in findings detail page (#7751) 2025-05-16 09:02:47 +02:00
sumit-tft 7d69cc4cd9 fix: Updated the high risk section provider icons to make it consistent (#7706) 2025-05-16 08:53:34 +02:00
Hugo Pereira BritoandMrCloudSec cdc4b362a4 feat(repository): add new check repository_default_branch_protection_applies_to_admins (#6205)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-16 08:29:45 +02:00
Pablo Lara 6417e6bbba feat: use getFindingsLatest when no scan or date filters are applied (#7756) 2025-05-16 08:18:12 +02:00
Víctor Fernández Poyatos b810d45d34 feat(findings): Add /findings/latest and /findings/metadata/latest endpoints (#7743) 2025-05-15 16:08:09 +02:00
Ogonna IwunzeandMrCloudSec f5a2695c3b fix(check): Add support for condition with restriction on SNS endpoint (#7750)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-15 16:00:00 +02:00
Hugo Pereira BritoandMrCloudSec 977c788fff feat(repository): add new check repository_default_branch_status_checks_required (#6204)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-15 15:33:49 +02:00
Hugo Pereira Brito 21f8b5dbad fix(check): add missing __init__.py files (#7748) 2025-05-15 11:22:58 +02:00
1c874d1283 feat(repository): add new check repository_default_branch_deletion_disabled (#6200)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-05-15 08:33:36 +02:00
Hugo Pereira BritoandMrCloudSec 8f9bdae2b7 feat(repository): add new check repository_default_branch_disallows_force_push (#6197)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-14 16:48:47 +02:00
Pablo LaraandStylusFrost 600813fb99 fix: force z-index componet select provider (#7744)
Co-authored-by: StylusFrost <pm.diaz.pena@gmail.com>
2025-05-14 15:19:41 +02:00
Hugo Pereira BritoandMrCloudSec 5a9ccd60a0 feat(repository): add new check repository_default_branch_requires_linear_history (#6162)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-14 14:37:27 +02:00
beb7a53efe feat(repository): add new check repository_default_branch_protection_enabled (#6161)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>
2025-05-14 13:42:59 +02:00
Hugo Pereira BritoandMrCloudSec 8431ce42a1 feat(organization): add new check organization_members_mfa_required (#6304)
Co-authored-by: MrCloudSec <hello@mistercloudsec.com>
2025-05-14 13:29:08 +02:00
Víctor Fernández Poyatos acfd1d99d6 fix(db-connectors): default values 2025-03-14 11:14:32 +01:00
Víctor Fernández Poyatos f6780ab0dc Merge branch 'master' into PRWLR-4758-django-must-support-read-only-replica-database-config 2025-03-13 12:50:16 +01:00
Víctor Fernández Poyatos 6ca3a8e076 ref: use same var names as before 2025-03-12 16:20:43 +01:00
Víctor Fernández Poyatos ac14f6f8d1 chore: update API changelog 2025-03-12 16:17:47 +01:00
Víctor Fernández Poyatos d9bc3fbf3c fix(tests): adapt unit tests to db connectors changes 2025-03-12 16:14:21 +01:00
Víctor Fernández Poyatos 178d398aa1 ref: refactor forced db connections to use replica when needed 2025-03-12 16:13:51 +01:00
Víctor Fernández Poyatos 36f9051167 feat(db): add read replica configuration through env vars 2025-03-12 16:11:58 +01:00
476 changed files with 68634 additions and 6351 deletions

No files matched your search

+8
View File
@@ -24,6 +24,10 @@ POSTGRES_USER=prowler
POSTGRES_PASSWORD=postgres
POSTGRES_DB=prowler_db
# Celery-Prowler task settings
TASK_RETRY_DELAY_SECONDS=0.1
TASK_RETRY_ATTEMPTS=5
# Valkey settings
# If running Valkey and celery on host, use localhost, else use 'valkey'
VALKEY_HOST=valkey
@@ -133,3 +137,7 @@ SOCIAL_GOOGLE_OAUTH_CLIENT_SECRET=""
SOCIAL_GITHUB_OAUTH_CALLBACK_URL="${AUTH_URL}/api/auth/callback/github"
SOCIAL_GITHUB_OAUTH_CLIENT_ID=""
SOCIAL_GITHUB_OAUTH_CLIENT_SECRET=""
# Single Sign-On (SSO)
SAML_PUBLIC_CERT=""
SAML_PRIVATE_KEY=""
@@ -81,7 +81,7 @@ jobs:
- name: Build and push container image (latest)
# Comment the following line for testing
if: github.event_name == 'push'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
# Set push: false for testing
@@ -94,7 +94,7 @@ jobs:
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
push: true
+2 -2
View File
@@ -48,12 +48,12 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/init@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/api-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/analyze@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
category: "/language:${{matrix.language}}"
+14 -2
View File
@@ -113,6 +113,12 @@ jobs:
python-version: ${{ matrix.python-version }}
cache: "poetry"
- name: Install system dependencies for xmlsec
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
sudo apt-get update
sudo apt-get install -y libxml2-dev libxmlsec1-dev libxmlsec1-openssl pkg-config
- name: Install dependencies
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
@@ -131,6 +137,12 @@ jobs:
run: |
poetry check --lock
- name: Prevents known compatibility error between lxml and libxml2/libxmlsec versions - https://github.com/xmlsec/python-xmlsec/issues/320
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
run: |
poetry run pip install --force-reinstall --no-binary lxml lxml
- name: Lint with ruff
working-directory: ./api
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
@@ -181,7 +193,7 @@ jobs:
- name: Upload coverage reports to Codecov
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: codecov/codecov-action@ad3126e916f78f00edff4ed0317cf185271ccc2d # v5.4.2
uses: codecov/codecov-action@18283e04ce6e62d37312384ff67231eb8fd56d24 # v5.4.3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
@@ -193,7 +205,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
- name: Build Container
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.API_WORKING_DIR }}
push: false
@@ -0,0 +1,67 @@
name: Create Backport Label
on:
release:
types: [published]
jobs:
create_label:
runs-on: ubuntu-latest
permissions:
contents: write
issues: write
steps:
- name: Create backport label
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_TAG: ${{ github.event.release.tag_name }}
OWNER_REPO: ${{ github.repository }}
run: |
VERSION_ONLY=${RELEASE_TAG#v} # Remove 'v' prefix if present (e.g., v3.2.0 -> 3.2.0)
# Check if it's a minor version (X.Y.0)
if [[ "$VERSION_ONLY" =~ ^[0-9]+\.[0-9]+\.0$ ]]; then
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is a minor version. Proceeding to create backport label."
TWO_DIGIT_VERSION=${VERSION_ONLY%.0} # Extract X.Y from X.Y.0 (e.g., 5.6 from 5.6.0)
FINAL_LABEL_NAME="backport-to-v${TWO_DIGIT_VERSION}"
FINAL_DESCRIPTION="Backport PR to the v${TWO_DIGIT_VERSION} branch"
echo "Effective label name will be: ${FINAL_LABEL_NAME}"
echo "Effective description will be: ${FINAL_DESCRIPTION}"
# Check if the label already exists
STATUS_CODE=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: token ${GITHUB_TOKEN}" "https://api.github.com/repos/${OWNER_REPO}/labels/${FINAL_LABEL_NAME}")
if [ "${STATUS_CODE}" -eq 200 ]; then
echo "Label '${FINAL_LABEL_NAME}' already exists."
elif [ "${STATUS_CODE}" -eq 404 ]; then
echo "Label '${FINAL_LABEL_NAME}' does not exist. Creating it..."
# Prepare JSON data payload
JSON_DATA=$(printf '{"name":"%s","description":"%s","color":"B60205"}' "${FINAL_LABEL_NAME}" "${FINAL_DESCRIPTION}")
CREATE_STATUS_CODE=$(curl -s -o /tmp/curl_create_response.json -w "%{http_code}" -X POST \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: token ${GITHUB_TOKEN}" \
--data "${JSON_DATA}" \
"https://api.github.com/repos/${OWNER_REPO}/labels")
CREATE_RESPONSE_BODY=$(cat /tmp/curl_create_response.json)
rm -f /tmp/curl_create_response.json
if [ "$CREATE_STATUS_CODE" -eq 201 ]; then
echo "Label '${FINAL_LABEL_NAME}' created successfully."
else
echo "Error creating label '${FINAL_LABEL_NAME}'. Status: $CREATE_STATUS_CODE"
echo "Response: $CREATE_RESPONSE_BODY"
exit 1
fi
else
echo "Error checking for label '${FINAL_LABEL_NAME}'. HTTP Status: ${STATUS_CODE}"
exit 1
fi
else
echo "Release ${RELEASE_TAG} (version ${VERSION_ONLY}) is not a minor version. Skipping backport label creation."
exit 0
fi
+1 -1
View File
@@ -11,7 +11,7 @@ jobs:
with:
fetch-depth: 0
- name: TruffleHog OSS
uses: trufflesecurity/trufflehog@b06f6d72a3791308bb7ba59c2b8cb7a083bd17e4 # v3.88.26
uses: trufflesecurity/trufflehog@90694bf9af66e7536abc5824e7a87246dbf933cb # v3.88.35
with:
path: ./
base: ${{ github.event.repository.default_branch }}
+2 -2
View File
@@ -18,7 +18,7 @@ jobs:
- name: Set short git commit SHA
id: vars
run: |
shortSha=$(git rev-parse --short ${{ github.sha }})
shortSha=$(git rev-parse --short ${{ github.event.pull_request.merge_commit_sha }})
echo "SHORT_SHA=${shortSha}" >> $GITHUB_ENV
- name: Trigger pull request
@@ -28,7 +28,7 @@ jobs:
repository: ${{ secrets.CLOUD_DISPATCH }}
event-type: prowler-pull-request-merged
client-payload: '{
"PROWLER_COMMIT_SHA": "${{ github.sha }}",
"PROWLER_COMMIT_SHA": "${{ github.event.pull_request.merge_commit_sha }}",
"PROWLER_COMMIT_SHORT_SHA": "${{ env.SHORT_SHA }}",
"PROWLER_PR_TITLE": "${{ github.event.pull_request.title }}",
"PROWLER_PR_LABELS": ${{ toJson(github.event.pull_request.labels.*.name) }},
@@ -127,7 +127,7 @@ jobs:
- name: Build and push container image (latest)
if: github.event_name == 'push'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
push: true
tags: |
@@ -140,7 +140,7 @@ jobs:
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
# Use local context to get changes
# https://github.com/docker/build-push-action#path-context
+2 -2
View File
@@ -56,12 +56,12 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/init@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/sdk-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/analyze@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
category: "/language:${{matrix.language}}"
+1 -1
View File
@@ -226,7 +226,7 @@ jobs:
# Codecov
- name: Upload coverage reports to Codecov
if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
uses: codecov/codecov-action@ad3126e916f78f00edff4ed0317cf185271ccc2d # v5.4.2
uses: codecov/codecov-action@18283e04ce6e62d37312384ff67231eb8fd56d24 # v5.4.3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
with:
@@ -38,7 +38,7 @@ jobs:
pip install boto3
- name: Configure AWS Credentials -- DEV
uses: aws-actions/configure-aws-credentials@ececac1a45f3b08a01d2dd070d28d111c5fe6722 # v4.1.0
uses: aws-actions/configure-aws-credentials@b47578312673ae6fa5b5096b330d9fbac3d116df # v4.2.1
with:
aws-region: ${{ env.AWS_REGION_DEV }}
role-to-assume: ${{ secrets.DEV_IAM_ROLE_ARN }}
@@ -81,7 +81,7 @@ jobs:
- name: Build and push container image (latest)
# Comment the following line for testing
if: github.event_name == 'push'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
@@ -96,7 +96,7 @@ jobs:
- name: Build and push container image (release)
if: github.event_name == 'release'
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.WORKING_DIRECTORY }}
build-args: |
+2 -2
View File
@@ -48,12 +48,12 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/init@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
languages: ${{ matrix.language }}
config-file: ./.github/codeql/ui-codeql-config.yml
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@28deaeda66b76a05916b6923827895f2b14ab387 # v3.28.16
uses: github/codeql-action/analyze@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
with:
category: "/language:${{matrix.language}}"
+1 -1
View File
@@ -50,7 +50,7 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
- name: Build Container
uses: docker/build-push-action@14487ce63c7a62a4a324b0bfb37086795e31c6c1 # v6.16.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ${{ env.UI_WORKING_DIR }}
# Always build using `prod` target
+1
View File
@@ -44,6 +44,7 @@ junit-reports/
# Cursor files
.cursorignore
.cursor/
# Terraform
.terraform*
+14 -8
View File
@@ -3,7 +3,7 @@
<img align="center" src="https://github.com/prowler-cloud/prowler/blob/master/docs/img/prowler-logo-white.png#gh-dark-mode-only" width="50%" height="50%">
</p>
<p align="center">
<b><i>Prowler Open Source</b> is as dynamic and adaptable as the environment it secures. It is trusted by the industry leaders to uphold the highest standards in security.
<b><i>Prowler</b> is the Open Cloud Security platform trusted by thousands to automate security and compliance in any cloud environment. With hundreds of ready-to-use checks and compliance frameworks, Prowler delivers real-time, customizable monitoring and seamless integrations, making cloud security simple, scalable, and cost-effective for organizations of any size.
</p>
<p align="center">
<b>Learn more at <a href="https://prowler.com">prowler.com</i></b>
@@ -86,21 +86,27 @@ prowler dashboard
| Provider | Checks | Services | [Compliance Frameworks](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/compliance/) | [Categories](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/misc/#categories) |
|---|---|---|---|---|
| AWS | 564 | 82 | 33 | 10 |
| GCP | 79 | 13 | 7 | 3 |
| Azure | 140 | 18 | 8 | 3 |
| Kubernetes | 83 | 7 | 4 | 7 |
| GitHub | 2 | 1 | 1 | 0 |
| M365 | 44 | 2 | 2 | 0 |
| AWS | 567 | 82 | 36 | 10 |
| GCP | 79 | 13 | 10 | 3 |
| Azure | 142 | 18 | 10 | 3 |
| Kubernetes | 83 | 7 | 5 | 7 |
| GitHub | 16 | 2 | 1 | 0 |
| M365 | 69 | 7 | 2 | 2 |
| NHN (Unofficial) | 6 | 2 | 1 | 0 |
> [!Note]
> The numbers in the table are updated periodically.
> [!Tip]
> For the most accurate and up-to-date information about checks, services, frameworks, and categories, visit [**Prowler Hub**](https://hub.prowler.com).
> [!Note]
> Use the following commands to list Prowler's available checks, services, compliance frameworks, and categories: `prowler <provider> --list-checks`, `prowler <provider> --list-services`, `prowler <provider> --list-compliance` and `prowler <provider> --list-categories`.
# 💻 Installation
## Prowler App
Installing Prowler App
Prowler App offers flexible installation methods tailored to various environments:
> For detailed instructions on using Prowler App, refer to the [Prowler App Usage Guide](https://docs.prowler.com/projects/prowler-open-source/en/latest/tutorials/prowler-app/).
+3
View File
@@ -29,6 +29,9 @@ DJANGO_SENTRY_DSN=
# If running django and celery on host, use 'localhost', else use 'postgres-db'
POSTGRES_HOST=[localhost|postgres-db]
POSTGRES_PORT=5432
# If you are running a replica only for read queries. Defaults to the same value as POSTGRES_HOST and POSTGRES_PORT
POSTGRES_HOST_READ_ONLY=[localhost|postgres-db]
POSTGRES_PORT_READ_ONLY=5432
POSTGRES_ADMIN_USER=prowler
POSTGRES_ADMIN_PASSWORD=S3cret
POSTGRES_USER=prowler_user
+55 -3
View File
@@ -2,12 +2,66 @@
All notable changes to the **Prowler API** are documented in this file.
## [v1.8.0] (Prowler UNRELEASED)
## [v1.9.0] (Prowler UNRELEASED)
### Added
- Added SSO with SAML support [(#7822)](https://github.com/prowler-cloud/prowler/pull/7822).
- Support GCP Service Account key. [(#7824)](https://github.com/prowler-cloud/prowler/pull/7824)
- Added new `GET /compliance-overviews` endpoints to retrieve compliance metadata and specific requirements statuses [(#7877)](https://github.com/prowler-cloud/prowler/pull/7877).
- Support for read only replicas in the database [(#7210)](https://github.com/prowler-cloud/prowler/pull/7210).
### Changed
- Reworked `GET /compliance-overviews` to return proper requirement metrics [(#7877)](https://github.com/prowler-cloud/prowler/pull/7877).
---
## [v1.8.4] (Prowler v5.7.4)
### Removed
- Reverted RLS transaction handling and DB custom backend [(#7994)](https://github.com/prowler-cloud/prowler/pull/7994).
---
## [v1.8.3] (Prowler v5.7.3)
### Added
- Database backend to handle already closed connections [(#7935)](https://github.com/prowler-cloud/prowler/pull/7935).
### Changed
- Renamed field encrypted_password to password for M365 provider [(#7784)](https://github.com/prowler-cloud/prowler/pull/7784)
### Fixed
- Fixed transaction persistence with RLS operations [(#7916)](https://github.com/prowler-cloud/prowler/pull/7916).
- Reverted the change `get_with_retry` to use the original `get` method for retrieving tasks [(#7932)](https://github.com/prowler-cloud/prowler/pull/7932).
---
## [v1.8.2] (Prowler v5.7.2)
### Fixed
- Fixed task lookup to use task_kwargs instead of task_args for scan report resolution. [(#7830)](https://github.com/prowler-cloud/prowler/pull/7830)
- Fixed Kubernetes UID validation to allow valid context names [(#7871)](https://github.com/prowler-cloud/prowler/pull/7871)
- Fixed the connection status verification before launching a scan [(#7831)](https://github.com/prowler-cloud/prowler/pull/7831)
- Fixed a race condition when creating background tasks [(#7876)](https://github.com/prowler-cloud/prowler/pull/7876).
- Fixed an error when modifying or retrieving tenants due to missing user UUID in transaction context [(#7890)](https://github.com/prowler-cloud/prowler/pull/7890).
---
## [v1.8.1] (Prowler v5.7.1)
### Fixed
- Added database index to improve performance on finding lookup [(#7800)](https://github.com/prowler-cloud/prowler/pull/7800).
---
## [v1.8.0] (Prowler v5.7.0)
### Added
- Added huge improvements to `/findings/metadata` and resource related filters for findings [(#7690)](https://github.com/prowler-cloud/prowler/pull/7690).
- Added improvements to `/overviews` endpoints [(#7690)](https://github.com/prowler-cloud/prowler/pull/7690).
- Added new queue to perform backfill background tasks [(#7690)](https://github.com/prowler-cloud/prowler/pull/7690).
- Added new endpoints to retrieve latest findings and metadata [(#7743)](https://github.com/prowler-cloud/prowler/pull/7743).
- Added export support for Prowler ThreatScore in M365 [(7783)](https://github.com/prowler-cloud/prowler/pull/7783)
---
@@ -24,7 +78,6 @@ All notable changes to the **Prowler API** are documented in this file.
## [v1.6.0] (Prowler v5.5.0)
### Added
- Support for developing new integrations [(#7167)](https://github.com/prowler-cloud/prowler/pull/7167).
- HTTP Security Headers [(#7289)](https://github.com/prowler-cloud/prowler/pull/7289).
- New endpoint to get the compliance overviews metadata [(#7333)](https://github.com/prowler-cloud/prowler/pull/7333).
@@ -62,7 +115,6 @@ All notable changes to the **Prowler API** are documented in this file.
- Fixed a race condition when deleting export files after the S3 upload [(#7172)](https://github.com/prowler-cloud/prowler/pull/7172).
- Handled exception when a provider has no secret in test connection [(#7283)](https://github.com/prowler-cloud/prowler/pull/7283).
---
## [v1.5.0] (Prowler v5.4.0)
+20 -5
View File
@@ -6,7 +6,19 @@ ARG POWERSHELL_VERSION=7.5.0
ENV POWERSHELL_VERSION=${POWERSHELL_VERSION}
# hadolint ignore=DL3008
RUN apt-get update && apt-get install -y --no-install-recommends wget libicu72 \
RUN apt-get update && apt-get install -y --no-install-recommends \
wget \
libicu72 \
gcc \
g++ \
make \
libxml2-dev \
libxmlsec1-dev \
libxmlsec1-openssl \
pkg-config \
libtool \
libxslt1-dev \
python3-dev \
&& rm -rf /var/lib/apt/lists/*
# Install PowerShell
@@ -37,18 +49,21 @@ COPY pyproject.toml ./
RUN pip install --no-cache-dir --upgrade pip && \
pip install --no-cache-dir poetry
COPY src/backend/ ./backend/
ENV PATH="/home/prowler/.local/bin:$PATH"
# Add `--no-root` to avoid installing the current project as a package
RUN poetry install --no-root && \
rm -rf ~/.cache/pip
COPY docker-entrypoint.sh ./docker-entrypoint.sh
RUN poetry run python "$(poetry env info --path)/src/prowler/prowler/providers/m365/lib/powershell/m365_powershell.py"
# Prevents known compatibility error between lxml and libxml2/libxmlsec versions.
# See: https://github.com/xmlsec/python-xmlsec/issues/320
RUN poetry run pip install --force-reinstall --no-binary lxml lxml
COPY src/backend/ ./backend/
COPY docker-entrypoint.sh ./docker-entrypoint.sh
WORKDIR /home/prowler/backend
# Development image
-125
View File
@@ -1,125 +0,0 @@
services:
api:
build:
dockerfile: Dockerfile
image: prowler-api
env_file:
- path: ./.env
required: false
ports:
- "${DJANGO_PORT:-8000}:${DJANGO_PORT:-8000}"
profiles:
- prod
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
entrypoint:
- "../docker-entrypoint.sh"
- "prod"
api-dev:
build:
dockerfile: Dockerfile
target: dev
image: prowler-api-dev
environment:
- DJANGO_SETTINGS_MODULE=config.django.devel
- DJANGO_LOGGING_FORMATTER=human_readable
env_file:
- path: ./.env
required: false
ports:
- "${DJANGO_PORT:-8080}:${DJANGO_PORT:-8080}"
volumes:
- "./src/backend:/home/prowler/backend"
- "./pyproject.toml:/home/prowler/pyproject.toml"
profiles:
- dev
depends_on:
postgres:
condition: service_healthy
valkey:
condition: service_healthy
entrypoint:
- "../docker-entrypoint.sh"
- "dev"
postgres:
image: postgres:16.3-alpine
ports:
- "${POSTGRES_PORT:-5432}:${POSTGRES_PORT:-5432}"
hostname: "postgres-db"
volumes:
- ./_data/postgres:/var/lib/postgresql/data
environment:
- POSTGRES_USER=${POSTGRES_ADMIN_USER:-prowler}
- POSTGRES_PASSWORD=${POSTGRES_ADMIN_PASSWORD:-S3cret}
- POSTGRES_DB=${POSTGRES_DB:-prowler_db}
env_file:
- path: ./.env
required: false
healthcheck:
test: ["CMD-SHELL", "sh -c 'pg_isready -U ${POSTGRES_ADMIN_USER:-prowler} -d ${POSTGRES_DB:-prowler_db}'"]
interval: 5s
timeout: 5s
retries: 5
valkey:
image: valkey/valkey:7-alpine3.19
ports:
- "${VALKEY_PORT:-6379}:6379"
hostname: "valkey"
volumes:
- ./_data/valkey:/data
env_file:
- path: ./.env
required: false
healthcheck:
test: ["CMD-SHELL", "sh -c 'valkey-cli ping'"]
interval: 10s
timeout: 5s
retries: 3
worker:
build:
dockerfile: Dockerfile
image: prowler-worker
environment:
- DJANGO_SETTINGS_MODULE=${DJANGO_SETTINGS_MODULE:-config.django.production}
env_file:
- path: ./.env
required: false
profiles:
- dev
- prod
depends_on:
valkey:
condition: service_healthy
postgres:
condition: service_healthy
entrypoint:
- "../docker-entrypoint.sh"
- "worker"
worker-beat:
build:
dockerfile: Dockerfile
image: prowler-worker
environment:
- DJANGO_SETTINGS_MODULE=${DJANGO_SETTINGS_MODULE:-config.django.production}
env_file:
- path: ./.env
required: false
profiles:
- dev
- prod
depends_on:
valkey:
condition: service_healthy
postgres:
condition: service_healthy
entrypoint:
- "../docker-entrypoint.sh"
- "beat"
+4
View File
@@ -3,6 +3,10 @@
apply_migrations() {
echo "Applying database migrations..."
# Fix Inconsistent migration history after adding sites app
poetry run python manage.py check_and_fix_socialaccount_sites_migration --database admin
poetry run python manage.py migrate --database admin
}
+213 -6
View File
@@ -1469,14 +1469,14 @@ with-social = ["django-allauth[socialaccount] (>=64.0.0)"]
[[package]]
name = "django"
version = "5.1.8"
version = "5.1.10"
description = "A high-level Python web framework that encourages rapid development and clean, pragmatic design."
optional = false
python-versions = ">=3.10"
groups = ["main", "dev"]
files = [
{file = "Django-5.1.8-py3-none-any.whl", hash = "sha256:11b28fa4b00e59d0def004e9ee012fefbb1065a5beb39ee838983fd24493ad4f"},
{file = "Django-5.1.8.tar.gz", hash = "sha256:42e92a1dd2810072bcc40a39a212b693f94406d0ba0749e68eb642f31dc770b4"},
{file = "django-5.1.10-py3-none-any.whl", hash = "sha256:19c9b771e9cf4de91101861aadd2daaa159bcf10698ca909c5755c88e70ccb84"},
{file = "django-5.1.10.tar.gz", hash = "sha256:73e5d191421d177803dbd5495d94bc7d06d156df9561f4eea9e11b4994c07137"},
]
[package.dependencies]
@@ -1490,19 +1490,20 @@ bcrypt = ["bcrypt"]
[[package]]
name = "django-allauth"
version = "65.4.1"
version = "65.8.0"
description = "Integrated set of Django applications addressing authentication, registration, account management as well as 3rd party (social) account authentication."
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "django_allauth-65.4.1.tar.gz", hash = "sha256:60b32aef7dbbcc213319aa4fd8f570e985266ea1162ae6ef7a26a24efca85c8c"},
{file = "django_allauth-65.8.0.tar.gz", hash = "sha256:9da589d99d412740629333a01865a90c95c97e0fae0cde789aa45a8fda90e83b"},
]
[package.dependencies]
asgiref = ">=3.8.1"
Django = ">=4.2.16"
pyjwt = {version = ">=1.7", extras = ["crypto"], optional = true, markers = "extra == \"socialaccount\""}
python3-saml = {version = ">=1.15.0,<2.0.0", optional = true, markers = "extra == \"saml\""}
requests = {version = ">=2.0.0", optional = true, markers = "extra == \"socialaccount\""}
requests-oauthlib = {version = ">=0.3.0", optional = true, markers = "extra == \"socialaccount\""}
@@ -2582,6 +2583,155 @@ websocket-client = ">=0.32.0,<0.40.0 || >0.40.0,<0.41.dev0 || >=0.43.dev0"
[package.extras]
adal = ["adal (>=1.0.2)"]
[[package]]
name = "lxml"
version = "5.4.0"
description = "Powerful and Pythonic XML processing library combining libxml2/libxslt with the ElementTree API."
optional = false
python-versions = ">=3.6"
groups = ["main"]
files = [
{file = "lxml-5.4.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:e7bc6df34d42322c5289e37e9971d6ed114e3776b45fa879f734bded9d1fea9c"},
{file = "lxml-5.4.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:6854f8bd8a1536f8a1d9a3655e6354faa6406621cf857dc27b681b69860645c7"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:696ea9e87442467819ac22394ca36cb3d01848dad1be6fac3fb612d3bd5a12cf"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6ef80aeac414f33c24b3815ecd560cee272786c3adfa5f31316d8b349bfade28"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3b9c2754cef6963f3408ab381ea55f47dabc6f78f4b8ebb0f0b25cf1ac1f7609"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7a62cc23d754bb449d63ff35334acc9f5c02e6dae830d78dab4dd12b78a524f4"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8f82125bc7203c5ae8633a7d5d20bcfdff0ba33e436e4ab0abc026a53a8960b7"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:b67319b4aef1a6c56576ff544b67a2a6fbd7eaee485b241cabf53115e8908b8f"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_28_ppc64le.whl", hash = "sha256:a8ef956fce64c8551221f395ba21d0724fed6b9b6242ca4f2f7beb4ce2f41997"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_28_s390x.whl", hash = "sha256:0a01ce7d8479dce84fc03324e3b0c9c90b1ece9a9bb6a1b6c9025e7e4520e78c"},
{file = "lxml-5.4.0-cp310-cp310-manylinux_2_28_x86_64.whl", hash = "sha256:91505d3ddebf268bb1588eb0f63821f738d20e1e7f05d3c647a5ca900288760b"},
{file = "lxml-5.4.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:a3bcdde35d82ff385f4ede021df801b5c4a5bcdfb61ea87caabcebfc4945dc1b"},
{file = "lxml-5.4.0-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:aea7c06667b987787c7d1f5e1dfcd70419b711cdb47d6b4bb4ad4b76777a0563"},
{file = "lxml-5.4.0-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:a7fb111eef4d05909b82152721a59c1b14d0f365e2be4c742a473c5d7372f4f5"},
{file = "lxml-5.4.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:43d549b876ce64aa18b2328faff70f5877f8c6dede415f80a2f799d31644d776"},
{file = "lxml-5.4.0-cp310-cp310-win32.whl", hash = "sha256:75133890e40d229d6c5837b0312abbe5bac1c342452cf0e12523477cd3aa21e7"},
{file = "lxml-5.4.0-cp310-cp310-win_amd64.whl", hash = "sha256:de5b4e1088523e2b6f730d0509a9a813355b7f5659d70eb4f319c76beea2e250"},
{file = "lxml-5.4.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:98a3912194c079ef37e716ed228ae0dcb960992100461b704aea4e93af6b0bb9"},
{file = "lxml-5.4.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0ea0252b51d296a75f6118ed0d8696888e7403408ad42345d7dfd0d1e93309a7"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b92b69441d1bd39f4940f9eadfa417a25862242ca2c396b406f9272ef09cdcaa"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:20e16c08254b9b6466526bc1828d9370ee6c0d60a4b64836bc3ac2917d1e16df"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7605c1c32c3d6e8c990dd28a0970a3cbbf1429d5b92279e37fda05fb0c92190e"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:ecf4c4b83f1ab3d5a7ace10bafcb6f11df6156857a3c418244cef41ca9fa3e44"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0cef4feae82709eed352cd7e97ae062ef6ae9c7b5dbe3663f104cd2c0e8d94ba"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_28_aarch64.whl", hash = "sha256:df53330a3bff250f10472ce96a9af28628ff1f4efc51ccba351a8820bca2a8ba"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_28_ppc64le.whl", hash = "sha256:aefe1a7cb852fa61150fcb21a8c8fcea7b58c4cb11fbe59c97a0a4b31cae3c8c"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_28_s390x.whl", hash = "sha256:ef5a7178fcc73b7d8c07229e89f8eb45b2908a9238eb90dcfc46571ccf0383b8"},
{file = "lxml-5.4.0-cp311-cp311-manylinux_2_28_x86_64.whl", hash = "sha256:d2ed1b3cb9ff1c10e6e8b00941bb2e5bb568b307bfc6b17dffbbe8be5eecba86"},
{file = "lxml-5.4.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:72ac9762a9f8ce74c9eed4a4e74306f2f18613a6b71fa065495a67ac227b3056"},
{file = "lxml-5.4.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:f5cb182f6396706dc6cc1896dd02b1c889d644c081b0cdec38747573db88a7d7"},
{file = "lxml-5.4.0-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:3a3178b4873df8ef9457a4875703488eb1622632a9cee6d76464b60e90adbfcd"},
{file = "lxml-5.4.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:e094ec83694b59d263802ed03a8384594fcce477ce484b0cbcd0008a211ca751"},
{file = "lxml-5.4.0-cp311-cp311-win32.whl", hash = "sha256:4329422de653cdb2b72afa39b0aa04252fca9071550044904b2e7036d9d97fe4"},
{file = "lxml-5.4.0-cp311-cp311-win_amd64.whl", hash = "sha256:fd3be6481ef54b8cfd0e1e953323b7aa9d9789b94842d0e5b142ef4bb7999539"},
{file = "lxml-5.4.0-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:b5aff6f3e818e6bdbbb38e5967520f174b18f539c2b9de867b1e7fde6f8d95a4"},
{file = "lxml-5.4.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:942a5d73f739ad7c452bf739a62a0f83e2578afd6b8e5406308731f4ce78b16d"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:460508a4b07364d6abf53acaa0a90b6d370fafde5693ef37602566613a9b0779"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:529024ab3a505fed78fe3cc5ddc079464e709f6c892733e3f5842007cec8ac6e"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7ca56ebc2c474e8f3d5761debfd9283b8b18c76c4fc0967b74aeafba1f5647f9"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a81e1196f0a5b4167a8dafe3a66aa67c4addac1b22dc47947abd5d5c7a3f24b5"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:00b8686694423ddae324cf614e1b9659c2edb754de617703c3d29ff568448df5"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_28_aarch64.whl", hash = "sha256:c5681160758d3f6ac5b4fea370495c48aac0989d6a0f01bb9a72ad8ef5ab75c4"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_28_ppc64le.whl", hash = "sha256:2dc191e60425ad70e75a68c9fd90ab284df64d9cd410ba8d2b641c0c45bc006e"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_28_s390x.whl", hash = "sha256:67f779374c6b9753ae0a0195a892a1c234ce8416e4448fe1e9f34746482070a7"},
{file = "lxml-5.4.0-cp312-cp312-manylinux_2_28_x86_64.whl", hash = "sha256:79d5bfa9c1b455336f52343130b2067164040604e41f6dc4d8313867ed540079"},
{file = "lxml-5.4.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3d3c30ba1c9b48c68489dc1829a6eede9873f52edca1dda900066542528d6b20"},
{file = "lxml-5.4.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1af80c6316ae68aded77e91cd9d80648f7dd40406cef73df841aa3c36f6907c8"},
{file = "lxml-5.4.0-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:4d885698f5019abe0de3d352caf9466d5de2baded00a06ef3f1216c1a58ae78f"},
{file = "lxml-5.4.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:aea53d51859b6c64e7c51d522c03cc2c48b9b5d6172126854cc7f01aa11f52bc"},
{file = "lxml-5.4.0-cp312-cp312-win32.whl", hash = "sha256:d90b729fd2732df28130c064aac9bb8aff14ba20baa4aee7bd0795ff1187545f"},
{file = "lxml-5.4.0-cp312-cp312-win_amd64.whl", hash = "sha256:1dc4ca99e89c335a7ed47d38964abcb36c5910790f9bd106f2a8fa2ee0b909d2"},
{file = "lxml-5.4.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:773e27b62920199c6197130632c18fb7ead3257fce1ffb7d286912e56ddb79e0"},
{file = "lxml-5.4.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ce9c671845de9699904b1e9df95acfe8dfc183f2310f163cdaa91a3535af95de"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:9454b8d8200ec99a224df8854786262b1bd6461f4280064c807303c642c05e76"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cccd007d5c95279e529c146d095f1d39ac05139de26c098166c4beb9374b0f4d"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:0fce1294a0497edb034cb416ad3e77ecc89b313cff7adbee5334e4dc0d11f422"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:24974f774f3a78ac12b95e3a20ef0931795ff04dbb16db81a90c37f589819551"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:497cab4d8254c2a90bf988f162ace2ddbfdd806fce3bda3f581b9d24c852e03c"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_28_aarch64.whl", hash = "sha256:e794f698ae4c5084414efea0f5cc9f4ac562ec02d66e1484ff822ef97c2cadff"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_28_ppc64le.whl", hash = "sha256:2c62891b1ea3094bb12097822b3d44b93fc6c325f2043c4d2736a8ff09e65f60"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_28_s390x.whl", hash = "sha256:142accb3e4d1edae4b392bd165a9abdee8a3c432a2cca193df995bc3886249c8"},
{file = "lxml-5.4.0-cp313-cp313-manylinux_2_28_x86_64.whl", hash = "sha256:1a42b3a19346e5601d1b8296ff6ef3d76038058f311902edd574461e9c036982"},
{file = "lxml-5.4.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4291d3c409a17febf817259cb37bc62cb7eb398bcc95c1356947e2871911ae61"},
{file = "lxml-5.4.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:4f5322cf38fe0e21c2d73901abf68e6329dc02a4994e483adbcf92b568a09a54"},
{file = "lxml-5.4.0-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:0be91891bdb06ebe65122aa6bf3fc94489960cf7e03033c6f83a90863b23c58b"},
{file = "lxml-5.4.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:15a665ad90054a3d4f397bc40f73948d48e36e4c09f9bcffc7d90c87410e478a"},
{file = "lxml-5.4.0-cp313-cp313-win32.whl", hash = "sha256:d5663bc1b471c79f5c833cffbc9b87d7bf13f87e055a5c86c363ccd2348d7e82"},
{file = "lxml-5.4.0-cp313-cp313-win_amd64.whl", hash = "sha256:bcb7a1096b4b6b24ce1ac24d4942ad98f983cd3810f9711bcd0293f43a9d8b9f"},
{file = "lxml-5.4.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:7be701c24e7f843e6788353c055d806e8bd8466b52907bafe5d13ec6a6dbaecd"},
{file = "lxml-5.4.0-cp36-cp36m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fb54f7c6bafaa808f27166569b1511fc42701a7713858dddc08afdde9746849e"},
{file = "lxml-5.4.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97dac543661e84a284502e0cf8a67b5c711b0ad5fb661d1bd505c02f8cf716d7"},
{file = "lxml-5.4.0-cp36-cp36m-manylinux_2_28_x86_64.whl", hash = "sha256:c70e93fba207106cb16bf852e421c37bbded92acd5964390aad07cb50d60f5cf"},
{file = "lxml-5.4.0-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:9c886b481aefdf818ad44846145f6eaf373a20d200b5ce1a5c8e1bc2d8745410"},
{file = "lxml-5.4.0-cp36-cp36m-musllinux_1_2_x86_64.whl", hash = "sha256:fa0e294046de09acd6146be0ed6727d1f42ded4ce3ea1e9a19c11b6774eea27c"},
{file = "lxml-5.4.0-cp36-cp36m-win32.whl", hash = "sha256:61c7bbf432f09ee44b1ccaa24896d21075e533cd01477966a5ff5a71d88b2f56"},
{file = "lxml-5.4.0-cp36-cp36m-win_amd64.whl", hash = "sha256:7ce1a171ec325192c6a636b64c94418e71a1964f56d002cc28122fceff0b6121"},
{file = "lxml-5.4.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:795f61bcaf8770e1b37eec24edf9771b307df3af74d1d6f27d812e15a9ff3872"},
{file = "lxml-5.4.0-cp37-cp37m-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:29f451a4b614a7b5b6c2e043d7b64a15bd8304d7e767055e8ab68387a8cacf4e"},
{file = "lxml-5.4.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:891f7f991a68d20c75cb13c5c9142b2a3f9eb161f1f12a9489c82172d1f133c0"},
{file = "lxml-5.4.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4aa412a82e460571fad592d0f93ce9935a20090029ba08eca05c614f99b0cc92"},
{file = "lxml-5.4.0-cp37-cp37m-manylinux_2_28_aarch64.whl", hash = "sha256:ac7ba71f9561cd7d7b55e1ea5511543c0282e2b6450f122672a2694621d63b7e"},
{file = "lxml-5.4.0-cp37-cp37m-manylinux_2_28_x86_64.whl", hash = "sha256:c5d32f5284012deaccd37da1e2cd42f081feaa76981f0eaa474351b68df813c5"},
{file = "lxml-5.4.0-cp37-cp37m-musllinux_1_2_aarch64.whl", hash = "sha256:ce31158630a6ac85bddd6b830cffd46085ff90498b397bd0a259f59d27a12188"},
{file = "lxml-5.4.0-cp37-cp37m-musllinux_1_2_x86_64.whl", hash = "sha256:31e63621e073e04697c1b2d23fcb89991790eef370ec37ce4d5d469f40924ed6"},
{file = "lxml-5.4.0-cp37-cp37m-win32.whl", hash = "sha256:be2ba4c3c5b7900246a8f866580700ef0d538f2ca32535e991027bdaba944063"},
{file = "lxml-5.4.0-cp37-cp37m-win_amd64.whl", hash = "sha256:09846782b1ef650b321484ad429217f5154da4d6e786636c38e434fa32e94e49"},
{file = "lxml-5.4.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:eaf24066ad0b30917186420d51e2e3edf4b0e2ea68d8cd885b14dc8afdcf6556"},
{file = "lxml-5.4.0-cp38-cp38-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2b31a3a77501d86d8ade128abb01082724c0dfd9524f542f2f07d693c9f1175f"},
{file = "lxml-5.4.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0e108352e203c7afd0eb91d782582f00a0b16a948d204d4dec8565024fafeea5"},
{file = "lxml-5.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a11a96c3b3f7551c8a8109aa65e8594e551d5a84c76bf950da33d0fb6dfafab7"},
{file = "lxml-5.4.0-cp38-cp38-manylinux_2_28_aarch64.whl", hash = "sha256:ca755eebf0d9e62d6cb013f1261e510317a41bf4650f22963474a663fdfe02aa"},
{file = "lxml-5.4.0-cp38-cp38-manylinux_2_28_x86_64.whl", hash = "sha256:4cd915c0fb1bed47b5e6d6edd424ac25856252f09120e3e8ba5154b6b921860e"},
{file = "lxml-5.4.0-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:226046e386556a45ebc787871d6d2467b32c37ce76c2680f5c608e25823ffc84"},
{file = "lxml-5.4.0-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:b108134b9667bcd71236c5a02aad5ddd073e372fb5d48ea74853e009fe38acb6"},
{file = "lxml-5.4.0-cp38-cp38-win32.whl", hash = "sha256:1320091caa89805df7dcb9e908add28166113dcd062590668514dbd510798c88"},
{file = "lxml-5.4.0-cp38-cp38-win_amd64.whl", hash = "sha256:073eb6dcdf1f587d9b88c8c93528b57eccda40209cf9be549d469b942b41d70b"},
{file = "lxml-5.4.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:bda3ea44c39eb74e2488297bb39d47186ed01342f0022c8ff407c250ac3f498e"},
{file = "lxml-5.4.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9ceaf423b50ecfc23ca00b7f50b64baba85fb3fb91c53e2c9d00bc86150c7e40"},
{file = "lxml-5.4.0-cp39-cp39-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:664cdc733bc87449fe781dbb1f309090966c11cc0c0cd7b84af956a02a8a4729"},
{file = "lxml-5.4.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:67ed8a40665b84d161bae3181aa2763beea3747f748bca5874b4af4d75998f87"},
{file = "lxml-5.4.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9b4a3bd174cc9cdaa1afbc4620c049038b441d6ba07629d89a83b408e54c35cd"},
{file = "lxml-5.4.0-cp39-cp39-manylinux_2_28_aarch64.whl", hash = "sha256:b0989737a3ba6cf2a16efb857fb0dfa20bc5c542737fddb6d893fde48be45433"},
{file = "lxml-5.4.0-cp39-cp39-manylinux_2_28_x86_64.whl", hash = "sha256:dc0af80267edc68adf85f2a5d9be1cdf062f973db6790c1d065e45025fa26140"},
{file = "lxml-5.4.0-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:639978bccb04c42677db43c79bdaa23785dc7f9b83bfd87570da8207872f1ce5"},
{file = "lxml-5.4.0-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:5a99d86351f9c15e4a901fc56404b485b1462039db59288b203f8c629260a142"},
{file = "lxml-5.4.0-cp39-cp39-win32.whl", hash = "sha256:3e6d5557989cdc3ebb5302bbdc42b439733a841891762ded9514e74f60319ad6"},
{file = "lxml-5.4.0-cp39-cp39-win_amd64.whl", hash = "sha256:a8c9b7f16b63e65bbba889acb436a1034a82d34fa09752d754f88d708eca80e1"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:1b717b00a71b901b4667226bba282dd462c42ccf618ade12f9ba3674e1fabc55"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:27a9ded0f0b52098ff89dd4c418325b987feed2ea5cc86e8860b0f844285d740"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4b7ce10634113651d6f383aa712a194179dcd496bd8c41e191cec2099fa09de5"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:53370c26500d22b45182f98847243efb518d268374a9570409d2e2276232fd37"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:c6364038c519dffdbe07e3cf42e6a7f8b90c275d4d1617a69bb59734c1a2d571"},
{file = "lxml-5.4.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:b12cb6527599808ada9eb2cd6e0e7d3d8f13fe7bbb01c6311255a15ded4c7ab4"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:5f11a1526ebd0dee85e7b1e39e39a0cc0d9d03fb527f56d8457f6df48a10dc0c"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:48b4afaf38bf79109bb060d9016fad014a9a48fb244e11b94f74ae366a64d252"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:de6f6bb8a7840c7bf216fb83eec4e2f79f7325eca8858167b68708b929ab2172"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:5cca36a194a4eb4e2ed6be36923d3cffd03dcdf477515dea687185506583d4c9"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:b7c86884ad23d61b025989d99bfdd92a7351de956e01c61307cb87035960bcb1"},
{file = "lxml-5.4.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:53d9469ab5460402c19553b56c3648746774ecd0681b1b27ea74d5d8a3ef5590"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:56dbdbab0551532bb26c19c914848d7251d73edb507c3079d6805fa8bba5b706"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:14479c2ad1cb08b62bb941ba8e0e05938524ee3c3114644df905d2331c76cd57"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:32697d2ea994e0db19c1df9e40275ffe84973e4232b5c274f47e7c1ec9763cdd"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:24f6df5f24fc3385f622c0c9d63fe34604893bc1a5bdbb2dbf5870f85f9a404a"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:151d6c40bc9db11e960619d2bf2ec5829f0aaffb10b41dcf6ad2ce0f3c0b2325"},
{file = "lxml-5.4.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:4025bf2884ac4370a3243c5aa8d66d3cb9e15d3ddd0af2d796eccc5f0244390e"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-macosx_10_15_x86_64.whl", hash = "sha256:9459e6892f59ecea2e2584ee1058f5d8f629446eab52ba2305ae13a32a059530"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:47fb24cc0f052f0576ea382872b3fc7e1f7e3028e53299ea751839418ade92a6"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:50441c9de951a153c698b9b99992e806b71c1f36d14b154592580ff4a9d0d877"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:ab339536aa798b1e17750733663d272038bf28069761d5be57cb4a9b0137b4f8"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:9776af1aad5a4b4a1317242ee2bea51da54b2a7b7b48674be736d463c999f37d"},
{file = "lxml-5.4.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:63e7968ff83da2eb6fdda967483a7a023aa497d85ad8f05c3ad9b1f2e8c84987"},
{file = "lxml-5.4.0.tar.gz", hash = "sha256:d12832e1dbea4be280b22fd0ea7c9b87f0d8fc51ba06e92dc62d52f804f78ebd"},
]
[package.extras]
cssselect = ["cssselect (>=0.7)"]
html-clean = ["lxml_html_clean"]
html5 = ["html5lib"]
htmlsoup = ["BeautifulSoup4"]
source = ["Cython (>=3.0.11,<3.1.0)"]
[[package]]
name = "markdown-it-py"
version = "3.0.0"
@@ -4236,6 +4386,27 @@ files = [
{file = "python_memcached-1.62-py2.py3-none-any.whl", hash = "sha256:1bdd8d2393ff53e80cd5e9442d750e658e0b35c3eebb3211af137303e3b729d1"},
]
[[package]]
name = "python3-saml"
version = "1.16.0"
description = "Saml Python Toolkit. Add SAML support to your Python software using this library"
optional = false
python-versions = "*"
groups = ["main"]
files = [
{file = "python3-saml-1.16.0.tar.gz", hash = "sha256:97c9669aecabc283c6e5fb4eb264f446b6e006f5267d01c9734f9d8bffdac133"},
{file = "python3_saml-1.16.0-py2-none-any.whl", hash = "sha256:c49097863c278ff669a337a96c46dc1f25d16307b4bb2679d2d1733cc4f5176a"},
{file = "python3_saml-1.16.0-py3-none-any.whl", hash = "sha256:20b97d11b04f01ee22e98f4a38242e2fea2e28fbc7fbc9bdd57cab5ac7fc2d0d"},
]
[package.dependencies]
isodate = ">=0.6.1"
lxml = ">=4.6.5,<4.7.0 || >4.7.0"
xmlsec = ">=1.3.9"
[package.extras]
test = ["coverage (>=4.5.2)", "flake8 (>=3.6.0,<=5.0.0)", "freezegun (>=0.3.11,<=1.1.0)", "pytest (>=4.6)"]
[[package]]
name = "pytz"
version = "2025.1"
@@ -5341,6 +5512,42 @@ files = [
{file = "xlsxwriter-3.2.3.tar.gz", hash = "sha256:ad6fd41bdcf1b885876b1f6b7087560aecc9ae5a9cc2ba97dcac7ab2e210d3d5"},
]
[[package]]
name = "xmlsec"
version = "1.3.15"
description = "Python bindings for the XML Security Library"
optional = false
python-versions = ">=3.5"
groups = ["main"]
files = [
{file = "xmlsec-1.3.15-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:60209f82a254a1d6083397c4eeae131e7ac2f64bfddb97f2b0b240369f03c4df"},
{file = "xmlsec-1.3.15-cp310-cp310-win32.whl", hash = "sha256:a62be0f8964bbec1efd2ca39b025c40da620a2ef9cb5440ff4ffa7e0c6906f70"},
{file = "xmlsec-1.3.15-cp310-cp310-win_amd64.whl", hash = "sha256:685b92860bbf048e3b725bd5e9310bd4d3515f7eafcb2c284dda62078a1ce90c"},
{file = "xmlsec-1.3.15-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c760230d4f77b7828857d076434e0810850eb2603775dc92fa9f760a98c2f694"},
{file = "xmlsec-1.3.15-cp311-cp311-win32.whl", hash = "sha256:901458034b7476e1fd0881a85814e184d00eec2b5df33b1ceeb312681e8cb9e8"},
{file = "xmlsec-1.3.15-cp311-cp311-win_amd64.whl", hash = "sha256:2ecbb65eea79a25769fbaa56c9e8bc4553aea63a9704795e962dfe06679b0191"},
{file = "xmlsec-1.3.15-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:0edff08e0442cdcc82bebf353ba4bcfd5a022f4b2751052ee1564afc5c78bef4"},
{file = "xmlsec-1.3.15-cp312-cp312-win32.whl", hash = "sha256:e5c402e5633fd39f75fe124219d66d383a040ba04d0de54e024afeb7fe7d3e3a"},
{file = "xmlsec-1.3.15-cp312-cp312-win_amd64.whl", hash = "sha256:0c47f2347e8dcc0a48648b9702af53179618c204414a8e36926a9f61214ebf0b"},
{file = "xmlsec-1.3.15-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6ac2154311d32a6571e22f224ed16356029e59bd5ca76edeb3922a809adfe89c"},
{file = "xmlsec-1.3.15-cp313-cp313-win32.whl", hash = "sha256:5ed218129f89b0592926ad2be42c017bece469db9b7380dc41bc09b01ca26d5d"},
{file = "xmlsec-1.3.15-cp313-cp313-win_amd64.whl", hash = "sha256:5fc29e69b064323317b3862751a3a8107670e0a17510ca4517bbdc1939a90b1a"},
{file = "xmlsec-1.3.15-cp36-cp36m-win32.whl", hash = "sha256:d0404dd76097b1f6dcbeff404c46cf045442a8cf9500f60c46a26ae03130ab9c"},
{file = "xmlsec-1.3.15-cp36-cp36m-win_amd64.whl", hash = "sha256:672bb43a12d6b8e2e4a392ef495ea731ded5acc1585f9358174295a6fb5df262"},
{file = "xmlsec-1.3.15-cp37-cp37m-win32.whl", hash = "sha256:96e24b22e862f0c50840a5af23cb7df186e7a1547b311a67ebca5b1e43ea0d86"},
{file = "xmlsec-1.3.15-cp37-cp37m-win_amd64.whl", hash = "sha256:bec066ce81a82a5a2b994b1e7be2af11715fd716a55754c645668acf9c5a64c0"},
{file = "xmlsec-1.3.15-cp38-cp38-win32.whl", hash = "sha256:95e80981b2e0ea74a7040cbf66b40072f4424298d7b50c3e587a026a7dab34ad"},
{file = "xmlsec-1.3.15-cp38-cp38-win_amd64.whl", hash = "sha256:c2a40f8549769ba5fdc223f0ae564d3b4d4ca52b6461d46bc508d3321267b2ad"},
{file = "xmlsec-1.3.15-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:a2d5692a683054dec769f4a1d6e8fade88ddcfc2cef89b20d0ecc1c75deb0dd6"},
{file = "xmlsec-1.3.15-cp39-cp39-macosx_13_0_arm64.whl", hash = "sha256:f0115d3b4f156df2cfee8424d75dcb7f5ca2cb4870af18b713098830493d3cb0"},
{file = "xmlsec-1.3.15-cp39-cp39-win32.whl", hash = "sha256:ffb32d3c5af289c8598d4f9215c9f8f6c208f1551e78f0180f525bc08c8a67d2"},
{file = "xmlsec-1.3.15-cp39-cp39-win_amd64.whl", hash = "sha256:3211da05c11c7a0d2b913a7834bff59e649150f41127949b3322442bc3986b56"},
{file = "xmlsec-1.3.15.tar.gz", hash = "sha256:baa856b83d0012e278e6f6cbec96ac8128de667ca9fa9a2eeb02c752e816f6d8"},
]
[package.dependencies]
lxml = ">=3.8"
[[package]]
name = "yarl"
version = "1.20.0"
@@ -5483,4 +5690,4 @@ type = ["pytest-mypy"]
[metadata]
lock-version = "2.1"
python-versions = ">=3.11,<3.13"
content-hash = "051924735a7069c8393fefc18fc2c310b196ea24ad41b8c984dc5852683d0407"
content-hash = "e25a4a8a62a8bad15ecb770bd3a6038e23ae545f55944b067816f59b3a96d5ca"
+3 -3
View File
@@ -7,8 +7,8 @@ authors = [{name = "Prowler Engineering", email = "engineering@prowler.com"}]
dependencies = [
"celery[pytest] (>=5.4.0,<6.0.0)",
"dj-rest-auth[with_social,jwt] (==7.0.1)",
"django==5.1.8",
"django-allauth==65.4.1",
"django==5.1.10",
"django-allauth[saml] (>=65.8.0,<66.0.0)",
"django-celery-beat (>=2.7.0,<3.0.0)",
"django-celery-results (>=2.5.1,<3.0.0)",
"django-cors-headers==4.4.0",
@@ -35,7 +35,7 @@ name = "prowler-api"
package-mode = false
# Needed for the SDK compatibility
requires-python = ">=3.11,<3.13"
version = "1.8.0"
version = "1.9.0"
[project.scripts]
celery = "src.backend.config.settings.celery"
+88 -26
View File
@@ -3,7 +3,14 @@ from django.db import transaction
from api.db_router import MainRouter
from api.db_utils import rls_transaction
from api.models import Membership, Role, Tenant, User, UserRoleRelationship
from api.models import (
Membership,
Role,
SAMLConfiguration,
Tenant,
User,
UserRoleRelationship,
)
class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
@@ -17,6 +24,8 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
def pre_social_login(self, request, sociallogin):
# Link existing accounts with the same email address
email = sociallogin.account.extra_data.get("email")
if sociallogin.account.provider == "saml":
email = sociallogin.user.email
if email:
existing_user = self.get_user_by_email(email)
if existing_user:
@@ -29,33 +38,86 @@ class ProwlerSocialAccountAdapter(DefaultSocialAccountAdapter):
"""
with transaction.atomic(using=MainRouter.admin_db):
user = super().save_user(request, sociallogin, form)
user.save(using=MainRouter.admin_db)
social_account_name = sociallogin.account.extra_data.get("name")
if social_account_name:
user.name = social_account_name
provider = sociallogin.account.provider
extra = sociallogin.account.extra_data
if provider == "saml":
# Handle SAML-specific logic
user.first_name = extra.get("firstName", [""])[0]
user.last_name = extra.get("lastName", [""])[0]
user.company_name = extra.get("organization", [""])[0]
user.name = f"{user.first_name} {user.last_name}".strip()
user.save(using=MainRouter.admin_db)
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
email_domain = user.email.split("@")[-1]
tenant = (
SAMLConfiguration.objects.using(MainRouter.admin_read)
.get(email_domain=email_domain)
.tenant
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
with rls_transaction(str(tenant.id)):
role_name = extra.get("userType", ["saml_default_role"])[0].strip()
try:
role = Role.objects.using(MainRouter.admin_read).get(
name=role_name, tenant_id=tenant.id
)
except Role.DoesNotExist:
role = Role.objects.using(MainRouter.admin_db).create(
name=role_name,
tenant_id=tenant.id,
manage_users=False,
manage_account=False,
manage_billing=False,
manage_providers=False,
manage_integrations=False,
manage_scans=False,
unlimited_visibility=False,
)
Membership.objects.using(MainRouter.admin_db).create(
user=user,
tenant=tenant,
role=Membership.RoleChoices.MEMBER,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
else:
# Handle other providers (e.g., GitHub, Google)
user.save(using=MainRouter.admin_db)
social_account_name = extra.get("name")
if social_account_name:
user.name = social_account_name
user.save(using=MainRouter.admin_db)
tenant = Tenant.objects.using(MainRouter.admin_db).create(
name=f"{user.email.split('@')[0]} default tenant"
)
with rls_transaction(str(tenant.id)):
Membership.objects.using(MainRouter.admin_db).create(
user=user, tenant=tenant, role=Membership.RoleChoices.OWNER
)
role = Role.objects.using(MainRouter.admin_db).create(
name="admin",
tenant_id=tenant.id,
manage_users=True,
manage_account=True,
manage_billing=True,
manage_providers=True,
manage_integrations=True,
manage_scans=True,
unlimited_visibility=True,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
return user
+28 -32
View File
@@ -15,6 +15,8 @@ from api.rbac.permissions import HasPermissions
class BaseViewSet(ModelViewSet):
_rls_ctx = None
authentication_classes = [JWTAuthentication]
required_permissions = []
permission_classes = [permissions.IsAuthenticated, HasPermissions]
@@ -45,25 +47,29 @@ class BaseViewSet(ModelViewSet):
def get_queryset(self):
raise NotImplementedError
def finalize_response(self, request, response, *args, **kwargs):
try:
return super().finalize_response(request, response, *args, **kwargs)
finally:
if self._rls_ctx:
self._rls_ctx.__exit__(None, None, None)
self._rls_ctx = None
class BaseRLSViewSet(BaseViewSet):
def dispatch(self, request, *args, **kwargs):
with transaction.atomic():
return super().dispatch(request, *args, **kwargs)
def initial(self, request, *args, **kwargs):
# Ideally, this logic would be in the `.setup()` method but DRF view sets don't call it
# https://docs.djangoproject.com/en/5.1/ref/class-based-views/base/#django.views.generic.base.View.setup
if request.auth is None:
raise NotAuthenticated
tenant_id = request.auth.get("tenant_id")
if tenant_id is None:
raise NotAuthenticated("Tenant ID is not present in token")
if not tenant_id:
raise NotAuthenticated("Tenant ID missing in JWT")
with rls_transaction(tenant_id):
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
self._rls_ctx = rls_transaction(tenant_id)
self._rls_ctx.__enter__()
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
def get_serializer_context(self):
context = super().get_serializer_context()
@@ -109,16 +115,6 @@ class BaseTenantViewset(BaseViewSet):
pass # Tenant might not exist, handle gracefully
def initial(self, request, *args, **kwargs):
if (
request.resolver_match.url_name != "tenant-detail"
and request.method != "DELETE"
):
user_id = str(request.user.id)
with rls_transaction(value=user_id, parameter=POSTGRES_USER_VAR):
return super().initial(request, *args, **kwargs)
# TODO: DRY this when we have time
if request.auth is None:
raise NotAuthenticated
@@ -126,18 +122,16 @@ class BaseTenantViewset(BaseViewSet):
if tenant_id is None:
raise NotAuthenticated("Tenant ID is not present in token")
with rls_transaction(tenant_id):
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
user_id = str(request.user.id)
self._rls_ctx = rls_transaction(value=user_id, parameter=POSTGRES_USER_VAR)
self._rls_ctx.__enter__()
return super().initial(request, *args, **kwargs)
class BaseUserViewset(BaseViewSet):
def dispatch(self, request, *args, **kwargs):
with transaction.atomic():
return super().dispatch(request, *args, **kwargs)
def initial(self, request, *args, **kwargs):
# TODO refactor after improving RLS on users
if request.stream is not None and request.stream.method == "POST":
return super().initial(request, *args, **kwargs)
if request.auth is None:
@@ -147,6 +141,8 @@ class BaseUserViewset(BaseViewSet):
if tenant_id is None:
raise NotAuthenticated("Tenant ID is not present in token")
with rls_transaction(tenant_id):
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
self._rls_ctx = rls_transaction(tenant_id)
self._rls_ctx.__enter__()
self.request.tenant_id = tenant_id
return super().initial(request, *args, **kwargs)
+7 -7
View File
@@ -190,6 +190,8 @@ def generate_compliance_overview_template(prowler_compliance: dict):
total_checks = len(requirement.Checks)
checks_dict = {check: None for check in requirement.Checks}
req_status_val = "MANUAL" if total_checks == 0 else "PASS"
# Build requirement dictionary
requirement_dict = {
"name": requirement.Name or requirement.Id,
@@ -204,20 +206,18 @@ def generate_compliance_overview_template(prowler_compliance: dict):
"manual": 0,
"total": total_checks,
},
"status": "PASS",
"status": req_status_val,
}
# Update requirements status
if total_checks == 0:
# Update requirements status counts for the framework
if req_status_val == "MANUAL":
requirements_status["manual"] += 1
elif req_status_val == "PASS":
requirements_status["passed"] += 1
# Add requirement to compliance requirements
compliance_requirements[requirement.Id] = requirement_dict
# Calculate pending requirements
pending_requirements = total_requirements - requirements_status["manual"]
requirements_status["passed"] = pending_requirements
# Build compliance dictionary
compliance_dict = {
"framework": compliance_data.Framework,
+11 -8
View File
@@ -3,27 +3,30 @@ ALLOWED_APPS = ("django", "socialaccount", "account", "authtoken", "silk")
class MainRouter:
default_db = "default"
default_read = "default_read"
prowler_user = "prowler_user"
admin_db = "admin"
admin_read = "admin_read"
def db_for_read(self, model, **hints): # noqa: F841
model_table_name = model._meta.db_table
if model_table_name.startswith("django_") or any(
model_table_name.startswith(f"{app}_") for app in ALLOWED_APPS
):
return self.admin_db
return None
if any(model_table_name.startswith(f"{app}_") for app in ALLOWED_APPS):
return self.admin_read
return self.default_read
def db_for_write(self, model, **hints): # noqa: F841
model_table_name = model._meta.db_table
if any(model_table_name.startswith(f"{app}_") for app in ALLOWED_APPS):
return self.admin_db
return None
return self.default_db
def allow_migrate(self, db, app_label, model_name=None, **hints): # noqa: F841
return db == self.admin_db
def allow_relation(self, obj1, obj2, **hints): # noqa: F841
# Allow relations if both objects are in either "default" or "admin" db connectors
if {obj1._state.db, obj2._state.db} <= {self.default_db, self.admin_db}:
# Allow relations if both objects are using one of our defined connectors
allowed = {self.default_db, self.default_read, self.admin_db, self.admin_read, self.prowler_user}
if {obj1._state.db, obj2._state.db} <= allowed:
return True
return None
+150 -21
View File
@@ -1,16 +1,19 @@
import re
import secrets
import uuid
from contextlib import contextmanager
from contextlib import ExitStack, contextmanager
from datetime import datetime, timedelta, timezone
from django.conf import settings
from django.contrib.auth.models import BaseUserManager
from django.db import connection, models, transaction
from django.db import connection, connections, models, transaction
from django_celery_beat.models import PeriodicTask
from psycopg2 import connect as psycopg2_connect
from psycopg2.extensions import AsIs, new_type, register_adapter, register_type
from rest_framework_json_api.serializers import ValidationError
from api.db_router import MainRouter
DB_USER = settings.DATABASES["default"]["USER"] if not settings.TESTING else "test"
DB_PASSWORD = (
settings.DATABASES["default"]["PASSWORD"] if not settings.TESTING else "test"
@@ -57,15 +60,29 @@ def rls_transaction(value: str, parameter: str = POSTGRES_TENANT_VAR):
value (str): Database configuration parameter value.
parameter (str): Database configuration parameter name, by default is 'api.tenant_id'.
"""
with transaction.atomic():
with connection.cursor() as cursor:
try:
# just in case the value is an UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
try:
# just in case the value is a UUID object
uuid.UUID(str(value))
except ValueError:
raise ValidationError("Must be a valid UUID")
aliases = (
MainRouter.default_db,
MainRouter.admin_db,
MainRouter.prowler_user,
MainRouter.default_read,
)
with ExitStack() as stack:
cursors = []
for alias in aliases:
stack.enter_context(transaction.atomic(using=alias))
cursor = connections[alias].cursor()
cursor.execute(SET_CONFIG_QUERY, [parameter, value])
yield cursor
cursors.append(cursor)
stack.callback(cursor.close)
yield
class CustomUserManager(BaseUserManager):
@@ -152,6 +169,28 @@ def delete_related_daily_task(provider_id: str):
PeriodicTask.objects.filter(name=task_name).delete()
def create_objects_in_batches(
tenant_id: str, model, objects: list, batch_size: int = 500
):
"""
Bulk-create model instances in repeated, per-tenant RLS transactions.
All chunks execute in their own transaction, so no single transaction
grows too large.
Args:
tenant_id (str): UUID string of the tenant under which to set RLS.
model: Django model class whose `.objects.bulk_create()` will be called.
objects (list): List of model instances (unsaved) to bulk-create.
batch_size (int): Maximum number of objects per bulk_create call.
"""
total = len(objects)
for i in range(0, total, batch_size):
chunk = objects[i : i + batch_size]
with rls_transaction(value=tenant_id, parameter=POSTGRES_TENANT_VAR):
model.objects.bulk_create(chunk, batch_size)
# Postgres Enums
@@ -227,6 +266,72 @@ def register_enum(apps, schema_editor, enum_class): # noqa: F841
register_adapter(enum_class, enum_adapter)
def _should_create_index_on_partition(
partition_name: str, all_partitions: bool = False
) -> bool:
"""
Determine if we should create an index on this partition.
Args:
partition_name: The name of the partition (e.g., "findings_2025_aug", "findings_default")
all_partitions: If True, create on all partitions. If False, only current/future partitions.
Returns:
bool: True if index should be created on this partition, False otherwise.
"""
if all_partitions:
return True
# Extract date from partition name if it follows the pattern
# Partition names look like: findings_2025_aug, findings_2025_jul, etc.
date_pattern = r"(\d{4})_([a-z]{3})$"
match = re.search(date_pattern, partition_name)
if not match:
# If we can't parse the date, include it to be safe (e.g., default partition)
return True
try:
year_str, month_abbr = match.groups()
year = int(year_str)
# Map month abbreviations to numbers
month_map = {
"jan": 1,
"feb": 2,
"mar": 3,
"apr": 4,
"may": 5,
"jun": 6,
"jul": 7,
"aug": 8,
"sep": 9,
"oct": 10,
"nov": 11,
"dec": 12,
}
month = month_map.get(month_abbr.lower())
if month is None:
# Unknown month abbreviation, include it to be safe
return True
partition_date = datetime(year, month, 1, tzinfo=timezone.utc)
# Get current month start
now = datetime.now(timezone.utc)
current_month_start = now.replace(
day=1, hour=0, minute=0, second=0, microsecond=0
)
# Include current month and future partitions
return partition_date >= current_month_start
except (ValueError, TypeError):
# If date parsing fails, include it to be safe
return True
def create_index_on_partitions(
apps, # noqa: F841
schema_editor,
@@ -235,16 +340,39 @@ def create_index_on_partitions(
columns: str,
method: str = "BTREE",
where: str = "",
all_partitions: bool = True,
):
"""
Create an index on every existing partition of `parent_table`.
Create an index on existing partitions of `parent_table`.
Args:
parent_table: The name of the root table (e.g. "findings").
index_name: A short name for the index (will be prefixed per-partition).
columns: The parenthesized column list, e.g. "tenant_id, scan_id, status".
method: The index method—BTREE, GIN, etc. Defaults to BTREE.
where: Optional WHERE clause (without the leading "WHERE"), e.g. "status = 'FAIL'".
method: The index method—BTREE, GIN, etc. Defaults to BTREE.
where: Optional WHERE clause (without the leading "WHERE"), e.g. "status = 'FAIL'".
all_partitions: Whether to create indexes on all partitions or just current/future ones.
Defaults to False (current/future only) to avoid maintenance overhead
on old partitions where the index may not be needed.
Examples:
# Create index only on current and future partitions (recommended for new indexes)
create_index_on_partitions(
apps, schema_editor,
parent_table="findings",
index_name="new_performance_idx",
columns="tenant_id, status, severity",
all_partitions=False # Default behavior
)
# Create index on all partitions (use when migrating existing critical indexes)
create_index_on_partitions(
apps, schema_editor,
parent_table="findings",
index_name="critical_existing_idx",
columns="tenant_id, scan_id",
all_partitions=True
)
"""
with connection.cursor() as cursor:
cursor.execute(
@@ -259,13 +387,14 @@ def create_index_on_partitions(
where_sql = f" WHERE {where}" if where else ""
for partition in partitions:
idx_name = f"{partition.replace('.', '_')}_{index_name}"
sql = (
f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {idx_name} "
f"ON {partition} USING {method} ({columns})"
f"{where_sql};"
)
schema_editor.execute(sql)
if _should_create_index_on_partition(partition, all_partitions):
idx_name = f"{partition.replace('.', '_')}_{index_name}"
sql = (
f"CREATE INDEX CONCURRENTLY IF NOT EXISTS {idx_name} "
f"ON {partition} USING {method} ({columns})"
f"{where_sql};"
)
schema_editor.execute(sql)
def drop_index_on_partitions(
@@ -279,7 +408,7 @@ def drop_index_on_partitions(
Args:
parent_table: The name of the root table (e.g. "findings").
index_name: The same short name used when creating them.
index_name: The same short name used when creating them.
"""
with connection.cursor() as cursor:
cursor.execute(
+3 -14
View File
@@ -1,10 +1,6 @@
import uuid
from functools import wraps
from django.db import connection, transaction
from rest_framework_json_api.serializers import ValidationError
from api.db_utils import POSTGRES_TENANT_VAR, SET_CONFIG_QUERY
from api.db_utils import rls_transaction
def set_tenant(func=None, *, keep_tenant=False):
@@ -42,7 +38,6 @@ def set_tenant(func=None, *, keep_tenant=False):
def decorator(func):
@wraps(func)
@transaction.atomic
def wrapper(*args, **kwargs):
try:
if not keep_tenant:
@@ -51,14 +46,8 @@ def set_tenant(func=None, *, keep_tenant=False):
tenant_id = kwargs["tenant_id"]
except KeyError:
raise KeyError("This task requires the tenant_id")
try:
uuid.UUID(tenant_id)
except ValueError:
raise ValidationError("Tenant ID must be a valid UUID")
with connection.cursor() as cursor:
cursor.execute(SET_CONFIG_QUERY, [POSTGRES_TENANT_VAR, tenant_id])
return func(*args, **kwargs)
with rls_transaction(tenant_id):
return func(*args, **kwargs)
return wrapper
+34 -4
View File
@@ -3,7 +3,7 @@ from rest_framework import status
from rest_framework.exceptions import APIException
from rest_framework_json_api.exceptions import exception_handler
from rest_framework_json_api.serializers import ValidationError
from rest_framework_simplejwt.exceptions import TokenError, InvalidToken
from rest_framework_simplejwt.exceptions import InvalidToken, TokenError
class ModelValidationError(ValidationError):
@@ -32,6 +32,31 @@ class InvitationTokenExpiredException(APIException):
default_code = "token_expired"
# Task Management Exceptions (non-HTTP)
class TaskManagementError(Exception):
"""Base exception for task management errors."""
def __init__(self, task=None):
self.task = task
super().__init__()
class TaskFailedException(TaskManagementError):
"""Raised when a task has failed."""
class TaskNotFoundException(TaskManagementError):
"""Raised when a task is not found."""
class TaskInProgressException(TaskManagementError):
"""Raised when a task is running but there's no related Task object to return."""
def __init__(self, task_result=None):
self.task_result = task_result
super().__init__()
def custom_exception_handler(exc, context):
if isinstance(exc, django_validation_error):
if hasattr(exc, "error_dict"):
@@ -39,7 +64,12 @@ def custom_exception_handler(exc, context):
else:
exc = ValidationError(detail=exc.messages[0], code=exc.code)
elif isinstance(exc, (TokenError, InvalidToken)):
exc.detail["messages"] = [
message_item["message"] for message_item in exc.detail["messages"]
]
if (
hasattr(exc, "detail")
and isinstance(exc.detail, dict)
and "messages" in exc.detail
):
exc.detail["messages"] = [
message_item["message"] for message_item in exc.detail["messages"]
]
return exception_handler(exc, context)
+138 -103
View File
@@ -22,7 +22,7 @@ from api.db_utils import (
StatusEnumField,
)
from api.models import (
ComplianceOverview,
ComplianceRequirementOverview,
Finding,
Integration,
Invitation,
@@ -81,6 +81,114 @@ class ChoiceInFilter(BaseInFilter, ChoiceFilter):
pass
class CommonFindingFilters(FilterSet):
# We filter providers from the scan in findings
provider = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_type__in = ChoiceInFilter(
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_uid = CharFilter(field_name="scan__provider__uid", lookup_expr="exact")
provider_uid__in = CharInFilter(field_name="scan__provider__uid", lookup_expr="in")
provider_uid__icontains = CharFilter(
field_name="scan__provider__uid", lookup_expr="icontains"
)
provider_alias = CharFilter(field_name="scan__provider__alias", lookup_expr="exact")
provider_alias__in = CharInFilter(
field_name="scan__provider__alias", lookup_expr="in"
)
provider_alias__icontains = CharFilter(
field_name="scan__provider__alias", lookup_expr="icontains"
)
updated_at = DateFilter(field_name="updated_at", lookup_expr="date")
uid = CharFilter(field_name="uid")
delta = ChoiceFilter(choices=Finding.DeltaChoices.choices)
status = ChoiceFilter(choices=StatusChoices.choices)
severity = ChoiceFilter(choices=SeverityChoices)
impact = ChoiceFilter(choices=SeverityChoices)
muted = BooleanFilter(
help_text="If this filter is not provided, muted and non-muted findings will be returned."
)
resources = UUIDInFilter(field_name="resource__id", lookup_expr="in")
region = CharFilter(method="filter_resource_region")
region__in = CharInFilter(field_name="resource_regions", lookup_expr="overlap")
region__icontains = CharFilter(
field_name="resource_regions", lookup_expr="icontains"
)
service = CharFilter(method="filter_resource_service")
service__in = CharInFilter(field_name="resource_services", lookup_expr="overlap")
service__icontains = CharFilter(
field_name="resource_services", lookup_expr="icontains"
)
resource_uid = CharFilter(field_name="resources__uid")
resource_uid__in = CharInFilter(field_name="resources__uid", lookup_expr="in")
resource_uid__icontains = CharFilter(
field_name="resources__uid", lookup_expr="icontains"
)
resource_name = CharFilter(field_name="resources__name")
resource_name__in = CharInFilter(field_name="resources__name", lookup_expr="in")
resource_name__icontains = CharFilter(
field_name="resources__name", lookup_expr="icontains"
)
resource_type = CharFilter(method="filter_resource_type")
resource_type__in = CharInFilter(field_name="resource_types", lookup_expr="overlap")
resource_type__icontains = CharFilter(
field_name="resources__type", lookup_expr="icontains"
)
# Temporarily disabled until we implement tag filtering in the UI
# resource_tag_key = CharFilter(field_name="resources__tags__key")
# resource_tag_key__in = CharInFilter(
# field_name="resources__tags__key", lookup_expr="in"
# )
# resource_tag_key__icontains = CharFilter(
# field_name="resources__tags__key", lookup_expr="icontains"
# )
# resource_tag_value = CharFilter(field_name="resources__tags__value")
# resource_tag_value__in = CharInFilter(
# field_name="resources__tags__value", lookup_expr="in"
# )
# resource_tag_value__icontains = CharFilter(
# field_name="resources__tags__value", lookup_expr="icontains"
# )
# resource_tags = CharInFilter(
# method="filter_resource_tag",
# lookup_expr="in",
# help_text="Filter by resource tags `key:value` pairs.\nMultiple values may be "
# "separated by commas.",
# )
def filter_resource_service(self, queryset, name, value):
return queryset.filter(resource_services__contains=[value])
def filter_resource_region(self, queryset, name, value):
return queryset.filter(resource_regions__contains=[value])
def filter_resource_type(self, queryset, name, value):
return queryset.filter(resource_types__contains=[value])
def filter_resource_tag(self, queryset, name, value):
overall_query = Q()
for key_value_pair in value:
tag_key, tag_value = key_value_pair.split(":", 1)
overall_query |= Q(
resources__tags__key__icontains=tag_key,
resources__tags__value__icontains=tag_value,
)
return queryset.filter(overall_query).distinct()
class TenantFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
updated_at = DateFilter(field_name="updated_at", lookup_expr="date")
@@ -257,94 +365,7 @@ class ResourceFilter(ProviderRelationshipFilterSet):
return queryset.filter(tags__text_search=value)
class FindingFilter(FilterSet):
# We filter providers from the scan in findings
provider = UUIDFilter(field_name="scan__provider__id", lookup_expr="exact")
provider__in = UUIDInFilter(field_name="scan__provider__id", lookup_expr="in")
provider_type = ChoiceFilter(
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_type__in = ChoiceInFilter(
choices=Provider.ProviderChoices.choices, field_name="scan__provider__provider"
)
provider_uid = CharFilter(field_name="scan__provider__uid", lookup_expr="exact")
provider_uid__in = CharInFilter(field_name="scan__provider__uid", lookup_expr="in")
provider_uid__icontains = CharFilter(
field_name="scan__provider__uid", lookup_expr="icontains"
)
provider_alias = CharFilter(field_name="scan__provider__alias", lookup_expr="exact")
provider_alias__in = CharInFilter(
field_name="scan__provider__alias", lookup_expr="in"
)
provider_alias__icontains = CharFilter(
field_name="scan__provider__alias", lookup_expr="icontains"
)
updated_at = DateFilter(field_name="updated_at", lookup_expr="date")
uid = CharFilter(field_name="uid")
delta = ChoiceFilter(choices=Finding.DeltaChoices.choices)
status = ChoiceFilter(choices=StatusChoices.choices)
severity = ChoiceFilter(choices=SeverityChoices)
impact = ChoiceFilter(choices=SeverityChoices)
muted = BooleanFilter(
help_text="If this filter is not provided, muted and non-muted findings will be returned."
)
resources = UUIDInFilter(field_name="resource__id", lookup_expr="in")
region = CharFilter(method="filter_resource_region")
region__in = CharInFilter(field_name="resource_regions", lookup_expr="overlap")
region__icontains = CharFilter(
field_name="resource_regions", lookup_expr="icontains"
)
service = CharFilter(method="filter_resource_service")
service__in = CharInFilter(field_name="resource_services", lookup_expr="overlap")
service__icontains = CharFilter(
field_name="resource_services", lookup_expr="icontains"
)
resource_uid = CharFilter(field_name="resources__uid")
resource_uid__in = CharInFilter(field_name="resources__uid", lookup_expr="in")
resource_uid__icontains = CharFilter(
field_name="resources__uid", lookup_expr="icontains"
)
resource_name = CharFilter(field_name="resources__name")
resource_name__in = CharInFilter(field_name="resources__name", lookup_expr="in")
resource_name__icontains = CharFilter(
field_name="resources__name", lookup_expr="icontains"
)
resource_type = CharFilter(method="filter_resource_type")
resource_type__in = CharInFilter(field_name="resource_types", lookup_expr="overlap")
resource_type__icontains = CharFilter(
field_name="resources__type", lookup_expr="icontains"
)
# Temporarily disabled until we implement tag filtering in the UI
# resource_tag_key = CharFilter(field_name="resources__tags__key")
# resource_tag_key__in = CharInFilter(
# field_name="resources__tags__key", lookup_expr="in"
# )
# resource_tag_key__icontains = CharFilter(
# field_name="resources__tags__key", lookup_expr="icontains"
# )
# resource_tag_value = CharFilter(field_name="resources__tags__value")
# resource_tag_value__in = CharInFilter(
# field_name="resources__tags__value", lookup_expr="in"
# )
# resource_tag_value__icontains = CharFilter(
# field_name="resources__tags__value", lookup_expr="icontains"
# )
# resource_tags = CharInFilter(
# method="filter_resource_tag",
# lookup_expr="in",
# help_text="Filter by resource tags `key:value` pairs.\nMultiple values may be "
# "separated by commas.",
# )
class FindingFilter(CommonFindingFilters):
scan = UUIDFilter(method="filter_scan_id")
scan__in = UUIDInFilter(method="filter_scan_id_in")
@@ -512,16 +533,6 @@ class FindingFilter(FilterSet):
return queryset.filter(id__lt=end)
def filter_resource_tag(self, queryset, name, value):
overall_query = Q()
for key_value_pair in value:
tag_key, tag_value = key_value_pair.split(":", 1)
overall_query |= Q(
resources__tags__key__icontains=tag_key,
resources__tags__value__icontains=tag_value,
)
return queryset.filter(overall_query).distinct()
@staticmethod
def maybe_date_to_datetime(value):
dt = value
@@ -530,6 +541,31 @@ class FindingFilter(FilterSet):
return dt
class LatestFindingFilter(CommonFindingFilters):
class Meta:
model = Finding
fields = {
"id": ["exact", "in"],
"uid": ["exact", "in"],
"delta": ["exact", "in"],
"status": ["exact", "in"],
"severity": ["exact", "in"],
"impact": ["exact", "in"],
"check_id": ["exact", "in", "icontains"],
}
filter_overrides = {
FindingDeltaEnumField: {
"filter_class": CharFilter,
},
StatusEnumField: {
"filter_class": CharFilter,
},
SeverityEnumField: {
"filter_class": CharFilter,
},
}
class ProviderSecretFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
updated_at = DateFilter(field_name="updated_at", lookup_expr="date")
@@ -601,12 +637,11 @@ class RoleFilter(FilterSet):
class ComplianceOverviewFilter(FilterSet):
inserted_at = DateFilter(field_name="inserted_at", lookup_expr="date")
provider_type = ChoiceFilter(choices=Provider.ProviderChoices.choices)
provider_type__in = ChoiceInFilter(choices=Provider.ProviderChoices.choices)
scan_id = UUIDFilter(field_name="scan__id")
scan_id = UUIDFilter(field_name="scan_id")
region = CharFilter(field_name="region")
class Meta:
model = ComplianceOverview
model = ComplianceRequirementOverview
fields = {
"inserted_at": ["date", "gte", "lte"],
"compliance_id": ["exact", "icontains"],
@@ -0,0 +1,80 @@
from django.contrib.sites.models import Site
from django.core.management.base import BaseCommand
from django.db import DEFAULT_DB_ALIAS, connection, connections, transaction
from django.db.migrations.recorder import MigrationRecorder
def table_exists(table_name):
with connection.cursor() as cursor:
cursor.execute(
"""
SELECT EXISTS (
SELECT 1 FROM information_schema.tables
WHERE table_name = %s
)
""",
[table_name],
)
return cursor.fetchone()[0]
class Command(BaseCommand):
help = "Fix migration inconsistency between socialaccount and sites"
def add_arguments(self, parser):
parser.add_argument(
"--database",
default=DEFAULT_DB_ALIAS,
help="Specifies the database to operate on.",
)
def handle(self, *args, **options):
db = options["database"]
connection = connections[db]
recorder = MigrationRecorder(connection)
applied = set(recorder.applied_migrations())
has_social = ("socialaccount", "0001_initial") in applied
with connection.cursor() as cursor:
cursor.execute(
"""
SELECT EXISTS (
SELECT FROM information_schema.tables
WHERE table_name = 'django_site'
);
"""
)
site_table_exists = cursor.fetchone()[0]
if has_social and not site_table_exists:
self.stdout.write(
f"Detected inconsistency in '{db}'. Creating 'django_site' table manually..."
)
with transaction.atomic(using=db):
with connection.schema_editor() as schema_editor:
schema_editor.create_model(Site)
recorder.record_applied("sites", "0001_initial")
recorder.record_applied("sites", "0002_alter_domain_unique")
self.stdout.write(
"Fixed: 'django_site' table created and migrations registered."
)
# Ensure the relationship table also exists
if not table_exists("socialaccount_socialapp_sites"):
self.stdout.write(
"Detected missing 'socialaccount_socialapp_sites' table. Creating manually..."
)
with connection.schema_editor() as schema_editor:
from allauth.socialaccount.models import SocialApp
schema_editor.create_model(
SocialApp._meta.get_field("sites").remote_field.through
)
self.stdout.write(
"Fixed: 'socialaccount_socialapp_sites' table created."
)
@@ -0,0 +1,29 @@
from functools import partial
from django.db import migrations
from api.db_utils import create_index_on_partitions, drop_index_on_partitions
class Migration(migrations.Migration):
atomic = False
dependencies = [
("api", "0023_resources_lookup_optimization"),
]
operations = [
migrations.RunPython(
partial(
create_index_on_partitions,
parent_table="findings",
index_name="find_tenant_uid_inserted_idx",
columns="tenant_id, uid, inserted_at DESC",
),
reverse_code=partial(
drop_index_on_partitions,
parent_table="findings",
index_name="find_tenant_uid_inserted_idx",
),
)
]
@@ -0,0 +1,17 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0024_findings_uid_index_partitions"),
]
operations = [
migrations.AddIndex(
model_name="finding",
index=models.Index(
fields=["tenant_id", "uid", "-inserted_at"],
name="find_tenant_uid_inserted_idx",
),
),
]
@@ -0,0 +1,14 @@
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("api", "0025_findings_uid_index_parent"),
]
operations = [
migrations.RunSQL(
"ALTER TYPE provider_secret_type ADD VALUE IF NOT EXISTS 'service_account';",
reverse_sql=migrations.RunSQL.noop,
),
]
@@ -0,0 +1,124 @@
# Generated by Django 5.1.8 on 2025-05-21 11:37
import uuid
import django.db.models.deletion
from django.db import migrations, models
import api.db_utils
import api.rls
from api.rls import RowLevelSecurityConstraint
class Migration(migrations.Migration):
dependencies = [
("api", "0026_provider_secret_gcp_service_account"),
]
operations = [
migrations.CreateModel(
name="ComplianceRequirementOverview",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
("inserted_at", models.DateTimeField(auto_now_add=True)),
("compliance_id", models.TextField(blank=False)),
("framework", models.TextField(blank=False)),
("version", models.TextField(blank=True)),
("description", models.TextField(blank=True)),
("region", models.TextField(blank=False)),
("requirement_id", models.TextField(blank=False)),
(
"requirement_status",
api.db_utils.StatusEnumField(
choices=[
("FAIL", "Fail"),
("PASS", "Pass"),
("MANUAL", "Manual"),
]
),
),
("passed_checks", models.IntegerField(default=0)),
("failed_checks", models.IntegerField(default=0)),
("total_checks", models.IntegerField(default=0)),
(
"scan",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="compliance_requirements_overviews",
related_query_name="compliance_requirements_overview",
to="api.scan",
),
),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "compliance_requirements_overviews",
"abstract": False,
"indexes": [
models.Index(
fields=["tenant_id", "scan_id"], name="cro_tenant_scan_idx"
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id"],
name="cro_scan_comp_idx",
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id", "region"],
name="cro_scan_comp_reg_idx",
),
models.Index(
fields=[
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
],
name="cro_scan_comp_req_idx",
),
models.Index(
fields=[
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
"region",
],
name="cro_scan_comp_req_reg_idx",
),
],
"constraints": [
models.UniqueConstraint(
fields=(
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
"region",
),
name="unique_tenant_compliance_requirement_overview",
)
],
},
),
migrations.AddConstraint(
model_name="ComplianceRequirementOverview",
constraint=RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_compliancerequirementoverview",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
]
@@ -0,0 +1,29 @@
from functools import partial
from django.db import migrations
from api.db_utils import create_index_on_partitions, drop_index_on_partitions
class Migration(migrations.Migration):
atomic = False
dependencies = [
("api", "0027_compliance_requirement_overviews"),
]
operations = [
migrations.RunPython(
partial(
create_index_on_partitions,
parent_table="findings",
index_name="find_tenant_scan_check_idx",
columns="tenant_id, scan_id, check_id",
),
reverse_code=partial(
drop_index_on_partitions,
parent_table="findings",
index_name="find_tenant_scan_check_idx",
),
)
]
@@ -0,0 +1,17 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("api", "0028_findings_check_index_partitions"),
]
operations = [
migrations.AddIndex(
model_name="finding",
index=models.Index(
fields=["tenant_id", "scan_id", "check_id"],
name="find_tenant_scan_check_idx",
),
),
]
@@ -0,0 +1,120 @@
# Generated by Django 5.1.8 on 2025-05-15 09:54
import uuid
import django.db.models.deletion
from django.db import migrations, models
import api.rls
class Migration(migrations.Migration):
dependencies = [
("api", "0029_findings_check_index_parent"),
]
operations = [
migrations.CreateModel(
name="SAMLDomainIndex",
fields=[
(
"id",
models.BigAutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("email_domain", models.CharField(max_length=254, unique=True)),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "saml_domain_index",
},
),
migrations.AddConstraint(
model_name="samldomainindex",
constraint=models.UniqueConstraint(
fields=("email_domain", "tenant"),
name="unique_resources_by_email_domain",
),
),
migrations.AddConstraint(
model_name="samldomainindex",
constraint=api.rls.BaseSecurityConstraint(
name="statements_on_samldomainindex",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.CreateModel(
name="SAMLConfiguration",
fields=[
(
"id",
models.UUIDField(
default=uuid.uuid4,
editable=False,
primary_key=True,
serialize=False,
),
),
(
"email_domain",
models.CharField(
help_text="Email domain used to identify the tenant, e.g. prowlerdemo.com",
max_length=254,
unique=True,
),
),
(
"metadata_xml",
models.TextField(
help_text="Raw IdP metadata XML to configure SingleSignOnService, certificates, etc."
),
),
("created_at", models.DateTimeField(auto_now_add=True)),
("updated_at", models.DateTimeField(auto_now=True)),
(
"tenant",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE, to="api.tenant"
),
),
],
options={
"db_table": "saml_configurations",
},
),
migrations.AddConstraint(
model_name="samlconfiguration",
constraint=api.rls.RowLevelSecurityConstraint(
"tenant_id",
name="rls_on_samlconfiguration",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
),
migrations.AddConstraint(
model_name="samlconfiguration",
constraint=models.UniqueConstraint(
fields=("tenant",), name="unique_samlconfig_per_tenant"
),
),
migrations.AlterField(
model_name="integration",
name="integration_type",
field=api.db_utils.IntegrationTypeEnumField(
choices=[
("amazon_s3", "Amazon S3"),
("aws_security_hub", "AWS Security Hub"),
("jira", "JIRA"),
("slack", "Slack"),
]
),
),
]
+310 -4
View File
@@ -1,13 +1,18 @@
import json
import re
import xml.etree.ElementTree as ET
from uuid import UUID, uuid4
from allauth.socialaccount.models import SocialApp
from config.settings.social_login import SOCIALACCOUNT_PROVIDERS
from cryptography.fernet import Fernet
from django.conf import settings
from django.contrib.auth.models import AbstractBaseUser
from django.contrib.postgres.fields import ArrayField
from django.contrib.postgres.indexes import GinIndex
from django.contrib.postgres.search import SearchVector, SearchVectorField
from django.contrib.sites.models import Site
from django.core.exceptions import ValidationError
from django.core.validators import MinLengthValidator
from django.db import models
from django.db.models import Q
@@ -19,6 +24,7 @@ from psqlextra.models import PostgresPartitionedModel
from psqlextra.types import PostgresPartitioningMethod
from uuid6 import uuid7
from api.db_router import MainRouter
from api.db_utils import (
CustomUserManager,
FindingDeltaEnumField,
@@ -218,9 +224,13 @@ class Provider(RowLevelSecurityProtectedModel):
@staticmethod
def validate_m365_uid(value):
if not re.match(r"^[a-zA-Z0-9-]+\.onmicrosoft\.com$", value):
if not re.match(
r"""^(?!-)[A-Za-z0-9](?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?(?:\.(?!-)[A-Za-z0-9]"""
r"""(?:[A-Za-z0-9-]{0,61}[A-Za-z0-9])?)*\.[A-Za-z]{2,}$""",
value,
):
raise ModelValidationError(
detail="M365 tenant ID must be a valid domain.",
detail="M365 domain ID must be a valid domain.",
code="m365-uid",
pointer="/data/attributes/uid",
)
@@ -238,7 +248,7 @@ class Provider(RowLevelSecurityProtectedModel):
@staticmethod
def validate_kubernetes_uid(value):
if not re.match(
r"^[a-z0-9][A-Za-z0-9_.:\/-]{1,250}$",
r"^[a-zA-Z0-9][a-zA-Z0-9._@:\/-]{1,250}$",
value,
):
raise ModelValidationError(
@@ -426,6 +436,7 @@ class Scan(RowLevelSecurityProtectedModel):
PeriodicTask, on_delete=models.CASCADE, null=True, blank=True
)
output_location = models.CharField(blank=True, null=True, max_length=200)
# TODO: mutelist foreign key
class Meta(RowLevelSecurityProtectedModel.Meta):
@@ -750,9 +761,17 @@ class Finding(PostgresPartitionedModel, RowLevelSecurityProtectedModel):
condition=Q(delta="new"),
name="find_delta_new_idx",
),
models.Index(
fields=["tenant_id", "uid", "-inserted_at"],
name="find_tenant_uid_inserted_idx",
),
GinIndex(fields=["resource_services"], name="gin_find_service_idx"),
GinIndex(fields=["resource_regions"], name="gin_find_region_idx"),
GinIndex(fields=["resource_types"], name="gin_find_rtype_idx"),
models.Index(
fields=["tenant_id", "scan_id", "check_id"],
name="find_tenant_scan_check_idx",
),
]
class JSONAPIMeta:
@@ -841,6 +860,7 @@ class ProviderSecret(RowLevelSecurityProtectedModel):
class TypeChoices(models.TextChoices):
STATIC = "static", _("Key-value pairs")
ROLE = "role", _("Role assumption")
SERVICE_ACCOUNT = "service_account", _("GCP Service Account Key")
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
@@ -1133,6 +1153,78 @@ class ComplianceOverview(RowLevelSecurityProtectedModel):
resource_name = "compliance-overviews"
class ComplianceRequirementOverview(RowLevelSecurityProtectedModel):
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
inserted_at = models.DateTimeField(auto_now_add=True, editable=False)
compliance_id = models.TextField(blank=False)
framework = models.TextField(blank=False)
version = models.TextField(blank=True)
description = models.TextField(blank=True)
region = models.TextField(blank=False)
requirement_id = models.TextField(blank=False)
requirement_status = StatusEnumField(choices=StatusChoices)
passed_checks = models.IntegerField(default=0)
failed_checks = models.IntegerField(default=0)
total_checks = models.IntegerField(default=0)
scan = models.ForeignKey(
Scan,
on_delete=models.CASCADE,
related_name="compliance_requirements_overviews",
related_query_name="compliance_requirements_overview",
)
class Meta(RowLevelSecurityProtectedModel.Meta):
db_table = "compliance_requirements_overviews"
constraints = [
models.UniqueConstraint(
fields=(
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
"region",
),
name="unique_tenant_compliance_requirement_overview",
),
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "DELETE"],
),
]
indexes = [
models.Index(fields=["tenant_id", "scan_id"], name="cro_tenant_scan_idx"),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id"],
name="cro_scan_comp_idx",
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id", "region"],
name="cro_scan_comp_reg_idx",
),
models.Index(
fields=["tenant_id", "scan_id", "compliance_id", "requirement_id"],
name="cro_scan_comp_req_idx",
),
models.Index(
fields=[
"tenant_id",
"scan_id",
"compliance_id",
"requirement_id",
"region",
],
name="cro_scan_comp_req_reg_idx",
),
]
class JSONAPIMeta:
resource_name = "compliance-requirements-overviews"
class ScanSummary(RowLevelSecurityProtectedModel):
objects = ActiveProviderManager()
all_objects = models.Manager()
@@ -1201,7 +1293,6 @@ class ScanSummary(RowLevelSecurityProtectedModel):
class Integration(RowLevelSecurityProtectedModel):
class IntegrationChoices(models.TextChoices):
S3 = "amazon_s3", _("Amazon S3")
SAML = "saml", _("SAML")
AWS_SECURITY_HUB = "aws_security_hub", _("AWS Security Hub")
JIRA = "jira", _("JIRA")
SLACK = "slack", _("Slack")
@@ -1275,6 +1366,221 @@ class IntegrationProviderRelationship(RowLevelSecurityProtectedModel):
]
class SAMLDomainIndex(models.Model):
"""
Public index of SAML domains. No RLS. Used for fast lookup in SAML login flow.
"""
email_domain = models.CharField(max_length=254, unique=True)
tenant = models.ForeignKey("Tenant", on_delete=models.CASCADE)
class Meta:
db_table = "saml_domain_index"
constraints = [
models.UniqueConstraint(
fields=("email_domain", "tenant"),
name="unique_resources_by_email_domain",
),
BaseSecurityConstraint(
name="statements_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
]
class SAMLConfiguration(RowLevelSecurityProtectedModel):
"""
Stores per-tenant SAML settings, including email domain and IdP metadata.
Automatically syncs to a SocialApp instance on save.
Note:
This model exists to provide a tenant-aware abstraction over SAML configuration.
It supports row-level security, custom validation, and metadata parsing, enabling
Prowler to expose a clean API and admin interface for managing SAML integrations.
Although Django Allauth uses the SocialApp model to store provider configuration,
it is not designed for multi-tenant use. SocialApp lacks support for tenant scoping,
email domain mapping, and structured metadata handling.
By managing SAMLConfiguration separately, we ensure:
- Strong isolation between tenants via RLS.
- Ownership of raw IdP metadata and its validation.
- An explicit link between SAML config and business-level identifiers (e.g. email domain).
- Programmatic transformation into the SocialApp format used by Allauth.
In short, this model acts as a secure and user-friendly layer over Allauth's lower-level primitives.
"""
id = models.UUIDField(primary_key=True, default=uuid4, editable=False)
email_domain = models.CharField(
max_length=254,
unique=True,
help_text="Email domain used to identify the tenant, e.g. prowlerdemo.com",
)
metadata_xml = models.TextField(
help_text="Raw IdP metadata XML to configure SingleSignOnService, certificates, etc."
)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class JSONAPIMeta:
resource_name = "saml-configurations"
class Meta:
db_table = "saml_configurations"
constraints = [
RowLevelSecurityConstraint(
field="tenant_id",
name="rls_on_%(class)s",
statements=["SELECT", "INSERT", "UPDATE", "DELETE"],
),
# 1 config per tenant
models.UniqueConstraint(
fields=["tenant"],
name="unique_samlconfig_per_tenant",
),
]
def clean(self, old_email_domain=None):
# Domain must not contain @
if "@" in self.email_domain:
raise ValidationError({"email_domain": "Domain must not contain @"})
# Enforce at most one config per tenant
qs = SAMLConfiguration.objects.filter(tenant=self.tenant)
# Exclude ourselves in case of update
if self.pk:
qs = qs.exclude(pk=self.pk)
if qs.exists():
raise ValidationError(
{"tenant": "A SAML configuration already exists for this tenant."}
)
# The email domain must be unique in the entire system
qs = SAMLConfiguration.objects.using(MainRouter.admin_read).filter(
email_domain__iexact=self.email_domain
)
if qs.exists() and old_email_domain != self.email_domain:
raise ValidationError(
{"tenant": "There is a problem with your email domain."}
)
def save(self, *args, **kwargs):
self.email_domain = self.email_domain.strip().lower()
is_create = not SAMLConfiguration.objects.filter(pk=self.pk).exists()
if not is_create:
old = SAMLConfiguration.objects.get(pk=self.pk)
old_email_domain = old.email_domain
old_metadata_xml = old.metadata_xml
else:
old_email_domain = None
old_metadata_xml = None
self.clean(old_email_domain)
super().save(*args, **kwargs)
if is_create or (
old_email_domain != self.email_domain
or old_metadata_xml != self.metadata_xml
):
self._sync_social_app(old_email_domain)
# Sync the public index
if not is_create and old_email_domain and old_email_domain != self.email_domain:
SAMLDomainIndex.objects.filter(email_domain=old_email_domain).delete()
# Create/update the new domain index
SAMLDomainIndex.objects.update_or_create(
email_domain=self.email_domain, defaults={"tenant": self.tenant}
)
def _parse_metadata(self):
"""
Parse the raw IdP metadata XML and extract:
- entity_id
- sso_url
- slo_url (may be None)
- x509cert (required)
"""
ns = {
"md": "urn:oasis:names:tc:SAML:2.0:metadata",
"ds": "http://www.w3.org/2000/09/xmldsig#",
}
try:
root = ET.fromstring(self.metadata_xml)
except ET.ParseError as e:
raise ValidationError({"metadata_xml": f"Invalid XML: {e}"})
# Entity ID
entity_id = root.attrib.get("entityID")
# SSO endpoint (must exist)
sso = root.find(".//md:IDPSSODescriptor/md:SingleSignOnService", ns)
if sso is None or "Location" not in sso.attrib:
raise ValidationError(
{"metadata_xml": "Missing SingleSignOnService in metadata."}
)
sso_url = sso.attrib["Location"]
# SLO endpoint (optional)
slo = root.find(".//md:IDPSSODescriptor/md:SingleLogoutService", ns)
slo_url = slo.attrib.get("Location") if slo is not None else None
# X.509 certificate (required)
cert = root.find(
'.//md:KeyDescriptor[@use="signing"]/ds:KeyInfo/ds:X509Data/ds:X509Certificate',
ns,
)
if cert is None or not cert.text or not cert.text.strip():
raise ValidationError(
{
"metadata_xml": 'Metadata must include a <ds:X509Certificate> under <KeyDescriptor use="signing">.'
}
)
x509cert = cert.text.strip()
return {
"entity_id": entity_id,
"sso_url": sso_url,
"slo_url": slo_url,
"x509cert": x509cert,
}
def _sync_social_app(self, previous_email_domain=None):
"""
Create or update the corresponding SocialApp based on email_domain.
If the domain changed, update the matching SocialApp.
"""
idp_settings = self._parse_metadata()
settings_dict = SOCIALACCOUNT_PROVIDERS["saml"].copy()
settings_dict["idp"] = idp_settings
current_site = Site.objects.get(id=settings.SITE_ID)
social_app_qs = SocialApp.objects.filter(
provider="saml", client_id=previous_email_domain or self.email_domain
)
if social_app_qs.exists():
social_app = social_app_qs.first()
social_app.client_id = self.email_domain
social_app.name = f"{self.tenant.name} SAML ({self.email_domain})"
social_app.settings = settings_dict
social_app.save()
social_app.sites.set([current_site])
else:
social_app = SocialApp.objects.create(
provider="saml",
client_id=self.email_domain,
name=f"{self.tenant.name} SAML ({self.email_domain})",
settings=settings_dict,
)
social_app.sites.set([current_site])
class ResourceScanSummary(RowLevelSecurityProtectedModel):
scan_id = models.UUIDField(default=uuid7, db_index=True)
resource_id = models.UUIDField(default=uuid4, db_index=True)
+1 -1
View File
@@ -1,4 +1,4 @@
from rest_framework_json_api.pagination import JsonApiPageNumberPagination
from drf_spectacular_jsonapi.schemas.pagination import JsonApiPageNumberPagination
class ComplianceOverviewPagination(JsonApiPageNumberPagination):
+3 -1
View File
@@ -30,7 +30,9 @@ class HasPermissions(BasePermission):
return True
user_roles = (
User.objects.using(MainRouter.admin_db).get(id=request.user.id).roles.all()
User.objects.using(MainRouter.admin_read)
.get(id=request.user.id)
.roles.all()
)
if not user_roles:
return False
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,82 @@
from unittest.mock import MagicMock
import pytest
from allauth.socialaccount.models import SocialLogin
from django.contrib.auth import get_user_model
from api.adapters import ProwlerSocialAccountAdapter
from api.db_router import MainRouter
from api.models import Membership, SAMLConfiguration, Tenant
User = get_user_model()
@pytest.mark.django_db
class TestProwlerSocialAccountAdapter:
def test_get_user_by_email_returns_user(self, create_test_user):
adapter = ProwlerSocialAccountAdapter()
user = adapter.get_user_by_email(create_test_user.email)
assert user == create_test_user
def test_get_user_by_email_returns_none_for_unknown_email(self):
adapter = ProwlerSocialAccountAdapter()
assert adapter.get_user_by_email("notfound@example.com") is None
def test_pre_social_login_links_existing_user(self, create_test_user, rf):
adapter = ProwlerSocialAccountAdapter()
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.account.provider = "saml"
sociallogin.account.extra_data = {}
sociallogin.user = create_test_user
sociallogin.connect = MagicMock()
adapter.pre_social_login(rf.get("/"), sociallogin)
call_args = sociallogin.connect.call_args
assert call_args is not None
called_request, called_user = call_args[0]
assert called_request.path == "/"
assert called_user.email == create_test_user.email
def test_pre_social_login_no_link_if_email_missing(self, rf):
adapter = ProwlerSocialAccountAdapter()
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = MagicMock()
sociallogin.account.provider = "github"
sociallogin.account.extra_data = {}
sociallogin.connect = MagicMock()
adapter.pre_social_login(rf.get("/"), sociallogin)
sociallogin.connect.assert_not_called()
def test_save_user_saml_flow(
self,
rf,
saml_setup,
saml_sociallogin,
):
adapter = ProwlerSocialAccountAdapter()
request = rf.get("/")
saml_sociallogin.user.email = saml_setup["email"]
tenant = Tenant.objects.using(MainRouter.admin_db).get(
id=saml_setup["tenant_id"]
)
saml_config = SAMLConfiguration.objects.using(MainRouter.admin_read).get(
tenant=tenant
)
assert saml_config.email_domain == saml_setup["domain"]
user = adapter.save_user(request, saml_sociallogin)
assert user.email == saml_setup["email"]
assert (
Membership.objects.using(MainRouter.admin_read)
.filter(user=user, tenant=tenant)
.exists()
)
+6 -6
View File
@@ -1,12 +1,12 @@
from unittest.mock import patch, MagicMock
from unittest.mock import MagicMock, patch
from api.compliance import (
generate_compliance_overview_template,
generate_scan_compliance,
get_prowler_provider_checks,
get_prowler_provider_compliance,
load_prowler_compliance,
load_prowler_checks,
generate_scan_compliance,
generate_compliance_overview_template,
load_prowler_compliance,
)
from api.models import Provider
@@ -69,7 +69,7 @@ class TestCompliance:
load_prowler_compliance()
from api.compliance import PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE, PROWLER_CHECKS
from api.compliance import PROWLER_CHECKS, PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE
assert PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE == {
"template_key": "template_value"
@@ -268,7 +268,7 @@ class TestCompliance:
"manual": 0,
"total": 0,
},
"status": "PASS",
"status": "MANUAL",
},
},
"requirements_status": {
+7 -5
View File
@@ -1,15 +1,17 @@
from unittest.mock import patch
import pytest
from config.django.base import DATABASE_ROUTERS as PROD_DATABASE_ROUTERS
from django.conf import settings
from django.db.migrations.recorder import MigrationRecorder
from django.db.utils import ConnectionRouter
from api.db_router import MainRouter
from api.rls import Tenant
from config.django.base import DATABASE_ROUTERS as PROD_DATABASE_ROUTERS
from unittest.mock import patch
@patch("api.db_router.MainRouter.admin_db", new="admin")
@patch("api.db_router.MainRouter.admin_read", new="admin_read")
class TestMainDatabaseRouter:
@pytest.fixture(scope="module")
def router(self):
@@ -20,12 +22,12 @@ class TestMainDatabaseRouter:
@pytest.mark.parametrize("api_model", [Tenant])
def test_router_api_models(self, api_model, router):
assert router.db_for_read(api_model) == "default"
assert router.db_for_read(api_model) == "prowler_user_read"
assert router.db_for_write(api_model) == "default"
assert router.allow_migrate_model(MainRouter.admin_db, api_model)
assert not router.allow_migrate_model("default", api_model)
def test_router_django_models(self, router):
assert router.db_for_read(MigrationRecorder.Migration) == MainRouter.admin_db
assert not router.db_for_read(MigrationRecorder.Migration) == "default"
assert router.db_for_read(MigrationRecorder.Migration) == MainRouter.admin_read
assert router.db_for_read(MigrationRecorder.Migration) != "default"
@@ -3,9 +3,13 @@ from enum import Enum
from unittest.mock import patch
import pytest
from django.conf import settings
from freezegun import freeze_time
from api.db_utils import (
_should_create_index_on_partition,
batch_delete,
create_objects_in_batches,
enum_to_choices,
generate_random_token,
one_week_from_now,
@@ -138,3 +142,88 @@ class TestBatchDelete:
)
assert Provider.objects.all().count() == 0
assert summary == {"api.Provider": create_test_providers}
class TestShouldCreateIndexOnPartition:
@freeze_time("2025-05-15 00:00:00Z")
@pytest.mark.parametrize(
"partition_name, all_partitions, expected",
[
("any_name", True, True),
("findings_default", True, True),
("findings_2022_jan", True, True),
("foo_bar", False, True),
("findings_2025_MAY", False, True),
("findings_2025_may", False, True),
("findings_2025_jun", False, True),
("findings_2025_apr", False, False),
("findings_2025_xyz", False, True),
],
)
def test_partition_inclusion_logic(self, partition_name, all_partitions, expected):
assert (
_should_create_index_on_partition(partition_name, all_partitions)
is expected
)
@freeze_time("2025-05-15 00:00:00Z")
def test_invalid_date_components(self):
# even if regex matches but int conversion fails, we fallback True
# (e.g. year too big, month number parse error)
bad_name = "findings_99999_jan"
assert _should_create_index_on_partition(bad_name, False) is True
bad_name2 = "findings_2025_abc"
# abc not in month_map → fallback True
assert _should_create_index_on_partition(bad_name2, False) is True
@pytest.mark.django_db
class TestCreateObjectsInBatches:
@pytest.fixture
def tenant(self, tenants_fixture):
return tenants_fixture[0]
def make_provider_instances(self, tenant, count):
"""
Return a list of `count` unsaved Provider instances for the given tenant.
"""
base_uid = 1000
return [
Provider(
tenant=tenant,
uid=str(base_uid + i),
provider=Provider.ProviderChoices.AWS,
)
for i in range(count)
]
def test_exact_multiple_of_batch(self, tenant):
total = 6
batch_size = 3
objs = self.make_provider_instances(tenant, total)
create_objects_in_batches(str(tenant.id), Provider, objs, batch_size=batch_size)
qs = Provider.objects.filter(tenant=tenant)
assert qs.count() == total
def test_non_multiple_of_batch(self, tenant):
total = 7
batch_size = 3
objs = self.make_provider_instances(tenant, total)
create_objects_in_batches(str(tenant.id), Provider, objs, batch_size=batch_size)
qs = Provider.objects.filter(tenant=tenant)
assert qs.count() == total
def test_batch_size_default(self, tenant):
default_size = settings.DJANGO_DELETION_BATCH_SIZE
total = default_size + 2
objs = self.make_provider_instances(tenant, total)
create_objects_in_batches(str(tenant.id), Provider, objs)
qs = Provider.objects.filter(tenant=tenant)
assert qs.count() == total
+379
View File
@@ -0,0 +1,379 @@
import json
from uuid import uuid4
import pytest
from django_celery_results.models import TaskResult
from rest_framework import status
from rest_framework.response import Response
from api.exceptions import (
TaskFailedException,
TaskInProgressException,
TaskNotFoundException,
)
from api.models import Task, User
from api.rls import Tenant
from api.v1.mixins import PaginateByPkMixin, TaskManagementMixin
@pytest.mark.django_db
class TestPaginateByPkMixin:
@pytest.fixture
def tenant(self):
return Tenant.objects.create(name="Test Tenant")
@pytest.fixture
def users(self, tenant):
# Create 5 users with proper email field
users = []
for i in range(5):
user = User.objects.create(email=f"user{i}@example.com", name=f"User {i}")
users.append(user)
return users
class DummyView(PaginateByPkMixin):
def __init__(self, page):
self._page = page
def paginate_queryset(self, qs):
return self._page
def get_serializer(self, queryset, many):
class S:
def __init__(self, data):
# serialize to list of ids
self.data = [obj.id for obj in data] if many else queryset.id
return S(queryset)
def get_paginated_response(self, data):
return Response({"results": data}, status=status.HTTP_200_OK)
def test_no_pagination(self, users):
base_qs = User.objects.all().order_by("id")
view = self.DummyView(page=None)
resp = view.paginate_by_pk(
request=None, base_queryset=base_qs, manager=User.objects
)
# since no pagination, should return all ids in order
expected = [u.id for u in base_qs]
assert isinstance(resp, Response)
assert resp.data == expected
def test_with_pagination(self, users):
base_qs = User.objects.all().order_by("id")
# simulate paging to first 2 ids
page = [base_qs[1].id, base_qs[3].id]
view = self.DummyView(page=page)
resp = view.paginate_by_pk(
request=None, base_queryset=base_qs, manager=User.objects
)
# should fetch only those two users, in the same order as page
assert resp.status_code == status.HTTP_200_OK
assert resp.data == {"results": page}
@pytest.mark.django_db
class TestTaskManagementMixin:
class DummyView(TaskManagementMixin):
pass
@pytest.fixture
def tenant(self):
return Tenant.objects.create(name="Test Tenant")
@pytest.fixture(autouse=True)
def cleanup(self):
Task.objects.all().delete()
TaskResult.objects.all().delete()
def test_no_task_and_no_taskresult_raises_not_found(self):
view = self.DummyView()
with pytest.raises(TaskNotFoundException):
view.check_task_status("task_xyz", {"foo": "bar"})
def test_no_task_and_no_taskresult_returns_none_when_not_raising(self):
view = self.DummyView()
result = view.check_task_status(
"task_xyz", {"foo": "bar"}, raise_on_not_found=False
)
assert result is None
def test_taskresult_pending_raises_in_progress(self):
task_kwargs = {"foo": "bar"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_xyz",
task_kwargs=json.dumps(task_kwargs),
status="PENDING",
)
view = self.DummyView()
with pytest.raises(TaskInProgressException) as excinfo:
view.check_task_status("task_xyz", task_kwargs, raise_on_not_found=False)
assert hasattr(excinfo.value, "task_result")
assert excinfo.value.task_result == tr
def test_taskresult_started_raises_in_progress(self):
task_kwargs = {"foo": "bar"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_xyz",
task_kwargs=json.dumps(task_kwargs),
status="STARTED",
)
view = self.DummyView()
with pytest.raises(TaskInProgressException) as excinfo:
view.check_task_status("task_xyz", task_kwargs, raise_on_not_found=False)
assert hasattr(excinfo.value, "task_result")
assert excinfo.value.task_result == tr
def test_taskresult_progress_raises_in_progress(self):
task_kwargs = {"foo": "bar"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_xyz",
task_kwargs=json.dumps(task_kwargs),
status="PROGRESS",
)
view = self.DummyView()
with pytest.raises(TaskInProgressException) as excinfo:
view.check_task_status("task_xyz", task_kwargs, raise_on_not_found=False)
assert hasattr(excinfo.value, "task_result")
assert excinfo.value.task_result == tr
def test_taskresult_failure_raises_failed(self):
task_kwargs = {"a": 1}
TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_fail",
task_kwargs=json.dumps(task_kwargs),
status="FAILURE",
)
view = self.DummyView()
with pytest.raises(TaskFailedException):
view.check_task_status("task_fail", task_kwargs, raise_on_not_found=False)
def test_taskresult_failure_returns_none_when_not_raising(self):
task_kwargs = {"a": 1}
TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_fail",
task_kwargs=json.dumps(task_kwargs),
status="FAILURE",
)
view = self.DummyView()
result = view.check_task_status(
"task_fail", task_kwargs, raise_on_failed=False, raise_on_not_found=False
)
assert result is None
def test_taskresult_success_returns_none(self):
task_kwargs = {"x": 2}
TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_ok",
task_kwargs=json.dumps(task_kwargs),
status="SUCCESS",
)
view = self.DummyView()
# should not raise, and returns None
assert (
view.check_task_status("task_ok", task_kwargs, raise_on_not_found=False)
is None
)
def test_taskresult_revoked_returns_none(self):
task_kwargs = {"x": 2}
TaskResult.objects.create(
task_id=str(uuid4()),
task_name="task_revoked",
task_kwargs=json.dumps(task_kwargs),
status="REVOKED",
)
view = self.DummyView()
# should not raise, and returns None
assert (
view.check_task_status(
"task_revoked", task_kwargs, raise_on_not_found=False
)
is None
)
def test_task_with_failed_status_raises_failed(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="FAILURE",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
with pytest.raises(TaskFailedException) as excinfo:
view.check_task_status("scan_task", task_kwargs)
# Check that the exception contains the expected task
assert hasattr(excinfo.value, "task")
assert excinfo.value.task == task
def test_task_with_cancelled_status_raises_failed(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="REVOKED",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
with pytest.raises(TaskFailedException) as excinfo:
view.check_task_status("scan_task", task_kwargs)
# Check that the exception contains the expected task
assert hasattr(excinfo.value, "task")
assert excinfo.value.task == task
def test_task_with_failed_status_returns_task_when_not_raising(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="FAILURE",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.check_task_status("scan_task", task_kwargs, raise_on_failed=False)
assert result == task
def test_task_with_completed_status_returns_none(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="SUCCESS",
)
Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.check_task_status("scan_task", task_kwargs)
assert result is None
def test_task_with_executing_status_returns_task(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="STARTED",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.check_task_status("scan_task", task_kwargs)
assert result is not None
assert result.pk == task.pk
def test_task_with_pending_status_returns_task(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="PENDING",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.check_task_status("scan_task", task_kwargs)
assert result is not None
assert result.pk == task.pk
def test_get_task_response_if_running_returns_none_for_completed_task(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="SUCCESS",
)
Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.get_task_response_if_running("scan_task", task_kwargs)
assert result is None
def test_get_task_response_if_running_returns_none_for_no_task(self):
view = self.DummyView()
result = view.get_task_response_if_running(
"nonexistent", {"foo": "bar"}, raise_on_not_found=False
)
assert result is None
def test_get_task_response_if_running_returns_202_for_executing_task(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="STARTED",
)
task = Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.get_task_response_if_running("scan_task", task_kwargs)
assert isinstance(result, Response)
assert result.status_code == status.HTTP_202_ACCEPTED
assert "Content-Location" in result.headers
# The response should contain the serialized task data
assert result.data is not None
assert "id" in result.data
assert str(result.data["id"]) == str(task.id)
def test_get_task_response_if_running_returns_none_for_available_task(self, tenant):
task_kwargs = {"provider_id": "test"}
tr = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs),
status="PENDING",
)
Task.objects.create(tenant=tenant, task_runner_task=tr)
view = self.DummyView()
result = view.get_task_response_if_running("scan_task", task_kwargs)
# PENDING maps to AVAILABLE, which is not EXECUTING, so should return None
assert result is None
def test_kwargs_filtering_works_correctly(self, tenant):
# Create tasks with different kwargs
task_kwargs_1 = {"provider_id": "test1", "scan_type": "full"}
task_kwargs_2 = {"provider_id": "test2", "scan_type": "quick"}
tr1 = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs_1),
status="STARTED",
)
tr2 = TaskResult.objects.create(
task_id=str(uuid4()),
task_name="scan_task",
task_kwargs=json.dumps(task_kwargs_2),
status="STARTED",
)
task1 = Task.objects.create(tenant=tenant, task_runner_task=tr1)
task2 = Task.objects.create(tenant=tenant, task_runner_task=tr2)
view = self.DummyView()
# Should find task1 when searching for its kwargs
result1 = view.check_task_status("scan_task", {"provider_id": "test1"})
assert result1 is not None
assert result1.pk == task1.pk
# Should find task2 when searching for its kwargs
result2 = view.check_task_status("scan_task", {"provider_id": "test2"})
assert result2 is not None
assert result2.pk == task2.pk
# Should not find anything when searching for non-existent kwargs
result3 = view.check_task_status(
"scan_task", {"provider_id": "test3"}, raise_on_not_found=False
)
assert result3 is None
+150 -1
View File
@@ -1,6 +1,9 @@
import pytest
from allauth.socialaccount.models import SocialApp
from django.core.exceptions import ValidationError
from api.models import Resource, ResourceTag
from api.db_router import MainRouter
from api.models import Resource, ResourceTag, SAMLConfiguration, Tenant
@pytest.mark.django_db
@@ -120,3 +123,149 @@ class TestResourceModel:
# compliance={},
# )
# assert Finding.objects.filter(uid=long_uid).exists()
@pytest.mark.django_db
class TestSAMLConfigurationModel:
VALID_METADATA = """<?xml version='1.0' encoding='UTF-8'?>
<md:EntityDescriptor entityID='TEST' xmlns:md='urn:oasis:names:tc:SAML:2.0:metadata'>
<md:IDPSSODescriptor WantAuthnRequestsSigned='false' protocolSupportEnumeration='urn:oasis:names:tc:SAML:2.0:protocol'>
<md:KeyDescriptor use='signing'>
<ds:KeyInfo xmlns:ds='http://www.w3.org/2000/09/xmldsig#'>
<ds:X509Data>
<ds:X509Certificate>FAKECERTDATA</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST' Location='https://idp.test/sso'/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>
"""
def test_creates_valid_configuration(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant A")
config = SAMLConfiguration.objects.using(MainRouter.admin_db).create(
email_domain="ssoexample.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant,
)
assert config.email_domain == "ssoexample.com"
assert SocialApp.objects.filter(client_id="ssoexample.com").exists()
def test_email_domain_with_at_symbol_fails(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant B")
config = SAMLConfiguration(
email_domain="invalid@domain.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant,
)
with pytest.raises(ValidationError) as exc_info:
config.clean()
errors = exc_info.value.message_dict
assert "email_domain" in errors
assert "Domain must not contain @" in errors["email_domain"][0]
def test_duplicate_email_domain_fails(self):
tenant1 = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant C1")
tenant2 = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant C2")
SAMLConfiguration.objects.using(MainRouter.admin_db).create(
email_domain="duplicate.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant1,
)
config = SAMLConfiguration(
email_domain="duplicate.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant2,
)
with pytest.raises(ValidationError) as exc_info:
config.clean()
errors = exc_info.value.message_dict
assert "tenant" in errors
assert "There is a problem with your email domain." in errors["tenant"][0]
def test_duplicate_tenant_config_fails(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant D")
SAMLConfiguration.objects.using(MainRouter.admin_db).create(
email_domain="unique1.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant,
)
config = SAMLConfiguration(
email_domain="unique2.com",
metadata_xml=TestSAMLConfigurationModel.VALID_METADATA,
tenant=tenant,
)
with pytest.raises(ValidationError) as exc_info:
config.clean()
errors = exc_info.value.message_dict
assert "tenant" in errors
assert (
"A SAML configuration already exists for this tenant."
in errors["tenant"][0]
)
def test_invalid_metadata_xml_fails(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant E")
config = SAMLConfiguration(
email_domain="brokenxml.com",
metadata_xml="<bad<xml>",
tenant=tenant,
)
with pytest.raises(ValidationError) as exc_info:
config._parse_metadata()
errors = exc_info.value.message_dict
assert "metadata_xml" in errors
assert "Invalid XML" in errors["metadata_xml"][0]
assert "not well-formed" in errors["metadata_xml"][0]
def test_metadata_missing_sso_fails(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant F")
xml = """<md:EntityDescriptor entityID="x" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata">
<md:IDPSSODescriptor></md:IDPSSODescriptor>
</md:EntityDescriptor>"""
config = SAMLConfiguration(
email_domain="nosso.com",
metadata_xml=xml,
tenant=tenant,
)
with pytest.raises(ValidationError) as exc_info:
config._parse_metadata()
errors = exc_info.value.message_dict
assert "metadata_xml" in errors
assert "Missing SingleSignOnService" in errors["metadata_xml"][0]
def test_metadata_missing_certificate_fails(self):
tenant = Tenant.objects.using(MainRouter.admin_db).create(name="Tenant G")
xml = """<md:EntityDescriptor entityID="x" xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata">
<md:IDPSSODescriptor>
<md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://example.com/sso"/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>"""
config = SAMLConfiguration(
email_domain="nocert.com",
metadata_xml=xml,
tenant=tenant,
)
with pytest.raises(ValidationError) as exc_info:
config._parse_metadata()
errors = exc_info.value.message_dict
assert "metadata_xml" in errors
assert "X509Certificate" in errors["metadata_xml"][0]
+667 -183
View File
@@ -9,15 +9,20 @@ from unittest.mock import ANY, MagicMock, Mock, patch
import jwt
import pytest
from allauth.socialaccount.models import SocialAccount, SocialApp
from botocore.exceptions import ClientError, NoCredentialsError
from conftest import API_JSON_CONTENT_TYPE, TEST_PASSWORD, TEST_USER
from django.conf import settings
from django.http import JsonResponse
from django.test import RequestFactory
from django.urls import reverse
from django_celery_results.models import TaskResult
from rest_framework import status
from rest_framework.response import Response
from api.compliance import get_compliance_frameworks
from api.db_router import MainRouter
from api.models import (
ComplianceOverview,
Integration,
Invitation,
Membership,
@@ -27,6 +32,7 @@ from api.models import (
ProviderSecret,
Role,
RoleProviderGroupRelationship,
SAMLConfiguration,
Scan,
StateChoices,
Task,
@@ -34,6 +40,7 @@ from api.models import (
UserRoleRelationship,
)
from api.rls import Tenant
from api.v1.views import ComplianceOverviewViewSet, TenantFinishACSView
TODAY = str(datetime.today().date())
@@ -913,11 +920,41 @@ class TestProviderViewSet:
"uid": "gke_aaaa-dev_europe-test1_dev-aaaa-test-cluster-long-name-123456789",
"alias": "GKE",
},
{
"provider": "kubernetes",
"uid": "gke_project/cluster-name",
"alias": "GKE",
},
{
"provider": "kubernetes",
"uid": "admin@k8s-demo",
"alias": "test",
},
{
"provider": "azure",
"uid": "8851db6b-42e5-4533-aa9e-30a32d67e875",
"alias": "test",
},
{
"provider": "m365",
"uid": "TestingPro.onmicrosoft.com",
"alias": "test",
},
{
"provider": "m365",
"uid": "subdomain.domain.es",
"alias": "test",
},
{
"provider": "m365",
"uid": "microsoft.net",
"alias": "test",
},
{
"provider": "m365",
"uid": "subdomain1.subdomain2.subdomain3.subdomain4.domain.net",
"alias": "test",
},
]
),
)
@@ -986,6 +1023,51 @@ class TestProviderViewSet:
"invalid_choice",
"provider",
),
(
{
"provider": "m365",
"uid": "https://test.com",
"alias": "test",
},
"m365-uid",
"uid",
),
(
{
"provider": "m365",
"uid": "thisisnotadomain",
"alias": "test",
},
"m365-uid",
"uid",
),
(
{
"provider": "m365",
"uid": "http://test.com",
"alias": "test",
},
"m365-uid",
"uid",
),
(
{
"provider": "m365",
"uid": f"{'a' * 64}.domain.com",
"alias": "test",
},
"m365-uid",
"uid",
),
(
{
"provider": "m365",
"uid": f"subdomain.{'a' * 64}.com",
"alias": "test",
},
"m365-uid",
"uid",
),
]
),
)
@@ -1613,6 +1695,26 @@ class TestProviderSecretViewSet:
"refresh_token": "refresh-token",
},
),
# GCP with Service Account Key secret
(
Provider.ProviderChoices.GCP.value,
ProviderSecret.TypeChoices.SERVICE_ACCOUNT,
{
"service_account_key": {
"type": "service_account",
"project_id": "project-id",
"private_key_id": "private-key-id",
"private_key": "private-key",
"client_email": "client-email",
"client_id": "client-id",
"auth_uri": "auth-uri",
"token_uri": "token-uri",
"auth_provider_x509_cert_url": "auth-provider-x509-cert-url",
"client_x509_cert_url": "client-x509-cert-url",
"universe_domain": "universe-domain",
},
},
),
# Kubernetes with STATIC secret
(
Provider.ProviderChoices.KUBERNETES.value,
@@ -2238,7 +2340,10 @@ class TestScanViewSet:
url = reverse("scan-report", kwargs={"pk": scan.id})
response = authenticated_client.get(url)
assert response.status_code == status.HTTP_404_NOT_FOUND
assert response.json()["errors"]["detail"] == "The scan has no reports."
assert (
response.json()["errors"]["detail"]
== "The scan has no reports, or the report generation task has not started yet."
)
def test_report_s3_no_credentials(
self, authenticated_client, scans_fixture, monkeypatch
@@ -2306,7 +2411,7 @@ class TestScanViewSet:
):
"""
When output_location is a local path and glob.glob returns an empty list,
the view should return HTTP 404 with detail "The scan has no reports."
the view should return HTTP 404 with detail "The scan has no reports, or the report generation task has not started yet."
"""
scan = scans_fixture[0]
scan.output_location = "/tmp/nonexistent_report_pattern.zip"
@@ -2318,7 +2423,10 @@ class TestScanViewSet:
response = authenticated_client.get(url)
assert response.status_code == 404
assert response.json()["errors"]["detail"] == "The scan has no reports."
assert (
response.json()["errors"]["detail"]
== "The scan has no reports, or the report generation task has not started yet."
)
def test_report_local_file(self, authenticated_client, scans_fixture, monkeypatch):
scan = scans_fixture[0]
@@ -2393,7 +2501,10 @@ class TestScanViewSet:
url = reverse("scan-compliance", kwargs={"pk": scan.id, "name": framework})
resp = authenticated_client.get(url)
assert resp.status_code == status.HTTP_404_NOT_FOUND
assert resp.json()["errors"]["detail"] == "The scan has no reports."
assert (
resp.json()["errors"]["detail"]
== "The scan has no reports, or the report generation task has not started yet."
)
def test_compliance_s3_no_credentials(
self, authenticated_client, scans_fixture, monkeypatch
@@ -2535,6 +2646,36 @@ class TestScanViewSet:
assert response.status_code == status.HTTP_404_NOT_FOUND
@patch("api.v1.views.TaskSerializer")
def test__get_task_status_finds_task_using_kwargs(
self, mock_task_serializer, authenticated_client, scans_fixture
):
scan = scans_fixture[0]
scan.state = StateChoices.COMPLETED
scan.output_location = "dummy"
scan.save()
task_result = TaskResult.objects.create(
task_name="scan-report",
task_kwargs={"scan_id": str(scan.id)},
)
task = Task.objects.create(
tenant_id=scan.tenant_id,
task_runner_task=task_result,
)
mock_task_serializer.return_value.data = {
"id": str(task.id),
"state": StateChoices.EXECUTING,
}
url = reverse("scan-report", kwargs={"pk": scan.id})
response = authenticated_client.get(url)
assert response.status_code == status.HTTP_202_ACCEPTED
assert response.data["id"] == str(task.id)
@patch("api.v1.views.get_s3_client")
@patch("api.v1.views.sentry_sdk.capture_exception")
def test_compliance_list_objects_client_error(
@@ -2585,7 +2726,10 @@ class TestScanViewSet:
response = authenticated_client.get(url)
assert response.status_code == status.HTTP_404_NOT_FOUND
assert response.json()["errors"]["detail"] == "The scan has no reports."
assert (
response.json()["errors"]["detail"]
== "The scan has no reports, or the report generation task has not started yet."
)
@patch("api.v1.views.get_s3_client")
def test_report_s3_client_error_other(
@@ -3185,6 +3329,29 @@ class TestFindingViewSet:
]
}
def test_findings_latest(self, authenticated_client, latest_scan_finding):
response = authenticated_client.get(
reverse("finding-latest"),
)
assert response.status_code == status.HTTP_200_OK
# The latest scan only has one finding, in comparison with `GET /findings`
assert len(response.json()["data"]) == 1
assert (
response.json()["data"][0]["attributes"]["status"]
== latest_scan_finding.status
)
def test_findings_metadata_latest(self, authenticated_client, latest_scan_finding):
response = authenticated_client.get(
reverse("finding-metadata_latest"),
)
assert response.status_code == status.HTTP_200_OK
attributes = response.json()["data"]["attributes"]
assert attributes["services"] == latest_scan_finding.resource_services
assert attributes["regions"] == latest_scan_finding.resource_regions
assert attributes["resource_types"] == latest_scan_finding.resource_types
@pytest.mark.django_db
class TestJWTFields:
@@ -4600,210 +4767,266 @@ class TestComplianceOverviewViewSet:
assert len(response.json()["data"]) == 0
def test_compliance_overview_list(
self, authenticated_client, compliance_overviews_fixture
self, authenticated_client, compliance_requirements_overviews_fixture
):
# List compliance overviews with existing data
compliance_overview1, compliance_overview2 = compliance_overviews_fixture
scan_id = str(compliance_overview1.scan.id)
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-list"),
{"filter[scan_id]": scan_id},
)
assert response.status_code == status.HTTP_200_OK
assert (
len(response.json()["data"]) == 1
) # Due to the custom get_queryset method, only one compliance_id
data = response.json()["data"]
assert len(data) == 2 # Two compliance frameworks
def test_compliance_overview_list_missing_scan_id(self, authenticated_client):
# Attempt to list compliance overviews without providing filter[scan_id]
response = authenticated_client.get(reverse("complianceoverview-list"))
# Check that we get aggregated data for each compliance framework
framework_ids = [item["id"] for item in data]
assert "aws_account_security_onboarding_aws" in framework_ids
assert "cis_1.4_aws" in framework_ids
# Check structure of response
for item in data:
assert "id" in item
assert "attributes" in item
attributes = item["attributes"]
assert "framework" in attributes
assert "version" in attributes
assert "requirements_passed" in attributes
assert "requirements_failed" in attributes
assert "requirements_manual" in attributes
assert "total_requirements" in attributes
def test_compliance_overview_metadata(
self, authenticated_client, compliance_requirements_overviews_fixture
):
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-metadata"),
{"filter[scan_id]": scan_id},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert "attributes" in data
assert "regions" in data["attributes"]
assert isinstance(data["attributes"]["regions"], list)
def test_compliance_overview_requirements(
self, authenticated_client, compliance_requirements_overviews_fixture
):
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
compliance_id = requirement_overview1.compliance_id
response = authenticated_client.get(
reverse("complianceoverview-requirements"),
{
"filter[scan_id]": scan_id,
"filter[compliance_id]": compliance_id,
},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert len(data) > 0
# Check structure of requirements response
for item in data:
assert "id" in item
assert "attributes" in item
attributes = item["attributes"]
assert "framework" in attributes
assert "version" in attributes
assert "description" in attributes
assert "status" in attributes
def test_compliance_overview_requirements_manual(
self, authenticated_client, compliance_requirements_overviews_fixture
):
scan_id = str(compliance_requirements_overviews_fixture[0].scan.id)
# Compliance with a manual requirement
compliance_id = "aws_account_security_onboarding_aws"
response = authenticated_client.get(
reverse("complianceoverview-requirements"),
{
"filter[scan_id]": scan_id,
"filter[compliance_id]": compliance_id,
},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert data[-1]["attributes"]["status"] == "MANUAL"
def test_compliance_overview_requirements_missing_scan_id(
self, authenticated_client
):
response = authenticated_client.get(
reverse("complianceoverview-requirements"),
{"filter[compliance_id]": "aws_account_security_onboarding_aws"},
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert response.json()["errors"][0]["source"]["pointer"] == "filter[scan_id]"
assert response.json()["errors"][0]["code"] == "required"
def test_compliance_overview_requirements_missing_compliance_id(
self, authenticated_client, compliance_requirements_overviews_fixture
):
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-requirements"),
{"filter[scan_id]": scan_id},
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_compliance_overview_attributes(self, authenticated_client):
response = authenticated_client.get(
reverse("complianceoverview-attributes"),
{"filter[compliance_id]": "aws_account_security_onboarding_aws"},
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert len(data) > 0
# Check structure of attributes response
for item in data:
assert "id" in item
assert "attributes" in item
attributes = item["attributes"]
assert "framework" in attributes
assert "version" in attributes
assert "description" in attributes
assert "attributes" in attributes
assert "metadata" in attributes["attributes"]
assert "check_ids" in attributes["attributes"]
def test_compliance_overview_attributes_missing_compliance_id(
self, authenticated_client
):
response = authenticated_client.get(
reverse("complianceoverview-attributes"),
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
def test_compliance_overview_task_management_integration(
self, authenticated_client, compliance_requirements_overviews_fixture
):
"""Test that task management mixin is properly integrated"""
from unittest.mock import patch
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
# Mock a running task
with patch.object(
ComplianceOverviewViewSet, "get_task_response_if_running"
) as mock_task_response:
mock_response = Response(
{"detail": "Task is running"}, status=status.HTTP_202_ACCEPTED
)
mock_task_response.return_value = mock_response
response = authenticated_client.get(
reverse("complianceoverview-list"),
{"filter[scan_id]": scan_id},
)
assert response.status_code == status.HTTP_202_ACCEPTED
mock_task_response.assert_called_once()
def test_compliance_overview_task_failed_exception(
self, authenticated_client, compliance_requirements_overviews_fixture
):
"""Test handling of TaskFailedException"""
from unittest.mock import patch
from api.exceptions import TaskFailedException
requirement_overview1 = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview1.scan.id)
# Mock a failed task
with patch.object(
ComplianceOverviewViewSet, "get_task_response_if_running"
) as mock_task_response:
mock_task_response.side_effect = TaskFailedException("Task failed")
response = authenticated_client.get(
reverse("complianceoverview-list"),
{"filter[scan_id]": scan_id},
)
assert response.status_code == status.HTTP_500_INTERNAL_SERVER_ERROR
assert "Task failed to generate compliance overview data" in str(
response.data
)
@pytest.mark.parametrize(
"filter_name, filter_value, expected_count",
"filter_name, filter_value_attr, expected_count_min",
[
("compliance_id", "aws_account_security_onboarding_aws", 1),
("compliance_id.icontains", "security_onboarding", 1),
("framework", "AWS-Account-Security-Onboarding", 1),
("framework.icontains", "security-onboarding", 1),
("version", "1.0", 1),
("version", "2.0", 0),
("version.icontains", "0", 1),
("region", "eu-west-1", 1),
("region.icontains", "west-1", 1),
("region.in", "eu-west-1,eu-west-2", 1),
("inserted_at.date", "2024-01-01", 0),
("inserted_at.date", TODAY, 1),
("inserted_at.gte", "2024-01-01", 1),
("scan_id", "scan.id", 1),
("compliance_id", "compliance_id", 1),
("framework", "framework", 1),
("version", "version", 1),
("region", "region", 1),
],
)
def test_compliance_overview_filters(
self,
authenticated_client,
compliance_overviews_fixture,
compliance_requirements_overviews_fixture,
filter_name,
filter_value,
expected_count,
filter_value_attr,
expected_count_min,
):
# Test filtering compliance overviews
compliance_overview1 = compliance_overviews_fixture[0]
scan_id = str(compliance_overview1.scan.id)
requirement_overview = compliance_requirements_overviews_fixture[0]
scan_id = str(requirement_overview.scan.id)
filter_value = requirement_overview
for attr in filter_value_attr.split("."):
filter_value = getattr(filter_value, attr)
filter_value = str(filter_value)
query_params = {
"filter[scan_id]": scan_id,
f"filter[{filter_name}]": filter_value,
}
if filter_name == "scan_id":
query_params = {"filter[scan_id]": filter_value}
response = authenticated_client.get(
reverse("complianceoverview-list"),
{
"filter[scan_id]": scan_id,
f"filter[{filter_name}]": filter_value,
},
)
assert response.status_code == status.HTTP_200_OK
assert len(response.json()["data"]) == expected_count
@pytest.mark.parametrize(
"filter_name",
["invalid_filter", "unknown_field"],
)
def test_compliance_overview_filters_invalid(
self, authenticated_client, compliance_overviews_fixture, filter_name
):
# Test handling of invalid filters
compliance_overview1 = compliance_overviews_fixture[0]
scan_id = str(compliance_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-list"),
{
"filter[scan_id]": scan_id,
f"filter[{filter_name}]": "some_value",
},
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
@pytest.mark.parametrize(
"sort_field",
["inserted_at", "-inserted_at", "compliance_id", "-compliance_id"],
)
def test_compliance_overview_sort(
self, authenticated_client, compliance_overviews_fixture, sort_field
):
# Test sorting compliance overviews
compliance_overview1 = compliance_overviews_fixture[0]
scan_id = str(compliance_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-list"),
{
"filter[scan_id]": scan_id,
"sort": sort_field,
},
)
assert response.status_code == status.HTTP_200_OK
def test_compliance_overview_sort_invalid(
self, authenticated_client, compliance_overviews_fixture
):
# Test handling of invalid sort parameters
compliance_overview1 = compliance_overviews_fixture[0]
scan_id = str(compliance_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-list"),
{
"filter[scan_id]": scan_id,
"sort": "invalid_field",
},
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert response.json()["errors"][0]["code"] == "invalid"
assert "invalid sort parameter" in response.json()["errors"][0]["detail"]
def test_compliance_overview_retrieve(
self, authenticated_client, compliance_overviews_fixture
):
# Retrieve a specific compliance overview
compliance_overview1 = compliance_overviews_fixture[0]
response = authenticated_client.get(
reverse(
"complianceoverview-detail",
kwargs={"pk": compliance_overview1.id},
),
)
assert response.status_code == status.HTTP_200_OK
data = response.json()["data"]
assert data["id"] == str(compliance_overview1.id)
attributes = data["attributes"]
assert attributes["compliance_id"] == compliance_overview1.compliance_id
assert attributes["framework"] == compliance_overview1.framework
assert attributes["version"] == compliance_overview1.version
assert attributes["region"] == compliance_overview1.region
assert attributes["description"] == compliance_overview1.description
assert "requirements" in attributes
def test_compliance_overview_invalid_retrieve(self, authenticated_client):
# Attempt to retrieve a compliance overview with an invalid ID
response = authenticated_client.get(
reverse(
"complianceoverview-detail",
kwargs={"pk": "invalid-id"},
),
)
assert response.status_code == status.HTTP_404_NOT_FOUND
def test_compliance_overview_list_queryset(
self, authenticated_client, compliance_overviews_fixture
):
compliance_overview1, compliance_overview2 = compliance_overviews_fixture
scan_id = str(compliance_overview1.scan.id)
response = authenticated_client.get(
reverse("complianceoverview-list"),
{"filter[scan_id]": scan_id},
)
# No filters, most fails should be returned
assert len(response.json()["data"]) == 1
assert response.json()["data"][0]["id"] == str(compliance_overview2.id)
compliance_overview1.requirements_failed = 5
compliance_overview1.save()
response = authenticated_client.get(
reverse("complianceoverview-list"),
{"filter[scan_id]": scan_id},
)
# No filters, now compliance_overview1 has more fails
assert len(response.json()["data"]) == 1
assert response.json()["data"][0]["id"] == str(compliance_overview1.id)
def test_compliance_overview_metadata(
self, authenticated_client, compliance_overviews_fixture
):
response = authenticated_client.get(
reverse("complianceoverview-metadata"),
{"filter[scan_id]": str(compliance_overviews_fixture[0].scan_id)},
)
data = response.json()
expected_regions = set(
ComplianceOverview.objects.all()
.values_list("region", flat=True)
.distinct("region")
query_params,
)
assert response.status_code == status.HTTP_200_OK
assert data["data"]["type"] == "compliance-overviews-metadata"
assert data["data"]["id"] is None
assert set(data["data"]["attributes"]["regions"]) == expected_regions
response_data = response.json()
def test_compliance_overview_metadata_missing_scan_id(self, authenticated_client):
# Attempt to list compliance overviews without providing filter[scan_id]
response = authenticated_client.get(reverse("complianceoverview-metadata"))
assert response.status_code == status.HTTP_400_BAD_REQUEST
assert response.json()["errors"][0]["source"]["pointer"] == "filter[scan_id]"
assert response.json()["errors"][0]["code"] == "required"
assert len(response_data["data"]) >= expected_count_min
if response_data["data"]:
first_item = response_data["data"][0]
assert "id" in first_item
assert "type" in first_item
assert first_item["type"] == "compliance-overviews"
assert "attributes" in first_item
attributes = first_item["attributes"]
assert "framework" in attributes
assert "version" in attributes
assert "requirements_passed" in attributes
assert "requirements_failed" in attributes
assert "requirements_manual" in attributes
assert "total_requirements" in attributes
if filter_name == "compliance_id":
assert first_item["id"] == filter_value
elif filter_name == "framework":
assert attributes["framework"] == filter_value
elif filter_name == "version":
assert attributes["version"] == filter_value
@pytest.mark.django_db
@@ -5307,3 +5530,264 @@ class TestIntegrationViewSet:
{f"filter[{filter_name}]": "whatever"},
)
assert response.status_code == status.HTTP_400_BAD_REQUEST
@pytest.mark.django_db
class TestSAMLInitiateAPIView:
def test_valid_email_domain_and_certificates(
self, authenticated_client, saml_setup, monkeypatch
):
monkeypatch.setenv("SAML_PUBLIC_CERT", "fake_cert")
monkeypatch.setenv("SAML_PRIVATE_KEY", "fake_key")
url = reverse("api_saml_initiate")
payload = {"email_domain": saml_setup["email"]}
response = authenticated_client.post(url, data=payload, format="json")
assert response.status_code == status.HTTP_302_FOUND
assert f"email={saml_setup['email']}" in response.url
assert (
reverse("saml_login", kwargs={"organization_slug": saml_setup["domain"]})
in response.url
)
def test_invalid_email_domain(self, authenticated_client):
url = reverse("api_saml_initiate")
payload = {"email_domain": "user@unauthorized.com"}
response = authenticated_client.post(url, data=payload, format="json")
assert response.status_code == status.HTTP_403_FORBIDDEN
assert response.json()["errors"]["detail"] == "Unauthorized domain."
def test_missing_certificates(self, authenticated_client, saml_setup, monkeypatch):
monkeypatch.setenv("SAML_PUBLIC_CERT", "")
monkeypatch.setenv("SAML_PRIVATE_KEY", "")
url = reverse("api_saml_initiate")
payload = {"email_domain": saml_setup["email"]}
response = authenticated_client.post(url, data=payload, format="json")
assert response.status_code == status.HTTP_403_FORBIDDEN
assert (
response.json()["errors"]["detail"]
== "SAML configuration is invalid: missing certificates."
)
@pytest.mark.django_db
class TestSAMLConfigurationViewSet:
def test_list_saml_configurations(self, authenticated_client, saml_setup):
config = SAMLConfiguration.objects.get(
email_domain=saml_setup["email"].split("@")[-1]
)
response = authenticated_client.get(reverse("saml-config-list"))
assert response.status_code == status.HTTP_200_OK
assert (
response.json()["data"][0]["attributes"]["email_domain"]
== config.email_domain
)
def test_retrieve_saml_configuration(self, authenticated_client, saml_setup):
config = SAMLConfiguration.objects.get(
email_domain=saml_setup["email"].split("@")[-1]
)
response = authenticated_client.get(
reverse("saml-config-detail", kwargs={"pk": config.id})
)
assert response.status_code == status.HTTP_200_OK
assert (
response.json()["data"]["attributes"]["metadata_xml"] == config.metadata_xml
)
def test_create_saml_configuration(self, authenticated_client, tenants_fixture):
payload = {
"email_domain": "newdomain.com",
"metadata_xml": """<?xml version='1.0' encoding='UTF-8'?>
<md:EntityDescriptor entityID='TEST' xmlns:md='urn:oasis:names:tc:SAML:2.0:metadata'>
<md:IDPSSODescriptor WantAuthnRequestsSigned='false' protocolSupportEnumeration='urn:oasis:names:tc:SAML:2.0:protocol'>
<md:KeyDescriptor use='signing'>
<ds:KeyInfo xmlns:ds='http://www.w3.org/2000/09/xmldsig#'>
<ds:X509Data>
<ds:X509Certificate>TEST</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
<md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST' Location='https://TEST/sso/saml'/>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect' Location='https://TEST/sso/saml'/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>
""",
}
response = authenticated_client.post(
reverse("saml-config-list"), data=payload, format="json"
)
assert response.status_code == status.HTTP_201_CREATED
assert SAMLConfiguration.objects.filter(email_domain="newdomain.com").exists()
def test_update_saml_configuration(self, authenticated_client, saml_setup):
config = SAMLConfiguration.objects.get(
email_domain=saml_setup["email"].split("@")[-1]
)
payload = {
"data": {
"type": "saml-configurations",
"id": str(config.id),
"attributes": {
"metadata_xml": """<?xml version='1.0' encoding='UTF-8'?>
<md:EntityDescriptor entityID='TEST' xmlns:md='urn:oasis:names:tc:SAML:2.0:metadata'>
<md:IDPSSODescriptor WantAuthnRequestsSigned='false' protocolSupportEnumeration='urn:oasis:names:tc:SAML:2.0:protocol'>
<md:KeyDescriptor use='signing'>
<ds:KeyInfo xmlns:ds='http://www.w3.org/2000/09/xmldsig#'>
<ds:X509Data>
<ds:X509Certificate>TEST2</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
<md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST' Location='https://TEST/sso/saml'/>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect' Location='https://TEST/sso/saml'/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>
"""
},
}
}
response = authenticated_client.patch(
reverse("saml-config-detail", kwargs={"pk": config.id}),
data=payload,
content_type="application/vnd.api+json",
)
assert response.status_code == status.HTTP_200_OK
config.refresh_from_db()
assert (
config.metadata_xml.strip()
== payload["data"]["attributes"]["metadata_xml"].strip()
)
def test_delete_saml_configuration(self, authenticated_client, saml_setup):
config = SAMLConfiguration.objects.get(
email_domain=saml_setup["email"].split("@")[-1]
)
response = authenticated_client.delete(
reverse("saml-config-detail", kwargs={"pk": config.id})
)
assert response.status_code == status.HTTP_204_NO_CONTENT
assert not SAMLConfiguration.objects.filter(id=config.id).exists()
@pytest.mark.django_db
class TestTenantFinishACSView:
def test_dispatch_skips_if_user_not_authenticated(self):
request = RequestFactory().get(
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
)
request.user = type("Anonymous", (), {"is_authenticated": False})()
with patch(
"allauth.socialaccount.providers.saml.views.get_app_or_404"
) as mock_get_app:
mock_get_app.return_value = SocialApp(
provider="saml",
client_id="testtenant",
name="Test App",
settings={},
)
view = TenantFinishACSView.as_view()
response = view(request, organization_slug="testtenant")
assert response.status_code in [200, 302]
def test_dispatch_skips_if_social_app_not_found(self, users_fixture):
request = RequestFactory().get(
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
)
request.user = users_fixture[0]
with patch(
"allauth.socialaccount.providers.saml.views.get_app_or_404"
) as mock_get_app:
mock_get_app.return_value = SocialApp(
provider="saml",
client_id="testtenant",
name="Test App",
settings={},
)
view = TenantFinishACSView.as_view()
response = view(request, organization_slug="testtenant")
assert isinstance(response, JsonResponse) or response.status_code in [200, 302]
def test_dispatch_sets_user_profile_and_assigns_role(
self, create_test_user, tenants_fixture, saml_setup
):
user = create_test_user
original_email = user.email
original_name = user.name
original_company = user.company_name
user.email = f"doe@{saml_setup['email']}"
social_account = SocialAccount(
user=user,
provider="saml",
extra_data={
"firstName": ["John"],
"lastName": ["Doe"],
"organization": ["TestOrg"],
"userType": ["saml_default_role"],
},
)
request = RequestFactory().get(
reverse("saml_finish_acs", kwargs={"organization_slug": "testtenant"})
)
request.user = user
with (
patch(
"allauth.socialaccount.providers.saml.views.get_app_or_404"
) as mock_get_app_or_404,
patch("allauth.socialaccount.models.SocialApp.objects.get"),
patch(
"allauth.socialaccount.models.SocialAccount.objects.get"
) as mock_socialaccount_get,
patch("api.v1.serializers.TokenSocialLoginSerializer") as mock_serializer,
):
mock_get_app_or_404.return_value = MagicMock(
provider="saml", client_id="testtenant", name="Test App", settings={}
)
mock_socialaccount_get.return_value = social_account
mock_instance = mock_serializer.return_value
mock_instance.is_valid.return_value = True
mock_instance.validated_data = {
"token": "mocktoken",
"refresh_token": "mockrefresh",
}
view = TenantFinishACSView.as_view()
response = view(request, organization_slug="testtenant")
assert response.status_code == 200
user.refresh_from_db()
assert user.name == "John Doe"
assert user.company_name == "TestOrg"
role = Role.objects.using(MainRouter.admin_db).get(name="saml_default_role")
assert role.tenant == tenants_fixture[0]
assert (
UserRoleRelationship.objects.using(MainRouter.admin_db)
.filter(user=user, tenant_id=tenants_fixture[0].id)
.exists()
)
user.email = original_email
user.name = original_name
user.company_name = original_company
user.save()
+1 -1
View File
@@ -186,7 +186,7 @@ def validate_invitation(
try:
# Admin DB connector is used to bypass RLS protection since the invitation belongs to a tenant the user
# is not a member of yet
invitation = Invitation.objects.using(MainRouter.admin_db).get(
invitation = Invitation.objects.using(MainRouter.admin_read).get(
token=invitation_token, email=email
)
except Invitation.DoesNotExist:
+222
View File
@@ -0,0 +1,222 @@
from django.urls import reverse
from django_celery_results.models import TaskResult
from rest_framework import status
from rest_framework.response import Response
from api.exceptions import (
TaskFailedException,
TaskInProgressException,
TaskNotFoundException,
)
from api.models import StateChoices, Task
from api.v1.serializers import TaskSerializer
class PaginateByPkMixin:
"""
Mixin to paginate on a list of PKs (cheaper than heavy JOINs),
re-fetch the full objects with the desired select/prefetch,
re-sort them to preserve DB ordering, then serialize + return.
"""
def paginate_by_pk(
self,
request, # noqa: F841
base_queryset,
manager,
select_related: list[str] | None = None,
prefetch_related: list[str] | None = None,
) -> Response:
pk_list = base_queryset.values_list("id", flat=True)
page = self.paginate_queryset(pk_list)
if page is None:
return Response(self.get_serializer(base_queryset, many=True).data)
queryset = manager.filter(id__in=page)
if select_related:
queryset = queryset.select_related(*select_related)
if prefetch_related:
queryset = queryset.prefetch_related(*prefetch_related)
queryset = sorted(queryset, key=lambda obj: page.index(obj.id))
serialized = self.get_serializer(queryset, many=True).data
return self.get_paginated_response(serialized)
class TaskManagementMixin:
"""
Mixin to manage task status checking.
This mixin provides functionality to check if a task with specific parameters
is running, completed, failed, or doesn't exist. It returns the task when running
and raises specific exceptions for failed/not found scenarios that can be handled
at the view level.
"""
def check_task_status(
self,
task_name: str,
task_kwargs: dict,
raise_on_failed: bool = True,
raise_on_not_found: bool = True,
) -> Task | None:
"""
Check the status of a task with given name and kwargs.
This method first checks for a related Task object, and if not found,
checks TaskResult directly. If a TaskResult is found and running but
there's no related Task, it raises TaskInProgressException.
Args:
task_name (str): The name of the task to check
task_kwargs (dict): The kwargs to match against the task
raise_on_failed (bool): Whether to raise exception if task failed
raise_on_not_found (bool): Whether to raise exception if task not found
Returns:
Task | None: The task instance if found (regardless of state), None if not found and raise_on_not_found=False
Raises:
TaskFailedException: If task failed and raise_on_failed=True
TaskNotFoundException: If task not found and raise_on_not_found=True
TaskInProgressException: If task is running but no related Task object exists
"""
# First, try to find a Task object with related TaskResult
try:
# Build the filter for task kwargs
task_filter = {
"task_runner_task__task_name": task_name,
}
# Add kwargs filters - we need to check if the task kwargs contain our parameters
for key, value in task_kwargs.items():
task_filter["task_runner_task__task_kwargs__contains"] = str(value)
task = (
Task.objects.filter(**task_filter)
.select_related("task_runner_task")
.order_by("-inserted_at")
.first()
)
if task:
# Get task state using the same logic as TaskSerializer
task_state_mapping = {
"PENDING": StateChoices.AVAILABLE,
"STARTED": StateChoices.EXECUTING,
"PROGRESS": StateChoices.EXECUTING,
"SUCCESS": StateChoices.COMPLETED,
"FAILURE": StateChoices.FAILED,
"REVOKED": StateChoices.CANCELLED,
}
celery_status = (
task.task_runner_task.status if task.task_runner_task else None
)
task_state = task_state_mapping.get(
celery_status or "", StateChoices.AVAILABLE
)
# Check task state and raise exceptions accordingly
if task_state in (StateChoices.FAILED, StateChoices.CANCELLED):
if raise_on_failed:
raise TaskFailedException(task=task)
return task
elif task_state == StateChoices.COMPLETED:
return None
return task
except Task.DoesNotExist:
pass
# If no Task found, check TaskResult directly
try:
# Build the filter for TaskResult
task_result_filter = {
"task_name": task_name,
}
# Add kwargs filters - check if the task kwargs contain our parameters
for key, value in task_kwargs.items():
task_result_filter["task_kwargs__contains"] = str(value)
task_result = (
TaskResult.objects.filter(**task_result_filter)
.order_by("-date_created")
.first()
)
if task_result:
# Check if the TaskResult indicates a running task
if task_result.status in ["PENDING", "STARTED", "PROGRESS"]:
# Task is running but no related Task object exists
raise TaskInProgressException(task_result=task_result)
elif task_result.status == "FAILURE":
if raise_on_failed:
raise TaskFailedException(task=None)
# For other statuses (SUCCESS, REVOKED), we don't have a Task to return,
# so we treat it as not found
except TaskResult.DoesNotExist:
pass
# No task found at all
if raise_on_not_found:
raise TaskNotFoundException()
return None
def get_task_response_if_running(
self,
task_name: str,
task_kwargs: dict,
raise_on_failed: bool = True,
raise_on_not_found: bool = True,
) -> Response | None:
"""
Get a 202 response with task details if the task is currently running.
This method is useful for endpoints that should return task status when
a background task is in progress, similar to the compliance overview endpoints.
Args:
task_name (str): The name of the task to check
task_kwargs (dict): The kwargs to match against the task
Returns:
Response | None: 202 response with task details if running, None otherwise
"""
task = self.check_task_status(
task_name=task_name,
task_kwargs=task_kwargs,
raise_on_failed=raise_on_failed,
raise_on_not_found=raise_on_not_found,
)
if not task:
return None
# Get task state
task_state_mapping = {
"PENDING": StateChoices.AVAILABLE,
"STARTED": StateChoices.EXECUTING,
"PROGRESS": StateChoices.EXECUTING,
"SUCCESS": StateChoices.COMPLETED,
"FAILURE": StateChoices.FAILED,
"REVOKED": StateChoices.CANCELLED,
}
celery_status = task.task_runner_task.status if task.task_runner_task else None
task_state = task_state_mapping.get(celery_status or "", StateChoices.AVAILABLE)
if task_state == StateChoices.EXECUTING:
self.response_serializer_class = TaskSerializer
serializer = TaskSerializer(task)
return Response(
data=serializer.data,
status=status.HTTP_202_ACCEPTED,
headers={
"Content-Location": reverse("task-detail", kwargs={"pk": task.id})
},
)
@@ -119,9 +119,9 @@ from rest_framework_json_api import serializers
"type": "email",
"description": "User microsoft email address.",
},
"encrypted_password": {
"password": {
"type": "string",
"description": "User encrypted password.",
"description": "User password.",
},
},
"required": [
@@ -129,7 +129,7 @@ from rest_framework_json_api import serializers
"client_secret",
"tenant_id",
"user",
"encrypted_password",
"password",
],
},
{
@@ -154,6 +154,17 @@ from rest_framework_json_api import serializers
},
"required": ["client_id", "client_secret", "refresh_token"],
},
{
"type": "object",
"title": "GCP Service Account Key",
"properties": {
"service_account_key": {
"type": "object",
"description": "The service account key for GCP.",
}
},
"required": ["service_account_key"],
},
{
"type": "object",
"title": "Kubernetes Static Credentials",
+76 -113
View File
@@ -14,7 +14,6 @@ from rest_framework_simplejwt.serializers import TokenObtainPairSerializer
from rest_framework_simplejwt.tokens import RefreshToken
from api.models import (
ComplianceOverview,
Finding,
Integration,
IntegrationProviderRelationship,
@@ -29,8 +28,10 @@ from api.models import (
ResourceTag,
Role,
RoleProviderGroupRelationship,
SAMLConfiguration,
Scan,
StateChoices,
StatusChoices,
Task,
User,
UserRoleRelationship,
@@ -1159,6 +1160,8 @@ class BaseWriteProviderSecretSerializer(BaseWriteSerializer):
)
elif secret_type == ProviderSecret.TypeChoices.ROLE:
serializer = AWSRoleAssumptionProviderSecret(data=secret)
elif secret_type == ProviderSecret.TypeChoices.SERVICE_ACCOUNT:
serializer = GCPServiceAccountProviderSecret(data=secret)
else:
raise serializers.ValidationError(
{"secret_type": f"Secret type not supported: {secret_type}"}
@@ -1197,7 +1200,7 @@ class M365ProviderSecret(serializers.Serializer):
client_secret = serializers.CharField()
tenant_id = serializers.CharField()
user = serializers.EmailField()
encrypted_password = serializers.CharField()
password = serializers.CharField()
class Meta:
resource_name = "provider-secrets"
@@ -1212,6 +1215,13 @@ class GCPProviderSecret(serializers.Serializer):
resource_name = "provider-secrets"
class GCPServiceAccountProviderSecret(serializers.Serializer):
service_account_key = serializers.JSONField()
class Meta:
resource_name = "provider-secrets"
class KubernetesProviderSecret(serializers.Serializer):
kubeconfig_content = serializers.CharField()
@@ -1670,130 +1680,61 @@ class RoleProviderGroupRelationshipSerializer(RLSSerializer, BaseWriteSerializer
# Compliance overview
class ComplianceOverviewSerializer(RLSSerializer):
class ComplianceOverviewSerializer(serializers.Serializer):
"""
Serializer for the ComplianceOverview model.
Serializer for compliance requirement status aggregated by compliance framework.
This serializer is used to format aggregated compliance framework data,
providing counts of passed, failed, and manual requirements along with
an overall global status for each framework.
"""
requirements_status = serializers.SerializerMethodField(
read_only=True, method_name="get_requirements_status"
)
provider_type = serializers.SerializerMethodField(read_only=True)
# Add ID field which will be used for resource identification
id = serializers.CharField()
framework = serializers.CharField()
version = serializers.CharField()
requirements_passed = serializers.IntegerField()
requirements_failed = serializers.IntegerField()
requirements_manual = serializers.IntegerField()
total_requirements = serializers.IntegerField()
class Meta:
model = ComplianceOverview
fields = [
"id",
"inserted_at",
"compliance_id",
"framework",
"version",
"requirements_status",
"region",
"provider_type",
"scan",
"url",
]
@extend_schema_field(
{
"type": "object",
"properties": {
"passed": {"type": "integer"},
"failed": {"type": "integer"},
"manual": {"type": "integer"},
"total": {"type": "integer"},
},
}
)
def get_requirements_status(self, obj):
return {
"passed": obj.requirements_passed,
"failed": obj.requirements_failed,
"manual": obj.requirements_manual,
"total": obj.total_requirements,
}
@extend_schema_field(serializers.CharField(allow_null=True))
def get_provider_type(self, obj):
"""
Retrieves the provider_type from scan.provider.provider_type.
"""
try:
return obj.scan.provider.provider
except AttributeError:
return None
class JSONAPIMeta:
resource_name = "compliance-overviews"
class ComplianceOverviewFullSerializer(ComplianceOverviewSerializer):
requirements = serializers.SerializerMethodField(read_only=True)
class ComplianceOverviewDetailSerializer(serializers.Serializer):
"""
Serializer for detailed compliance requirement information.
class Meta(ComplianceOverviewSerializer.Meta):
fields = ComplianceOverviewSerializer.Meta.fields + [
"description",
"requirements",
]
This serializer formats the aggregated requirement data, showing detailed status
and counts for each requirement across all regions.
"""
@extend_schema_field(
{
"type": "object",
"properties": {
"requirement_id": {
"type": "object",
"properties": {
"name": {"type": "string"},
"checks": {
"type": "object",
"properties": {
"check_name": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": ["PASS", "FAIL", None],
},
},
}
},
"description": "Each key in the 'checks' object is a check name, with values as "
"'PASS', 'FAIL', or null.",
},
"status": {
"type": "string",
"enum": ["PASS", "FAIL", "MANUAL"],
},
"attributes": {
"type": "array",
"items": {
"type": "object",
},
},
"description": {"type": "string"},
"checks_status": {
"type": "object",
"properties": {
"total": {"type": "integer"},
"pass": {"type": "integer"},
"fail": {"type": "integer"},
"manual": {"type": "integer"},
},
},
},
}
},
}
)
def get_requirements(self, obj):
"""
Returns the detailed structure of requirements.
"""
return obj.requirements
id = serializers.CharField()
framework = serializers.CharField()
version = serializers.CharField()
description = serializers.CharField()
status = serializers.ChoiceField(choices=StatusChoices.choices)
class JSONAPIMeta:
resource_name = "compliance-requirements-details"
class ComplianceOverviewAttributesSerializer(serializers.Serializer):
id = serializers.CharField()
framework = serializers.CharField()
version = serializers.CharField()
description = serializers.CharField()
attributes = serializers.JSONField()
class JSONAPIMeta:
resource_name = "compliance-requirements-attributes"
class ComplianceOverviewMetadataSerializer(serializers.Serializer):
regions = serializers.ListField(child=serializers.CharField(), allow_empty=True)
class Meta:
class JSONAPIMeta:
resource_name = "compliance-overviews-metadata"
@@ -2119,3 +2060,25 @@ class IntegrationUpdateSerializer(BaseWriteIntegrationSerializer):
IntegrationProviderRelationship.objects.bulk_create(new_relationships)
return super().update(instance, validated_data)
# SSO
class SamlInitiateSerializer(serializers.Serializer):
email_domain = serializers.CharField()
class JSONAPIMeta:
resource_name = "saml-initiate"
class SamlMetadataSerializer(serializers.Serializer):
class JSONAPIMeta:
resource_name = "saml-meta"
class SAMLConfigurationSerializer(RLSSerializer):
class Meta:
model = SAMLConfiguration
fields = ["id", "email_domain", "metadata_xml", "created_at", "updated_at"]
read_only_fields = ["id", "created_at", "updated_at"]
+15
View File
@@ -22,10 +22,13 @@ from api.v1.views import (
ResourceViewSet,
RoleProviderGroupRelationshipView,
RoleViewSet,
SAMLConfigurationViewSet,
SAMLInitiateAPIView,
ScanViewSet,
ScheduleViewSet,
SchemaView,
TaskViewSet,
TenantFinishACSView,
TenantMembersViewSet,
TenantViewSet,
UserRoleRelationshipView,
@@ -49,6 +52,7 @@ router.register(
router.register(r"overviews", OverviewViewSet, basename="overview")
router.register(r"schedules", ScheduleViewSet, basename="schedule")
router.register(r"integrations", IntegrationViewSet, basename="integration")
router.register(r"saml-config", SAMLConfigurationViewSet, basename="saml-config")
tenants_router = routers.NestedSimpleRouter(router, r"tenants", lookup="tenant")
tenants_router.register(
@@ -112,6 +116,17 @@ urlpatterns = [
),
name="provider_group-providers-relationship",
),
# API endpoint to start SAML SSO flow
path(
"auth/saml/initiate/", SAMLInitiateAPIView.as_view(), name="api_saml_initiate"
),
# Allauth SAML endpoints for tenants
path("accounts/", include("allauth.urls")),
path(
"api/v1/accounts/saml/<organization_slug>/acs/finish/",
TenantFinishACSView.as_view(),
name="saml_finish_acs",
),
path("tokens/google", GoogleSocialLoginView.as_view(), name="token-google"),
path("tokens/github", GithubSocialLoginView.as_view(), name="token-github"),
path("", include(router.urls)),
+675 -75
View File
@@ -3,8 +3,10 @@ import os
from datetime import datetime, timedelta, timezone
import sentry_sdk
from allauth.socialaccount.models import SocialAccount, SocialApp
from allauth.socialaccount.providers.github.views import GitHubOAuth2Adapter
from allauth.socialaccount.providers.google.views import GoogleOAuth2Adapter
from allauth.socialaccount.providers.saml.views import FinishACSView
from botocore.exceptions import ClientError, NoCredentialsError, ParamValidationError
from celery.result import AsyncResult
from config.env import env
@@ -17,19 +19,20 @@ from django.conf import settings as django_settings
from django.contrib.postgres.aggregates import ArrayAgg
from django.contrib.postgres.search import SearchQuery
from django.db import transaction
from django.db.models import Count, Exists, F, OuterRef, Prefetch, Q, Subquery, Sum
from django.db.models import Count, Exists, F, OuterRef, Prefetch, Q, Sum
from django.db.models.functions import Coalesce
from django.http import HttpResponse
from django.http import HttpResponse, JsonResponse
from django.shortcuts import redirect
from django.urls import reverse
from django.utils.dateparse import parse_date
from django.utils.decorators import method_decorator
from django.views.decorators.cache import cache_control
from django_celery_beat.models import PeriodicTask
from drf_spectacular.settings import spectacular_settings
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import (
OpenApiParameter,
OpenApiResponse,
OpenApiTypes,
extend_schema,
extend_schema_view,
)
@@ -58,13 +61,19 @@ from tasks.tasks import (
)
from api.base_views import BaseRLSViewSet, BaseTenantViewset, BaseUserViewset
from api.compliance import get_compliance_frameworks
from api.compliance import (
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
get_compliance_frameworks,
)
from api.db_router import MainRouter
from api.db_utils import rls_transaction
from api.exceptions import TaskFailedException
from api.filters import (
ComplianceOverviewFilter,
FindingFilter,
IntegrationFilter,
InvitationFilter,
LatestFindingFilter,
MembershipFilter,
ProviderFilter,
ProviderGroupFilter,
@@ -80,6 +89,7 @@ from api.filters import (
)
from api.models import (
ComplianceOverview,
ComplianceRequirementOverview,
Finding,
Integration,
Invitation,
@@ -93,6 +103,8 @@ from api.models import (
ResourceScanSummary,
Role,
RoleProviderGroupRelationship,
SAMLConfiguration,
SAMLDomainIndex,
Scan,
ScanSummary,
SeverityChoices,
@@ -110,8 +122,10 @@ from api.utils import (
validate_invitation,
)
from api.uuid_utils import datetime_to_uuid7, uuid7_start
from api.v1.mixins import PaginateByPkMixin, TaskManagementMixin
from api.v1.serializers import (
ComplianceOverviewFullSerializer,
ComplianceOverviewAttributesSerializer,
ComplianceOverviewDetailSerializer,
ComplianceOverviewMetadataSerializer,
ComplianceOverviewSerializer,
FindingDynamicFilterSerializer,
@@ -144,6 +158,8 @@ from api.v1.serializers import (
RoleProviderGroupRelationshipSerializer,
RoleSerializer,
RoleUpdateSerializer,
SAMLConfigurationSerializer,
SamlInitiateSerializer,
ScanComplianceReportSerializer,
ScanCreateSerializer,
ScanReportSerializer,
@@ -258,7 +274,7 @@ class SchemaView(SpectacularAPIView):
def get(self, request, *args, **kwargs):
spectacular_settings.TITLE = "Prowler API"
spectacular_settings.VERSION = "1.8.0"
spectacular_settings.VERSION = "1.9.0"
spectacular_settings.DESCRIPTION = (
"Prowler API specification.\n\nThis file is auto-generated."
)
@@ -375,6 +391,163 @@ class GithubSocialLoginView(SocialLoginView):
return original_response
@extend_schema(exclude=True)
class SAMLInitiateAPIView(GenericAPIView):
serializer_class = SamlInitiateSerializer
permission_classes = []
def post(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
email = serializer.validated_data["email_domain"]
domain = email.split("@", 1)[-1].lower()
try:
check = SAMLDomainIndex.objects.get(email_domain=domain)
with rls_transaction(str(check.tenant_id)):
config = SAMLConfiguration.objects.get(tenant_id=str(check.tenant_id))
except (SAMLDomainIndex.DoesNotExist, SAMLConfiguration.DoesNotExist):
return Response(
{"detail": "Unauthorized domain."}, status=status.HTTP_403_FORBIDDEN
)
# Check certificates are not empty
saml_public_cert = os.getenv("SAML_PUBLIC_CERT", "").strip()
saml_private_key = os.getenv("SAML_PRIVATE_KEY", "").strip()
if not saml_public_cert or not saml_private_key:
return Response(
{"detail": "SAML configuration is invalid: missing certificates."},
status=status.HTTP_403_FORBIDDEN,
)
saml_login_url = reverse(
"saml_login", kwargs={"organization_slug": config.email_domain}
)
return redirect(f"{saml_login_url}?email={email}")
@extend_schema_view(
list=extend_schema(
tags=["SAML"],
summary="List all SSO configurations",
description="Returns all the SAML-based SSO configurations associated with the current tenant.",
),
retrieve=extend_schema(
tags=["SAML"],
summary="Retrieve SSO configuration details",
description="Returns the details of a specific SAML configuration belonging to the current tenant.",
),
create=extend_schema(
tags=["SAML"],
summary="Create the SSO configuration",
description="Creates a new SAML SSO configuration for the current tenant, including email domain and metadata XML.",
),
partial_update=extend_schema(
tags=["SAML"],
summary="Update the SSO configuration",
description="Partially updates an existing SAML SSO configuration. Supports changes to email domain and metadata XML.",
),
destroy=extend_schema(
tags=["SAML"],
summary="Delete the SSO configuration",
description="Deletes an existing SAML SSO configuration associated with the current tenant.",
),
)
@method_decorator(CACHE_DECORATOR, name="retrieve")
@method_decorator(CACHE_DECORATOR, name="list")
class SAMLConfigurationViewSet(BaseRLSViewSet):
"""
ViewSet for managing SAML SSO configurations per tenant.
This endpoint allows authorized users to perform CRUD operations on SAMLConfiguration,
which define how a tenant integrates with an external SAML Identity Provider (IdP).
Typical use cases include:
- Listing all existing configurations for auditing or UI display.
- Retrieving a single configuration to show setup details.
- Creating or updating a configuration to onboard or modify SAML integration.
- Deleting a configuration when deactivating SAML for a tenant.
"""
serializer_class = SAMLConfigurationSerializer
required_permissions = [Permissions.MANAGE_INTEGRATIONS]
queryset = SAMLConfiguration.objects.all()
def get_queryset(self):
# If called during schema generation, return an empty queryset
if getattr(self, "swagger_fake_view", False):
return SAMLConfiguration.objects.none()
return SAMLConfiguration.objects.filter(tenant=self.request.tenant_id)
class TenantFinishACSView(FinishACSView):
def dispatch(self, request, organization_slug):
response = super().dispatch(request, organization_slug)
user = getattr(request, "user", None)
if not user or not user.is_authenticated:
return response
try:
social_app = SocialApp.objects.get(
provider="saml", client_id=organization_slug
)
social_account = SocialAccount.objects.get(
user=user, provider=social_app.provider
)
except (SocialApp.DoesNotExist, SocialAccount.DoesNotExist):
return response
extra = social_account.extra_data
user.first_name = extra.get("firstName", [""])[0]
user.last_name = extra.get("lastName", [""])[0]
user.company_name = extra.get("organization", [""])[0]
user.name = f"{user.first_name} {user.last_name}".strip()
user.save()
email_domain = user.email.split("@")[-1]
tenant = (
SAMLConfiguration.objects.using(MainRouter.admin_read)
.get(email_domain=email_domain)
.tenant
)
role_name = extra.get("userType", ["saml_default_role"])[0].strip()
try:
role = Role.objects.using(MainRouter.admin_read).get(
name=role_name, tenant=tenant
)
except Role.DoesNotExist:
role = Role.objects.using(MainRouter.admin_db).create(
name=role_name,
tenant=tenant,
manage_users=False,
manage_account=False,
manage_billing=False,
manage_providers=False,
manage_integrations=False,
manage_scans=False,
unlimited_visibility=False,
)
UserRoleRelationship.objects.using(MainRouter.admin_db).filter(
user=user,
tenant_id=tenant.id,
).delete()
UserRoleRelationship.objects.using(MainRouter.admin_db).create(
user=user,
role=role,
tenant_id=tenant.id,
)
serializer = TokenSocialLoginSerializer(data={"email": user.email})
serializer.is_valid(raise_exception=True)
return JsonResponse(
{
"type": "saml-social-tokens",
"attributes": serializer.validated_data,
}
)
@extend_schema_view(
list=extend_schema(
tags=["User"],
@@ -1158,7 +1331,9 @@ class ProviderViewSet(BaseRLSViewSet):
200: OpenApiResponse(description="Report obtained successfully"),
202: OpenApiResponse(description="The task is in progress"),
403: OpenApiResponse(description="There is a problem with credentials"),
404: OpenApiResponse(description="The scan has no reports"),
404: OpenApiResponse(
description="The scan has no reports, or the report generation task has not started yet"
),
},
),
compliance=extend_schema(
@@ -1279,7 +1454,7 @@ class ScanViewSet(BaseRLSViewSet):
try:
task = Task.objects.get(
task_runner_task__task_name="scan-report",
task_runner_task__task_args__contains=str(scan_instance.id),
task_runner_task__task_kwargs__contains=str(scan_instance.id),
)
except Task.DoesNotExist:
return None
@@ -1361,7 +1536,9 @@ class ScanViewSet(BaseRLSViewSet):
code = e.response.get("Error", {}).get("Code")
if code == "NoSuchKey":
return Response(
{"detail": "The scan has no reports."},
{
"detail": "The scan has no reports, or the report generation task has not started yet."
},
status=status.HTTP_404_NOT_FOUND,
)
return Response(
@@ -1374,7 +1551,9 @@ class ScanViewSet(BaseRLSViewSet):
files = glob.glob(path_pattern)
if not files:
return Response(
{"detail": "The scan has no reports."},
{
"detail": "The scan has no reports, or the report generation task has not started yet."
},
status=status.HTTP_404_NOT_FOUND,
)
filepath = files[0]
@@ -1400,7 +1579,10 @@ class ScanViewSet(BaseRLSViewSet):
if not scan.output_location:
return Response(
{"detail": "The scan has no reports."}, status=status.HTTP_404_NOT_FOUND
{
"detail": "The scan has no reports, or the report generation task has not started yet."
},
status=status.HTTP_404_NOT_FOUND,
)
if scan.output_location.startswith("s3://"):
@@ -1438,7 +1620,10 @@ class ScanViewSet(BaseRLSViewSet):
if not scan.output_location:
return Response(
{"detail": "The scan has no reports."}, status=status.HTTP_404_NOT_FOUND
{
"detail": "The scan has no reports, or the report generation task has not started yet."
},
status=status.HTTP_404_NOT_FOUND,
)
if scan.output_location.startswith("s3://"):
@@ -1475,10 +1660,10 @@ class ScanViewSet(BaseRLSViewSet):
},
)
prowler_task = Task.objects.get(id=task.id)
scan.task_id = task.id
scan.save(update_fields=["task_id"])
prowler_task = Task.objects.get(id=task.id)
self.response_serializer_class = TaskSerializer
output_serializer = self.get_serializer(prowler_task)
@@ -1671,10 +1856,24 @@ class ResourceViewSet(BaseRLSViewSet):
],
filters=True,
),
latest=extend_schema(
tags=["Finding"],
summary="List the latest findings",
description="Retrieve a list of the latest findings from the latest scans for each provider with options for "
"filtering by various criteria.",
filters=True,
),
metadata_latest=extend_schema(
tags=["Finding"],
summary="Retrieve metadata values from the latest findings",
description="Fetch unique metadata values from a set of findings from the latest scans for each provider. "
"This is useful for dynamic filtering.",
filters=True,
),
)
@method_decorator(CACHE_DECORATOR, name="list")
@method_decorator(CACHE_DECORATOR, name="retrieve")
class FindingViewSet(BaseRLSViewSet):
class FindingViewSet(PaginateByPkMixin, BaseRLSViewSet):
queryset = Finding.all_objects.all()
serializer_class = FindingSerializer
filterset_class = FindingFilter
@@ -1706,11 +1905,16 @@ class FindingViewSet(BaseRLSViewSet):
def get_serializer_class(self):
if self.action == "findings_services_regions":
return FindingDynamicFilterSerializer
elif self.action == "metadata":
elif self.action in ["metadata", "metadata_latest"]:
return FindingMetadataSerializer
return super().get_serializer_class()
def get_filterset_class(self):
if self.action in ["latest", "metadata_latest"]:
return LatestFindingFilter
return FindingFilter
def get_queryset(self):
tenant_id = self.request.tenant_id
user_roles = get_role(self.request.user)
@@ -1750,21 +1954,14 @@ class FindingViewSet(BaseRLSViewSet):
return super().filter_queryset(queryset)
def list(self, request, *args, **kwargs):
base_qs = self.filter_queryset(self.get_queryset())
paginated_ids = self.paginate_queryset(base_qs.values_list("id", flat=True))
if paginated_ids is not None:
ids = list(paginated_ids)
findings = (
Finding.all_objects.filter(tenant_id=self.request.tenant_id, id__in=ids)
.select_related("scan")
.prefetch_related("resources")
)
# Re-sort in Python to preserve ordering:
findings = sorted(findings, key=lambda x: ids.index(x.id))
serializer = self.get_serializer(findings, many=True)
return self.get_paginated_response(serializer.data)
serializer = self.get_serializer(base_qs, many=True)
return Response(serializer.data)
filtered_queryset = self.filter_queryset(self.get_queryset())
return self.paginate_by_pk(
request,
filtered_queryset,
manager=Finding.all_objects,
select_related=["scan"],
prefetch_related=["resources"],
)
@action(detail=False, methods=["get"], url_name="findings_services_regions")
def findings_services_regions(self, request):
@@ -1897,6 +2094,107 @@ class FindingViewSet(BaseRLSViewSet):
serializer.is_valid(raise_exception=True)
return Response(serializer.data)
@action(detail=False, methods=["get"], url_name="latest")
def latest(self, request):
tenant_id = request.tenant_id
filtered_queryset = self.filter_queryset(self.get_queryset())
latest_scan_ids = (
Scan.all_objects.filter(tenant_id=tenant_id, state=StateChoices.COMPLETED)
.order_by("provider_id", "-inserted_at")
.distinct("provider_id")
.values_list("id", flat=True)
)
filtered_queryset = filtered_queryset.filter(
tenant_id=tenant_id, scan_id__in=latest_scan_ids
)
return self.paginate_by_pk(
request,
filtered_queryset,
manager=Finding.all_objects,
select_related=["scan"],
prefetch_related=["resources"],
)
@action(
detail=False,
methods=["get"],
url_name="metadata_latest",
url_path="metadata/latest",
)
def metadata_latest(self, request):
tenant_id = request.tenant_id
query_params = request.query_params
latest_scans_queryset = (
Scan.all_objects.filter(tenant_id=tenant_id, state=StateChoices.COMPLETED)
.order_by("provider_id", "-inserted_at")
.distinct("provider_id")
)
latest_scans_ids = list(latest_scans_queryset.values_list("id", flat=True))
queryset = ResourceScanSummary.objects.filter(
tenant_id=tenant_id,
scan_id__in=latest_scans_queryset.values_list("id", flat=True),
)
# ToRemove: Temporary fallback mechanism
present_ids = set(
ResourceScanSummary.objects.filter(
tenant_id=tenant_id, scan_id__in=latest_scans_ids
)
.values_list("scan_id", flat=True)
.distinct()
)
missing_scan_ids = [sid for sid in latest_scans_ids if sid not in present_ids]
if missing_scan_ids:
for scan_id in missing_scan_ids:
backfill_scan_resource_summaries_task.apply_async(
kwargs={"tenant_id": tenant_id, "scan_id": scan_id}
)
return Response(
get_findings_metadata_no_aggregations(
tenant_id, self.filter_queryset(self.get_queryset())
)
)
if service_filter := query_params.get("filter[service]") or query_params.get(
"filter[service__in]"
):
queryset = queryset.filter(service__in=service_filter.split(","))
if region_filter := query_params.get("filter[region]") or query_params.get(
"filter[region__in]"
):
queryset = queryset.filter(region__in=region_filter.split(","))
if resource_type_filter := query_params.get(
"filter[resource_type]"
) or query_params.get("filter[resource_type__in]"):
queryset = queryset.filter(
resource_type__in=resource_type_filter.split(",")
)
services = list(
queryset.values_list("service", flat=True).distinct().order_by("service")
)
regions = list(
queryset.values_list("region", flat=True).distinct().order_by("region")
)
resource_types = list(
queryset.values_list("resource_type", flat=True)
.distinct()
.order_by("resource_type")
)
result = {
"services": services,
"regions": regions,
"resource_types": resource_types,
}
serializer = self.get_serializer(data=result)
serializer.is_valid(raise_exception=True)
return Response(serializer.data)
@extend_schema_view(
list=extend_schema(
@@ -2082,7 +2380,7 @@ class InvitationAcceptViewSet(BaseRLSViewSet):
)
# Proceed with accepting the invitation
user = User.objects.using(MainRouter.admin_db).get(email=user_email)
user = User.objects.using(MainRouter.admin_read).get(email=user_email)
membership = Membership.objects.using(MainRouter.admin_db).create(
user=user,
tenant=invitation.tenant,
@@ -2264,8 +2562,7 @@ class RoleProviderGroupRelationshipView(RelationshipView, BaseRLSViewSet):
list=extend_schema(
tags=["Compliance Overview"],
summary="List compliance overviews for a scan",
description="Retrieve an overview of all the compliance in a given scan. If no region filters are provided, the"
" region with the most fails will be returned by default.",
description="Retrieve an overview of all the compliance in a given scan.",
parameters=[
OpenApiParameter(
name="filter[scan_id]",
@@ -2275,12 +2572,18 @@ class RoleProviderGroupRelationshipView(RelationshipView, BaseRLSViewSet):
description="Related scan ID.",
),
],
),
retrieve=extend_schema(
tags=["Compliance Overview"],
summary="Retrieve data from a specific compliance overview",
description="Fetch detailed information about a specific compliance overview by its ID, including detailed "
"requirement information and check's status.",
responses={
200: OpenApiResponse(
description="Compliance overviews obtained successfully",
response=ComplianceOverviewSerializer(many=True),
),
202: OpenApiResponse(
description="The task is in progress", response=TaskSerializer
),
500: OpenApiResponse(
description="Compliance overviews generation task failed"
),
},
),
metadata=extend_schema(
tags=["Compliance Overview"],
@@ -2296,19 +2599,84 @@ class RoleProviderGroupRelationshipView(RelationshipView, BaseRLSViewSet):
description="Related scan ID.",
),
],
responses={
200: OpenApiResponse(
description="Compliance overviews metadata obtained successfully",
response=ComplianceOverviewMetadataSerializer,
),
202: OpenApiResponse(description="The task is in progress"),
500: OpenApiResponse(
description="Compliance overviews generation task failed"
),
},
),
requirements=extend_schema(
tags=["Compliance Overview"],
summary="List compliance requirements overview for a scan",
description="Retrieve a detailed overview of compliance requirements in a given scan, grouped by compliance "
"framework. This endpoint provides requirement-level details and aggregates status across regions.",
parameters=[
OpenApiParameter(
name="filter[scan_id]",
required=True,
type=OpenApiTypes.UUID,
location=OpenApiParameter.QUERY,
description="Related scan ID.",
),
OpenApiParameter(
name="filter[compliance_id]",
required=True,
type=OpenApiTypes.STR,
location=OpenApiParameter.QUERY,
description="Compliance ID.",
),
],
responses={
200: OpenApiResponse(
description="Compliance requirement details obtained successfully",
response=ComplianceOverviewDetailSerializer(many=True),
),
202: OpenApiResponse(description="The task is in progress"),
500: OpenApiResponse(
description="Compliance overviews generation task failed"
),
},
filters=True,
),
attributes=extend_schema(
tags=["Compliance Overview"],
summary="Get compliance requirement attributes",
description="Retrieve detailed attribute information for all requirements in a specific compliance framework "
"along with the associated check IDs for each requirement.",
parameters=[
OpenApiParameter(
name="filter[compliance_id]",
required=True,
type=str,
location=OpenApiParameter.QUERY,
description="Compliance framework ID to get attributes for.",
),
],
responses={
200: OpenApiResponse(
description="Compliance attributes obtained successfully",
response=ComplianceOverviewAttributesSerializer(many=True),
),
},
),
)
@method_decorator(CACHE_DECORATOR, name="list")
@method_decorator(CACHE_DECORATOR, name="retrieve")
class ComplianceOverviewViewSet(BaseRLSViewSet):
@method_decorator(CACHE_DECORATOR, name="requirements")
@method_decorator(CACHE_DECORATOR, name="attributes")
class ComplianceOverviewViewSet(BaseRLSViewSet, TaskManagementMixin):
pagination_class = ComplianceOverviewPagination
queryset = ComplianceOverview.objects.all()
queryset = ComplianceRequirementOverview.objects.all()
serializer_class = ComplianceOverviewSerializer
filterset_class = ComplianceOverviewFilter
http_method_names = ["get"]
search_fields = ["compliance_id"]
ordering = ["compliance_id"]
ordering_fields = ["inserted_at", "compliance_id", "framework", "region"]
ordering_fields = ["compliance_id"]
# RBAC required permissions (implicit -> MANAGE_PROVIDERS enable unlimited visibility or check the visibility of
# the provider through the provider group)
required_permissions = []
@@ -2319,51 +2687,44 @@ class ComplianceOverviewViewSet(BaseRLSViewSet):
role, Permissions.UNLIMITED_VISIBILITY.value, False
)
if self.action == "retrieve":
if unlimited_visibility:
# User has unlimited visibility, return all compliance
return ComplianceOverview.objects.filter(
tenant_id=self.request.tenant_id
)
providers = get_providers(role)
return ComplianceOverview.objects.filter(
tenant_id=self.request.tenant_id, scan__provider__in=providers
)
if unlimited_visibility:
base_queryset = self.filter_queryset(
ComplianceOverview.objects.filter(tenant_id=self.request.tenant_id)
ComplianceRequirementOverview.objects.filter(
tenant_id=self.request.tenant_id
)
)
else:
providers = Provider.objects.filter(
provider_groups__in=role.provider_groups.all()
).distinct()
base_queryset = self.filter_queryset(
ComplianceOverview.objects.filter(
ComplianceRequirementOverview.objects.filter(
tenant_id=self.request.tenant_id, scan__provider__in=providers
)
)
max_failed_ids = (
base_queryset.filter(compliance_id=OuterRef("compliance_id"))
.order_by("-requirements_failed")
.values("id")[:1]
)
return base_queryset.filter(id__in=Subquery(max_failed_ids)).order_by(
"compliance_id"
)
return base_queryset
def get_serializer_class(self):
if self.action == "retrieve":
return ComplianceOverviewFullSerializer
if hasattr(self, "response_serializer_class"):
return self.response_serializer_class
elif self.action == "list":
return ComplianceOverviewSerializer
elif self.action == "metadata":
return ComplianceOverviewMetadataSerializer
elif self.action == "attributes":
return ComplianceOverviewAttributesSerializer
elif self.action == "requirements":
return ComplianceOverviewDetailSerializer
return super().get_serializer_class()
@extend_schema(exclude=True)
def retrieve(self, request, *args, **kwargs):
raise MethodNotAllowed(method="GET")
def list(self, request, *args, **kwargs):
if not request.query_params.get("filter[scan_id]"):
scan_id = request.query_params.get("filter[scan_id]")
if not scan_id:
raise ValidationError(
[
{
@@ -2374,7 +2735,82 @@ class ComplianceOverviewViewSet(BaseRLSViewSet):
}
]
)
return super().list(request, *args, **kwargs)
try:
if task := self.get_task_response_if_running(
task_name="scan-compliance-overviews",
task_kwargs={"tenant_id": self.request.tenant_id, "scan_id": scan_id},
raise_on_not_found=False,
):
return task
except TaskFailedException:
return Response(
{"detail": "Task failed to generate compliance overview data."},
status=status.HTTP_500_INTERNAL_SERVER_ERROR,
)
queryset = self.filter_queryset(self.filter_queryset(self.get_queryset()))
requirement_status_subquery = queryset.values(
"compliance_id", "requirement_id"
).annotate(
fail_count=Count("id", filter=Q(requirement_status="FAIL")),
pass_count=Count("id", filter=Q(requirement_status="PASS")),
total_count=Count("id"),
)
compliance_data = {}
framework_info = {}
for item in queryset.values("compliance_id", "framework", "version").distinct():
framework_info[item["compliance_id"]] = {
"framework": item["framework"],
"version": item["version"],
}
for item in requirement_status_subquery:
compliance_id = item["compliance_id"]
if item["fail_count"] > 0:
req_status = "FAIL"
elif item["pass_count"] == item["total_count"]:
req_status = "PASS"
else:
req_status = "MANUAL"
if compliance_id not in compliance_data:
compliance_data[compliance_id] = {
"total_requirements": 0,
"requirements_passed": 0,
"requirements_failed": 0,
"requirements_manual": 0,
}
compliance_data[compliance_id]["total_requirements"] += 1
if req_status == "PASS":
compliance_data[compliance_id]["requirements_passed"] += 1
elif req_status == "FAIL":
compliance_data[compliance_id]["requirements_failed"] += 1
else:
compliance_data[compliance_id]["requirements_manual"] += 1
response_data = []
for compliance_id, data in compliance_data.items():
framework = framework_info.get(compliance_id, {})
response_data.append(
{
"id": compliance_id,
"compliance_id": compliance_id,
"framework": framework.get("framework", ""),
"version": framework.get("version", ""),
"requirements_passed": data["requirements_passed"],
"requirements_failed": data["requirements_failed"],
"requirements_manual": data["requirements_manual"],
"total_requirements": data["total_requirements"],
}
)
serializer = self.get_serializer(response_data, many=True)
return Response(serializer.data)
@action(detail=False, methods=["get"], url_name="metadata")
def metadata(self, request):
@@ -2390,11 +2826,21 @@ class ComplianceOverviewViewSet(BaseRLSViewSet):
}
]
)
tenant_id = self.request.tenant_id
try:
if task := self.get_task_response_if_running(
task_name="scan-compliance-overviews",
task_kwargs={"tenant_id": self.request.tenant_id, "scan_id": scan_id},
raise_on_not_found=False,
):
return task
except TaskFailedException:
return Response(
{"detail": "Task failed to generate compliance overview data."},
status=status.HTTP_500_INTERNAL_SERVER_ERROR,
)
regions = list(
ComplianceOverview.objects.filter(tenant_id=tenant_id, scan_id=scan_id)
self.get_queryset()
.filter(scan_id=scan_id)
.values_list("region", flat=True)
.order_by("region")
.distinct()
@@ -2405,6 +2851,160 @@ class ComplianceOverviewViewSet(BaseRLSViewSet):
serializer.is_valid(raise_exception=True)
return Response(serializer.data, status=status.HTTP_200_OK)
@action(detail=False, methods=["get"], url_name="requirements")
def requirements(self, request):
scan_id = request.query_params.get("filter[scan_id]")
compliance_id = request.query_params.get("filter[compliance_id]")
if not scan_id:
raise ValidationError(
[
{
"detail": "This query parameter is required.",
"status": 400,
"source": {"pointer": "filter[scan_id]"},
"code": "required",
}
]
)
if not compliance_id:
raise ValidationError(
[
{
"detail": "This query parameter is required.",
"status": 400,
"source": {"pointer": "filter[compliance_id]"},
"code": "required",
}
]
)
try:
if task := self.get_task_response_if_running(
task_name="scan-compliance-overviews",
task_kwargs={"tenant_id": self.request.tenant_id, "scan_id": scan_id},
raise_on_not_found=False,
):
return task
except TaskFailedException:
return Response(
{"detail": "Task failed to generate compliance overview data."},
status=status.HTTP_500_INTERNAL_SERVER_ERROR,
)
filtered_queryset = self.filter_queryset(self.get_queryset())
all_requirements = (
filtered_queryset.values(
"requirement_id", "framework", "version", "description"
)
.distinct()
.annotate(
total_instances=Count("id"),
manual_count=Count("id", filter=Q(requirement_status="MANUAL")),
)
)
passed_instances = (
filtered_queryset.filter(requirement_status="PASS")
.values("requirement_id")
.annotate(pass_count=Count("id"))
)
passed_counts = {
item["requirement_id"]: item["pass_count"] for item in passed_instances
}
requirements_summary = []
for requirement in all_requirements:
requirement_id = requirement["requirement_id"]
total_instances = requirement["total_instances"]
passed_count = passed_counts.get(requirement_id, 0)
is_manual = requirement["manual_count"] == total_instances
if is_manual:
requirement_status = "MANUAL"
elif passed_count == total_instances:
requirement_status = "PASS"
else:
requirement_status = "FAIL"
requirements_summary.append(
{
"id": requirement_id,
"framework": requirement["framework"],
"version": requirement["version"],
"description": requirement["description"],
"status": requirement_status,
}
)
serializer = self.get_serializer(requirements_summary, many=True)
return Response(serializer.data, status=status.HTTP_200_OK)
@action(detail=False, methods=["get"], url_name="attributes")
def attributes(self, request):
compliance_id = request.query_params.get("filter[compliance_id]")
if not compliance_id:
raise ValidationError(
[
{
"detail": "This query parameter is required.",
"status": 400,
"source": {"pointer": "filter[compliance_id]"},
"code": "required",
}
]
)
provider_type = None
try:
sample_requirement = (
self.get_queryset().filter(compliance_id=compliance_id).first()
)
if sample_requirement:
provider_type = sample_requirement.scan.provider.provider
except Exception:
pass
# If we couldn't determine from database, try each provider type
if not provider_type:
for pt in Provider.ProviderChoices.values:
if compliance_id in PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE.get(pt, {}):
provider_type = pt
break
if not provider_type:
raise NotFound(detail=f"Compliance framework '{compliance_id}' not found.")
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE.get(
provider_type, {}
)
compliance_framework = compliance_template.get(compliance_id)
if not compliance_framework:
raise NotFound(detail=f"Compliance framework '{compliance_id}' not found.")
attribute_data = []
for requirement_id, requirement in compliance_framework.get(
"requirements", {}
).items():
check_ids = list(requirement.get("checks", {}).keys())
metadata = requirement.get("attributes", [])
attribute_data.append(
{
"id": requirement_id,
"framework": compliance_framework.get("framework", ""),
"version": compliance_framework.get("version", ""),
"description": requirement.get("description", ""),
"attributes": {"metadata": metadata, "check_ids": check_ids},
}
)
serializer = self.get_serializer(attribute_data, many=True)
return Response(serializer.data, status=status.HTTP_200_OK)
@extend_schema(tags=["Overview"])
@extend_schema_view(
@@ -2451,7 +3051,7 @@ class ComplianceOverviewViewSet(BaseRLSViewSet):
class OverviewViewSet(BaseRLSViewSet):
queryset = ComplianceOverview.objects.all()
http_method_names = ["get"]
ordering = ["-id"]
ordering = ["-inserted_at"]
# RBAC required permissions (implicit -> MANAGE_PROVIDERS enable unlimited visibility or check the visibility of
# the provider through the provider group)
required_permissions = []
+16 -8
View File
@@ -1,6 +1,13 @@
import warnings
from celery import Celery, Task
from config.env import env
# Suppress specific warnings from django-rest-auth: https://github.com/iMerica/dj-rest-auth/issues/684
warnings.filterwarnings(
"ignore", category=UserWarning, module="dj_rest_auth.registration.serializers"
)
BROKER_VISIBILITY_TIMEOUT = env.int("DJANGO_BROKER_VISIBILITY_TIMEOUT", default=86400)
celery_app = Celery("tasks")
@@ -33,6 +40,7 @@ class RLSTask(Task):
):
from django_celery_results.models import TaskResult
from api.db_router import MainRouter
from api.models import Task as APITask
result = super().apply_async(
@@ -45,14 +53,14 @@ class RLSTask(Task):
shadow=shadow,
**options,
)
task_result_instance = TaskResult.objects.get(task_id=result.task_id)
from api.db_utils import rls_transaction
task_result_instance = TaskResult.objects.using(MainRouter.admin_db).get(
task_id=result.task_id
)
tenant_id = kwargs.get("tenant_id")
with rls_transaction(tenant_id):
APITask.objects.update_or_create(
id=task_result_instance.task_id,
tenant_id=tenant_id,
defaults={"task_runner_task": task_result_instance},
)
APITask.objects.using(MainRouter.admin_db).update_or_create(
id=task_result_instance.task_id,
tenant_id=tenant_id,
defaults={"task_runner_task": task_result_instance},
)
return result
+4
View File
@@ -10,6 +10,7 @@ from config.settings.social_login import * # noqa
SECRET_KEY = env("SECRET_KEY", default="secret")
DEBUG = env.bool("DJANGO_DEBUG", default=False)
ALLOWED_HOSTS = ["localhost", "127.0.0.1"]
SECURE_PROXY_SSL_HEADER = ("HTTP_X_FORWARDED_PROTO", "https")
# Application definition
@@ -26,16 +27,19 @@ INSTALLED_APPS = [
"rest_framework",
"corsheaders",
"drf_spectacular",
"drf_spectacular_jsonapi",
"django_guid",
"rest_framework_json_api",
"django_celery_results",
"django_celery_beat",
"rest_framework_simplejwt.token_blacklist",
"allauth",
"django.contrib.sites",
"allauth.account",
"allauth.socialaccount",
"allauth.socialaccount.providers.google",
"allauth.socialaccount.providers.github",
"allauth.socialaccount.providers.saml",
"dj_rest_auth.registration",
"rest_framework.authtoken",
]
+26
View File
@@ -14,6 +14,19 @@ DATABASES = {
"HOST": env("POSTGRES_HOST", default="postgres-db"),
"PORT": env("POSTGRES_PORT", default="5432"),
},
"default_read": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB", default="prowler_db"),
"USER": env("POSTGRES_USER", default="prowler_user"),
"PASSWORD": env("POSTGRES_PASSWORD", default="prowler"),
"HOST": env(
"POSTGRES_HOST_READ_ONLY",
default=env("POSTGRES_HOST", default="postgres-db"),
),
"PORT": env(
"POSTGRES_PORT_READ_ONLY", default=env("POSTGRES_PORT", default="5432")
),
},
"admin": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB", default="prowler_db"),
@@ -22,6 +35,19 @@ DATABASES = {
"HOST": env("POSTGRES_HOST", default="postgres-db"),
"PORT": env("POSTGRES_PORT", default="5432"),
},
"admin_read": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB", default="prowler_db"),
"USER": env("POSTGRES_ADMIN_USER", default="prowler"),
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD", default="S3cret"),
"HOST": env(
"POSTGRES_HOST_READ_ONLY",
default=env("POSTGRES_HOST", default="postgres-db"),
),
"PORT": env(
"POSTGRES_PORT_READ_ONLY", default=env("POSTGRES_PORT", default="5432")
),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
+16 -1
View File
@@ -5,7 +5,6 @@ DEBUG = env.bool("DJANGO_DEBUG", default=False)
ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=["localhost", "127.0.0.1"])
# Database
# TODO Use Django database routers https://docs.djangoproject.com/en/5.0/topics/db/multi-db/#automatic-database-routing
DATABASES = {
"prowler_user": {
"ENGINE": "django.db.backends.postgresql",
@@ -15,6 +14,14 @@ DATABASES = {
"HOST": env("POSTGRES_HOST"),
"PORT": env("POSTGRES_PORT"),
},
"default_read": {
"ENGINE": "django.db.backends.postgresql",
"NAME": env("POSTGRES_DB"),
"USER": env("POSTGRES_USER"),
"PASSWORD": env("POSTGRES_PASSWORD"),
"HOST": env("POSTGRES_HOST_READ_ONLY", default=env("POSTGRES_HOST")),
"PORT": env("POSTGRES_PORT_READ_ONLY", default=env("POSTGRES_PORT")),
},
"admin": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB"),
@@ -23,5 +30,13 @@ DATABASES = {
"HOST": env("POSTGRES_HOST"),
"PORT": env("POSTGRES_PORT"),
},
"admin_read": {
"ENGINE": "psqlextra.backend",
"NAME": env("POSTGRES_DB"),
"USER": env("POSTGRES_ADMIN_USER"),
"PASSWORD": env("POSTGRES_ADMIN_PASSWORD"),
"HOST": env("POSTGRES_HOST_READ_ONLY", default=env("POSTGRES_HOST")),
"PORT": env("POSTGRES_PORT_READ_ONLY", default=env("POSTGRES_PORT")),
},
}
DATABASES["default"] = DATABASES["prowler_user"]
@@ -11,8 +11,7 @@ GITHUB_OAUTH_CALLBACK_URL = env("SOCIAL_GITHUB_OAUTH_CALLBACK_URL", default="")
# Allauth settings
ACCOUNT_LOGIN_METHODS = {"email"} # Use Email / Password authentication
ACCOUNT_USERNAME_REQUIRED = False
ACCOUNT_EMAIL_REQUIRED = True
ACCOUNT_SIGNUP_FIELDS = ["email*", "password1*", "password2*"]
ACCOUNT_EMAIL_VERIFICATION = "none" # Do not require email confirmation
ACCOUNT_USER_MODEL_USERNAME_FIELD = None
REST_AUTH = {
@@ -25,6 +24,11 @@ SOCIALACCOUNT_EMAIL_AUTHENTICATION = True
# Connect local account and social account if local account with that email address already exists
SOCIALACCOUNT_EMAIL_AUTHENTICATION_AUTO_CONNECT = True
SOCIALACCOUNT_ADAPTER = "api.adapters.ProwlerSocialAccountAdapter"
# SAML keys
SAML_PUBLIC_CERT = env("SAML_PUBLIC_CERT", default="")
SAML_PRIVATE_KEY = env("SAML_PRIVATE_KEY", default="")
SOCIALACCOUNT_PROVIDERS = {
"google": {
"APP": {
@@ -50,4 +54,18 @@ SOCIALACCOUNT_PROVIDERS = {
"read:org",
],
},
"saml": {
"use_nameid_for_email": True,
"sp": {
"entity_id": "urn:prowler.com:sp",
},
"advanced": {
"x509cert": SAML_PUBLIC_CERT,
"private_key": SAML_PRIVATE_KEY,
"name_id_format": "urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress",
"authn_request_signed": True,
"want_assertion_signed": True,
"want_message_signed": True,
},
},
}
+216 -1
View File
@@ -1,8 +1,9 @@
import logging
from datetime import datetime, timedelta, timezone
from unittest.mock import patch
from unittest.mock import MagicMock, patch
import pytest
from allauth.socialaccount.models import SocialLogin
from django.conf import settings
from django.db import connection as django_connection
from django.db import connections as django_connections
@@ -15,6 +16,7 @@ from tasks.jobs.backfill import backfill_resource_scan_summaries
from api.db_utils import rls_transaction
from api.models import (
ComplianceOverview,
ComplianceRequirementOverview,
Finding,
Integration,
IntegrationProviderRelationship,
@@ -26,9 +28,12 @@ from api.models import (
Resource,
ResourceTag,
Role,
SAMLConfiguration,
SAMLDomainIndex,
Scan,
ScanSummary,
StateChoices,
StatusChoices,
Task,
User,
UserRoleRelationship,
@@ -777,6 +782,114 @@ def compliance_overviews_fixture(scans_fixture, tenants_fixture):
return compliance_overview1, compliance_overview2
@pytest.fixture
def compliance_requirements_overviews_fixture(scans_fixture, tenants_fixture):
"""Fixture for ComplianceRequirementOverview objects used by the new ComplianceOverviewViewSet."""
tenant = tenants_fixture[0]
scan1, scan2, scan3 = scans_fixture
# Create ComplianceRequirementOverview objects for scan1
requirement_overview1 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="aws_account_security_onboarding_aws",
framework="AWS-Account-Security-Onboarding",
version="1.0",
description="Description for AWS Account Security Onboarding",
region="eu-west-1",
requirement_id="requirement1",
requirement_status=StatusChoices.PASS,
passed_checks=2,
failed_checks=0,
total_checks=2,
)
requirement_overview2 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="aws_account_security_onboarding_aws",
framework="AWS-Account-Security-Onboarding",
version="1.0",
description="Description for AWS Account Security Onboarding",
region="eu-west-1",
requirement_id="requirement2",
requirement_status=StatusChoices.PASS,
passed_checks=2,
failed_checks=0,
total_checks=2,
)
requirement_overview3 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="aws_account_security_onboarding_aws",
framework="AWS-Account-Security-Onboarding",
version="1.0",
description="Description for AWS Account Security Onboarding",
region="eu-west-2",
requirement_id="requirement1",
requirement_status=StatusChoices.PASS,
passed_checks=2,
failed_checks=0,
total_checks=2,
)
requirement_overview4 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="aws_account_security_onboarding_aws",
framework="AWS-Account-Security-Onboarding",
version="1.0",
description="Description for AWS Account Security Onboarding",
region="eu-west-2",
requirement_id="requirement2",
requirement_status=StatusChoices.FAIL,
passed_checks=1,
failed_checks=1,
total_checks=2,
)
requirement_overview5 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="aws_account_security_onboarding_aws",
framework="AWS-Account-Security-Onboarding",
version="1.0",
description="Description for AWS Account Security Onboarding (MANUAL)",
region="eu-west-2",
requirement_id="requirement3",
requirement_status=StatusChoices.MANUAL,
passed_checks=0,
failed_checks=0,
total_checks=0,
)
# Create a different compliance framework for testing
requirement_overview6 = ComplianceRequirementOverview.objects.create(
tenant=tenant,
scan=scan1,
compliance_id="cis_1.4_aws",
framework="CIS-1.4-AWS",
version="1.4",
description="CIS AWS Foundations Benchmark v1.4.0",
region="eu-west-1",
requirement_id="cis_requirement1",
requirement_status=StatusChoices.FAIL,
passed_checks=0,
failed_checks=3,
total_checks=3,
)
return (
requirement_overview1,
requirement_overview2,
requirement_overview3,
requirement_overview4,
requirement_overview5,
requirement_overview6,
)
def get_api_tokens(
api_client, user_email: str, user_password: str, tenant_id: str = None
) -> tuple[str, str]:
@@ -929,6 +1042,105 @@ def backfill_scan_metadata_fixture(scans_fixture, findings_fixture):
backfill_resource_scan_summaries(tenant_id=tenant_id, scan_id=scan_id)
@pytest.fixture(scope="function")
def latest_scan_finding(authenticated_client, providers_fixture, resources_fixture):
provider = providers_fixture[0]
tenant_id = str(providers_fixture[0].tenant_id)
resource = resources_fixture[0]
scan = Scan.objects.create(
name="latest completed scan",
provider=provider,
trigger=Scan.TriggerChoices.MANUAL,
state=StateChoices.COMPLETED,
tenant_id=tenant_id,
)
finding = Finding.objects.create(
tenant_id=tenant_id,
uid="test_finding_uid_1",
scan=scan,
delta="new",
status=Status.FAIL,
status_extended="test status extended ",
impact=Severity.critical,
impact_extended="test impact extended one",
severity=Severity.critical,
raw_result={
"status": Status.FAIL,
"impact": Severity.critical,
"severity": Severity.critical,
},
tags={"test": "dev-qa"},
check_id="test_check_id",
check_metadata={
"CheckId": "test_check_id",
"Description": "test description apple sauce",
},
first_seen_at="2024-01-02T00:00:00Z",
)
finding.add_resources([resource])
backfill_resource_scan_summaries(tenant_id, str(scan.id))
return finding
@pytest.fixture
def saml_setup(tenants_fixture):
tenant_id = tenants_fixture[0].id
domain = "example.com"
SAMLDomainIndex.objects.create(email_domain=domain, tenant_id=tenant_id)
metadata_xml = """<?xml version='1.0' encoding='UTF-8'?>
<md:EntityDescriptor entityID='TEST' xmlns:md='urn:oasis:names:tc:SAML:2.0:metadata'>
<md:IDPSSODescriptor WantAuthnRequestsSigned='false' protocolSupportEnumeration='urn:oasis:names:tc:SAML:2.0:protocol'>
<md:KeyDescriptor use='signing'>
<ds:KeyInfo xmlns:ds='http://www.w3.org/2000/09/xmldsig#'>
<ds:X509Data>
<ds:X509Certificate>TEST</ds:X509Certificate>
</ds:X509Data>
</ds:KeyInfo>
</md:KeyDescriptor>
<md:NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</md:NameIDFormat>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST' Location='https://TEST/sso/saml'/>
<md:SingleSignOnService Binding='urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect' Location='https://TEST/sso/saml'/>
</md:IDPSSODescriptor>
</md:EntityDescriptor>
"""
SAMLConfiguration.objects.create(
tenant_id=str(tenant_id),
email_domain=domain,
metadata_xml=metadata_xml,
)
return {
"email": f"user@{domain}",
"domain": domain,
"tenant_id": tenant_id,
}
@pytest.fixture
def saml_sociallogin(users_fixture):
user = users_fixture[0]
user.email = "samlsso@acme.com"
extra_data = {
"firstName": ["Test"],
"lastName": ["User"],
"organization": ["Prowler"],
"userType": ["member"],
}
account = MagicMock()
account.provider = "saml"
account.extra_data = extra_data
sociallogin = MagicMock(spec=SocialLogin)
sociallogin.account = account
sociallogin.user = user
return sociallogin
def get_authorization_header(access_token: str) -> dict:
return {"Authorization": f"Bearer {access_token}"}
@@ -942,6 +1154,9 @@ def pytest_configure(config):
# Apply the mock before the test session starts. This is necessary to avoid admin error when running the
# 0004_rbac_missing_admin_roles migration
patch("api.db_router.MainRouter.admin_db", new="default").start()
patch("api.db_router.MainRouter.admin_read", new="default").start()
patch("api.db_router.MainRouter.prowler_user", new="default").start()
patch("api.db_router.MainRouter.default_read", new="default").start()
def pytest_unconfigure(config):
+6
View File
@@ -3,6 +3,12 @@
import os
import sys
import warnings
# Suppress specific warnings from django-rest-auth: https://github.com/iMerica/dj-rest-auth/issues/684
warnings.filterwarnings(
"ignore", category=UserWarning, module="dj_rest_auth.registration.serializers"
)
def main():
+1 -1
View File
@@ -64,7 +64,7 @@ def delete_tenant(pk: str):
"""
deletion_summary = {}
for provider in Provider.objects.using(MainRouter.admin_db).filter(tenant_id=pk):
for provider in Provider.objects.using(MainRouter.admin_read).filter(tenant_id=pk):
summary = delete_provider(pk, provider.id)
deletion_summary.update(summary)
+4
View File
@@ -45,6 +45,9 @@ from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore_azur
from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore_gcp import (
ProwlerThreatScoreGCP,
)
from prowler.lib.outputs.compliance.prowler_threatscore.prowler_threatscore_m365 import (
ProwlerThreatScoreM365,
)
from prowler.lib.outputs.csv.csv import CSV
from prowler.lib.outputs.html.html import HTML
from prowler.lib.outputs.ocsf.ocsf import OCSF
@@ -85,6 +88,7 @@ COMPLIANCE_CLASS_MAP = {
],
"m365": [
(lambda name: name.startswith("cis_"), M365CIS),
(lambda name: name == "prowler_threatscore_m365", ProwlerThreatScoreM365),
],
}
+120 -71
View File
@@ -13,9 +13,9 @@ from api.compliance import (
PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE,
generate_scan_compliance,
)
from api.db_utils import rls_transaction
from api.db_utils import create_objects_in_batches, rls_transaction
from api.models import (
ComplianceOverview,
ComplianceRequirementOverview,
Finding,
Provider,
Resource,
@@ -119,11 +119,11 @@ def perform_prowler_scan(
ValueError: If the provider cannot be connected.
"""
check_status_by_region = {}
exception = None
unique_resources = set()
scan_resource_cache: set[tuple[str, str, str, str]] = set()
start_time = time.time()
exc = None
with rls_transaction(tenant_id):
provider_instance = Provider.objects.get(pk=provider_id)
@@ -139,7 +139,7 @@ def perform_prowler_scan(
provider_instance.connected = True
except Exception as e:
provider_instance.connected = False
raise ValueError(
exc = ValueError(
f"Provider {provider_instance.provider} is not connected: {e}"
)
finally:
@@ -148,6 +148,11 @@ def perform_prowler_scan(
)
provider_instance.save()
# If the provider is not connected, raise an exception outside the transaction.
# If raised within the transaction, the transaction will be rolled back and the provider will not be marked as not connected.
if exc:
raise exc
prowler_scan = ProwlerScan(provider=prowler_provider, checks=checks_to_execute)
resource_cache = {}
@@ -287,16 +292,6 @@ def perform_prowler_scan(
)
finding_instance.add_resources([resource_instance])
# Update compliance data if applicable
if finding.status.value == "MUTED":
continue
region_dict = check_status_by_region.setdefault(finding.region, {})
current_status = region_dict.get(finding.check_id)
if current_status == "FAIL":
continue
region_dict[finding.check_id] = finding.status.value
# Update scan resource summaries
scan_resource_cache.add(
(
@@ -329,63 +324,6 @@ def perform_prowler_scan(
if exception is not None:
raise exception
try:
regions = prowler_provider.get_regions()
except AttributeError:
regions = set()
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
provider_instance.provider
]
compliance_overview_by_region = {
region: deepcopy(compliance_template) for region in regions
}
for region, check_status in check_status_by_region.items():
compliance_data = compliance_overview_by_region.setdefault(
region, deepcopy(compliance_template)
)
for check_name, status in check_status.items():
generate_scan_compliance(
compliance_data,
provider_instance.provider,
check_name,
status,
)
# Prepare compliance overview objects
compliance_overview_objects = []
for region, compliance_data in compliance_overview_by_region.items():
for compliance_id, compliance in compliance_data.items():
compliance_overview_objects.append(
ComplianceOverview(
tenant_id=tenant_id,
scan=scan_instance,
region=region,
compliance_id=compliance_id,
framework=compliance["framework"],
version=compliance["version"],
description=compliance["description"],
requirements=compliance["requirements"],
requirements_passed=compliance["requirements_status"]["passed"],
requirements_failed=compliance["requirements_status"]["failed"],
requirements_manual=compliance["requirements_status"]["manual"],
total_requirements=compliance["total_requirements"],
)
)
try:
with rls_transaction(tenant_id):
ComplianceOverview.objects.bulk_create(
compliance_overview_objects, batch_size=500
)
except Exception as overview_exception:
import sentry_sdk
sentry_sdk.capture_exception(overview_exception)
logger.error(
f"Error storing compliance overview for scan {scan_id}: {overview_exception}"
)
try:
resource_scan_summaries = [
ResourceScanSummary(
@@ -564,3 +502,114 @@ def aggregate_findings(tenant_id: str, scan_id: str):
for agg in aggregation
}
ScanSummary.objects.bulk_create(scan_aggregations, batch_size=3000)
def create_compliance_requirements(tenant_id: str, scan_id: str):
"""
Create detailed compliance requirement overview records for a scan.
This function processes the compliance data collected during a scan and creates
individual records for each compliance requirement in each region. These detailed
records provide a granular view of compliance status.
Args:
tenant_id (str): The ID of the tenant for which to create records.
scan_id (str): The ID of the scan for which to create records.
Returns:
dict: A dictionary containing the number of requirements created and the regions processed.
Raises:
ValidationError: If tenant_id is not a valid UUID.
"""
try:
with rls_transaction(tenant_id):
scan_instance = Scan.objects.get(pk=scan_id)
provider_instance = scan_instance.provider
prowler_provider = initialize_prowler_provider(provider_instance)
# Get check status data by region from findings
check_status_by_region = {}
with rls_transaction(tenant_id):
findings = Finding.objects.filter(scan_id=scan_id, muted=False)
for finding in findings:
# Get region from resources
for resource in finding.resources.all():
region = resource.region
region_dict = check_status_by_region.setdefault(region, {})
current_status = region_dict.get(finding.check_id)
if current_status == "FAIL":
continue
region_dict[finding.check_id] = finding.status
try:
# Try to get regions from provider
regions = prowler_provider.get_regions()
except (AttributeError, Exception):
# If not available, use regions from findings
regions = set(check_status_by_region.keys())
# Get compliance template for the provider
compliance_template = PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE[
provider_instance.provider
]
# Create compliance data by region
compliance_overview_by_region = {
region: deepcopy(compliance_template) for region in regions
}
# Apply check statuses to compliance data
for region, check_status in check_status_by_region.items():
compliance_data = compliance_overview_by_region.setdefault(
region, deepcopy(compliance_template)
)
for check_name, status in check_status.items():
generate_scan_compliance(
compliance_data,
provider_instance.provider,
check_name,
status,
)
# Prepare compliance requirement objects
compliance_requirement_objects = []
for region, compliance_data in compliance_overview_by_region.items():
for compliance_id, compliance in compliance_data.items():
# Create an overview record for each requirement within each compliance framework
for requirement_id, requirement in compliance["requirements"].items():
compliance_requirement_objects.append(
ComplianceRequirementOverview(
tenant_id=tenant_id,
scan=scan_instance,
region=region,
compliance_id=compliance_id,
framework=compliance["framework"],
version=compliance["version"],
requirement_id=requirement_id,
description=requirement["description"],
passed_checks=requirement["checks_status"]["pass"],
failed_checks=requirement["checks_status"]["fail"],
total_checks=requirement["checks_status"]["total"],
requirement_status=requirement["status"],
)
)
# Bulk create requirement records
create_objects_in_batches(
tenant_id, ComplianceRequirementOverview, compliance_requirement_objects
)
return {
"requirements_created": len(compliance_requirement_objects),
"regions_processed": list(regions),
"compliance_frameworks": (
list(compliance_overview_by_region.get(list(regions)[0], {}).keys())
if regions
else []
),
}
except Exception as e:
logger.error(f"Error creating compliance requirements for scan {scan_id}: {e}")
raise e
+25 -1
View File
@@ -17,7 +17,11 @@ from tasks.jobs.export import (
_generate_output_directory,
_upload_to_s3,
)
from tasks.jobs.scan import aggregate_findings, perform_prowler_scan
from tasks.jobs.scan import (
aggregate_findings,
create_compliance_requirements,
perform_prowler_scan,
)
from tasks.utils import batched, get_next_execution_datetime
from api.compliance import get_compliance_frameworks
@@ -101,6 +105,7 @@ def perform_scan_task(
chain(
perform_scan_summary_task.si(tenant_id, scan_id),
create_compliance_requirements_task.si(tenant_id=tenant_id, scan_id=scan_id),
generate_outputs.si(
scan_id=scan_id, provider_id=provider_id, tenant_id=tenant_id
),
@@ -211,6 +216,9 @@ def perform_scheduled_scan_task(self, tenant_id: str, provider_id: str):
chain(
perform_scan_summary_task.si(tenant_id, scan_instance.id),
create_compliance_requirements_task.si(
tenant_id=tenant_id, scan_id=str(scan_instance.id)
),
generate_outputs.si(
scan_id=str(scan_instance.id), provider_id=provider_id, tenant_id=tenant_id
),
@@ -371,3 +379,19 @@ def backfill_scan_resource_summaries_task(tenant_id: str, scan_id: str):
scan_id (str): The scan identifier.
"""
return backfill_resource_scan_summaries(tenant_id=tenant_id, scan_id=scan_id)
@shared_task(base=RLSTask, name="scan-compliance-overviews")
def create_compliance_requirements_task(tenant_id: str, scan_id: str):
"""
Creates detailed compliance requirement records for a scan.
This task processes the compliance data collected during a scan and creates
individual records for each compliance requirement in each region. These detailed
records provide a granular view of compliance status.
Args:
tenant_id (str): The tenant ID for which to create records.
scan_id (str): The ID of the scan for which to create records.
"""
return create_compliance_requirements(tenant_id=tenant_id, scan_id=scan_id)
+821 -8
View File
@@ -1,16 +1,19 @@
import json
import uuid
from datetime import datetime
from unittest.mock import MagicMock, patch
import pytest
from tasks.jobs.scan import (
_create_finding_delta,
_store_resources,
create_compliance_requirements,
perform_prowler_scan,
)
from tasks.utils import CustomEncoder
from api.models import (
ComplianceRequirementOverview,
Finding,
Provider,
Resource,
@@ -206,6 +209,10 @@ class TestPerformScan:
scan.refresh_from_db()
assert scan.state == StateChoices.FAILED
provider.refresh_from_db()
assert provider.connected is False
assert isinstance(provider.connection_last_checked_at, datetime)
@pytest.mark.parametrize(
"last_status, new_status, expected_delta",
[
@@ -230,7 +237,7 @@ class TestPerformScan:
):
tenant_id = uuid.uuid4()
provider_instance = MagicMock()
provider_instance.id = "provider456"
provider_instance.id = "provider123"
finding = MagicMock()
finding.resource_uid = "resource_uid_123"
@@ -245,15 +252,16 @@ class TestPerformScan:
resource_instance.region = finding.region
mock_get_or_create_resource.return_value = (resource_instance, True)
tag_instance = MagicMock()
mock_get_or_create_tag.return_value = (tag_instance, True)
resource, resource_uid_tuple = _store_resources(
finding, tenant_id, provider_instance
finding, str(tenant_id), provider_instance
)
mock_get_or_create_resource.assert_called_once_with(
tenant_id=tenant_id,
tenant_id=str(tenant_id),
provider=provider_instance,
uid=finding.resource_uid,
defaults={
@@ -300,11 +308,11 @@ class TestPerformScan:
mock_get_or_create_tag.return_value = (tag_instance, True)
resource, resource_uid_tuple = _store_resources(
finding, tenant_id, provider_instance
finding, str(tenant_id), provider_instance
)
mock_get_or_create_resource.assert_called_once_with(
tenant_id=tenant_id,
tenant_id=str(tenant_id),
provider=provider_instance,
uid=finding.resource_uid,
defaults={
@@ -358,14 +366,14 @@ class TestPerformScan:
]
resource, resource_uid_tuple = _store_resources(
finding, tenant_id, provider_instance
finding, str(tenant_id), provider_instance
)
mock_get_or_create_tag.assert_any_call(
tenant_id=tenant_id, key="tag1", value="value1"
tenant_id=str(tenant_id), key="tag1", value="value1"
)
mock_get_or_create_tag.assert_any_call(
tenant_id=tenant_id, key="tag2", value="value2"
tenant_id=str(tenant_id), key="tag2", value="value2"
)
resource_instance.upsert_or_delete_tags.assert_called_once()
tags_passed = resource_instance.upsert_or_delete_tags.call_args[1]["tags"]
@@ -377,3 +385,808 @@ class TestPerformScan:
# TODO Add tests for aggregations
@pytest.mark.django_db
class TestCreateComplianceRequirements:
def test_create_compliance_requirements_success(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
findings_fixture,
resources_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch("tasks.jobs.scan.generate_scan_compliance"),
patch("tasks.jobs.scan.create_objects_in_batches") as mock_create_objects,
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.return_value = [
"us-east-1",
"us-west-2",
]
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {
"cis_1.4_aws": {
"framework": "CIS AWS Foundations Benchmark",
"version": "1.4.0",
"requirements": {
"1.1": {
"description": "Ensure root access key does not exist",
"checks_status": {
"pass": 0,
"fail": 0,
"manual": 0,
"total": 1,
},
"status": "PASS",
},
"1.2": {
"description": "Ensure MFA is enabled for root account",
"checks_status": {
"pass": 0,
"fail": 1,
"manual": 0,
"total": 1,
},
"status": "FAIL",
},
},
},
"aws_account_security_onboarding_aws": {
"framework": "AWS Account Security Onboarding",
"version": "1.0",
"requirements": {
"requirement1": {
"description": "Basic security requirement",
"checks_status": {
"pass": 1,
"fail": 0,
"manual": 0,
"total": 1,
},
"status": "PASS",
},
},
},
}
mock_findings_filter.return_value = []
result = create_compliance_requirements(tenant_id, scan_id)
assert "requirements_created" in result
assert "regions_processed" in result
assert "compliance_frameworks" in result
assert result["regions_processed"] == ["us-east-1", "us-west-2"]
assert result["requirements_created"] == 6
assert len(result["compliance_frameworks"]) == 2
mock_create_objects.assert_called_once()
call_args = mock_create_objects.call_args[0]
assert call_args[0] == tenant_id
assert call_args[1] == ComplianceRequirementOverview
assert len(call_args[2]) == 6
compliance_objects = call_args[2]
for obj in compliance_objects:
assert isinstance(obj, ComplianceRequirementOverview)
assert obj.tenant.id == tenant.id
assert obj.scan == scan
assert obj.region in ["us-east-1", "us-west-2"]
assert obj.compliance_id in [
"cis_1.4_aws",
"aws_account_security_onboarding_aws",
]
def test_create_compliance_requirements_with_findings(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches"),
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_finding1 = MagicMock()
mock_finding1.check_id = "check1"
mock_finding1.status = "PASS"
mock_resource1 = MagicMock()
mock_resource1.region = "us-east-1"
mock_finding1.resources.all.return_value = [mock_resource1]
mock_finding2 = MagicMock()
mock_finding2.check_id = "check2"
mock_finding2.status = "FAIL"
mock_resource2 = MagicMock()
mock_resource2.region = "us-west-2"
mock_finding2.resources.all.return_value = [mock_resource2]
mock_findings_filter.return_value = [mock_finding1, mock_finding2]
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.return_value = [
"us-east-1",
"us-west-2",
]
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {"check_1": None},
"checks_status": {
"pass": 2,
"fail": 1,
"manual": 0,
"total": 3,
},
"status": "FAIL",
},
"req_2": {
"description": "Test Requirement 2",
"checks": {"check_2": None},
"checks_status": {
"pass": 2,
"fail": 0,
"manual": 0,
"total": 2,
},
"status": "PASS",
},
},
}
}
result = create_compliance_requirements(tenant_id, scan_id)
mock_findings_filter.assert_called_once_with(scan_id=scan_id, muted=False)
assert mock_generate_compliance.call_count == 2
assert result["requirements_created"] == 4
assert set(result["regions_processed"]) == {"us-east-1", "us-west-2"}
def test_create_compliance_requirements_no_provider_regions(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch("tasks.jobs.scan.generate_scan_compliance"),
patch("tasks.jobs.scan.create_objects_in_batches"),
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.KUBERNETES
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_finding = MagicMock()
mock_finding.check_id = "check1"
mock_finding.status = "PASS"
mock_resource = MagicMock()
mock_resource.region = "default"
mock_finding.resources.all.return_value = [mock_resource]
mock_findings_filter.return_value = [mock_finding]
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.side_effect = AttributeError(
"No get_regions method"
)
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {
"kubernetes_cis": {
"framework": "CIS Kubernetes Benchmark",
"version": "1.6.0",
"requirements": {
"1.1": {
"description": "Test requirement",
"checks_status": {
"pass": 0,
"fail": 0,
"manual": 0,
"total": 1,
},
"status": "PASS",
},
},
},
}
result = create_compliance_requirements(tenant_id, scan_id)
assert result["regions_processed"] == ["default"]
def test_create_compliance_requirements_empty_findings(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches"),
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_findings_filter.return_value = []
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.return_value = ["us-east-1"]
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {
"cis_1.4_aws": {
"framework": "CIS AWS Foundations Benchmark",
"version": "1.4.0",
"requirements": {
"1.1": {
"description": "Test requirement",
"checks_status": {
"pass": 0,
"fail": 0,
"manual": 0,
"total": 1,
},
"status": "PASS",
},
},
},
}
mock_findings_filter.return_value = []
result = create_compliance_requirements(tenant_id, scan_id)
assert result["regions_processed"] == ["us-east-1"]
assert result["requirements_created"] == 1
mock_generate_compliance.assert_not_called()
def test_create_compliance_requirements_error_handling(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_initialize_prowler_provider.side_effect = Exception(
"Provider initialization failed"
)
with pytest.raises(Exception, match="Provider initialization failed"):
create_compliance_requirements(tenant_id, scan_id)
def test_create_compliance_requirements_muted_findings_excluded(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch("tasks.jobs.scan.generate_scan_compliance"),
patch("tasks.jobs.scan.create_objects_in_batches"),
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_findings_filter.return_value = []
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.return_value = ["us-east-1"]
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {}
mock_findings_filter.return_value = []
create_compliance_requirements(tenant_id, scan_id)
mock_findings_filter.assert_called_once_with(scan_id=scan_id, muted=False)
def test_create_compliance_requirements_check_status_priority(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches"),
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
provider = providers_fixture[0]
provider.provider = Provider.ProviderChoices.AWS
provider.save()
scan.provider = provider
scan.save()
tenant_id = str(tenant.id)
scan_id = str(scan.id)
mock_finding1 = MagicMock()
mock_finding1.check_id = "check1"
mock_finding1.status = "PASS"
mock_resource1 = MagicMock()
mock_resource1.region = "us-east-1"
mock_finding1.resources.all.return_value = [mock_resource1]
mock_finding2 = MagicMock()
mock_finding2.check_id = "check1"
mock_finding2.status = "FAIL"
mock_resource2 = MagicMock()
mock_resource2.region = "us-east-1"
mock_finding2.resources.all.return_value = [mock_resource2]
mock_findings_filter.return_value = [mock_finding1, mock_finding2]
mock_prowler_provider_instance = MagicMock()
mock_prowler_provider_instance.get_regions.return_value = ["us-east-1"]
mock_initialize_prowler_provider.return_value = (
mock_prowler_provider_instance
)
mock_compliance_template.__getitem__.return_value = {
"cis_1.4_aws": {
"framework": "CIS AWS Foundations Benchmark",
"version": "1.4.0",
"requirements": {
"1.1": {
"description": "Test requirement",
"checks_status": {
"pass": 0,
"fail": 0,
"manual": 0,
"total": 1,
},
"status": "PASS",
},
},
},
}
create_compliance_requirements(tenant_id, scan_id)
assert mock_generate_compliance.call_count == 1
def test_compliance_overview_aggregation_requirement_fail_priority(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches") as mock_create_objects,
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
providers_fixture[0]
mock_findings_filter.return_value = []
mock_prowler_provider = MagicMock()
mock_prowler_provider.get_regions.return_value = [
"us-east-1",
"us-west-2",
"eu-west-1",
]
mock_initialize_prowler_provider.return_value = mock_prowler_provider
mock_compliance_template.__getitem__.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {"check_1": None},
"checks_status": {
"pass": 2,
"fail": 1,
"manual": 0,
"total": 3,
},
"status": "FAIL",
}
},
}
}
mock_generate_compliance.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {
"check_1": {
"us-east-1": {"status": "PASS"},
"us-west-2": {"status": "FAIL"},
"eu-west-1": {"status": "PASS"},
}
},
"checks_status": {
"pass": 2,
"fail": 1,
"manual": 0,
"total": 3,
},
"status": "FAIL",
}
},
}
}
created_objects = []
mock_create_objects.side_effect = (
lambda tenant_id, model, objs, batch_size=500: created_objects.extend(
objs
)
)
create_compliance_requirements(str(tenant.id), str(scan.id))
assert len(created_objects) == 3
assert all(obj.requirement_status == "FAIL" for obj in created_objects)
def test_compliance_overview_aggregation_requirement_pass_all_regions(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches") as mock_create_objects,
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
providers_fixture[0]
mock_findings_filter.return_value = []
mock_prowler_provider = MagicMock()
mock_prowler_provider.get_regions.return_value = ["us-east-1", "us-west-2"]
mock_initialize_prowler_provider.return_value = mock_prowler_provider
mock_compliance_template.__getitem__.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {"check_1": None},
"checks_status": {
"pass": 2,
"fail": 0,
"manual": 0,
"total": 2,
},
"status": "PASS",
}
},
}
}
mock_generate_compliance.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {
"check_1": {
"us-east-1": {"status": "PASS"},
"us-west-2": {"status": "PASS"},
}
},
"checks_status": {
"pass": 2,
"fail": 0,
"manual": 0,
"total": 2,
},
"status": "PASS",
}
},
}
}
created_objects = []
mock_create_objects.side_effect = (
lambda tenant_id, model, objs, batch_size=500: created_objects.extend(
objs
)
)
create_compliance_requirements(str(tenant.id), str(scan.id))
assert len(created_objects) == 2
assert all(obj.requirement_status == "PASS" for obj in created_objects)
def test_compliance_overview_aggregation_multiple_requirements_mixed_status(
self,
tenants_fixture,
scans_fixture,
providers_fixture,
):
with (
patch("api.db_utils.rls_transaction"),
patch(
"tasks.jobs.scan.initialize_prowler_provider"
) as mock_initialize_prowler_provider,
patch(
"tasks.jobs.scan.PROWLER_COMPLIANCE_OVERVIEW_TEMPLATE"
) as mock_compliance_template,
patch(
"tasks.jobs.scan.generate_scan_compliance"
) as mock_generate_compliance,
patch("tasks.jobs.scan.create_objects_in_batches") as mock_create_objects,
patch("api.models.Finding.objects.filter") as mock_findings_filter,
):
tenant = tenants_fixture[0]
scan = scans_fixture[0]
providers_fixture[0]
mock_findings_filter.return_value = []
mock_prowler_provider = MagicMock()
mock_prowler_provider.get_regions.return_value = ["us-east-1", "us-west-2"]
mock_initialize_prowler_provider.return_value = mock_prowler_provider
mock_compliance_template.__getitem__.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {"check_1": None},
"checks_status": {
"pass": 2,
"fail": 0,
"manual": 0,
"total": 2,
},
"status": "PASS",
},
"req_2": {
"description": "Test Requirement 2",
"checks": {"check_2": None},
"checks_status": {
"pass": 1,
"fail": 1,
"manual": 0,
"total": 2,
},
"status": "FAIL",
},
},
}
}
mock_generate_compliance.return_value = {
"test_compliance": {
"framework": "Test Framework",
"version": "1.0",
"requirements": {
"req_1": {
"description": "Test Requirement 1",
"checks": {
"check_1": {
"us-east-1": {"status": "PASS"},
"us-west-2": {"status": "PASS"},
}
},
"checks_status": {
"pass": 2,
"fail": 0,
"manual": 0,
"total": 2,
},
"status": "PASS",
},
"req_2": {
"description": "Test Requirement 2",
"checks": {
"check_2": {
"us-east-1": {"status": "PASS"},
"us-west-2": {"status": "FAIL"},
}
},
"checks_status": {
"pass": 1,
"fail": 1,
"manual": 0,
"total": 2,
},
"status": "FAIL",
},
},
}
}
created_objects = []
mock_create_objects.side_effect = (
lambda tenant_id, model, objs, batch_size=500: created_objects.extend(
objs
)
)
create_compliance_requirements(str(tenant.id), str(scan.id))
assert len(created_objects) == 4
req_1_objects = [
obj for obj in created_objects if obj.requirement_id == "req_1"
]
req_2_objects = [
obj for obj in created_objects if obj.requirement_id == "req_2"
]
assert len(req_1_objects) == 2
assert len(req_2_objects) == 2
assert all(obj.requirement_status == "PASS" for obj in req_1_objects)
assert all(obj.requirement_status == "FAIL" for obj in req_2_objects)
+1 -1
View File
@@ -161,7 +161,7 @@ def update_nav_bar(pathname):
html.Span(
[
html.Img(src="assets/favicon.ico", className="w-5"),
"Subscribe to prowler SaaS",
"Subscribe to Prowler Cloud",
],
className="flex items-center gap-x-3 text-white",
),
+350
View File
@@ -2569,6 +2569,356 @@ def get_section_containers_3_levels(data, section_1, section_2, section_3):
return html.Div(section_containers, className="compliance-data-layout")
def get_section_containers_threatscore(data, section_1, section_2, section_3):
data["STATUS"] = data["STATUS"].apply(map_status_to_icon)
findings_counts_marco = (
data.groupby([section_1, "STATUS"]).size().unstack(fill_value=0)
)
section_containers = []
data[section_1] = data[section_1].astype(str)
data[section_2] = data[section_2].astype(str)
data[section_3] = data[section_3].astype(str)
data.sort_values(
by=section_3,
key=lambda x: x.map(extract_numeric_values),
ascending=True,
inplace=True,
)
for marco in data[section_1].unique():
success_marco = findings_counts_marco.loc[marco].get(pass_emoji, 0)
failed_marco = findings_counts_marco.loc[marco].get(fail_emoji, 0)
fig_name = go.Figure(
[
go.Bar(
name="Failed",
x=[failed_marco],
y=[""],
orientation="h",
marker=dict(color="#e77676"),
width=[0.8],
),
go.Bar(
name="Success",
x=[success_marco],
y=[""],
orientation="h",
marker=dict(color="#45cc6e"),
width=[0.8],
),
]
)
fig_name.update_layout(
barmode="stack",
margin=dict(l=10, r=10, t=10, b=10),
paper_bgcolor="rgba(0,0,0,0)",
plot_bgcolor="rgba(0,0,0,0)",
showlegend=False,
width=350,
height=30,
xaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
yaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
annotations=[
dict(
x=success_marco + failed_marco,
y=0,
xref="x",
yref="y",
text=str(success_marco),
showarrow=False,
font=dict(color="#45cc6e", size=14),
xanchor="left",
yanchor="middle",
),
dict(
x=0,
y=0,
xref="x",
yref="y",
text=str(failed_marco),
showarrow=False,
font=dict(color="#e77676", size=14),
xanchor="right",
yanchor="middle",
),
],
)
fig_name.add_annotation(
x=failed_marco,
y=0.3,
text="|",
showarrow=False,
font=dict(size=20),
xanchor="center",
yanchor="middle",
)
graph_div = html.Div(
dcc.Graph(
figure=fig_name, config={"staticPlot": True}, className="info-bar"
),
className="graph-section",
)
direct_internal_items = []
for categoria in data[data[section_1] == marco][section_2].unique():
specific_data = data[
(data[section_1] == marco) & (data[section_2] == categoria)
]
findings_counts_categoria = (
specific_data.groupby([section_2, "STATUS"])
.size()
.unstack(fill_value=0)
)
success_categoria = findings_counts_categoria.loc[categoria].get(
pass_emoji, 0
)
failed_categoria = findings_counts_categoria.loc[categoria].get(
fail_emoji, 0
)
fig_section = go.Figure(
[
go.Bar(
name="Failed",
x=[failed_categoria],
y=[""],
orientation="h",
marker=dict(color="#e77676"),
width=[0.8],
),
go.Bar(
name="Success",
x=[success_categoria],
y=[""],
orientation="h",
marker=dict(color="#45cc6e"),
width=[0.8],
),
]
)
fig_section.update_layout(
barmode="stack",
margin=dict(l=10, r=10, t=10, b=10),
paper_bgcolor="rgba(0,0,0,0)",
plot_bgcolor="rgba(0,0,0,0)",
showlegend=False,
width=350,
height=30,
xaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
yaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
annotations=[
dict(
x=success_categoria + failed_categoria,
y=0,
xref="x",
yref="y",
text=str(success_categoria),
showarrow=False,
font=dict(color="#45cc6e", size=14),
xanchor="left",
yanchor="middle",
),
dict(
x=0,
y=0,
xref="x",
yref="y",
text=str(failed_categoria),
showarrow=False,
font=dict(color="#e77676", size=14),
xanchor="right",
yanchor="middle",
),
],
)
fig_section.add_annotation(
x=failed_categoria,
y=0.3,
text="|",
showarrow=False,
font=dict(size=20),
xanchor="center",
yanchor="middle",
)
graph_div_section = html.Div(
dcc.Graph(
figure=fig_section,
config={"staticPlot": True},
className="info-bar-child",
),
className="graph-section-req",
)
direct_internal_items_idgrupocontrol = []
for idgrupocontrol in specific_data[section_3].unique():
specific_data2 = specific_data[
specific_data[section_3] == idgrupocontrol
]
findings_counts_idgrupocontrol = (
specific_data2.groupby([section_3, "STATUS"])
.size()
.unstack(fill_value=0)
)
success_idgrupocontrol = findings_counts_idgrupocontrol.loc[
idgrupocontrol
].get(pass_emoji, 0)
failed_idgrupocontrol = findings_counts_idgrupocontrol.loc[
idgrupocontrol
].get(fail_emoji, 0)
fig_idgrupocontrol = go.Figure(
[
go.Bar(
name="Failed",
x=[failed_idgrupocontrol],
y=[""],
orientation="h",
marker=dict(color="#e77676"),
width=[0.8],
),
go.Bar(
name="Success",
x=[success_idgrupocontrol],
y=[""],
orientation="h",
marker=dict(color="#45cc6e"),
width=[0.8],
),
]
)
fig_idgrupocontrol.update_layout(
barmode="stack",
margin=dict(l=10, r=10, t=10, b=10),
paper_bgcolor="rgba(0,0,0,0)",
plot_bgcolor="rgba(0,0,0,0)",
showlegend=False,
width=350,
height=30,
xaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
yaxis=dict(showticklabels=False, showgrid=False, zeroline=False),
annotations=[
dict(
x=success_idgrupocontrol + failed_idgrupocontrol,
y=0,
xref="x",
yref="y",
text=str(success_idgrupocontrol),
showarrow=False,
font=dict(color="#45cc6e", size=14),
xanchor="left",
yanchor="middle",
),
dict(
x=0,
y=0,
xref="x",
yref="y",
text=str(failed_idgrupocontrol),
showarrow=False,
font=dict(color="#e77676", size=14),
xanchor="right",
yanchor="middle",
),
],
)
fig_idgrupocontrol.add_annotation(
x=failed_idgrupocontrol,
y=0.3,
text="|",
showarrow=False,
font=dict(size=20),
xanchor="center",
yanchor="middle",
)
graph_div_idgrupocontrol = html.Div(
dcc.Graph(
figure=fig_idgrupocontrol,
config={"staticPlot": True},
className="info-bar-child",
),
className="graph-section-req",
)
data_table = dash_table.DataTable(
data=specific_data2.to_dict("records"),
columns=[
{"name": i, "id": i}
for i in [
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
],
style_table={"overflowX": "auto"},
style_as_list_view=True,
style_cell={"textAlign": "left", "padding": "5px"},
)
title_internal = f"{idgrupocontrol} - {specific_data2['REQUIREMENTS_DESCRIPTION'].iloc[0]}"
# Cut the title if it's too long
title_internal = (
title_internal[:130] + " ..."
if len(title_internal) > 130
else title_internal
)
internal_accordion_item_2 = dbc.AccordionItem(
title=title_internal,
children=[
graph_div_idgrupocontrol,
html.Div([data_table], className="inner-accordion-content"),
],
)
direct_internal_items_idgrupocontrol.append(
html.Div(
[
graph_div_idgrupocontrol,
dbc.Accordion(
[internal_accordion_item_2],
start_collapsed=True,
flush=True,
),
],
className="accordion-inner--child",
)
)
internal_accordion_item = dbc.AccordionItem(
title=categoria,
children=direct_internal_items_idgrupocontrol,
)
internal_section_container = html.Div(
[
graph_div_section,
dbc.Accordion(
[internal_accordion_item], start_collapsed=True, flush=True
),
],
className="accordion-inner--child",
)
direct_internal_items.append(internal_section_container)
accordion_item = dbc.AccordionItem(title=marco, children=direct_internal_items)
section_container = html.Div(
[
graph_div,
dbc.Accordion([accordion_item], start_collapsed=True, flush=True),
],
className="accordion-inner",
)
section_containers.append(section_container)
return html.Div(section_containers, className="compliance-data-layout")
# This function extracts and compares up to two numeric values, ensuring correct sorting for version-like strings.
def extract_numeric_values(value):
numbers = re.findall(r"\d+", str(value))
@@ -0,0 +1,24 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
warnings.filterwarnings("ignore")
def get_table(data):
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
)
+24
View File
@@ -0,0 +1,24 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
warnings.filterwarnings("ignore")
def get_table(data):
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
)
+24
View File
@@ -0,0 +1,24 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
warnings.filterwarnings("ignore")
def get_table(data):
aux = data[
[
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
)
+43
View File
@@ -0,0 +1,43 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_DESCRIPTION"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SECTION"
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
aux = data[
[
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_DESCRIPTION",
)
+43
View File
@@ -0,0 +1,43 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_DESCRIPTION"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SECTION"
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
aux = data[
[
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_DESCRIPTION",
)
+43
View File
@@ -0,0 +1,43 @@
import warnings
from dashboard.common_methods import get_section_containers_3_levels
warnings.filterwarnings("ignore")
def get_table(data):
data["REQUIREMENTS_DESCRIPTION"] = (
data["REQUIREMENTS_ID"] + " - " + data["REQUIREMENTS_DESCRIPTION"]
)
data["REQUIREMENTS_DESCRIPTION"] = data["REQUIREMENTS_DESCRIPTION"].apply(
lambda x: x[:150] + "..." if len(str(x)) > 150 else x
)
data["REQUIREMENTS_ATTRIBUTES_SECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SECTION"
].apply(lambda x: x[:80] + "..." if len(str(x)) > 80 else x)
data["REQUIREMENTS_ATTRIBUTES_SUBSECTION"] = data[
"REQUIREMENTS_ATTRIBUTES_SUBSECTION"
].apply(lambda x: x[:150] + "..." if len(str(x)) > 150 else x)
aux = data[
[
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
"ACCOUNTID",
"RESOURCEID",
]
]
return get_section_containers_3_levels(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_DESCRIPTION",
)
@@ -1,6 +1,6 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
from dashboard.common_methods import get_section_containers_threatscore
warnings.filterwarnings("ignore")
@@ -11,6 +11,7 @@ def get_table(data):
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
@@ -19,6 +20,9 @@ def get_table(data):
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
return get_section_containers_threatscore(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
@@ -1,6 +1,6 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
from dashboard.common_methods import get_section_containers_threatscore
warnings.filterwarnings("ignore")
@@ -11,6 +11,7 @@ def get_table(data):
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
@@ -19,6 +20,9 @@ def get_table(data):
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
return get_section_containers_threatscore(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
@@ -1,6 +1,6 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
from dashboard.common_methods import get_section_containers_threatscore
warnings.filterwarnings("ignore")
@@ -11,6 +11,7 @@ def get_table(data):
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
@@ -19,6 +20,9 @@ def get_table(data):
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
return get_section_containers_threatscore(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
@@ -1,6 +1,6 @@
import warnings
from dashboard.common_methods import get_section_containers_cis
from dashboard.common_methods import get_section_containers_threatscore
warnings.filterwarnings("ignore")
@@ -11,6 +11,7 @@ def get_table(data):
"REQUIREMENTS_ID",
"REQUIREMENTS_DESCRIPTION",
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"CHECKID",
"STATUS",
"REGION",
@@ -19,6 +20,9 @@ def get_table(data):
]
].copy()
return get_section_containers_cis(
aux, "REQUIREMENTS_ID", "REQUIREMENTS_ATTRIBUTES_SECTION"
return get_section_containers_threatscore(
aux,
"REQUIREMENTS_ATTRIBUTES_SECTION",
"REQUIREMENTS_ATTRIBUTES_SUBSECTION",
"REQUIREMENTS_ID",
)
+23 -7
View File
@@ -90,12 +90,28 @@ def create_layout_overview(
),
html.Div(
[
(
html.Label(
"Table Rows:",
className="text-prowler-stone-900 font-bold text-sm",
style={"margin-right": "10px"},
)
html.Label(
"Search:",
className="text-prowler-stone-900 font-bold text-sm",
style={"margin-right": "10px"},
),
dcc.Input(
id="search-input",
type="text",
placeholder="Search by check title, service, region...",
debounce=True,
style={
"padding": "4px 8px",
"border": "1px solid #ccc",
"borderRadius": "4px",
"marginRight": "20px",
"width": "250px",
},
),
html.Label(
"Table Rows:",
className="text-prowler-stone-900 font-bold text-sm",
style={"margin-right": "10px"},
),
table_row_dropdown,
download_button_csv,
@@ -132,7 +148,7 @@ def create_layout_compliance(
html.A(
[
html.Img(src="assets/favicon.ico", className="w-5 mr-3"),
html.Span("Subscribe to prowler SaaS"),
html.Span("Subscribe to Prowler Cloud"),
],
href="https://prowler.pro/",
target="_blank",
+102 -43
View File
@@ -651,58 +651,114 @@ def get_table(current_compliance, table):
def get_threatscore_mean_by_pillar(df):
modified_df = df[df["STATUS"] == "FAIL"]
score_per_pillar = {}
max_score_per_pillar = {}
modified_df["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"] = pd.to_numeric(
modified_df["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"], errors="coerce"
)
for _, row in df.iterrows():
pillar = (
row["REQUIREMENTS_ATTRIBUTES_SECTION"].split(" - ")[0]
if isinstance(row["REQUIREMENTS_ATTRIBUTES_SECTION"], str)
else "Unknown"
)
pillar_means = (
modified_df.groupby("REQUIREMENTS_ATTRIBUTES_SECTION")[
"REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"
]
.mean()
.round(2)
)
if pillar not in score_per_pillar:
score_per_pillar[pillar] = 0
max_score_per_pillar[pillar] = 0
level_of_risk = pd.to_numeric(
row["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"], errors="coerce"
)
level_of_risk = 1 if pd.isna(level_of_risk) else level_of_risk
weight = 1
if "REQUIREMENTS_ATTRIBUTES_WEIGHT" in row and not pd.isna(
row["REQUIREMENTS_ATTRIBUTES_WEIGHT"]
):
weight = pd.to_numeric(
row["REQUIREMENTS_ATTRIBUTES_WEIGHT"], errors="coerce"
)
weight = 1 if pd.isna(weight) else weight
max_score_per_pillar[pillar] += level_of_risk * weight
if row["STATUS"] == "PASS":
score_per_pillar[pillar] += level_of_risk * weight
output = []
for pillar, mean in pillar_means.items():
output.append(f"{pillar} - [{mean}]")
for pillar in max_score_per_pillar:
risk_score = 0
if max_score_per_pillar[pillar] > 0:
risk_score = (score_per_pillar[pillar] / max_score_per_pillar[pillar]) * 100
output.append(f"{pillar} - [{risk_score:.1f}%]")
for value in output:
if value.split(" - ")[0] in df["REQUIREMENTS_ATTRIBUTES_SECTION"].values:
base_pillar = value.split(" - ")[0]
if base_pillar in df["REQUIREMENTS_ATTRIBUTES_SECTION"].values:
df.loc[
df["REQUIREMENTS_ATTRIBUTES_SECTION"] == value.split(" - ")[0],
df["REQUIREMENTS_ATTRIBUTES_SECTION"] == base_pillar,
"REQUIREMENTS_ATTRIBUTES_SECTION",
] = value
return df
def get_table_prowler_threatscore(df):
df = df[df["STATUS"] == "FAIL"]
score_per_pillar = {}
max_score_per_pillar = {}
pillars = {}
# Delete " - " from the column REQUIREMENTS_ATTRIBUTES_SECTION
df["REQUIREMENTS_ATTRIBUTES_SECTION"] = (
df["REQUIREMENTS_ATTRIBUTES_SECTION"].str.split(" - ").str[0]
)
df_copy = df.copy()
df["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"] = pd.to_numeric(
df["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"], errors="coerce"
)
score_df = (
df.groupby("REQUIREMENTS_ATTRIBUTES_SECTION")[
"REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"
]
.mean()
.reset_index()
.rename(
columns={
"REQUIREMENTS_ATTRIBUTES_SECTION": "Pillar",
"REQUIREMENTS_ATTRIBUTES_LEVELOFRISK": "Score",
}
for _, row in df_copy.iterrows():
pillar = (
row["REQUIREMENTS_ATTRIBUTES_SECTION"].split(" - ")[0]
if isinstance(row["REQUIREMENTS_ATTRIBUTES_SECTION"], str)
else "Unknown"
)
)
if pillar not in pillars:
pillars[pillar] = {"FAIL": 0, "PASS": 0, "MUTED": 0}
score_per_pillar[pillar] = 0
max_score_per_pillar[pillar] = 0
level_of_risk = pd.to_numeric(
row["REQUIREMENTS_ATTRIBUTES_LEVELOFRISK"], errors="coerce"
)
level_of_risk = 1 if pd.isna(level_of_risk) else level_of_risk
weight = 1
if "REQUIREMENTS_ATTRIBUTES_WEIGHT" in row and not pd.isna(
row["REQUIREMENTS_ATTRIBUTES_WEIGHT"]
):
weight = pd.to_numeric(
row["REQUIREMENTS_ATTRIBUTES_WEIGHT"], errors="coerce"
)
weight = 1 if pd.isna(weight) else weight
max_score_per_pillar[pillar] += level_of_risk * weight
if row["STATUS"] == "PASS":
pillars[pillar]["PASS"] += 1
score_per_pillar[pillar] += level_of_risk * weight
elif row["STATUS"] == "FAIL":
pillars[pillar]["FAIL"] += 1
if "MUTED" in row and row["MUTED"] == "True":
pillars[pillar]["MUTED"] += 1
result_df = []
for pillar in pillars.keys():
risk_score = 0
if max_score_per_pillar[pillar] > 0:
risk_score = (score_per_pillar[pillar] / max_score_per_pillar[pillar]) * 100
result_df.append({"Pillar": pillar, "Score": risk_score})
score_df = pd.DataFrame(result_df)
score_df = score_df.sort_values("Score", ascending=True)
fig = px.bar(
score_df,
@@ -710,22 +766,25 @@ def get_table_prowler_threatscore(df):
y="Score",
color="Score",
color_continuous_scale=[
"#45cc6e",
"#f4d44d",
"#e77676",
], # verde → amarillo → rojo
hover_data={"Score": True, "Pillar": True},
labels={"Score": "Average Risk Score", "Pillar": "Section"},
"#f4d44d",
"#45cc6e",
],
labels={"Score": "Risk Score (%)", "Pillar": "Section"},
height=400,
text="Score",
)
fig.update_traces(texttemplate="%{text:.1f}%", textposition="outside")
fig.update_layout(
xaxis_title="Pillar",
yaxis_title="Level of Risk",
yaxis_title="Risk Score (%)",
margin=dict(l=20, r=20, t=30, b=20),
plot_bgcolor="rgba(0,0,0,0)",
paper_bgcolor="rgba(0,0,0,0)",
coloraxis_colorbar=dict(title="Risk"),
coloraxis_colorbar=dict(title="Risk %"),
yaxis=dict(range=[0, 110]),
)
return dcc.Graph(
+25 -4
View File
@@ -83,7 +83,18 @@ def load_csv_files(csv_files):
"""Load CSV files into a single pandas DataFrame."""
dfs = []
for file in csv_files:
df = pd.read_csv(file, sep=";", on_bad_lines="skip")
account_columns = ["ACCOUNT_ID", "ACCOUNT_UID", "SUBSCRIPTION"]
df_sample = pd.read_csv(file, sep=";", on_bad_lines="skip", nrows=1)
dtype_dict = {}
for col in account_columns:
if col in df_sample.columns:
dtype_dict[col] = str
# Read the full file with proper dtypes
df = pd.read_csv(file, sep=";", on_bad_lines="skip", dtype=dtype_dict)
if "CHECK_ID" in df.columns:
if "TIMESTAMP" in df.columns or df["PROVIDER"].unique() == "aws":
dfs.append(df.astype(str))
@@ -120,7 +131,6 @@ if data is None:
]
)
else:
# This handles the case where we are using v3 outputs
if "ASSESSMENT_START_TIME" in data.columns:
data["ASSESSMENT_START_TIME"] = data["ASSESSMENT_START_TIME"].str.replace(
@@ -518,6 +528,7 @@ else:
Input("service-filter", "value"),
Input("table-rows", "value"),
Input("status-filter", "value"),
Input("search-input", "value"),
Input("aws_card", "n_clicks"),
Input("azure_card", "n_clicks"),
Input("gcp_card", "n_clicks"),
@@ -540,6 +551,7 @@ def filter_data(
service_values,
table_row_values,
status_values,
search_value,
aws_clicks,
azure_clicks,
gcp_clicks,
@@ -1144,6 +1156,15 @@ def filter_data(
}
index_count = 0
if search_value:
search_value = search_value.lower()
filtered_data = filtered_data[
filtered_data["CHECK_TITLE"].str.lower().str.contains(search_value)
| filtered_data["SERVICE_NAME"].str.lower().str.contains(search_value)
| filtered_data["REGION"].str.lower().str.contains(search_value)
| filtered_data["STATUS"].str.lower().str.contains(search_value)
]
full_filtered_data = filtered_data.copy()
filtered_data = filtered_data.head(table_row_values)
# Sort the filtered_data
@@ -1342,13 +1363,13 @@ def filter_data(
"m365", m365_provider_logo, "Accounts", full_filtered_data
)
# Subscribe to prowler SaaS card
# Subscribe to Prowler Cloud card
subscribe_card = [
html.Div(
html.A(
[
html.Img(src="assets/favicon.ico", className="w-5 mr-3"),
html.Span("Subscribe to prowler SaaS"),
html.Span("Subscribe to Prowler Cloud"),
],
href="https://prowler.pro/",
target="_blank",
+21 -59
View File
@@ -70,7 +70,7 @@ The other three cases does not need additional configuration, `--az-cli-auth` an
Prowler for Azure needs two types of permission scopes to be set:
- **Microsoft Entra ID permissions**: used to retrieve metadata from the identity assumed by Prowler and specific Entra checks (not mandatory to have access to execute the tool). The permissions required by the tool are the following:
- `Directory.Read.All`
- `Domain.Read.All`
- `Policy.Read.All`
- `UserAuthenticationMethod.Read.All` (used only for the Entra checks related with multifactor authentication)
- **Subscription scope permissions**: required to launch the checks against your resources, mandatory to launch the tool. It is required to add the following RBAC builtin roles per subscription to the entity that is going to be assumed by the tool:
@@ -81,6 +81,9 @@ Prowler for Azure needs two types of permission scopes to be set:
To assign the permissions, follow the instructions in the [Microsoft Entra ID permissions](../tutorials/azure/create-prowler-service-principal.md#assigning-the-proper-permissions) section and the [Azure subscriptions permissions](../tutorials/azure/subscriptions.md#assign-the-appropriate-permissions-to-the-identity-that-is-going-to-be-assumed-by-prowler) section, respectively.
???+ warning
Some permissions in `ProwlerRole` are considered **write** permissions, so if you have a `ReadOnly` lock attached to some resources you may get an error and will not get a finding for that check.
#### Checks that require ProwlerRole
The following checks require the `ProwlerRole` permissions to be executed, if you want to run them, make sure you have assigned the role to the identity that is going to be assumed by Prowler:
@@ -153,77 +156,34 @@ With this credentials you will only be able to run the checks that work through
Authentication flag: `--env-auth`
This authentication method follows the same approach as the service principal method but introduces two additional environment variables for user credentials: `M365_USER` and `M365_ENCRYPTED_PASSWORD`.
This authentication method follows the same approach as the service principal method but introduces two additional environment variables for user credentials: `M365_USER` and `M365_PASSWORD`.
```console
export AZURE_CLIENT_ID="XXXXXXXXX"
export AZURE_CLIENT_SECRET="XXXXXXXXX"
export AZURE_TENANT_ID="XXXXXXXXX"
export M365_USER="your_email@example.com"
export M365_ENCRYPTED_PASSWORD="6500780061006d0070006c006500700061007300730077006f0072006400" # replace this to yours
export M365_PASSWORD="examplepassword"
```
These two new environment variables are **required** to execute the PowerShell modules needed to retrieve information from M365 services. Prowler uses Service Principal authentication to access Microsoft Graph and user credentials to authenticate to Microsoft PowerShell modules.
- `M365_USER` should be your Microsoft account email using the default domain. This means it must look like `example@YourCompany.onmicrosoft.com`.
- `M365_USER` should be your Microsoft account email using the **assigned domain in the tenant**. This means it must look like `example@YourCompany.onmicrosoft.com` or `example@YourCompany.com`, but it must be the exact domain assigned to that user in the tenant.
To ensure that you are using the default domain you can see how to verify it [here](../tutorials/microsoft365/getting-started-m365.md#step-1-obtain-your-domain).
???+ warning
The user must not be MFA capable. Microsoft does not allow MFA capable users to authenticate programmatically. See [Microsoft documentation](https://learn.microsoft.com/en-us/entra/identity-platform/scenario-desktop-acquire-token-username-password?tabs=dotnet) for more information.
If you don't have a user created with that domain, Prowler will not work as it will not be able to ensure both app an user belong to the same tenant. To proceed, you can either create a new user with that domain or modify the domain of an existing user.
???+ warning
Using a tenant domain other than the one assigned — even if it belongs to the same tenant — will cause Prowler to fail, as Microsoft authentication will not succeed.
Ensure you are using the right domain for the user you are trying to authenticate with.
![User Domains](../tutorials/microsoft365/img/user-domains.png)
- `M365_ENCRYPTED_PASSWORD` must be an encrypted SecureString. To convert your password into a valid encrypted string, you need to use PowerShell.
???+ warning
Passwords encrypted using ConvertTo-SecureString can only be decrypted on the same OS/user context. If you generate an encrypted password on macOS or Linux (both UNIX), it should fail on Windows and vice versa. As Prowler Cloud runs on UNIX if you generate your password using Windows it won't work so you'll need to generate a new password using any UNIX distro (example above)
If you are working from Windows and you will use your encrypted password in a different system (like for example executing Prowler in macOS or adding your password to Prowler Cloud), you will need to generate a "UNIX compatible" version of your encrypted password. This can be done using WSL which is so easy to install on Windows.
=== "UNIX"
Open a PowerShell cmd with a [supported version](requirements.md#supported-powershell-versions) and then run the following command:
```console
$securePassword = ConvertTo-SecureString "examplepassword" -AsPlainText -Force
$encryptedPassword = $securePassword | ConvertFrom-SecureString
Write-Output $encryptedPassword
6500780061006d0070006c006500700061007300730077006f0072006400
```
If everything is done correctly, you will see the encrypted string that you need to set as the `M365_ENCRYPTED_PASSWORD` environment variable.
=== "Windows"
How to install WSL and PowerShell on it to generate that password (you can use a different distro but this one will work for sure):
```console
wsl --install -d Ubuntu-22.04
```
Then, open the Ubuntu terminal and run the following commands:
```console
sudo apt update && sudo apt install -y wget apt-transport-https software-properties-common
wget -q "https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb"
sudo dpkg -i packages-microsoft-prod.deb
sudo apt update
sudo apt install -y powershell
pwsh
```
With this done you will see now that a prompt running PowerShell with the latest version is open so here you will be able to generate your encrypted password:
```console
$securePassword = ConvertTo-SecureString "examplepassword" -AsPlainText -Force
$encryptedPassword = $securePassword | ConvertFrom-SecureString
Write-Output $encryptedPassword
6500780061006d0070006c006500700061007300730077006f0072006400
```
If everything is done correctly, you will see the encrypted string that you need to set as the `M365_ENCRYPTED_PASSWORD` environment variable.
- `M365_PASSWORD` must be the user password.
???+ note
Before we asked for a encrypted password, but now we ask for the user password directly. Prowler will now handle the password encryption for you.
### Interactive Browser authentication
@@ -242,11 +202,13 @@ Since this is a delegated permission authentication method, necessary permission
Prowler for M365 requires two types of permission scopes to be set (if you want to run the full provider including PowerShell checks). Both must be configured using Microsoft Entra ID:
- **Service Principal Application Permissions**: These are set at the **application** level and are used to retrieve data from the identity being assessed:
- `Directory.Read.All`: Required for all services.
- `AuditLog.Read.All`: Required for Entra service.
- `Domain.Read.All`: Required for all services.
- `Organization.Read.All`: Required for retrieving tenant information.
- `Policy.Read.All`: Required for all services.
- `User.Read` (IMPORTANT: this must be set as **delegated**): Required for the sign-in.
- `Sites.Read.All`: Required for SharePoint service.
- `SharePointTenantSettings.Read.All`: Required for SharePoint service.
- `User.Read` (IMPORTANT: this must be set as **delegated**): Required for the sign-in.
- **Powershell Modules Permissions**: These are set at the `M365_USER` level, so the user used to run Prowler must have one of the following roles:
- `Global Reader` (recommended): this allows you to read all roles needed.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 376 KiB

After

Width:  |  Height:  |  Size: 383 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 347 KiB

After

Width:  |  Height:  |  Size: 119 KiB

@@ -40,7 +40,7 @@ az ad sp create-for-rbac --name "ProwlerApp"
To allow Prowler to retrieve metadata from the identity assumed and run specific Entra checks, it is needed to assign the following permissions:
- `Directory.Read.All`
- `Domain.Read.All`
- `Policy.Read.All`
- `UserAuthenticationMethod.Read.All` (used only for the Entra checks related with multifactor authentication)
@@ -58,7 +58,7 @@ To assign the permissions you can make it from the Azure Portal or using the Azu
5. Then click on "+ Add a permission" and select "Microsoft Graph"
6. Once in the "Microsoft Graph" view, select "Application permissions"
7. Finally, search for "Directory", "Policy" and "UserAuthenticationMethod" select the following permissions:
- `Directory.Read.All`
- `Domain.Read.All`
- `Policy.Read.All`
- `UserAuthenticationMethod.Read.All`
8. Click on "Add permissions" to apply the new permissions.
@@ -90,7 +90,7 @@ A Service Principal is required to grant Prowler the necessary privileges.
Assign the following Microsoft Graph permissions:
- Directory.Read.All
- Domain.Read.All
- Policy.Read.All
@@ -107,11 +107,11 @@ Assign the following Microsoft Graph permissions:
3. Search and select:
- `Directory.Read.All`
- `Domain.Read.All`
- `Policy.Read.All`
- `UserAuthenticationMethod.Read.All`
![Permission Screenshots](./img/directory-permission.png)
![Permission Screenshots](./img/domain-permission.png)
4. Click `Add permissions`, then grant admin consent
Binary file not shown.

Before

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 160 KiB

+14
View File
@@ -109,6 +109,15 @@ The following list includes all the Microsoft 365 checks with configurable varia
| `exchange_organization_mailtips_enabled` | `recommended_mailtips_large_audience_threshold` | Integer |
## GitHub
### Configurable Checks
The following list includes all the GitHub checks with configurable variables that can be changed in the configuration yaml file:
| Check Name | Value | Type |
|--------------------------------------------|---------------------------------------------|---------|
| `repository_inactive_not_archived` | `inactive_not_archived_days_threshold` | Integer |
## Config YAML File Structure
???+ note
@@ -525,5 +534,10 @@ m365:
# m365.exchange_organization_mailtips_enabled
recommended_mailtips_large_audience_threshold: 25 # maximum number of recipients
# GitHub Configuration
github:
# github.repository_inactive_not_archived
inactive_not_archived_days_threshold: 180
```
@@ -1,7 +1,5 @@
# Getting Started with GCP on Prowler Cloud/App
<iframe width="560" height="380" src="https://www.youtube-nocookie.com/embed/v1as8vTFlMg" title="Prowler Cloud Onboarding GCP" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="1"></iframe>
Set up your GCP project to enable security scanning using Prowler Cloud/App.
## Requirements
+20
View File
@@ -21,3 +21,23 @@ To specify the namespace(s) to be scanned, use the `--namespace` flag followed b
```console
prowler --namespace namespace1 namespace2
```
## Proxy and TLS Verification
If your Kubernetes cluster is only accessible via an internal proxy, Prowler will respect the `HTTPS_PROXY` or `https_proxy` environment variable:
```console
export HTTPS_PROXY=http://my.internal.proxy:8888
prowler kubernetes ...
```
If you need to skip TLS verification for internal proxies, you can set the `K8S_SKIP_TLS_VERIFY` environment variable:
```console
export K8S_SKIP_TLS_VERIFY=true
prowler kubernetes ...
```
This will allow Prowler to connect to the cluster even if the proxy uses a self-signed certificate.
These environment variables are supported both when using an external `kubeconfig` and in in-cluster mode.
@@ -4,9 +4,9 @@ Set up your M365 account to enable security scanning using Prowler Cloud/App.
## Requirements
To configure your M365 account, you’ll need:
To configure your M365 account, you'll need:
1. Obtain your `Default Domain` from the Entra ID portal.
1. Obtain a domain from the Entra ID portal.
2. Access Prowler Cloud/App and add a new cloud provider `Microsoft 365`.
@@ -18,8 +18,6 @@ To configure your M365 account, you’ll need:
3.3 Assign the required roles to your user.
3.4 Retrieve your encrypted password.
4. Add the credentials to Prowler Cloud/App.
## Step 1: Obtain your Domain
@@ -32,9 +30,7 @@ Go to the Entra ID portal, then you can search for `Domain` or go to Identity >
![Custom Domain Names](./img/custom-domain-names.png)
Once you are there just look for the `Default Domain` this should be something similar to `YourCompany.onmicrosoft.com`. To ensure that you are picking the correct domain just click on it and verify that the type is `Initial` and you can't delete it.
![Search Default Domain](./img/search-default-domain.png)
Once you are there just select the domain you want to use.
---
@@ -78,11 +74,11 @@ A Service Principal is required to grant Prowler the necessary privileges.
![New Registration](./img/new-registration.png)
4. Go to `Certificates & secrets` > `+ New client secret`
4. Go to `Certificates & secrets` > `Client secrets` > `+ New client secret`
![Certificate & Secrets nav](./img/certificates-and-secrets.png)
5. Fill in the required fields and click `Add`, then copy the generated value (that value will be `AZURE_CLIENT_SECRET`)
5. Fill in the required fields and click `Add`, then copy the generated `value` (that value will be `AZURE_CLIENT_SECRET`)
![New Client Secret](./img/new-client-secret.png)
@@ -99,12 +95,11 @@ With this done you will have all the needed keys, summarized in the following ta
### Grant required API permissions
Assign the following Microsoft Graph permissions:
- `Directory.Read.All`: Required for all services.
- `AuditLog.Read.All`: Required for Entra service.
- `Domain.Read.All`: Required for all services.
- `Policy.Read.All`: Required for all services.
- `User.Read` (IMPORTANT: this is set as **delegated**): Required for the sign-in.
- `Sites.Read.All`: Required for SharePoint service.
- `SharePointTenantSettings.Read.All`: Required for SharePoint service.
- `User.Read` (IMPORTANT: this is set as **delegated**): Required for the sign-in.
Follow these steps to assign the permissions:
@@ -117,31 +112,35 @@ Follow these steps to assign the permissions:
![Add API Permission](./img/add-app-api-permission.png)
3. Search and select every permission below and once all are selected click on `Add permissions`:
- `Directory.Read.All`
- `AuditLog.Read.All`: Required for Entra service.
- `Domain.Read.All`
- `Organization.Read.All`
- `Policy.Read.All`
- `Sites.Read.All`
- `SharePointTenantSettings.Read.All`
![Permission Screenshots](./img/directory-permission.png)
4. Click `Add permissions`, then grant admin consent
![Application Permissions](./img/app-permissions.png)
![Grant Admin Consent](./img/grant-admin-consent.png)
5. Click `+ Add a permission` > `Microsoft Graph` > `Delegated permissions`
4. Click `+ Add a permission` > `Microsoft Graph` > `Delegated permissions`
![Add API Permission](./img/add-delegated-api-permission.png)
6. Search and select:
5. Search and select:
- `User.Read`
![Permission Screenshots](./img/directory-permission-delegated.png)
7. Click `Add permissions`, then grant admin consent
6. Click `Add permissions`, then **grant admin consent**
![Grant Admin Consent](./img/grant-admin-consent-delegated.png)
![Grant Admin Consent](./img/grant-admin-consent.png)
The final result of permission assignment should be this:
![Final Permission Assignment](./img/final-permissions-m365.png)
---
@@ -174,25 +173,20 @@ Follow these steps to assign the role:
---
### Get your encrypted password
For this step you will need to use PowerShell, here you will have to create your Encrypted Password based on the password of the User that you are going to use. For more information about how to generate this Password go [here](../../getting-started/requirements.md#service-principal-and-user-credentials-authentication-recommended) and follow the steps needed to obtain `M365_ENCRYPTED_PASSWORD`.
---
## Step 4: Add credentials to Prowler Cloud/App
1. Go to your App Registration overview and copy the `Client ID` and `Tenant ID`
![App Overview](./img/app-overview.png)
2. Go to Prowler Cloud/App and paste:
- `Client ID`
- `Tenant ID`
- `AZURE_CLIENT_SECRET` from earlier
- `M365_USER` your user using the default domain, more info [here](../../getting-started/requirements.md#service-principal-and-user-credentials-authentication-recommended)
- `M365_ENCRYPTED_PASSWORD` generated before
- `M365_USER` the user using the correct assigned domain, more info [here](../../getting-started/requirements.md#service-principal-and-user-credentials-authentication-recommended)
- `M365_PASSWORD` the password of the user
![Prowler Cloud M365 Credentials](./img/m365-credentials.png)
Binary file not shown.

After

Width:  |  Height:  |  Size: 159 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 63 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 459 KiB

After

Width:  |  Height:  |  Size: 453 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 438 KiB

After

Width:  |  Height:  |  Size: 183 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 347 KiB

After

Width:  |  Height:  |  Size: 119 KiB

+2 -2
View File
@@ -38,11 +38,11 @@ prowler <provider> --list-checks
```
- Execute specific check(s):
```console
prowler <provider> -c/--checks s3_bucket_public_access
prowler <provider> -c/--checks s3_bucket_public_access iam_root_mfa_enabled
```
- Exclude specific check(s):
```console
prowler <provider> -e/--excluded-checks ec2 rds
prowler <provider> -e/--excluded-checks s3_bucket_public_access iam_root_mfa_enabled
```
- Execute checks that appears in a json file:
```json
Loaded 100 of 476 files, more files were not shown because too many files have changed in this diff. Show more