266 Commits

Author SHA1 Message Date
Toni de la Fuente 72d6d3f535 feat(inventory): Prowler quick inventory including IAM resources (#1258)
* chore(inventory): option included in main

* chore(inventory): quick inventory

* chore(inventory): functional version

* chore(inventory): functional version without echo

* Update include/quick_inventory

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* Update prowler

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>

* Added new line at report line

* Added more information from IAM

Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>
2022-07-21 10:37:28 +02:00
Mitch ddd34dc9cc fix(extra7173): Correct check and alternative name (#1270) 2022-07-20 08:36:34 +02:00
Sergio Garcia 03b1c10d13 fix(codebuild): expired token error using Instance Metadata
Co-authored-by: sergargar <sergio@verica.io>
2022-07-14 07:32:01 +02:00
Sergio Garcia 4cd5b8fd04 fix(codebuild): expired token error (#1262) 2022-07-12 07:38:44 +02:00
Toni de la Fuente f691046c1f feat(inventory): Prowler quick inventory (#1245)
* chore(inventory): option included in main

* chore(inventory): quick inventory

* chore(inventory): functional version

* chore(inventory): functional version without echo

* Update include/quick_inventory

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* Update prowler

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>

* Added new line at report line

Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>
2022-07-08 12:41:54 +02:00
Sergio Garcia fd850790d5 fix(add-checks-regions): Missing regions in checks (#1247)
* add regions to checks

* add root as resource

Co-authored-by: sergargar <sergio@verica.io>
2022-07-04 09:46:08 +02:00
Sergio Garcia 912d5d7f8c fix(postgres): Fix postgres connector issues. (#1244)
* fix(postgres): Fix postgres connector issues.

* fix(postgres): Update documentation

Co-authored-by: sergargar <sergio@verica.io>
2022-06-30 18:12:33 +02:00
Pepe Fagoaga d88a136ac3 feat(db-connector): Include env variables (#1236)
* feat(db-connector): Include env variables

* fix(typo)

* fix(psql-test): Remove PGPASSWORD
2022-06-30 08:43:41 +02:00
Pepe Fagoaga 821083639a fix(bckCredentials): Do nothing if no initial creds (#1239) 2022-06-29 16:52:08 +02:00
Sergio Garcia 0ad149942b fix(security_hub_integration): Treat failed findings as failed in Security Hub (#1219)
Co-authored-by: sergargar <sergio@verica.io>
2022-06-22 14:03:16 +02:00
Nacho Rivera 66305768c0 fix(instance metadata): Missing raw flag in JQ parser (#1214) 2022-06-21 10:14:12 +02:00
Sergio Garcia 05f98fe993 fix(junit_xml output): Fix XML output integration (#1210)
Co-authored-by: sergargar <sergio@verica.io>
2022-06-20 13:27:54 +02:00
Sergio Garcia 633d7bd8a8 fix(instance-metadata): Credentials recovering (#1207)
* fix(instance-metadata): Credentials recovering

* fix(expr): Dockerfile to root and expr in SESSION_TIME_REMAINING.

Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: sergargar <sergio@verica.io>
2022-06-17 14:23:56 +02:00
Pepe Fagoaga 3944ea2055 fix(session_duration): Use jq with TZ=UTC (#1195) 2022-06-15 13:25:43 +02:00
Pepe Fagoaga d32a7986a5 fix(shellcheck): Main variables (#1194) 2022-06-14 10:43:15 +02:00
Pepe Fagoaga da000b54ca refactor(Prowler): Main logic refactor (#1189)
* fix(aws_profile_loader): New functions

* fix(shellcheck): Temporary remove Shellcheck

* fix(aws_cli_detector): new function

* fix(jq_detector): New function

* fix(os_detector): New function

* fix(output_bucket): Output bucket input check in main

* fix(python_detector): deleted unused python detector

* fix(credentials): credentials check out of whoami

* [break]refactor(main)

* [BREAK] Get list of checks parsing all input options

* [break]refactor(main): execute checks functions

* [break]refactor(main): move functions to libs

* fix(validations): custom check validation and typos

* refactor(validate_options): Include comments

* fix(custom_checks): Minor fixes

* refactor(closing_files): include libraries

* refactor(loader): Include ignored checks

* refactor(main): Fix shellcheck

* refactor(loader): beautify

* refactor(monochrome): without variables

* refactor(modes): MODES array not needed

* refactor(whoami): get error from AWSCLI

* refactor(secrets-detector)

* refactor(secrets-detector)

* fix(html_scoring): html scoring was fixed.

* fix(load_checks_from_file)

* fix(color-code): Print if not mono

* fix(not extra): Fixed if EXCLUDE_CHECK_ID is empty

* fix(IFS): Restore default IFS once modes are parsed

* fix(bucket): validate before whoami

* fix(bucket): validate before whoami

Co-authored-by: n4ch04 <nachor1992@gmail.com>
Co-authored-by: sergargar <sergio@verica.io>
Co-authored-by: Nacho Rivera <59198746+n4ch04@users.noreply.github.com>
2022-06-13 17:34:31 +02:00
Nacho Rivera f9322ab3aa fix(outputs): Replace each comma occurrence before sending to csv file (#1188) 2022-06-08 09:19:50 +02:00
Sergio Garcia 50a670fbc4 fix(codebuild_update): AWS CLI and permissions update. (#1183) 2022-06-07 14:49:22 +02:00
Nacho Rivera bc56c4242e refactor(outputs): Consolidate Prowler output functions (#1180)
* chore(db providers): db providers first version

* chore(db provider): added db provider setup into Readme

* fix(csv_line): csv_line out of conditional

* fix(README): text instead of varchar in table

* fix(help): help message extended

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>

* fix(typo): Update README.md

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(table): add if not exists

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(typo): Readme postgreSQL

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(db_connector): details to add a new provider

* fix(typo): Uppercase Prowler

Co-authored-by: Toni de la Fuente <toni@blyx.com>

* fix(prowler): deleted unused variable

* chore(checks): test db connector previous to send data

* chore(input tests): input tests moved to main

* fix(typo): Readme typos

* chore(table): table name from pgpass file

* fix(grep test): Added missing -E flag

* chore(table): check of table name and Readme

* chore(error colors): Added error colors

* chore(inputcheck): checks about mode and output inputs into main

* fix(inputs) custom output file name

* fix(outputs): comment profile

* chore(textXXX): both 3 textfunctions using general

* fix(allowlist): allowlist check included as function

* fix(headers): Add headers to certain output files

* fix(reformulate): change structure and delete comments

* fix(testing): Input test after load includes

* fix(variables): Added named vars

* fix(colors): Deleted unused colors

* fix(outputs): fine tuning

* fix(outputs): allowlist parameters read

* fix(allowlist): allowlist logic reformulated

* fix(REPREGION): REPREGION change by REGION_FROM_CHECK

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Toni de la Fuente <toni@blyx.com>
2022-06-06 12:56:21 +02:00
Pepe Fagoaga 1b63256b9c fix(assume_role): Use date instead of jq (#1181)
* fix(date): Use  instead of date

* fix(assume_role): Use date instead of jq

JQ parses datetimes using the local timezone and not UTC
2022-06-03 08:31:43 -07:00
Pepe Fagoaga e5cd42da55 fix(typo): Max session duration error message (#1179) 2022-06-02 15:08:30 +02:00
Nacho Rivera 2e134ed947 feat(db_connector): Create a PostgreSQL connector for Prowler (#1171)
* chore(db providers): db providers first version

* chore(db provider): added db provider setup into Readme

* fix(csv_line): csv_line out of conditional

* fix(README): text instead of varchar in table

* fix(help): help message extended

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>

* fix(typo): Update README.md

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(table): add if not exists

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(typo): Readme postgreSQL

Co-authored-by: Pepe Fagoaga <pepe@verica.io>

* fix(db_connector): details to add a new provider

* fix(typo): Uppercase Prowler

Co-authored-by: Toni de la Fuente <toni@blyx.com>

* fix(prowler): deleted unused variable

* chore(checks): test db connector previous to send data

* chore(input tests): input tests moved to main

* fix(typo): Readme typos

* chore(table): table name from pgpass file

* fix(grep test): Added missing -E flag

* chore(table): check of table name and Readme

* chore(error colors): Added error colors

* fix(tablename): table name in readme

* fix(typo)

* fix(db_provider): Exact match

* fix(error): One line message

* chore(pgpass check): Check added for pgpass file

* fix(pgpass): pgpass file and permissions test

* fix(unused vars): Deleted unused vars

* fix(TOP_PID): Deleted TOP_PID unused var and comment

* chore(db tests): Credentials, database and table tests added

* fix(empty pgpass): Look for empty fields at pgpass file

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Toni de la Fuente <toni@blyx.com>
2022-06-02 13:15:14 +02:00
Nacho Rivera f4bae78730 Timestamp to date casting issues solved (#1154)
* fix(date): Deleted @ char before date argument

* fix(date): Use @ only when input is epoch
2022-05-23 09:28:56 +02:00
Sergio Garcia d60eea5e2f fix(copyToS3): Upload to S3 only when indicated (#1134)
Co-authored-by: sergargar <sergio@verica.io>
2022-05-12 17:30:49 +02:00
Sergio Garcia 210f44f66f fix(custom-file-in-bucket): Custom file names are also support for S3 output. (#1129) 2022-05-11 10:16:29 +02:00
Sergio Garcia b78e4ad6a1 fix(allowlist_db): Improve DynamoDB regex for allowlisting. (#1127) 2022-05-06 13:46:53 +02:00
stof 4146566f92 feat(assume-role): Properly handle External ID variable 2022-05-05 16:10:52 +02:00
Sergio Garcia de77a33341 fix(allowlist_db): Improve DynamoDB regex for allowlisting. (#1126)
Co-authored-by: sergargar <sergio@verica.io>
2022-05-03 11:57:23 +02:00
Sergio Garcia ecbfbfb960 fix(allowlist_db): Improve DynamoDB regex for allowlisting. (#1125)
Co-authored-by: sergargar <sergio@verica.io>
2022-05-03 11:31:12 +02:00
Sergio Garcia aa16bf4084 feat(dynamodb_allowlist): Support DynamoDB tables ARN for allowlist input (#1118)
* feat(dynamodb_allowlist): Support dynamodb tables arn for allowlist input.

* feat(allowlist): Include logging messages for input file

* fix(allowlist): Modify DynamoDB key name

Co-authored-by: sergargar <sergio@verica.io>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
2022-04-28 17:04:44 +02:00
Pepe Fagoaga d6ade7694e chore(allowlist): Rename references (#1108)
* chore(allowlist): rename file

* chore(allowlist): remove old references
2022-04-13 11:31:50 +02:00
n4ch04 819b52687c Replace comma from csv input info (#1102)
* fix(output): replace comma from csv input info

* fix(outputs): parameter expansion done in echo to csv
2022-04-11 16:04:47 +02:00
Sergio Garcia 28fff104a1 feat(S3_in_w_x_flags): Support S3 URIs for custom checks paths and whitelist files. (#1090)
* feat(S3_in_w_x_flags): Support S3 URIs for custom checks paths and whitelist files.

* feat(S3_in_w_x_flags): README document was updated.

* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Toni de la Fuente <toni@blyx.com>
Co-authored-by: Sergio Garcia Garcia
2022-04-07 14:37:02 -04:00
Sergio Garcia 8582e40edf fix(secrets_library): Verify if detect-secrets library is missing (#1080) 2022-03-25 13:19:05 +01:00
Toni de la Fuente 365b396f9a feat(metadata): Include account metadata in Prowler assessments (#1049)
* Add support for organizations accounts metadata part 1

* Add support for organizations accounts metadata part 2

* Add gathering account metadata from org

* chore(prowler): get accounts metadata

Use assume_role backing up normal assumed credentials to assume management account and then restore it to old ones

* fix(orgs metadata): deleted assume_role_orgs

* refactor(organization_metadata)

Reformulate to extract AWS Organizations metadata

* doc(org_metadata): include required -R in usage

* docs(org-metadata): Update README

Co-authored-by: n4ch04 <nachor1992@gmail.com>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
2022-03-16 16:27:19 +01:00
n4ch04 f04b174e67 fix(whitelist): Whitelist logic reformulated (#1061)
* fix(whitelist): Whitelist logic reformulated again

* chore(whitelist): reformulate style
2022-03-11 10:15:58 +01:00
n4ch04 3097ba6c66 fix(include/outputs):Rolling back whitelist checking to RE check (#1037)
* fix(include/outputs):Rolling back whitelist checking to RE check

* fix(include/ouputs): Clarified variable assignation coming from argument
2022-02-14 13:04:47 +01:00
Pepe Fagoaga 5c6902b459 fix(extra730): Handle invalid date formats checking ACM certificates (#1033) 2022-02-09 17:56:55 +01:00
n4ch04 30ce25300f fix(include/outputs): Whitelist logic reformulated to exactly match input (#1029)
* fix(inlcude/outputs) Whitelist logic reformulated to exactly match input

* fix(include/outputs): Changed name of iterative variable that browses whitelisted values

* fix(include/outputs): Deleted missing echo and include and put variables in brackets
2022-02-04 12:07:48 -05:00
n4ch04 d9561d5d22 fix(check32): filterName base64encoded to avoid space problems in filter names (#1020)
* fix(check32): filterName base64encoded to avoid space problems in filter names

* fix(check32): base64 decoding atomic expression

* fix(check32): Variable enclosing

Co-authored-by: Nacho Rivera <nachor1992@gmail>
2022-02-02 11:09:38 -05:00
Pepe Fagoaga 90565099bd Change references from toniblyx to prowler-cloud (#1003)
Co-authored-by: Toni de la Fuente <toni@blyx.com>
2022-01-27 12:17:38 +01:00
Toni de la Fuente 2b2814723f Prowler 2.7.0 - Brave (#998)
* Extra7161 EFS encryption at rest check

* Added check_extra7162 which checks if Log groups have 365 days retention

* fixed code to handle all regions and formatted output

* changed check title, resource type and service name as well as making the code more dynamic

* Extra7161 EFS encryption at rest check

* New check_extra7163 Secrets Manager key rotation enabled

* New check7160 Enabled AutomaticVersionUpgrade on RedShift Cluster

* Update ProwlerRole.yaml to have same permissions as util/org-multi-account/ProwlerRole.yaml

* Fix link to quicksight dashboard

* Install detect-secrets (e.g. for check_extra742)

* Updating check_extra7163 with requested changes

* fix(assumed-role): Check if -T and -A options are set

* docs(Readme): `-T` option is not mandatory

* fix(assume-role): Handle AWS STS CLI errors

* fix(assume-role): Handle AWS STS CLI errors

* Update group25_FTR

When trying to run the group 25 (Amazon FTR related security checks) nothing happens, after looking at the code there is a misconfiguration in 2 params: GROUP_RUN_BY_DEFAULT[9] and GROUP_CHECKS[9]. Updating values to 25 fixed the issue.

* Update README.md

broken link for capital letters in group file (group25_FTR)

* #938 issue assume_role multiple times should be fixed

* Label 2.7.0-1December2021 for tests

* Fixed error that appeared if the number of findings was very high.

* Adjusted the batch to only do 50 at a time. 100 caused capacity issues. Also added a check for an edge case where if the updated findings was a multiple of the batch size, it would throw an error for attempting to import 0 findings.

* Added line to delete the temp folder after everything is done.

* New check 7164 Check if Cloudwatch log groups are protected by AWS KMS@maisenhe

* updated CHECK_RISK

* Added checks extra7160,extra7161,extra7162,extra7163 to group Extras

* Added checks extra7160,extra7161,extra7162,extra7163 to group Extras

* Added issue templates

* New check 7165 DynamoDB: DAX encrypted at rest @Daniel-Peladeau

* New check 7165 DynamoDB: DAX encrypted at rest @Daniel-Peladeau

* Fix #963 check 792 to force json in ELB queries

* Fix #957 check 763 had us-east-1 region hardcoded

* Fix #962 check 7147 ALTERNATE NAME

* Fix #940 handling error when can not list functions

* Added new checks 7164 and 7165 to group extras

* Added invalid check or group id to the error message #962

* Fix Broken Link

* Add docker volume example to README.md

* Updated Dockerfile to use amazonlinux container

* Updated Dockerfile with AWS cli v2

* Added upgrade to the RUN

* Added cache purge to Dockerfile

* Backup AWS Credentials before AssumeRole and Restore them before CopyToS3

* exporting the ENV variables

* fixed bracket

* Improved documentation for install process

* fix checks with comma issues

* Added -D option to copy to S3 with the initial AWS credentials

* Cosmetic variable name change

* Added $PROFILE_OPT to CopyToS3 commands

* remove commas

* removed file as it is not needed

* Improved help usage options -h

* Fixed CIS LEVEL on 7163 through 7165

* When performing a restoreInitialAWSCredentials, unset the credentials ENV variables if they were never set

* New check 7166 Elastic IP addresses with associations are protected by AWS Shield Advanced

* New check 7167 Cloudfront distributions are protected by AWS Shield Advanced

* New check 7168 Route53 hosted zones are protected by AWS Shield Advanced

* New check 7169 Global accelerators are protected by AWS Shield Advanced

* New check 7170 Application load balancers are protected by AWS Shield Advanced

* New check 7171 Classic load balancers are protected by AWS Shield Advanced

* Include example for global resources

* Add AWS Advance Shield protection checks corrections

* Added Shield actions GetSubscriptionState and DescribeProtection

* Added Shield actions GetSubscriptionState and DescribeProtection

* docs(templates): Improve bug template with more info (#982)

* Removed echoes after role chaining fix

* Changed Route53 checks7152 and 7153 to INFO when no domains found

* Changed Route53 checks 7152 and 7153 title to clarify

* Added passed security groups in output to check 778

* Added passed security groups and updated title to check 777

* Added FAIL as error handling when SCP prevents queries to regions

* Label version 2.7.0-6January2022

* Updated .dockerignore with .github/

* Fix: issue #758 and #984

* Fix: issue #741 CloudFront and real-time logs

* Fix issues #971 set all as INFO instead of FAIL when no access to resource

* Fix: issue #986

* Add additional action permissions for Glue and Shield Advanced checks @lazize

* Add extra shield action permission

Allows the shield:GetSubscriptionState action

* Add permission actions

Make sure all files where permission actions are necessary will have the same actions

* Fix: Credential chaining from environment variables @lazize #996f

If profile is not defined, restore original credentials from environment variables,
if they exists, before assume-role

* Lable version 2.7.0-24January2022

Co-authored-by: Lee Myers <ichilegend@gmail.com>
Co-authored-by: Chinedu Obiakara <obiakac@amazon.com>
Co-authored-by: Daniel Peladeau <dcpeladeau@gmail.com>
Co-authored-by: Jonathan Lozano <jonloza@amazon.com>
Co-authored-by: Daniel Lorch <dlorch@gmail.com>
Co-authored-by: Pepe Fagoaga <jose.fagoaga@smartprotection.com>
Co-authored-by: Israel <6672089+lopmoris@users.noreply.github.com>
Co-authored-by: root <halfluke@gmail.com>
Co-authored-by: nikirby <nikirby@amazon.com>
Co-authored-by: Joel Maisenhelder <maisenhe@gmail.com>
Co-authored-by: RT <35173068+rtcms@users.noreply.github.com>
Co-authored-by: Andrea Di Fabio <39841198+sectoramen@users.noreply.github.com>
Co-authored-by: Joseph de CLERCK <clerckj@amazon.fr>
Co-authored-by: Michael Dickinson <45626543+michael-dickinson-sainsburys@users.noreply.github.com>
Co-authored-by: Pepe Fagoaga <pepe@verica.io>
Co-authored-by: Leonardo Azize Martins <lazize@users.noreply.github.com>
2022-01-24 13:49:47 +01:00
Toni de la Fuente d272fad4c2 Enhancement IAM assumed role session duration error handling by @jfagoagas
Enhancement IAM assumed role session duration error handling by @jfagoagas
2021-11-15 18:17:09 +01:00
Toni de la Fuente 50de9f2ab4 Fix output for checks check3x when no CW group is in place 2021-11-15 15:49:33 +01:00
Pepe Fagoaga 563cd71060 fix(iam-role): Delete temporary prowler.sts_assumed* if error 2021-11-13 16:25:43 +01:00
Pepe Fagoaga e4edb5e39e fix(iam-role): IAM assumed role session duration 2021-11-12 18:32:02 +01:00
Toni de la Fuente dd398a994b Fix issue #904 2021-11-11 14:05:14 +01:00
Toni de la Fuente 140e96e5e1 Fix issue #848 CIS LEVEL added to CSV and other formats 2021-11-11 13:40:40 +01:00
Toni de la Fuente 48d7381822 Fix issue #871 2021-11-09 14:21:32 +01:00
Toni de la Fuente 2c81b383e8 Fix scoring check counter 2021-11-09 12:59:36 +01:00