chore(release): 3.12.1

Co-authored-by: sergargar <sergargar@users.noreply.github.com>

.github/workflows/codeql.yml

@@ -13,10 +13,10 @@ name: "CodeQL"
 
 on:
   push:
-    branches: [ "master", prowler-2, prowler-3.0-dev ]
+    branches: [ "master", "prowler-4.0-dev" ]
   pull_request:
     # The branches below must be a subset of the branches above
-    branches: [ "master" ]
+    branches: [ "master", "prowler-4.0-dev" ]
   schedule:
     - cron: '00 12 * * *'
 

.github/workflows/pull-request.yml

@@ -4,9 +4,11 @@ on:
   push:
     branches:
       - "master"
+      - "prowler-4.0-dev"
   pull_request:
     branches:
       - "master"
+      - "prowler-4.0-dev"
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -18,7 +20,7 @@ jobs:
       - uses: actions/checkout@v3
       - name: Test if changes are in not ignored paths
         id: are-non-ignored-files-changed
-        uses: tj-actions/changed-files@v39
+        uses: tj-actions/changed-files@v41
         with:
           files: ./**
           files_ignore: |
@@ -26,6 +28,7 @@ jobs:
             README.md
             docs/**
             permissions/**
+            mkdocs.yml
       - name: Install poetry
         if: steps.are-non-ignored-files-changed.outputs.any_changed == 'true'
         run: |

.pre-commit-config.yaml

@@ -80,9 +80,9 @@ repos:
       - id: trufflehog
         name: TruffleHog
         description: Detect secrets in your data.
-        # entry: bash -c 'trufflehog git file://. --only-verified --fail'
+        entry: bash -c 'trufflehog --no-update git file://. --only-verified --fail'
         # For running trufflehog in docker, use the following entry instead:
-        entry: bash -c 'docker run -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --only-verified --fail'
+        # entry: bash -c 'docker run -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --only-verified --fail'
         language: system
         stages: ["commit", "push"]
 

README.md

@@ -14,11 +14,11 @@
   <a href="https://pypi.org/project/prowler/"><img alt="Python Version" src="https://img.shields.io/pypi/v/prowler.svg"></a>
   <a href="https://pypi.python.org/pypi/prowler/"><img alt="Python Version" src="https://img.shields.io/pypi/pyversions/prowler.svg"></a>
   <a href="https://pypistats.org/packages/prowler"><img alt="PyPI Prowler Downloads" src="https://img.shields.io/pypi/dw/prowler.svg?label=prowler%20downloads"></a>
-  <a href="https://pypistats.org/packages/prowler-cloud"><img alt="PyPI Prowler-Cloud Downloads" src="https://img.shields.io/pypi/dw/prowler-cloud.svg?label=prowler-cloud%20downloads"></a>
   <a href="https://hub.docker.com/r/toniblyx/prowler"><img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/toniblyx/prowler"></a>
   <a href="https://hub.docker.com/r/toniblyx/prowler"><img alt="Docker" src="https://img.shields.io/docker/cloud/build/toniblyx/prowler"></a>
   <a href="https://hub.docker.com/r/toniblyx/prowler"><img alt="Docker" src="https://img.shields.io/docker/image-size/toniblyx/prowler"></a>
   <a href="https://gallery.ecr.aws/prowler-cloud/prowler"><img width="120" height=19" alt="AWS ECR Gallery" src="https://user-images.githubusercontent.com/3985464/151531396-b6535a68-c907-44eb-95a1-a09508178616.png"></a>
+  <a href="https://codecov.io/gh/prowler-cloud/prowler"><img src="https://codecov.io/gh/prowler-cloud/prowler/graph/badge.svg?token=OflBGsdpDl"/></a>
 </p>
 <p align="center">
   <a href="https://github.com/prowler-cloud/prowler"><img alt="Repo size" src="https://img.shields.io/github/repo-size/prowler-cloud/prowler"></a>

docs/index.md

@@ -136,26 +136,16 @@ Prowler is available as a project in [PyPI](https://pypi.org/project/prowler-clo
 
 === "AWS CloudShell"
 
-    Prowler can be easely executed in AWS CloudShell but it has some prerequsites to be able to to so. AWS CloudShell is a container running with `Amazon Linux release 2 (Karoo)` that comes with Python 3.7, since Prowler requires Python >= 3.9 we need to first install a newer version of Python. Follow the steps below to successfully execute Prowler v3 in AWS CloudShell:
+    After the migration of AWS CloudShell from Amazon Linux 2 to Amazon Linux 2023 [[1]](https://aws.amazon.com/about-aws/whats-new/2023/12/aws-cloudshell-migrated-al2023/) [2](https://docs.aws.amazon.com/cloudshell/latest/userguide/cloudshell-AL2023-migration.html), there is no longer a need to manually compile Python 3.9 as it's already included in AL2023. Prowler can thus be easily installed following the Generic method of installation via pip. Follow the steps below to successfully execute Prowler v3 in AWS CloudShell:
 
     _Requirements_:
 
-    * First install all dependences and then Python, in this case we need to compile it because there is not a package available at the time this document is written:
-    ```
-    sudo yum -y install gcc openssl-devel bzip2-devel libffi-devel
-    wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz
-    tar zxf Python-3.9.16.tgz
-    cd Python-3.9.16/
-    ./configure --enable-optimizations
-    sudo make altinstall
-    python3.9 --version
-    cd
-    ```
+    * Open AWS CloudShell `bash`.
+
     _Commands_:
 
-    * Once Python 3.9 is available we can install Prowler from pip:
     ```
-    pip3.9 install prowler
+    pip install prowler
     prowler -v
     ```
 

docs/tutorials/aws/boto3-configuration.md

@@ -32,3 +32,14 @@ Prowler's AWS Provider uses the Boto3 [Standard](https://boto3.amazonaws.com/v1/
 - Retry attempts on nondescriptive, transient error codes. Specifically, these HTTP status codes: 500, 502, 503, 504.
 
 - Any retry attempt will include an exponential backoff by a base factor of 2 for a maximum backoff time of 20 seconds.
+
+## Notes for validating retry attempts
+
+If you are making changes to Prowler, and want to validate if requests are being retried or given up on, you can take the following approach
+
+* Run prowler with `--log-level DEBUG` and `--log-file debuglogs.txt`
+* Search for retry attempts using `grep -i 'Retry needed' debuglogs.txt`
+
+This is based off of the [AWS documentation](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html#checking-retry-attempts-in-your-client-logs), which states that if a retry is performed, you will see a message starting with "Retry needed".
+
+You can determine the total number of calls made using `grep -i 'Sending http request' debuglogs.txt | wc -l`

docs/tutorials/aws/cloudshell.md

@@ -1,26 +1,26 @@
 # AWS CloudShell
 
-Prowler can be easily executed in AWS CloudShell but it has some prerequisites to be able to to so. AWS CloudShell is a container running with `Amazon Linux release 2 (Karoo)` that comes with Python 3.7, since Prowler requires Python >= 3.9 we need to first install a newer version of Python. Follow the steps below to successfully execute Prowler v3 in AWS CloudShell:
-
-- First install all dependences and then Python, in this case we need to compile it because there is not a package available at the time this document is written:
-```
-sudo yum -y install gcc openssl-devel bzip2-devel libffi-devel
-wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz
-tar zxf Python-3.9.16.tgz
-cd Python-3.9.16/
-./configure --enable-optimizations
-sudo make altinstall
-python3.9 --version
-cd 
-```
-- Once Python 3.9 is available we can install Prowler from pip:
-```
-pip3.9 install prowler
-```
-- Now enjoy Prowler:
-```
+## Installation
+After the migration of AWS CloudShell from Amazon Linux 2 to Amazon Linux 2023 [[1]](https://aws.amazon.com/about-aws/whats-new/2023/12/aws-cloudshell-migrated-al2023/) [[2]](https://docs.aws.amazon.com/cloudshell/latest/userguide/cloudshell-AL2023-migration.html), there is no longer a need to manually compile Python 3.9 as it's already included in AL2023. Prowler can thus be easily installed following the Generic method of installation via pip. Follow the steps below to successfully execute Prowler v3 in AWS CloudShell:
+```shell
+pip install prowler
 prowler -v
-prowler 
 ```
 
-- To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/home/cloudshell-user/output/prowler-output-123456789012-20221220191331.csv`
+## Download Files
+
+To download the results from AWS CloudShell, select Actions -> Download File and add the full path of each file. For the CSV file it will be something like `/home/cloudshell-user/output/prowler-output-123456789012-20221220191331.csv`
+
+## Clone Prowler from Github
+
+The limited storage that AWS CloudShell provides for the user's home directory causes issues when installing the poetry dependencies to run Prowler from GitHub. Here is a workaround:
+```shell
+git clone https://github.com/prowler-cloud/prowler.git
+cd prowler
+pip install poetry
+mkdir /tmp/pypoetry
+poetry config cache-dir /tmp/pypoetry
+poetry shell
+poetry install
+python prowler.py -v
+```

docs/tutorials/aws/role-assumption.md

@@ -23,6 +23,15 @@ prowler aws -R arn:aws:iam::<account_id>:role/<role_name>
 prowler aws -T/--session-duration <seconds> -I/--external-id <external_id> -R arn:aws:iam::<account_id>:role/<role_name>
 ```
 
+## Custom Role Session Name
+
+Prowler can use your custom Role Session name with:
+```console
+prowler aws --role-session-name <role_session_name>
+```
+
+> It defaults to `ProwlerAssessmentSession`
+
 ## STS Endpoint Region
 
 If you are using Prowler in AWS regions that are not enabled by default you need to use the argument `--sts-endpoint-region` to point the AWS STS API calls `assume-role` and `get-caller-identity` to the non-default region, e.g.: `prowler aws --sts-endpoint-region eu-south-2`.

docs/tutorials/custom-checks-metadata.md

@@ -0,0 +1,43 @@
+# Custom Checks Metadata
+
+In certain organizations, the severity of specific checks might differ from the default values defined in the check's metadata. For instance, while `s3_bucket_level_public_access_block` could be deemed `critical` for some organizations, others might assign a different severity level.
+
+The custom metadata option offers a means to override default metadata set by Prowler
+
+You can utilize `--custom-checks-metadata-file` followed by the path to your custom checks metadata YAML file.
+
+## Available Fields
+
+The list of supported check's metadata fields that can be override are listed as follows:
+
+- Severity
+
+## File Syntax
+
+This feature is available for all the providers supported in Prowler since the metadata format is common between all the providers. The following is the YAML format for the custom checks metadata file:
+```yaml title="custom_checks_metadata.yaml"
+CustomChecksMetadata:
+  aws:
+    Checks:
+      s3_bucket_level_public_access_block:
+        Severity: high
+      s3_bucket_no_mfa_delete:
+        Severity: high
+  azure:
+    Checks:
+      storage_infrastructure_encryption_is_enabled:
+        Severity: medium
+  gcp:
+    Checks:
+      compute_instance_public_ip:
+        Severity: critical
+```
+
+## Usage
+
+Executing the following command will assess all checks and generate a report while overriding the metadata for those checks:
+```sh
+prowler <provider> --custom-checks-metadata-file <path/to/custom/metadata>
+```
+
+This customization feature enables organizations to tailor the severity of specific checks based on their unique requirements, providing greater flexibility in security assessment and reporting.

docs/tutorials/parallel-execution.md

@@ -0,0 +1,187 @@
+# Parallel Execution
+
+The strategy used here will be to execute Prowler once per service. You can modify this approach as per your requirements.
+
+This can help for really large accounts, but please be aware of AWS API rate limits:
+
+1. **Service-Specific Limits**: Each AWS service has its own rate limits. For instance, Amazon EC2 might have different rate limits for launching instances versus making API calls to describe instances.
+2. **API Rate Limits**: Most of the rate limits in AWS are applied at the API level. Each API call to an AWS service counts towards the rate limit for that service.
+3. **Throttling Responses**: When you exceed the rate limit for a service, AWS responds with a throttling error. In AWS SDKs, these are typically represented as `ThrottlingException` or `RateLimitExceeded` errors.
+
+For information on Prowler's retrier configuration please refer to this [page](https://docs.prowler.cloud/en/latest/tutorials/aws/boto3-configuration/).
+
+> Note: You might need to increase the `--aws-retries-max-attempts` parameter from the default value of 3. The retrier follows an exponential backoff strategy.
+
+## Linux
+
+Generate a list of services that Prowler supports, and populate this info into a file:
+
+```bash
+prowler aws --list-services | awk -F"- " '{print $2}' | sed '/^$/d' > services
+```
+
+Make any modifications for services you would like to skip scanning by modifying this file.
+
+Then create a new PowerShell script file `parallel-prowler.sh` and add the following contents. Update the `$profile` variable to the AWS CLI profile you want to run Prowler with.
+
+```bash
+#!/bin/bash
+
+# Change these variables as needed
+profile="your_profile"
+account_id=$(aws sts get-caller-identity --profile "${profile}" --query 'Account' --output text)
+
+echo "Executing in account: ${account_id}"
+
+# Maximum number of concurrent processes
+MAX_PROCESSES=5
+
+# Loop through the services
+while read service; do
+    echo "$(date '+%Y-%m-%d %H:%M:%S'): Starting job for service: ${service}"
+
+    # Run the command in the background
+    (prowler -p "$profile" -s "$service" -F "${account_id}-${service}" --ignore-unused-services --only-logs; echo "$(date '+%Y-%m-%d %H:%M:%S') - ${service} has completed") &
+
+    # Check if we have reached the maximum number of processes
+    while [ $(jobs -r | wc -l) -ge ${MAX_PROCESSES} ]; do
+        # Wait for a second before checking again
+        sleep 1
+    done
+done < ./services
+
+# Wait for all background processes to finish
+wait
+echo "All jobs completed"
+```
+
+Output will be stored in the `output/` folder that is in the same directory from which you executed the script.
+
+## Windows
+
+Generate a list of services that Prowler supports, and populate this info into a file:
+
+```powershell
+prowler aws --list-services | ForEach-Object {
+    # Capture lines that are likely service names
+    if ($_ -match '^\- \w+$') {
+        $_.Trim().Substring(2)
+    }
+} | Where-Object {
+    # Filter out empty or null lines
+    $_ -ne $null -and $_ -ne ''
+} | Set-Content -Path "services"
+```
+
+Make any modifications for services you would like to skip scanning by modifying this file.
+
+Then create a new PowerShell script file `parallel-prowler.ps1` and add the following contents. Update the `$profile` variable to the AWS CLI profile you want to run prowler with.
+
+Change any parameters you would like when calling prowler in the `Start-Job -ScriptBlock` section. Note that you need to keep the `--only-logs` parameter, else some encoding issue occurs when trying to render the progress-bar and prowler won't successfully execute.
+
+```powershell
+$profile = "your_profile"
+$account_id = Invoke-Expression -Command "aws sts get-caller-identity --profile $profile --query 'Account' --output text"
+
+Write-Host "Executing Prowler in $account_id"
+
+# Maximum number of concurrent jobs
+$MAX_PROCESSES = 5
+
+# Read services from a file
+$services = Get-Content -Path "services"
+
+# Array to keep track of started jobs
+$jobs = @()
+
+foreach ($service in $services) {
+    # Start the command as a job
+    $job = Start-Job -ScriptBlock {
+        prowler -p ${using:profile} -s ${using:service} -F "${using:account_id}-${using:service}" --ignore-unused-services --only-logs
+	      $endTimestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
+        Write-Output "${endTimestamp} - $using:service has completed"
+    }
+    $jobs += $job
+    Write-Host "$(Get-Date -Format 'yyyy-MM-dd HH:mm:ss') - Starting job for service: $service"
+
+    # Check if we have reached the maximum number of jobs
+    while (($jobs | Where-Object { $_.State -eq 'Running' }).Count -ge $MAX_PROCESSES) {
+        Start-Sleep -Seconds 1
+        # Check for any completed jobs and receive their output
+        $completedJobs = $jobs | Where-Object { $_.State -eq 'Completed' }
+        foreach ($completedJob in $completedJobs) {
+            Receive-Job -Job $completedJob -Keep | ForEach-Object { Write-Host $_ }
+            $jobs = $jobs | Where-Object { $_.Id -ne $completedJob.Id }
+            Remove-Job -Job $completedJob
+        }
+    }
+}
+
+# Check for any remaining completed jobs
+$remainingCompletedJobs = $jobs | Where-Object { $_.State -eq 'Completed' }
+foreach ($remainingJob in $remainingCompletedJobs) {
+    Receive-Job -Job $remainingJob -Keep | ForEach-Object { Write-Host $_ }
+    Remove-Job -Job $remainingJob
+}
+
+Write-Host "$(Get-Date -Format 'yyyy-MM-dd HH:mm:ss') - All jobs completed"
+```
+
+Output will be stored in `C:\Users\YOUR-USER\Documents\output\`
+
+## Combining the output files
+
+Guidance is provided for the CSV file format. From the ouput directory, execute either the following Bash or PowerShell script. The script will collect the output from the CSV files, only include the header from the first file, and then output the result as CombinedCSV.csv in the current working directory.
+
+There is no logic implemented in terms of which CSV files it will combine. If you have additional CSV files from other actions, such as running a quick inventory, you will need to move that out of the current (or any nested) directory, or move the output you want to combine into its own folder and run the script from there.
+
+```bash
+#!/bin/bash
+
+# Initialize a variable to indicate the first file
+firstFile=true
+
+# Find all CSV files and loop through them
+find . -name "*.csv" -print0 | while IFS= read -r -d '' file; do
+    if [ "$firstFile" = true ]; then
+        # For the first file, keep the header
+        cat "$file" > CombinedCSV.csv
+        firstFile=false
+    else
+        # For subsequent files, skip the header
+        tail -n +2 "$file" >> CombinedCSV.csv
+    fi
+done
+```
+
+```powershell
+# Get all CSV files from current directory and its subdirectories
+$csvFiles = Get-ChildItem -Recurse -Filter "*.csv"
+
+# Initialize a variable to track if it's the first file
+$firstFile = $true
+
+# Loop through each CSV file
+foreach ($file in $csvFiles) {
+    if ($firstFile) {
+        # For the first file, keep the header and change the flag
+        $combinedCsv = Import-Csv -Path $file.FullName
+        $firstFile = $false
+    } else {
+        # For subsequent files, skip the header
+        $tempCsv = Import-Csv -Path $file.FullName
+        $combinedCsv += $tempCsv | Select-Object * -Skip 1
+    }
+}
+
+# Export the combined data to a new CSV file
+$combinedCsv | Export-Csv -Path "CombinedCSV.csv" -NoTypeInformation
+```
+
+## TODO: Additional Improvements
+
+Some services need to instantiate another service to perform a check. For instance, `cloudwatch` will instantiate Prowler's `iam` service to perform the `cloudwatch_cross_account_sharing_disabled` check. When the `iam` service is instantiated, it will perform the `__init__` function, and pull all the information required for that service. This provides an opportunity for an improvement in the above script to group related services together so that the `iam` services (or any other cross-service references) isn't repeatedily instantiated by grouping dependant services together. A complete mapping between these services still needs to be further investigated, but these are the cross-references that have been noted:
+
+* inspector2 needs lambda and ec2
+* cloudwatch needs iam
+* dlm needs ec2

docs/tutorials/reporting.md

@@ -43,46 +43,71 @@ Hereunder is the structure for each of the supported report formats by Prowler:
 ![HTML Output](../img/output-html.png)
 ### CSV
 
-The following are the columns present in the CSV format:
+CSV format has a set of common columns for all the providers, and then provider specific columns.  
+The common columns are the following:
 
 - ASSESSMENT_START_TIME
 - FINDING_UNIQUE_ID
 - PROVIDER
+- CHECK_ID
+- CHECK_TITLE
+- CHECK_TYPE
+- STATUS
+- STATUS_EXTENDED
+- SERVICE_NAME
+- SUBSERVICE_NAME
+- SEVERITY
+- RESOURCE_TYPE
+- RESOURCE_DETAILS
+- RESOURCE_TAGS
+- DESCRIPTION
+- RISK
+- RELATED_URL
+- REMEDIATION_RECOMMENDATION_TEXT
+- REMEDIATION_RECOMMENDATION_URL
+- REMEDIATION_RECOMMENDATION_CODE_NATIVEIAC
+- REMEDIATION_RECOMMENDATION_CODE_TERRAFORM
+- REMEDIATION_RECOMMENDATION_CODE_CLI
+- REMEDIATION_RECOMMENDATION_CODE_OTHER
+- COMPLIANCE
+- CATEGORIES
+- DEPENDS_ON
+- RELATED_TO
+- NOTES
+
+And then by the provider specific columns:
+
+#### AWS
+
 - PROFILE
 - ACCOUNT_ID
--  ACCOUNT_NAME
--  ACCOUNT_EMAIL
--  ACCOUNT_ARN
--  ACCOUNT_ORG
--  ACCOUNT_TAGS
--  REGION
--  CHECK_ID
--  CHECK_TITLE
--  CHECK_TYPE
--  STATUS
--  STATUS_EXTENDED
--  SERVICE_NAME
--  SUBSERVICE_NAME
--  SEVERITY
--  RESOURCE_ID
--  RESOURCE_ARN
--  RESOURCE_TYPE
--  RESOURCE_DETAILS
--  RESOURCE_TAGS
--  DESCRIPTION
--  COMPLIANCE
--  RISK
--  RELATED_URL
--  REMEDIATION_RECOMMENDATION_TEXT
--  REMEDIATION_RECOMMENDATION_URL
--  REMEDIATION_RECOMMENDATION_CODE_NATIVEIAC
--  REMEDIATION_RECOMMENDATION_CODE_TERRAFORM
--  REMEDIATION_RECOMMENDATION_CODE_CLI
--  REMEDIATION_RECOMMENDATION_CODE_OTHER
--  CATEGORIES
--  DEPENDS_ON
--  RELATED_TO
--  NOTES
+- ACCOUNT_NAME
+- ACCOUNT_EMAIL
+- ACCOUNT_ARN
+- ACCOUNT_ORG
+- ACCOUNT_TAGS
+- REGION
+- RESOURCE_ID
+- RESOURCE_ARN
+
+
+#### AZURE
+
+- TENANT_DOMAIN
+- SUBSCRIPTION
+- RESOURCE_ID
+- RESOURCE_NAME
+
+
+#### GCP
+
+- PROJECT_ID
+- LOCATION
+- RESOURCE_ID
+- RESOURCE_NAME
+
+
+
 
 > Since Prowler v3 the CSV column delimiter is the semicolon (`;`)
 ### JSON

mkdocs.yml

@@ -38,8 +38,10 @@ nav:
       - Logging: tutorials/logging.md
       - Allowlist: tutorials/allowlist.md
       - Check Aliases: tutorials/check-aliases.md
+      - Custom Metadata: tutorials/custom-checks-metadata.md
       - Ignore Unused Services: tutorials/ignore-unused-services.md
       - Pentesting: tutorials/pentesting.md
+      - Parallel Execution: tutorials/parallel-execution.md
       - Developer Guide: developer-guide/introduction.md
       - AWS:
           - Authentication: tutorials/aws/authentication.md

prowler/__main__.py

@@ -26,6 +26,10 @@ from prowler.lib.check.check import (
 )
 from prowler.lib.check.checks_loader import load_checks_to_execute
 from prowler.lib.check.compliance import update_checks_metadata_with_compliance
+from prowler.lib.check.custom_checks_metadata import (
+    parse_custom_checks_metadata_file,
+    update_checks_metadata,
+)
 from prowler.lib.cli.parser import ProwlerArgumentParser
 from prowler.lib.logger import logger, set_logging_config
 from prowler.lib.outputs.compliance import display_compliance_table
@@ -47,7 +51,6 @@ from prowler.providers.common.audit_info import (
     set_provider_audit_info,
     set_provider_execution_parameters,
 )
-from prowler.providers.common.clean import clean_provider_local_output_directories
 from prowler.providers.common.outputs import set_provider_output_options
 from prowler.providers.common.quick_inventory import run_provider_quick_inventory
 
@@ -68,6 +71,7 @@ def prowler():
     checks_folder = args.checks_folder
     severities = args.severity
     compliance_framework = args.compliance
+    custom_checks_metadata_file = args.custom_checks_metadata_file
 
     if not args.no_banner:
         print_banner(args)
@@ -97,9 +101,19 @@ def prowler():
 
     bulk_compliance_frameworks = bulk_load_compliance_frameworks(provider)
     # Complete checks metadata with the compliance framework specification
-    update_checks_metadata_with_compliance(
+    bulk_checks_metadata = update_checks_metadata_with_compliance(
         bulk_compliance_frameworks, bulk_checks_metadata
     )
+    # Update checks metadata if the --custom-checks-metadata-file is present
+    custom_checks_metadata = None
+    if custom_checks_metadata_file:
+        custom_checks_metadata = parse_custom_checks_metadata_file(
+            provider, custom_checks_metadata_file
+        )
+        bulk_checks_metadata = update_checks_metadata(
+            bulk_checks_metadata, custom_checks_metadata
+        )
+
     if args.list_compliance:
         print_compliance_frameworks(bulk_compliance_frameworks)
         sys.exit()
@@ -175,7 +189,11 @@ def prowler():
     findings = []
     if len(checks_to_execute):
         findings = execute_checks(
-            checks_to_execute, provider, audit_info, audit_output_options
+            checks_to_execute,
+            provider,
+            audit_info,
+            audit_output_options,
+            custom_checks_metadata,
         )
     else:
         logger.error(
@@ -305,9 +323,6 @@ def prowler():
     if checks_folder:
         remove_custom_checks_module(checks_folder, provider)
 
-    # clean local directories
-    clean_provider_local_output_directories(args)
-
     # If there are failed findings exit code 3, except if -z is input
     if not args.ignore_exit_code_3 and stats["total_fail"] > 0:
         sys.exit(3)

prowler/compliance/aws/ens_rd2022_aws.json

@@ -211,6 +211,31 @@
         "iam_avoid_root_usage"
       ]
     },
+    {
+      "Id": "op.acc.4.aws.iam.8",
+      "Description": "Proceso de gestión de derechos de acceso",
+      "Attributes": [
+        {
+          "IdGrupoControl": "op.acc.4",
+          "Marco": "operacional",
+          "Categoria": "control de acceso",
+          "DescripcionControl": "Se restringirá todo acceso a las acciones especificadas para el usuario root de una cuenta.",
+          "Nivel": "alto",
+          "Tipo": "requisito",
+          "Dimensiones": [
+            "confidencialidad",
+            "integridad",
+            "trazabilidad",
+            "autenticidad"
+          ],
+          "ModoEjecucion": "automático"
+        }
+      ],
+      "Checks": [
+        "organizations_account_part_of_organizations",
+        "organizations_scp_check_deny_regions"
+      ]
+    },
     {
       "Id": "op.acc.4.aws.iam.9",
       "Description": "Proceso de gestión de derechos de acceso",
@@ -1121,6 +1146,30 @@
         "cloudtrail_insights_exist"
       ]
     },
+    {
+      "Id": "op.exp.8.r1.aws.ct.3",
+      "Description": "Revisión de los registros",
+      "Attributes": [
+        {
+          "IdGrupoControl": "op.exp.8.r1",
+          "Marco": "operacional",
+          "Categoria": "explotación",
+          "DescripcionControl": "Registrar los eventos de lectura y escritura de datos.",
+          "Nivel": "alto",
+          "Tipo": "refuerzo",
+          "Dimensiones": [
+            "trazabilidad"
+          ],
+          "ModoEjecucion": "automático"
+        }
+      ],
+      "Checks": [
+        "cloudwatch_log_metric_filter_and_alarm_for_cloudtrail_configuration_changes_enabled",
+        "cloudtrail_s3_dataevents_write_enabled",
+        "cloudtrail_s3_dataevents_read_enabled",
+        "cloudtrail_insights_exist"
+      ]
+    },
     {
       "Id": "op.exp.8.r1.aws.ct.4",
       "Description": "Revisión de los registros",
@@ -1233,6 +1282,33 @@
         "iam_role_cross_service_confused_deputy_prevention"
       ]
     },
+    {
+      "Id": "op.exp.8.r4.aws.ct.1",
+      "Description": "Control de acceso",
+      "Attributes": [
+        {
+          "IdGrupoControl": "op.exp.8.r4",
+          "Marco": "operacional",
+          "Categoria": "explotación",
+          "DescripcionControl": "Asignar correctamente las políticas AWS IAM para el acceso y borrado de los registros y sus copias de seguridad haciendo uso del principio de mínimo privilegio.",
+          "Nivel": "alto",
+          "Tipo": "refuerzo",
+          "Dimensiones": [
+            "trazabilidad"
+          ],
+          "ModoEjecucion": "automático"
+        }
+      ],
+      "Checks": [
+        "iam_policy_allows_privilege_escalation",
+        "iam_customer_attached_policy_no_administrative_privileges",
+        "iam_customer_unattached_policy_no_administrative_privilege",
+        "iam_no_custom_policy_permissive_role_assumption",
+        "iam_policy_attached_only_to_group_or_roles",
+        "iam_role_cross_service_confused_deputy_prevention",
+        "iam_policy_no_full_access_to_cloudtrail"
+      ]
+    },
     {
       "Id": "op.exp.8.r4.aws.ct.2",
       "Description": "Control de acceso",
@@ -2110,7 +2186,7 @@
         }
       ],
       "Checks": [
-        "networkfirewall_in_all_vpc"
+        "fms_policy_compliant"
       ]
     },
     {
@@ -2251,6 +2327,31 @@
         "cloudfront_distributions_https_enabled"
       ]
     },
+    {
+      "Id": "mp.com.4.aws.ws.1",
+      "Description": "Separación de flujos de información en la red",
+      "Attributes": [
+        {
+          "IdGrupoControl": "mp.com.4",
+          "Marco": "medidas de protección",
+          "Categoria": "segregación de redes",
+          "DescripcionControl": "Se deberán abrir solo los puertos necesarios para el uso del servicio AWS WorkSpaces.",
+          "Nivel": "alto",
+          "Tipo": "requisito",
+          "Dimensiones": [
+            "confidencialidad",
+            "integridad",
+            "trazabilidad",
+            "autenticidad",
+            "disponibilidad"
+          ],
+          "ModoEjecucion": "automático"
+        }
+      ],
+      "Checks": [
+        "workspaces_vpc_2private_1public_subnets_nat"
+      ]
+    },
     {
       "Id": "mp.com.4.aws.vpc.1",
       "Description": "Separación de flujos de información en la red",
@@ -2323,7 +2424,8 @@
         }
       ],
       "Checks": [
-        "vpc_subnet_separate_private_public"
+        "vpc_subnet_separate_private_public",
+        "vpc_different_regions"
       ]
     },
     {
@@ -2370,7 +2472,8 @@
         }
       ],
       "Checks": [
-        "vpc_subnet_different_az"
+        "vpc_subnet_different_az",
+        "vpc_different_regions"
       ]
     },
     {

prowler/config/config.py

@@ -11,7 +11,7 @@ from prowler.lib.logger import logger
 
 timestamp = datetime.today()
 timestamp_utc = datetime.now(timezone.utc).replace(tzinfo=timezone.utc)
-prowler_version = "3.11.3"
+prowler_version = "3.12.1"
 html_logo_url = "https://github.com/prowler-cloud/prowler/"
 html_logo_img = "https://user-images.githubusercontent.com/3985464/113734260-7ba06900-96fb-11eb-82bc-d4f68a1e2710.png"
 square_logo_img = "https://user-images.githubusercontent.com/38561120/235905862-9ece5bd7-9aa3-4e48-807a-3a9035eb8bfb.png"
@@ -22,6 +22,9 @@ gcp_logo = "https://user-images.githubusercontent.com/38561120/235928332-eb4accd
 orange_color = "\033[38;5;208m"
 banner_color = "\033[1;92m"
 
+# Severities
+valid_severities = ["critical", "high", "medium", "low", "informational"]
+
 # Compliance
 actual_directory = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
 

prowler/config/config.yaml

@@ -69,8 +69,8 @@ aws:
   # AWS Organizations
   # organizations_scp_check_deny_regions
   # organizations_enabled_regions: [
-  #   'eu-central-1',
-  #   'eu-west-1',
+  #   "eu-central-1",
+  #   "eu-west-1",
   #   "us-east-1"
   # ]
   organizations_enabled_regions: []

prowler/config/custom_checks_metadata_example.yaml

@@ -0,0 +1,15 @@
+CustomChecksMetadata:
+  aws:
+    Checks:
+      s3_bucket_level_public_access_block:
+        Severity: high
+      s3_bucket_no_mfa_delete:
+        Severity: high
+  azure:
+    Checks:
+      storage_infrastructure_encryption_is_enabled:
+        Severity: medium
+  gcp:
+    Checks:
+      compute_instance_public_ip:
+        Severity: critical

prowler/lib/check/check.py

@@ -16,6 +16,7 @@ from colorama import Fore, Style
 import prowler
 from prowler.config.config import orange_color
 from prowler.lib.check.compliance_models import load_compliance_framework
+from prowler.lib.check.custom_checks_metadata import update_check_metadata
 from prowler.lib.check.models import Check, load_check_metadata
 from prowler.lib.logger import logger
 from prowler.lib.outputs.outputs import report
@@ -106,14 +107,20 @@ def exclude_services_to_run(
 
 # Load checks from checklist.json
 def parse_checks_from_file(input_file: str, provider: str) -> set:
-    checks_to_execute = set()
-    with open_file(input_file) as f:
-        json_file = parse_json_file(f)
+    """parse_checks_from_file returns a set of checks read from the given file"""
+    try:
+        checks_to_execute = set()
+        with open_file(input_file) as f:
+            json_file = parse_json_file(f)
 
-    for check_name in json_file[provider]:
-        checks_to_execute.add(check_name)
+        for check_name in json_file[provider]:
+            checks_to_execute.add(check_name)
 
-    return checks_to_execute
+        return checks_to_execute
+    except Exception as error:
+        logger.error(
+            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}] -- {error}"
+        )
 
 
 # Load checks from custom folder
@@ -309,7 +316,7 @@ def print_checks(
 def parse_checks_from_compliance_framework(
     compliance_frameworks: list, bulk_compliance_frameworks: dict
 ) -> list:
-    """Parse checks from compliance frameworks specification"""
+    """parse_checks_from_compliance_framework returns a set of checks from the given compliance_frameworks"""
     checks_to_execute = set()
     try:
         for framework in compliance_frameworks:
@@ -416,6 +423,7 @@ def execute_checks(
     provider: str,
     audit_info: Any,
     audit_output_options: Provider_Output_Options,
+    custom_checks_metadata: Any,
 ) -> list:
     # List to store all the check's findings
     all_findings = []
@@ -461,6 +469,7 @@ def execute_checks(
                     audit_info,
                     services_executed,
                     checks_executed,
+                    custom_checks_metadata,
                 )
                 all_findings.extend(check_findings)
 
@@ -506,6 +515,7 @@ def execute_checks(
                         audit_info,
                         services_executed,
                         checks_executed,
+                        custom_checks_metadata,
                     )
                     all_findings.extend(check_findings)
 
@@ -531,6 +541,7 @@ def execute(
     audit_info: Any,
     services_executed: set,
     checks_executed: set,
+    custom_checks_metadata: Any,
 ):
     # Import check module
     check_module_path = (
@@ -541,6 +552,10 @@ def execute(
     check_to_execute = getattr(lib, check_name)
     c = check_to_execute()
 
+    # Update check metadata to reflect that in the outputs
+    if custom_checks_metadata and custom_checks_metadata["Checks"].get(c.CheckID):
+        c = update_check_metadata(c, custom_checks_metadata["Checks"][c.CheckID])
+
     # Run check
     check_findings = run_check(c, audit_output_options)
 
@@ -598,22 +613,32 @@ def update_audit_metadata(
         )
 
 
-def recover_checks_from_service(service_list: list, provider: str) -> list:
-    checks = set()
-    service_list = [
-        "awslambda" if service == "lambda" else service for service in service_list
-    ]
-    for service in service_list:
-        modules = recover_checks_from_provider(provider, service)
-        if not modules:
-            logger.error(f"Service '{service}' does not have checks.")
+def recover_checks_from_service(service_list: list, provider: str) -> set:
+    """
+    Recover all checks from the selected provider and service
 
-        else:
-            for check_module in modules:
-                # Recover check name and module name from import path
-                # Format: "providers.{provider}.services.{service}.{check_name}.{check_name}"
-                check_name = check_module[0].split(".")[-1]
-                # If the service is present in the group list passed as parameters
-                # if service_name in group_list: checks_from_arn.add(check_name)
-                checks.add(check_name)
-    return checks
+    Returns a set of checks from the given services
+    """
+    try:
+        checks = set()
+        service_list = [
+            "awslambda" if service == "lambda" else service for service in service_list
+        ]
+        for service in service_list:
+            service_checks = recover_checks_from_provider(provider, service)
+            if not service_checks:
+                logger.error(f"Service '{service}' does not have checks.")
+
+            else:
+                for check in service_checks:
+                    # Recover check name and module name from import path
+                    # Format: "providers.{provider}.services.{service}.{check_name}.{check_name}"
+                    check_name = check[0].split(".")[-1]
+                    # If the service is present in the group list passed as parameters
+                    # if service_name in group_list: checks_from_arn.add(check_name)
+                    checks.add(check_name)
+        return checks
+    except Exception as error:
+        logger.error(
+            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+        )

prowler/lib/check/checks_loader.py

@@ -1,5 +1,6 @@
 from colorama import Fore, Style
 
+from prowler.config.config import valid_severities
 from prowler.lib.check.check import (
     parse_checks_from_compliance_framework,
     parse_checks_from_file,
@@ -10,7 +11,6 @@ from prowler.lib.logger import logger
 
 
 # Generate the list of checks to execute
-# PENDING Test for this function
 def load_checks_to_execute(
     bulk_checks_metadata: dict,
     bulk_compliance_frameworks: dict,
@@ -22,73 +22,93 @@ def load_checks_to_execute(
     categories: set,
     provider: str,
 ) -> set:
-    """Generate the list of checks to execute based on the cloud provider and input arguments specified"""
-    checks_to_execute = set()
+    """Generate the list of checks to execute based on the cloud provider and the input arguments given"""
+    try:
+        # Local subsets
+        checks_to_execute = set()
+        check_aliases = {}
+        check_severities = {key: [] for key in valid_severities}
+        check_categories = {}
 
-    # Handle if there are checks passed using -c/--checks
-    if check_list:
-        for check_name in check_list:
-            checks_to_execute.add(check_name)
+        # First, loop over the bulk_checks_metadata to extract the needed subsets
+        for check, metadata in bulk_checks_metadata.items():
+            # Aliases
+            for alias in metadata.CheckAliases:
+                check_aliases[alias] = check
 
-    # Handle if there are some severities passed using --severity
-    elif severities:
-        for check in bulk_checks_metadata:
-            # Check check's severity
-            if bulk_checks_metadata[check].Severity in severities:
-                checks_to_execute.add(check)
-        if service_list:
-            checks_to_execute = (
-                recover_checks_from_service(service_list, provider) & checks_to_execute
-            )
+            # Severities
+            if metadata.Severity:
+                check_severities[metadata.Severity].append(check)
 
-    # Handle if there are checks passed using -C/--checks-file
-    elif checks_file:
-        try:
+            # Categories
+            for category in metadata.Categories:
+                if category not in check_categories:
+                    check_categories[category] = []
+                check_categories[category].append(check)
+
+        # Handle if there are checks passed using -c/--checks
+        if check_list:
+            for check_name in check_list:
+                checks_to_execute.add(check_name)
+
+        # Handle if there are some severities passed using --severity
+        elif severities:
+            for severity in severities:
+                checks_to_execute.update(check_severities[severity])
+
+            if service_list:
+                checks_to_execute = (
+                    recover_checks_from_service(service_list, provider)
+                    & checks_to_execute
+                )
+
+        # Handle if there are checks passed using -C/--checks-file
+        elif checks_file:
             checks_to_execute = parse_checks_from_file(checks_file, provider)
-        except Exception as e:
-            logger.error(f"{e.__class__.__name__}[{e.__traceback__.tb_lineno}] -- {e}")
 
-    # Handle if there are services passed using -s/--services
-    elif service_list:
-        checks_to_execute = recover_checks_from_service(service_list, provider)
+        # Handle if there are services passed using -s/--services
+        elif service_list:
+            checks_to_execute = recover_checks_from_service(service_list, provider)
 
-    # Handle if there are compliance frameworks passed using --compliance
-    elif compliance_frameworks:
-        try:
+        # Handle if there are compliance frameworks passed using --compliance
+        elif compliance_frameworks:
             checks_to_execute = parse_checks_from_compliance_framework(
                 compliance_frameworks, bulk_compliance_frameworks
             )
-        except Exception as e:
-            logger.error(f"{e.__class__.__name__}[{e.__traceback__.tb_lineno}] -- {e}")
 
-    # Handle if there are categories passed using --categories
-    elif categories:
-        for cat in categories:
-            for check in bulk_checks_metadata:
-                # Check check's categories
-                if cat in bulk_checks_metadata[check].Categories:
-                    checks_to_execute.add(check)
+        # Handle if there are categories passed using --categories
+        elif categories:
+            for category in categories:
+                checks_to_execute.update(check_categories[category])
 
-    # If there are no checks passed as argument
-    else:
-        try:
+        # If there are no checks passed as argument
+        else:
             # Get all check modules to run with the specific provider
             checks = recover_checks_from_provider(provider)
-        except Exception as e:
-            logger.error(f"{e.__class__.__name__}[{e.__traceback__.tb_lineno}] -- {e}")
-        else:
+
             for check_info in checks:
                 # Recover check name from import path (last part)
                 # Format: "providers.{provider}.services.{service}.{check_name}.{check_name}"
                 check_name = check_info[0]
                 checks_to_execute.add(check_name)
 
-    # Get Check Aliases mapping
-    check_aliases = {}
-    for check, metadata in bulk_checks_metadata.items():
-        for alias in metadata.CheckAliases:
-            check_aliases[alias] = check
+        # Check Aliases
+        checks_to_execute = update_checks_to_execute_with_aliases(
+            checks_to_execute, check_aliases
+        )
 
+        return checks_to_execute
+
+    except Exception as error:
+        logger.error(
+            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}] -- {error}"
+        )
+
+
+def update_checks_to_execute_with_aliases(
+    checks_to_execute: set, check_aliases: dict
+) -> set:
+    """update_checks_to_execute_with_aliases returns the checks_to_execute updated using the check aliases."""
     # Verify if any input check is an alias of another check
     for input_check in checks_to_execute:
         if (
@@ -101,5 +121,4 @@ def load_checks_to_execute(
             print(
                 f"\nUsing alias {Fore.YELLOW}{input_check}{Style.RESET_ALL} for check {Fore.YELLOW}{check_aliases[input_check]}{Style.RESET_ALL}...\n"
             )
-
     return checks_to_execute

prowler/lib/check/custom_checks_metadata.py

@@ -0,0 +1,77 @@
+import sys
+
+import yaml
+from jsonschema import validate
+
+from prowler.config.config import valid_severities
+from prowler.lib.logger import logger
+
+custom_checks_metadata_schema = {
+    "type": "object",
+    "properties": {
+        "Checks": {
+            "type": "object",
+            "patternProperties": {
+                ".*": {
+                    "type": "object",
+                    "properties": {
+                        "Severity": {
+                            "type": "string",
+                            "enum": valid_severities,
+                        }
+                    },
+                    "required": ["Severity"],
+                    "additionalProperties": False,
+                }
+            },
+            "additionalProperties": False,
+        }
+    },
+    "required": ["Checks"],
+    "additionalProperties": False,
+}
+
+
+def parse_custom_checks_metadata_file(provider: str, parse_custom_checks_metadata_file):
+    """parse_custom_checks_metadata_file returns the custom_checks_metadata object if it is valid, otherwise aborts the execution returning the ValidationError."""
+    try:
+        with open(parse_custom_checks_metadata_file) as f:
+            custom_checks_metadata = yaml.safe_load(f)["CustomChecksMetadata"][provider]
+            validate(custom_checks_metadata, schema=custom_checks_metadata_schema)
+        return custom_checks_metadata
+    except Exception as error:
+        logger.critical(
+            f"{error.__class__.__name__} -- {error}[{error.__traceback__.tb_lineno}]"
+        )
+        sys.exit(1)
+
+
+def update_checks_metadata(bulk_checks_metadata, custom_checks_metadata):
+    """update_checks_metadata returns the bulk_checks_metadata with the check's metadata updated based on the custom_checks_metadata provided."""
+    try:
+        # Update checks metadata from CustomChecksMetadata file
+        for check, custom_metadata in custom_checks_metadata["Checks"].items():
+            check_metadata = bulk_checks_metadata.get(check)
+            if check_metadata:
+                bulk_checks_metadata[check] = update_check_metadata(
+                    check_metadata, custom_metadata
+                )
+        return bulk_checks_metadata
+    except Exception as error:
+        logger.critical(
+            f"{error.__class__.__name__} -- {error}[{error.__traceback__.tb_lineno}]"
+        )
+        sys.exit(1)
+
+
+def update_check_metadata(check_metadata, custom_metadata):
+    """update_check_metadata updates the check_metadata fields present in the custom_metadata and returns the updated version of the check_metadata. If some field is not present or valid the check_metadata is returned with the original fields."""
+    try:
+        if custom_metadata:
+            for attribute in custom_metadata:
+                try:
+                    setattr(check_metadata, attribute, custom_metadata[attribute])
+                except ValueError:
+                    pass
+    finally:
+        return check_metadata

prowler/lib/cli/parser.py

@@ -7,6 +7,7 @@ from prowler.config.config import (
     check_current_version,
     default_config_file_path,
     default_output_directory,
+    valid_severities,
 )
 from prowler.providers.common.arguments import (
     init_providers_parser,
@@ -49,6 +50,7 @@ Detailed documentation at https://docs.prowler.cloud
         self.__init_exclude_checks_parser__()
         self.__init_list_checks_parser__()
         self.__init_config_parser__()
+        self.__init_custom_checks_metadata_parser__()
 
         # Init Providers Arguments
         init_providers_parser(self)
@@ -223,8 +225,8 @@ Detailed documentation at https://docs.prowler.cloud
         common_checks_parser.add_argument(
             "--severity",
             nargs="+",
-            help="List of severities to be executed [informational, low, medium, high, critical]",
-            choices=["informational", "low", "medium", "high", "critical"],
+            help=f"List of severities to be executed {valid_severities}",
+            choices=valid_severities,
         )
         group.add_argument(
             "--compliance",
@@ -286,3 +288,15 @@ Detailed documentation at https://docs.prowler.cloud
             default=default_config_file_path,
             help="Set configuration file path",
         )
+
+    def __init_custom_checks_metadata_parser__(self):
+        # CustomChecksMetadata
+        custom_checks_metadata_subparser = (
+            self.common_providers_parser.add_argument_group("Custom Checks Metadata")
+        )
+        custom_checks_metadata_subparser.add_argument(
+            "--custom-checks-metadata-file",
+            nargs="?",
+            default=None,
+            help="Path for the custom checks metadata YAML file. See example prowler/config/custom_checks_metadata_example.yaml for reference and format. See more in https://docs.prowler.cloud/en/latest/tutorials/custom-checks-metadata/",
+        )

prowler/lib/outputs/compliance.py

@@ -401,7 +401,8 @@ def display_compliance_table(
                                         "Bajo": 0,
                                     }
                                 if finding.status == "FAIL":
-                                    fail_count += 1
+                                    if attribute.Tipo != "recomendacion":
+                                        fail_count += 1
                                     marcos[marco_categoria][
                                         "Estado"
                                     ] = f"{Fore.RED}NO CUMPLE{Style.RESET_ALL}"

prowler/lib/outputs/file_descriptors.py

@@ -12,8 +12,6 @@ from prowler.config.config import (
 from prowler.lib.logger import logger
 from prowler.lib.outputs.html import add_html_header
 from prowler.lib.outputs.models import (
-    Aws_Check_Output_CSV,
-    Azure_Check_Output_CSV,
     Check_Output_CSV_AWS_CIS,
     Check_Output_CSV_AWS_ISO27001_2013,
     Check_Output_CSV_AWS_Well_Architected,
@@ -21,19 +19,18 @@ from prowler.lib.outputs.models import (
     Check_Output_CSV_GCP_CIS,
     Check_Output_CSV_Generic_Compliance,
     Check_Output_MITRE_ATTACK,
-    Gcp_Check_Output_CSV,
     generate_csv_fields,
 )
 from prowler.lib.utils.utils import file_exists, open_file
 from prowler.providers.aws.lib.audit_info.models import AWS_Audit_Info
-from prowler.providers.azure.lib.audit_info.models import Azure_Audit_Info
+from prowler.providers.common.outputs import get_provider_output_model
 from prowler.providers.gcp.lib.audit_info.models import GCP_Audit_Info
 
 
 def initialize_file_descriptor(
     filename: str,
     output_mode: str,
-    audit_info: AWS_Audit_Info,
+    audit_info: Any,
     format: Any = None,
 ) -> TextIOWrapper:
     """Open/Create the output file. If needed include headers or the required format"""
@@ -75,27 +72,15 @@ def fill_file_descriptors(output_modes, output_directory, output_filename, audit
             for output_mode in output_modes:
                 if output_mode == "csv":
                     filename = f"{output_directory}/{output_filename}{csv_file_suffix}"
-                    if isinstance(audit_info, AWS_Audit_Info):
-                        file_descriptor = initialize_file_descriptor(
-                            filename,
-                            output_mode,
-                            audit_info,
-                            Aws_Check_Output_CSV,
-                        )
-                    if isinstance(audit_info, Azure_Audit_Info):
-                        file_descriptor = initialize_file_descriptor(
-                            filename,
-                            output_mode,
-                            audit_info,
-                            Azure_Check_Output_CSV,
-                        )
-                    if isinstance(audit_info, GCP_Audit_Info):
-                        file_descriptor = initialize_file_descriptor(
-                            filename,
-                            output_mode,
-                            audit_info,
-                            Gcp_Check_Output_CSV,
-                        )
+                    output_model = get_provider_output_model(
+                        audit_info.__class__.__name__
+                    )
+                    file_descriptor = initialize_file_descriptor(
+                        filename,
+                        output_mode,
+                        audit_info,
+                        output_model,
+                    )
                     file_descriptors.update({output_mode: file_descriptor})
 
                 elif output_mode == "json":

prowler/lib/outputs/html.py

@@ -407,7 +407,7 @@ def get_azure_html_assessment_summary(audit_info):
         if isinstance(audit_info, Azure_Audit_Info):
             printed_subscriptions = []
             for key, value in audit_info.identity.subscriptions.items():
-                intermediate = key + " : " + value
+                intermediate = f"{key} : {value}"
                 printed_subscriptions.append(intermediate)
 
             # check if identity is str(coming from SP) or dict(coming from browser or)

prowler/lib/outputs/json.py

@@ -31,6 +31,7 @@ from prowler.lib.outputs.models import (
     unroll_dict_to_list,
 )
 from prowler.lib.utils.utils import hash_sha512, open_file, outputs_unix_timestamp
+from prowler.providers.aws.lib.audit_info.models import AWS_Audit_Info
 
 
 def fill_json_asff(finding_output, audit_info, finding, output_options):
@@ -155,7 +156,11 @@ def fill_json_ocsf(audit_info, finding, output_options) -> Check_Output_JSON_OCS
         aws_org_uid = ""
         account = None
         org = None
-        profile = audit_info.profile if audit_info.profile is not None else "default"
+        profile = ""
+        if isinstance(audit_info, AWS_Audit_Info):
+            profile = (
+                audit_info.profile if audit_info.profile is not None else "default"
+            )
         if (
             hasattr(audit_info, "organizations_metadata")
             and audit_info.organizations_metadata

prowler/lib/outputs/slack.py

@@ -13,7 +13,7 @@ def send_slack_message(token, channel, stats, provider, audit_info):
         response = client.chat_postMessage(
             username="Prowler",
             icon_url=square_logo_img,
-            channel="#" + channel,
+            channel=f"#{channel}",
             blocks=create_message_blocks(identity, logo, stats),
         )
         return response
@@ -35,7 +35,7 @@ def create_message_identity(provider, audit_info):
         elif provider == "azure":
             printed_subscriptions = []
             for key, value in audit_info.identity.subscriptions.items():
-                intermediate = "- *" + key + ": " + value + "*\n"
+                intermediate = f"- *{key}: {value}*\n"
                 printed_subscriptions.append(intermediate)
             identity = f"Azure Subscriptions:\n{''.join(printed_subscriptions)}"
             logo = azure_logo

prowler/providers/aws/aws_provider.py

@@ -10,7 +10,10 @@ from prowler.config.config import aws_services_json_file
 from prowler.lib.check.check import list_modules, recover_checks_from_service
 from prowler.lib.logger import logger
 from prowler.lib.utils.utils import open_file, parse_json_file
-from prowler.providers.aws.config import AWS_STS_GLOBAL_ENDPOINT_REGION
+from prowler.providers.aws.config import (
+    AWS_STS_GLOBAL_ENDPOINT_REGION,
+    ROLE_SESSION_NAME,
+)
 from prowler.providers.aws.lib.audit_info.models import AWS_Assume_Role, AWS_Audit_Info
 from prowler.providers.aws.lib.credentials.credentials import create_sts_session
 
@@ -113,9 +116,15 @@ def assume_role(
     sts_endpoint_region: str = None,
 ) -> dict:
     try:
+        role_session_name = (
+            assumed_role_info.role_session_name
+            if assumed_role_info.role_session_name
+            else ROLE_SESSION_NAME
+        )
+
         assume_role_arguments = {
             "RoleArn": assumed_role_info.role_arn,
-            "RoleSessionName": "ProwlerAsessmentSession",
+            "RoleSessionName": role_session_name,
             "DurationSeconds": assumed_role_info.session_duration,
         }
 
@@ -152,23 +161,31 @@ def input_role_mfa_token_and_code() -> tuple[str]:
 
 
 def generate_regional_clients(
-    service: str, audit_info: AWS_Audit_Info, global_service: bool = False
+    service: str,
+    audit_info: AWS_Audit_Info,
 ) -> dict:
+    """generate_regional_clients returns a dict with the following format for the given service:
+
+    Example:
+        {"eu-west-1": boto3_service_client}
+    """
     try:
         regional_clients = {}
         service_regions = get_available_aws_service_regions(service, audit_info)
-        # Check if it is global service to gather only one region
-        if global_service:
-            if service_regions:
-                if audit_info.profile_region in service_regions:
-                    service_regions = [audit_info.profile_region]
-                service_regions = service_regions[:1]
-        for region in service_regions:
+
+        # Get the regions enabled for the account and get the intersection with the service available regions
+        if audit_info.enabled_regions:
+            enabled_regions = service_regions.intersection(audit_info.enabled_regions)
+        else:
+            enabled_regions = service_regions
+
+        for region in enabled_regions:
             regional_client = audit_info.audit_session.client(
                 service, region_name=region, config=audit_info.session_config
             )
             regional_client.region = region
             regional_clients[region] = regional_client
+
         return regional_clients
     except Exception as error:
         logger.error(
@@ -176,6 +193,26 @@ def generate_regional_clients(
         )
 
 
+def get_aws_enabled_regions(audit_info: AWS_Audit_Info) -> set:
+    """get_aws_enabled_regions returns a set of enabled AWS regions"""
+
+    # EC2 Client to check enabled regions
+    service = "ec2"
+    default_region = get_default_region(service, audit_info)
+    ec2_client = audit_info.audit_session.client(service, region_name=default_region)
+
+    enabled_regions = set()
+    try:
+        # With AllRegions=False we only get the enabled regions for the account
+        for region in ec2_client.describe_regions(AllRegions=False).get("Regions", []):
+            enabled_regions.add(region.get("RegionName"))
+    except Exception as error:
+        logger.warning(
+            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+        )
+    return enabled_regions
+
+
 def get_aws_available_regions():
     try:
         actual_directory = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
@@ -216,6 +253,8 @@ def get_checks_from_input_arn(audit_resources: list, provider: str) -> set:
                     service = "efs"
                 elif service == "logs":
                     service = "cloudwatch"
+                elif service == "cognito":
+                    service = "cognito-idp"
                 # Check if Prowler has checks in service
                 try:
                     list_modules(provider, service)
@@ -267,17 +306,18 @@ def get_regions_from_audit_resources(audit_resources: list) -> set:
     return audited_regions
 
 
-def get_available_aws_service_regions(service: str, audit_info: AWS_Audit_Info) -> list:
+def get_available_aws_service_regions(service: str, audit_info: AWS_Audit_Info) -> set:
     # Get json locally
     actual_directory = pathlib.Path(os.path.dirname(os.path.realpath(__file__)))
     with open_file(f"{actual_directory}/{aws_services_json_file}") as f:
         data = parse_json_file(f)
-    # Check if it is a subservice
-    json_regions = data["services"][service]["regions"][audit_info.audited_partition]
-    if audit_info.audited_regions:  # Check for input aws audit_info.audited_regions
-        regions = list(
-            set(json_regions).intersection(audit_info.audited_regions)
-        )  # Get common regions between input and json
+    json_regions = set(
+        data["services"][service]["regions"][audit_info.audited_partition]
+    )
+    # Check for input aws audit_info.audited_regions
+    if audit_info.audited_regions:
+        # Get common regions between input and json
+        regions = json_regions.intersection(audit_info.audited_regions)
     else:  # Get all regions from json of the service and partition
         regions = json_regions
     return regions

prowler/providers/aws/config.py

@@ -1,2 +1,3 @@
 AWS_STS_GLOBAL_ENDPOINT_REGION = "us-east-1"
 BOTO3_USER_AGENT_EXTRA = "APN_1826889"
+ROLE_SESSION_NAME = "ProwlerAssessmentSession"

prowler/providers/aws/lib/allowlist/allowlist.py

@@ -143,29 +143,23 @@ def is_allowlisted(
     finding_tags,
 ):
     try:
-        allowlisted_checks = {}
         # By default is not allowlisted
         is_finding_allowlisted = False
-        # First set account key from allowlist dict
-        if audited_account in allowlist["Accounts"]:
-            allowlisted_checks = allowlist["Accounts"][audited_account]["Checks"]
-        # If there is a *, it affects to all accounts
-        # This cannot be elif since in the case of * and single accounts we
-        # want to merge allowlisted checks from * to the other accounts check list
-        if "*" in allowlist["Accounts"]:
-            checks_multi_account = allowlist["Accounts"]["*"]["Checks"]
-            allowlisted_checks.update(checks_multi_account)
 
-        # Test if it is allowlisted
-        if is_allowlisted_in_check(
-            allowlisted_checks,
-            audited_account,
-            check,
-            finding_region,
-            finding_resource,
-            finding_tags,
-        ):
-            is_finding_allowlisted = True
+        # We always check all the accounts present in the allowlist
+        # if one allowlists the finding we set the finding as allowlisted
+        for account in allowlist["Accounts"]:
+            if account == audited_account or account == "*":
+                if is_allowlisted_in_check(
+                    allowlist["Accounts"][account]["Checks"],
+                    audited_account,
+                    check,
+                    finding_region,
+                    finding_resource,
+                    finding_tags,
+                ):
+                    is_finding_allowlisted = True
+                    break
 
         return is_finding_allowlisted
     except Exception as error:
@@ -310,10 +304,17 @@ def is_excepted(
             is_tag_excepted = __is_item_matched__(excepted_tags, finding_tags)
 
             if (
-                is_account_excepted
-                and is_region_excepted
-                and is_resource_excepted
-                and is_tag_excepted
+                not is_account_excepted
+                and not is_region_excepted
+                and not is_resource_excepted
+                and not is_tag_excepted
+            ):
+                excepted = False
+            elif (
+                (is_account_excepted or not excepted_accounts)
+                and (is_region_excepted or not excepted_regions)
+                and (is_resource_excepted or not excepted_resources)
+                and (is_tag_excepted or not excepted_tags)
             ):
                 excepted = True
         return excepted

prowler/providers/aws/lib/arguments/arguments.py

@@ -1,6 +1,8 @@
 from argparse import ArgumentTypeError, Namespace
+from re import fullmatch, search
 
 from prowler.providers.aws.aws_provider import get_aws_available_regions
+from prowler.providers.aws.config import ROLE_SESSION_NAME
 from prowler.providers.aws.lib.arn.arn import arn_type
 
 
@@ -26,6 +28,13 @@ def init_parser(self):
         help="ARN of the role to be assumed",
         # Pending ARN validation
     )
+    aws_auth_subparser.add_argument(
+        "--role-session-name",
+        nargs="?",
+        default=ROLE_SESSION_NAME,
+        help="An identifier for the assumed role session. Defaults to ProwlerAssessmentSession",
+        type=validate_role_session_name,
+    )
     aws_auth_subparser.add_argument(
         "--sts-endpoint-region",
         nargs="?",
@@ -84,6 +93,11 @@ def init_parser(self):
         action="store_true",
         help="Skip updating previous findings of Prowler in Security Hub",
     )
+    aws_security_hub_subparser.add_argument(
+        "--send-sh-only-fails",
+        action="store_true",
+        help="Send only Prowler failed findings to SecurityHub",
+    )
     # AWS Quick Inventory
     aws_quick_inventory_subparser = aws_parser.add_argument_group("Quick Inventory")
     aws_quick_inventory_subparser.add_argument(
@@ -99,6 +113,7 @@ def init_parser(self):
         "-B",
         "--output-bucket",
         nargs="?",
+        type=validate_bucket,
         default=None,
         help="Custom output bucket, requires -M <mode> and it can work also with -o flag.",
     )
@@ -106,6 +121,7 @@ def init_parser(self):
         "-D",
         "--output-bucket-no-assume",
         nargs="?",
+        type=validate_bucket,
         default=None,
         help="Same as -B but do not use the assumed role credentials to put objects to the bucket, instead uses the initial credentials.",
     )
@@ -126,6 +142,7 @@ def init_parser(self):
         default=None,
         help="Path for allowlist yaml file. See example prowler/config/aws_allowlist.yaml for reference and format. It also accepts AWS DynamoDB Table or Lambda ARNs or S3 URIs, see more in https://docs.prowler.cloud/en/latest/tutorials/allowlist/",
     )
+
     # Based Scans
     aws_based_scans_subparser = aws_parser.add_argument_group("AWS Based Scans")
     aws_based_scans_parser = aws_based_scans_subparser.add_mutually_exclusive_group()
@@ -178,9 +195,37 @@ def validate_arguments(arguments: Namespace) -> tuple[bool, str]:
 
     # Handle if session_duration is not the default value or external_id is set
     if (
-        arguments.session_duration and arguments.session_duration != 3600
-    ) or arguments.external_id:
+        (arguments.session_duration and arguments.session_duration != 3600)
+        or arguments.external_id
+        or arguments.role_session_name != ROLE_SESSION_NAME
+    ):
         if not arguments.role:
-            return (False, "To use -I/-T options -R option is needed")
+            return (
+                False,
+                "To use -I/--external-id, -T/--session-duration or --role-session-name options -R/--role option is needed",
+            )
 
     return (True, "")
+
+
+def validate_bucket(bucket_name):
+    """validate_bucket validates that the input bucket_name is valid"""
+    if search("(?!(^xn--|.+-s3alias$))^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$", bucket_name):
+        return bucket_name
+    else:
+        raise ArgumentTypeError(
+            "Bucket name must be valid (https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html)"
+        )
+
+
+def validate_role_session_name(session_name):
+    """
+    validates that the role session name is valid
+    Documentation: https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html
+    """
+    if fullmatch("[\w+=,.@-]{2,64}", session_name):
+        return session_name
+    else:
+        raise ArgumentTypeError(
+            "Role Session Name must be 2-64 characters long and consist only of upper- and lower-case alphanumeric characters with no spaces. You can also include underscores or any of the following characters: =,.@-"
+        )

prowler/providers/aws/lib/audit_info/audit_info.py

@@ -30,6 +30,7 @@ current_audit_info = AWS_Audit_Info(
         session_duration=None,
         external_id=None,
         mfa_enabled=None,
+        role_session_name=None,
     ),
     mfa_enabled=None,
     audit_resources=None,
@@ -38,4 +39,5 @@ current_audit_info = AWS_Audit_Info(
     audit_metadata=None,
     audit_config=None,
     ignore_unused_services=False,
+    enabled_regions=set(),
 )

prowler/providers/aws/lib/audit_info/models.py

@@ -1,4 +1,4 @@
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from datetime import datetime
 from typing import Any, Optional
 
@@ -20,6 +20,7 @@ class AWS_Assume_Role:
     session_duration: int
     external_id: str
     mfa_enabled: bool
+    role_session_name: str
 
 
 @dataclass
@@ -53,3 +54,4 @@ class AWS_Audit_Info:
     audit_metadata: Optional[Any] = None
     audit_config: Optional[dict] = None
     ignore_unused_services: bool = False
+    enabled_regions: set = field(default_factory=set)

prowler/providers/aws/lib/policy_condition_parser/policy_condition_parser.py

@@ -1,8 +1,11 @@
-def is_account_only_allowed_in_condition(
-    condition_statement: dict, source_account: str
+def is_condition_block_restrictive(
+    condition_statement: dict, source_account: str, is_cross_account_allowed=False
 ):
     """
-    is_account_only_allowed_in_condition parses the IAM Condition policy block and returns True if the source_account passed as argument is within, False if not.
+    is_condition_block_restrictive parses the IAM Condition policy block and, by default, returns True if the source_account passed as argument is within, False if not.
+
+    If argument is_cross_account_allowed is True it tests if the Condition block includes any of the operators allowlisted returning True if does, False if not.
+
 
     @param condition_statement: dict with an IAM Condition block, e.g.:
         {
@@ -54,13 +57,16 @@ def is_account_only_allowed_in_condition(
                         condition_statement[condition_operator][value],
                         list,
                     ):
-                        # if there is an arn/account without the source account -> we do not consider it safe
-                        # here by default we assume is true and look for false entries
                         is_condition_key_restrictive = True
-                        for item in condition_statement[condition_operator][value]:
-                            if source_account not in item:
-                                is_condition_key_restrictive = False
-                                break
+                        # if cross account is not allowed check for each condition block looking for accounts
+                        # different than default
+                        if not is_cross_account_allowed:
+                            # if there is an arn/account without the source account -> we do not consider it safe
+                            # here by default we assume is true and look for false entries
+                            for item in condition_statement[condition_operator][value]:
+                                if source_account not in item:
+                                    is_condition_key_restrictive = False
+                                    break
 
                         if is_condition_key_restrictive:
                             is_condition_valid = True
@@ -70,10 +76,13 @@ def is_account_only_allowed_in_condition(
                         condition_statement[condition_operator][value],
                         str,
                     ):
-                        if (
-                            source_account
-                            in condition_statement[condition_operator][value]
-                        ):
+                        if is_cross_account_allowed:
                             is_condition_valid = True
+                        else:
+                            if (
+                                source_account
+                                in condition_statement[condition_operator][value]
+                            ):
+                                is_condition_valid = True
 
     return is_condition_valid

prowler/providers/aws/lib/s3/s3.py

@@ -1,5 +1,3 @@
-import sys
-
 from prowler.config.config import (
     csv_file_suffix,
     html_file_suffix,
@@ -29,7 +27,7 @@ def send_to_s3_bucket(
         else:  # Compliance output mode
             filename = f"{output_filename}_{output_mode}{csv_file_suffix}"
 
-        logger.info(f"Sending outputs to S3 bucket {output_bucket_name}")
+        logger.info(f"Sending output file {filename} to S3 bucket {output_bucket_name}")
         # File location
         file_name = output_directory + "/" + filename
 
@@ -41,10 +39,9 @@ def send_to_s3_bucket(
         s3_client.upload_file(file_name, output_bucket_name, object_name)
 
     except Exception as error:
-        logger.critical(
+        logger.error(
             f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}] -- {error}"
         )
-        sys.exit(1)
 
 
 def get_s3_object_path(output_directory: str) -> str:

prowler/providers/aws/lib/security_hub/security_hub.py

@@ -29,7 +29,9 @@ def prepare_security_hub_findings(
             continue
 
         # Handle quiet mode
-        if output_options.is_quiet and finding.status != "FAIL":
+        if (
+            output_options.is_quiet or output_options.send_sh_only_fails
+        ) and finding.status != "FAIL":
             continue
 
         # Get the finding region

prowler/providers/aws/lib/service/service.py

@@ -1,17 +1,21 @@
-import threading
+from concurrent.futures import ThreadPoolExecutor, as_completed
 
+from prowler.lib.logger import logger
 from prowler.providers.aws.aws_provider import (
     generate_regional_clients,
     get_default_region,
 )
 from prowler.providers.aws.lib.audit_info.models import AWS_Audit_Info
 
+MAX_WORKERS = 10
+
 
 class AWSService:
     """The AWSService class offers a parent class for each AWS Service to generate:
     - AWS Regional Clients
     - Shared information like the account ID and ARN, the the AWS partition and the checks audited
     - AWS Session
+    - Thread pool for the __threading_call__
     - Also handles if the AWS Service is Global
     """
 
@@ -34,9 +38,7 @@ class AWSService:
 
         # Generate Regional Clients
         if not global_service:
-            self.regional_clients = generate_regional_clients(
-                self.service, audit_info, global_service
-            )
+            self.regional_clients = generate_regional_clients(self.service, audit_info)
 
         # Get a single region and client if the service needs it (e.g. AWS Global Service)
         # We cannot include this within an else because some services needs both the regional_clients
@@ -44,14 +46,40 @@ class AWSService:
         self.region = get_default_region(self.service, audit_info)
         self.client = self.session.client(self.service, self.region)
 
+        # Thread pool for __threading_call__
+        self.thread_pool = ThreadPoolExecutor(max_workers=MAX_WORKERS)
+
     def __get_session__(self):
         return self.session
 
-    def __threading_call__(self, call):
-        threads = []
-        for regional_client in self.regional_clients.values():
-            threads.append(threading.Thread(target=call, args=(regional_client,)))
-        for t in threads:
-            t.start()
-        for t in threads:
-            t.join()
+    def __threading_call__(self, call, iterator=None):
+        # Use the provided iterator, or default to self.regional_clients
+        items = iterator if iterator is not None else self.regional_clients.values()
+        # Determine the total count for logging
+        item_count = len(items)
+
+        # Trim leading and trailing underscores from the call's name
+        call_name = call.__name__.strip("_")
+        # Add Capitalization
+        call_name = " ".join([x.capitalize() for x in call_name.split("_")])
+
+        # Print a message based on the call's name, and if its regional or processing a list of items
+        if iterator is None:
+            logger.info(
+                f"{self.service.upper()} - Starting threads for '{call_name}' function across {item_count} regions..."
+            )
+        else:
+            logger.info(
+                f"{self.service.upper()} - Starting threads for '{call_name}' function to process {item_count} items..."
+            )
+
+        # Submit tasks to the thread pool
+        futures = [self.thread_pool.submit(call, item) for item in items]
+
+        # Wait for all tasks to complete
+        for future in as_completed(futures):
+            try:
+                future.result()  # Raises exceptions from the thread, if any
+            except Exception:
+                # Handle exceptions if necessary
+                pass  # Replace 'pass' with any additional exception handling logic. Currently handled within the called function

prowler/providers/aws/services/accessanalyzer/accessanalyzer_service.py

@@ -85,21 +85,36 @@ class AccessAnalyzer(AWSService):
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
 
+    # TODO: We need to include ListFindingsV2
+    # https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/accessanalyzer/client/list_findings_v2.html
     def __list_findings__(self):
         logger.info("AccessAnalyzer - Listing Findings per Analyzer...")
         try:
             for analyzer in self.analyzers:
-                if analyzer.status == "ACTIVE":
-                    regional_client = self.regional_clients[analyzer.region]
-                    list_findings_paginator = regional_client.get_paginator(
-                        "list_findings"
+                try:
+                    if analyzer.status == "ACTIVE":
+                        regional_client = self.regional_clients[analyzer.region]
+                        list_findings_paginator = regional_client.get_paginator(
+                            "list_findings"
+                        )
+                        for page in list_findings_paginator.paginate(
+                            analyzerArn=analyzer.arn
+                        ):
+                            for finding in page["findings"]:
+                                analyzer.findings.append(Finding(id=finding["id"]))
+                except ClientError as error:
+                    if error.response["Error"]["Code"] == "ValidationException":
+                        logger.warning(
+                            f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                        )
+                    else:
+                        logger.error(
+                            f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                        )
+                except Exception as error:
+                    logger.error(
+                        f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
                     )
-                    for page in list_findings_paginator.paginate(
-                        analyzerArn=analyzer.arn
-                    ):
-                        for finding in page["findings"]:
-                            analyzer.findings.append(Finding(id=finding["id"]))
-
         except Exception as error:
             logger.error(
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"

prowler/providers/aws/services/apigateway/apigateway_restapi_authorizers_enabled/apigateway_restapi_authorizers_enabled.metadata.json

@@ -1,7 +1,7 @@
 {
   "Provider": "aws",
   "CheckID": "apigateway_restapi_authorizers_enabled",
-  "CheckTitle": "Check if API Gateway has configured authorizers.",
+  "CheckTitle": "Check if API Gateway has configured authorizers at api or method level.",
   "CheckAliases": [
     "apigateway_authorizers_enabled"
   ],
@@ -13,7 +13,7 @@
   "ResourceIdTemplate": "arn:partition:service:region:account-id:resource-id",
   "Severity": "medium",
   "ResourceType": "AwsApiGatewayRestApi",
-  "Description": "Check if API Gateway has configured authorizers.",
+  "Description": "Check if API Gateway has configured authorizers at api or method level.",
   "Risk": "If no authorizer is enabled anyone can use the service.",
   "RelatedUrl": "",
   "Remediation": {

prowler/providers/aws/services/apigateway/apigateway_restapi_authorizers_enabled/apigateway_restapi_authorizers_enabled.py

@@ -13,12 +13,41 @@ class apigateway_restapi_authorizers_enabled(Check):
             report.resource_id = rest_api.name
             report.resource_arn = rest_api.arn
             report.resource_tags = rest_api.tags
+            # it there are not authorizers at api level and resources without methods (default case) ->
+            report.status = "FAIL"
+            report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} does not have an authorizer configured at api level."
             if rest_api.authorizer:
                 report.status = "PASS"
-                report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} has an authorizer configured."
+                report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} has an authorizer configured at api level"
             else:
-                report.status = "FAIL"
-                report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} does not have an authorizer configured."
+                # we want to know if api has not authorizers and all the resources don't have methods configured
+                resources_have_methods = False
+                all_methods_authorized = True
+                resource_paths_with_unathorized_methods = []
+                for resource in rest_api.resources:
+                    # if the resource has methods test if they have all configured authorizer
+                    if resource.resource_methods:
+                        resources_have_methods = True
+                        for (
+                            http_method,
+                            authorization_method,
+                        ) in resource.resource_methods.items():
+                            if authorization_method == "NONE":
+                                all_methods_authorized = False
+                                unauthorized_method = (
+                                    f"{resource.path} -> {http_method}"
+                                )
+                                resource_paths_with_unathorized_methods.append(
+                                    unauthorized_method
+                                )
+                # if there are methods in at least one resource and are all authorized
+                if all_methods_authorized and resources_have_methods:
+                    report.status = "PASS"
+                    report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} has all methods authorized"
+                # if there are methods in at least one result but some of then are not authorized-> list it
+                elif not all_methods_authorized:
+                    report.status_extended = f"API Gateway {rest_api.name} ID {rest_api.id} does not have authorizers at api level and the following paths and methods are unauthorized: {'; '.join(resource_paths_with_unathorized_methods)}."
+
             findings.append(report)
 
         return findings

prowler/providers/aws/services/apigateway/apigateway_service.py

@@ -17,6 +17,7 @@ class APIGateway(AWSService):
         self.__get_authorizers__()
         self.__get_rest_api__()
         self.__get_stages__()
+        self.__get_resources__()
 
     def __get_rest_apis__(self, regional_client):
         logger.info("APIGateway - Getting Rest APIs...")
@@ -53,7 +54,9 @@ class APIGateway(AWSService):
                 if authorizers:
                     rest_api.authorizer = True
         except Exception as error:
-            logger.error(f"{error.__class__.__name__}: {error}")
+            logger.error(
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
 
     def __get_rest_api__(self):
         logger.info("APIGateway - Describing Rest API...")
@@ -64,7 +67,9 @@ class APIGateway(AWSService):
                 if rest_api_info["endpointConfiguration"]["types"] == ["PRIVATE"]:
                     rest_api.public_endpoint = False
         except Exception as error:
-            logger.error(f"{error.__class__.__name__}: {error}")
+            logger.error(
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
 
     def __get_stages__(self):
         logger.info("APIGateway - Getting stages for Rest APIs...")
@@ -95,7 +100,46 @@ class APIGateway(AWSService):
                         )
                     )
         except Exception as error:
-            logger.error(f"{error.__class__.__name__}: {error}")
+            logger.error(
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
+
+    def __get_resources__(self):
+        logger.info("APIGateway - Getting API resources...")
+        try:
+            for rest_api in self.rest_apis:
+                regional_client = self.regional_clients[rest_api.region]
+                get_resources_paginator = regional_client.get_paginator("get_resources")
+                for page in get_resources_paginator.paginate(restApiId=rest_api.id):
+                    for resource in page["items"]:
+                        id = resource["id"]
+                        resource_methods = []
+                        methods_auth = {}
+                        for resource_method in resource.get(
+                            "resourceMethods", {}
+                        ).keys():
+                            resource_methods.append(resource_method)
+
+                        for resource_method in resource_methods:
+                            if resource_method != "OPTIONS":
+                                method_config = regional_client.get_method(
+                                    restApiId=rest_api.id,
+                                    resourceId=id,
+                                    httpMethod=resource_method,
+                                )
+                                auth_type = method_config["authorizationType"]
+                                methods_auth.update({resource_method: auth_type})
+
+                        rest_api.resources.append(
+                            PathResourceMethods(
+                                path=resource["path"], resource_methods=methods_auth
+                            )
+                        )
+
+        except Exception as error:
+            logger.error(
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
 
 
 class Stage(BaseModel):
@@ -107,6 +151,11 @@ class Stage(BaseModel):
     tags: Optional[list] = []
 
 
+class PathResourceMethods(BaseModel):
+    path: str
+    resource_methods: dict
+
+
 class RestAPI(BaseModel):
     id: str
     arn: str
@@ -116,3 +165,4 @@ class RestAPI(BaseModel):
     public_endpoint: bool = True
     stages: list[Stage] = []
     tags: Optional[list] = []
+    resources: list[PathResourceMethods] = []

prowler/providers/aws/services/apigatewayv2/apigatewayv2_api_access_logging_enabled/apigatewayv2_api_access_logging_enabled.py

@@ -14,13 +14,13 @@ class apigatewayv2_api_access_logging_enabled(Check):
                 if stage.logging:
                     report.status = "PASS"
                     report.status_extended = f"API Gateway V2 {api.name} ID {api.id} in stage {stage.name} has access logging enabled."
-                    report.resource_id = api.name
+                    report.resource_id = f"{api.name}-{stage.name}"
                     report.resource_arn = api.arn
                     report.resource_tags = api.tags
                 else:
                     report.status = "FAIL"
                     report.status_extended = f"API Gateway V2 {api.name} ID {api.id} in stage {stage.name} has access logging disabled."
-                    report.resource_id = api.name
+                    report.resource_id = f"{api.name}-{stage.name}"
                     report.resource_arn = api.arn
                     report.resource_tags = api.tags
                 findings.append(report)

prowler/providers/aws/services/awslambda/awslambda_function_no_secrets_in_code/awslambda_function_no_secrets_in_code.py

@@ -11,57 +11,55 @@ from prowler.providers.aws.services.awslambda.awslambda_client import awslambda_
 class awslambda_function_no_secrets_in_code(Check):
     def execute(self):
         findings = []
-        for function in awslambda_client.functions.values():
-            if function.code:
-                report = Check_Report_AWS(self.metadata())
-                report.region = function.region
-                report.resource_id = function.name
-                report.resource_arn = function.arn
-                report.resource_tags = function.tags
+        if awslambda_client.functions:
+            for function, function_code in awslambda_client.__get_function_code__():
+                if function_code:
+                    report = Check_Report_AWS(self.metadata())
+                    report.region = function.region
+                    report.resource_id = function.name
+                    report.resource_arn = function.arn
+                    report.resource_tags = function.tags
 
-                report.status = "PASS"
-                report.status_extended = (
-                    f"No secrets found in Lambda function {function.name} code."
-                )
-                with tempfile.TemporaryDirectory() as tmp_dir_name:
-                    function.code.code_zip.extractall(tmp_dir_name)
-                    # List all files
-                    files_in_zip = next(os.walk(tmp_dir_name))[2]
-                    secrets_findings = []
-                    for file in files_in_zip:
-                        secrets = SecretsCollection()
-                        with default_settings():
-                            secrets.scan_file(f"{tmp_dir_name}/{file}")
-                        detect_secrets_output = secrets.json()
-                        if detect_secrets_output:
-                            for (
-                                file_name
-                            ) in (
-                                detect_secrets_output.keys()
-                            ):  # Appears that only 1 file is being scanned at a time, so could rework this
-                                output_file_name = file_name.replace(
-                                    f"{tmp_dir_name}/", ""
-                                )
-                                secrets_string = ", ".join(
-                                    [
-                                        f"{secret['type']} on line {secret['line_number']}"
-                                        for secret in detect_secrets_output[file_name]
-                                    ]
-                                )
-                                secrets_findings.append(
-                                    f"{output_file_name}: {secrets_string}"
-                                )
+                    report.status = "PASS"
+                    report.status_extended = (
+                        f"No secrets found in Lambda function {function.name} code."
+                    )
+                    with tempfile.TemporaryDirectory() as tmp_dir_name:
+                        function_code.code_zip.extractall(tmp_dir_name)
+                        # List all files
+                        files_in_zip = next(os.walk(tmp_dir_name))[2]
+                        secrets_findings = []
+                        for file in files_in_zip:
+                            secrets = SecretsCollection()
+                            with default_settings():
+                                secrets.scan_file(f"{tmp_dir_name}/{file}")
+                            detect_secrets_output = secrets.json()
+                            if detect_secrets_output:
+                                for (
+                                    file_name
+                                ) in (
+                                    detect_secrets_output.keys()
+                                ):  # Appears that only 1 file is being scanned at a time, so could rework this
+                                    output_file_name = file_name.replace(
+                                        f"{tmp_dir_name}/", ""
+                                    )
+                                    secrets_string = ", ".join(
+                                        [
+                                            f"{secret['type']} on line {secret['line_number']}"
+                                            for secret in detect_secrets_output[
+                                                file_name
+                                            ]
+                                        ]
+                                    )
+                                    secrets_findings.append(
+                                        f"{output_file_name}: {secrets_string}"
+                                    )
 
-                    if secrets_findings:
-                        final_output_string = "; ".join(secrets_findings)
-                        report.status = "FAIL"
-                        # report.status_extended = f"Potential {'secrets' if len(secrets_findings)>1 else 'secret'} found in Lambda function {function.name} code. {final_output_string}."
-                        if len(secrets_findings) > 1:
-                            report.status_extended = f"Potential secrets found in Lambda function {function.name} code -> {final_output_string}."
-                        else:
-                            report.status_extended = f"Potential secret found in Lambda function {function.name} code -> {final_output_string}."
-                        # break // Don't break as there may be additional findings
+                        if secrets_findings:
+                            final_output_string = "; ".join(secrets_findings)
+                            report.status = "FAIL"
+                            report.status_extended = f"Potential {'secrets' if len(secrets_findings) > 1 else 'secret'} found in Lambda function {function.name} code -> {final_output_string}."
 
-                findings.append(report)
+                    findings.append(report)
 
         return findings

prowler/providers/aws/services/awslambda/awslambda_service.py

@@ -1,6 +1,7 @@
 import io
 import json
 import zipfile
+from concurrent.futures import as_completed
 from enum import Enum
 from typing import Any, Optional
 
@@ -21,15 +22,6 @@ class Lambda(AWSService):
         self.functions = {}
         self.__threading_call__(self.__list_functions__)
         self.__list_tags_for_resource__()
-
-        # We only want to retrieve the Lambda code if the
-        # awslambda_function_no_secrets_in_code check is set
-        if (
-            "awslambda_function_no_secrets_in_code"
-            in audit_info.audit_metadata.expected_checks
-        ):
-            self.__threading_call__(self.__get_function__)
-
         self.__threading_call__(self.__get_policy__)
         self.__threading_call__(self.__get_function_url_config__)
 
@@ -70,28 +62,45 @@ class Lambda(AWSService):
                 f" {error}"
             )
 
-    def __get_function__(self, regional_client):
-        logger.info("Lambda - Getting Function...")
-        try:
-            for function in self.functions.values():
-                if function.region == regional_client.region:
-                    function_information = regional_client.get_function(
-                        FunctionName=function.name
-                    )
-                    if "Location" in function_information["Code"]:
-                        code_location_uri = function_information["Code"]["Location"]
-                        raw_code_zip = requests.get(code_location_uri).content
-                        self.functions[function.arn].code = LambdaCode(
-                            location=code_location_uri,
-                            code_zip=zipfile.ZipFile(io.BytesIO(raw_code_zip)),
-                        )
+    def __get_function_code__(self):
+        logger.info("Lambda - Getting Function Code...")
+        # Use a thread pool handle the queueing and execution of the __fetch_function_code__ tasks, up to max_workers tasks concurrently.
+        lambda_functions_to_fetch = {
+            self.thread_pool.submit(
+                self.__fetch_function_code__, function.name, function.region
+            ): function
+            for function in self.functions.values()
+        }
 
+        for fetched_lambda_code in as_completed(lambda_functions_to_fetch):
+            function = lambda_functions_to_fetch[fetched_lambda_code]
+            try:
+                function_code = fetched_lambda_code.result()
+                if function_code:
+                    yield function, function_code
+            except Exception as error:
+                logger.error(
+                    f"{function.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+
+    def __fetch_function_code__(self, function_name, function_region):
+        try:
+            regional_client = self.regional_clients[function_region]
+            function_information = regional_client.get_function(
+                FunctionName=function_name
+            )
+            if "Location" in function_information["Code"]:
+                code_location_uri = function_information["Code"]["Location"]
+                raw_code_zip = requests.get(code_location_uri).content
+                return LambdaCode(
+                    location=code_location_uri,
+                    code_zip=zipfile.ZipFile(io.BytesIO(raw_code_zip)),
+                )
         except Exception as error:
             logger.error(
-                f"{regional_client.region} --"
-                f" {error.__class__.__name__}[{error.__traceback__.tb_lineno}]:"
-                f" {error}"
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
+            raise
 
     def __get_policy__(self, regional_client):
         logger.info("Lambda - Getting Policy...")

prowler/providers/aws/services/cloudtrail/cloudtrail_service.py

@@ -140,7 +140,16 @@ class Cloudtrail(AWSService):
                                 error.response["Error"]["Code"]
                                 == "InsightNotEnabledException"
                             ):
-                                continue
+                                logger.warning(
+                                    f"{client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                                )
+                            elif (
+                                error.response["Error"]["Code"]
+                                == "UnsupportedOperationException"
+                            ):
+                                logger.warning(
+                                    f"{client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                                )
                             else:
                                 logger.error(
                                     f"{client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"

prowler/providers/aws/services/cloudwatch/cloudwatch_changes_to_network_acls_alarm_configured/cloudwatch_changes_to_network_acls_alarm_configured.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_changes_to_network_acls_alarm_configured(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_changes_to_network_gateways_alarm_configured/cloudwatch_changes_to_network_gateways_alarm_configured.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_changes_to_network_gateways_alarm_configured(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_changes_to_network_route_tables_alarm_configured/cloudwatch_changes_to_network_route_tables_alarm_configured.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_changes_to_network_route_tables_alarm_configured(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_changes_to_vpcs_alarm_configured/cloudwatch_changes_to_vpcs_alarm_configured.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_changes_to_vpcs_alarm_configured(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_and_alarm_for_aws_config_configuration_changes_enabled/cloudwatch_log_metric_filter_and_alarm_for_aws_config_configuration_changes_enabled.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -24,26 +25,13 @@ class cloudwatch_log_metric_filter_and_alarm_for_aws_config_configuration_change
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_and_alarm_for_cloudtrail_configuration_changes_enabled/cloudwatch_log_metric_filter_and_alarm_for_cloudtrail_configuration_changes_enabled.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -24,26 +25,13 @@ class cloudwatch_log_metric_filter_and_alarm_for_cloudtrail_configuration_change
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_authentication_failures/cloudwatch_log_metric_filter_authentication_failures.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_authentication_failures(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_aws_organizations_changes/cloudwatch_log_metric_filter_aws_organizations_changes.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_aws_organizations_changes(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_disable_or_scheduled_deletion_of_kms_cmk/cloudwatch_log_metric_filter_disable_or_scheduled_deletion_of_kms_cmk.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_disable_or_scheduled_deletion_of_kms_cmk(Chec
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_for_s3_bucket_policy_changes/cloudwatch_log_metric_filter_for_s3_bucket_policy_changes.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,14 @@ class cloudwatch_log_metric_filter_for_s3_bucket_policy_changes(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_policy_changes/cloudwatch_log_metric_filter_policy_changes.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_policy_changes(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_root_usage/cloudwatch_log_metric_filter_root_usage.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_root_usage(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_security_group_changes/cloudwatch_log_metric_filter_security_group_changes.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_security_group_changes(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_sign_in_without_mfa/cloudwatch_log_metric_filter_sign_in_without_mfa.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_sign_in_without_mfa(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/cloudwatch_log_metric_filter_unauthorized_api_calls/cloudwatch_log_metric_filter_unauthorized_api_calls.py

@@ -1,5 +1,3 @@
-import re
-
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
     cloudtrail_client,
@@ -7,6 +5,9 @@ from prowler.providers.aws.services.cloudtrail.cloudtrail_client import (
 from prowler.providers.aws.services.cloudwatch.cloudwatch_client import (
     cloudwatch_client,
 )
+from prowler.providers.aws.services.cloudwatch.lib.metric_filters import (
+    check_cloudwatch_log_metric_filter,
+)
 from prowler.providers.aws.services.cloudwatch.logs_client import logs_client
 
 
@@ -22,26 +23,13 @@ class cloudwatch_log_metric_filter_unauthorized_api_calls(Check):
         report.region = cloudwatch_client.region
         report.resource_id = cloudtrail_client.audited_account
         report.resource_arn = cloudtrail_client.audited_account_arn
-        # 1. Iterate for CloudWatch Log Group in CloudTrail trails
-        log_groups = []
-        for trail in cloudtrail_client.trails:
-            if trail.log_group_arn:
-                log_groups.append(trail.log_group_arn.split(":")[6])
-        # 2. Describe metric filters for previous log groups
-        for metric_filter in logs_client.metric_filters:
-            if metric_filter.log_group in log_groups:
-                if re.search(pattern, metric_filter.pattern, flags=re.DOTALL):
-                    report.resource_id = metric_filter.log_group
-                    report.resource_arn = metric_filter.arn
-                    report.region = metric_filter.region
-                    report.status = "FAIL"
-                    report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
-                    # 3. Check if there is an alarm for the metric
-                    for alarm in cloudwatch_client.metric_alarms:
-                        if alarm.metric == metric_filter.metric:
-                            report.status = "PASS"
-                            report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
-                            break
+        report = check_cloudwatch_log_metric_filter(
+            pattern,
+            cloudtrail_client.trails,
+            logs_client.metric_filters,
+            cloudwatch_client.metric_alarms,
+            report,
+        )
 
         findings.append(report)
         return findings

prowler/providers/aws/services/cloudwatch/lib/metric_filters.py

@@ -0,0 +1,34 @@
+import re
+
+from prowler.lib.check.models import Check_Report_AWS
+
+
+def check_cloudwatch_log_metric_filter(
+    metric_filter_pattern: str,
+    trails: list,
+    metric_filters: list,
+    metric_alarms: list,
+    report: Check_Report_AWS,
+):
+    # 1. Iterate for CloudWatch Log Group in CloudTrail trails
+    log_groups = []
+    for trail in trails:
+        if trail.log_group_arn:
+            log_groups.append(trail.log_group_arn.split(":")[6])
+    # 2. Describe metric filters for previous log groups
+    for metric_filter in metric_filters:
+        if metric_filter.log_group in log_groups:
+            if re.search(metric_filter_pattern, metric_filter.pattern, flags=re.DOTALL):
+                report.resource_id = metric_filter.log_group
+                report.resource_arn = metric_filter.arn
+                report.region = metric_filter.region
+                report.status = "FAIL"
+                report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} but no alarms associated."
+                # 3. Check if there is an alarm for the metric
+                for alarm in metric_alarms:
+                    if alarm.metric == metric_filter.metric:
+                        report.status = "PASS"
+                        report.status_extended = f"CloudWatch log group {metric_filter.log_group} found with metric filter {metric_filter.name} and alarms set."
+                        break
+
+    return report

prowler/providers/aws/services/codeartifact/codeartifact_packages_external_public_publishing_disabled/codeartifact_packages_external_public_publishing_disabled.py

@@ -16,7 +16,7 @@ class codeartifact_packages_external_public_publishing_disabled(Check):
                 report = Check_Report_AWS(self.metadata())
                 report.region = repository.region
                 report.resource_id = package.name
-                report.resource_arn = repository.arn
+                report.resource_arn = f"{repository.arn}/{package.namespace + ':' if package.namespace else ''}{package.name}"
                 report.resource_tags = repository.tags
 
                 if package.latest_version.origin.origin_type in (

prowler/providers/aws/services/codeartifact/codeartifact_service.py

@@ -63,7 +63,7 @@ class CodeArtifact(AWSService):
                     list_packages_parameters = {
                         "domain": self.repositories[repository].domain_name,
                         "domainOwner": self.repositories[repository].domain_owner,
-                        "repository": repository,
+                        "repository": self.repositories[repository].name,
                     }
                     packages = []
                     for page in list_packages_paginator.paginate(
@@ -83,18 +83,37 @@ class CodeArtifact(AWSService):
                                 ]
                             )
                             # Get Latest Package Version
-                            latest_version_information = (
-                                regional_client.list_package_versions(
-                                    domain=self.repositories[repository].domain_name,
-                                    domainOwner=self.repositories[
-                                        repository
-                                    ].domain_owner,
-                                    repository=repository,
-                                    format=package_format,
-                                    package=package_name,
-                                    sortBy="PUBLISHED_TIME",
+                            if package_namespace:
+                                latest_version_information = (
+                                    regional_client.list_package_versions(
+                                        domain=self.repositories[
+                                            repository
+                                        ].domain_name,
+                                        domainOwner=self.repositories[
+                                            repository
+                                        ].domain_owner,
+                                        repository=self.repositories[repository].name,
+                                        format=package_format,
+                                        namespace=package_namespace,
+                                        package=package_name,
+                                        sortBy="PUBLISHED_TIME",
+                                    )
+                                )
+                            else:
+                                latest_version_information = (
+                                    regional_client.list_package_versions(
+                                        domain=self.repositories[
+                                            repository
+                                        ].domain_name,
+                                        domainOwner=self.repositories[
+                                            repository
+                                        ].domain_owner,
+                                        repository=self.repositories[repository].name,
+                                        format=package_format,
+                                        package=package_name,
+                                        sortBy="PUBLISHED_TIME",
+                                    )
                                 )
-                            )
                             latest_version = ""
                             latest_origin_type = "UNKNOWN"
                             latest_status = "Published"

prowler/providers/aws/services/cognito/cognito_idp_client.py

@@ -0,0 +1,4 @@
+from prowler.providers.aws.lib.audit_info.audit_info import current_audit_info
+from prowler.providers.aws.services.cognito.cognito_service import CognitoIDP
+
+cognito_idp_client = CognitoIDP(current_audit_info)

prowler/providers/aws/services/cognito/cognito_service.py

@@ -0,0 +1,122 @@
+from datetime import datetime
+from typing import Optional
+
+from pydantic import BaseModel
+
+from prowler.lib.logger import logger
+from prowler.lib.scan_filters.scan_filters import is_resource_filtered
+from prowler.providers.aws.lib.service.service import AWSService
+
+
+################## CognitoIDP
+class CognitoIDP(AWSService):
+    def __init__(self, audit_info):
+        super().__init__("cognito-idp", audit_info)
+        self.user_pools = {}
+        self.__threading_call__(self.__list_user_pools__)
+        self.__describe_user_pools__()
+        self.__get_user_pool_mfa_config__()
+
+    def __list_user_pools__(self, regional_client):
+        logger.info("Cognito - Listing User Pools...")
+        try:
+            user_pools_paginator = regional_client.get_paginator("list_user_pools")
+            for page in user_pools_paginator.paginate(MaxResults=60):
+                for user_pool in page["UserPools"]:
+                    arn = f"arn:{self.audited_partition}:cognito-idp:{regional_client.region}:{self.audited_account}:userpool/{user_pool['Id']}"
+                    if not self.audit_resources or (
+                        is_resource_filtered(arn, self.audit_resources)
+                    ):
+                        try:
+                            self.user_pools[arn] = UserPool(
+                                id=user_pool["Id"],
+                                arn=arn,
+                                name=user_pool["Name"],
+                                region=regional_client.region,
+                                last_modified=user_pool["LastModifiedDate"],
+                                creation_date=user_pool["CreationDate"],
+                                status=user_pool.get("Status", "Disabled"),
+                            )
+                        except Exception as error:
+                            logger.error(
+                                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                            )
+        except Exception as error:
+            logger.error(
+                f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
+
+    def __describe_user_pools__(self):
+        logger.info("Cognito - Describing User Pools...")
+        try:
+            for user_pool in self.user_pools.values():
+                try:
+                    user_pool_details = self.regional_clients[
+                        user_pool.region
+                    ].describe_user_pool(UserPoolId=user_pool.id)["UserPool"]
+                    user_pool.password_policy = user_pool_details.get(
+                        "Policies", {}
+                    ).get("PasswordPolicy", {})
+                    user_pool.deletion_protection = user_pool_details.get(
+                        "DeletionProtection", "INACTIVE"
+                    )
+                    user_pool.advanced_security_mode = user_pool_details.get(
+                        "UserPoolAddOns", {}
+                    ).get("AdvancedSecurityMode", "OFF")
+                    user_pool.tags = [user_pool_details.get("UserPoolTags", "")]
+                except Exception as error:
+                    logger.error(
+                        f"{user_pool.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                    )
+        except Exception as error:
+            logger.error(
+                f"{user_pool.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
+
+    def __get_user_pool_mfa_config__(self):
+        logger.info("Cognito - Getting User Pool MFA Configuration...")
+        try:
+            for user_pool in self.user_pools.values():
+                try:
+                    mfa_config = self.regional_clients[
+                        user_pool.region
+                    ].get_user_pool_mfa_config(UserPoolId=user_pool.id)
+                    if mfa_config["MfaConfiguration"] != "OFF":
+                        user_pool.mfa_config = MFAConfig(
+                            sms_authentication=mfa_config.get(
+                                "SmsMfaConfiguration", {}
+                            ),
+                            software_token_mfa_authentication=mfa_config.get(
+                                "SoftwareTokenMfaConfiguration", {}
+                            ),
+                            status=mfa_config["MfaConfiguration"],
+                        )
+                except Exception as error:
+                    logger.error(
+                        f"{user_pool.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                    )
+        except Exception as error:
+            logger.error(
+                f"{user_pool.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
+
+
+class MFAConfig(BaseModel):
+    sms_authentication: Optional[dict]
+    software_token_mfa_authentication: Optional[dict]
+    status: str
+
+
+class UserPool(BaseModel):
+    id: str
+    arn: str
+    name: str
+    region: str
+    advanced_security_mode: str = "OFF"
+    deletion_protection: str = "INACTIVE"
+    last_modified: datetime
+    creation_date: datetime
+    status: str
+    password_policy: Optional[dict]
+    mfa_config: Optional[MFAConfig]
+    tags: Optional[list] = []

prowler/providers/aws/services/ec2/ec2_service.py

@@ -17,7 +17,7 @@ class EC2(AWSService):
         super().__init__(__class__.__name__, audit_info)
         self.instances = []
         self.__threading_call__(self.__describe_instances__)
-        self.__get_instance_user_data__()
+        self.__threading_call__(self.__get_instance_user_data__, self.instances)
         self.security_groups = []
         self.regions_with_sgs = []
         self.__threading_call__(self.__describe_security_groups__)
@@ -27,7 +27,7 @@ class EC2(AWSService):
         self.volumes_with_snapshots = {}
         self.regions_with_snapshots = {}
         self.__threading_call__(self.__describe_snapshots__)
-        self.__get_snapshot_public__()
+        self.__threading_call__(self.__determine_public_snapshots__, self.snapshots)
         self.network_interfaces = []
         self.__threading_call__(self.__describe_public_network_interfaces__)
         self.__threading_call__(self.__describe_sg_network_interfaces__)
@@ -36,12 +36,11 @@ class EC2(AWSService):
         self.volumes = []
         self.__threading_call__(self.__describe_volumes__)
         self.ebs_encryption_by_default = []
-        self.__threading_call__(self.__get_ebs_encryption_by_default__)
+        self.__threading_call__(self.__get_ebs_encryption_settings__)
         self.elastic_ips = []
-        self.__threading_call__(self.__describe_addresses__)
+        self.__threading_call__(self.__describe_ec2_addresses__)
 
     def __describe_instances__(self, regional_client):
-        logger.info("EC2 - Describing EC2 Instances...")
         try:
             describe_instances_paginator = regional_client.get_paginator(
                 "describe_instances"
@@ -106,7 +105,6 @@ class EC2(AWSService):
             )
 
     def __describe_security_groups__(self, regional_client):
-        logger.info("EC2 - Describing Security Groups...")
         try:
             describe_security_groups_paginator = regional_client.get_paginator(
                 "describe_security_groups"
@@ -155,7 +153,6 @@ class EC2(AWSService):
             )
 
     def __describe_network_acls__(self, regional_client):
-        logger.info("EC2 - Describing Network ACLs...")
         try:
             describe_network_acls_paginator = regional_client.get_paginator(
                 "describe_network_acls"
@@ -186,7 +183,6 @@ class EC2(AWSService):
             )
 
     def __describe_snapshots__(self, regional_client):
-        logger.info("EC2 - Describing Snapshots...")
         try:
             snapshots_in_region = False
             describe_snapshots_paginator = regional_client.get_paginator(
@@ -219,35 +215,30 @@ class EC2(AWSService):
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
 
-    def __get_snapshot_public__(self):
-        logger.info("EC2 - Getting snapshot volume attribute permissions...")
-        for snapshot in self.snapshots:
-            try:
-                regional_client = self.regional_clients[snapshot.region]
-                snapshot_public = regional_client.describe_snapshot_attribute(
-                    Attribute="createVolumePermission", SnapshotId=snapshot.id
-                )
-                for permission in snapshot_public["CreateVolumePermissions"]:
-                    if "Group" in permission:
-                        if permission["Group"] == "all":
-                            snapshot.public = True
+    def __determine_public_snapshots__(self, snapshot):
+        try:
+            regional_client = self.regional_clients[snapshot.region]
+            snapshot_public = regional_client.describe_snapshot_attribute(
+                Attribute="createVolumePermission", SnapshotId=snapshot.id
+            )
+            for permission in snapshot_public["CreateVolumePermissions"]:
+                if "Group" in permission:
+                    if permission["Group"] == "all":
+                        snapshot.public = True
 
-            except ClientError as error:
-                if error.response["Error"]["Code"] == "InvalidSnapshot.NotFound":
-                    logger.warning(
-                        f"{snapshot.region} --"
-                        f" {error.__class__.__name__}[{error.__traceback__.tb_lineno}]:"
-                        f" {error}"
-                    )
-                    continue
-
-            except Exception as error:
-                logger.error(
-                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "InvalidSnapshot.NotFound":
+                logger.warning(
+                    f"{snapshot.region} --"
+                    f" {error.__class__.__name__}[{error.__traceback__.tb_lineno}]:"
+                    f" {error}"
                 )
+        except Exception as error:
+            logger.error(
+                f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
 
     def __describe_public_network_interfaces__(self, regional_client):
-        logger.info("EC2 - Describing Network Interfaces...")
         try:
             # Get Network Interfaces with Public IPs
             describe_network_interfaces_paginator = regional_client.get_paginator(
@@ -274,7 +265,6 @@ class EC2(AWSService):
             )
 
     def __describe_sg_network_interfaces__(self, regional_client):
-        logger.info("EC2 - Describing Network Interfaces...")
         try:
             # Get Network Interfaces for Security Groups
             for sg in self.security_groups:
@@ -299,30 +289,25 @@ class EC2(AWSService):
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
 
-    def __get_instance_user_data__(self):
-        logger.info("EC2 - Getting instance user data...")
-        for instance in self.instances:
-            try:
-                regional_client = self.regional_clients[instance.region]
-                user_data = regional_client.describe_instance_attribute(
-                    Attribute="userData", InstanceId=instance.id
-                )["UserData"]
-                if "Value" in user_data:
-                    instance.user_data = user_data["Value"]
-
-            except ClientError as error:
-                if error.response["Error"]["Code"] == "InvalidInstanceID.NotFound":
-                    logger.warning(
-                        f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
-                    )
-                    continue
-            except Exception as error:
-                logger.error(
+    def __get_instance_user_data__(self, instance):
+        try:
+            regional_client = self.regional_clients[instance.region]
+            user_data = regional_client.describe_instance_attribute(
+                Attribute="userData", InstanceId=instance.id
+            )["UserData"]
+            if "Value" in user_data:
+                instance.user_data = user_data["Value"]
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "InvalidInstanceID.NotFound":
+                logger.warning(
                     f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
                 )
+        except Exception as error:
+            logger.error(
+                f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )
 
     def __describe_images__(self, regional_client):
-        logger.info("EC2 - Describing Images...")
         try:
             for image in regional_client.describe_images(Owners=["self"])["Images"]:
                 arn = f"arn:{self.audited_partition}:ec2:{regional_client.region}:{self.audited_account}:image/{image['ImageId']}"
@@ -345,7 +330,6 @@ class EC2(AWSService):
             )
 
     def __describe_volumes__(self, regional_client):
-        logger.info("EC2 - Describing Volumes...")
         try:
             describe_volumes_paginator = regional_client.get_paginator(
                 "describe_volumes"
@@ -370,8 +354,7 @@ class EC2(AWSService):
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
 
-    def __describe_addresses__(self, regional_client):
-        logger.info("EC2 - Describing Elastic IPs...")
+    def __describe_ec2_addresses__(self, regional_client):
         try:
             for address in regional_client.describe_addresses()["Addresses"]:
                 public_ip = None
@@ -402,8 +385,7 @@ class EC2(AWSService):
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
 
-    def __get_ebs_encryption_by_default__(self, regional_client):
-        logger.info("EC2 - Get EBS Encryption By Default...")
+    def __get_ebs_encryption_settings__(self, regional_client):
         try:
             volumes_in_region = False
             for volume in self.volumes:

prowler/providers/aws/services/eks/eks_service.py

@@ -4,7 +4,6 @@ from pydantic import BaseModel
 
 from prowler.lib.logger import logger
 from prowler.lib.scan_filters.scan_filters import is_resource_filtered
-from prowler.providers.aws.aws_provider import generate_regional_clients
 from prowler.providers.aws.lib.service.service import AWSService
 
 
@@ -13,7 +12,6 @@ class EKS(AWSService):
     def __init__(self, audit_info):
         # Call AWSService's __init__
         super().__init__(__class__.__name__, audit_info)
-        self.regional_clients = generate_regional_clients(self.service, audit_info)
         self.clusters = []
         self.__threading_call__(self.__list_clusters__)
         self.__describe_cluster__(self.regional_clients)

prowler/providers/aws/services/elasticache/elasticache_service.py

@@ -1,5 +1,6 @@
 from typing import Optional
 
+from botocore.exceptions import ClientError
 from pydantic import BaseModel
 
 from prowler.lib.logger import logger
@@ -73,7 +74,15 @@ class ElastiCache(AWSService):
                     cluster.tags = regional_client.list_tags_for_resource(
                         ResourceName=cluster.arn
                     )["TagList"]
-
+                except ClientError as error:
+                    if error.response["Error"]["Code"] == "CacheClusterNotFound":
+                        logger.warning(
+                            f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                        )
+                    else:
+                        logger.error(
+                            f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                        )
                 except Exception as error:
                     logger.error(
                         f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"

prowler/providers/aws/services/elbv2/elbv2_insecure_ssl_ciphers/elbv2_insecure_ssl_ciphers.py

@@ -33,7 +33,7 @@ class elbv2_insecure_ssl_ciphers(Check):
                     and listener.ssl_policy not in secure_ssl_policies
                 ):
                     report.status = "FAIL"
-                    report.status_extended = f"ELBv2 {lb.name} has listeners with insecure SSL protocols or ciphers."
+                    report.status_extended = f"ELBv2 {lb.name} has listeners with insecure SSL protocols or ciphers ({listener.ssl_policy})."
 
             findings.append(report)
 

prowler/providers/aws/services/fms/fms_policy_compliant/fms_policy_compliant.py

@@ -13,17 +13,24 @@ class fms_policy_compliant(Check):
             report.status = "PASS"
             report.status_extended = "FMS enabled with all compliant accounts."
             non_compliant_policy = False
-            for policy in fms_client.fms_policies:
-                for policy_to_account in policy.compliance_status:
-                    if policy_to_account.status == "NON_COMPLIANT":
-                        report.status = "FAIL"
-                        report.status_extended = f"FMS with non-compliant policy {policy.name} for account {policy_to_account.account_id}."
-                        report.resource_id = policy.id
-                        report.resource_arn = policy.arn
-                        non_compliant_policy = True
+            if fms_client.fms_policies:
+                for policy in fms_client.fms_policies:
+                    for policy_to_account in policy.compliance_status:
+                        if (
+                            policy_to_account.status == "NON_COMPLIANT"
+                            or not policy_to_account.status
+                        ):
+                            report.status = "FAIL"
+                            report.status_extended = f"FMS with non-compliant policy {policy.name} for account {policy_to_account.account_id}."
+                            report.resource_id = policy.id
+                            report.resource_arn = policy.arn
+                            non_compliant_policy = True
+                            break
+                    if non_compliant_policy:
                         break
-                if non_compliant_policy:
-                    break
+            else:
+                report.status = "FAIL"
+                report.status_extended = f"FMS without any compliant policy for account {fms_client.audited_account}."
 
             findings.append(report)
         return findings

prowler/providers/aws/services/fms/fms_service.py

@@ -5,8 +5,6 @@ from prowler.lib.logger import logger
 from prowler.lib.scan_filters.scan_filters import is_resource_filtered
 from prowler.providers.aws.lib.service.service import AWSService
 
-# from prowler.providers.aws.aws_provider import generate_regional_clients
-
 
 ################## FMS
 class FMS(AWSService):
@@ -68,14 +66,19 @@ class FMS(AWSService):
                 for page in list_compliance_status_paginator.paginate(
                     PolicyId=fms_policy.id
                 ):
-                    for fms_compliance_status in page["PolicyComplianceStatusList"]:
+                    for fms_compliance_status in page.get(
+                        "PolicyComplianceStatusList", []
+                    ):
+                        compliance_status = ""
+                        if fms_compliance_status.get("EvaluationResults"):
+                            compliance_status = fms_compliance_status.get(
+                                "EvaluationResults"
+                            )[0].get("ComplianceStatus", "")
                         fms_policy.compliance_status.append(
                             PolicyAccountComplianceStatus(
                                 account_id=fms_compliance_status.get("MemberAccount"),
                                 policy_id=fms_compliance_status.get("PolicyId"),
-                                status=fms_compliance_status.get("EvaluationResults")[
-                                    0
-                                ].get("ComplianceStatus"),
+                                status=compliance_status,
                             )
                         )
 

prowler/providers/aws/services/iam/iam_role_cross_service_confused_deputy_prevention/iam_role_cross_service_confused_deputy_prevention.py

@@ -1,6 +1,6 @@
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.lib.policy_condition_parser.policy_condition_parser import (
-    is_account_only_allowed_in_condition,
+    is_condition_block_restrictive,
 )
 from prowler.providers.aws.services.iam.iam_client import iam_client
 
@@ -30,7 +30,7 @@ class iam_role_cross_service_confused_deputy_prevention(Check):
                         and "Service" in statement["Principal"]
                         # Check to see if the appropriate condition statements have been implemented
                         and "Condition" in statement
-                        and is_account_only_allowed_in_condition(
+                        and is_condition_block_restrictive(
                             statement["Condition"], iam_client.audited_account
                         )
                     ):

prowler/providers/aws/services/iam/iam_service.py

@@ -494,11 +494,30 @@ class IAM(AWSService):
                                     document=inline_group_policy_doc,
                                 )
                             )
+                        except ClientError as error:
+                            if error.response["Error"]["Code"] == "NoSuchEntity":
+                                logger.warning(
+                                    f"{self.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                                )
+                            else:
+                                logger.error(
+                                    f"{self.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                                )
+
                         except Exception as error:
                             logger.error(
                                 f"{self.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
                             )
                 group.inline_policies = inline_group_policies
+            except ClientError as error:
+                if error.response["Error"]["Code"] == "NoSuchEntity":
+                    logger.warning(
+                        f"{self.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                    )
+                else:
+                    logger.error(
+                        f"{self.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                    )
 
             except Exception as error:
                 logger.error(

prowler/providers/aws/services/organizations/organizations_scp_check_deny_regions/organizations_scp_check_deny_regions.py

@@ -48,11 +48,12 @@ class organizations_scp_check_deny_regions(Check):
                                 and "aws:RequestedRegion"
                                 in statement["Condition"]["StringNotEquals"]
                             ):
-                                if (
-                                    organizations_enabled_regions
-                                    == statement["Condition"]["StringNotEquals"][
+                                if all(
+                                    region
+                                    in statement["Condition"]["StringNotEquals"][
                                         "aws:RequestedRegion"
                                     ]
+                                    for region in organizations_enabled_regions
                                 ):
                                     # All defined regions are restricted, we exit here, no need to continue.
                                     report.status = "PASS"
@@ -73,11 +74,12 @@ class organizations_scp_check_deny_regions(Check):
                                 and "aws:RequestedRegion"
                                 in statement["Condition"]["StringEquals"]
                             ):
-                                if (
-                                    organizations_enabled_regions
-                                    == statement["Condition"]["StringEquals"][
+                                if all(
+                                    region
+                                    in statement["Condition"]["StringEquals"][
                                         "aws:RequestedRegion"
                                     ]
+                                    for region in organizations_enabled_regions
                                 ):
                                     # All defined regions are restricted, we exit here, no need to continue.
                                     report.status = "PASS"

prowler/providers/aws/services/rds/rds_service.py

@@ -232,7 +232,15 @@ class RDS(AWSService):
                     for att in response["DBClusterSnapshotAttributes"]:
                         if "all" in att["AttributeValues"]:
                             snapshot.public = True
-
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "DBClusterSnapshotNotFoundFault":
+                logger.warning(
+                    f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+            else:
+                logger.error(
+                    f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
         except Exception as error:
             logger.error(
                 f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"

prowler/providers/aws/services/s3/s3_service.py

@@ -28,6 +28,7 @@ class S3(AWSService):
         self.__threading_call__(self.__get_bucket_tagging__)
 
     # In the S3 service we override the "__threading_call__" method because we spawn a process per bucket instead of per region
+    # TODO: Replace the above function with the service __threading_call__ using the buckets as the iterator
     def __threading_call__(self, call):
         threads = []
         for bucket in self.buckets:
@@ -101,6 +102,15 @@ class S3(AWSService):
             if "MFADelete" in bucket_versioning:
                 if "Enabled" == bucket_versioning["MFADelete"]:
                     bucket.mfa_delete = True
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "NoSuchBucket":
+                logger.warning(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+            else:
+                logger.error(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
         except Exception as error:
             if bucket.region:
                 logger.error(
@@ -153,6 +163,15 @@ class S3(AWSService):
                 bucket.logging_target_bucket = bucket_logging["LoggingEnabled"][
                     "TargetBucket"
                 ]
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "NoSuchBucket":
+                logger.warning(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+            else:
+                logger.error(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
         except Exception as error:
             if regional_client:
                 logger.error(
@@ -224,6 +243,15 @@ class S3(AWSService):
                     grantee.permission = grant["Permission"]
                 grantees.append(grantee)
             bucket.acl_grantees = grantees
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "NoSuchBucket":
+                logger.warning(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+            else:
+                logger.error(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
         except Exception as error:
             if regional_client:
                 logger.error(
@@ -241,18 +269,26 @@ class S3(AWSService):
             bucket.policy = json.loads(
                 regional_client.get_bucket_policy(Bucket=bucket.name)["Policy"]
             )
-        except Exception as error:
-            if "NoSuchBucketPolicy" in str(error):
+        except ClientError as error:
+            if error.response["Error"]["Code"] == "NoSuchBucketPolicy":
                 bucket.policy = {}
+            elif error.response["Error"]["Code"] == "NoSuchBucket":
+                logger.warning(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
             else:
-                if regional_client:
-                    logger.error(
-                        f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
-                    )
-                else:
-                    logger.error(
-                        f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
-                    )
+                logger.error(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+        except Exception as error:
+            if regional_client:
+                logger.error(
+                    f"{regional_client.region} -- {error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
+            else:
+                logger.error(
+                    f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+                )
 
     def __get_bucket_ownership_controls__(self, bucket):
         logger.info("S3 - Get buckets ownership controls...")

prowler/providers/aws/services/sns/sns_topics_not_publicly_accessible/sns_topics_not_publicly_accessible.py

@@ -1,6 +1,6 @@
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.lib.policy_condition_parser.policy_condition_parser import (
-    is_account_only_allowed_in_condition,
+    is_condition_block_restrictive,
 )
 from prowler.providers.aws.services.sns.sns_client import sns_client
 
@@ -35,7 +35,7 @@ class sns_topics_not_publicly_accessible(Check):
                         ):
                             if (
                                 "Condition" in statement
-                                and is_account_only_allowed_in_condition(
+                                and is_condition_block_restrictive(
                                     statement["Condition"], sns_client.audited_account
                                 )
                             ):

prowler/providers/aws/services/sqs/sqs_queues_not_publicly_accessible/sqs_queues_not_publicly_accessible.py

@@ -1,6 +1,6 @@
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.lib.policy_condition_parser.policy_condition_parser import (
-    is_account_only_allowed_in_condition,
+    is_condition_block_restrictive,
 )
 from prowler.providers.aws.services.sqs.sqs_client import sqs_client
 
@@ -32,8 +32,10 @@ class sqs_queues_not_publicly_accessible(Check):
                             )
                         ):
                             if "Condition" in statement:
-                                if is_account_only_allowed_in_condition(
-                                    statement["Condition"], sqs_client.audited_account
+                                if is_condition_block_restrictive(
+                                    statement["Condition"],
+                                    sqs_client.audited_account,
+                                    True,
                                 ):
                                     report.status_extended = f"SQS queue {queue.id} is not public because its policy only allows access from the same account."
                                 else:

prowler/providers/aws/services/trustedadvisor/trustedadvisor_premium_support_plan_subscribed/trustedadvisor_premium_support_plan_subscribed.metadata.json

@@ -7,7 +7,7 @@
   "SubServiceName": "",
   "ResourceIdTemplate": "arn:aws:iam::AWS_ACCOUNT_NUMBER:root",
   "Severity": "low",
-  "ResourceType": "",
+  "ResourceType": "Other",
   "Description": "Check if a Premium support plan is subscribed.",
   "Risk": "Ensure that the appropriate support level is enabled for the necessary AWS accounts. For example, if an AWS account is being used to host production systems and environments, it is highly recommended that the minimum AWS Support Plan should be Business.",
   "RelatedUrl": "https://aws.amazon.com/premiumsupport/plans/",

prowler/providers/aws/services/trustedadvisor/trustedadvisor_service.py

@@ -34,9 +34,9 @@ class TrustedAdvisor(AWSService):
     def __describe_trusted_advisor_checks__(self):
         logger.info("TrustedAdvisor - Describing Checks...")
         try:
-            for check in self.client.describe_trusted_advisor_checks(language="en")[
-                "checks"
-            ]:
+            for check in self.client.describe_trusted_advisor_checks(language="en").get(
+                "checks", []
+            ):
                 self.checks.append(
                     Check(
                         id=check["id"],

prowler/providers/aws/services/vpc/vpc_different_regions/vpc_different_regions.py

@@ -5,22 +5,23 @@ from prowler.providers.aws.services.vpc.vpc_client import vpc_client
 class vpc_different_regions(Check):
     def execute(self):
         findings = []
-        vpc_regions = set()
-        for vpc in vpc_client.vpcs.values():
-            if not vpc.default:
-                vpc_regions.add(vpc.region)
+        if len(vpc_client.vpcs) > 0:
+            vpc_regions = set()
+            for vpc in vpc_client.vpcs.values():
+                if not vpc.default:
+                    vpc_regions.add(vpc.region)
 
-        report = Check_Report_AWS(self.metadata())
-        # This is a global check under the vpc service: region, resource_id and tags are not relevant here but we keep them for consistency
-        report.region = vpc_client.region
-        report.resource_id = vpc_client.audited_account
-        report.resource_arn = vpc_client.audited_account_arn
-        report.status = "FAIL"
-        report.status_extended = "VPCs found only in one region."
-        if len(vpc_regions) > 1:
-            report.status = "PASS"
-            report.status_extended = "VPCs found in more than one region."
+            report = Check_Report_AWS(self.metadata())
+            report.region = vpc_client.region
+            report.resource_id = vpc_client.audited_account
+            report.resource_arn = vpc_client.audited_account_arn
 
-        findings.append(report)
+            report.status = "FAIL"
+            report.status_extended = "VPCs found only in one region."
+
+            if len(vpc_regions) > 1:
+                report.status = "PASS"
+                report.status_extended = "VPCs found in more than one region."
+            findings.append(report)
 
         return findings

prowler/providers/aws/services/vpc/vpc_endpoint_connections_trust_boundaries/vpc_endpoint_connections_trust_boundaries.py

@@ -2,7 +2,7 @@ from re import compile
 
 from prowler.lib.check.models import Check, Check_Report_AWS
 from prowler.providers.aws.lib.policy_condition_parser.policy_condition_parser import (
-    is_account_only_allowed_in_condition,
+    is_condition_block_restrictive,
 )
 from prowler.providers.aws.services.vpc.vpc_client import vpc_client
 
@@ -35,7 +35,7 @@ class vpc_endpoint_connections_trust_boundaries(Check):
 
                         if "Condition" in statement:
                             for account_id in trusted_account_ids:
-                                if is_account_only_allowed_in_condition(
+                                if is_condition_block_restrictive(
                                     statement["Condition"], account_id
                                 ):
                                     access_from_trusted_accounts = True
@@ -70,7 +70,7 @@ class vpc_endpoint_connections_trust_boundaries(Check):
                                 access_from_trusted_accounts = False
                                 if "Condition" in statement:
                                     for account_id in trusted_account_ids:
-                                        if is_account_only_allowed_in_condition(
+                                        if is_condition_block_restrictive(
                                             statement["Condition"], account_id
                                         ):
                                             access_from_trusted_accounts = True
@@ -102,7 +102,7 @@ class vpc_endpoint_connections_trust_boundaries(Check):
 
                                 if "Condition" in statement:
                                     for account_id in trusted_account_ids:
-                                        if is_account_only_allowed_in_condition(
+                                        if is_condition_block_restrictive(
                                             statement["Condition"], account_id
                                         ):
                                             access_from_trusted_accounts = True

prowler/providers/azure/lib/exception/exception.py

@@ -0,0 +1,11 @@
+class AzureException(Exception):
+    """
+    Exception raised when dealing with Azure Provider/Azure audit info instance
+
+    Attributes:
+        message -- message to be displayed
+    """
+
+    def __init__(self, message):
+        self.message = message
+        super().__init__(self.message)

prowler/providers/common/audit_info.py

@@ -8,6 +8,7 @@ from prowler.lib.logger import logger
 from prowler.providers.aws.aws_provider import (
     AWS_Provider,
     assume_role,
+    get_aws_enabled_regions,
     get_checks_from_input_arn,
     get_regions_from_audit_resources,
 )
@@ -30,6 +31,7 @@ from prowler.providers.azure.lib.audit_info.models import (
     Azure_Audit_Info,
     Azure_Region_Config,
 )
+from prowler.providers.azure.lib.exception.exception import AzureException
 from prowler.providers.gcp.gcp_provider import GCP_Provider
 from prowler.providers.gcp.lib.audit_info.audit_info import gcp_audit_info
 from prowler.providers.gcp.lib.audit_info.models import GCP_Audit_Info
@@ -61,7 +63,7 @@ GCP Account: {Fore.YELLOW}[{profile}]{Style.RESET_ALL}  GCP Project IDs: {Fore.Y
     def print_azure_credentials(self, audit_info: Azure_Audit_Info):
         printed_subscriptions = []
         for key, value in audit_info.identity.subscriptions.items():
-            intermediate = key + " : " + value
+            intermediate = f"{key} : {value}"
             printed_subscriptions.append(intermediate)
         report = f"""
 This report is being generated using the identity below:
@@ -83,6 +85,7 @@ Azure Identity Type: {Fore.YELLOW}[{audit_info.identity.identity_type}]{Style.RE
         current_audit_info.assumed_role_info.role_arn = input_role
         input_session_duration = arguments.get("session_duration")
         input_external_id = arguments.get("external_id")
+        input_role_session_name = arguments.get("role_session_name")
 
         # STS Endpoint Region
         sts_endpoint_region = arguments.get("sts_endpoint_region")
@@ -151,6 +154,9 @@ Azure Identity Type: {Fore.YELLOW}[{audit_info.identity.identity_type}]{Style.RE
             )
             current_audit_info.assumed_role_info.external_id = input_external_id
             current_audit_info.assumed_role_info.mfa_enabled = input_mfa
+            current_audit_info.assumed_role_info.role_session_name = (
+                input_role_session_name
+            )
 
             # Check if role arn is valid
             try:
@@ -256,6 +262,9 @@ Azure Identity Type: {Fore.YELLOW}[{audit_info.identity.identity_type}]{Style.RE
         if arguments.get("resource_arn"):
             current_audit_info.audit_resources = arguments.get("resource_arn")
 
+        # Get Enabled Regions
+        current_audit_info.enabled_regions = get_aws_enabled_regions(current_audit_info)
+
         return current_audit_info
 
     def set_aws_execution_parameters(self, provider, audit_info) -> list[str]:
@@ -295,11 +304,11 @@ Azure Identity Type: {Fore.YELLOW}[{audit_info.identity.identity_type}]{Style.RE
             and not browser_auth
             and not managed_entity_auth
         ):
-            raise Exception(
+            raise AzureException(
                 "Azure provider requires at least one authentication method set: [--az-cli-auth | --sp-env-auth | --browser-auth | --managed-identity-auth]"
             )
         if (not browser_auth and tenant_id) or (browser_auth and not tenant_id):
-            raise Exception(
+            raise AzureException(
                 "Azure Tenant ID (--tenant-id) is required only for browser authentication mode"
             )
 

prowler/providers/common/clean.py

@@ -1,32 +0,0 @@
-import importlib
-import sys
-from shutil import rmtree
-
-from prowler.config.config import default_output_directory
-from prowler.lib.logger import logger
-
-
-def clean_provider_local_output_directories(args):
-    """
-    clean_provider_local_output_directories cleans deletes local custom dirs when output is sent to remote provider storage
-    """
-    try:
-        # import provider cleaning function
-        provider_clean_function = f"clean_{args.provider}_local_output_directories"
-        getattr(importlib.import_module(__name__), provider_clean_function)(args)
-    except AttributeError as attribute_exception:
-        logger.info(
-            f"Cleaning local output directories not initialized for provider {args.provider}: {attribute_exception}"
-        )
-    except Exception as error:
-        logger.critical(
-            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
-        )
-        sys.exit(1)
-
-
-def clean_aws_local_output_directories(args):
-    """clean_aws_provider_local_output_directories deletes local custom dirs when output is sent to remote provider storage for aws provider"""
-    if args.output_bucket or args.output_bucket_no_assume:
-        if args.output_directory != default_output_directory:
-            rmtree(args.output_directory)

prowler/providers/common/outputs.py

@@ -29,6 +29,21 @@ def set_provider_output_options(
         return provider_output_options
 
 
+def get_provider_output_model(audit_info_class_name):
+    """
+    get_provider_output_model returns the model _Check_Output_CSV for each provider
+    """
+    # from AWS_Audit_Info -> AWS -> aws -> Aws
+    output_provider = audit_info_class_name.split("_", 1)[0].lower().capitalize()
+    output_provider_model_name = f"{output_provider}_Check_Output_CSV"
+    output_provider_models_path = "prowler.lib.outputs.models"
+    output_provider_model = getattr(
+        importlib.import_module(output_provider_models_path), output_provider_model_name
+    )
+
+    return output_provider_model
+
+
 @dataclass
 class Provider_Output_Options:
     is_quiet: bool
@@ -54,7 +69,8 @@ class Provider_Output_Options:
         if arguments.output_directory:
             if not isdir(arguments.output_directory):
                 if arguments.output_modes:
-                    makedirs(arguments.output_directory)
+                    # exist_ok is set to True not to raise FileExistsError
+                    makedirs(arguments.output_directory, exist_ok=True)
 
 
 class Azure_Output_Options(Provider_Output_Options):
@@ -119,6 +135,7 @@ class Aws_Output_Options(Provider_Output_Options):
 
         # Security Hub Outputs
         self.security_hub_enabled = arguments.security_hub
+        self.send_sh_only_fails = arguments.send_sh_only_fails
         if arguments.security_hub:
             if not self.output_modes:
                 self.output_modes = ["json-asff"]

prowler/providers/gcp/gcp_provider.py

@@ -1,12 +1,11 @@
 import os
 import sys
 
+from colorama import Fore, Style
 from google import auth
 from googleapiclient import discovery
-from googleapiclient.discovery import Resource
 
 from prowler.lib.logger import logger
-from prowler.providers.gcp.lib.audit_info.models import GCP_Audit_Info
 
 
 class GCP_Provider:
@@ -91,17 +90,7 @@ class GCP_Provider:
             logger.error(
                 f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
             )
+            print(
+                f"\n{Fore.YELLOW}Cloud Resource Manager API {Style.RESET_ALL}has not been used before or it is disabled.\nEnable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/ then retry."
+            )
             return []
-
-
-def generate_client(
-    service: str,
-    api_version: str,
-    audit_info: GCP_Audit_Info,
-) -> Resource:
-    try:
-        return discovery.build(service, api_version, credentials=audit_info.credentials)
-    except Exception as error:
-        logger.error(
-            f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
-        )

prowler/providers/gcp/lib/service/service.py

@@ -3,10 +3,11 @@ import threading
 import google_auth_httplib2
 import httplib2
 from colorama import Fore, Style
+from google.oauth2.credentials import Credentials
 from googleapiclient import discovery
+from googleapiclient.discovery import Resource
 
 from prowler.lib.logger import logger
-from prowler.providers.gcp.gcp_provider import generate_client
 from prowler.providers.gcp.lib.audit_info.models import GCP_Audit_Info
 
 
@@ -25,7 +26,9 @@ class GCPService:
         self.api_version = api_version
         self.default_project_id = audit_info.default_project_id
         self.region = region
-        self.client = generate_client(service, api_version, audit_info)
+        self.client = self.__generate_client__(
+            self.service, api_version, audit_info.credentials
+        )
         # Only project ids that have their API enabled will be scanned
         self.project_ids = self.__is_api_active__(audit_info.project_ids)
 
@@ -59,10 +62,23 @@ class GCPService:
                     project_ids.append(project_id)
                 else:
                     print(
-                        f"\n{Fore.YELLOW}{self.service} API {Style.RESET_ALL}has not been used in project {project_id} before or it is disabled.\nEnable it by visiting https://console.developers.google.com/apis/api/dataproc.googleapis.com/overview?project={project_id} then retry."
+                        f"\n{Fore.YELLOW}{self.service} API {Style.RESET_ALL}has not been used in project {project_id} before or it is disabled.\nEnable it by visiting https://console.developers.google.com/apis/api/{self.service}.googleapis.com/overview?project={project_id} then retry."
                     )
             except Exception as error:
                 logger.error(
                     f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
                 )
         return project_ids
+
+    def __generate_client__(
+        self,
+        service: str,
+        api_version: str,
+        credentials: Credentials,
+    ) -> Resource:
+        try:
+            return discovery.build(service, api_version, credentials=credentials)
+        except Exception as error:
+            logger.error(
+                f"{error.__class__.__name__}[{error.__traceback__.tb_lineno}]: {error}"
+            )

pyproject.toml

@@ -22,7 +22,7 @@ packages = [
   {include = "prowler"}
 ]
 readme = "README.md"
-version = "3.11.3"
+version = "3.12.1"
 
 [tool.poetry.dependencies]
 alive-progress = "3.1.5"
@@ -38,37 +38,39 @@ boto3 = "1.26.165"
 botocore = "1.29.165"
 colorama = "0.4.6"
 detect-secrets = "1.4.0"
-google-api-python-client = "2.107.0"
-google-auth-httplib2 = "^0.1.0"
+google-api-python-client = "2.113.0"
+google-auth-httplib2 = ">=0.1,<0.3"
+jsonschema = "4.20.0"
 mkdocs = {version = "1.5.3", optional = true}
-mkdocs-material = {version = "9.4.8", optional = true}
+mkdocs-material = {version = "9.5.3", optional = true}
 msgraph-core = "0.2.2"
 msrestazure = "^0.6.4"
 pydantic = "1.10.13"
-python = "^3.9"
+python = ">=3.9,<3.12"
 schema = "0.7.5"
-shodan = "1.30.1"
-slack-sdk = "3.23.0"
+shodan = "1.31.0"
+slack-sdk = "3.26.1"
 tabulate = "0.9.0"
 
 [tool.poetry.extras]
 docs = ["mkdocs", "mkdocs-material"]
 
 [tool.poetry.group.dev.dependencies]
-bandit = "1.7.5"
+bandit = "1.7.6"
 black = "22.12.0"
-coverage = "7.3.2"
-docker = "6.1.3"
-flake8 = "6.1.0"
-freezegun = "1.2.2"
+coverage = "7.4.0"
+docker = "7.0.0"
+flake8 = "7.0.0"
+freezegun = "1.4.0"
 mock = "5.1.0"
-moto = "4.2.8"
+moto = {extras = ["all"], version = "4.2.13"}
 openapi-spec-validator = "0.7.1"
-pylint = "3.0.2"
-pytest = "7.4.3"
+openapi-schema-validator = "0.6.2"
+pylint = "3.0.3"
+pytest = "7.4.4"
 pytest-cov = "4.1.0"
 pytest-randomly = "3.15.0"
-pytest-xdist = "3.4.0"
+pytest-xdist = "3.5.0"
 safety = "2.3.5"
 vulture = "2.10"
 

tests/config/config_test.py

@@ -54,7 +54,7 @@ config_aws = {
 
 class Test_Config:
     def test_get_aws_available_regions(self):
-        assert len(get_aws_available_regions()) == 32
+        assert len(get_aws_available_regions()) == 33
 
     @mock.patch(
         "prowler.config.config.requests.get", new=mock_prowler_get_latest_release

tests/lib/check/check_loader_test.py

@@ -0,0 +1,319 @@
+from mock import patch
+
+from prowler.lib.check.checks_loader import (
+    load_checks_to_execute,
+    update_checks_to_execute_with_aliases,
+)
+from prowler.lib.check.models import (
+    Check_Metadata_Model,
+    Code,
+    Recommendation,
+    Remediation,
+)
+
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME = "s3_bucket_level_public_access_block"
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_CUSTOM_ALIAS = (
+    "s3_bucket_level_public_access_block"
+)
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY = "medium"
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_SERVICE = "s3"
+
+
+class TestCheckLoader:
+    provider = "aws"
+
+    def get_custom_check_metadata(self):
+        return Check_Metadata_Model(
+            Provider="aws",
+            CheckID=S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME,
+            CheckTitle="Check S3 Bucket Level Public Access Block.",
+            CheckType=["Data Protection"],
+            CheckAliases=[S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_CUSTOM_ALIAS],
+            ServiceName=S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_SERVICE,
+            SubServiceName="",
+            ResourceIdTemplate="arn:partition:s3:::bucket_name",
+            Severity=S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY,
+            ResourceType="AwsS3Bucket",
+            Description="Check S3 Bucket Level Public Access Block.",
+            Risk="Public access policies may be applied to sensitive data buckets.",
+            RelatedUrl="https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-block-public-access.html",
+            Remediation=Remediation(
+                Code=Code(
+                    NativeIaC="",
+                    Terraform="https://docs.bridgecrew.io/docs/bc_aws_s3_20#terraform",
+                    CLI="aws s3api put-public-access-block --region <REGION_NAME> --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true --bucket <BUCKET_NAME>",
+                    Other="https://github.com/cloudmatos/matos/tree/master/remediations/aws/s3/s3/block-public-access",
+                ),
+                Recommendation=Recommendation(
+                    Text="You can enable Public Access Block at the bucket level to prevent the exposure of your data stored in S3.",
+                    Url="https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-block-public-access.html",
+                ),
+            ),
+            Categories=["internet-exposed"],
+            DependsOn=[],
+            RelatedTo=[],
+            Notes="",
+            Compliance=[],
+        )
+
+    def test_load_checks_to_execute(self):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = None
+        service_list = None
+        severities = None
+        compliance_frameworks = None
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.recover_checks_from_provider",
+            return_value=[
+                (
+                    f"{S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME}",
+                    "path/to/{S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME}",
+                )
+            ],
+        ):
+            assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_check_list(self):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME]
+        service_list = None
+        severities = None
+        compliance_frameworks = None
+        categories = None
+
+        assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+            bulk_checks_metatada,
+            bulk_compliance_frameworks,
+            checks_file,
+            check_list,
+            service_list,
+            severities,
+            compliance_frameworks,
+            categories,
+            self.provider,
+        )
+
+    def test_load_checks_to_execute_with_severities(self):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = None
+        severities = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY]
+        compliance_frameworks = None
+        categories = None
+
+        assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+            bulk_checks_metatada,
+            bulk_compliance_frameworks,
+            checks_file,
+            check_list,
+            service_list,
+            severities,
+            compliance_frameworks,
+            categories,
+            self.provider,
+        )
+
+    def test_load_checks_to_execute_with_severities_and_services(self):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_SERVICE]
+        severities = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY]
+        compliance_frameworks = None
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.recover_checks_from_service",
+            return_value={S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME},
+        ):
+            assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_severities_and_services_not_within_severity(
+        self,
+    ):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = ["ec2"]
+        severities = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY]
+        compliance_frameworks = None
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.recover_checks_from_service",
+            return_value={"ec2_ami_public"},
+        ):
+            assert set() == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_checks_file(
+        self,
+    ):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = "path/to/test_file"
+        check_list = []
+        service_list = []
+        severities = []
+        compliance_frameworks = None
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.parse_checks_from_file",
+            return_value={S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME},
+        ):
+            assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_service_list(
+        self,
+    ):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = [S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME_SERVICE]
+        severities = []
+        compliance_frameworks = None
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.recover_checks_from_service",
+            return_value={S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME},
+        ):
+            assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_compliance_frameworks(
+        self,
+    ):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = []
+        severities = []
+        compliance_frameworks = ["test-compliance-framework"]
+        categories = None
+
+        with patch(
+            "prowler.lib.check.checks_loader.parse_checks_from_compliance_framework",
+            return_value={S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME},
+        ):
+            assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+                bulk_checks_metatada,
+                bulk_compliance_frameworks,
+                checks_file,
+                check_list,
+                service_list,
+                severities,
+                compliance_frameworks,
+                categories,
+                self.provider,
+            )
+
+    def test_load_checks_to_execute_with_categories(
+        self,
+    ):
+        bulk_checks_metatada = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata()
+        }
+        bulk_compliance_frameworks = None
+        checks_file = None
+        check_list = []
+        service_list = []
+        severities = []
+        compliance_frameworks = []
+        categories = {"internet-exposed"}
+
+        assert {S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME} == load_checks_to_execute(
+            bulk_checks_metatada,
+            bulk_compliance_frameworks,
+            checks_file,
+            check_list,
+            service_list,
+            severities,
+            compliance_frameworks,
+            categories,
+            self.provider,
+        )
+
+    def test_update_checks_to_execute_with_aliases(self):
+        checks_to_execute = {"renamed_check"}
+        check_aliases = {"renamed_check": "check_name"}
+        assert {"check_name"} == update_checks_to_execute_with_aliases(
+            checks_to_execute, check_aliases
+        )

tests/lib/check/check_test.py

@@ -3,7 +3,7 @@ import pathlib
 from importlib.machinery import FileFinder
 from pkgutil import ModuleInfo
 
-from boto3 import client, session
+from boto3 import client
 from fixtures.bulk_checks_metadata import test_bulk_checks_metadata
 from mock import patch
 from moto import mock_s3
@@ -27,8 +27,7 @@ from prowler.providers.aws.aws_provider import (
     get_checks_from_input_arn,
     get_regions_from_audit_resources,
 )
-from prowler.providers.aws.lib.audit_info.models import AWS_Audit_Info
-from prowler.providers.common.models import Audit_Metadata
+from tests.providers.aws.audit_info_utils import set_mocked_aws_audit_info
 
 AWS_ACCOUNT_NUMBER = "123456789012"
 AWS_REGION = "us-east-1"
@@ -257,37 +256,11 @@ def mock_recover_checks_from_aws_provider_rds_service(*_):
     ]
 
 
-class Test_Check:
-    def set_mocked_audit_info(self):
-        audit_info = AWS_Audit_Info(
-            session_config=None,
-            original_session=None,
-            audit_session=session.Session(
-                profile_name=None,
-                botocore_session=None,
-            ),
-            audited_account=AWS_ACCOUNT_NUMBER,
-            audited_account_arn=f"arn:aws:iam::{AWS_ACCOUNT_NUMBER}:root",
-            audited_user_id=None,
-            audited_partition="aws",
-            audited_identity_arn=None,
-            profile=None,
-            profile_region=None,
-            credentials=None,
-            assumed_role_info=None,
-            audited_regions=None,
-            organizations_metadata=None,
-            audit_resources=None,
-            mfa_enabled=False,
-            audit_metadata=Audit_Metadata(
-                services_scanned=0,
-                expected_checks=[],
-                completed_checks=0,
-                audit_progress=0,
-            ),
-        )
-        return audit_info
+def mock_recover_checks_from_aws_provider_cognito_service(*_):
+    return []
 
+
+class Test_Check:
     def test_load_check_metadata(self):
         test_cases = [
             {
@@ -363,7 +336,7 @@ class Test_Check:
             provider = test["input"]["provider"]
             assert (
                 parse_checks_from_folder(
-                    self.set_mocked_audit_info(), check_folder, provider
+                    set_mocked_aws_audit_info(), check_folder, provider
                 )
                 == test["expected"]
             )
@@ -596,6 +569,19 @@ class Test_Check:
         recovered_checks = get_checks_from_input_arn(audit_resources, provider)
         assert recovered_checks == expected_checks
 
+    @patch(
+        "prowler.lib.check.check.recover_checks_from_provider",
+        new=mock_recover_checks_from_aws_provider_cognito_service,
+    )
+    def test_get_checks_from_input_arn_cognito(self):
+        audit_resources = [
+            f"arn:aws:cognito-idp:us-east-1:{AWS_ACCOUNT_NUMBER}:userpool/test"
+        ]
+        provider = "aws"
+        expected_checks = []
+        recovered_checks = get_checks_from_input_arn(audit_resources, provider)
+        assert recovered_checks == expected_checks
+
     @patch(
         "prowler.lib.check.check.recover_checks_from_provider",
         new=mock_recover_checks_from_aws_provider_ec2_service,

tests/lib/check/custom_checks_metadata_test.py

@@ -0,0 +1,164 @@
+import logging
+import os
+
+import pytest
+
+from prowler.lib.check.custom_checks_metadata import (
+    parse_custom_checks_metadata_file,
+    update_check_metadata,
+    update_checks_metadata,
+)
+from prowler.lib.check.models import (
+    Check_Metadata_Model,
+    Code,
+    Recommendation,
+    Remediation,
+)
+
+CUSTOM_CHECKS_METADATA_FIXTURE_FILE = f"{os.path.dirname(os.path.realpath(__file__))}/fixtures/custom_checks_metadata_example.yaml"
+CUSTOM_CHECKS_METADATA_FIXTURE_FILE_NOT_VALID = f"{os.path.dirname(os.path.realpath(__file__))}/fixtures/custom_checks_metadata_example_not_valid.yaml"
+
+AWS_PROVIDER = "aws"
+AZURE_PROVIDER = "azure"
+GCP_PROVIDER = "gcp"
+
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME = "s3_bucket_level_public_access_block"
+S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY = "medium"
+
+
+class TestCustomChecksMetadata:
+    def get_custom_check_metadata(self):
+        return Check_Metadata_Model(
+            Provider="aws",
+            CheckID=S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME,
+            CheckTitle="Check S3 Bucket Level Public Access Block.",
+            CheckType=["Data Protection"],
+            CheckAliases=[],
+            ServiceName="s3",
+            SubServiceName="",
+            ResourceIdTemplate="arn:partition:s3:::bucket_name",
+            Severity=S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY,
+            ResourceType="AwsS3Bucket",
+            Description="Check S3 Bucket Level Public Access Block.",
+            Risk="Public access policies may be applied to sensitive data buckets.",
+            RelatedUrl="https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-block-public-access.html",
+            Remediation=Remediation(
+                Code=Code(
+                    NativeIaC="",
+                    Terraform="https://docs.bridgecrew.io/docs/bc_aws_s3_20#terraform",
+                    CLI="aws s3api put-public-access-block --region <REGION_NAME> --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true --bucket <BUCKET_NAME>",
+                    Other="https://github.com/cloudmatos/matos/tree/master/remediations/aws/s3/s3/block-public-access",
+                ),
+                Recommendation=Recommendation(
+                    Text="You can enable Public Access Block at the bucket level to prevent the exposure of your data stored in S3.",
+                    Url="https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-block-public-access.html",
+                ),
+            ),
+            Categories=[],
+            DependsOn=[],
+            RelatedTo=[],
+            Notes="",
+            Compliance=[],
+        )
+
+    def test_parse_custom_checks_metadata_file_for_aws(self):
+        assert parse_custom_checks_metadata_file(
+            AWS_PROVIDER, CUSTOM_CHECKS_METADATA_FIXTURE_FILE
+        ) == {
+            "Checks": {
+                "s3_bucket_level_public_access_block": {"Severity": "high"},
+                "s3_bucket_no_mfa_delete": {"Severity": "high"},
+            }
+        }
+
+    def test_parse_custom_checks_metadata_file_for_azure(self):
+        assert parse_custom_checks_metadata_file(
+            AZURE_PROVIDER, CUSTOM_CHECKS_METADATA_FIXTURE_FILE
+        ) == {"Checks": {"sqlserver_auditing_enabled": {"Severity": "high"}}}
+
+    def test_parse_custom_checks_metadata_file_for_gcp(self):
+        assert parse_custom_checks_metadata_file(
+            GCP_PROVIDER, CUSTOM_CHECKS_METADATA_FIXTURE_FILE
+        ) == {"Checks": {"bigquery_dataset_cmk_encryption": {"Severity": "low"}}}
+
+    def test_parse_custom_checks_metadata_file_for_aws_validation_error(self, caplog):
+        caplog.set_level(logging.CRITICAL)
+
+        with pytest.raises(SystemExit) as error:
+            parse_custom_checks_metadata_file(
+                AWS_PROVIDER, CUSTOM_CHECKS_METADATA_FIXTURE_FILE_NOT_VALID
+            )
+        assert error.type == SystemExit
+        assert error.value.code == 1
+        assert "'Checks' is a required property" in caplog.text
+
+    def test_update_checks_metadata(self):
+        updated_severity = "high"
+        bulk_checks_metadata = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata(),
+        }
+        custom_checks_metadata = {
+            "Checks": {
+                S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: {
+                    "Severity": updated_severity
+                },
+            }
+        }
+
+        bulk_checks_metadata_updated = update_checks_metadata(
+            bulk_checks_metadata, custom_checks_metadata
+        ).get(S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME)
+
+        assert bulk_checks_metadata_updated.Severity == updated_severity
+
+    def test_update_checks_metadata_not_present_field(self):
+        bulk_checks_metadata = {
+            S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: self.get_custom_check_metadata(),
+        }
+        custom_checks_metadata = {
+            "Checks": {
+                S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME: {
+                    "RandomField": "random_value"
+                },
+            }
+        }
+
+        bulk_checks_metadata_updated = update_checks_metadata(
+            bulk_checks_metadata, custom_checks_metadata
+        ).get(S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_NAME)
+
+        assert (
+            bulk_checks_metadata_updated.Severity
+            == S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY
+        )
+
+    def test_update_check_metadata(self):
+        updated_severity = "high"
+        custom_checks_metadata = {"Severity": updated_severity}
+
+        check_metadata_updated = update_check_metadata(
+            self.get_custom_check_metadata(), custom_checks_metadata
+        )
+        assert check_metadata_updated.Severity == updated_severity
+
+    def test_update_check_metadata_not_present_field(self):
+        custom_checks_metadata = {"RandomField": "random_value"}
+
+        check_metadata_updated = update_check_metadata(
+            self.get_custom_check_metadata(), custom_checks_metadata
+        )
+        assert (
+            check_metadata_updated.Severity
+            == S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY
+        )
+
+    def test_update_check_metadata_none_custom_metadata(self):
+        custom_checks_metadata = None
+
+        check_metadata_updated = update_check_metadata(
+            self.get_custom_check_metadata(), custom_checks_metadata
+        )
+        assert (
+            check_metadata_updated.Severity
+            == S3_BUCKET_LEVEL_PUBLIC_ACCESS_BLOCK_SEVERITY
+        )

tests/lib/check/fixtures/custom_checks_metadata_example.yaml

@@ -0,0 +1,15 @@
+CustomChecksMetadata:
+  aws:
+    Checks:
+      s3_bucket_level_public_access_block:
+        Severity: high
+      s3_bucket_no_mfa_delete:
+        Severity: high
+  azure:
+    Checks:
+      sqlserver_auditing_enabled:
+        Severity: high
+  gcp:
+    Checks:
+      bigquery_dataset_cmk_encryption:
+        Severity: low

tests/lib/check/fixtures/custom_checks_metadata_example_not_valid.yaml

@@ -0,0 +1,5 @@
+CustomChecksMetadata:
+  aws:
+    Check:
+      s3_bucket_level_public_access_block:
+        Severity: high

tests/lib/cli/parser_test.py

@@ -5,6 +5,11 @@ import pytest
 from mock import patch
 
 from prowler.lib.cli.parser import ProwlerArgumentParser
+from prowler.providers.aws.config import ROLE_SESSION_NAME
+from prowler.providers.aws.lib.arguments.arguments import (
+    validate_bucket,
+    validate_role_session_name,
+)
 from prowler.providers.azure.lib.arguments.arguments import validate_azure_region
 
 prowler_command = "prowler"
@@ -739,7 +744,7 @@ class Test_Parser:
         assert wrapped_exit.value.code == 2
         assert (
             capsys.readouterr().err
-            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/-T options -R option is needed\n"
+            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/--external-id, -T/--session-duration or --role-session-name options -R/--role option is needed\n"
         )
 
     def test_aws_parser_session_duration_long(self, capsys):
@@ -752,7 +757,7 @@ class Test_Parser:
         assert wrapped_exit.value.code == 2
         assert (
             capsys.readouterr().err
-            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/-T options -R option is needed\n"
+            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/--external-id, -T/--session-duration or --role-session-name options -R/--role option is needed\n"
         )
 
     # TODO
@@ -773,7 +778,7 @@ class Test_Parser:
         assert wrapped_exit.value.code == 2
         assert (
             capsys.readouterr().err
-            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/-T options -R option is needed\n"
+            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/--external-id, -T/--session-duration or --role-session-name options -R/--role option is needed\n"
         )
 
     def test_aws_parser_external_id_long(self, capsys):
@@ -786,7 +791,7 @@ class Test_Parser:
         assert wrapped_exit.value.code == 2
         assert (
             capsys.readouterr().err
-            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/-T options -R option is needed\n"
+            == f"{prowler_default_usage_error}\nprowler: error: aws: To use -I/--external-id, -T/--session-duration or --role-session-name options -R/--role option is needed\n"
         )
 
     def test_aws_parser_region_f(self):
@@ -882,6 +887,12 @@ class Test_Parser:
         parsed = self.parser.parse(command)
         assert parsed.skip_sh_update
 
+    def test_aws_parser_send_only_fail(self):
+        argument = "--send-sh-only-fails"
+        command = [prowler_command, argument]
+        parsed = self.parser.parse(command)
+        assert parsed.send_sh_only_fails
+
     def test_aws_parser_quick_inventory_short(self):
         argument = "-i"
         command = [prowler_command, argument]
@@ -1005,6 +1016,13 @@ class Test_Parser:
         parsed = self.parser.parse(command)
         assert parsed.sts_endpoint_region == sts_endpoint_region
 
+    def test_aws_parser_role_session_name(self):
+        argument = "--role-session-name"
+        role_session_name = ROLE_SESSION_NAME
+        command = [prowler_command, argument, role_session_name]
+        parsed = self.parser.parse(command)
+        assert parsed.role_session_name == role_session_name
+
     def test_parser_azure_auth_sp(self):
         argument = "--sp-env-auth"
         command = [prowler_command, "azure", argument]
@@ -1132,3 +1150,50 @@ class Test_Parser:
             match=f"Region {invalid_region} not allowed, allowed regions are {' '.join(expected_regions)}",
         ):
             validate_azure_region(invalid_region)
+
+    def test_validate_bucket_invalid_bucket_names(self):
+        bad_bucket_names = [
+            "xn--bucket-name",
+            "mrryadfpcwlscicvnrchmtmyhwrvzkgfgdxnlnvaaummnywciixnzvycnzmhhpwb",
+            "192.168.5.4",
+            "bucket-name-s3alias",
+            "bucket-name-s3alias-",
+            "bucket-n$ame",
+            "bu",
+        ]
+        for bucket_name in bad_bucket_names:
+            with pytest.raises(ArgumentTypeError) as argument_error:
+                validate_bucket(bucket_name)
+
+            assert argument_error.type == ArgumentTypeError
+            assert (
+                argument_error.value.args[0]
+                == "Bucket name must be valid (https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html)"
+            )
+
+    def test_validate_bucket_valid_bucket_names(self):
+        valid_bucket_names = ["bucket-name" "test" "test-test-test"]
+        for bucket_name in valid_bucket_names:
+            assert validate_bucket(bucket_name) == bucket_name
+
+    def test_validate_role_session_name_invalid_role_names(self):
+        bad_role_names = [
+            "role name",
+            "adasD*",
+            "test#",
+            "role-name?",
+        ]
+        for role_name in bad_role_names:
+            with pytest.raises(ArgumentTypeError) as argument_error:
+                validate_role_session_name(role_name)
+
+            assert argument_error.type == ArgumentTypeError
+            assert (
+                argument_error.value.args[0]
+                == "Role Session Name must be 2-64 characters long and consist only of upper- and lower-case alphanumeric characters with no spaces. You can also include underscores or any of the following characters: =,.@-"
+            )
+
+    def test_validate_role_session_name_valid_role_names(self):
+        valid_role_names = ["prowler-role" "test@" "test=test+test,."]
+        for role_name in valid_role_names:
+            assert validate_role_session_name(role_name) == role_name

tests/providers/aws/audit_info_utils.py

@@ -1,15 +1,43 @@
 from boto3 import session
 
-from prowler.providers.aws.lib.audit_info.models import AWS_Audit_Info
+from prowler.providers.aws.lib.audit_info.models import AWS_Assume_Role, AWS_Audit_Info
 from prowler.providers.common.models import Audit_Metadata
 
-AWS_REGION_US_EAST_1 = "us-east-1"
-AWS_REGION_EU_WEST_1 = "eu-west-1"
-AWS_REGION_EU_WEST_2 = "eu-west-2"
-AWS_PARTITION = "aws"
+# Root AWS Account
 AWS_ACCOUNT_NUMBER = "123456789012"
 AWS_ACCOUNT_ARN = f"arn:aws:iam::{AWS_ACCOUNT_NUMBER}:root"
+
+# Commercial Regions
+AWS_REGION_US_EAST_1 = "us-east-1"
+AWS_REGION_US_EAST_1_AZA = "us-east-1a"
+AWS_REGION_US_EAST_1_AZB = "us-east-1b"
+AWS_REGION_EU_WEST_1 = "eu-west-1"
+AWS_REGION_EU_WEST_1_AZA = "eu-west-1a"
+AWS_REGION_EU_WEST_1_AZB = "eu-west-1b"
+AWS_REGION_EU_WEST_2 = "eu-west-2"
+AWS_REGION_CN_NORTHWEST_1 = "cn-northwest-1"
+AWS_REGION_CN_NORTH_1 = "cn-north-1"
+AWS_REGION_EU_SOUTH_2 = "eu-south-2"
+AWS_REGION_EU_SOUTH_3 = "eu-south-3"
+AWS_REGION_US_WEST_2 = "us-west-2"
+AWS_REGION_US_EAST_2 = "us-east-2"
+AWS_REGION_EU_CENTRAL_1 = "eu-central-1"
+
+
+# China Regions
+AWS_REGION_CHINA_NORHT_1 = "cn-north-1"
+
+# Gov Cloud Regions
+AWS_REGION_GOV_CLOUD_US_EAST_1 = "us-gov-east-1"
+
+# Iso Regions
+AWS_REGION_ISO_GLOBAL = "aws-iso-global"
+
+# AWS Partitions
 AWS_COMMERCIAL_PARTITION = "aws"
+AWS_GOV_CLOUD_PARTITION = "aws-us-gov"
+AWS_CHINA_PARTITION = "aws-cn"
+AWS_ISO_PARTITION = "aws-iso"
 
 
 # Mocked AWS Audit Info
@@ -17,32 +45,44 @@ def set_mocked_aws_audit_info(
     audited_regions: [str] = [],
     audited_account: str = AWS_ACCOUNT_NUMBER,
     audited_account_arn: str = AWS_ACCOUNT_ARN,
+    audited_partition: str = AWS_COMMERCIAL_PARTITION,
+    expected_checks: [str] = [],
+    profile_region: str = None,
+    audit_config: dict = {},
+    ignore_unused_services: bool = False,
+    assumed_role_info: AWS_Assume_Role = None,
+    audit_session: session.Session = session.Session(
+        profile_name=None,
+        botocore_session=None,
+    ),
+    original_session: session.Session = None,
+    enabled_regions: set = None,
 ):
     audit_info = AWS_Audit_Info(
         session_config=None,
-        original_session=None,
-        audit_session=session.Session(
-            profile_name=None,
-            botocore_session=None,
-        ),
+        original_session=original_session,
+        audit_session=audit_session,
         audited_account=audited_account,
         audited_account_arn=audited_account_arn,
         audited_user_id=None,
-        audited_partition=AWS_PARTITION,
+        audited_partition=audited_partition,
         audited_identity_arn=None,
         profile=None,
-        profile_region=None,
+        profile_region=profile_region,
         credentials=None,
-        assumed_role_info=None,
+        assumed_role_info=assumed_role_info,
         audited_regions=audited_regions,
         organizations_metadata=None,
-        audit_resources=None,
+        audit_resources=[],
         mfa_enabled=False,
         audit_metadata=Audit_Metadata(
             services_scanned=0,
-            expected_checks=[],
+            expected_checks=expected_checks,
             completed_checks=0,
             audit_progress=0,
         ),
+        audit_config=audit_config,
+        ignore_unused_services=ignore_unused_services,
+        enabled_regions=enabled_regions if enabled_regions else set(audited_regions),
     )
     return audit_info