chore(merge): Merge master with Prowler 4.0 (#3467)

Co-authored-by: Sergio Garcia <38561120+sergargar@users.noreply.github.com>
This commit is contained in:
Pepe Fagoaga
2024-02-29 11:19:17 +01:00
committed by GitHub
parent b0f2f34d3b
commit f0c027f54e
932 changed files with 38787 additions and 20014 deletions
+5 -2
View File
@@ -196,14 +196,17 @@ aws:
As you can see in the above code, within the service client, in this case the `ec2_client`, there is an object called `audit_config` which is a Python dictionary containing the values read from the configuration file.
In order to use it, you have to check first if the value is present in the configuration file. If the value is not present, you can create it in the `config.yaml` file and then, read it from the check.
> It is mandatory to always use the `dictionary.get(value, default)` syntax to set a default value in the case the configuration value is not present.
???+ note
It is mandatory to always use the `dictionary.get(value, default)` syntax to set a default value in the case the configuration value is not present.
## Check Metadata
Each Prowler check has metadata associated which is stored at the same level of the check's folder in a file called A `check_name.metadata.json` containing the check's metadata.
> We are going to include comments in this example metadata JSON but they cannot be included because the JSON format does not allow comments.
???+ note
We are going to include comments in this example metadata JSON but they cannot be included because the JSON format does not allow comments.
```json
{
+45
View File
@@ -0,0 +1,45 @@
# Debugging
Debugging in Prowler make things easier!
If you are developing Prowler, it's possible that you will encounter some situations where you have to inspect the code in depth to fix some unexpected issues during the execution. To do that, if you are using VSCode you can run the code using the integrated debugger. Please, refer to this [documentation](https://code.visualstudio.com/docs/editor/debugging) for guidance about the debugger in VSCode.
The following file is an example of the [debugging configuration](https://code.visualstudio.com/docs/editor/debugging#_launch-configurations) file that you can add to [Virtual Studio Code](https://code.visualstudio.com/).
This file should inside the *.vscode* folder and its name has to be *launch.json*:
```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: Current File",
"type": "python",
"request": "launch",
"program": "prowler.py",
"args": [
"aws",
"-f",
"eu-west-1",
"--service",
"cloudwatch",
"--log-level",
"ERROR",
"-p",
"dev",
],
"console": "integratedTerminal",
"justMyCode": false
},
{
"name": "Python: Debug Tests",
"type": "python",
"request": "launch",
"program": "${file}",
"purpose": [
"debug-test"
],
"console": "integratedTerminal",
"justMyCode": false
}
]
}
```
+5 -3
View File
@@ -1,6 +1,6 @@
# Developer Guide
You can extend Prowler in many different ways, in most cases you will want to create your own checks and compliance security frameworks, here is where you can learn about how to get started with it. We also include how to create custom outputs, integrations and more.
You can extend Prowler Open Source in many different ways, in most cases you will want to create your own checks and compliance security frameworks, here is where you can learn about how to get started with it. We also include how to create custom outputs, integrations and more.
## Get the code and install all dependencies
@@ -16,7 +16,7 @@ pip install poetry
```
Then install all dependencies including the ones for developers:
```
poetry install
poetry install --with dev
poetry shell
```
@@ -31,7 +31,9 @@ You should get an output like the following:
pre-commit installed at .git/hooks/pre-commit
```
Before we merge any of your pull requests we pass checks to the code, we use the following tools and automation to make sure the code is secure and dependencies up-to-dated (these should have been already installed if you ran `pipenv install -d`):
Before we merge any of your pull requests we pass checks to the code, we use the following tools and automation to make sure the code is secure and dependencies up-to-dated:
???+ note
These should have been already installed if you ran `poetry install --with dev`
- [`bandit`](https://pypi.org/project/bandit/) for code security review.
- [`safety`](https://pypi.org/project/safety/) and [`dependabot`](https://github.com/features/security) for dependencies.
+15 -11
View File
@@ -40,13 +40,15 @@ Other commands to run tests:
- Run tests for a provider service: `pytest -n auto -vvv -s -x tests/providers/<provider>/services/<service>`
- Run tests for a provider check: `pytest -n auto -vvv -s -x tests/providers/<provider>/services/<service>/<check>`
> Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) documentation for more information.
???+ note
Refer to the [pytest documentation](https://docs.pytest.org/en/7.1.x/getting-started.html) documentation for more information.
## AWS
For the AWS provider we have ways to test a Prowler check based on the following criteria:
> Note: We use and contribute to the [Moto](https://github.com/getmoto/moto) library which allows us to easily mock out tests based on AWS infrastructure. **It's awesome!**
???+ note
We use and contribute to the [Moto](https://github.com/getmoto/moto) library which allows us to easily mock out tests based on AWS infrastructure. **It's awesome!**
- AWS API calls covered by [Moto](https://github.com/getmoto/moto):
- Service tests with `@mock_<service>`
@@ -195,7 +197,8 @@ class Test_iam_password_policy_uppercase:
If the IAM service for the check's we want to test is not covered by Moto, we have to inject the objects in the service client using [MagicMock](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.MagicMock). As we have pointed above, we cannot instantiate the service since it will make real calls to the AWS APIs.
> The following example uses the IAM GetAccountPasswordPolicy which is covered by Moto but this is only for demonstration purposes.
???+ note
The following example uses the IAM GetAccountPasswordPolicy which is covered by Moto but this is only for demonstration purposes.
The following code shows how to use MagicMock to create the service objects.
@@ -325,7 +328,8 @@ class Test_iam_password_policy_uppercase:
Note that this does not use Moto, to keep it simple, but if you use any `moto`-decorators in addition to the patch, the call to `orig(self, operation_name, kwarg)` will be intercepted by Moto.
> The above code comes from here https://docs.getmoto.org/en/latest/docs/services/patching_other_services.html
???+ note
The above code comes from here https://docs.getmoto.org/en/latest/docs/services/patching_other_services.html
#### Mocking more than one service
@@ -385,7 +389,7 @@ with mock.patch(
"prowler.providers.<provider>.lib.audit_info.audit_info.audit_info",
new=audit_info,
), mock.patch(
"prowler.providers.aws.services.<service>.<check>.<check>.<service>_client",
"prowler.providers.<provider>.services.<service>.<check>.<check>.<service>_client",
new=<SERVICE>(audit_info),
):
```
@@ -407,10 +411,10 @@ with mock.patch(
"prowler.providers.<provider>.lib.audit_info.audit_info.audit_info",
new=audit_info,
), mock.patch(
"prowler.providers.aws.services.<service>.<SERVICE>",
"prowler.providers.<provider>.services.<service>.<SERVICE>",
new=<SERVICE>(audit_info),
) as service_client, mock.patch(
"prowler.providers.aws.services.<service>.<service>_client.<service>_client",
"prowler.providers.<provider>.services.<service>.<service>_client.<service>_client",
new=service_client,
):
```
@@ -523,7 +527,7 @@ from unittest import mock
from uuid import uuid4
# Azure Constants
AZURE_SUSCRIPTION = str(uuid4())
from tests.providers.azure.azure_fixtures import AZURE_SUBSCRIPTION
@@ -542,7 +546,7 @@ class Test_defender_ensure_defender_for_arm_is_on:
# Create the custom Defender object to be tested
defender_client.pricings = {
AZURE_SUSCRIPTION: {
AZURE_SUBSCRIPTION: {
"Arm": Defender_Pricing(
resource_id=resource_id,
pricing_tier="Not Standard",
@@ -580,9 +584,9 @@ class Test_defender_ensure_defender_for_arm_is_on:
assert result[0].status == "FAIL"
assert (
result[0].status_extended
== f"Defender plan Defender for ARM from subscription {AZURE_SUSCRIPTION} is set to OFF (pricing tier not standard)"
== f"Defender plan Defender for ARM from subscription {AZURE_SUBSCRIPTION} is set to OFF (pricing tier not standard)"
)
assert result[0].subscription == AZURE_SUSCRIPTION
assert result[0].subscription == AZURE_SUBSCRIPTION
assert result[0].resource_name == "Defender plan ARM"
assert result[0].resource_id == resource_id
```