diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml index b5de1af83f..9ab59f017d 100644 --- a/docker-compose-dev.yml +++ b/docker-compose-dev.yml @@ -144,6 +144,10 @@ services: condition: service_healthy neo4j: condition: service_healthy + ulimits: + nofile: + soft: 65536 + hard: 65536 entrypoint: - "/home/prowler/docker-entrypoint.sh" - "worker" @@ -166,6 +170,10 @@ services: condition: service_healthy neo4j: condition: service_healthy + ulimits: + nofile: + soft: 65536 + hard: 65536 entrypoint: - "../docker-entrypoint.sh" - "beat" diff --git a/docker-compose.yml b/docker-compose.yml index 992a753ac0..798d1ecaba 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -117,6 +117,10 @@ services: condition: service_healthy postgres: condition: service_healthy + ulimits: + nofile: + soft: 65536 + hard: 65536 entrypoint: - "/home/prowler/docker-entrypoint.sh" - "worker" @@ -131,6 +135,10 @@ services: condition: service_healthy postgres: condition: service_healthy + ulimits: + nofile: + soft: 65536 + hard: 65536 entrypoint: - "../docker-entrypoint.sh" - "beat" diff --git a/docs/troubleshooting.mdx b/docs/troubleshooting.mdx index 8e1cca7b7c..a6a28c0ad6 100644 --- a/docs/troubleshooting.mdx +++ b/docs/troubleshooting.mdx @@ -49,15 +49,13 @@ AWS_PROFILE=prowler-profile - If you are scanning multiple AWS accounts, you may need to add multiple profiles to your AWS config. Note that this workaround is mainly for local testing; for production or multi-account setups, follow the [CloudFormation Template guide](https://github.com/prowler-cloud/prowler/issues/7745) and ensure the correct IAM roles and permissions are set up in each account. -### Scans complete but reports are missing or compliance data is empty (`Too many open files` error) +### Scans Complete but Reports Are Missing or Compliance Data Is Empty (`Too many open files` Error) -When running Prowler App via Docker Compose, you may encounter situations where scans complete successfully but reports are not available for download, compliance data shows as empty, or you see 404 errors when trying to access scan reports. Checking the `worker` container logs may reveal errors like `[Errno 24] Too many open files`. +When running Prowler App via Docker Compose, scans may complete successfully but reports are not available for download, compliance data shows as empty, or 404 errors appear when trying to access scan reports. Checking the `worker` container logs may reveal errors like `[Errno 24] Too many open files`. -This issue occurs because the default file descriptor limits in Docker containers are too low for Prowler's operations. +This issue occurs because the default file descriptor limits in Docker containers are too low for Prowler's operations. The default `docker-compose.yml` already includes `ulimits` configuration with `nofile` set to `65536` for the `worker` and `worker-beat` services to prevent this issue. -**Solution:** - -Add `ulimits` configuration to the `worker` and `worker-beat` services in your `docker-compose.yaml`: +If a custom `docker-compose.yml` is being used or the default configuration has been modified, ensure the `ulimits` configuration is present in both the `worker` and `worker-beat` services: ```yaml services: @@ -76,17 +74,13 @@ services: # ... rest of service configuration ``` -After making these changes, restart your Docker Compose stack: +After making these changes, restart the Docker Compose stack: ```bash docker compose down docker compose up -d ``` - -We are evaluating adding these values to the default `docker-compose.yml` to avoid this issue in future releases. - - ### API Container Fails to Start with JWT Key Permission Error See [GitHub Issue #8897](https://github.com/prowler-cloud/prowler/issues/8897) for more details. diff --git a/prowler/CHANGELOG.md b/prowler/CHANGELOG.md index 8901c6f4a1..4bfea57e81 100644 --- a/prowler/CHANGELOG.md +++ b/prowler/CHANGELOG.md @@ -20,6 +20,7 @@ All notable changes to the **Prowler SDK** are documented in this file. - OpenStack compute 7 new checks [(#9944)](https://github.com/prowler-cloud/prowler/pull/9944) - CSA CCM 4.0 for the Alibaba Cloud provider [(#10061)](https://github.com/prowler-cloud/prowler/pull/10061) - ECS Exec (ECS-006) privilege escalation detection via `ecs:ExecuteCommand` + `ecs:DescribeTasks` [(#10066)](https://github.com/prowler-cloud/prowler/pull/10066) +- Add file descriptor limits (`ulimits`) to Docker Compose worker services to prevent `Too many open files` errors [(#10107)](https://github.com/prowler-cloud/prowler/pull/10107) ### 🔄 Changed