mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-03-22 03:08:23 +00:00
494 lines
13 KiB
Plaintext
494 lines
13 KiB
Plaintext
---
|
||
title: 'AWS Organizations Bulk Provisioning in Prowler'
|
||
---
|
||
|
||
Prowler offers an automated tool to discover and provision all AWS accounts within an AWS Organization. This streamlines onboarding for organizations managing multiple AWS accounts by automatically generating the configuration needed for bulk provisioning.
|
||
|
||
The tool, `aws_org_generator.py`, complements the [Bulk Provider Provisioning](./bulk-provider-provisioning) tool and is available in the Prowler repository at: [util/prowler-bulk-provisioning](https://github.com/prowler-cloud/prowler/tree/master/util/prowler-bulk-provisioning)
|
||
|
||
<Note>
|
||
**Native AWS Organizations support is now available in Prowler Cloud.** You can onboard all accounts via the UI wizard — with automatic discovery, hierarchical tree selection, connection testing, and bulk scan launch — without any scripts or YAML files.
|
||
|
||
See [AWS Organizations in Prowler Cloud](/user-guide/tutorials/prowler-cloud-aws-organizations).
|
||
|
||
The CLI-based tool below remains useful for self-hosted Prowler App and advanced automation scenarios.
|
||
</Note>
|
||
|
||
{/* TODO: Add screenshot of the tool in action */}
|
||
|
||
## Overview
|
||
|
||
The AWS Organizations Bulk Provisioning tool simplifies multi-account onboarding by:
|
||
|
||
* Automatically discovering all active accounts in an AWS Organization
|
||
* Generating YAML configuration files for bulk provisioning
|
||
* Supporting account filtering and custom role configurations
|
||
* Eliminating manual entry of account IDs and role ARNs
|
||
|
||
## Prerequisites
|
||
|
||
### Requirements
|
||
|
||
* Python 3.7 or higher
|
||
* AWS credentials with Organizations read access
|
||
* ProwlerRole (or custom role) deployed across all target accounts
|
||
* Prowler API key (from Prowler Cloud or self-hosted Prowler App)
|
||
* For self-hosted Prowler App, remember to [point to your API base URL](./bulk-provider-provisioning#custom-api-endpoints)
|
||
* Learn how to create API keys: [Prowler App API Keys](../tutorials/prowler-app-api-keys)
|
||
|
||
### Deploying ProwlerRole Across AWS Organizations
|
||
|
||
Before using the AWS Organizations generator, deploy the ProwlerRole across all accounts in the organization using CloudFormation StackSets.
|
||
|
||
<Note>
|
||
**Follow the official documentation:**
|
||
[Deploying Prowler IAM Roles Across AWS Organizations](../providers/aws/organizations#deploying-prowler-iam-roles-across-aws-organizations)
|
||
|
||
**Key points:**
|
||
|
||
* Use CloudFormation StackSets from the management account
|
||
* Deploy to all organizational units (OUs) or specific OUs
|
||
* Use an external ID for enhanced security
|
||
* Ensure the role has necessary permissions for Prowler scans
|
||
</Note>
|
||
|
||
### Installation
|
||
|
||
Clone the repository and install required dependencies:
|
||
|
||
```bash
|
||
git clone https://github.com/prowler-cloud/prowler.git
|
||
cd prowler/util/prowler-bulk-provisioning
|
||
pip install -r requirements-aws-org.txt
|
||
```
|
||
|
||
### AWS Credentials Setup
|
||
|
||
Configure AWS credentials with Organizations read access:
|
||
|
||
* **Management account credentials**, or
|
||
* **Delegated administrator account** with `organizations:ListAccounts` permission
|
||
|
||
Required IAM permissions:
|
||
|
||
```json
|
||
{
|
||
"Version": "2012-10-17",
|
||
"Statement": [
|
||
{
|
||
"Effect": "Allow",
|
||
"Action": [
|
||
"organizations:ListAccounts",
|
||
"organizations:DescribeOrganization"
|
||
],
|
||
"Resource": "*"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
### Prowler API Key Setup
|
||
|
||
Configure your Prowler API key:
|
||
|
||
```bash
|
||
export PROWLER_API_KEY="pk_example-api-key"
|
||
```
|
||
|
||
To create an API key:
|
||
|
||
1. Log in to Prowler Cloud or Prowler App
|
||
2. Click **Profile** → **Account**
|
||
3. Click **Create API Key**
|
||
4. Provide a descriptive name and optionally set an expiration date
|
||
5. Copy the generated API key (it will only be shown once)
|
||
|
||
For detailed instructions, see: [Prowler App API Keys](../tutorials/prowler-app-api-keys)
|
||
|
||
## Basic Usage
|
||
|
||
### Generate Configuration for All Accounts
|
||
|
||
To generate a YAML configuration file for all active accounts in the organization:
|
||
|
||
```bash
|
||
python aws_org_generator.py -o aws-accounts.yaml --external-id prowler-ext-id-2024
|
||
```
|
||
|
||
This command:
|
||
|
||
1. Lists all ACTIVE accounts in the organization
|
||
2. Generates YAML entries for each account
|
||
3. Saves the configuration to `aws-accounts.yaml`
|
||
|
||
**Output:**
|
||
|
||
```
|
||
Fetching accounts from AWS Organizations...
|
||
Found 47 active accounts in organization
|
||
Generated configuration for 47 accounts
|
||
|
||
Configuration written to: aws-accounts.yaml
|
||
|
||
Next steps:
|
||
1. Review the generated file: cat aws-accounts.yaml | head -n 20
|
||
2. Run bulk provisioning: python prowler_bulk_provisioning.py aws-accounts.yaml
|
||
```
|
||
|
||
### Review Generated Configuration
|
||
|
||
Review the generated YAML configuration:
|
||
|
||
```bash
|
||
head -n 20 aws-accounts.yaml
|
||
```
|
||
|
||
**Example output:**
|
||
|
||
```yaml
|
||
- provider: aws
|
||
uid: '111111111111'
|
||
alias: Production-Account
|
||
auth_method: role
|
||
credentials:
|
||
role_arn: arn:aws:iam::111111111111:role/ProwlerRole
|
||
external_id: prowler-ext-id-2024
|
||
|
||
- provider: aws
|
||
uid: '222222222222'
|
||
alias: Development-Account
|
||
auth_method: role
|
||
credentials:
|
||
role_arn: arn:aws:iam::222222222222:role/ProwlerRole
|
||
external_id: prowler-ext-id-2024
|
||
```
|
||
|
||
### Dry Run Mode
|
||
|
||
Test the configuration without writing a file:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
--external-id prowler-ext-id-2024 \
|
||
--dry-run
|
||
```
|
||
|
||
## Advanced Configuration
|
||
|
||
### Using a Specific AWS Profile
|
||
|
||
Specify an AWS profile when multiple profiles are configured:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--profile org-management-admin \
|
||
--external-id prowler-ext-id-2024
|
||
```
|
||
|
||
### Excluding Specific Accounts
|
||
|
||
Exclude the management account or other accounts from provisioning:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--external-id prowler-ext-id-2024 \
|
||
--exclude 123456789012,210987654321
|
||
```
|
||
|
||
Common exclusion scenarios:
|
||
|
||
* Management account (requires different permissions)
|
||
* Break-glass accounts (emergency access)
|
||
* Suspended or archived accounts
|
||
|
||
### Including Only Specific Accounts
|
||
|
||
Generate configuration for specific accounts only:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--external-id prowler-ext-id-2024 \
|
||
--include 111111111111,222222222222,333333333333
|
||
```
|
||
|
||
### Custom Role Name
|
||
|
||
Specify a custom role name if not using the default `ProwlerRole`:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--role-name ProwlerExecutionRole \
|
||
--external-id prowler-ext-id-2024
|
||
```
|
||
|
||
### Custom Alias Format
|
||
|
||
Customize account aliases using template variables:
|
||
|
||
```bash
|
||
# Use account name and ID
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--alias-format "{name}-{id}" \
|
||
--external-id prowler-ext-id-2024
|
||
|
||
# Use email prefix
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--alias-format "{email}" \
|
||
--external-id prowler-ext-id-2024
|
||
```
|
||
|
||
Available template variables:
|
||
|
||
* `{name}` - Account name
|
||
* `{id}` - Account ID
|
||
* `{email}` - Account email
|
||
|
||
### Additional Role Assumption Options
|
||
|
||
Configure optional role assumption parameters:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--role-name ProwlerRole \
|
||
--external-id prowler-ext-id-2024 \
|
||
--session-name prowler-scan-session \
|
||
--duration-seconds 3600
|
||
```
|
||
|
||
## Complete Workflow Example
|
||
|
||
<Steps>
|
||
<Step title="Deploy ProwlerRole Using StackSets">
|
||
1. Log in to the AWS management account
|
||
2. Open CloudFormation → StackSets
|
||
3. Create a new StackSet using the [Prowler role template](https://github.com/prowler-cloud/prowler/blob/master/permissions/templates/cloudformation/prowler-scan-role.yml)
|
||
4. Deploy to all organizational units
|
||
5. Use a unique external ID (e.g., `prowler-org-2024-abc123`)
|
||
|
||
{/* TODO: Add screenshot of CloudFormation StackSets deployment */}
|
||
</Step>
|
||
|
||
<Step title="Generate YAML Configuration">
|
||
Configure AWS credentials and generate the YAML file:
|
||
|
||
```bash
|
||
# Using management account credentials
|
||
export AWS_PROFILE=org-management
|
||
|
||
# Generate configuration
|
||
python aws_org_generator.py \
|
||
-o aws-org-accounts.yaml \
|
||
--external-id prowler-org-2024-abc123 \
|
||
--exclude 123456789012
|
||
```
|
||
|
||
**Output:**
|
||
|
||
```
|
||
Fetching accounts from AWS Organizations...
|
||
Using AWS profile: org-management
|
||
Found 47 active accounts in organization
|
||
Generated configuration for 46 accounts
|
||
|
||
Configuration written to: aws-org-accounts.yaml
|
||
|
||
Next steps:
|
||
1. Review the generated file: cat aws-org-accounts.yaml | head -n 20
|
||
2. Run bulk provisioning: python prowler_bulk_provisioning.py aws-org-accounts.yaml
|
||
```
|
||
</Step>
|
||
|
||
<Step title="Review Generated Configuration">
|
||
Verify the generated YAML configuration:
|
||
|
||
```bash
|
||
# View first 20 lines
|
||
head -n 20 aws-org-accounts.yaml
|
||
|
||
# Check for unexpected accounts
|
||
grep "uid:" aws-org-accounts.yaml
|
||
|
||
# Verify role ARNs
|
||
grep "role_arn:" aws-org-accounts.yaml | head -5
|
||
|
||
# Count accounts
|
||
grep "provider: aws" aws-org-accounts.yaml | wc -l
|
||
```
|
||
</Step>
|
||
|
||
<Step title="Run Bulk Provisioning">
|
||
Provision all accounts to Prowler Cloud or Prowler App:
|
||
|
||
```bash
|
||
# Set Prowler API key
|
||
export PROWLER_API_KEY="pk_example-api-key"
|
||
|
||
# Run bulk provisioning with connection testing
|
||
python prowler_bulk_provisioning.py aws-org-accounts.yaml
|
||
```
|
||
|
||
**With custom options:**
|
||
|
||
```bash
|
||
python prowler_bulk_provisioning.py aws-org-accounts.yaml \
|
||
--concurrency 10 \
|
||
--timeout 120
|
||
```
|
||
|
||
**Successful output:**
|
||
|
||
```
|
||
[1] ✅ Created provider (id=db9a8985-f9ec-4dd8-b5a0-e05ab3880bed)
|
||
[1] ✅ Created secret (id=466f76c6-5878-4602-a4bc-13f9522c1fd2)
|
||
[1] ✅ Connection test: Connected
|
||
|
||
[2] ✅ Created provider (id=7a99f789-0cf5-4329-8279-2d443a962676)
|
||
[2] ✅ Created secret (id=c5702180-f7c4-40fd-be0e-f6433479b126)
|
||
[2] ✅ Connection test: Connected
|
||
|
||
Done. Success: 47 Failures: 0
|
||
```
|
||
|
||
{/* TODO: Add screenshot of successful bulk provisioning output */}
|
||
</Step>
|
||
</Steps>
|
||
|
||
## Command Reference
|
||
|
||
### Full Command-Line Options
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o OUTPUT_FILE \
|
||
--role-name ROLE_NAME \
|
||
--external-id EXTERNAL_ID \
|
||
--session-name SESSION_NAME \
|
||
--duration-seconds SECONDS \
|
||
--alias-format FORMAT \
|
||
--exclude ACCOUNT_IDS \
|
||
--include ACCOUNT_IDS \
|
||
--profile AWS_PROFILE \
|
||
--region AWS_REGION \
|
||
--dry-run
|
||
```
|
||
|
||
## Troubleshooting
|
||
|
||
### Error: "No AWS credentials found"
|
||
|
||
**Solution:** Configure AWS credentials using one of these methods:
|
||
|
||
```bash
|
||
# Method 1: AWS CLI configure
|
||
aws configure
|
||
|
||
# Method 2: Environment variables
|
||
export AWS_ACCESS_KEY_ID=your-key-id
|
||
export AWS_SECRET_ACCESS_KEY=your-secret-key
|
||
|
||
# Method 3: Use AWS profile
|
||
export AWS_PROFILE=org-management
|
||
```
|
||
|
||
### Error: "Access denied to AWS Organizations API"
|
||
|
||
**Cause:** Current credentials don't have permission to list organization accounts.
|
||
|
||
**Solution:**
|
||
|
||
* Ensure management account credentials are used
|
||
* Verify IAM permissions include `organizations:ListAccounts`
|
||
* Check IAM policies for Organizations access
|
||
|
||
### Error: "AWS Organizations is not enabled"
|
||
|
||
**Cause:** The account is not part of an organization.
|
||
|
||
**Solution:** This tool requires an AWS Organization. Create one in the AWS Organizations console or use standard bulk provisioning for standalone accounts.
|
||
|
||
### No Accounts Generated After Filters
|
||
|
||
**Cause:** All accounts were filtered out by `--exclude` or `--include` options.
|
||
|
||
**Solution:** Review filter options and verify account IDs are correct:
|
||
|
||
```bash
|
||
# List all accounts in organization
|
||
aws organizations list-accounts --query "Accounts[?Status=='ACTIVE'].[Id,Name]" --output table
|
||
```
|
||
|
||
### Connection Test Failures During Bulk Provisioning
|
||
|
||
**Cause:** ProwlerRole may not be deployed correctly or credentials are invalid.
|
||
|
||
**Solution:**
|
||
|
||
* Verify StackSet deployment status in CloudFormation
|
||
* Check role trust policy includes correct external ID
|
||
* Test role assumption manually:
|
||
|
||
```bash
|
||
aws sts assume-role \
|
||
--role-arn arn:aws:iam::123456789012:role/ProwlerRole \
|
||
--role-session-name test \
|
||
--external-id prowler-ext-id-2024
|
||
```
|
||
|
||
## Security Best Practices
|
||
|
||
### Use External ID
|
||
|
||
Always use an external ID when assuming cross-account roles:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--external-id $(uuidgen | tr '[:upper:]' '[:lower:]')
|
||
```
|
||
|
||
The external ID must match the one configured in the ProwlerRole trust policy across all accounts.
|
||
|
||
### Exclude Sensitive Accounts
|
||
|
||
Exclude accounts that shouldn't be scanned or require special handling:
|
||
|
||
```bash
|
||
python aws_org_generator.py \
|
||
-o aws-accounts.yaml \
|
||
--external-id prowler-ext-id \
|
||
--exclude 123456789012,111111111111 # management, break-glass accounts
|
||
```
|
||
|
||
### Review Generated Configuration
|
||
|
||
Always review the generated YAML before provisioning:
|
||
|
||
```bash
|
||
# Check for unexpected accounts
|
||
grep "uid:" aws-org-accounts.yaml
|
||
|
||
# Verify role ARNs
|
||
grep "role_arn:" aws-org-accounts.yaml | head -5
|
||
|
||
# Count accounts
|
||
grep "provider: aws" aws-org-accounts.yaml | wc -l
|
||
```
|
||
|
||
## Next Steps
|
||
|
||
<Columns cols={2}>
|
||
<Card title="Bulk Provider Provisioning" icon="terminal" href="/user-guide/tutorials/bulk-provider-provisioning">
|
||
Learn how to bulk provision providers in Prowler.
|
||
</Card>
|
||
<Card title="Prowler App" icon="pen-to-square" href="/user-guide/tutorials/prowler-app">
|
||
Detailed instructions on how to use Prowler.
|
||
</Card>
|
||
</Columns>
|