feat(llm): add LLM provider (#8555)

Co-authored-by: Andoni A. <14891798+andoniaf@users.noreply.github.com>

docs/developer-guide/llm-details.md

@@ -0,0 +1,102 @@
+# LLM Provider
+
+This page details the [Large Language Model (LLM)](https://en.wikipedia.org/wiki/Large_language_model) provider implementation in Prowler.
+
+The LLM provider enables security testing of language models using red team techniques. By default, Prowler uses the built-in LLM configuration that targets OpenAI models with comprehensive security test suites. To configure it, follow the [LLM getting started guide](../tutorials/llm/getting-started-llm.md).
+
+## LLM Provider Classes Architecture
+
+The LLM provider implementation follows the general [Provider structure](./provider.md). This section focuses on the LLM-specific implementation, highlighting how the generic provider concepts are realized for LLM security testing in Prowler. For a full overview of the provider pattern, base classes, and extension guidelines, see [Provider documentation](./provider.md).
+
+### Main Class
+
+- **Location:** [`prowler/providers/llm/llm_provider.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/llm/llm_provider.py)
+- **Base Class:** Inherits from `Provider` (see [base class details](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/common/provider.py)).
+- **Purpose:** Central orchestrator for LLM-specific logic, configuration management, and integration with promptfoo for red team testing.
+- **Key LLM Responsibilities:**
+    - Initializes and manages LLM configuration using promptfoo.
+    - Validates configuration and sets up the LLM testing context.
+    - Loads and manages red team test configuration, plugins, and target models.
+    - Provides properties and methods for downstream LLM security testing.
+    - Integrates with promptfoo for comprehensive LLM security evaluation.
+
+### Data Models
+
+- **Location:** [`prowler/providers/llm/models.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/llm/models.py)
+- **Purpose:** Define structured data for LLM output options and configuration.
+- **Key LLM Models:**
+    - `LLMOutputOptions`: Customizes output filename logic for LLM-specific reporting.
+
+### LLM Security Testing Integration
+
+- **Location:** [`prowler/providers/llm/llm_provider.py`](https://github.com/prowler-cloud/prowler/blob/master/prowler/providers/llm/llm_provider.py)
+- **Purpose:** Integrates with promptfoo for comprehensive LLM security testing.
+- **Key LLM Responsibilities:**
+    - Executes promptfoo red team evaluations against target LLMs.
+    - Processes security test results and converts them to Prowler reports.
+    - Manages test concurrency and progress tracking.
+    - Handles real-time streaming of test results.
+
+### Configuration Management
+
+The LLM provider uses promptfoo configuration files to define:
+
+- **Target Models**: The LLM models to test (e.g., OpenAI GPT, Anthropic Claude)
+- **Red Team Plugins**: Security test suites (OWASP, MITRE, NIST, EU AI Act)
+- **Test Parameters**: Concurrency, test counts, and evaluation criteria
+
+### Default Configuration
+
+Prowler includes a comprehensive default LLM configuration that:
+
+- Targets OpenAI models by default
+- Includes multiple security test frameworks (OWASP, MITRE, NIST, EU AI Act)
+- Provides extensive test coverage for LLM security vulnerabilities
+- Supports custom configuration for specific testing needs
+
+## Specific Patterns in LLM Security Testing
+
+The LLM provider implements security testing through integration with promptfoo, following these patterns:
+
+### Red Team Testing Framework
+
+- **Plugin-based Architecture**: Uses promptfoo plugins for different security test categories
+- **Comprehensive Coverage**: Includes OWASP LLM Top 10, MITRE ATLAS, NIST AI Risk Management, and EU AI Act compliance
+- **Real-Time Evaluation**: Streams test results as they are generated
+- **Progress Tracking**: Provides detailed progress information during test execution
+
+### Test Execution Flow
+
+1. **Configuration Loading**: Loads promptfoo configuration with target models and test plugins
+2. **Test Generation**: Generates security test cases based on configured plugins
+3. **Concurrent Execution**: Runs tests with configurable concurrency limits
+4. **Result Processing**: Converts promptfoo results to Prowler security reports
+5. **Progress Monitoring**: Tracks and displays test execution progress
+
+### Security Test Categories
+
+The LLM provider supports comprehensive security testing across multiple frameworks:
+
+- **OWASP LLM Top 10**: Covers prompt injection, data leakage, and model security
+- **MITRE ATLAS**: Adversarial threat landscape for AI systems
+- **NIST AI Risk Management**: AI system risk assessment and mitigation
+- **EU AI Act**: European Union AI regulation compliance
+- **Custom Tests**: Support for organization-specific security requirements
+
+## Error Handling and Validation
+
+The LLM provider includes comprehensive error handling for:
+
+- **Configuration Validation**: Ensures valid promptfoo configuration files
+- **Model Access**: Handles authentication and access issues with target LLMs
+- **Test Execution**: Manages test failures and timeout scenarios
+- **Result Processing**: Handles malformed or incomplete test results
+
+## Integration with Prowler Ecosystem
+
+The LLM provider seamlessly integrates with Prowler's existing infrastructure:
+
+- **Output Formats**: Supports all Prowler output formats (JSON, CSV, HTML, etc.)
+- **Compliance Frameworks**: Integrates with Prowler's compliance reporting
+- **Fixer Integration**: Supports automated remediation recommendations
+- **Dashboard Integration**: Compatible with Prowler App for centralized management

docs/index.md

@@ -14,6 +14,7 @@ The official supported providers right now are:
 | **Github** | Official | Stable | UI, API, CLI |
 | **IaC** | Official | Beta | CLI |
 | **MongoDB Atlas** | Official | Beta | CLI |
+| **LLM** | Official | Beta | CLI |
 | **NHN** | Unofficial | Beta | CLI |
 
 Prowler supports **auditing, incident response, continuous monitoring, hardening, forensic readiness, and remediation**.

docs/tutorials/llm/getting-started-llm.md

@@ -0,0 +1,142 @@
+# Getting Started With LLM on Prowler
+
+## Overview
+
+Prowler's LLM provider enables comprehensive security testing of large language models using red team techniques. It integrates with [promptfoo](https://promptfoo.dev/) to provide extensive security evaluation capabilities.
+
+## Prerequisites
+
+Before using the LLM provider, ensure the following requirements are met:
+
+- **promptfoo installed**: The LLM provider requires promptfoo to be installed on the system
+- **LLM API access**: Valid API keys for the target LLM models to test
+- **Email verification**: promptfoo requires email verification for red team evaluations
+
+## Installation
+
+### Install promptfoo
+
+Install promptfoo using one of the following methods:
+
+**Using npm:**
+```bash
+npm install -g promptfoo
+```
+
+**Using Homebrew (macOS):**
+```bash
+brew install promptfoo
+```
+
+**Using other package managers:**
+See the [promptfoo installation guide](https://promptfoo.dev/docs/installation/) for additional installation methods.
+
+### Verify Installation
+
+```bash
+promptfoo --version
+```
+
+## Configuration
+
+### Step 1: Email Verification
+
+promptfoo requires email verification for red team evaluations. Set the email address:
+
+```bash
+promptfoo config set email your-email@company.com
+```
+
+### Step 2: Configure LLM API Keys
+
+Set up API keys for the target LLM models. For OpenAI (default configuration):
+
+```bash
+export OPENAI_API_KEY="your-openai-api-key"
+```
+
+For other providers, see the [promptfoo documentation](https://promptfoo.dev/docs/providers/) for specific configuration requirements.
+
+### Step 3: Generate Test Cases (Optional)
+
+Prowler provides a default suite of red team tests but to customize the test cases, generate them first:
+
+```bash
+promptfoo redteam generate
+```
+
+This creates test cases based on your configuration.
+
+## Usage
+
+### Basic Usage
+
+Run LLM security testing with the default configuration:
+
+```bash
+prowler llm
+```
+
+### Custom Configuration
+
+Use a custom promptfoo configuration file:
+
+```bash
+prowler llm --config-path /path/to/your/config.yaml
+```
+
+### Output Options
+
+Generate reports in various formats:
+
+```bash
+# JSON output
+prowler llm --output-format json
+
+# CSV output
+prowler llm --output-format csv
+
+# HTML report
+prowler llm --output-format html
+```
+
+### Concurrency Control
+
+Adjust the number of concurrent tests:
+
+```bash
+prowler llm --max-concurrency 5
+```
+
+## Default Configuration
+
+Prowler includes a comprehensive default LLM configuration that provides:
+
+- **Target Models**: OpenAI GPT models by default
+- **Security Frameworks**:
+  - OWASP LLM Top 10
+  - OWASP API Top 10
+  - MITRE ATLAS
+  - NIST AI Risk Management Framework
+  - EU AI Act compliance
+- **Test Coverage**: Over 5,000 security test cases
+- **Plugin Support**: Multiple security testing plugins
+
+## Advanced Configuration
+
+### Custom Test Suites
+
+Create custom test configurations by modifying the promptfoo config file in `prowler/config/llm_config.yaml` or pass a custom configuration with `--config-file` flag:
+
+```yaml
+description: Custom LLM Security Tests
+targets:
+  - id: openai:gpt-4
+redteam:
+  plugins:
+    - id: owasp:llm
+      numTests: 10
+    - id: mitre:atlas
+      numTests: 5
+```
+