prowler/GITHUB_INTEGRATION_IMPLEMENTATION.md

GitHub Integration Implementation Summary

This document summarizes the complete GitHub integration implementation for Prowler, which allows sending findings as GitHub Issues similar to the existing Jira integration.

Implementation Overview

The GitHub integration has been fully implemented across all layers of the Prowler application:

• API client layer
• Backend models and serializers
• API endpoints and views
• Async tasks and job processing
• URL routing

Files Created

1. GitHub API Client (prowler/lib/outputs/github/)

prowler/lib/outputs/github/exceptions/exceptions.py

• Comprehensive exception classes for GitHub integration errors
• Includes exceptions for authentication, repository access, issue creation, etc.

prowler/lib/outputs/github/exceptions/__init__.py

• Exports all GitHub exception classes

prowler/lib/outputs/github/github.py

• Main GitHub class for interacting with GitHub API
• Supports Personal Access Token (PAT) authentication
• Key methods:
  • __init__(): Initialize and authenticate GitHub client
  • test_connection(): Test connection and fetch accessible repositories (static method)
  • get_repositories(): Get all accessible repositories for the authenticated user
  • get_repository_labels(): Get available labels for a repository
  • send_finding(): Create a GitHub issue from a Prowler finding

prowler/lib/outputs/github/__init__.py

• Exports GitHub and GitHubConnection classes

Key Features of GitHub Client:

• Native markdown support (GitHub natively supports markdown, unlike Jira's ADF)
• Comprehensive finding details in issue body with formatted tables
• Severity and status indicators with emojis
• Code blocks for remediation steps (CLI, Terraform, Native IaC)
• Resource tags and compliance framework information
• Error handling and logging

Files Modified

1. Backend Models

api/src/backend/api/models.py

• Added GITHUB = "github", _("GitHub") to Integration.IntegrationChoices

2. Serializers and Validators

api/src/backend/api/v1/serializer_utils/integrations.py

• Added GitHubConfigSerializer: Serializer for GitHub configuration (owner, repositories)
• Added GitHubCredentialSerializer: Serializer for GitHub credentials (token, owner)
• Updated IntegrationCredentialField schema to include GitHub credentials documentation
• Updated IntegrationConfigField schema to include GitHub configuration

api/src/backend/api/v1/serializers.py

• Added IntegrationGitHubDispatchSerializer: Serializer for dispatching findings to GitHub
• Updated BaseWriteIntegrationSerializer.validate_integration_data() to handle GitHub integration
• Updated IntegrationSerializer.to_representation() to include GitHub owner in configuration
• Added imports for GitHubConfigSerializer and GitHubCredentialSerializer

3. API Filters

api/src/backend/api/filters.py

• Added IntegrationGitHubFindingsFilter: Filter for GitHub findings dispatch

4. API Views

api/src/backend/api/v1/views.py

• Added IntegrationGitHubViewSet: ViewSet for GitHub integration dispatch
  • Handles POST requests to send findings to GitHub as issues
  • Validates repository access
  • Triggers async GitHub integration task
• Added imports for IntegrationGitHubDispatchSerializer, IntegrationGitHubFindingsFilter, and github_integration_task

5. URL Routing

api/src/backend/api/v1/urls.py

• Added GitHub integration router: /integrations/{integration_id}/github/dispatches
• Added import for IntegrationGitHubViewSet

6. Backend Utilities

api/src/backend/api/utils.py

• Updated initialize_prowler_integration() to support GitHub integration
  • Initializes GitHub client from integration credentials
  • Handles authentication errors
• Updated prowler_integration_connection_test() to test GitHub connections
  • Fetches repositories on successful connection
  • Updates integration configuration with repository list

7. Async Tasks

api/src/backend/tasks/tasks.py

• Added github_integration_task(): Celery task for GitHub integration
  • Queued on "integrations" queue
  • Delegates to send_findings_to_github() job
• Added import for send_findings_to_github

8. Integration Jobs

api/src/backend/tasks/jobs/integrations.py

• Added send_findings_to_github(): Business logic for sending findings to GitHub
  • Fetches findings with related resources and metadata
  • Extracts remediation information
  • Calls GitHub API client to create issues
  • Returns success/failure counts

API Endpoints

Create GitHub Integration

POST /api/v1/integrations
Content-Type: application/json

{
  "integration_type": "github",
  "enabled": true,
  "credentials": {
    "token": "ghp_xxxxxxxxxxxx",
    "owner": "myorg"  // optional
  },
  "configuration": {},
  "providers": []
}

Test GitHub Connection

POST /api/v1/integrations/{integration_id}/connection

Send Findings to GitHub

POST /api/v1/integrations/{integration_id}/github/dispatches
Content-Type: application/json

{
  "repository": "owner/repo",
  "labels": ["security", "prowler"],  // optional
  "finding_id": "uuid",  // or finding_id__in: ["uuid1", "uuid2"]
}

Data Flow

1. Integration Creation:

   • User provides GitHub PAT and optional owner
   • Backend validates credentials
   • GitHub API client tests authentication
   • Repositories are fetched and stored in configuration

2. Connection Testing:

   • User triggers connection test
   • Async task fetches repositories
   • Configuration updated with latest repository list
   • Connection status saved

3. Dispatching Findings:

   • User selects findings and target repository
   • API validates repository exists in configuration
   • Async task processes each finding:
     • Fetches finding details, resources, metadata
     • Builds markdown issue body
     • Creates GitHub issue via API
   • Returns success/failure counts

GitHub Issue Format

Created issues include:

• Title: [Prowler] SEVERITY - CHECK_ID - RESOURCE_UID
• Body:
  • Finding details table (severity, status, provider, region, resource info)
  • Risk description
  • Recommendations
  • Remediation code blocks (CLI, Terraform, Native IaC)
  • Resource tags
  • Compliance frameworks
  • Link back to finding in Prowler

Configuration

GitHub Personal Access Token Requirements

The PAT must have the following scopes:

• repo - Full control of private repositories (to create issues)

Integration Configuration Structure

{
  "repositories": {
    "owner/repo1": "repo1",
    "owner/repo2": "repo2"
  },
  "owner": "myorg"
}

Credentials Structure (Encrypted)

{
  "token": "ghp_xxxxxxxxxxxx",
  "owner": "myorg"
}

Next Steps

1. Database Migration (Required)

Create a Django migration to add GitHub to the Integration model choices:

cd api/src
python manage.py makemigrations
python manage.py migrate

2. UI Implementation (To Be Done)

Following the Jira integration UI pattern, create:

ui/components/integrations/github/

• github-integrations-manager.tsx - List, add, edit, delete integrations
• github-integration-form.tsx - Form for creating/editing integrations
• github-integration-card.tsx - Display integration status

ui/actions/integrations/

• github-dispatch.ts - Server actions for dispatching findings
  • sendFindingToGitHub()
  • pollGitHubDispatchTask()

Key UI Components:

• GitHub token input (with validation)
• Repository owner input (optional)
• Test connection button
• Repository selector dropdown
• Labels input (multi-select or comma-separated)
• Dispatch findings interface

3. Testing Checklist

• Create GitHub integration with valid PAT
• Test connection and verify repositories are fetched
• Update integration credentials
• Send single finding to GitHub repository
• Send multiple findings in batch
• Verify issue creation in GitHub
• Test with invalid token (should fail gracefully)
• Test with repository user doesn't have access to
• Verify labels are applied correctly
• Check markdown rendering in GitHub issues

Architecture Consistency

This implementation follows the exact same pattern as the Jira integration:

• ✅ Same file structure and organization
• ✅ Same serializer and validator patterns
• ✅ Same ViewSet and URL routing structure
• ✅ Same async task and job processing flow
• ✅ Same connection testing mechanism
• ✅ Same error handling patterns

Security Considerations

• GitHub PAT is encrypted using Fernet encryption before storage
• PAT is never exposed in API responses
• Repository access is validated before allowing dispatch
• All API calls use HTTPS
• Rate limiting should be considered for GitHub API calls

Performance Notes

• Repository fetching is paginated (100 per page)
• Findings are processed individually (can be parallelized in future)
• Async tasks prevent API timeout on large batches
• Connection testing is cached in integration configuration

Compatibility

• Works with GitHub.com (default)
• Can be configured for GitHub Enterprise Server (via api_url parameter)
• Supports both user and organization repositories
• Compatible with GitHub's REST API v3

---

Implementation Status: ✅ Backend Complete | ⏳ Database Migration Needed | ⏳ UI Pending