mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-10-05 03:12:14 +00:00
chore: remove benchmark report file
Benchmark data moved to PR description for better visibility.
This commit is contained in:
alejandrobailo
@@ -1,224 +0,0 @@
|
||||
# pre-commit vs prek: Performance Benchmark Report
|
||||
|
||||
**Project:** Prowler
|
||||
**Date:** 2026-01-26
|
||||
**Environment:** macOS Darwin 25.2.0, Apple Silicon
|
||||
**Versions:** pre-commit 4.5.1 | prek 0.3.0
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
| Metric | pre-commit | prek | Improvement |
|
||||
|--------|-----------|------|-------------|
|
||||
| Cold Install | 25.18s | 3.42s | **7.37x faster** |
|
||||
| Warm Execution (avg) | 87.53s | 14.80s | **5.92x faster** |
|
||||
| Cache Size | 170 MB | 154 MB | **9.4% smaller** |
|
||||
|
||||
**Recommendation:** ✅ **Migrate to prek** — significant performance gains with full compatibility.
|
||||
|
||||
---
|
||||
|
||||
## Detailed Benchmarks
|
||||
|
||||
### 1. Cold Installation (`install-hooks`)
|
||||
|
||||
Caches were cleared before each test to simulate first-time setup.
|
||||
|
||||
| Tool | Time | CPU Usage |
|
||||
|------|------|-----------|
|
||||
| pre-commit | **25.183s** | 54% |
|
||||
| prek | **3.418s** | 252% |
|
||||
|
||||
**Analysis:** prek is **7.37x faster** due to:
|
||||
- Parallel repository cloning
|
||||
- Parallel environment installation
|
||||
- Rust-based implementation with better concurrency
|
||||
|
||||
### 2. Warm Execution (`run --all-files`)
|
||||
|
||||
Running all 18 hooks on the entire codebase with cached environments.
|
||||
|
||||
#### pre-commit Results
|
||||
| Run | Time |
|
||||
|-----|------|
|
||||
| 1 | 117.17s |
|
||||
| 2 | 83.44s |
|
||||
| 3 | 61.97s |
|
||||
| **Average** | **87.53s** |
|
||||
|
||||
#### prek Results
|
||||
| Run | Time |
|
||||
|-----|------|
|
||||
| 1 | 15.81s |
|
||||
| 2 | 14.40s |
|
||||
| 3 | 14.20s |
|
||||
| **Average** | **14.80s** |
|
||||
|
||||
**Analysis:** prek is **5.92x faster** due to:
|
||||
- Concurrent hook execution based on priority
|
||||
- Optimized file change detection
|
||||
- Native Rust performance
|
||||
|
||||
### 3. Disk Space Usage
|
||||
|
||||
| Tool | Cache Size |
|
||||
|------|------------|
|
||||
| pre-commit | 170 MB |
|
||||
| prek | 154 MB |
|
||||
|
||||
**Savings:** 16 MB (9.4% reduction)
|
||||
|
||||
---
|
||||
|
||||
## Hooks Compatibility Matrix
|
||||
|
||||
All 18 hooks in Prowler's `.pre-commit-config.yaml` were tested:
|
||||
|
||||
| Hook | Type | pre-commit | prek | Notes |
|
||||
|------|------|------------|------|-------|
|
||||
| check-merge-conflict | python | ✅ | ✅ | |
|
||||
| check-yaml | python | ✅ | ✅ | |
|
||||
| check-json | python | ✅ | ✅ | |
|
||||
| end-of-file-fixer | python | ✅ | ✅ | |
|
||||
| trailing-whitespace | python | ✅ | ✅ | |
|
||||
| no-commit-to-branch | python | ✅ | ✅ | |
|
||||
| pretty-format-json | python | ✅ | ✅ | |
|
||||
| pretty-format-toml | python | ✅ | ✅ | |
|
||||
| shellcheck | script | ✅ | ✅ | |
|
||||
| autoflake | python | ✅ | ✅ | |
|
||||
| isort | python | ✅ | ✅ | |
|
||||
| black | python | ✅ | ✅ | |
|
||||
| flake8 | python | ✅ | ✅ | |
|
||||
| poetry-check (API) | python | ✅ | ✅ | |
|
||||
| poetry-lock (API) | python | ✅ | ✅ | |
|
||||
| poetry-check (SDK) | python | ✅ | ✅ | |
|
||||
| poetry-lock (SDK) | python | ✅ | ✅ | |
|
||||
| hadolint | docker | ✅ | ✅ | Requires hadolint binary |
|
||||
| pylint | system | ✅ | ✅ | |
|
||||
| trufflehog | system | ✅ | ✅ | |
|
||||
| bandit | system | ✅ | ✅ | |
|
||||
| safety | system | ✅ | ✅ | |
|
||||
| vulture | system | ✅ | ✅ | |
|
||||
| ui-checks (husky) | system | ✅ | ✅ | |
|
||||
|
||||
**Result:** 100% compatibility — no configuration changes required.
|
||||
|
||||
---
|
||||
|
||||
## Annual Time Savings Projection
|
||||
|
||||
Assumptions:
|
||||
- 50 developers
|
||||
- 10 commits/developer/day
|
||||
- 250 working days/year
|
||||
|
||||
| Scenario | pre-commit | prek | Annual Savings |
|
||||
|----------|-----------|------|----------------|
|
||||
| Per commit (warm) | 87.5s | 14.8s | 72.7s saved |
|
||||
| Per developer/day | 14.6 min | 2.5 min | 12.1 min saved |
|
||||
| **Per developer/year** | 60.8 hrs | 10.3 hrs | **50.5 hrs saved** |
|
||||
| **Team annual** | 3,042 hrs | 515 hrs | **2,527 hrs saved** |
|
||||
|
||||
**Monetary Impact** (assuming $75/hr developer cost):
|
||||
- **Annual savings: ~$189,525**
|
||||
|
||||
---
|
||||
|
||||
## CI/CD Impact
|
||||
|
||||
### GitHub Actions Cache Benefits
|
||||
|
||||
| Metric | pre-commit | prek |
|
||||
|--------|-----------|------|
|
||||
| Cache upload/download | 170 MB | 154 MB |
|
||||
| Cold install in CI | ~30s | ~5s |
|
||||
| Warm execution in CI | ~90s | ~15s |
|
||||
|
||||
**Estimated CI time savings per PR:** ~100 seconds
|
||||
|
||||
---
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### Phase 1: Validation (This PR)
|
||||
- [x] Create isolated worktree
|
||||
- [x] Run benchmark comparisons
|
||||
- [x] Verify hook compatibility
|
||||
- [ ] Document any edge cases
|
||||
|
||||
### Phase 2: Documentation Updates
|
||||
- [ ] Update `CLAUDE.md`: `poetry run pre-commit` → `prek`
|
||||
- [ ] Update `CONTRIBUTING.md` (if exists)
|
||||
- [ ] Update developer setup guides
|
||||
|
||||
### Phase 3: CI/CD Migration
|
||||
```yaml
|
||||
# .github/workflows/*.yml
|
||||
- name: Install prek
|
||||
run: pipx install prek
|
||||
|
||||
- name: Run hooks
|
||||
run: prek run --all-files
|
||||
```
|
||||
|
||||
### Phase 4: Developer Migration
|
||||
```bash
|
||||
# One-time migration per developer
|
||||
pre-commit uninstall
|
||||
prek install
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risks and Mitigations
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| prek is newer/less mature | Medium | Low | Backed by major projects (Airflow, FastAPI) |
|
||||
| Missing subcommands | Low | Low | Core functionality is complete |
|
||||
| Team unfamiliarity | Low | Low | Commands are identical to pre-commit |
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
**prek delivers substantial performance improvements with zero migration friction:**
|
||||
|
||||
1. **7.37x faster** cold installation
|
||||
2. **5.92x faster** warm execution
|
||||
3. **9.4% smaller** cache footprint
|
||||
4. **100% compatible** with existing configuration
|
||||
5. **No code changes** required
|
||||
|
||||
The migration is low-risk and high-reward, particularly beneficial for:
|
||||
- New contributor onboarding (faster setup)
|
||||
- CI/CD pipelines (reduced build times)
|
||||
- Developer productivity (less waiting)
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Raw Benchmark Data
|
||||
|
||||
```
|
||||
=== COLD INSTALL ===
|
||||
pre-commit install-hooks: 25.183s (54% CPU)
|
||||
prek install-hooks: 3.418s (252% CPU)
|
||||
|
||||
=== WARM EXECUTION (3 runs each) ===
|
||||
pre-commit run --all-files:
|
||||
Run 1: 117.17s
|
||||
Run 2: 83.44s
|
||||
Run 3: 61.97s
|
||||
Average: 87.53s
|
||||
|
||||
prek run --all-files:
|
||||
Run 1: 15.81s
|
||||
Run 2: 14.40s
|
||||
Run 3: 14.20s
|
||||
Average: 14.80s
|
||||
|
||||
=== CACHE SIZES ===
|
||||
pre-commit: 170 MB
|
||||
prek: 154 MB
|
||||
```
|
||||
Reference in New Issue
Block a user