Commit Graph

36 Commits

Author SHA1 Message Date
Víctor Fernández Poyatos 8a2cfea677 feat/PRWLR-4368 Remove TenantMiddleware and set tenant_id at view level (#31)
* feat(API): PRWLR-4368 remove TenantMiddleware in favour of transaction based setup

* feat(API): PRWLR-4368 override initial request method to perform atomic transactions on RLS viewsets
2024-08-21 10:10:01 +02:00
Víctor Fernández Poyatos 308f52c6f9 build(deps): bump gunicorn version to 23.0.0 (#29) 2024-08-13 16:28:39 +02:00
Víctor Fernández Poyatos 4f99518d52 fix(Dockerfile): add curl-dev to base image (#27) 2024-08-12 12:34:55 +02:00
Jon Young a2ab216531 chore(Celery): add basic Celery worker with broker only (#20)
* chore(Celery): add basic Celery worker with broker only

* chore(Celery): saving progress. Not able to schedule tasks

* fix(Celery): add celery app for use by django

* fix(Celery): register tasks

* fix(Docker): add celery workers to docker-compose

* chore(Celery): add django-celery-results backend

to store results using Django ORM

* fix(Celery): get app config the correct way

* fix(Docker): start connecting docker Celery workers to Valkey

not yet operational

* fix(Celery): get celery & django to work in docker-compose

* docs(Celery): document how to run Celery in development environment

includes changes to support the configuration and deployment
of Celery worker and its dependencies, Valkey and Postgres.

* fix(GHA): add valkey to CI services

* fix(GHA): add valkey to CI services

* fix(GHA): add valkey-cli ping to CI services

* fix(GHA): use right port for valkey

* fix(Views): remove debug task code

* test(Celery): start adding celery task tests

not yet working!

* fix(pyproject): rollback django upgrade

* fix(docker): updated docker runtime and env vars

based on feedback from #20

* fix(Dockerfile): include dependencies for psutil

psutil was introduced by pytest-celery

* fix(Backend): PRWLR-4013 fix celery settings structure

* fix(Celery): update celery app to work with new settings structure

* fix(Views): remove debug task code

* fix(Config): remove debug code

* fix(Celery): update celery app name when running worker

---------

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2024-08-09 16:47:35 +02:00
dependabot[bot] 43c9e70a65 chore(deps): bump django from 5.0.7 to 5.0.8 (#26)
Bumps [django](https://github.com/django/django) from 5.0.7 to 5.0.8.
- [Commits](https://github.com/django/django/compare/5.0.7...5.0.8)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-08 10:57:10 -04:00
Víctor Fernández Poyatos e2fc83c81b docs(README): add migration and fixtures commands (#23)
* docs(README): add migration and fixtures commands

* docs(README): add env variables info
2024-08-08 10:56:03 -04:00
Víctor Fernández Poyatos 857edd9aa6 chore(Backend,API): PRWLR-4341 restructure project based on versions (#25) 2024-08-07 16:40:18 -04:00
Jon Young 35128b89b1 chore(API): rename project to be Prowler API (#22) 2024-08-07 10:05:05 +02:00
Víctor Fernández Poyatos 4371ad1535 feat/PRWLR-3999 Implement providers endpoints (#21)
* feat(Backend): PRWLR-3989 add RLS to postgresql db and base models

* feat(API): PRWLR-3989 add TenantMiddleware

* chore(API, Backend): PRWLR-3989 create new db user without RLS bypass on migrations

* chore(Backend): PRWLR-3989 fix RLS bypasser for POST requests

* fix(Backend): PRWLR-3989 fix user permissions when migrating new models

* chore(Backend): PRWLR-3989 add testing view for RLS manual tests

* feat(API): PRWLR-3989 add tenant_id to API logging

* chore(API, Backend): PRWLR-3989 add TODOs

* test(API): PRWLR-3989 add new middleware unit tests

* chore(API): PRWLR-3989 refactor RLS code

* fix(tests): PRWLR-3989 fix testing db connector

* chore: PRWLR-3989 add references to JIRA tickets

* fix: PRWLR-3989 remove bypass logic and fix serializers

* fix: PRWLR-3989 improve drop SQL query for RLS models

* feat(Backend): PRWLR-3989 add specific permissions on each model

* fix(Backend): PRWLR-3989 fix database routing and grant select perms

* fix(test): PRWLR-3989 fix routing issues with unit tests

* chore: PRWLR-3989 remove references to JIRA tickets

* feat(Backend): PRWLR-3999 add Provider model

* feat: PRWLR-3999 add providers view logic

* fix: PRWLR-3999 fix unique index fields

* feat(API): PRWLR-3999 add custom exception handlers

* feat(API): PRWLR-3999 add /providers/{provider_id}/connection endpoint

* test(API): PRWLR-3999 add base unit tests for providers

* fix(API, Backend): PRWLR-3999 fix bugs after rebase

* chore(API, docker): PRWLR-3999 add dev feats for demo

* feat(API): PRWLR-3999 add drf-spectacular-jsonapi and improve docs

* test(API): PRWLR-3999 add providers unit tests

* chore(Backend): PRWLR-3989 adjust privileges for user on table tenant

* chore: PRWLR-3999 rename model custom validation error

* chore: PRWLR-3999 remove Test references

* chore: PRWLR-3999 update API v1 spec

* fix: PRWLR-3999 apply requested changes to filter and models

* feat: PRWLR-3999 add validation to PATCH /providers payload

* fix: PRWLR-3999 fix providers enum description

* chore: add more providers fixtures

* fix: PRWLR-3999 make providers.alias optional
2024-08-06 12:39:08 -04:00
Víctor Fernández Poyatos e1a12bcb14 feat/PRWLR-3989 Add RLS full implementation (#18)
* feat(Backend): PRWLR-3989 add RLS to postgresql db and base models

* feat(API): PRWLR-3989 add TenantMiddleware

* chore(API, Backend): PRWLR-3989 create new db user without RLS bypass on migrations

* chore(Backend): PRWLR-3989 fix RLS bypasser for POST requests

* fix(Backend): PRWLR-3989 fix user permissions when migrating new models

* chore(Backend): PRWLR-3989 add testing view for RLS manual tests

* feat(API): PRWLR-3989 add tenant_id to API logging

* chore(API, Backend): PRWLR-3989 add TODOs

* test(API): PRWLR-3989 add new middleware unit tests

* chore(API): PRWLR-3989 refactor RLS code

* fix(tests): PRWLR-3989 fix testing db connector

* chore: PRWLR-3989 add references to JIRA tickets

* fix: PRWLR-3989 remove bypass logic and fix serializers

* fix: PRWLR-3989 improve drop SQL query for RLS models

* feat(Backend): PRWLR-3989 add specific permissions on each model

* fix(Backend): PRWLR-3989 fix database routing and grant select perms

* fix(test): PRWLR-3989 fix routing issues with unit tests

* chore: PRWLR-3989 remove references to JIRA tickets

* chore(Backend): PRWLR-3989 adjust privileges for user on table tenant

* chore: PRWLR-3989 add comments on migrations
2024-08-02 11:29:05 -04:00
Jon Young 6a135cb47c PRWLR-4236: fix(CI): use postgres for unit tests (#19)
* fix(Config): use local postgres for unit tests

* fix(CI): use postgres for unit tests in GHA
2024-07-25 10:00:58 +02:00
Víctor Fernández Poyatos de97b9f298 feat/PRWLR-3999 Add providers endpoint with mock data (#17)
* chore: PRWLR-3996 add drf-jsonapi library to project

* feat: PRWLR-3996 implement /tenants endpoint and basic JSON:API spec

* feat: PRWLR-3996 implement new middleware to ingest metadata on API responses

* chore: PRWLR-3996 annotate views with drf-spectacular

* feat: PRWLR-3996 add cache-control to tenants endpoints

* feat: PRWLR-3996 update views filters

* chore: PRWLR-3996 remove meta middleware in favour of meta fields in serializers

* test(API): PRWLR-3996 add unit tests for /tenants

* chore: PRWLR-3996 turn DEBUG on on devel environment

* test: PRWLR-3996 rework tenants unit tests

* chore: PRWLR-3996 refactor filter classes

* fix(Backend,API): PRWLR-3996 use correct filter backend

* feat(API): PRWLR-3999 implement mocked /providers endpoints

* feat(API): PRWLR-3999 add /providers/check_connection mocked endpoint and loc headers
2024-07-23 12:34:48 -04:00
Víctor Fernández Poyatos 7cb3a4e16e feat/PRWLR-3996 implement /tenants endpoint and basic filters logic (#14)
* chore: PRWLR-3996 add drf-jsonapi library to project

* feat: PRWLR-3996 implement /tenants endpoint and basic JSON:API spec

* feat: PRWLR-3996 implement new middleware to ingest metadata on API responses

* chore: PRWLR-3996 annotate views with drf-spectacular

* feat: PRWLR-3996 add cache-control to tenants endpoints

* feat: PRWLR-3996 update views filters

* chore: PRWLR-3996 remove meta middleware in favour of meta fields in serializers

* test(API): PRWLR-3996 add unit tests for /tenants

* chore: PRWLR-3996 turn DEBUG on on devel environment

* test: PRWLR-3996 rework tenants unit tests

* chore: PRWLR-3996 refactor filter classes

* fix(Backend,API): PRWLR-3996 use correct filter backend
2024-07-22 16:06:57 -04:00
Jon Young 43f79663d9 chore(CODEOWNERS): Update for team name change (#16)
* chore(CODEOWNERS): update CODEOWNERS

* chore(CODEOWNERS): update CODEOWNERS

* chore(CODEOWNERS): update CODEOWNERS
2024-07-22 08:43:50 +02:00
Jon Young cd7d7c303a PRWLR-3997: fix(Models): create BaseModel & TenantModel (#13)
* fix(Models): create BaseModel & TenantModel

* update description to match class

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>

* fix(Tenants): rename TenantModel to Tenant

---------

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2024-07-16 15:44:59 +02:00
Víctor Fernández Poyatos 8f41b38bbf chore: PRWLR-4055 Update project structure and add env vars (#12)
* chore: PRWLR-4055 update project structure and add env vars

* chore: PRWLR-4055 make production settings default

* chore: PRWLR-4055 change default values to env variables

* chore: PRWLR-4055 apply requested changes on default values

* chore: PRWLR-4055 adapt testing environment
2024-07-16 09:33:56 -04:00
Víctor Fernández Poyatos 9df759da60 feat: PRWLR-3989 add PostgreSQL to project (#10)
* feat: PRWLR-3989 add postgresql to project

* chore: PRWLR-3989 add requested changes to docker structure

* chore: PRWLR-3989 update poetry.lock
2024-07-11 10:39:09 -04:00
dependabot[bot] 157a54f30c chore(deps): bump django from 5.0.6 to 5.0.7 (#11)
Bumps [django](https://github.com/django/django) from 5.0.6 to 5.0.7.
- [Commits](https://github.com/django/django/compare/5.0.6...5.0.7)

---
updated-dependencies:
- dependency-name: django
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-11 10:34:41 +02:00
Víctor Fernández Poyatos f0a276773e feat(backend): PRWLR-4030 Implement Gunicorn as Django WSGI (#9)
* feat: PRWLR-4030 implement Gunicorn server and logging

* build: PRWLR-4030 update docker files

* docs: PRWLR-4030 update README

* chore: PRWLR-4030 change binding address to 0.0.0.0 by default
2024-07-10 16:32:56 +02:00
Víctor Fernández Poyatos 457801f752 chore: update CODEOWNERS to include restful-api team (#8) 2024-07-08 13:09:04 +02:00
Víctor Fernández Poyatos 3a502c5b3d feat(Backend): PRWLR-3990 Add custom logging (#6)
* chore: PRWLR-4022 add django-cors-headers

* chore: PRWLR-4022 configure basic CORS

* feat: PRWLR-4022 add drf-spectacular and spec/docs endpoints

* build: PRWLR-4022 add basic docker development deployment

* chore: PRWLR-4022 update schema generation

* chore: PRWLR-4022 bump required Python version to 3.12

* feat: PRWLR-3990 add basic custom Django logging

* feat: PRWLR-3990 add tasks logger

* chore: PRWLR-3990 update API logging middleware to follow updated syntax

* docs: PRWLR-3990 add docstring for APILoggingMiddleware class

* chore: PRWLR-3990 add testing dependencies to poetry

* test(API): PRWLR-3990 add new middleware unit tests and base setup

* feat(API, Backend): PRWLR-3990 add NDJSONFormatter and transaction_id

* feat(API): PRWLR-3990 add dev/test model, views and urls

* chore(backend): PRWLR-3990 refactor logging settings and modules

* chore(backend): PRWLR-3990 add BackendLogger enum and refactor logging modules

* chore(backend): PRWLR-3990 refactor module importing

* style(backend): PRWLR-3990 remove extra endlines

* chore: PRWLR-3990 update poetry.lock

* chore: PRWLR-3990 run pre-commit
2024-07-05 10:13:04 -04:00
Jon Young 30775373dc chore(CI): add labeler definition (#7) 2024-07-05 09:30:24 -04:00
Jon Young 7e194407f6 chore(CI): PRWLR-3897: add Continuous Integration workflows (#3)
* chore(CI): import pre-commit-config.yaml from prowler-cloud/prowler

* chore(CI): import pre-commit-config.yaml from prowler-cloud/prowler

* chore(CI): add safety package to pre-commit

* chore(CI): use poetry run to access venv for local checks

* docs(repo): add community focused docs

* chore(GitHub): add ISSUE_TEMPLATEs

same as prowler-cloud/prowler repo

* chore(CI): add pull-request.yml workflow

runs on PRs and pushes to main

* chore(CI): add additional workflows

same as those on prowler-cloud/prowler

* fix(CI): use correct path

* fix(API): fix flake8 F401 failures

* fix(API, Backend): format with black

* fix( Backend): format with black

* fix(CI): use correct path

* fix(CI): move pyproject.toml

* fix(CI): simplify tools with ruff

* fix(CI): use ruff instead of flake8, black

* fix(CI): change ruff commands for GHA

* fix(CI): use correct ruff command name

* fix(Backend): ruff format

* fix(CI): ignore safety jinja2 vuln id 70612

and add mypy for future usage

* fix(CI): set file path for hadolint

* fix(CI): add dockle

* fix(CI): update dockle version

* fix(CI): fix hadolint errors

* fix(CI): add simple tests for CI

* fix(Docker): multi-stage docker build

* docs(README): update docs for poetry location
2024-07-05 09:01:02 -04:00
Víctor Fernández Poyatos a69a155679 docs(backend): PRWLR-3988 Include Django development guide (#5)
* docs: PRWLR-3988 add development guide to README

* docs: PRWLR-3988 add description to README
2024-07-03 11:45:54 -04:00
Víctor Fernández Poyatos 7ff6d860ce feat(backend): PRWLR-4022 Basic Django and DRF setup and dev tools (#4)
* chore: PRWLR-4022 add django-cors-headers

* chore: PRWLR-4022 configure basic CORS

* feat: PRWLR-4022 add drf-spectacular and spec/docs endpoints

* build: PRWLR-4022 add basic docker development deployment

* chore: PRWLR-4022 update schema generation

* chore: PRWLR-4022 bump required Python version to 3.12
2024-07-03 09:34:29 -04:00
dependabot[bot] b440be717c chore(deps): bump djangorestframework from 3.15.1 to 3.15.2 in /src (#2)
Bumps [djangorestframework](https://github.com/encode/django-rest-framework) from 3.15.1 to 3.15.2.
- [Release notes](https://github.com/encode/django-rest-framework/releases)
- [Commits](https://github.com/encode/django-rest-framework/compare/3.15.1...3.15.2)

---
updated-dependencies:
- dependency-name: djangorestframework
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-02 10:00:59 +02:00
Jon Young d8752719c1 Merge pull request #1 from prowler-cloud/PRWLR-3980-project-setup
feat: PRWLR-3980 Setup base project structure
2024-07-01 12:20:20 -04:00
Víctor Fernández Poyatos 737a0ff9cb chore: update .gitignore 2024-07-01 18:04:13 +02:00
Víctor Fernández Poyatos 1c8e676822 chore: update CODEOWNERS 2024-07-01 18:03:27 +02:00
Víctor Fernández Poyatos 7b98f0fc92 chore: update pull request template 2024-07-01 18:00:47 +02:00
Víctor Fernández Poyatos 45865f2e71 chore: add required files for project base structure 2024-07-01 17:43:05 +02:00
Víctor Fernández Poyatos eded2df687 chore: add .github folder and related files 2024-07-01 17:42:29 +02:00
Víctor Fernández Poyatos 766d3f6670 chore: PRWLR-3980 add Django project and API 2024-07-01 17:42:03 +02:00
Víctor Fernández Poyatos 3f2d0a13af chore: PRWLR-3980 add poetry files 2024-07-01 17:41:18 +02:00
Víctor Fernández Poyatos 690957e1c3 chore: PRWLR-3980 update .gitignore 2024-07-01 17:19:19 +02:00
Jon Young 3092b56fd6 Initial commit 2024-07-01 10:42:55 -04:00