Commit Graph

11 Commits

Author SHA1 Message Date
Víctor Fernández Poyatos 24857eaa7f feat/PRWLR-4002 add /scans endpoints (without SDK integration) (#33)
* feat(Backend): PRWLR-4992 add Scan db model

* feat(API): PRWLR-4002 add viewset and required tools for /scans endpoint

* feat(API): PRWLR-4002 improve /scans filters

* feat(API): PRWLR-4002 add relationships links

* feat(API): PRWLR-4002 implement POST /scan custom logic

* fix(API): PRWLR-4002 fix Scan.type keyword usage

* feat(API): PRWLR-4002 implement PATCH /scans

* feat(API): PRWLR-4002 refactor serializers for write operations

* fix(API): PRWLR-4002 fix providers on ScanSerializer

* test(API): PRWLR-4002 add unit tests for /scans

* feat(Backend, Test): PRWLR-4002 refactor routing and unit tests db connections

* build(CI): PRWLR-4002 update CI env vars for postgres

* fix(Tests): PRWLR-4002 fix close_db_connections fixture

* feat(Backend, API): PRWLR-4002 apply requested changes to Scan model and filters

* chore(Tests): PRWLR-4002 rename pytest fixtures

* feat(Backend): PRWLR-4002 remove unique constraint from Scan model

* fix(Backend, Tests): PRWLR-4002 fix db routing and migration mechanism for tests

* chore(deps): PRWLR-4002 add uuid6 to poetry deps

* chore(Backend): PRWLR-4002 refactor filter methods

* fix(Tests): PRWLR-4002 fix wrong postgres credentials for testing

* feat(API): PRWLR-4002 implement merging mechanism for Scans.scanner_args

* feat(API): PRWLR-4002 implement merging mechanism for Scans.scanner_args

* feat(Backend): PRWLR-4002 add indexes to Scan model

* feat(Backend): PRWLR-4002 remove id index from Scan model (redundant)

* feat(API): PRWLR-4002 add datetime fields to Scan serializer
2024-08-30 14:32:13 -04:00
Jon Young a2ab216531 chore(Celery): add basic Celery worker with broker only (#20)
* chore(Celery): add basic Celery worker with broker only

* chore(Celery): saving progress. Not able to schedule tasks

* fix(Celery): add celery app for use by django

* fix(Celery): register tasks

* fix(Docker): add celery workers to docker-compose

* chore(Celery): add django-celery-results backend

to store results using Django ORM

* fix(Celery): get app config the correct way

* fix(Docker): start connecting docker Celery workers to Valkey

not yet operational

* fix(Celery): get celery & django to work in docker-compose

* docs(Celery): document how to run Celery in development environment

includes changes to support the configuration and deployment
of Celery worker and its dependencies, Valkey and Postgres.

* fix(GHA): add valkey to CI services

* fix(GHA): add valkey to CI services

* fix(GHA): add valkey-cli ping to CI services

* fix(GHA): use right port for valkey

* fix(Views): remove debug task code

* test(Celery): start adding celery task tests

not yet working!

* fix(pyproject): rollback django upgrade

* fix(docker): updated docker runtime and env vars

based on feedback from #20

* fix(Dockerfile): include dependencies for psutil

psutil was introduced by pytest-celery

* fix(Backend): PRWLR-4013 fix celery settings structure

* fix(Celery): update celery app to work with new settings structure

* fix(Views): remove debug task code

* fix(Config): remove debug code

* fix(Celery): update celery app name when running worker

---------

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
2024-08-09 16:47:35 +02:00
Jon Young 6a135cb47c PRWLR-4236: fix(CI): use postgres for unit tests (#19)
* fix(Config): use local postgres for unit tests

* fix(CI): use postgres for unit tests in GHA
2024-07-25 10:00:58 +02:00
Jon Young 43f79663d9 chore(CODEOWNERS): Update for team name change (#16)
* chore(CODEOWNERS): update CODEOWNERS

* chore(CODEOWNERS): update CODEOWNERS

* chore(CODEOWNERS): update CODEOWNERS
2024-07-22 08:43:50 +02:00
Víctor Fernández Poyatos 457801f752 chore: update CODEOWNERS to include restful-api team (#8) 2024-07-08 13:09:04 +02:00
Víctor Fernández Poyatos 3a502c5b3d feat(Backend): PRWLR-3990 Add custom logging (#6)
* chore: PRWLR-4022 add django-cors-headers

* chore: PRWLR-4022 configure basic CORS

* feat: PRWLR-4022 add drf-spectacular and spec/docs endpoints

* build: PRWLR-4022 add basic docker development deployment

* chore: PRWLR-4022 update schema generation

* chore: PRWLR-4022 bump required Python version to 3.12

* feat: PRWLR-3990 add basic custom Django logging

* feat: PRWLR-3990 add tasks logger

* chore: PRWLR-3990 update API logging middleware to follow updated syntax

* docs: PRWLR-3990 add docstring for APILoggingMiddleware class

* chore: PRWLR-3990 add testing dependencies to poetry

* test(API): PRWLR-3990 add new middleware unit tests and base setup

* feat(API, Backend): PRWLR-3990 add NDJSONFormatter and transaction_id

* feat(API): PRWLR-3990 add dev/test model, views and urls

* chore(backend): PRWLR-3990 refactor logging settings and modules

* chore(backend): PRWLR-3990 add BackendLogger enum and refactor logging modules

* chore(backend): PRWLR-3990 refactor module importing

* style(backend): PRWLR-3990 remove extra endlines

* chore: PRWLR-3990 update poetry.lock

* chore: PRWLR-3990 run pre-commit
2024-07-05 10:13:04 -04:00
Jon Young 30775373dc chore(CI): add labeler definition (#7) 2024-07-05 09:30:24 -04:00
Jon Young 7e194407f6 chore(CI): PRWLR-3897: add Continuous Integration workflows (#3)
* chore(CI): import pre-commit-config.yaml from prowler-cloud/prowler

* chore(CI): import pre-commit-config.yaml from prowler-cloud/prowler

* chore(CI): add safety package to pre-commit

* chore(CI): use poetry run to access venv for local checks

* docs(repo): add community focused docs

* chore(GitHub): add ISSUE_TEMPLATEs

same as prowler-cloud/prowler repo

* chore(CI): add pull-request.yml workflow

runs on PRs and pushes to main

* chore(CI): add additional workflows

same as those on prowler-cloud/prowler

* fix(CI): use correct path

* fix(API): fix flake8 F401 failures

* fix(API, Backend): format with black

* fix( Backend): format with black

* fix(CI): use correct path

* fix(CI): move pyproject.toml

* fix(CI): simplify tools with ruff

* fix(CI): use ruff instead of flake8, black

* fix(CI): change ruff commands for GHA

* fix(CI): use correct ruff command name

* fix(Backend): ruff format

* fix(CI): ignore safety jinja2 vuln id 70612

and add mypy for future usage

* fix(CI): set file path for hadolint

* fix(CI): add dockle

* fix(CI): update dockle version

* fix(CI): fix hadolint errors

* fix(CI): add simple tests for CI

* fix(Docker): multi-stage docker build

* docs(README): update docs for poetry location
2024-07-05 09:01:02 -04:00
Víctor Fernández Poyatos 1c8e676822 chore: update CODEOWNERS 2024-07-01 18:03:27 +02:00
Víctor Fernández Poyatos 7b98f0fc92 chore: update pull request template 2024-07-01 18:00:47 +02:00
Víctor Fernández Poyatos eded2df687 chore: add .github folder and related files 2024-07-01 17:42:29 +02:00