feat(Findings): Partitioned database tables (#45)

* feat(Findings): initial findings model

* fix(Findings): add view, serializers, migration for enums

* fix(Findings): incomplete jsonb_to_tsvector wrapper

will not run as written

* fix(Findings): use Severity and Status enums from prowler SDK

* tests(Findings): add failing view tests

* fix(Finding): add resource relationship

not returning correct data from serializer, missing links

* fix(FindingSerializer): get Scan & Resource relationships to show up

* fix(FindingFilter): add more filter fields

* fix(FindingFilter): filter on provider id

* fix(FindingSerializer): return Resource in relationship

not ResourceFindingMapping

* fix(FindingModel): update migration

* fix(FindingFilter): full text search on findings

* fix(Resources): include Findings in ResourceSerializer

* fix(FindingFilter): expand text search columns

* fix(DbUtils): docstring, not comment

* fix(BaseViews): remove TODO

comment not applicable right now

* fix(Fixtures): add more findings to fixture file

and change on_delete behavior for resource_finding_mapping

* fix(Resources): rename index to match others

* fix(Findings): update Findigns RLS to allow for full CRUD

eventually we'll let users enter a manual finding
which implies INSERT, UPDATE, DELETE

* fix(Findings): use TextChoices directly for Status enum

* fix(FindingSerializer): build a set instead of a list

* consistency in fixtures

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>

* fix(API): update v1 spec for findings

* feat(Findings): store findings in a partitioned table

* fix(Settings): change unit of partition max age

to match unit when creating

* docs(Partitions): document how to manage partitions via manage.py

* fix(Findings): add tag for spec/docs

* fix(Findings): clean up migrations

* fix(Findings): convert scan_id and inserted_at filters into finding.id filters

* fix(Findings): add default filter for today

and set default sort order

* fix(Findings): add default filter for today

and set default sort order

* fix(Findings): update fixtures so datetime matches id

* fix(Findings): partition the ResourceFindingMapping table to match Findings

* docs(Partitions): document postgres config values more

* docs(UUIDUtils): do not use raw query strigns (typo)

* docs(Partitions): change unit in comment description

* fix(Findings): change resource_name & tags to be Finding

* docs(Partitions): change unit in partitions settings docstring

* fix(Findings): remove conflicting logic & filters

* chore: apply suggested changes

* chore: optimize imports

---------

Co-authored-by: Víctor Fernández Poyatos <victor@prowler.com>
This commit is contained in:
Jon YoungandVíctor Fernández Poyatos authored and GitHub committed 2024-09-23 11:39:03 +02:00
1 parent 489ac20141
commit 4ca95b08e2
23 files changed
+816 -94

No files matched your search

+3 -4
View File
@@ -3,9 +3,8 @@ import json
from django.contrib.auth.password_validation import validate_password
from drf_spectacular.utils import extend_schema_field
from rest_framework_json_api import serializers
from rest_framework_json_api.serializers import ValidationError
from rest_framework_json_api.relations import SerializerMethodResourceRelatedField
from rest_framework_json_api.serializers import ValidationError
from api.models import (
StateChoices,
@@ -438,7 +437,7 @@ class ResourceSerializer(RLSSerializer):
return fields
class FindingVisbilityEnumSerializerField(serializers.ChoiceField):
class FindingDeltaEnumSerializerField(serializers.ChoiceField):
def __init__(self, **kwargs):
kwargs["choices"] = Finding.DeltaChoices.choices
super().__init__(**kwargs)
@@ -498,7 +497,7 @@ class FindingSerializer(RLSSerializer):
}
class JSONAPIMeta:
resource_name = "findings"
resource_name = "Findings"
def get_resources(self, obj):
mappings = ResourceFindingMapping.objects.filter(finding=obj)