mirror of
https://github.com/prowler-cloud/prowler.git
synced 2026-10-25 12:54:18 +00:00
test(mcp): add test foundation for the MCP server (#12291)
This commit is contained in:
1 parent
c74eac1369
commit
138d643119
32 files changed
+2332
-22
No files matched your search
@@ -426,6 +426,150 @@ For complete installation and deployment options, see:
|
||||
|
||||
For development I recommend to use the [Model Context Protocol Inspector](https://github.com/modelcontextprotocol/inspector) as MCP client to test and debug your tools.
|
||||
|
||||
## Testing
|
||||
|
||||
Tests live in `mcp_server/tests/`, mirroring the source tree, and use the `test_*.py`
|
||||
prefix (the same convention as the API, not the SDK's `*_test.py` suffix).
|
||||
|
||||
From `mcp_server/`:
|
||||
|
||||
```bash
|
||||
cd mcp_server
|
||||
|
||||
uv run pytest # Whole suite
|
||||
uv run pytest tests/prowler_app/models # One area
|
||||
uv run pytest --cov=./prowler_mcp_server # With coverage
|
||||
```
|
||||
|
||||
From the repository root:
|
||||
|
||||
```bash
|
||||
make test-mcp # Runs the MCP suite exactly as CI does
|
||||
```
|
||||
|
||||
Async tests need no marker — `asyncio_mode` is set to `auto`.
|
||||
|
||||
### Reading the Coverage Numbers
|
||||
|
||||
<Warning>
|
||||
Coverage here has a high floor that means nothing. `coverage.py` measures
|
||||
*statements*, and in a Pydantic model module nearly every statement is a class-body
|
||||
field declaration that runs at **import** time. `prowler_app/server.py` imports
|
||||
every tool module — and therefore every model module — when it is first imported,
|
||||
so all of those declarations execute and count as covered before a single test runs.
|
||||
|
||||
Importing the package and executing no tests at all already reports **36% overall**,
|
||||
with individual model modules between 54% and 84%. A model module sitting at ~68%
|
||||
with no tests written for it has **none** of its behaviour covered: the covered lines
|
||||
are its imports, `class` statements and `Field(...)` declarations, and the missing
|
||||
ranges are its `from_api_response()` bodies.
|
||||
|
||||
Judge a module against that import-only floor, not against zero, and do not set a
|
||||
Codecov target from the raw total.
|
||||
</Warning>
|
||||
|
||||
### Shared Fixtures
|
||||
|
||||
All fixtures live in `mcp_server/tests/conftest.py`. Three are autouse and apply to
|
||||
every test: the environment is pinned to deterministic values, real socket
|
||||
connections are blocked, and the API client singleton registry is snapshotted and
|
||||
restored.
|
||||
|
||||
| Fixture | What it gives you |
|
||||
|---------|-------------------|
|
||||
| `mock_api_client` | The API client singleton with its transport mocked. The workhorse. |
|
||||
| `mock_router` | Route registry and request recorder |
|
||||
| `mcp_root_server` | The mounted root server, for in-memory client tests |
|
||||
| `health_client` | Starlette `TestClient` for the `/health` route |
|
||||
| `http_request_headers` | Injects request headers for HTTP-transport auth tests |
|
||||
| `hub_router` / `docs_router` | Mock the Hub and Docs sub-servers' sync HTTP clients |
|
||||
| `api_client` / `isolated_api_client` | The live singleton / a freshly-constructed one |
|
||||
|
||||
Helpers live in `mcp_server/tests/helpers/`: JSON:API document builders
|
||||
(`jsonapi.py`), the `MockRouter` (`http.py`), tool-contract assertions
|
||||
(`assertions.py`) and fake credentials (`tokens.py`).
|
||||
|
||||
### Writing a Tool Test
|
||||
|
||||
Drive tools through an in-memory MCP client, and open the client inside the test —
|
||||
FastMCP warns that holding a client in a fixture causes event-loop problems.
|
||||
|
||||
```python
|
||||
from fastmcp import Client
|
||||
|
||||
from tests.helpers.jsonapi import jsonapi_collection, jsonapi_resource
|
||||
|
||||
FINDING_ATTRIBUTES = {
|
||||
"uid": "prowler-aws-s3_bucket_public_access-123456789012-us-east-1-my-bucket",
|
||||
"status": "FAIL",
|
||||
"severity": "high",
|
||||
"status_extended": "S3 bucket my-bucket is publicly accessible.",
|
||||
"delta": "new",
|
||||
"muted": False,
|
||||
"muted_reason": None,
|
||||
"check_metadata": {"checkid": "s3_bucket_public_access"},
|
||||
}
|
||||
|
||||
|
||||
async def test_search_without_dates_queries_the_latest_scan_endpoint(
|
||||
mcp_root_server, mock_api_client, mock_router
|
||||
):
|
||||
"""With no date range the tool targets the cheaper `/findings/latest`."""
|
||||
mock_router.add(
|
||||
"GET",
|
||||
"/api/v1/findings/latest",
|
||||
json=jsonapi_collection(
|
||||
[jsonapi_resource("findings", "f1", FINDING_ATTRIBUTES)]
|
||||
),
|
||||
)
|
||||
|
||||
async with Client(mcp_root_server) as client:
|
||||
result = await client.call_tool("prowler_search_security_findings", {})
|
||||
|
||||
assert result.data["findings"][0]["check_id"] == "s3_bucket_public_access"
|
||||
assert mock_router.paths() == ["GET /api/v1/findings/latest"]
|
||||
```
|
||||
|
||||
The exemplar suite covers `findings` end to end — `tests/prowler_app/models/test_findings.py`
|
||||
and `tests/prowler_app/tools/test_findings.py`. It is deliberately one feature
|
||||
across both layers rather than a scattering of unrelated samples, and `findings`
|
||||
is the feature that exercises the whole foundation: two-tier models, nested
|
||||
sub-models, both relationship shapes, endpoint switching on a date range,
|
||||
list-to-CSV filter encoding, and a tool that returns prose instead of a model.
|
||||
|
||||
Note the two files share a name. That is why `__init__.py` is required in every
|
||||
`tests/` subdirectory here — without it they would collide on import.
|
||||
|
||||
<Warning>
|
||||
Tool parameters are declared with pydantic `Field(default=...)`, and only FastMCP's
|
||||
tool wrapper resolves those defaults. Calling a tool method directly with an
|
||||
argument omitted leaves it as a raw `FieldInfo` object, which is truthy — so a
|
||||
filter such as `if email:` silently builds a query out of the `FieldInfo` repr.
|
||||
Call tools through the client, or pass every argument explicitly.
|
||||
</Warning>
|
||||
|
||||
### Why the API Key Is Pinned, Not Stripped
|
||||
|
||||
`prowler_app/server.py` builds every tool at import time. Constructing a tool
|
||||
reaches `ProwlerAppAuth`, which raises when `PROWLER_API_KEY` is missing, and
|
||||
`load_all_tools` swallows that error per tool class. The result is that the whole
|
||||
`prowler_*` namespace registers **zero** tools while the server still logs
|
||||
"Successfully mounted Prowler tools server".
|
||||
|
||||
The suite therefore pins a fake key in `[tool.pytest_env]`, which is applied before
|
||||
any test module is imported, and `tests/test_server.py` asserts each namespace is
|
||||
non-empty so this failure can never return silently.
|
||||
|
||||
<Note>
|
||||
`ProwlerAppAuth` resolves `PROWLER_MCP_TRANSPORT_MODE` and `API_BASE_URL` in its
|
||||
default arguments, which Python evaluates once at module import. `monkeypatch.setenv`
|
||||
cannot change them — pass `mode=` and `base_url=` explicitly in auth tests.
|
||||
</Note>
|
||||
|
||||
For the full set of rules and templates, see the
|
||||
[`prowler-test-mcp` skill](https://github.com/prowler-cloud/prowler/blob/master/skills/prowler-test-mcp/SKILL.md)
|
||||
and the [official FastMCP testing guide](https://gofastmcp.com/development/tests).
|
||||
|
||||
## Related Documentation
|
||||
|
||||
<CardGroup cols={2}>
|
||||
|
||||
Reference in new issue
Block a user