Compare commits

...
Author SHA1 Message Date
Rubén De la Torre Vico cbc5f58998 chore(lighthouse): change wording in new MCP Server class 2026-08-13 15:41:52 +02:00
Rubén De la Torre Vico 975c3f082d feat(mcp): report every tool failure as an MCP error
Adds ProwlerMCP, the FastMCP subclass every sub-server is now built
from. Its tool() wraps whatever it registers -- the decorator forms and
the direct call BaseTool uses -- so a failure leaves any tool as a
ToolError, which the client reads as isError: true.

Applied at the base class rather than by hand because forgetting it is
silent: every server now sets mask_error_details=True, so an unwrapped
tool would answer "Error calling tool 'x'" and nothing else. ToolError
bypasses that masking, which is what lets the servers mask by default
and still say something useful.

No tool changes yet. Tools that still return {"error": ...} keep working
exactly as before; they are converted surface by surface in the PRs
above this one. What changes here is that a failure which used to escape
as a raw exception is now described by render_tool_error.

The rules this establishes are in AGENTS.md and the developer guide,
so the conversions have something to be checked against.
2026-08-12 13:26:39 +02:00
Rubén De la Torre Vico c37e2b90dc feat(mcp): add a single vocabulary for tool failures
Adds lib/errors.py: the exception types the sub-servers raise for a
failure they can describe -- an API rejection, a task that never
finished, bad credentials, a Hub outage -- and render_tool_error, the
one place any of them becomes text a client reads.

Nothing imports it yet. The wrapper that turns these into an MCP error
on every tool, and the servers that raise them, follow in this stack.
2026-08-12 13:16:38 +02:00
13 changed files with 1040 additions and 25 deletions
+101 -13
View File
@@ -44,6 +44,11 @@ The main server orchestrates three sub-servers with prefixed namespacing:
mcp_server/prowler_mcp_server/
├── server.py # Main orchestrator
├── main.py # CLI entry point
├── lib/
│ ├── server.py # ProwlerMCP, the base class of every sub-server
│ ├── errors.py # Exception types and the one error renderer
│ ├── logger.py
│ └── analytics.py
├── prowler_hub/
├── prowler_app/
│ ├── tools/ # Tool implementations
@@ -59,6 +64,8 @@ The MCP Server uses two patterns for tool registration:
1. **Direct Decorators** (Prowler Hub/Docs): Tools are registered using `@mcp.tool()` decorators
2. **Auto-Discovery** (`prowler_app`): All public methods of `BaseTool` subclasses are auto-registered
Both funnel through `ProwlerMCP.tool` (`lib/server.py`), which is what applies the error contract to every tool no matter how it was registered. Build sub-servers with `ProwlerMCP`, never `FastMCP` directly.
## Adding Tools to the `prowler_app` Sub-Server
### Step 1: Create the Tool Class
@@ -120,12 +127,10 @@ class NewFeatureTools(BaseTool):
Returns complete feature details including configuration and metadata.
"""
try:
response = await self.api_client.get(f"/api/v1/features/{feature_id}")
return DetailedFeature.from_api_response(response["data"]).model_dump()
except Exception as e:
self.logger.error(f"Failed to get feature {feature_id}: {e}")
return {"error": str(e), "status": "failed"}
# No try/except: a failure here raises, and the tool wrapper turns it into a
# ToolError the client sees as `isError: true`. See "Error Handling" below.
response = await self.api_client.get(f"/api/v1/features/{feature_id}")
return DetailedFeature.from_api_response(response["data"]).model_dump()
```
### Step 2: Create the Models
@@ -369,18 +374,101 @@ async def search_items(self, status: str = Field(...)) -> dict:
### Error Handling
Return structured error responses instead of raising exceptions:
Let failures raise. Every sub-server is a `ProwlerMCP` (`prowler_mcp_server/lib/server.py`),
whose `tool()` wraps whatever it registers in `tool_errors`, turning any exception into a
`ToolError`. The client sees `isError: true` and a message it can act on.
That wrapping is not something you apply — the two registration styles (the `@mcp.tool()`
decorators, and the direct `mcp.tool(fn)` call `BaseTool` uses) both funnel through
`ProwlerMCP.tool`. Build sub-servers with `ProwlerMCP`, never `FastMCP` directly: masking
is on everywhere, so a tool that escaped the funnel would answer `Error calling tool 'x'`
with no detail at all.
Never `return {"error": ...}`: a returned payload is `isError: false` at the MCP protocol
level, so the client is told the call succeeded and only finds out otherwise if it happens
to inspect the right key.
```python
async def get_item(self, item_id: str) -> dict:
try:
response = await self.api_client.get(f"/api/v1/items/{item_id}")
return DetailedItem.from_api_response(response["data"]).model_dump()
except Exception as e:
self.logger.error(f"Failed to get item {item_id}: {e}")
return {"error": str(e), "status": "failed"}
"""A rejected request, a timeout and a malformed payload all raise from here.
Each is rendered with the API's own words plus what it implies about retrying.
"""
response = await self.api_client.get(f"/api/v1/items/{item_id}")
return DetailedItem.from_api_response(response["data"]).model_dump()
```
Raise `ToolError` whenever the message is one you wrote for the caller. Its text reaches
the client verbatim, so anything they need in order to recover has to be *in* the message
— an error carries nothing else:
```python
from fastmcp.exceptions import ToolError
if not data:
raise ToolError(
f"Item '{item_id}' was not found. Use prowler_list_items to find valid IDs."
)
```
**Do not raise `ValueError` from a tool.** The two are not interchangeable: anything that
is not a `ToolError` is described as a bug in this server. That is right for a model
factory rejecting an API payload or a pydantic `ValidationError`, and wrong for a
refusal — so the exception type is what carries the distinction:
```text
Date range cannot exceed 2 days. Requested range: 2025-01-01 to 2025-01-10 (10 days)
The Prowler MCP Server hit an unexpected ValueError: Missing pagination metadata in API
response. This is a bug in the server, not something you can fix by changing the
arguments.
```
If you surface an exception yourself — into a `ToolError` you build, or into a field of a
structured result — pass it through `render_tool_error(e)` rather than `str(e)`, so the
same failure is never described two ways. Pass `warn=False` when the result already
reports the outcome.
#### Deciding between an error and a result
Ask two questions, in order:
1. **Did the tool finish its own job?** `test_integration_connection`'s job is to run the
check and report what happened, so `connected: false` is the job finished.
`get_finding_details`' job is to return the finding, so no finding means it did not.
2. **Is the reported state a fact about the remote world or about our call?** The world
(Jira refused the credentials, 3 of 40 items failed, a discovery found nothing) is a
**result**. Our call (403, connection reset, invalid UUID, a bug in a model factory) is
an **error**.
One rule overrides both: **if a write may have partially landed, that fact travels in a
successful structured result, never in an error.** An agent reads `isError: true` as
"nothing happened, safe to retry"; reporting "I may have created 17 Jira issues" that way
invites a duplicate dispatch.
#### What the client reads
`render_tool_error` describes the failure in one plain sentence: the call, the status and
whatever the API said, with the field named when it named one.
```text
GET /findings/b1ca536c failed with HTTP 404. No Finding matches the given query.
POST /integrations failed with HTTP 400. This field may not be blank. (/data/attributes/configuration/bucket_name); Enter a valid URL.
Date range cannot exceed 2 days. Requested range: 2025-01-01 to 2025-01-10 (10 days)
```
Nothing is added that the status code already implies. The one exception is a request that
could have changed something and never came back with a verdict — a 5xx or a timeout on a
write — which gets a warning, because an agent otherwise reads any failure as "nothing
happened" and sends the write again:
```text
DELETE /integrations/i1 failed with HTTP 500. A server error occurred. It may have been carried out anyway, so check the current state before retrying.
```
Every server sets `mask_error_details=True`. That costs nothing, because `ToolError`
bypasses masking; it only stops raw internals escaping from code paths outside a tool.
### Parameter Descriptions
Use Pydantic `Field()` with clear descriptions. This also helps LLMs understand
+38 -1
View File
@@ -26,6 +26,8 @@ The Prowler MCP Server provides AI agents access to the Prowler ecosystem throug
## CRITICAL RULES
### Tool Implementation
- ALWAYS: Build sub-servers with `ProwlerMCP`, never `FastMCP` directly. It is what
applies the error contract to every tool, whichever way it is registered
- ALWAYS: Extend `BaseTool` ABC for Prowler tools (auto-registration)
- ALWAYS: Use `@mcp.tool()` decorator for Hub/Docs tools
- NEVER: Manually register BaseTool subclasses
@@ -42,6 +44,37 @@ The Prowler MCP Server provides AI agents access to the Prowler ecosystem throug
- ALWAYS: Use `build_filter_params()` for query parameters
- NEVER: Create new httpx clients in tools
### Errors
One rule: **`ToolError` is a message you wrote for the caller. Any other exception is
a bug or an upstream failure**, and `render_tool_error` describes it.
- ALWAYS: `raise ToolError(...)` for anything the caller can act on — a rejected
argument, a lookup that found nothing, a workflow step they must do first. Its text
reaches the client verbatim, past `mask_error_details`
- NEVER: `raise ValueError(...)` in a tool. It is reported as a bug in this server,
which is correct for a model factory rejecting an API payload and wrong for a
refusal
- ALWAYS: Let an upstream failure propagate untouched. `ProwlerMCP.tool` wraps every
registration, so it becomes a `ToolError` describing the call, the status and what
the API said. There is nothing to remember to apply
- NEVER: `return {"error": ...}` or `{"success": False}`. A returned payload is
`isError: false`, so the client is told the call succeeded
- NEVER: Raise a plain exception *after* a write has been accepted. `ToolError` is the
only kind whose message reaches the client exactly as written
- ALWAYS: `render_tool_error(e)` when you surface an exception yourself, so a failure
is never described two different ways. Use `warn=False` when embedding it in a
result that already reports the outcome
- ALWAYS: Return a structured result, not an error, when a write may have partially
landed (`status="unknown"`, `deleted="unknown"`, `safe_to_retry=False`). An agent
reads `isError: true` as "nothing happened, safe to retry"
- ALWAYS: Return a structured result for an outcome that *is* the tool's job to
report: `connected: false`, an empty list, an idempotent no-op
- NEVER: Wrap a whole tool body in `except Exception`. It reports bugs in this
server as API failures, and the wrapper already handles the rest
See `prowler_mcp_server/lib/errors.py` and
`docs/developer-guide/mcp-server.mdx` for the message format.
---
## ARCHITECTURE
@@ -72,6 +105,9 @@ Python 3.12+ | FastMCP 3.4.4 | httpx (async) | Pydantic | uv | pytest
```text
mcp_server/prowler_mcp_server/
├── server.py # Main orchestration
├── lib/
│ ├── server.py # ProwlerMCP: base class of every sub-server
│ └── errors.py # Exception types + render_tool_error
├── prowler_hub/server.py # Hub tools (no auth)
├── prowler_app/
│ ├── server.py
@@ -113,7 +149,8 @@ make test-mcp # Run the MCP test suite exactly as CI does
- [ ] Models use `MinimalSerializerMixin`
- [ ] API responses transformed to simplified models
- [ ] No hardcoded secrets
- [ ] Error handling returns structured responses
- [ ] Failures raise (never `return {"error": ...}`); outcomes that may have changed
something return a structured result
- [ ] Parameter descriptions use Pydantic `Field()`
- [ ] Tests added under `mcp_server/tests/`, mirroring the source path below the
package root (`prowler_mcp_server/prowler_app/tools/` -> `tests/prowler_app/tools/`),
+347
View File
@@ -0,0 +1,347 @@
"""One way to fail: every tool failure reaches the client as a `ToolError`.
MCP draws a line this server used to blur. A tool that *returns* `{"error": ...}`
produces a successful result (`isError: false`) whose failure is only discoverable by
guessing which key to look at; a tool that *raises* produces `isError: true`, which
every client and model already understands as "this call did not work".
`ToolError` is a `FastMCPError`, and `FastMCP._call_tool` re-raises those untouched
(fastmcp/server/server.py:1241). So a message built here is what the client reads,
verbatim, past every mount and past `mask_error_details`. That is what lets the servers
mask by default while still telling the caller everything relevant.
`render_tool_error` is the single place an exception becomes that text, and
`tool_errors` is what makes sure no tool can escape it.
The exception types live here rather than next to the API client because the hub and the
documentation sub-servers must be able to raise and render them without taking a
dependency on `prowler_app`.
"""
from __future__ import annotations
import functools
import inspect
import json
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any
import httpx
from fastmcp.exceptions import ToolError
from prowler_mcp_server.lib.logger import logger
# Upstream bodies are not ours and may be large or HTML; enough to diagnose, not enough
# to flood the model's context.
_MAX_UPSTREAM_BODY = 500
# The one thing a status code does not say. Appended only when a request that could have
# changed something did not come back with a verdict, because an agent reads a failure as
# "nothing happened" and will happily send the write again.
_UNKNOWN_OUTCOME = (
" It may have been carried out anyway, so check the current state before retrying."
)
@dataclass(frozen=True, slots=True)
class ApiErrorDetail:
"""One entry of a JSON:API `errors` array.
The API answers a rejected write with one error *per invalid field*, each naming the
field in `source`. Keeping the whole shape is what turns "the request was invalid"
into "these two fields were invalid, and here is which".
Field names are JSON:API's own. `source.pointer` and `source.parameter` are not two
spellings of one thing: a pointer is a JSON Pointer into the request *document*
(`/data/attributes/provider_id`, as `alerts/errors.py` and `tasks/beat.py` send),
while a parameter is a *query parameter* name (`page[size]`, `lookback_days`, as
`api/v1/views.py` sends). The spec has a third, `source.header`; the Prowler API
never emits one, so there is nothing here to read it into.
"""
detail: str | None = None
title: str | None = None
pointer: str | None = None
"""JSON:API `source.pointer`: a JSON Pointer into the request document."""
parameter: str | None = None
"""JSON:API `source.parameter`: the query parameter that caused the error."""
@classmethod
def from_jsonapi(cls, error: dict[str, Any]) -> ApiErrorDetail:
"""Build from a single member of a JSON:API `errors` array.
`source` is optional and most errors omit it, so it supplies the location only,
never whether there is a detail worth reporting.
"""
source = error.get("source", {})
return cls(
detail=error.get("detail"),
title=error.get("title"),
pointer=source.get("pointer"),
parameter=source.get("parameter"),
)
def render(self) -> str:
"""The error text, and where the API said it is.
A pointer is left as-is because a leading `/` already reads as a path into the
body. A parameter is labelled, since `(page[size])` on its own would read like
one.
"""
text = self.detail or self.title or ""
if not text:
return ""
if self.pointer:
return f"{text} ({self.pointer})"
if self.parameter:
return f"{text} (parameter {self.parameter})"
return text
def parse_jsonapi_errors(payload: Any) -> tuple[ApiErrorDetail, ...]:
"""Extract every error from a JSON:API error document.
Tolerant on purpose: this runs while handling a failure, and a body that is not the
document it should be must not turn a useful API error into a parsing traceback.
"""
if not isinstance(payload, dict):
return ()
errors = payload.get("errors")
if not isinstance(errors, list):
return ()
return tuple(
ApiErrorDetail.from_jsonapi(error)
for error in errors
if isinstance(error, dict)
)
class ProwlerAPIError(Exception):
"""An error response returned by the Prowler API.
Raised only when the API answered with an error status, which tells a caller
something no plain exception can: the request reached Prowler and was
rejected, so it changed nothing. A timeout or a dropped connection stays a
bare exception because the request may well have been processed.
"""
def __init__(
self,
message: str,
status_code: int,
*,
method: str | None = None,
path: str | None = None,
errors: tuple[ApiErrorDetail, ...] = (),
) -> None:
super().__init__(message)
self.status_code: int = status_code
# Stored as plain strings so this module stays independent of the API client's
# HTTPMethod enum; StrEnum members compare equal to their value either way.
self.method: str | None = str(method) if method is not None else None
self.path: str | None = path
self.errors: tuple[ApiErrorDetail, ...] = tuple(errors)
@property
def rejected(self) -> bool:
"""The request reached Prowler and was refused, so it changed nothing."""
return 400 <= self.status_code < 500
class ProwlerTaskError(Exception):
"""A background task this server was waiting on did not complete.
Separate from `ProwlerAPIError` because the API already accepted the work: the
task exists and may still be running, so the outcome is unknown rather than refused.
"""
def __init__(self, message: str, *, task_id: str, state: str) -> None:
super().__init__(message)
self.task_id: str = task_id
self.state: str = state
"""One of `timeout`, `failed` or `cancelled`."""
class ProwlerAuthError(ValueError):
"""The credentials are missing, malformed or expired.
Subclasses `ValueError` so that the handlers which already treat an
authentication failure as a refusal-before-send keep working. It is matched by name
in `render_tool_error` rather than by that base class, so it is described as the
credential problem it is instead of falling through to the bug branch.
"""
class ProwlerHubError(Exception):
"""The Prowler Hub answered with an error status.
The Hub is a separate public service with its own client, so its failures cannot be
`ProwlerAPIError`. Everything the Hub exposes is a read.
"""
def __init__(
self,
message: str,
*,
status_code: int,
path: str,
body: str | None = None,
) -> None:
super().__init__(message)
self.status_code: int = status_code
self.path: str = path
self.body: str | None = body
def _upstream_detail(body: str | None) -> str:
"""A readable line from an error body that is not JSON:API.
The Prowler API answers with a JSON:API document, which is parsed into
`ApiErrorDetail`. Every other host this server talks to has its own shape: the Hub
answers `{"error": "Not found"}`, GitHub answers plain text, a proxy in between may
answer HTML. Relaying any of those verbatim puts braces and markup in front of the
model, so the message is pulled out when there is one and truncated when there is
not.
"""
if not body:
return ""
text = body.strip()
try:
parsed = json.loads(text)
except ValueError:
parsed = None
if isinstance(parsed, dict):
for key in ("error", "message", "detail"):
value = parsed.get(key)
if isinstance(value, str) and value:
return value
return f"{text[:_MAX_UPSTREAM_BODY]}..." if len(text) > _MAX_UPSTREAM_BODY else text
def render_tool_error(error: Exception, *, warn: bool = True) -> str:
"""Describe an exception in the plainest sentence that keeps every useful detail.
Callers that surface a failure from anywhere other than a raised exception -- a
message they write themselves, a field of a structured result -- go through here
too, which is what keeps one failure from being described two different ways.
Pass `warn=False` when the caller already states the outcome, which a structured
result reporting `status="unknown"` does by definition. Otherwise the generic
warning lands next to a more specific one saying the same thing.
Ordered most specific first: `ProwlerAuthError` is a `ValueError` and
`httpx.TimeoutException` is a `RequestError`, so the general branches come last.
"""
unknown = _UNKNOWN_OUTCOME if warn else ""
if isinstance(error, ProwlerAPIError):
operation = (
" ".join(p for p in (error.method, error.path) if p) or "The request"
)
details = "; ".join(text for text in (d.render() for d in error.errors) if text)
message = f"{operation} failed with HTTP {error.status_code}."
if details:
message = f"{message} {details}"
# A 4xx is a refusal, so it changed nothing and needs no warning.
if error.rejected or error.method == "GET":
return message
return message + unknown
if isinstance(error, ProwlerTaskError):
# The API accepted the work before the wait failed, so the outcome is open
# whichever way the task ended.
return f"{error}{unknown}"
if isinstance(error, ProwlerAuthError):
return f"Prowler authentication failed: {error}"
if isinstance(error, ProwlerHubError):
# Same shape as the API branch, with the service named because the Hub can be
# down while the API is fine. Everything the Hub exposes is a GET.
message = f"Prowler Hub GET {error.path} failed with HTTP {error.status_code}."
detail = _upstream_detail(error.body)
return f"{message} {detail}" if detail else message
if isinstance(error, httpx.HTTPStatusError):
# An upstream that is not the Prowler API, such as the external-URL fetch.
request = error.request
message = (
f"{request.method} {request.url} failed with HTTP "
f"{error.response.status_code}."
)
detail = _upstream_detail(error.response.text)
return f"{message} {detail}" if detail else message
if isinstance(error, httpx.RequestError):
# No answer at all: a timeout, a dropped connection, a DNS failure.
try:
operation = f"{error.request.method} {error.request.url}"
method = error.request.method
except RuntimeError:
# httpx only attaches the request once it has one, and reading it before
# then raises. Never let that hide the failure being reported.
operation, method = "The request", None
suffix = "" if method == "GET" else unknown
return f"{operation} got no answer ({type(error).__name__}: {error}).{suffix}"
# No `ValueError` branch, deliberately. A message written for the caller is raised
# as a `ToolError`, which never reaches here. What is left -- a model factory
# rejecting an API payload, a pydantic `ValidationError`, an `int()` on something
# that is not a number -- is this server or the API breaking its own contract, and
# saying so is the only useful thing to tell a caller who cannot fix it.
return (
f"The Prowler MCP Server hit an unexpected {type(error).__name__}: {error}. "
"This is a bug in the server, not something you can fix by changing the "
"arguments."
)
def tool_errors(fn: Callable[..., Any]) -> Callable[..., Any]:
"""Wrap a tool so that every failure leaves it as a `ToolError`.
Applied by `ProwlerMCP.tool()` rather than by hand, so no registration can miss
it. It wraps the callable handed to `mcp.tool()`, not the class attribute, so only
the MCP boundary is normalised: a tool calling another tool internally still sees
the real, typed exception and can branch on it.
Two constraints worth knowing before changing this:
- Never register the result with `exclude_args=`. That path
(fastmcp/utilities/types.py) rebuilds the function from `__code__`, which on a
wrapper is the wrapper's own. Nothing in this server passes it today.
- `inspect.iscoroutinefunction`, not the `asyncio` one, which is deprecated from
Python 3.14 and would be an error under this project's warning filters.
"""
name = getattr(fn, "__qualname__", repr(fn))
def mark(wrapper: Callable[..., Any]) -> Callable[..., Any]:
"""Flag the wrapper so a test can prove every registered tool went through it."""
wrapper.__prowler_tool_errors__ = True # ty: ignore[unresolved-attribute]
return wrapper
if inspect.iscoroutinefunction(fn):
@functools.wraps(fn)
async def async_wrapper(*args: Any, **kwargs: Any) -> Any:
try:
return await fn(*args, **kwargs)
except ToolError:
raise
except Exception as error:
logger.exception(f"Tool {name} failed: {error}")
raise ToolError(render_tool_error(error)) from error
return mark(async_wrapper)
@functools.wraps(fn)
def sync_wrapper(*args: Any, **kwargs: Any) -> Any:
try:
return fn(*args, **kwargs)
except ToolError:
raise
except Exception as error:
logger.exception(f"Tool {name} failed: {error}")
raise ToolError(render_tool_error(error)) from error
return mark(sync_wrapper)
@@ -0,0 +1,32 @@
"""The FastMCP subclass every Prowler sub-server is built from."""
from typing import Any
from fastmcp import FastMCP
from prowler_mcp_server.lib.errors import tool_errors
class ProwlerMCP(FastMCP):
"""A FastMCP server whose tools all report failures the same way.
`FastMCP.tool()` is the single funnel every registration goes through, the
`@server.tool()` and bare `@server.tool` decorator forms, and the direct
`mcp.tool(fn)` call `BaseTool` uses to auto-register, so applying
`tool_errors` here covers all of them at once.
"""
def tool(self, name_or_fn: Any = None, **kwargs: Any) -> Any:
"""Register a tool, wrapped so its failures reach the client as `ToolError`."""
if callable(name_or_fn):
# Direct call: mcp.tool(fn), or the bare @mcp.tool decorator.
return super().tool(tool_errors(name_or_fn), **kwargs)
# Parameterised decorator: @mcp.tool() or @mcp.tool(name="..."). FastMCP hands
# back the decorator that does the registering, so the wrap goes in front of it.
register = super().tool(name_or_fn, **kwargs)
def decorator(fn: Any) -> Any:
return register(tool_errors(fn))
return decorator
@@ -1,9 +1,8 @@
from fastmcp import FastMCP
from prowler_mcp_server.lib.server import ProwlerMCP
from prowler_mcp_server.prowler_app.utils.tool_loader import load_all_tools
# Initialize MCP server
app_mcp_server = FastMCP("prowler-app")
app_mcp_server = ProwlerMCP("prowler-app", mask_error_details=True)
# Auto-discover and load all tools from the tools package
load_all_tools(app_mcp_server)
@@ -72,6 +72,9 @@ class BaseTool(ABC):
async methods (not starting with '_') as tools. Subclasses do not need
to override this method.
Failures need no handling here: `ProwlerMCP.tool` wraps whatever it is
given, so every tool reports them the same way.
Args:
mcp: The FastMCP instance to register tools with
"""
@@ -1,14 +1,14 @@
from typing import Any
from fastmcp import FastMCP
from pydantic import Field
from prowler_mcp_server.lib.server import ProwlerMCP
from prowler_mcp_server.prowler_documentation.search_engine import (
ProwlerDocsSearchEngine,
)
# Initialize FastMCP server
docs_mcp_server = FastMCP("prowler-docs")
# Initialize MCP server
docs_mcp_server = ProwlerMCP("prowler-docs", mask_error_details=True)
prowler_docs_search_engine = ProwlerDocsSearchEngine()
@@ -5,13 +5,13 @@ Provides access to Prowler Hub API for security checks and compliance frameworks
"""
import httpx
from fastmcp import FastMCP
from pydantic import Field
from prowler_mcp_server import __version__
from prowler_mcp_server.lib.server import ProwlerMCP
# Initialize FastMCP for Prowler Hub
hub_mcp_server = FastMCP("prowler-hub")
# Initialize MCP server for Prowler Hub
hub_mcp_server = ProwlerMCP("prowler-hub", mask_error_details=True)
# API base URL
BASE_URL = "https://hub.prowler.com/api"
+2 -2
View File
@@ -1,10 +1,10 @@
from fastmcp import FastMCP
from starlette.responses import JSONResponse
from prowler_mcp_server import __version__
from prowler_mcp_server.lib.logger import logger
from prowler_mcp_server.lib.server import ProwlerMCP
prowler_mcp_server = FastMCP("prowler-mcp-server")
prowler_mcp_server = ProwlerMCP("prowler-mcp-server", mask_error_details=True)
def setup_main_server():
View File
+311
View File
@@ -0,0 +1,311 @@
"""Tests for the sentence a failure is described with.
These assert on the *text* a model reads, because that text is the whole contract: a
`ToolError` carries nothing else. What matters is that the API's own words survive
intact, and that a write whose outcome nobody can report says so.
How that sentence reaches a client -- and that no tool can escape it -- is
`test_server.py`.
"""
import httpx
import pytest
from prowler_mcp_server.lib.errors import (
ApiErrorDetail,
ProwlerAPIError,
ProwlerAuthError,
ProwlerHubError,
ProwlerTaskError,
parse_jsonapi_errors,
render_tool_error,
)
MAY_HAVE_LANDED = "It may have been carried out anyway"
# --------------------------------------------------------------------------- parsing
def test_every_error_of_the_document_is_preserved():
"""A rejected write names one error per invalid field; all of them matter."""
errors = parse_jsonapi_errors(
{
"errors": [
{"status": "400", "detail": "This field may not be blank."},
{"status": "400", "detail": "Enter a valid URL."},
]
}
)
assert [error.detail for error in errors] == [
"This field may not be blank.",
"Enter a valid URL.",
]
def test_a_query_parameter_error_is_not_dressed_up_as_a_body_path():
"""`source.parameter` and `source.pointer` are different places, per JSON:API.
The API sends a parameter for a bad query string (`api/v1/views.py` answers
`page[size]` and `lookback_days` that way) and a pointer for a bad body field.
Rendering a parameter bare would read as though `page[size]` were a path into the
document, which is somewhere the caller never put it.
"""
(error,) = parse_jsonapi_errors(
{
"errors": [
{
"detail": "invalid parameter 'page[size]'",
"source": {"parameter": "page[size]"},
}
]
}
)
assert error.parameter == "page[size]"
assert error.pointer is None
assert error.render() == "invalid parameter 'page[size]' (parameter page[size])"
def test_the_field_an_error_points_at_is_kept():
"""`source.pointer` is what turns "invalid" into "this field is invalid"."""
(error,) = parse_jsonapi_errors(
{
"errors": [
{
"detail": "This field may not be blank.",
"source": {"pointer": "/data/attributes/name"},
}
]
}
)
assert error.render() == ("This field may not be blank. (/data/attributes/name)")
def test_an_error_with_only_a_title_still_says_something():
"""`detail` is the useful field, but the API does not always send one."""
assert ApiErrorDetail(title="Not Found").render() == "Not Found"
@pytest.mark.parametrize(
"payload",
[None, "not a document", {}, {"errors": "not a list"}, {"errors": [None]}],
ids=["none", "text", "empty", "errors-not-a-list", "member-not-a-dict"],
)
def test_a_body_that_is_not_an_error_document_is_tolerated(payload):
"""Parsing runs while handling a failure; it must not become the failure."""
assert parse_jsonapi_errors(payload) == ()
def test_an_error_with_nothing_in_it_renders_empty():
"""Rendered to nothing rather than to punctuation, so composition can skip it."""
assert ApiErrorDetail().render() == ""
# --------------------------------------------------------------------- api failures
def test_a_failed_read_names_the_call_and_the_reason():
message = render_tool_error(
ProwlerAPIError(
"API request failed: 404 - Not found.",
404,
method="GET",
path="/findings/nope",
errors=parse_jsonapi_errors(
{"errors": [{"detail": "No Finding matches the given query."}]}
),
)
)
assert message == (
"GET /findings/nope failed with HTTP 404. No Finding matches the given query."
)
def test_every_error_of_a_rejected_write_reaches_the_client():
"""The API rejects a write with one error per invalid field, and all of them help."""
message = render_tool_error(
ProwlerAPIError(
"API request failed: 400 - blank",
400,
method="POST",
path="/integrations",
errors=parse_jsonapi_errors(
{
"errors": [
{
"detail": "This field may not be blank.",
"source": {"pointer": "/data/attributes/bucket_name"},
},
{"detail": "Enter a valid URL."},
]
}
),
)
)
assert message == (
"POST /integrations failed with HTTP 400. "
"This field may not be blank. (/data/attributes/bucket_name); "
"Enter a valid URL."
)
def test_a_rejected_write_gets_no_warning():
"""A 4xx changed nothing, so there is nothing to warn about."""
message = render_tool_error(
ProwlerAPIError("boom", 400, method="POST", path="/scans")
)
assert MAY_HAVE_LANDED not in message
def test_a_write_that_hit_a_server_error_warns_it_may_have_landed():
"""The API validates and queues before answering, so a 500 may have gone through."""
message = render_tool_error(
ProwlerAPIError("boom", 500, method="DELETE", path="/integrations/i1")
)
assert message.startswith("DELETE /integrations/i1 failed with HTTP 500.")
assert MAY_HAVE_LANDED in message
def test_a_failed_read_never_warns():
"""A read cannot have changed anything, whatever went wrong."""
message = render_tool_error(
ProwlerAPIError("boom", 500, method="GET", path="/scans")
)
assert MAY_HAVE_LANDED not in message
# ------------------------------------------------------------------ no answer at all
def test_a_write_that_got_no_answer_warns_it_may_have_landed():
"""A timeout is the case the warning exists for."""
request = httpx.Request("POST", "https://api.testing.invalid/api/v1/scans")
message = render_tool_error(httpx.ReadTimeout("timed out", request=request))
assert "POST https://api.testing.invalid/api/v1/scans got no answer" in message
assert "ReadTimeout" in message
assert MAY_HAVE_LANDED in message
def test_a_read_that_got_no_answer_does_not_warn():
request = httpx.Request("GET", "https://api.testing.invalid/api/v1/findings")
message = render_tool_error(httpx.ReadTimeout("timed out", request=request))
assert MAY_HAVE_LANDED not in message
def test_a_dropped_connection_names_what_went_wrong():
request = httpx.Request("POST", "https://api.testing.invalid/api/v1/scans")
message = render_tool_error(httpx.ConnectError("connection reset", request=request))
assert "ConnectError: connection reset" in message
def test_an_unfinished_task_warns_it_may_have_landed():
"""The API already accepted the work, so the outcome is open, not refused."""
message = render_tool_error(
ProwlerTaskError(
"Task t1 polling timed out after 60 seconds.", task_id="t1", state="timeout"
)
)
assert message.startswith("Task t1 polling timed out after 60 seconds.")
assert MAY_HAVE_LANDED in message
# -------------------------------------------------------------- refusals before send
def test_a_stray_value_error_is_reported_as_a_bug():
"""The distinction the previous passthrough branch could not make.
A model factory rejecting an API payload, or an `int()` on something that is not a
number, is not the caller's mistake. Describing it like a validation message sends
an agent off rewriting arguments that were never the problem.
"""
message = render_tool_error(
ValueError("Missing pagination metadata in API response")
)
assert "unexpected ValueError" in message
assert "bug in the server" in message
def test_an_authentication_failure_says_so():
"""A `ValueError` subclass, so it must be recognised before the generic branch."""
assert render_tool_error(ProwlerAuthError("Token has expired")) == (
"Prowler authentication failed: Token has expired"
)
# ---------------------------------------------------------------------- other hosts
def test_a_hub_failure_reads_like_an_api_failure():
"""Same sentence as the Prowler API, with the service named.
The Hub can be down while the API is fine, so which one failed is worth the two
extra words -- but the shape must not differ, or the two look like two contracts.
"""
message = render_tool_error(
ProwlerHubError(
"hub failed",
status_code=404,
path="/check/test",
body='{"error": "Not found"}',
)
)
assert message == "Prowler Hub GET /check/test failed with HTTP 404. Not found"
@pytest.mark.parametrize(
("body", "expected"),
[
('{"error": "Not found"}', "Not found"),
('{"message": "Bad gateway"}', "Bad gateway"),
('{"detail": "Rate limited"}', "Rate limited"),
("Service Unavailable", "Service Unavailable"),
('{"unexpected": "shape"}', '{"unexpected": "shape"}'),
],
ids=["error", "message", "detail", "plain-text", "unknown-json"],
)
def test_an_upstream_message_is_pulled_out_of_whatever_shape_it_came_in(body, expected):
"""Hosts that are not the Prowler API each have their own error shape.
Relaying the raw body puts JSON braces, or a whole HTML page, in front of the model.
"""
message = render_tool_error(
ProwlerHubError("hub failed", status_code=500, path="/checks", body=body)
)
assert message.endswith(expected)
def test_an_upstream_body_is_truncated():
"""An HTML error page must not flood the model's context."""
message = render_tool_error(
ProwlerHubError("hub failed", status_code=500, path="/checks", body="x" * 2000)
)
assert "x" * 500 in message
assert "x" * 501 not in message
# --------------------------------------------------------------------- server bugs
def test_a_bug_in_this_server_is_reported_as_a_bug():
"""Named as ours, so the caller stops trying to fix it by changing arguments."""
message = render_tool_error(KeyError("attributes"))
assert "unexpected KeyError" in message
assert "bug in the server" in message
+170
View File
@@ -0,0 +1,170 @@
"""Tests for the server class every sub-server is built from.
These drive a real `ProwlerMCP` through an in-memory MCP client rather than calling
`tool_errors` directly, because applying that wrapper by hand is exactly what this
class exists to make unnecessary. What matters is that a tool registered *any* of the
ways this server registers them ends up with the error contract, and that it keeps the
name, description and schema FastMCP publishes.
"""
import pytest
from fastmcp import Client
from fastmcp.exceptions import ToolError
from pydantic import Field
from prowler_mcp_server.lib.errors import ProwlerAPIError
from prowler_mcp_server.lib.server import ProwlerMCP
async def call(server: ProwlerMCP, name: str, arguments: dict | None = None):
"""Call a tool the way a client does, without raising on failure."""
async with Client(server) as client:
return await client.call_tool(name, arguments or {}, raise_on_error=False)
async def test_the_parameterised_decorator_form_is_wrapped():
"""`@mcp.tool()` -- how the hub and documentation sub-servers register."""
server = ProwlerMCP("test", mask_error_details=True)
@server.tool()
async def failing() -> dict:
"""A tool that fails."""
raise ProwlerAPIError("boom", 404, method="GET", path="/x")
result = await call(server, "failing")
assert result.is_error
assert result.content[0].text == "GET /x failed with HTTP 404."
async def test_the_bare_decorator_form_is_wrapped():
"""`@mcp.tool` without parentheses is a different code path in FastMCP."""
server = ProwlerMCP("test", mask_error_details=True)
@server.tool
async def failing() -> dict:
"""A tool that fails."""
raise ProwlerAPIError("boom", 500, method="GET", path="/y")
result = await call(server, "failing")
assert result.is_error
assert "GET /y failed with HTTP 500." in result.content[0].text
async def test_the_direct_call_form_is_wrapped():
"""`mcp.tool(fn)` -- how `BaseTool` auto-registers its methods."""
server = ProwlerMCP("test", mask_error_details=True)
async def failing() -> dict:
"""A tool that fails."""
raise ProwlerAPIError("boom", 403, method="DELETE", path="/z")
server.tool(failing)
result = await call(server, "failing")
assert result.is_error
assert "DELETE /z failed with HTTP 403." in result.content[0].text
async def test_a_synchronous_tool_is_wrapped():
"""The documentation sub-server registers plain `def` tools."""
server = ProwlerMCP("test", mask_error_details=True)
@server.tool()
def failing() -> dict:
"""A synchronous tool that fails."""
raise KeyError("attributes")
result = await call(server, "failing")
assert result.is_error
assert "unexpected KeyError" in result.content[0].text
async def test_a_refusal_reaches_the_caller_word_for_word():
"""A `ToolError` is passed through untouched, masking included.
That is the whole reason refusals are raised as one: the tool already wrote the
sentence the caller needs, and nothing downstream improves on it.
"""
server = ProwlerMCP("test", mask_error_details=True)
@server.tool()
async def refusing() -> dict:
"""A tool that refuses its arguments."""
raise ToolError(
"Date range cannot exceed 2 days. Requested range: 2025-01-01 to "
"2025-01-10 (10 days)"
)
result = await call(server, "refusing")
assert result.is_error
assert result.content[0].text == (
"Date range cannot exceed 2 days. Requested range: 2025-01-01 to "
"2025-01-10 (10 days)"
)
async def test_a_result_is_passed_through_untouched():
server = ProwlerMCP("test")
@server.tool()
async def succeeding(value: int) -> dict:
"""A tool that works."""
return {"value": value}
result = await call(server, "succeeding", {"value": 3})
assert not result.is_error
assert result.data == {"value": 3}
async def test_wrapping_does_not_disturb_the_published_tool():
"""The wrapper must be invisible to FastMCP's schema generation.
A wrapper that loses the signature takes the parameters with it, and a tool with no
parameters and no description is unusable while still looking registered.
"""
server = ProwlerMCP("test")
@server.tool()
async def search(
query: str = Field(description="What to search for"),
limit: int = Field(default=10, description="How many results"),
) -> dict:
"""Search for things."""
return {"query": query, "limit": limit}
async with Client(server) as client:
(tool,) = await client.list_tools()
assert tool.name == "search"
assert tool.description == "Search for things."
properties = tool.inputSchema["properties"]
assert properties["query"]["description"] == "What to search for"
assert properties["limit"]["default"] == 10
@pytest.mark.parametrize("name", ["decorated", "direct"])
async def test_every_registration_carries_the_marker(name):
"""The marker is what lets the contract test prove no tool slipped past."""
server = ProwlerMCP("test")
async def direct() -> dict:
"""Registered by direct call."""
return {}
@server.tool()
async def decorated() -> dict:
"""Registered by decorator."""
return {}
server.tool(direct)
tool = await server.get_tool(name)
assert tool is not None, f"{name!r} was not registered at all"
# `get_tool` is typed as the base `Tool`; only `FunctionTool` carries `fn`.
assert getattr(getattr(tool, "fn", None), "__prowler_tool_errors__", False)
+28
View File
@@ -33,6 +33,34 @@ async def test_every_sub_server_contributes_tools(mcp_root_server):
assert tools_in_namespace(tools, "prowler_"), "Prowler App registered no tools"
async def test_no_tool_disappears_between_registration_and_the_client(mcp_root_server):
"""Every tool registered on a sub-server must still be reachable through the mount.
`ProwlerMCP.tool` wraps every tool before handing it to FastMCP, whether it arrived
by decorator or by the direct call `BaseTool.register_tools` makes. A wrapper that
loses the signature, the name or the coroutine-ness of what it wraps drops the tool
silently: the mount still succeeds and the count is the only thing that moves.
"""
from prowler_mcp_server.prowler_app.server import app_mcp_server
from prowler_mcp_server.prowler_documentation.server import docs_mcp_server
from prowler_mcp_server.prowler_hub.server import hub_mcp_server
async with Client(mcp_root_server) as client:
tools = await client.list_tools()
for namespace, sub_server in (
("prowler_hub_", hub_mcp_server),
("prowler_docs_", docs_mcp_server),
("prowler_", app_mcp_server),
):
expected = len(await sub_server.list_tools())
published = len(tools_in_namespace(tools, namespace))
assert published == expected, (
f"'{namespace}' publishes {published} tools but its sub-server registered "
f"{expected}"
)
async def test_every_tool_is_namespaced(mcp_root_server):
"""Tool names are a published interface; nothing may escape the namespaces."""
async with Client(mcp_root_server) as client: