Agentic Pentesting: Scope, Evidence, and Data Exposure

Last updated: Sep 9, 2026

DarkMoon and Strix are autonomous penetration-testing systems that can complement a development workflow. Strix provides skills for invoking its testing workflows from Claude Code; DarkMoon uses OpenCode and MCP with a dedicated privacy gateway. Neither integration makes the platform’s agent loop a Claude Code feature.

Evidence snapshot: September 9, 2026. This comparison is based on the linked public source code. It supports architectural comparison and a privacy case study. No shared pentest benchmark or deployment validation was performed for this page, so it does not rank detection quality or recommend either system for sensitive targets. The supporting resource evaluation records sources and missing evidence.

Choose the security task

Start with what you need to verify. Configuration scanning, source review, and active exploitation answer different questions.

Task	Example	Relevant route
Secure the coding agent	Check MCP permissions and startup hooks	Security Hardening
Review application source	Trace an authorization check through a changed endpoint	Security auditor template
Test application behavior	Verify that one test account cannot retrieve another account’s record	An authorized laboratory assessment with a pentesting system

For the last row, define the target, accounts, permitted actions, stop conditions, and cleanup before execution. An agent allowed to read a repository does not thereby have permission to attack every service referenced in its configuration.

Compare the inspected implementations

Dimension	Strix	DarkMoon
Execution architecture	Python application using the OpenAI Agents SDK	OpenCode agent configuration plus MCP tools
Claude Code connection	Repository supplies agent skills for its CLI and cloud workflows	Inspected prompt-tokenization plugin targets OpenCode; equivalent Claude Code wiring was not established
Documented target emphasis	Applications, APIs, source repositories, and web targets	Web targets plus specialized infrastructure agents, including AD, Kubernetes, cloud, and LLM endpoints
Local inference	Documents Ollama and OpenAI-compatible endpoints	Documents Ollama and llama.cpp
Data minimization	No equivalent reversible gateway established in this review	Session-local placeholders, selective restoration for execution, output sanitation
Finding checks	Required evidence, PoC, assumptions, counterevidence, confidence, and CVSS fields	Published reports and findings claims; benchmark validation remains a separate task
Deduplication	Explicit report comparison, including a model judge	A distinct-finding count requires inspection of the campaign export
Repository license	Apache-2.0	GPL-3.0

Sources: Strix integration documentation, Strix SDK dependency, Strix local providers, and DarkMoon architecture and configuration.

The DarkMoon README at this revision marks Strix as cloud-only. Strix’s local-provider documentation contradicts that row. Compare the configured execution paths rather than copying either vendor’s comparison table. Local inference also requires checking secondary model calls and telemetry before claiming an offline run.

Follow the data through a privacy gateway

DarkMoon’s gateway addresses a useful problem: running a capable remote model while reducing the target values included in its context. Its OpenCode plugin tokenizes user text through a local socket. The command gateway restores placeholders in permitted execution positions and sanitizes returned strings.

An illustrative flow, using synthetic data:

Operator supplies a laboratory target: 192.0.2.10
    -> local vault assigns a placeholder
Model works with the placeholder
    -> gateway evaluates where the command uses it
Approved local execution receives the target value
    -> tool output is sanitized before the next inference

This separates two boundaries: what reaches the model and where a tool can send a restored value. At the inspected revision, the command gateway’s default DEGRADE policy leaves a placeholder in a disallowed position while allowing the command to run. Its STRICT policy can refuse the command. Those policies do not describe every sanitation failure.

The prompt plugin normally withholds raw text when tokenization is unavailable, unless its fallback configuration allows it. By contrast, the global result sanitizer returns the original value if sanitation raises an exception. A fail-closed prompt hook therefore does not establish fail-closed output handling.

Recognition coverage matters even when nothing crashes. In a local function test with a fresh vault for each input, password=DemoSecret987! was masked, while the same synthetic value remained in a JSON string and a dictionary. This tests sanitize_result, not a running campaign. The evaluation includes the reproducible fixture. Test previously unseen synthetic secrets in tool errors and encoded output too, then inspect requests sent to the model endpoint. Vault logs explain transformations; outgoing requests show the resulting exposure for that run. See Data Privacy for the broader data-flow controls.

Strix has a separate consideration: its deduplication judge receives report fields and can use its own provider endpoint. Include that route in an inference inventory. Strix documents a telemetry opt-out, STRIX_TELEMETRY=0, in its telemetry policy; the setting does not by itself prove that the entire assessment is offline.

Read findings as claims to verify

Strix’s report validator rejects missing evidence text and requests counterevidence and assumptions. This improves the report contract. The check for a nonempty PoC field does not execute that PoC. Its deduplication code also returns a nonduplicate result with an error when the comparison fails, so report consumers need to inspect failures.

For example, an access-control finding should identify the test account, request, unauthorized object, response, and the expected authorization rule. Repeat requests that demonstrate the same defect should not inflate the count without an explicit counting convention. A retrieved JWT public key establishes key availability; token forgery requires proof that the application accepted the forged token. A stored XSS payload establishes persistence; browser execution requires separate evidence.

The inspected DarkMoon README advertises 57 Juice Shop vulnerabilities. Treat this as a publisher-reported campaign count until its export has been reviewed and deduplicated. Strix’s XBEN publication reports 100 of 104 flag-extraction challenges solved using Strix v0.4.0 and gemini-3-pro-preview. Neither figure is a common measure of the two systems, and neither was reproduced for this page.

DarkMoon’s Juice Shop article describes seven findings, four classified as exploited. Its privacy article reports zero occurrences of the real target address in approximately 3.5 MB of model traffic, but identifies a separate LLM-endpoint assessment on a sealed build. Keep those campaigns distinct; the traffic claim does not validate the 57-finding campaign or every sensitive-data category.

Evaluate on the same laboratory

A useful comparison fixes the target and defines success before the agents run. Record the following in each campaign manifest:

Record	Required detail
Software	Agent commit, edition, container image digests, target version
Inference	Exact model, provider endpoints, local-server configuration, secondary judges
Scope	Authorized targets, test accounts, prohibited actions, cleanup procedure
Resources	Token and tool-call limits, monetary budget, stop conditions
Interventions	Human hints, retries, manual corrections, interrupted runs
Results	Distinct defects, reproduction evidence, false positives, missed seeded defects
Exposure	Synthetic canaries, actual outgoing requests, exceptions, telemetry destinations

Use the same target snapshot and counting rules for both tools. Separate a fixed-model experiment from an experiment using each publisher’s recommended model. For DarkMoon’s gateway, compare enabled and disabled runs on synthetic data, recording both exposure and accepted findings. Repeat runs to expose variance; a single success cannot establish reliability.

Review host access separately from model privacy. DarkMoon’s inspected Compose file gives its execution container root privileges, host networking, added network capabilities, and access to the Docker socket. Strix’s Docker client adds NET_ADMIN and NET_RAW; its local source mounts permit writes. Use an isolated laboratory host for evaluation and inspect the selected backend’s mounts and network access. Container packaging alone does not establish containment. The Sandbox Isolation guide explains the distinctions.

Evidence and adoption limits

Strix is the more direct candidate for an investigation beginning in Claude Code because the repository supplies that integration route. DarkMoon is a useful case study for reversible tokenization and specialized MCP execution. These are editorial placement decisions, not measured detection rankings.

Before publishing a tested setup or recommending use with sensitive data, obtain a reproducible campaign manifest, outgoing-request evidence, failure-policy tests, and a reviewed finding export. Keep public source observations separate from cloud-product promises and from private communications. A pinned source review establishes what was inspected; it does not establish the behavior of an image tagged latest or a managed service.