AI Agent API Key Leakage: Output Detection and Limits

What 1-SEC checks

Output analysis reads the supplied output or response string and applies implemented secret and sensitive-data patterns. Matching rules produce LLM output violation events and alerts. The pattern names and categories explain which representation triggered the finding.

Read the implementation: analyzeOutput

Required telemetry and context

Relevant input events: llm_output, llm_response, completion. Use a trusted producer at the real application boundary.

• output
• response
• session_id

Field names above describe relevant context, not a complete API request. Consult the current event contract for envelope and detail requirements. Preserve booleans, numbers, and strings as the receiving handler expects them. Agent-provided text cannot independently verify identity or authorization.

Deployment, prevention, and response

Use synthetic keys in tests and redact production secrets from logs. If an output really exposed a credential, revoke/rotate it through the issuing service and investigate downstream use. A synchronous output gate must be implemented at the application boundary if release prevention is required. For a tool-result integration, compare the synthetic API-key fixture with an ordinary result and verify the event reaches analyzeOutput. Do not log real credentials to demonstrate the detector.

Record the source event, affected agent/host, actual target, policy decision, and response result. Review enforcement configuration separately from detection. A recommendation, selected action, dry-run record, and completed action are different outcomes.

Reproduce the source fixtures

From a checkout of the linked 1-SEC source revision, run the selected fixtures with the repository's Go toolchain. These commands run tests; they do not execute a real attack against your systems.

go test ./internal/modules/llmfirewall -run '^(TestAnalyzeOutput_APIKey_Leak|TestAnalyzeOutput_CleanOutput_NoAlert)$' -count=1

The referenced source fixtures ran in the selected local package suites. Helper tests and synthetic events establish only the assertions shown, not production prevention or framework compatibility.

func TestAnalyzeOutput_APIKey_Leak(t *testing.T) {
	pipeline := makeCapturingPipeline()
	f := startedFirewallWithPipeline(t, pipeline)
	ev := makeLLMOutputEvent("Your OpenAI key is sk-abcdefghijklmnopqrstuvwxyz123456789012345")
	f.HandleEvent(ev)

	if pipeline.count() == 0 {
		t.Error("expected alert for API key leak in LLM output")
	}
}

func TestAnalyzeOutput_CleanOutput_NoAlert(t *testing.T) {
	pipeline := makeCapturingPipeline()
	f := startedFirewallWithPipeline(t, pipeline)
	ev := makeLLMOutputEvent("The capital of France is Paris. Have a great day!")
	f.HandleEvent(ev)

	if pipeline.count() > 0 {
		t.Errorf("expected no alerts for clean output, got %d", pipeline.count())
	}
}

Tuning, evaluation, and coverage limits

This is not a complete secret inventory or a universal DLP system. Encoded, split, proprietary, or unrecognized credentials can evade the patterns. Output telemetry delivered after release cannot retroactively prevent disclosure. MCP tool results are not automatically intercepted by this LLM output path. Explicitly map the returned text into an output or response event if it is the material being inspected; other tool formats and binary content need their own handling.

Evaluate this behavior with representative authorized activity and a controlled known-positive case. Investigate the exact rule, inputs, and state before suppressing an alert. Do not assume a hard-coded heuristic is a configurable setting. Measure your own deployment; the source fixtures do not establish a benchmark, detection rate, compliance certification, or incident-prevention claim.