1-SEC contributors · Reviewed · Module: data_poisoning
Model Weight Integrity: Comparing Reported and Expected Hashes
A model-update event with a model ID and two unequal nonempty weight hashes produces a critical model-tampering alert.
What 1-SEC checks
A model-update event with a model ID and two unequal nonempty weight hashes produces a critical model-tampering alert. The comparison uses the supplied strings directly. The same handler also evaluates the confidence drift state collected for that model; a weight mismatch and a distribution shift are separate observations.
Required telemetry and context
Relevant input events: model_update, model_deploy, weight_change. Use a trusted producer at the real application boundary.
model_idversionweight_hashexpected_hashsource
Field names above describe relevant context, not a complete API request. Consult the current event contract for envelope and detail requirements. Preserve booleans, numbers, and strings as the receiving handler expects them. Agent-provided text cannot independently verify identity or authorization.
Deployment, prevention, and response
Compute a digest of the exact artifact in a trusted release pipeline and obtain the approved expected value through an independently controlled manifest. Supply both values at the deployment boundary. Refuse an unapproved artifact in that pipeline rather than waiting for a monitoring alert. Preserve the artifact version and deployment decision for investigation; do not overwrite the expected digest merely to clear an alert.
Record the source event, affected agent/host, actual target, policy decision, and response result. Review enforcement configuration separately from detection. A recommendation, selected action, dry-run record, and completed action are different outcomes.
Reproduce the source fixtures
From a checkout of the linked 1-SEC source revision, run the selected fixtures with the repository's Go toolchain. These commands run tests; they do not execute a real attack against your systems.
go test ./internal/modules/datapoisoning -run '^(TestGuard_HandleEvent_ModelUpdate_Tampering)$' -count=1The existing synthetic event fixture asserts model_tampering when reported and expected hashes differ. Source inspection establishes the nonempty guards and comparison; no live artifact signing, file hashing or deployment refusal was exercised.
TestGuard_HandleEvent_ModelUpdate_Tampering
Full fixture and assertions — run in its original package with its helpers.
func TestGuard_HandleEvent_ModelUpdate_Tampering(t *testing.T) {
cp := makeCapturingPipeline()
g := startedModuleWithPipeline(t, cp)
defer g.Stop()
ev := core.NewSecurityEvent("test", "model_update", core.SeverityInfo, "model update")
ev.Details["model_id"] = "model-prod"
ev.Details["version"] = "2.0"
ev.Details["weight_hash"] = "tampered-hash-abc"
ev.Details["expected_hash"] = "expected-hash-xyz"
ev.Details["source"] = "registry"
ev.SourceIP = "10.0.0.1"
if err := g.HandleEvent(ev); err != nil {
t.Fatalf("HandleEvent() error: %v", err)
}
if cp.count() == 0 {
t.Error("expected alert for model tampering")
}
if !cp.hasAlertType("model_tampering") {
t.Error("expected model_tampering alert type")
}
}Tuning, evaluation, and coverage limits
1-SEC does not open the model file or compute weight_hash in this handler. It does not authenticate the expected hash, verify a model signature, inspect learned behavior or prove that matching bytes are safe. Missing either hash skips the mismatch branch. A trusted manifest and a digest taken from the same compromised source are not independent evidence. The fixture below verifies mismatch alerting only; it is not a full signed-model deployment test.
Evaluate this behavior with representative authorized activity and a controlled known-positive case. Investigate the exact rule, inputs, and state before suppressing an alert. Do not assume a hard-coded heuristic is a configurable setting. Measure your own deployment; the source fixtures do not establish a benchmark, detection rate, compliance certification, or incident-prevention claim.
Related implemented controls
- RAG Document Poisoning: Retrieved-Content Verification
- Training Dataset Integrity: Hash Changes and Bulk-Update Signals
- Model Confidence Drift: Low-Confidence Signals and Distribution Windows
- Model Registry Review: Missing Integrity Metadata and Lookalike Names
- Model Artifact Inspection: Pickle Indicators and SafeTensors Bounds