1-SEC contributors · Reviewed · Module: data_poisoning
Model Registry Review: Missing Integrity Metadata and Lookalike Names
Model registry events flag missing integrity metadata only when both signature and hash strings are empty.
What 1-SEC checks
Model registry events flag missing integrity metadata only when both signature and hash strings are empty. Additional heuristics identify recent low-download entries, suspicious model-name words and approximate matches to a short built-in list of popular normalized model names. The lookalike comparison accepts an edit distance of at most two, excludes exact normalized matches and additionally depends on absent hash and reported age or download count.
Required telemetry and context
Relevant input events: model_download, model_registry. Use a trusted producer at the real application boundary.
model_nameregistryhashsignaturedownload_count (number)created_days_ago (number)author
Field names above describe relevant context, not a complete API request. Consult the current event contract for envelope and detail requirements. Preserve booleans, numbers, and strings as the receiving handler expects them. Agent-provided text cannot independently verify identity or authorization.
Deployment, prevention, and response
Emit registry metadata from a trusted download or promotion gateway and verify signatures and digests there using your approved publisher policy. Investigate spelling, publisher identity, version and exact resolved artifact together. Keep legitimate forks in your release review instead of treating download popularity as trust. Use the separate package guide for package-manager names and the artifact guide for bounded Pickle/SafeTensors inspection.
Record the source event, affected agent/host, actual target, policy decision, and response result. Review enforcement configuration separately from detection. A recommendation, selected action, dry-run record, and completed action are different outcomes.
Reproduce the source fixtures
From a checkout of the linked 1-SEC source revision, run the selected fixtures with the repository's Go toolchain. These commands run tests; they do not execute a real attack against your systems.
go test ./internal/modules/datapoisoning -run '^(TestGuard_HandleEvent_ModelDownload_Slopsquatting)$' -count=1The cited fixture verifies a synthetic lookalike model name produces model_slopsquatting. It does not validate a registry service, a real signature or the precision of the name/age/download heuristics.
TestGuard_HandleEvent_ModelDownload_Slopsquatting
Full fixture and assertions — run in its original package with its helpers.
func TestGuard_HandleEvent_ModelDownload_Slopsquatting(t *testing.T) {
cp := makeCapturingPipeline()
g := startedModuleWithPipeline(t, cp)
defer g.Stop()
ev := core.NewSecurityEvent("test", "model_download", core.SeverityInfo, "model download")
ev.Details["model_name"] = "llamma-3"
ev.Details["registry"] = "huggingface"
ev.Details["author"] = "random-user"
ev.Details["created_days_ago"] = 1
ev.Details["download_count"] = 12
ev.SourceIP = "10.0.0.1"
if err := g.HandleEvent(ev); err != nil {
t.Fatalf("HandleEvent() error: %v", err)
}
if !cp.hasAlertType("model_slopsquatting") {
t.Error("expected model_slopsquatting alert type")
}
}Tuning, evaluation, and coverage limits
A nonempty signature or hash string is not cryptographic verification. Registry names, authors, age and download counts are producer-supplied; missing numeric values can behave as zero and trigger suspicious-new-model heuristics. Approximate naming can flag legitimate forks and miss unrelated malicious models. No live registry lookup, author authentication, model-card verification or model-behavior evaluation occurs here. These signals cannot establish whether AI hallucinated a package or whether a model is backdoored.
Evaluate this behavior with representative authorized activity and a controlled known-positive case. Investigate the exact rule, inputs, and state before suppressing an alert. Do not assume a hard-coded heuristic is a configurable setting. Measure your own deployment; the source fixtures do not establish a benchmark, detection rate, compliance certification, or incident-prevention claim.
Related implemented controls
- RAG Document Poisoning: Retrieved-Content Verification
- Training Dataset Integrity: Hash Changes and Bulk-Update Signals
- Model Confidence Drift: Low-Confidence Signals and Distribution Windows
- Model Weight Integrity: Comparing Reported and Expected Hashes
- Model Artifact Inspection: Pickle Indicators and SafeTensors Bounds