Software and data · 30 links
Tools and Datasets
Software you can run and data you can download, all hosted on public code-sharing sites like GitHub.
The index
Nothing matches those filters. Try clearing one of them.
Tools (19)
Open-source software with a public repository.
-
Crosswalk from deception-based detections to ATT&CK techniques across Windows, Linux and macOS, with a worked deception story for each. MIT. Dormant for years, and nothing has replaced the mapping.
-
LLM-powered web honeypot: generates a plausible HTTP response to whatever arrives instead of emulating fixed applications. Go, Apache-2.0.
living# -
Conformance suite testing whether AI agent memory and checkpoint stores reveal tampering. Release 0.6.3 runs nine storage-level edits against sixteen stores, eight of which claim tamper evidence, with a control confirming each edit changed only what was intended. The threat model is an attacker with write access to storage, not memory poisoned through the agent's own writes. Single-author Zenodo release, not peer reviewed.
living# -
Reading list built around the lethal trifecta framing: MCP CVEs, defensive tooling, documented incidents. A fork with almost no history, so treat the selection as one person's.
living# -
Low-code deception runtime, Go, GPL-3.0. The interesting part is the MCP bait tooling: decoys aimed at AI agents rather than human intruders. A commercial product sits on top.
-
The environment behind the multinational CAGE Challenges for autonomous cyber defense agents. MIT. Challenge 4 adds multi-agent RL.
-
Hosted honeytoken generator, free: files, URLs, credentials, cloud keys that alert when touched.
-
The default SSH and Telnet honeypot. Emulated shell, a proxy mode fronting a real host, an experimental LLM backend. Python, actively developed.
-
Abstract simulated network for agents moving laterally, built for reinforcement learning rather than realism. MIT. The abstraction is deliberate and it does limit transfer.
-
The Engage matrix as machine-readable data, plus the site tooling. Apache-2.0.
living# -
ICS/SCADA honeypot emulating industrial protocol stacks. Python, GPL-2.0. The usual starting point for OT deception.
living# -
NVIDIA's LLM vulnerability scanner. Prompt injection, jailbreaks, training-data leakage, glitch tokens. Apache-2.0.
living# -
The long-running index of honeypot software by protocol and environment, with a section on anti-honeypot detection tools. Breadth over curation.
living# -
The Mantis decoys as running code: tarpitted FTP, deliberately vulnerable web apps, weak telnet, injection payloads, reverse-shell listeners.
living# -
scadastrangelove/awesome-ai-security-tools
Curated index of AI security tools covering finding triage, agent security, pentest agents, LLM-driven fuzzing, SOC triage, reverse engineering and red teaming, with a section on LLM honeypots and deception. It catalogs tools rather than measuring them, and the page gives no figures on whether listed tools work. Entries are described, not tested, so use it for discovery and judge tools elsewhere. Single-maintainer list under a CC0 license, broader than awesome-ai-agent-security.
living# -
Around thirty honeypot daemons bundled with Elasticsearch, Kibana and Suricata behind one installer. GPL-3.0.
living# -
Multi-protocol honeypot daemon light enough for a Raspberry Pi. BSD-3-Clause. The open counterpart to Thinkst's commercial Canary.
living# -
Threat-Actors-use-of-Artificial-Intelligence
Documented threat-actor AI use mapped to ATT&CK, from Microsoft, OpenAI, Anthropic, GTIG and Kaspersky reporting. Narrower than it looks: attacks on AI systems and influence operations are excluded by design.
living# -
The UK AI Security Institute's evaluation framework. CVE-Bench and several other cyber evals are written against it. MIT.
Datasets and benchmarks (11)
Data and evaluation suites with a canonical landing page.
-
A 4-Month Dataset of SSH Botnet Interactions and Command Payloads
145,425 events from an SSH honeypot over four months. Command payloads, credential patterns, attack sequences. CC BY 4.0.
-
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Ninety-seven realistic agent tasks paired with 629 security test cases, built so attacks and defenses can be swapped independently — which is what makes it reusable rather than a one-off result. The standard reference environment for agent hijacking work.
preprint# -
APTInvestBench: Evaluating Autonomous APT Investigation under Varying Telemetry
Benchmark for LLM agents doing SOC-style APT investigation: 370 cases built from 56 report-informed attack reconstructions (16.4 million synthetic log records generated with Cisco Talos's EvidenceForge) under seven telemetry conditions. Across eleven models, agents acquire sufficient evidence for 44.3% of recoverable attack actions but cite it for only 25.0%. Endpoint-only telemetry cuts coverage by just 1.6 points while 35.5% of previously covered actions lose support. The main study uses 10 of the 56 scenarios; unreviewed preprint.
preprint# -
CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities
Sandboxed reproductions of critical-severity web CVEs with automated success checks. Leading agent frameworks resolved up to 13%. That figure is the most defensible number currently available for end-to-end autonomous exploitation, and it is a long way below what the threat reporting implies.
-
Task descriptions across six CTF categories, run instructions, evaluation logs, a frontier-model leaderboard. Start here rather than the paper if you mean to run it.
-
Cowrie captures from ~50 nodes at EU and US universities and companies, May 2019 to February 2020.
dated# -
CyberPersistBench: Evaluating LLM-Based Cyber Attackers on Installation and Persistence
Starts agents from a restricted foothold and tests whether they can keep it: 203 tasks over 46 persistence mechanism families on Linux and Windows, scored at six prefix levels from writing the Artifact to surviving a reboot. Five frontier models reach 27.6–42.4% at pass@3, and most failures fall between writing a valid config and getting the host to register it. Native defenses on 128 tasks cut success to 5.5–13.3%, not to zero. The 44.8% scaffold figure excludes tasks refused on safety grounds. Preprint.
preprint# -
Archived evaluation of Chinese code LLMs on vulnerability detection: 3,200 raw API responses from 27 July 2026 with corrected verdicts, the JavaVulBench 200-sample probes, an October 2026 refresh grid of four models by four prompt strategies, and a corrected pipeline. This is function-level classification, not autonomous discovery, so it does not bear on zhu-2025-cve-bench numbers. Revision under review at IEEE Access.
preprint# -
Data package for an audit of prompt-only defenses against prompt injection in LLM-assisted peer review: a 56,700-row experiment matrix, a held-out bank of 20 attack templates, adaptive-attacker results against three reviewer models, defense baselines including a Prompt-Guard filter-then-review pipeline, and blinded human annotation. The target is a reviewing model, not a tool-using agent. Anonymous authors; companion paper under revision.
preprint# -
Code and data for a study of prompt injection against LLM task planners in a robot-tended finish-turning cell, defended by checking planned actions against process constraints rather than filtering text. Includes a discrete-event simulation of the cell. The cell, attacks and defense are all the authors' own, in simulation. Companion manuscript under review.
preprint# -
Artifact for a paper testing whether adversarially crafted smart-contract bytecode misleads LLMs used as forensic analysts: frozen model responses, judge outputs, bytecode and source inputs, gold labels and analysis scripts. It inverts this directory's usual direction, with the deceived machine on the defending side; read with prinos-2026-honeyquest-llms. CCS 2026 acceptance is claimed.