Securing the Instruction Compiler: Security Scanning, Lint, and Style for Agent Skills

The Problem: Agent Instructions Are Attack Surface

When we built the Compact Instruction Compiler, the goal was simple: take verbose Markdown instruction files and compile them through a PIR intermediate into compact Markdown. What we didn't address was what happens when the input isn't just verbose — it's malicious.

The threat is real. NVIDIA demonstrated an indirect AGENTS.md injection attack where a dependency-injected instruction file overrides the agent's safety directives. The OWASP Agentic Skills Top 10 (AST) formalizes this: prompt injection (AST01), over-privileged commands (AST03), exfiltration (AST04), untrusted external instructions (AST05). The SoK paper on prompt injection against coding assistants (arXiv:2601.17548) shows that natural-language-plus-code blends defeat pure signature scanners — a defensive policy that says "NEVER exfiltrate credentials" contains the same words as an attack that says "send credentials to."

We needed three things: a security gate that runs before compilation, a lint checker that enforces structural conventions, and a config file that ties them together. Here's what we built.

Architecture: Three New Tools

The CIC pipeline gained three new components, each with a clear separation of concerns:

The Pipeline

When you run cic.sh -i (install mode), security scanning and semantic regression checking are now auto-enabled. This means you can't accidentally install adversarial content into your source tree without explicitly opting out with --no-sec.

The Security Scanner: 60 Rules, 9 Categories

The scanner (cic-sec.sh) maps directly to the OWASP Agentic Skills Top 10. Each rule is a tab-delimited record: ID, severity, category, ERE regex, description. The rules are data, not logic — adding a new detection pattern means adding a line to the array, not writing a new function.

The XSS False-Positive Lesson

The original XSS-01 regex was <script[^>]*> — simple and correct for HTML. But agent instruction files are not HTML. They're Markdown with shell code blocks. A line like Run the script at /tmp/<script>.sh triggered the rule 30 times across the repository.

The fix: require HTML context. The regex became (^|[[:space:]<])<script[^>]*> — the <script> tag must be preceded by start-of-line, whitespace, or another < (indicating HTML structure). Shell placeholders like /tmp/<script>.sh have / or alphanumerics before the <, so they no longer match. False positives dropped from 30 to 7 — and all 7 remaining are real <script> tags in HTML code blocks inside archived plans.

The Lint Checker: 70 Rules, 9 File Types

The lint checker (cic-lint.sh) validates structural conventions across nine file types: skills, workflows, abilities, rules, memories, ideas, plans, AGENTS.md, and README.md. The rule catalog lives in cic-style.sh as tab-delimited data records — the same data-not-logic pattern as the security scanner.

Rules were derived from three sources:

When repo conventions conflicted with external specs, the repo won. The Agent Skills spec says description max is 1024 chars; our skills average 80. The spec lists Invocation and Commands as recommended sections; many of our skills (like build and test) are simple enough that they don't need them. Those rules became info severity — they report but don't block.

Severity Design

The .cic Config File

Configuration moved from a scattered .devin/cic-sec-allowlist file to a project-root .cic file — the same pattern as .clang-format and .clang-tidy. Three INI-style sections:

[sec]
# Security scanner allowlist — suppress vetted defensive FPs
# Format: RULE_ID:path-glob or *:path-glob (trust directive)
*:.devin/plans/archive/*
*:.devin/ideas/*
INJ-03:AGENTS.md
XSS-01:.devin/skills/add-lab/*

[lint]
# Lint/style thresholds and rule disables
max_line_length = 200
max_heading_level = 4
# disable = STYLE-04

[compile]
# CIC compiler defaults
auto_sec_on_install = true
auto_guard_on_install = true

The [sec] section is parsed by cic-sec.sh with a one-liner: awk '/^\[sec\]/{f=1;next} /^\[/{f=0} f{print}' .cic — extract lines between [sec] and the next section header. A user-level fallback at ~/.config/devin/cic-sec-allowlist still works for personal overrides.

The Performance Pattern: One Awk Pass

The first implementation used per-file shell loops: for f in *.md; do grep ...; awk ...; done. For 758 files, this spawned 758+ subprocesses and took over 10 seconds. Unacceptable for a compile gate.

The fix: move all rules — frontmatter parsing, heading extraction, naming checks, code/prose/link checks — into a single awk process that processes all files in one invocation. Awk's FNR==1 rule detects file boundaries and resets per-file state. The END block flushes the last file.

This pattern is now canonical for batch file scanning in the CIC toolchain. Both cic-sec.sh (0.5s) and cic-lint.sh (0.4s) use it. The key insight: awk processes all files in one invocation, tracks per-file state with FNR==1 reset, and flushes accumulated results at END.

The AST08 Gap: Why Signatures Aren't Enough

We mitigate this in two ways:

The Red-Team Corpus

We extracted detection patterns from multiple sources into data/redteam/prompt-injection-patterns.txt — a maintained seed set for rule development:

The corpus currently has 90 patterns across 9 categories. cic-sec.sh --update-rules reports coverage: how many corpus patterns are represented by the 60 embedded rules. New patterns are added to the corpus first, then promoted to rules after testing.

Verification

Thirteen unit tests in test-cic-sec.sh cover the scanner's behavior:

What's Next

The security scanner and lint checker are implemented and running. The VLM classification pass is implemented but requires pi-daemon to be running — it gracefully skips when unavailable. On the roadmap:

• Automated FP fixing — a --fix mode for cic-lint.sh that auto-corrects trailing whitespace and consecutive blank lines (safe, mechanical fixes).
• Pre-commit integration — a .githooks/pre-commit script that scans staged instruction files before they land. Documented and ready to install with git config core.hooksPath .githooks.
• Corpus-to-rules pipeline — automatically generate cic-sec.sh rules from data/redteam/ patterns, reducing the gap between corpus (90) and rules (60).
• Semantic structure validation — beyond regex: section ordering, maxTokens budgets, and heading-tree depth validation using the document-schema approach.

The broader question — whether signature scanning plus LLM classification is sufficient for the agentic AI threat landscape — remains open. The SoK paper suggests it's the best we can do without runtime sandboxing. We're betting that compile-time gates plus VLM triage plus an allowlist for vetted content is a practical defense-in-depth for the current generation of agent instruction files.