Search terms: secret scanning · pre-commit hook · leaked password · API key in repo ·
sanitize doc · redact · kb-sanitize.py · AKIA · connection string · share a runbook externally
Tool: ~/.local/bin/kb-sanitize.py. Written 2026-08-08.
Documentation hygiene is two different problems, and treating them as one produces bad docs. Redact everything and your runbook becomes useless; redact nothing and you eventually commit a password.
| Tier | What | Rule |
|---|---|---|
| CRITICAL | Passwords, API keys, tokens, private keys, connection strings | Never in any repo, however private. Private repos get cloned, forked, backed up, and made public by accident. |
| REVIEW | Hostnames, internal IPs, UUIDs, usernames, paths, naming schemes | These belong in an internal runbook — they’re what makes it usable. They only matter when a doc crosses an audience boundary. |
The second row is the one people get wrong. An internal runbook that says
“connect to <NAS-HOST> as <USER>” helps nobody. Put the real hostname in. The question is
never “is this sensitive?” — it’s “who is going to read this?”
Boundaries worth naming:
Scan only — nothing is written:
kb-sanitize.py docs/runbook.md
Credentials only, for internal repos where hostnames legitimately belong:
kb-sanitize.py docs/runbook.md --creds-only
Write a redacted copy for external sharing. Name your own terms with OLD=LABEL:
kb-sanitize.py docs/runbook.md --out /tmp/runbook-external.md \
--term "acme-nas=NAS-HOST" \
--term "jsmith=USER" \
--term "prod-payments=SHARE"
Exit codes: 0 clean · 1 error · 2 CRITICAL findings — usable in CI or a hook.
CRITICAL
| Rule | Example it matches |
|---|---|
| password assignment | password: correcthorsebattery, PASSWD=... |
| api key / secret / token | API_KEY=sk_live_9fQ2xLmZ8vTbNw41PdRc |
| AWS access key id | AKIAIOSFODNN7EXAMPLE |
| private key block | -----BEGIN RSA PRIVATE KEY----- |
| connection string with password | postgres://user:pass@host/db |
| GitHub / Slack token | ghp_…, xoxb-… |
Placeholders are deliberately ignored, so template docs don’t spam you:
password=YOUR_PASSWORD, password=<REDACTED>, password=$ENV_VAR, password=xxx.
REVIEW — IPv4, UUID, MAC, enx… interface names, /home/<user> paths, email addresses,
.local hostnames. Documentation ranges (192.0.2.x, 198.51.100.x, 203.0.113.x) and
loopback are never flagged; those exist precisely for writing examples.
kb-sanitize.py runbook.md
!! CRITICAL — CREDENTIALS (3 occurrence(s))
These do not belong in ANY repo, however private. Remove before committing.
password assignment — 1 occurrence(s), 1 unique
line 7: password: correcthorsebattery
api key / secret / token — 1 occurrence(s), 1 unique
line 5: API_KEY=sk_live_9fQ2xLmZ8vTbNw41PdRc
connection string with password — 1 occurrence(s), 1 unique
line 4: postgres://svc_deploy:Hunter2Rocks@
REVIEW — identifying detail (6 occurrence(s))
Fine in an internal runbook. Redact only when crossing an audience boundary.
IPv4 address — 3 occurrence(s), 1 unique
line 2: 10.20.30.40
...
Exit code 2. The three CRITICAL items get removed and rotated — a credential committed
once is compromised, even if the next commit deletes it, because it’s still in the history.
The six REVIEW items stay if this is an internal doc.
Blocks a commit when CRITICAL findings appear. Save as .git/hooks/pre-commit, chmod +x:
#!/bin/sh
fail=0
for f in $(git diff --cached --name-only --diff-filter=ACM | grep -E '\.(md|txt|yaml|yml|env|conf)$'); do
kb-sanitize.py "$f" --creds-only >/tmp/scan.$$ 2>&1 || {
[ $? -eq 2 ] && { cat /tmp/scan.$$; fail=1; }
}
done
rm -f /tmp/scan.$$
[ $fail -eq 1 ] && { echo "Commit blocked: credentials found. Remove and rotate them."; exit 1; }
exit 0
--creds-only is the right mode here — you do not want a hook nagging about hostnames in
your own internal docs every single commit. Noisy hooks get bypassed with --no-verify, and a
hook everyone bypasses protects nothing.
It cannot judge context. These all passed a clean scan and still had to be rewritten by hand when moving a doc to a different repo:
{{< relref >}} link to a page that only exists in the source wikiNone contain an IP, a username, or a path. The mechanical half automates; the contextual half does not.
It only knows the terms you name. A first run passed --term "acme-nas" while the doc said
ACME NAS with a space — no match, no warning. Name every spelling.
1. Don’t build a placeholder out of the thing you’re hiding. The first version replaced
jsmith with <JSMITH>. It reported success while the string was still in the file — wrapped
in angle brackets, still fully searchable. Fixed by requiring OLD=LABEL.
The lesson generalises past this tool: verify the artifact, not the tool’s report. Always finish with an independent check:
for s in acme-nas jsmith 10.20.30.40 prod-payments; do
printf "%-16s %s\n" "$s" "$(grep -ic "$s" external-copy.md)"
done
Every count must be 0. That check found the bug the tool’s own success message hid.
2. A malformed test makes a working tool look broken. A test key of
AKIAQ7RZ4NEXAMPLE99 never matched, and the AWS rule looked dead. It’s AKIA + 15
characters; a real key is AKIA + 16. Re-tested with a correctly-formed sample and it fired
immediately. When a pattern matcher “fails,” check the test data before the pattern.
3. Documentation about secrets trips secret scanners. This very page flagged 10 CRITICAL findings on its first scan — every one a deliberate example. Left alone, the pre-commit hook would have blocked commits on the doc that explains the hook. Same self-referential trap as a security scanner whose own rule table looks like malware to another scanner.
4. \b silently misses the most common secret naming convention. The first version used
\bpassword — and reported “Nothing flagged” on a real .env that contained a live credential.
In DB_PASSWORD the character before PASSWORD is an underscore, and _ is a word
character, so \b never matches there. Tested against realistic variable names it caught
1 of 5:
password=classicform caught
EMAIL_PASS=... MISSED
DB_PASSWORD=... MISSED
API_SECRET=... MISSED
SMTP_PASS=... MISSED
DB_PASSWORD, API_SECRET, AWS_SECRET_ACCESS_KEY, SMTP_PASS — that’s how secrets are
actually named in the wild. Fixed with (?<![A-Za-z0-9]), a zero-width lookbehind that permits
a leading underscore while still refusing a match inside a word like surpass=.
A scanner’s silence is not evidence. “Nothing flagged” means either there is nothing there or my patterns do not cover this, and the two are indistinguishable from outside. Test any detector against inputs you know should fire before you trust a clean result. This one reported clean on a file that was not.
kb-sanitize: ignorePut the marker on a line and its matches are skipped. Put it on a fence opener and the whole block is skipped.
| password assignment | `password: hunter2` | <!-- kb-sanitize: ignore -->
``` kb-sanitize: ignore
...an example block full of sample secrets...
```
It is deliberately line-scoped, not file-scoped. A file-level opt-out would mean one example paragraph disables scanning for the entire document — and that’s precisely the file where a real secret would later go unnoticed. Marking each line costs more effort and keeps the rest of the file protected.
Verify after marking: an unmarked test file must still fail. If exemptions silence everything, they’re too broad.
git filter-repo,
BFG) is secondary and does nothing for anyone who already cloned..smbcredentials, .env) with 600
permissions, created via install -m 600 /dev/null <file> so the contents are never briefly
world-readable — and edited in an editor, so they never enter shell history.gitleaks, trufflehog). It runs before the commit; those
catch what got through.