Search terms: smartctl · smartmontools · drive failing · reallocated sectors ·
pending sectors · SSD wear · TBW · Power On Hours · drive alerts · discord webhook ·
smart-health-check.sh · false all-clear
Built 2026-08-10. Watches every local drive on the laptop and media server, alerts to a Discord channel on degradation, and sends a weekly digest so a dead monitor is distinguishable from healthy drives.
Script: ~/.local/bin/smart-health-check.sh (both machines, identical path).
sudo smart-health-check.sh # alert ONLY on problems (daily cron)
sudo smart-health-check.sh --digest # always report (weekly cron)
sudo smart-health-check.sh --dry-run # print locally, post nothing
Cron — root’s crontab, not the user one:
sudo crontab -e
0 8 * * * /home/aztechguy/.local/bin/smart-health-check.sh
5 8 * * 0 /home/aztechguy/.local/bin/smart-health-check.sh --digest
Webhook at ~/.config/discord-webhooks/drives.txt, mode 600. Log at ~/drive-health.log —
every run since 2026-08-11, including clean ones (see bug 5).
There are three device classes, not two. Branching is on
/sys/block/<dev>/queue/rotational, never on the device name — see bug 6 below.
Spinning disks (rotational=1) — attributes that predict mechanical failure:
| Attribute | Meaning |
|---|---|
Reallocated_Sector_Ct (5) |
Bad blocks already remapped. Should be 0 |
Current_Pending_Sector (197) |
Failed reads awaiting reallocation. Strongest predictor |
Offline_Uncorrectable (198) |
Unrecoverable. Non-zero means data loss already happened |
Temperature_Celsius (194) |
Sustained >45-50°C shortens life |
Power_On_Hours (9) |
÷8760 = years. On SATA this is roughly calendar age |
SATA SSDs (rotational=0, not nvme*) — 197 and 198 do not exist on these, and
temperature is on 190, not 194:
| Attribute | Meaning |
|---|---|
Wear_Leveling_Count (177) |
Read the normalised VALUE, not the raw. 100 = new; 100 − VALUE = % consumed |
Used_Rsvd_Blk_Cnt_Tot (179) |
Reserve blocks eaten. The SATA equivalent of NVMe’s Available Spare |
Uncorrectable_Error_Cnt (187) |
Non-zero means data loss already happened |
Airflow_Temperature_Cel (190) |
Not 194. Asking for 194 returns nothing |
Total_LBAs_Written (241) |
× 512 = bytes written |
Every SATA device, spinning or not:
| Attribute | Meaning |
|---|---|
CRC_Error_Count (199) |
Interface errors — a failing cable or connector, not the platters. A failing SATA cable presents exactly like a dying drive |
NVMe — a completely different set:
| Field | Meaning |
|---|---|
Critical Warning |
Any non-zero bit is serious |
Percentage Used |
Controller’s own wear estimate (includes write amplification) |
Available Spare |
Reserve blocks left. Falling below 20% is late-stage |
Data Units Written |
Actual host writes — compare against the drive’s rated TBW |
Media and Data Integrity Errors |
Should be 0 |
NVMe Power On Hours counts active time, not calendar time. The spec permits drives to
exclude time spent in non-operational power states, so an idle drive accumulates hours slowly.
Real numbers from this machine:
Samsung 980 PRO (OS drive) 4,802 h 51.4 TB written
WD Blue SN570 (data drive) 1,081 h 30.6 TB written
The data drive had been installed for over a year and reported the equivalent of 45 days. Converting that to “0.1 years” produced a figure that was flatly wrong, and it briefly made a genuine 509-day-old ext4 error look impossible.
So the script prints raw hours for NVMe and years only for SATA. That asymmetry is deliberate — do not “fix” it.
For SSD life, Data Units Written against rated TBW is the number that means something:
980 PRO 1TB: 51.4 TB / ~600 TBW = 8.6% (drive's own estimate: 17%)
SN570 2TB: 30.6 TB / ~900 TBW = 3.4% (drive's own estimate: 1%)
The controller’s Percentage Used runs more conservative because it counts write
amplification, not just host writes.
Every one of these would have shipped silently. They are all the same species: a code path that was written but never executed.
--dry-run printed nothing on a healthy systemThe “no problems” branch hit exit 0 before the dry-run print. Healthy drives plus dry-run
equalled silence, which looked like a crash.
Covered above. The parser was correct; the metric was misunderstood. Worth noting the
sequence — the output was doubted, validated against raw smartctl, and the code turned out
to be right while the interpretation was wrong. Validating still paid off.
HTTP 403 Forbidden posting to DiscordDiscord sits behind Cloudflare, which rejects Python’s default Python-urllib/3.x User-Agent.
The fix was already in another script on the same machine. run-buffer-automation.sh had
been posting successfully for months using python to build the JSON and curl to send it.
Match the working pattern before inventing one. A mechanism already proven in your own environment beats a fresh one, however clean it looks.
smartctl needs root. Run as a normal user it returns nothing, the drive list came back empty,
and the script cheerfully reported “✅ all clear” over zero drives.
This nearly shipped, because most cron jobs on this box live in the user crontab
(crontab -e) and only root’s is empty. Pasting the lines into the familiar place would have
produced a green weekly digest every Sunday while nothing was actually being checked.
Two guards now:
[ "$(id -u)" -eq 0 ] || exit 1 # refuse to run as non-root
if [ ${#SUMMARY[@]} -eq 0 ]; then # empty list is a PROBLEM, never "all clear"
PROBLEMS+=("No drives could be read at all.")
fi
A monitor that cannot see must never report success. Absence of findings and absence of capability look identical from the outside, and only one of them is good news.
The first real scheduled run was checked the next morning. drive-health.log had no entry for
that day, and on the media server the log file did not exist at all. It looked exactly like
a cron job that never fired.
It had fired. The journal proved it:
Aug 11 08:00:01 CRON[276408]: (root) CMD (/home/aztechguy/.local/bin/smart-health-check.sh)
The clean path returned before writing anything:
else
exit 0 # nothing wrong, not digest day — stay silent <-- the bug
fi
“Stay silent” was meant to mean don’t post to Discord. It also silenced the log, so a healthy run and a dead job were indistinguishable for up to five days — until the Sunday digest either arrived or didn’t.
This page already said it logged every run. The code never did. The documentation described the intended behaviour and was believed over the artifact, which is what made the diagnosis take twenty minutes instead of two. Fixed by logging before the silent exit:
log "clean: ${#SUMMARY[@]} drive(s) checked, no problems, nothing posted"
exit 0
Silent on the alert channel is not the same as silent in the log. Decide those two separately, and prefer a run record that exists even when nothing happened. A monitor with no heartbeat cannot be distinguished from a corpse.
Verifying a monitor’s first scheduled run is a distinct task from installing it, and it is worth doing by hand rather than waiting for the digest. The check that finds this class of bug is always the same shape: did the thing run, and did it leave evidence? — two questions, not one.
The media server’s newly cloned Samsung 870 EVO, in the weekly digest:
sda Samsung SSD 870 EVO 50 0.0y ?°C realloc=0 pend=? uncorr=? PASSED
Three fields unknown, and still PASSED. Two faults compounding:
for dev in $(lsblk -dno NAME,TYPE | awk '$2=="disk" && $1 !~ /^nvme/ {print $1}'); do
realloc=$(a 5); pending=$(a 197); uncorr=$(a 198); temp=$(a 194)
[ "${pending:-0}" -gt 0 ] && PROBLEMS+=(...)
nvme* took the spinning-disk
path. A SATA SSD has no 197 and no 198 at all, and reports temperature on 190. Three
of four questions were unanswerable by construction.${pending:-0} turned “unreadable” into “0”, and 0 passes. Absence of a measurement
became a clean bill of health.The kernel already knew which it was — cat /sys/block/sda/queue/rotational. Fix: branch on
rotational, read the right attribute set per class, and route every expected-but-unreadable
attribute into a named Could not be read block that changes the headline from ✅ to ⚠️.
sda Samsung SSD 870 EVO 50 0.0y 37°C SSD wear=0% rsvd=0 uncorr=0 crc=0 written=0.13 TB PASSED
Leading-zero trap: smartctl’s VALUE column returns
063,099. Bash reads a leading zero as octal, so$((100 - 099))dies with “value too great for base 8”. Force base 10:$((10#$x)).
Adding attribute 199 immediately found something real: sdd (a USB-enclosure drive) carried
one lifetime CRC error. The check fired 🔴 correctly — and would have fired 🔴 every single
run from then on, because CRC is a cumulative counter that never resets.
Alert fatigue kills a channel faster than no monitoring at all. A daily red alert about a seven-month-old event trains you to ignore the channel that also carries real drive failures.
Fix — alert on growth, not on lifetime totals. Baselines are stored in
~/.smart-health-crc.state, keyed by drive serial (device letters shift between boots):
S6PXNL0W611817D 0
979TDLTAS 0
979TG3KAS 0
WD-WXW1E753CP2L 1
CRC errors rose 1 → 2 since the last check <- signal, alerts
1 lifetime CRC error, unchanged <- noise, reported once then silent
The distinction generalises: interface counters get delta alerting; media-damage counters (reallocated / pending / uncorrectable) keep absolute thresholds, because any non-zero there is real damage whenever it happened.
--dry-run deliberately writes no baseline, so testing never mutates real state — and the note
says so rather than claiming “recorded” when it recorded nothing.
smartctl lives in /usr/sbin, which is not in cron’s default PATH (/usr/bin:/bin).
Under cron the script would have exited with “smartctl not installed” into a cron mail nobody
reads.
Fixed inside the script rather than the crontab, so it behaves identically from cron, a shell, or ssh:
export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Test under real cron conditions — root and restricted PATH — before trusting it:
sudo env -i PATH=/usr/bin:/bin /home/aztechguy/.local/bin/smart-health-check.sh --dry-run
A daily check that only speaks on problems is indistinguishable from a broken one. Silence could mean “drives are fine” or “the job hasn’t run since March.”
The Sunday digest is proof of life. If it stops arriving, the monitoring is broken — and
that is information you want. The same reasoning applies to the ~/drive-health.log entries,
which record every run whether or not anything was posted.
The NAS is not covered. smartctl isn’t installed on UGOS, and SSH is deliberately disabled
by default there — enabled manually for 2-hour windows behind MFA. Automated polling would mean
weakening a control chosen on purpose. Use the NAS’s own built-in drive notifications instead.
Current NAS drive ages, read manually on 2026-08-09: IronWolf 12TB 2.4 y, Red Pro 10TB 4.0 y, Red Pro 6TB 5.6 y — all zero reallocated, zero pending, SMART PASSED.