Projects online backup

Projects Backup — Setup Guide

1. Install 7-Zip (if not already installed)

sudo apt update
sudo apt install p7zip-full

2. Create the password file

mkdir -p ~/.config/backup
nano ~/.config/backup/backup.pass
# type your password on a single line, save, exit
chmod 600 ~/.config/backup/backup.pass

Verify it’s locked down:

ls -l ~/.config/backup/backup.pass
# should show: -rw------- 1 youruser youruser

3. Edit the script

Open backup-projects.sh and set the three paths at the top:

  • SOURCE_DIR — your actual Projects folder path
  • DEST_DIR — the backup folder on your secondary drive
  • PASSFILE — path to the password file from step 2

4. Make it executable and move it into place

chmod +x backup-projects.sh
sudo mv backup-projects.sh /usr/local/bin/backup-projects.sh

5. Test it manually first

sudo -u YOURUSER /usr/local/bin/backup-projects.sh

Check that the .7z file shows up in DEST_DIR, and confirm you can open it:

7z t /mnt/secondary/Backups/projects-<date>.7z
# it will prompt for the password — test that it actually decrypts

6. Set up the cron job

Edit your user’s crontab (NOT root’s, so it has normal access to your home dir/password file):

crontab -e

Add this line to run nightly at 1:00 AM:

0 1 * * * /usr/local/bin/backup-projects.sh >> /home/YOURUSER/backup-projects.log 2>&1

7. Connect it to MEGA

MEGA’s sync client only watches the folders you’ve explicitly added (Desktop, Documents, Downloads, Music, Pictures per your setup). Your secondary drive backup folder isn’t in that list by default, so you have two options:

  • Preferred: Open the MEGA app → Preferences → Syncs → add /mnt/secondary/Backups as an additional synced folder.
  • Alternative: Symlink the backup folder into an already-synced location, e.g.: 
    ln -s /mnt/secondary/Backups ~/Documents/ProjectBackups
    (Note: some sync clients don’t follow symlinks — test this before relying on it. Adding the folder directly in MEGA’s sync settings is more reliable.)

Testing / restoring the archive

Because the script uses -mhe=on (encrypted headers — file names are hidden, not just contents), GUI archive tools generally can’t open this file. GNOME Archive Manager, the 7-Zip desktop app, etc. will either error out or show an empty/unreadable listing. This is expected and not a sign of corruption. Always use the 7z command line for this archive.

Verify integrity (no extraction, just checks the archive is intact):

7z t /mnt/secondary/Backups/projects-<date>.7z

You’ll be prompted for the password. A healthy archive ends with Everything is Ok.

Extract / restore to a scratch folder:

7z x /mnt/secondary/Backups/projects-<date>.7z -o/tmp/restore-test

Prompts for the password, then extracts the full folder tree into /tmp/restore-test.

Extract to the original location (overwrites in place — be careful):

7z x /mnt/secondary/Backups/projects-<date>.7z -o/home/YOURUSER/Projects -aoa

-aoa = overwrite all existing files without asking. Omit it if you want to be prompted per file instead.

Automated weekly verification (added 2026-07-12)

The “test restores periodically” note below used to be a manual chore. It is now automated by verify-backups.sh, which runs the drill every week and alerts if anything is wrong. It is strictly read-only — it never creates, deletes, or modifies a backup, only tests the newest one.

Each run, in order (any hard failure → red Discord alert + exit 1):

  1. an archive exists on the secondary drive;
  2. the newest one is younger than 26h (catches a nightly backup that silently stopped running);
  3. it is at least 10 MB (catches an empty/truncated archive);
  4. 7z t integrity test passes — real CRC decompression of every file, using the password;
  5. a sentinel path (Projects/shared_trading_lib) actually exists inside — catches the nasty case where the archive is intact but doesn’t contain the real tree (e.g. backed up the wrong or an empty folder).

A soft yellow warning (still passes) fires if the newest archive shrank more than 50% versus the previous one — an early signal that the backup started capturing less than it should.

  • Script: /home/aztechguy/.local/bin/verify-backups.sh (in user bin, not /usr/local/bin, so it runs as your user with normal access to the password file — same reasoning as running the backup from your crontab, not root’s)
  • Log: /home/aztechguy/backup-verify.log
  • Schedule: Sundays at 8:00 AM via user crontab, with --always-notify so a green “backups verified” confirmation lands in Discord once a week (so you know it ran, not just silence). Runs without the flag are silent unless something fails.
  • Manual run any time: verify-backups.sh --always-notify
  • Note: the current archive destination in the live script is /media/aztechguy/2a3a6a06-.../projects-backup/, not the /mnt/secondary/Backups placeholder used in the examples above.

Discord webhook configuration (segregated 2026-07-12)

Alerts are split across three Discord channels so a backup failure never hides in trading noise. All three URLs live in one place: ~/.config/discord-webhooks/ (dir 700, files 600).

File Feeds How the consumer reads it
stocks.txt all trading bots (options, DIA, QQQ, 0DTE, Portfolio 2), plus daily check-in, watchdog, fill logger the bots read discord_webhook from their own state.json; stocks.txt is the source of truth, synced to those files by apply-stocks-webhook.sh
calendar.txt run-buffer-automation.sh (Sniffspot/Swimply calendar) symlink to ~/.config/backup/discord_webhook.txt, which that script already reads live each run
backups.txt verify-backups.sh read live each run (falls back to the stocks webhook only if the file is missing, so a config slip never silences a real alert)

To rotate a webhook:

  • Backups or calendar: just edit the file (backups.txt, or calendar.txt). Read live next run — no other step.
  • Stocks: edit stocks.txt, then run apply-stocks-webhook.sh to push it into all five trading state.json files (it backs up each one first). Run it outside market hours (not 6 AM-1 PM PT weekdays) so it doesn’t race a bot writing its own state.

Current segregation: stocks and calendar each already have their own webhook; backups gets a dedicated one (paste it into backups.txt). Helper scripts live in ~/.local/bin/.

Notes / things worth knowing

  • Password loss = data loss. There’s no recovery if you forget it. Consider keeping a copy in a password manager (not on the same machine).
  • Test restores periodically. An untested backup is a hope, not a backup. Every month or so, run the 7z t (or 7z x) command above and confirm it actually opens.
  • Retention is 30 days (RETENTION_DAYS in both scripts, raised from 7/14 on 2026-08-09). The limit was never disk space — ~144 MB per Projects archive is ~4 GB for a month against 665 GB free. It’s detection lag: 7 days only protects you from problems you notice within a week, and if a file is silently corrupted then faithfully backed up for eight nights, every copy you hold is already bad. Problems on this box have surfaced far later than a week (a 509-day-old ext4 error; automation scripts unbacked-up for a month).
  • How the layers actually stack. Local retention is the only thing setting history depth — MEGA is a mirror, so when retention deletes an archive locally the sync deletes it there too. Those are not missing; they were deleted upstream. MEGA’s REWIND (30 days) is what recovers them after that. So the real window is 30 days live + 30 days recoverable. MEGA versioning is irrelevant to these archives — each night writes a new filename and nothing is ever overwritten. Turn versioning on for a synced Home folder, where files really do get edited in place; it does nothing here.
  • The archive count wobbles by one. find -mtime +7 counts whole 24-hour blocks and truncates, and the sweep runs after the new archive is built — so a file sitting within minutes of the boundary survives an extra night and is swept the next one. Expect “N or N+1”, not exactly N.
  • The log file (~/backup-projects.log) will grow forever; if that bothers you, add logrotate config or just periodically truncate it.