A Long Day’s Journey Into Backups – Rachel Kelly
- A journey in four stages
- Version 1 – State of Backups Aug 2017
- Needed to look though old logs to see how far back problem wentDaily diffing from duply/duplicity – But where was the original?
- Tried to restore data from old backups
- Couldn’t restore from backup since original was too old
- Couldn’t get it to work, needed something new fast
- Version 2
- Created tarball, uploaded to AWS S3, via daily cron
- Done quickly
- Not reliable sending to s3
- Needed ongoing work
- Big Win: But at least complient, and we could get data back out
- Try 3 : Shiny EFS
- EFS is AWS’s NFS solution
- tarball created on local EFS disk, easypeasy
- Big Win: Reliable backups, incorporated into infrastructure, retrievable data. 8 weeks off backups
- Miscalculated cost: About 10x original estimate
- Try 4: Tarball to s3 redux
- Tarball to s3. PLUS infrastructure
- Would work going forward
- S3 approx 1/10 the cost of EFS
- Big Win: Reliable, inexpensive, functional, dependable
- Discovered that EFS not encrypted
- Able to manage well
- Current Solution
- cron job calls a bash script
- Safety in bash: set -euo pipefail
- tar up the filesystem (with exclusions)
- Set it to s3 with aws cli (with hosts’s hostname)
- After 56 days is sent to glacier
- restore script to get a file (uses AWS credientials of current instance)
- What’s Next?
- I could work on backups forever and they will never be perfect
- Ability to restore to a new instance
- Want to be able to get files from anywhere
- Microservice backups
- Deglacierisation
- What we need
- Better CLI tool – safer
- What I really really want
- A decent enterprise solution
- NIH is a dangerous habit
- Speaker knows no one using a hosted enterprise solution
- Vendor solutions seem to be crazy overkill
- Feedbacks
- Amanda Recomended
- Every morning restore dev enviroment from anon prodouting one. Comet Backup
- Wasabi compatable with s3
- Recoment “Retrospec”
- tarsnap
- bacula
- rsnapshot
- Looking at Borg Back
- rsync to zfs, zfs send
- Personal backups, using duplicity
- Industry 4.0
- After mechanization, mass production and automation
- The machines run everything and we reap all the benefits – maybe
- Robot Hype
- Post AI winter
- Implementoer’s bias (top down design of neutral network setup, choose number of layers etc)
