Memoir of a small-city mayor who grew up gay in Indiana. Timed to come out for his presidential run in 2019. Nice enough read with a good mix of stories. 3/5
James Bond investigates the mysterious industrialist Hugo Drax and his nuclear missile project which is vital to Britain’s security. Exciting and well written. 3/5
A series of stories about individuals, not just about the Internet but about women and early computing, hypertext, etc. Interesting and well written. 3/5
Lewis interviews people involved in the Obama to Trump transition at 3 major government agencies. He profiles the people, their jobs and in most cases how the Trump people underestimated the Dept’s importance. 3/5
Mostly a stats dump with a few profiles and accounts of struggling millennials sprinkled in. With a weird tone shift to boomer-love in the last chapter. Okay I guess 3/5
A first-hand account of a training class in 1974/75 where only 11 of the 71 starters graduated. Fun read although some interviews with non-graduates would have provided a contrast. 3/5
Three Laws of Nature: A Little Book on Thermodynamics by R Stephen Berry
Science mixed in with some history, designed for those with minimal science. The equations were simple but numerous & didn’t work in audiobook format. Try the printed version. 2/5
A detailed account of the film’s making from pre-production though to the bad reviews of the first release. Covers most aspects of the film and people involved. 4/5
Pulitzer Prize winning story of a team creating a new model of minicomputer in the late-1970s. Good portraits of the team members and aspects of the tech. 4/5
I’ve recently moved my home backups over to restic . I’m using restic to backup the /etc and /home folders and on all machines are my website files and databases. Media files are backed up separately.
I have around 220 Gigabytes of data, about half of that is photos.
My Home setup
I currently have 4 regularly-used physical machines at home: two desktops, one laptop and server. I also have a VPS hosted at Linode and a VM running on the home server. Everything is running Linux.
Existing Backup Setup
For at least 15 years I’ve been using rsnaphot for backup. rsnapshot works by keeping a local copy of the folders to be backed up. To update the local copy it uses rsync over ssh to pull down a copy from the remote machine. It then keeps multiple old versions of files by making a series of copies.
I’d end up with around 12 older versions of the filesystem (something like 5 daily, 4 weekly and 3 monthly) so I could recover files that had been deleted. To save space rsnapshot uses hard links so only one copy of a file is kept if the contents didn’t change.
I also backed up a copy to external hard drives regularly and kept one copy offsite.
The main problem with rsnapshot was it was a little clunky. It took a long time to run because it copied and deleted a lot of files every time it ran. It also is difficult to exclude folders from being backed up and it is also not compatible with any cloud based filesystems. It also requires ssh keys to login to remote machines as root.
Getting started with restic
I started playing around with restic after seeing some recommendations online. As a single binary with a few commands it seemed a little simpler than other solutions. It has a push model so needs to be on each machine and it will upload from there to the archive.
Restic supports around a dozen storage backends for repositories. These include local file system, sftp and Amazon S3. When you create an archive via “restic init” it creates a simple file structure for the repository in most backends:
You can then use simple commands like “restic backup /etc” to backup files to there. The restic documentation site makes things pretty easy to follow.
Restic automatically encrypts backups and each server needs a key to read/write to it’s backups. However any key can see all files in a repository even those belonging to other hosts.
Backup Strategy with Restic
I decided on the followup strategy for my backups:
Make a daily copy of /etc, /home and other files for each machine
Keep 5 daily and 3 weekly copies
Have one copy of data on Backblaze B2
Have another copy on my home server
Export the copies on the home server to external disk regularly
Backblaze B2 is very similar Amazon S3 and is supported directly by restic. It is however cheaper. Storage is 0.5 cents per gigabyte/month and downloads are 1 cent per gigabyte. In comparison AWS S3 One Zone Infrequent access charges 1 cent per gigabyte/month for storage and 9 cents per gigabyte for downloads.
What
Backblaze B2
AWS S3
Store 250 GB per month
$1.25
$2.50
Download 250 GB
$2.50
$22.50
AWS S3 Glacier is cheaper for storage but hard to work with and retrieval costs would be even higher.
Backblaze B2 is less reliable than S3 (they had an outage when I was testing) but this isn’t a big problem when I’m using them just for backups.
Setting up Backblaze B2
To setup B2 I went to the website and created an account. I would advise putting in your credit card once you finish initial testing as it will not let you add more than 10GB of data without one.
I decided that for security I would have each server use a separate restic repository. This means that I would use a bit of extra space since restic will only keep one copy of a file that is identical on most machines. I ended up using around 15% more.
For each machine I created an B2 application key and set it to have a namePrefix with the name of the machine. This means that each application key can only see files in it’s own folder
On each machine I installed restic and then created an /etc/restic folder. I then added the file b2_env:
The “source” command loads in the api key and passwords.
The restic backup lines do the actual backup. I have restricted my upload speed to 20 Megabits/second . The /etc/restic/home_exclude lists folders that shouldn’t be backed up. For this I have:
as these are folders with regularly changing contents that I don’t need to backup.
The “restic forget” command removes older snapshots. I’m telling it to keep 6 daily copies and 3 weekly copies of my data, plus at least the most recent 5 no matter how old then are.
This command doesn’t actually free up the space taken up by the removed snapshots. I need to run the “restic prune” command for that. However according to this analysis the prune operation generates so many API calls and data transfers that the payback time on disk space saved can be months(!). So I only run the command approx once every 45 days. Here is the code for this:
prune_run() {
echo "Running restic Prune"
/usr/local/bin/restic prune --cleanup-cache --max-unused 20%
echo " "
touch /etc/restic/last_prune_b2
echo "Updating restic if required"
echo " "
/usr/local/bin/restic self-update
}
prune_check() {
if [[ ! -f /etc/restic/last_prune_b2 ]]; then
touch -d "2 days ago" /etc/restic/last_prune_b2
fi
if [[ $(find /etc/restic/last_prune_b2 -mtime -30 -print) ]]; then
echo "Last backup was less than 30 days ago so wont run prune"
echo " "
else
echo "Chance of running prune is 1/30"
RANDOM=$(date +%N | cut -b4-9)
flip=$((1 + RANDOM %30))
if [[ $flip = 15 ]]; then
prune_run
fi
fi
}
prune_check
Setting up sftp
As well as backing up to B2 I wanted to backup my data to my home server. In this case I decided to have a single repository shared by all the servers.
First of all I created a “restic” account on my server with a home of /home/restic. I then created a folder /media/backups/restic owned by the restic user.
I then followed this guide for sftp-only accounts to restrict the restic user. Relevant lines I changed were “Match User restic” and “ChrootDirectory /media/backups/restic”
On each host I also needed to run “cp /etc/ssh/ssh_host_rsa_key /root/.ssh/id_rsa ” and also add the host’s public ssh_key to /home/restic/.ssh/authorized_keys on the server.
Then it is just a case of creating a sftp_env file like in the b2 example above. Except this is a little shorter:
For backing up my VPS I had to do another step since this couldn’t push files to my home. What I did was instead add a script that ran on the home server and used rsync to copy down folders from by VPS to local. I used rrsync to restrict this script.
Once I had a local folder I ran “restic –home vps-name backup /copy-of-folder” to backup over sftpd. The –host option made sure the backups were listed for the right machine.
Since the restic folder is just a bunch of files, I’m copying up it directly to external disk which I keep outside the house.
Parting Thoughts
I’m fairly happy with restic so far. I don’t have not run into too many problems or gotchas yet although if you are starting up I’d suggest testing with a small repository to get used to the commands etc.
I have copies of keys in my password manager for recovery.
There are a few things I still have to do including setup up some monitoring and also decide how often to run the prune operation.
24 Lectures about various aspects of Franklin and his life. Each lecture is on a theme so they are not chronological. I hadn’t read any biographies previously but this might help. 4/5
3rd book in the Lady Astronaut series. Mostly concerned with trying to find and stop agents sabotaging the Moonbase. Works well and held my interest. 3/5
A collection of long New Yorker articles from the 1960s. One on a stock corner even has parallels with Gamestop in 2021. Interesting and well told even when dated. 3/5
James Bond takes on Gangster/Agent/Voodoo leader ‘Mr Big’ in Harlem, Florida and Jamaica. The racial stereotypes are dated but could be worse. The story held my interest. 3/5
An account of the fire that killed a five-person firefighter crew. Minute by minute of the fire itself, plus the investigation and the trial of the arsonist. 4/5
The unexpected connections between creativity and mess. Lots of examples although as one commentator noticed most of them were from people already masters not beginners. 3/5
A book on how the most famous and successful are often there because of their upbringing, practice or chance events pushed them to the top rather than just raw talent. 4/5
How the latest research that reveals the extent to which behaviors once thought exclusively human are also found in other species. Spoiler: except Culture. 3/5
The author is thrown into the war as a 19 year old officer in command of 4 tanks 5 days after D-Day. Very well written and lots of detail of the good and the bad. 4/5
Lots of examples of how people are wrong about usually crime rates or levels of immigration. Divided into topics with some comments on why and how to fix. 3/5
A how-to on rebooting civilization following a worldwide disaster. The tone is addressed to a present-day person rather than someone from the future which makes it more readable. 4/5
Almost solely devoted to America it devotes sections to major events around the metal including it’s demonetization, government and private price manipulation and speculation including the Hunt Brothers. 3/5
About half the length of the other books in the series and published posthumously. Laura and Almanzo try to make a success farming for 4 years. Things don’t go well. The book is a bit more adult than some of the others 3/5
Interesting how close it is to the 2006 Movie. Also since it is set in ~1951, World War 2 looms large in many places & most characters are veterans. Very good and fairly quick read. 4/5
An account of the failed airborne operation. Mostly a day-by-day & sources including interviews with participants. A little confusing without maps. 4/5
“The definitive history of American policy on nuclear war”. Lots of “War Plans” and “Targeting Policy” with back and forth between service factions. 3/5
“Combines elements of memoir from Johnson with the history and science of attempts to discover life on Mars”. I liked this book a lot, very nicely written and inspiring. 4/5
Each year I do the majority of my Charity donations in early December (just after my birthday) spread over a few days (so as not to get my credit card suspended).
I also blog about it to hopefully inspire others. See: 2019, 2018, 2017, 2016, 2015
All amounts this year are in $US unless otherwise stated
My main donations was $750 to Givewell (to allocate to projects as they prioritize). Once again I’m happy that Givewell make efficient use of money donated. I decided this year to give a higher proportion of my giving to them than last year.
Software and Internet Infrastructure Projects
€20 to Syncthing which I’ve started to use instead of Dropbox.
A series of short chapters arranged in seasonal sections on Paris, People, the Author’s life and the French Revolutionary Calendar. Plenty of Interest. 3/5
Covering Laura’s short time as a schoolteacher (aged 15!) and her courting with husband-to-be Almanzo. Most action is in the first half of the book though. 3/5
In depth chapters on things the Walkman, Game Boy and Hello Kitty traces Japans rise first in hardware and then in cultural influence. Excellent story 4/5
A conflict-reporter memoir of her life and career. Based mainly in Moscow, Baghdad, and Beirut she especially goes into detail of her missions into Syria during it’s civil war. 3/5
Various incidents with 15 year old Laura now studying to become a school teacher while being courted. The family farm progresses and town grows. 3/5
Bold They Rise: The Space Shuttle Early Years (1972-1986) by David Hitt and Heather R. Smith
Covering up to and including the Challenger Disaster. Largely quotes from astronauts and people involved. Interesting seeing how missions quickly went to routine. 3/5
A history of the forces that led to the Atlas program from the end of the War to 1954. Covers a wide range of led-up rocket programs, technical advances and political, cold-war and inter-service rivalries. 3/5
A memoir of the author’s life and career from 16 to her mid 20s. Mixture of story, information about meat (including recipes), the butchery trade and meat industry. 3/5
A history of the iPhone and various technologies that went into it. Plus some tours of components and manufacturing. No cooperation from Apple so some big gaps but does okay. 4/5
Lots of examples of where Maths went wrong. From Financial errors and misbuilt bridges to failed game shows. Mix of well-known and some more obscure stories. Well told. 4/5
