Simon Lyall's Blog – Page 24 – New Zealand, Sysadmin, Linux, Curry, Chess

Feeds I follow: Citylab, Commitstrip, MKBHD, Offsetting Bahaviour

I thought I’d list of the feeds/blogs/sites I currently follow. Mostly I do this via RSS using Newsblur.

MKBHD : Quality Tech Videos (RSS) – Marques Brownlee is a youtube tech reviewer. He puts out a video every week or two (probably more soon since he recently graduated and has more time). Mostly covers phones and phablets but also laptops and other tech. The videos are beautifully filmed (thats the HD part) and mostly 5-10 minutes long. A great way to keep up with the newest stuff, I do disagree with his love of phone speakers though.
CityLab (RSS) – Originally part of the The Atlantic magazine this not has been spun off to it’s own website. Looks of stores about design, transport, issues and other stuff for city-dwellers. Really a huge range of stories. eg “America’s Egg Shortage Threatens Austin’s Breakfast Taco Supply” , “The Peculiar Inequality of Singapore’s Famed Public Housing” and “Tattooed New Yorkers Report All Kinds of Nasty Skin Conditions“.
CommitStrip : The blog relating the daily life of web agencies developers (RSS) – A webcomic about coders (and others). A bit more realistic (in both technology and “reality) than Dilbert and others.
Offsetting Behaviour (RSS) – Blog by economists Eric Crampton (mostly) and
Seamus Hogan. Covers a wide-range of topics with a lot of NZ orientated content. Especially some Christchurch Earthquake orientated posts. A little less interesting since Eric moved to Wellington to be part of a right-wing think tank. Tends to back up statements with references!

Reading the Lord of the Rings aloud

The reading project that I am working on is a re-read of the Lord of the Rings. I’ve read the book/trilogy around a dozen times over the years but the two main differences this time are that I am reading it aloud and that I am consulting a couple of commentaries as I go. The references works I am using are The Lord of the Rings: A Reader’s Companion and the The Lord of the Rings Reread series by Kate Nepveu. The Companion is a fairly large book (860 pages) that follows the text page by page and gives explanations for words, characters and the history/development of the text. These can range from a simple definition to a couple of pages on a specific topic or character. The reread has a quick synopsis at the start of the article for each chapter and then some commentary by Kate followed by some comments from her readers (which I usually only quickly skim).

I started my read-aloud on February 15th 2015 and I am now ( April 7th ) just past the half-way point ( I completed The Fellowship of the Ring on March 27th) . My process is to read the text for 30-60 minutes ( I’m reading the three-book 1979 3rd edition paperback edition, which amusingly has various errors that the Reader’s Companion points out as I go) which gets me though 5-10 pages. I read aloud everything on the page including chapter titles, songs, non-English words and footnotes. A few times I have checked the correct pronunciation of words ( Eomer is one ) but otherwise I try not to get distracted. Once I finish for the session I open the Reader’s Companion and check the entries for the pages I have just read and at the end of each chapter ( chapters are usually around 20-30 pages) I have a look at Kate’s blog entry. I try an read most days and sometimes do extras on weekends.

One thing I really need to say is that I really am enjoying the whole thing. I love the book (like I said I’ve read it over a dozen times) and reading it aloud makes the experience even better. The main difference is that I do not skip over words/sentences/paragraphs which tends to happen when I read normally. So I don’t miss phrases like the description of Lake Hithoel:

The sun, already long fallen from the noon, was shining in a windy sky. The pent waters spread out into a long oval lake, pale Nen Hithoel, fenced by steep grey hills whose sides were clad with trees. At the far southern end rose three peaks. The midmost stood somewhat forward from the others and sundered from them, an island in the waters, about which the flowing River flung pale shimmering arms. Distant but deep there came up on the wind a roaring sound like the roll of thunder heard far away.

Nor do I skip the other little details that are easy to miss, like Grishnakh and his Mordor Orcs leaving the rest of the group for a couple of days on the plains of Rohan or the description of country leading up to the west gate of Moria. Although I do wish I’d seen the link to the map of Helm’s Deep halfway down this page before I’d read the chapter as it would have made things clearer. The Companion is also good at pointing out how things fit in the chronology, so when somebody gazes at the horizon and sees a cloud of smoke it will say what event elsewhere in the book (or other writing) that is from. You also get a great feel for Tolkien’s language and words and his vivid descriptions of scenes and landscape (often up to a page long) such the example above. Although I do find he uses “suddenly” an awful lot when he has new events/people break into the narrative.

The readers companion is a great resource, written by two serious Tolkien scholars but intended for general readers rather than academics. Kate Nepveu’s articles are also very useful in giving a more opinionated and subjective commentary. I would definitely recommend the experience to others who are fans of the Lord of the Rings. I’m not sure how well it would work with other books but certainly it enhances a work I already know well and love.

At the current rate I am expecting to finish some time in June or July. The next project I’m planning is Shakespeare’s plays. I am planning on reading each one (multiple times including possibly at least once aloud) and watching the BBC Television Shakespeare and other adaptations and commentaries. My plan is that I’ll cover the majority of them but I’ll see how I go, However I’d like to at least get though the major ones.

Parallel Importing vs The Economist

For the last few years I have subscribed to the online edition of The Economist magazine. Previously I read it via their website but for the last year or two I have used their mobile app. Both feature the full-text of each week’s magazine. Since I subscribed near 15 years ago I have paid:

Launched Jun 1997   US$ 48
Jun 1999            US$ 48
Oct 2002            US$ 69
Oct 2003            US$ 69
Dec 2006            US$ 79
Oct 2009            US$ 79
Oct 2010            US$ 95
Oct 2011            US$ 95
Mar 2014            NZ$ 400 (approx US$ 300)

You will note the steady creep for a few years followed by the huge jump in 2014.

Note: I reviewed by credit card bill for 2012 and 2013 and I didn’t see any payments, it is possible I was getting it for free for two years 🙂 . Possibly this was due to the transition between using an outside card processor (Worldpay) and doing the subscriptions in-house.

Last year I paid the bill in a bit of a rush and while I was surprised at the amount I didn’t think to hard. This year however I had a closer look. What seems to have happened is that The Economist has changed their online pricing model from “cheap online product” to “discount from the printed price”. This means that instead of online subscribers paying the same everywhere they now pay slightly less than it would cost to get the printed magazine delivered to the home.

Unfortunately the New Zealand price is very high to (I assume) cover the cost of shipping a relatively small number of magazines via air all the way from the nearest printing location.

So readers in New Zealand are now charged NZ$ 736 for a two-year digital subscription while readers in the US are now charged US$ 223 ( NZ$ 293) for the same product. Thus New Zealanders pay 2.5 times as much as Americans.

Fortunately since I am a globe-trotting member of the world elite® I was able to change my subscription address to my US office and save a bunch of cash. However for a magazine that publishes the Big Mac Index comparing prices of products around the world the huge different in prices for the same digital product seems a little weird.

LInks: WW1 Maps, Shawshank, Microservices, Dev Interviewing

40 Maps that Explain World War One – A bunch of maps from all sorts of sources that explain different aspects of the origins and path of WW1. Huge rang of stuff.
On Interviewing as a Junior Dev – Liz Rush kept a record of her hunt for a job. Lots of anecdotes, advice and data.
The Little-Known Story of How The Shawshank Redemption Became One of the Most Beloved Films of All Time – Good long article about the making of the movie with quotes from those involved.
State of the Art in Microservices by Adrian Cockcroft (video 38min) – A good intro to Microservices and an overview of what is currently happening in the area. Slides are here

Books for Sale – Part 2

I’m doing a book clean-out. The following are all for sale. Remainders will be given away to charity or something. Pickup is from either my house (Dominion Rd/Balmoral, Auckland) or my I can meet during the week near my work in Wyndham Street in the Auckland CBD.

Prices as mark, discount if you want to by more than 5 or so. Links may not match the exact edition I am selling.

If you are interested in any please contact me via email ( simon@darkmere.gen.nz ) or over twitter ( @slyall ). Sale will run to end of April or so.

See Part 1 for more books

Business

Business @ The Speed of Stupid: Building Companies after the technology Shakeout by Dan Burke – $5
Fortune Strategy by Ean Higgins & Arun Abey – $2
What Management is : How it works and why it is everyone’s business (200 edition) Joan Magretta $10
The Company of the Future – Frances Caincross – $10
The mind of the CEO – Jeffrey Garten – $5

Commentary / Opinion / Speculation / Politics

The Pentagon’s New Map – Thomas P. M. Barnett – $5
Hot, Flat and Crowded by Thomas L. Friedman – $10
Radical Evolution by Joel Garreau – $10
Super Crunches by Ian Ayres – $10
The Waste makers by Vance Packard (1963 edition) – $4
The Hidden Persuaders by Vance Packard (1961 edition) – $4
Illiberal Education: The politics of race and Sex on Campus by Dinesh D’Souza – $10
Colossus: The rise and fall of the American Empire by Niall Ferguson $10
Antifragile: Things that gain from Disorder by Nassim Nicholas Taleb – $10

Technical

Voices from the Sky: Previews of the coming Space age (1966) by Arthur C Clarke – $8
Flight into Space: The facts about space exploration (1957) by John Norton Leonard – $4
Teach Yourself Physics by David Bryant – $4
The Subatomic Monster: Essays on Science by Isaac Asimov – $5
Building Service Provider Networks by Howard Berkowitz – $10
Homepage Usability: 50 websites Deconstructed by Jacob Neilsen – $10
Permissions A survival Guide: Blunt talk about Art as Intellectual Property – Susan Bielstein – $20
Who Owns Culture? Appropriation and Authenticity in American law by Susan Scafidi – $10

Travel / Misc

London Orbital : A walk around the M25 by Iain Sinclair – $10
Simon’s Cat (hardback) by Simon Tofield – $10
Hotel Babylon: Inside the Extravagance and Mayhem of a Luxury Fire-Star Hotel – Anonymous and Imogen Edward-Jones – $5
Raggle-taggle: Adventures with a fiddle in Hungary and Roumania (Hardcover 1935) – Walter Starkie – $20

Books for sale – Part 1

Prices as mark, discount if you want to by more than 5 or so. Links may not match the exact edition I am selling.

If you are interested in any please contact me via email ( simon@darkmere.gen.nz ) or over twitter ( @slyall ) Sale will run to end of April or so.

See Part 2 for more books

Science Fiction / Fantasy

Deryni Books by Katherine Kurtz, all paperbacks of used quality unless otherwise named.

Deryni Rising – $4
Deryni Checkmate – $4
High Deryni – $4
Camber of Culdi (2 copies) -$4 each
The Bishops Heir (Hardback, ripped jacket) – $4
The Quest for Saint Camber – $4
The Deryni Archives – $4

Science Fiction Short Story Collections

Apeman, Spaceman (Anthropological Science Fiction) – 1970 by Leon E. Stover (Editor), Carleton S. Coon (Foreword), Harry Harrison (Editor) – $4
Star Streak, Stories of Space Paperback – Edited Betty M. Owen – $4
Quark 1 – edited by Samuel R. Delany, Marilyn Hacker – $4
The Seventh Galaxy Reader Edited by Frederik Pohl – $4
Impossible Places– Short stories by Alan dean Foster – $4

Sci-Fi Novels

The Jesus factor By Edward Corley (poor quality paperback) – $2
Killer Pine – Lidsay Gutteridge – $4

Other Fiction

Florance of Arabia: A novel by Christopher Buckley – $10
Boomsday: A novel by Christopher Buckley (HardCover) – $10
The Third World War; The untold Story by John Hackett (Hardcover) – $5

History

The Way We Never Were: American Families And The Nostalgia Trap by Stephanie Coontz – $10
The Making of Modern Russia by Lionel Kochan – $4
Lenin’s Tomb: The Last Days of the Soviet Empire by David Remnick – $4
The Tunnels of Cu Chi: A Harrowing Account of America’s Tunnel Rats in the Underground Battlefields of Vietnam by Tom Mangold – $10
The IRA – Tim Pat Coogan (1970 Edition) – $5
Who Dares Wins: The Story of the SAS 1950-1982 by Tony Geraghty – $10
Guardians: Strategic Reconnaissance Satellites by Curtis Peebles – $5
Empire: How Britain made the Modern World by Niall Ferguson (big hardback with lots of photos) – $15
Safety in Numbers: The Mysterious world of Swiss banking by Nicholas Faith – $10
Centenary: 100 Years of All Black Rugby – R.H Chester – $20
What’s Gnu: A history of the Crossword puzzle by Michelle Arnot – $10

Static networking in Ansible the quick and dirty way

I’m in the process of setting up a server at home to replace an old one. I’m maintaining the new one via Ansible to try and get keep as tidy as possible. Part of the setup involves setting up a bridge interface so that I can run kvm virtual machines on the box.

In order to make the box a little more stable I decided to make the ethernet settings static rather than via DHCP. Unfortunately ansible doesn’t really have a nice standard way of setting up network ports (there are a few modules around but none in the main distribution). After looking around I decided just to make a simple ansible role to handle the files.

The machine is running centos7. The networking initially looked like:

/etc/sysconfig/network-scripts/ifcfg-enp2s0
::::::::::::::
HWADDR=9C:B6:54:07:E8:49
TYPE=Ethernet
BOOTPROTO=dhcp
NAME=enp2s0
ONBOOT=yes
#

I decided the easiest way was to just manually create and copy the files. So I created a static_networking role.

roles/static_networking/handlers/main.yml
roles/static_networking/files/grey/ifcfg-enp2s0
roles/static_networking/files/grey/ifcfg-bridge0
roles/static_networking/tasks/main.yml
roles/static_networking/tasks/setup-redhat.yml

Inside the tasks the main.yml just loads up the setup-redhat.yml which is:

---
- name: copy files if they are listed in var
  copy: src={{ ansible_hostname }}/ifcfg-{{ item }} dest=/etc/sysconfig/network-scripts/ owner=root mode=0644
  with_items: static_interfaces
  notify:
  - restart network

Which is fairly simple. It just goes though a list of “static_interfaces” for a host and copies these files from the local machine to the machine I am setting up. If the copy makes any changes it sends a notify.

For the machine “grey” I just create some entries in hosts_vars/grey.yml

static_interfaces:
 - enp2s0
 - bridge0

and then the files themselves:

roles/static_networking/files/grey/ifcfg-bridge0
::::::::::::::
DEVICE="bridge0"
ONBOOT="yes"
TYPE=Bridge
BOOTPROTO=static
IPADDR=10.1.1.28
NETMASK=255.255.255.0
GATEWAY=10.1.1.1
::::::::::::::
roles/static_networking/files/grey/ifcfg-enp2s0
::::::::::::::
DEVICE="enp2s0"
ONBOOT="yes"
NM_CONTROLLED="no"
BOOTPROTO="none"
BRIDGE=bridge0
HWADDR="9c:b6:54:07:e8:49"

which are the actual files to be copied. If any files are actually updated the handler will be triggered

roles/static_networking/handlers/main.yml 
---
# Called by "name" when network config files are changed
- name: restart network
  service: name=network state=restarted

Overall it seems to work and I only broke networking once (the ip on enp2s0 keep getting re-added until I forced network manager to forget about it). I wouldn’t really recommend this sort of thing for non-trivial sites though. Keeping per-site configs in roles isn’t really the best way to do things.

Linux.conf.au 2015 – Day 5 – Session 3

NoOps with Ansible and Puppet – Monty Taylor

NoOps
- didn’t know it was a contentious term
- “devs can code and let a service deploy, manage and scale their code”
- I want to change the system by landing commits. don’t want to “do ops”
- if I have to use my root access it is a bug
Cloud Native
- Ephemeral Compute
- Data services
- Design your applications to be resilient via scale out
- Cloud scale out, forget HA for one system, forget long-lived system, shared-nothing for everything. Cloud provides the hard scale-out/HA/9s stuff
- Great for new applications
OpenStack Infra
- Tooling, automation, and CI for the openstack project
- 2000 devs
- every commit is fully tested.
- each test runs on a single use cloud slave
- 1.7 million test jobs in the last 6 months. 18 TB of log data
- all runs in HP and rackspace public clouds
Create Servers manually at 1st
Step 1 – Puppet
- extra hipster because it is in ruby
- If you like ruby it is awesome. If don’t is it less-awesome
- collaboration from non-root users
- code review
- problem that it blows up when you try and install the same thing in two different places
- 3 ways to run. masterless puppet apply. master + puppet agent daemon . master + puppet agent non-daemons
Secret stuff that you don’t want into you puppet git repo
- hiera
Step 2 – Ansible for orchestration
- Control the puppet agent so it runs it nicely and in schedule and on correct hosts first
- Open source system management tool
- Sequence of steps not description of state like puppet
- ad-hoc operation. run random commands
- easy to slowly grow over time till it takes over puppet
- yaml syntax of config files
Step 3 – Ansible for cloud management
Ansible config currently mixed in with puppet under – http://git.openstack.org/cgit/openstack-infra/system-config/

Conference Closing

Steve Walsh wins Rusty Wrench award
Preview of Linux.conf.au 2016 in Geelong
- Much flatter than Auckland
- Deakin University – Waterfront Campus
- Waurn Ponds student accomadation 15 minutes with shuttles
- Feb 8th – 12th 2016
- CFP 1st of June 2015
- Theme “life is better with linux”
- 4 keynotes confirmed or in final stages of discussion, 2 female, 2 male
- NFS keytags
- lcabythebay.org.au
Announcement for Linux.conf.au 2017 will be in Hobart

Linux.conf.au 2015 – Day 5 – Session 2

When Everything Falls Apart: Stories of Version Control System Scaling – Ben Kero

Sysadmin at Mozilla looking after VCS
Primarily covering mercurial
Background
- Primarily mercurial
- 3445 repos (1223 unique)
- 32 million commits
- 2TB+ transfer per day
- 1000+ clones per day
- Biggest customer = ourselves
- tested platforms > 12
Also use git (a lot) and a bit of: subversion, CVS, Bazaar, RCS
2 * ssh servers, 10 machines mirror http traffic behind load balancer
1st story – know what you are hosting
- Big git repo 1.7G somebody asked to move off github
- Turned out to be mozilla git mirror, so important to move
- plenty of spare resources
- But high load straight away
- turned out to be mercurial->git converter, huge load
- Ran garbage collection – took several hours
- tweaked some other settings
2nd story
- 2003 . “Try” CI system
- Simple CI system (before the term existed or they were common)
- flicks off to build server, sends status back to dev
- mercurial had history being immutable up until v2.1 and mozilla was stuck on old version
- ended up with 29,000 brashes in repo
- Around 10,000 heads some operations just start to fail
- Wait times for pushes over 45 minutes. Manual fixes for this
- process was “hg serve” only just freezein gup, not any debug info
- had to attached debugging. trying to update the cache.
- cache got nuked by cached push, long process to rebuild it.
- mercurial bug 4255 in process of being looked at, no fix yet
The new system
- More web-scalable to replace old the system
- Closer to the pull-request model
- multi-homing
- leverage mercurial bundles
- stores bundles in scalable object store
- hopefully minimal retooling from other groups (lots of weird systems supported)
Planet release engineering @ mozilla

SL[AUO]B: Kernel memory allocator design and philosophy – Christopher Lameter

NOTE: I don’t do kernel stuff so much of this is over my head.
Role of the allocator
- page allocator only works in full page size (4k) and is fairly slow
- slab allocator for smaller allocation
- SLAB is one of the “slab allocators”
kmeme_cache , numa aware, etc
History
- SLOB: K&R 1991-1999 . compact
- SLAB: Solaris 199-2008 . cache friendly, benchmark friendly
- SLUB: 2008-today , simple and instruction costs count, better debugging, defrag, execution time friendly
2013 – work to split out common code for allocators
SOLB
- manages list of free objects with the space of free objects
- have to traverse list to find object of sufficient size
- rapid fragmentation of memory
SLAB
- queues per cpu and per node to track cache hotness
- queues for each remote node
- complete data structures
- cold object expiration every 2 seconds on each CPU
- large systems with LOTS of CPUs have huge amount of memory trapped, spending lots of time cleaning cache
SLUB
- A lot less queuing
- Pages associated with per-cpu. increased locality
- page based policies and interleave
- de-fragmentation on multiple levels
- current default in the kernel
slabinfo tool for SLUB. tune, modify, query, control objects and settings
can be asked to go into debug mode even when debugging not enabled with rest of the kernel
Comparing
- SLUB faster (SLAB good for benchmarks)
- SLOB slow
- SLOB less memory overhead for small/simple systems (only, doesn’t handle lots of reallocations that fragment)
Roadmap
- More common framework
- Various other speedups and features

Linux.conf.au 2015 – Day 5 – Session 1

How to get one of those Open Source jobs – Mark Atwood

Warns talk might still have some US-centric stuff still in it
“Open Source Job” – most important word is “Job”
- The Open Source bit means you are a bit more transferable than a closed-source programmer
- Don’t have to move to major tech city
Communication skills
- Have to learn to Write clearly in English
- Heave to learn how to speak, including in meetings and give some talks
- Reachable – Have a public email address
- Don’t be a jerk, reputation very important
Technical skills
- Learn how to program
- Start with python and javascript
- Learn other languages eg scale, erlang, clojure, c, C++
- How to use debugger and IDE
- Learn to use git well
- Learn how to code test (especially to work with CI testers like jenkins)
- Idea: Do lots of simple practise problems in programming using specific technique or language
Relationships & Peers
- Work with people remote and nearby
- stackoverflow
- Don’t be a jerk
Work
- Have to “do the work” then “get the job”
- Start by fixing bugs on a project
- Your skills will improve and others will see you have those skills
Collaborate
- Many projects use IRC
- Most projects have bug tracker
- Learn how to use the non-basic stuff in git
- Peer programming
Reputation
- Portfolio vs resume
- github account is your portfolio
- Need to be on social media, at least a little bit, most be reachable
Getting the Job
- If you have a good enough a rep the jobs will seek you out
- Keywords on github and linkedin will attract recruiters
- People will suggest you that apply
- Conferences like linux.conf.au
- Remember to counter-offer the offer letter
- Once you are working for them, work out what is job related an the company might have a claim on. make sure you list in your agreement any projects you are already working on
Health
- Don’t work longer than 40h a week regularly
- 60h weeks can only be sustained for a couple of weeks
- Just eat junk-food
- Don’t work for jerks
Money
- Startups – bad for your health. Do not kill yourself for a nickle, have real equity
Keep Learning
3 books to read
- Oh the palces you will go – Dr Seuss
- Getting things Done – David Allen
- How to fail at almost everything and still win big – Scott Adams

Pettycoin: Towards 1.0 – Rusty Russell

Problem it bitcoining mining is expensive, places lower limit on transaction fees
Took 6 months of to mostly work on pettycoin
Petty coin
- Simple
- gateway to bitcoin
- small amounts
- partial knowledge, don’t need to know everything
- fast block times
Altcoins – bitcoin like things that are not bitcoin
- 2 million posts to altcoin announce forum
- lots of noise to talk to people
review
- Paper released saying how it should have been done
- hash functions
- bitcoin blocks
- Bitcoin transactions
Sidechain
- alternative chains that use real bitcoins
- Lots of wasted work? – bitcoin miners can mine other chains at the same time
- too fast to keep notes
- Compact CVP Proofs (reduce length of block header to go all the way back )