Audiobooks – June 2024

Bush by Jean Edward Smith

A biography of President George W. Bush mostly concentrating on the Invasion and Occupation of Iraq. Openly hostile to the subject. 3/5

The Longest Campaign: Britain’s Maritime Struggle in the Atlantic and Northwest Europe, 1939–1945 by Brian E. Walter

A very good overview of the navel war that covers almost all aspects and actions. 4/5

Adventures in the Screen Trade by William Goldman

The author’s experiences and thoughts on screenwriting and the Hollywood movies business. Lots of interesting stories. 4/5

My Scoring System

  • 5/5 = Brilliant, top 5 book of the year
  • 4/5 = Above average, strongly recommend
  • 3/5 = Average. in the middle 70% of books I read
  • 2/5 = Disappointing
  • 1/5 = Did not like at all
Share

Audiobooks – May 2024

Oscar Wars: A History of Hollywood in Gold, Sweat, and Tears by Michael Schulman

The evolution of the awards sprinkled with lots of stories of campaigns and shows in a changing Hollywood. A fun read. 4/5

The Master Switch: The Rise and Fall of Information Empires by Tim Wu

A chronicle of the America’s Radio, Phone, Film and TV industries and how they all ended up as monopolies or cartels. 4/5

Nuclear War: A Scenario by Annie Jacobsen

A minute by minute account of a present-day nuclear war with other chapters explaining background to what is happening. Pretty good 4/5

The Shadow Puppet by Georges Simenon

After a businessman is robbed and murdered, Maigret is convince one of the residents of an adjoining apartment building is responsible. 3/5

My Scoring System

  • 5/5 = Brilliant, top 5 book of the year
  • 4/5 = Above average, strongly recommend
  • 3/5 = Average. in the middle 70% of books I read
  • 2/5 = Disappointing
  • 1/5 = Did not like at all
Share

Two Metro Rail lines Auckland should build

Introduction

In my previous article I covered why Light Metro is the best technology of the next major stage of Auckland’s train network. Here I present a couple of lines that could be the basis for a future network.

The lines are designed to form a mesh an enable transfers (especially in the CBD) but are of course just ideas. One problem I have encountered is steep sections of track, these will require the track to smooth our the height differences and for trains able to handle climbs of around 5%.

I am estimating costs as $300m/km for elevated sections and $1b/km for underground sections. Hence I have used elevated line wherever possible.

Light Metro Technology

As outlined in my previous article Light Metro is Automated, Grade Separated with Short Trains and High Frequencies. It is well suited to Auckland where requirements exceed Light Rail but a full metro would be overkill.

The key advantages of Light Metro over street running light rail is it’s high capacity, frequency and higher speed. Attempting to push Light Rail beyond it’s natural sweet-spot result in a grade-separated system that costs as much as Light Metro but is worse and often costs more to run.

The below table shows the capacity of a Light Metro line (in each direction). For Auckland the stations outside the CBD could be serviced by buses to further increase coverage area. Trains could start at short length and frequency increased as high as possible before longer trains should be used.

Headway / Trains per Hour2 Cars3 Cars4 Cars6 Cars
5 min / 12 tph2,4003,6004,8007,200
3 min / 20 tph4,0006,0008,00012,000
2 min / 30 tph6,0009,00012,00018,000
90 sec / 40 tph8,00012,00016,00024,000
Max Passengers per hour per direction

If the system is run with 4-car trains then each has the capacity over double one of the major Auckland motorways such as the Western or Southern.

Line 1 – A North/South Metro Line from Albany to the Airport

This line would upgrade the Northern Busway on the North Shore, run under the CBD and connect to the Airport in the South.

The line would be grade separated above the road as much as possible since this is cheaper than under-grounding. It would be underground though the central city however.

Total length would be around 36km of which around 5.5 would be underground. Cost would be something like $15b

Northern Section

This would start at the exiting Albany bus centre and follow the Northern busway to Akoranga station. It would then go along the shore until roughly opposite Sulphur point where it would either go in a tunnel or bridge over the Harbour to Wynyard Quarter. Stations would be Albany, Rosedale, Constellation, Sunnynook, Smales Farm, Akoranga

The Northern Busway should be kept South of Akoranga Station for use by buses from Takapuna, Northcote and Brikenhead. This would give the system more capacity and is easier than those people transferring from a bus to a train for such a short ride.

Travel time from Albany to the Te Waihorotiu Station (Aotea) should hopefully be around 25 minutes.

City Section

Once over the Harbour the line should head underground and have a series of stops in the Central City. I would suggest

  • Central Wynyard Quarter near Madden St
  • Near Les Mills on Victoria St West
  • Te Waihorotiu Station (Aotea)
  • University / Symonds St

The Te Waihorotiu CRL station is apparently already future-proofed with space for a North/South line. The station will effectively be the centre of the Auckland System. There should also be a surface Light Rail line nearby on Queen Street.

The University station would be quite deep and probably be a an elevator-only station.

Southern Section

South of Grafton Valley the line would go under the domain before going through Newmarket. The line could either be above or below ground though Newmarket but will be above ground once it reach Manukau Road.

I don’t think having a station for the Museum is justified but there could also be a station at the North of Newmarket near Sarawia St. There should be at least one station in Central Newmarket near the existing Train station to allow transfers

South of Newmarket the line will travel above Manukau Rd and continue South through Onehunga and Mangere Bridge.

Possible stations could be (at roughly 1km intervals):

  • Near corner Manukau and Great South Road
  • Corner of Manukau Rd and Ranfurly Road
  • Corner of Manukau Rd and Queen Mary Ave (Alexandra Park, Green Lane West Rd)
  • Corner of Manukau and Pah Roads
  • Royal Oak Mall
  • Corner of Manukau Rd and Trafalgar St
  • Onehunga Mall Road near Grey Street
  • Onehunga Station
  • Mangere Bridge Village
  • Corner of McKenzie and Millar Rd
  • Corner of Bader Dr and Idlewild Ave
  • Mangere Town Centre (see below)
  • Airport Drive Area
  • Airport Terminal

The Southern Section would have roughly 16 stations and take over 18km and would take around 30 minutes to cover from the Airport to Te Waihorotiu/Aoetea Station.

Previous proposals have followed the motorway but I’ve switched this to following roads inside the suburb of Mangere Bridge and giving the suburb 3 stations with the Millar Rd one having good connectivity to Favona.

The Mangere Town Centre station would be a branch off the Bader Drive station. It could be run as a shuttle. Eventually the line could be extended East along Buckland Road to Papatoetoe Station then North to Otara and/or South to Manukau

Line 2: North-West Metro Line from Westgate to the City

New line in yellow, existing rail line in blue

This line is intended to fill the gaps to the North of the existing Western Rail Line and use the Motorway corridor. Closer to town it will go above Great South Rd and Karangahape Rd.

It will then do an above-ground spiral around the city to improve coverage and transfers.

Total length would be around 20km and all above ground. Cost would be something like $6b

Western Section

This would run from Westgate to Karangahape Road mainly along the North Western Motorway and Great North Road (GNR). It would be roughly 16km long and 100% overhead.

Stops could be: Westgate Shopping Centre, Royal Road, Huruhuru Road, Lincoln Road, Te Atatu Road, Rosebank, Point Chevalier Shops, Zoo / MOTAT, Corner GNR & Bond St, Corner GNR & Williamson Ave, Corner GNR and Newton Rd, St Kevins Arcade.

City Section

The St Kevins Arcade stop on Karangahape Rd should be designed to allow people to easily transfer to either the Dominion Rd Light rail on Queen St or the Karanga-a-Hape CRL station.

After the St Kevins Arcade stop the line continues east along Karangahape Rd and then turns down Symonds Street, Anzac Ave, Customs Street and then across the Viaduct Basin to Madden street.

  • St Kevins Arcade
  • Symonds St near City Rd
  • Symonds St near the Engineering School
  • Symond St near Parliament St
  • Customs St near Britomart
  • Customs St West near Market lane
  • Madden St near Daldy St

The line has seven stations in the CBD and intersects all the other lines twice. This enhances the coverage of the other lines via transfers. Extra stations are also a lot easier and cheaper to build on this line than the underground lines.

eg Someone coming from the North Shore on the N/S Metro could get off at Wynyard Station and Transfer to the Western Metro Line. They would then only have to wait a couple of minutes to catch a train to the Britomart station.

Dominion Road Light Rail

This has been covered elsewhere in detail but building a Lower Queen St to SH20 surface Light rail line fills a gap in coverage and provides additional capacity along Queen Street fairly cheaply.

The line would be mostly separate from car traffic on dedicated lanes in the Center of Dominion Rd and Queen Street. Length would be 8km.

Followups Lines

The above two lines probably give Central Auckland significant metro coverage to last many years. Future lines in Ponsonby, Sandringham, Mt Eden, The CDB, and Newmarket would probably be best served by cheaper street running light rail.

Further out Light Metro may suit the longer distances. Lines or Branches like:

  • Te Atatu
  • Point Chev to Onehunga
  • Mangere to Papatoetoe
  • Papatoetoe to Otara and then on to Botany
  • Manukau to Papatotoe, Howick and Manurewa
  • Takapuna
  • Orewa

Areas like Remuera could use either technology or just retain bus-based feeders

Questions

Q: Why not Light Rail?

A: Street running light rail is suitable for many sections but it lacks the higher capacity and speed of Light Metro. This is need for long busy routes like the link to the North Shore. If you Grade Separate the Light Rail then you end up spending as much as Light Metro for an inferior product.

However Light Rail is suitable for many routes that don’t justify the extra speed/capacity. This includes Dominion Road and additional filler routes around the CBD that need a step-up from buses.

Q: Why not Heavy Rail?

A: A System compatible with New Zealand’s current service would not work. It would not be able to handle turns, climbs and automated operation without extremely expensive changes which would lose all compatibility. The existing routes (including the CRL) are already full so no savings via reuse is gained.

Q: Why elevated instead of tunneled?

A: Because it is cheaper. Cost for tunneled is usually at least twice that of overhead and can often be more. Yes, not everybody likes the look of overhead lines but going underground can increase the cost by enough to derail the project.

Q: What about steep sections?

A: Certain section of the lines are quite steep due to Auckland’s terrain. This may cause a problem with the route. Light Metro can handle steep slopes than Heavy Rail but handling it may require additional measure like altering the height of lines so they smooth out slopes.

Share

Revisiting Light Metro for Auckland

As of 2024 the Auckland Light Rail proposal has died. What started as a cheap surface light rail running down the centre of roads became a largely underground system and the cost spiral to at least $15 billion.

In this post I will outline why I think that medium-distance high-traffic routes should instead be built using Automated Light Metro technology.

In a followup post I will suggest a possible 2 line backbone for Auckland. Combining Light Metro with the existing heavy-rail system, some street-running light rail and buses this could be a system which could be built for less than the cost of the governments light-rail proposals and leave lots of room to be extended

Much as motorway project are planned years in advance this could be something that is largely designed and ready to be built when funding and political will are both available.

What is Automated Light Metro?

Automated Light Metro is what Wikipedia classifies as a Medium-capacity rail system , it has greater capacity than light rail but less than a Full Metro System.

For this article will be based the Hitachi Rail Italy Driverless Metro but there are similar systems from other vendors. The Hitachi Rail Italy Driverless Metro system is deployed in several metros including Copenhagen, Brescia, Milan and Taipei.

The main characteristics of the system are:

  • Automated (Driverless) Operation
  • Grade Separated – Mainly overhead
  • Short trains ( 30 – 100m )
  • Trains up to every few minutes (or less).
  • Large capacity of passengers per hour
  • Platform Screen Doors
  • Off the shelf Solution

Automated Operation

An important part of the system is that the trains are driverless. Only around 4 people in a control room are needed to run a whole system (which could be 100 trains). Automated operation allows trains to be scheduled close together and also makes it cheap to run lots of trains even off-peak. A train every 10 minutes at 1am for instance.

Grade Separation – Overhead vs Underground

Grade Separation means that there are no crossings of the track. It either runs underground or overhead. This means the track can’t be blocked and is basically required for automated operation.

I’m advocating that lines should be overhead whenever possible. The main advantage of this is lower cost (a third to a half of underground), speed of construction and in some cases easier access to stations.

While some people don’t like the look of overhead lines these will be running on existing motorway or road corridors. Spending billions to put the lines underground is a waste of money that can be used to build more tracks (or result in nothing being constructed at all due to cost)

I think people who worry about the look of overhead rail forget how ugly 4-5 lanes of car-filled roads really are. Not to mention the noise and fumes.

Compare these shots of the Vancouver Skytrain running alongside roads.

Some underground track will probably be required for the CBD but in light of the high extra cost it should be minimised.

Short Trains

Compared to the existing Auckland trains the Light Metro cars are around half the length. This means a 4-car Light Metro train is around 52m vs 145m for a 6 car NZ AM Class train. However the denser loading and greater frequency means that the capacity is around the same.

The shorter Light Metro trains allow for smaller and cheaper stations. Platforms can be smaller and lifts and stairs can be sized for smaller “waves” of passengers.

Very Frequent Trains

Automated Light Metro Systems can have very short intervals (headway) between trains. 90 seconds is available off-the-shelf which allows 40 trains/hour in each direction. This means that passengers are not sitting around waiting for trains, you don’t even have to time when you turn up. See the video below for an example of what 90 seconds between trains feels like.

Because the trains are automated the cost of running trains half as long and twice as often is about the same. Whereas with conventional trains you need another driver and better training for the driver to handle tighter tolerances.

High Capacity

As stated above an Automated Light Metro can typically handle a train every 90 seconds or 40 trains per hour. A system like Hitachi Rail Italy Driverless Metro consists of 13m cars that can carry up to 100 people each (1/3 seated). These can be arranged in 2,3,4 or 6 car trains.

Headway / Trains per Hour2 Cars3 Cars4 Cars6 Cars
5 min / 12 tph2,400 3,600 4,800 7,200
3 min / 20 tph4,0006,0008,00012,000
2 min / 30 tph6,0009,00012,00018,000
90 sec / 40 tph8,00012,00016,00024,000
Max Passengers per hour per direction

I would suggest the system be designed for 4 car trains ( 52m ) giving a maximum capacity of 400 people per train and 16,000 people per hour in each direction. That capacity would probably not be needed initially. So at the start 2 or 3 car trains could be run. Capacity can be increased by more frequent intervals and eventually longer trains.

Stations (especially the underground ones) should be even future-proofed to be upgradable to 6-car trains. But probably upgrade will be decades away. This means that relatively small and cheap stations can be built.

16,000 people/hour is equivalent to 10 lanes of cars, plus another 10 in the other direction. It exceeds the total number of people going South over the Auckland Harbour Bridge during the morning peak ( around 4500/h in buses and 9000/h in cars according to this article )

As a comparison the 66m Light Rail vehicles proposed for Dominion Road would be able to do around 8400 passengers per hour at a 3 minute headway. This would probably be the maximum for a street running system.

Platform Screen Doors

These are standard for all modern systems for safety.

Off the shelf Solution

A system such as Hitachi Rail Italy Driverless Metro is very much an industry standard. The system is operating successfully in several other cities around the world. This is much lower risk than going with a bespoke solution or unusual technology.

How does it compare to other options?

Advantages of Light Metro over Light Rail

The main advantage of Light Metro over Light Rail is capacity. While a basic street-running light rail is cheaper increasing that capacity beyond a certain point results in very long trains, cross roads being blocked and if full grade separation is introduced a big increase in cost to around what Light Metro would be.

Light Metro is also faster (see next section) and since it is a completed separated less prone to disruption.

However street-running light rail is cheaper and better suited for some routes that do not require high-capacity or speed and which have frequent stops. So routes covering local trips around Queen Street, Ponsonby and Parnell would be better suited to Light Rail.

Speed estimate

The Copenhagen Metro has an average speed (including stops) of 40km/h. This would point to a journey time of around 30 minutes from Westgate, The Airport or Albany to the CBD. The maximum speed is 80km/h so means that long sections could be practical. For example an 13km extension between Albany and Silverdale would take 10 minutes.

Partially Separated Light Rail usually averages at best than 30km/h and less than that in sections with many stops and/or crossings.

Relationship with exist Heavy rail

Light Metro will be to a different standard to the existing rail system. This is not a disadvantage since it will run on completely separate and new lines. The existing lines (including the CRL) are already at capacity in trains/hour so new lines will always be needed.

A system that is designed to be compatible with existing rail will probably unable to be automated and would have many other compromises.

More information.

Greater Auckland has pushed several articles about Light Metro previously

Share

Audiobooks – April 2024

The Time Traveller’s Guide to Regency Britain by Ian Mortimer

England 1789–1830 written as a guide on what a visitor would see and should know and do. Quite fun. 4/5

Charles III : The Inside Story by Robert Hardman

Covers the death of the Queen through to the first year of the new King’s reign. Semi-authorised. Recommended for those with interest in the topic. 4/5

Nuts and Bolts: Seven Small Inventions That Changed the World (in a Big Way) by Roma Agrawal

Cover the development of 7 inventions their spinoffs and refinements. An easy and interesting read. 4/5

My Scoring System

  • 5/5 = Brilliant, top 5 book of the year
  • 4/5 = Above average, strongly recommend
  • 3/5 = Average. in the middle 70% of books I read
  • 2/5 = Disappointing
  • 1/5 = Did not like at all
Share

Everything Open 2024 – Day 3 talks

Keynote: Intelligent Interfaces: Challenges and Opportunities by Aaron Quigley

  • Eye Tracking of the user
    • DiffDisplays – Eye tracking and when you looked away from a screen it frooze it. When you looked back it gave you a summary/diff of what you missed
    • Bought this down to the widget level, a widget got notification when user looking or away and could decide what to do
  • Change Blindness (different from attention blindness)
    • When phone far away simplify phone interface, more detail when closer
    • People don’t see details of displays slowly fading in and out as distance from display changed
  • Phone on table, screen up or screen down
    • SpeCam – Facedown screen can have light and detect what it is sitting on. Guess material it is sitting on
    • Accuracy same/better than a proper spectrometer
  • MicroCam – Phone placed with screen face up
    • Placement aware computing
  • OmniSense
    • 360 Camera
    • Track what the user’s whole body is doing
    • Tracks what is happening all around the user. Danger sensors, context aware output
  • BreathIn control. Breath pattern to control phone
    • User camera in a watch potion to detect handle gestures (looking at top/back of hand)
  • RotoSwype – Smart ring to do gesture keyboard input
  • RadarCat – Radar + Categorization
    • More Socially acceptable that cameras everywhere and always on
    • Used to detect material
    • Complex pattern of reflection and absorption that returns lots of information
    • Trained on 661 feature and 512 bins
    • Radar signal can ever detect different colours. Different dyes interact differently
    • Can detect if people are wearing gloves
    • Application – Scales at self-checkout supermarket to detect what is being weighed
    • Radar in shoe can recognise the surface and layers below (carpet on weed etc)

Passwordless Linux – Passkey and External IdP support in FreeIPA by Fraser Tweedale

  • Passwords
    • Users are diligent (weak reuse)
    • Using passwords securely imposes friction and cognitive load
    • Phishable
  • Objectives – Reduce password picking risks, phishing, friction,frequency of login
  • Alternatives
    • 2FA, Smartcard, Passkeys / WebAuthn, Web SSO Providers
  • 2FA
    • HOTP / TOTP etc
    • phishable
  • Smart Cards
    • Phishing Resistant
  • Passkeys
    • Better versions of MFA Cards
    • Phishing resistant
    • “passkey” term is a little vague
  • Web SSO
    • SAML, OAuth2
    • Using an existing account to authenticate
    • Some privacy concern
    • Keycloak, Redhat SSO, Okta, Facebook
    • Great on the web, harder in other context
  • What about our workstations?
    • pam has hooks for most of the above (Web SSO less common) or pam_sss does all
  • FreeIPA / Red Hat Identity Management
  • DEMO

Locknote: Who gets to work in STEM? And who is being left out? by Rae Johnston

  • Poor diversity affects the development of AI
  • False identification much higher by facial recognition for non-white people
  • Feed the AI more data sets?
  • Bias might not even be noticed if the developers are not diverse
  • Only around 25% of STEM people are Women
  • Only 15% of UK scientist came from Working Class backgrounds (35% of the population)
  • 11% of Australians don’t have access to affordable Internet or don’t use it.
  • The digital divide is narrowing but getting deeper. Increasing harder to function if you are not online
  • Male STEM graduates are 1.8x more likely to be in jobs that required the array than women. Mush worse for indigenous people

Lightning Talks

  • Creating test networks with Network Namespace
    • ip netns add test-lan
  • Rerap Micron
  • Haystack Storage System
    • Time-bases key/value store
  • AgOpenGPS
    • Self Steering System for Tractors
  • Common Network Myths
    • End to end packet loss is the only thing that matters
    • Single broadcast domain is a SPOF, broadcast storms etc
    • Ping and ICMP is your friend. Please allow ping
    • Don’t force 1500 MTU
    • Asymmetric routing is normal
    • non-standard port number doesn’t make you secure
  • radio:console for remote radio
  • WASM
    • FileSender – Share large datasets over the Internet
Share

Everything Open 2024 – Day 2 talks

Keynote: How Adversaries Use AI by Jana Dekanovska

  • Adversary
    • Nation States
    • Ecrime
    • Hactivism
  • Trends
    • High Profile Ecrime attacks – Ransomware -> Data extortion
    • Malware-Free Attacks – Phish, Social engineering to get in rather than malware
    • Cloud Consciousness
    • Espionage – Focuses in Eastern Europe and Middle East
    • Vulnerability Exploitation – Not just zero days, Takes while to learn to leverage vuls
    • Cloud Consciousness – Adversary knows they are in the cloud, have to operate in it.
  • Generative AI
    • Code Generation
    • Social Engineer – Help people sound like Native Speakers, improve wording
    • Prompt Injection
  • Big Four States sponsoring attacks – China, North Korea, Iran, Russia
  • North Korea – Often after money
  • Russia, Iran – Concentrating on local adversaries
  • China
    • 1m personal in Cyber Security
    • Get as much data as possible
  • Elections
    • Won’t be hacking into voting systems
    • Will be generating news, stories, content and targeting populations
  • Crime Operations
    • GenAI helps efficiency and Speed of attacks
    • Average Breakout time faster from 10h in 2018 to 1h now
    • Members from around the world, at leats one from Australia
    • Using ChatGPT to help out during intrusions to understand what they are seeing
    • Using ChatGPT to generate scripts

Consistent Eventually Replication Database by William Brown

  • Sites go down. Lets have multiple sites for our database
  • CAP Theorem
  • PostgresSQL Database
    • Active Primary + Standby
    • Always Consistent
    • Promote passive to active in event of outage
    • Availability
    • But not partition tolerant
  • etcd
    • Nodes elect active node which handles writes. Passive nodes go offline then others are still happy
    • If active node fails then new active node elected and handles writes
    • Not availbale. Since if only one node then it will go to sleep cause it doesn’t know state of other nodes (dead or just unreachable)
  • Active Directory
    • If node disconnected then it will just keep serving old data
    • reads and writes always services even if they are out of contact with other nodes
    • Not consistent
  • Kanidm
    • identity management database
    • Want availability and partition tolerance
    • Because we want disconnected nodes to still handle reads and writes (eg for branch office that is off internet)
    • Also want to be able to scale very high, single node can’t handle all the writes
  • Building and Design
    • Simultaneous writes have to happen on multiple servers, what happens if writes overlap. Changes to same record on different servers
    • ” What would Postgres do? “
    • Have nanosecond timestamps. Apply events nicely in order, only worry about conflicts. Use Lamport Clock (which only goes forward)
    • What happens if the timestamps match?
    • Servers get a uuid, timestamp gets uuid added to it so one server is slightly newer
    • Both servers can go though process in isolation and get the same outputted database content
  • Lots more stuff but I got lost
    • Attribute State + CRDT
  • Most of your code will be doing weird paths. And they must all be tested.
  • Complaint that academic papers are very hard to read. Difficult to translate into code.

Next Generation Authorisation – a developers guide to Cedar by Ricardo Sueiras

  • Authorisation is hard
  • Ceder
    • DSL around authorisation
    • Policy Language
    • Evaluation and Authorisation Engine
    • Easy to Analise
  • Authorisation Language

Managing the Madness of Cloud Logging by Alistair Chapman

  • The use case
  • All vendors put their logs in weird places and in weird sorts of ways. All differently
  • Different defaults for different events
  • Inconsistent event formats –
  • Changes must be proactive – You have to turn on before you need it
  • Configuration isn’t static – VEndor can change around the format with little worning
  • Very easy to access the platform APIs from a VM.
  • Easy to get on a VM if you have access to the Cloud platform
  • Platform Security Tools
    • Has access to all logs and can correlate events
    • Doesn’t work well if you are not 100% using their product. ie Multi-cloud
    • Can cost a lot, requires agents to be deployed
  • Integrating with your own SIEM platform
    • Hard to push logs out to external sources sometimes
    • Can get all 3 into splunk, loki, elastic
    • You have to duplicate with the cloud provider has already done
  • Assess your requirements
    • How much do you need live correlation vs reviewing after something happened
    • Need to plan ahead
    • OSCF, OTel, ECS – Standards. Pick one and use for everything
    • Try log everything. Audit events, Performance metrics, Billing
    • But obvious lots of logs cost logs of money
    • Make it actionable – Discoverability and correlation. Automation
  • Taming log Chaos
    • Learn from Incidents – What sort of thing happens, what did you need availbale
    • Test assumptions – eg How trusted is “internal”
    • Log your logging – How would you know it is not working
    • Document everything – Make it easier to detect deviations from norm
    • Have processes/standards for the teams generating the events (eg what tags to use)
  • Prioritise common mistakes
    • Opportunity for learning
    • Don’t forget to train the humans
  • Think Holistically
    • App security is more than just code
    • Automation and tooling will help but not solve anything
    • If you don’t have a security plan… Make one
  • Common problems
    • Devs will often post key to github
    • github has a feature to block common keys, must be enabled
  • Summary
    • The logs you gather must be actionable
    • Get familiar with the logs, and verify they actually work they way you think
    • Put the logs in one place if you can
    • Plan for the worst
    • Don’t let the logs overwhelm you. But don’t leave important events unlogged
    • The fewer platforms you use the easier it is
Share

Everything Open 2024 – Day 1 talks

Developing in the open, building a product with our users by Toby Bellwood

  • The Lagoon Story
    • At amazee.io . Is Lagoon Lead
    • What is Lagoon
    • Application to Kubernetes (docker build for customer, converts to k8s)
    • Docker based
    • Based on git workflows. Mostly Drupal, WordPress, PHP and NodeJS apps
    • Presets for the extra stuff like monitoring etc
  • Why
    • Cause Developers are too busy to do all that extra stuff
    • and it means Ops prefer if it was all automated away (the right way)
  • 8 full-time team members
    • Knows a lot about application, not so much about the users (apart from Amazee.io)
    • Users: Hosting providers, Agencies, Developers
    • The Adopter: Someone using it for something else, weird use cases
    • Agencies: Need things to go out quickly, want automation, like documentation to be good. Often will need weird technologies cause customers wants that.
    • Developers: Just want it stabele. Only worried about one project at at time. Often OS minded
  • User Mindset
    • Building own tools using application
    • Do walking tours of the system, recorded zoom session
    • Use developer tools
    • Discord, Slack, Office Hours, Events, Easy Access to the team
  • Balance priorities
    • eg stuff customers will use even those Amazee won’t use
  • Engaging Upstream
    • Try to be a good participant, What they would want their customers to be
    • Encourage our teams to “contribute first”. Usually works well
  • Empowering the Team
    • Contribute under your own name
    • Participate in communities
  • How to stay Open Source forever?
    • Widening the Core Contributor Group
    • Learn from others in the Community. But most companies are not open sourcing the main component of their business.
    • Unsuccessful CNCF Sandbox project

Presenting n3n – A simple Peer to Peer VPN by Hamish Coleman

  • How to compares to other VPNs?
    • Peer to peer
    • NAT piecing
    • Not all packets need to go via the server
    • Distributed ethernet switch – gives extra features
    • Userspace except for tuntap driver which is pretty common
    • Low deployment requirements, easy to install in multiple environments
    • Relatively simple security, not super secure
  • History
    • Based off n2n (developed by the people who did ntop)
    • But they changed the license in October 2023
    • Decided to fork into a new project
    • First release of n3n in April 2024
  • Big change was they introduced a CLA (contributor licensing agreement)
  • CLAs have problems
    • Legal document
    • Needs real day, contributor hostile, asymmetry of power
    • Can lead to surprise relicencing
  • Alternatives to a CLA
  • Preserving Git history
    • Developer’s Certificate of Origin
    • Or it could be a CLA
  • Handling Changes
    • Don’t surprise your Volunteers
    • Don’t ignore your Volunteers
    • Do discuss with you Volunteers and bring them along
  • Alternatives
    • Wireguard – No NAT piercing
    • OpenVPN – Mostly client to Server. Also Too configurable
  • Why prefer
    • One simple access method (Speaker uses 4x OS)
    • A single access method
    • p2p avoid latency delays because local instances to talk directly
  • Goals
    • Protocol compatibility with n2n
    • Don’t break user visible APIs
    • Incrementally clean and improve codebase
  • How it works now
    • Supernode – Central co-ordination point, public IP, Some access control, Last-resort for packet forwarding
    • Communities – Nodes join, form a virtual segment
  • IP addresses
    • Can just run a DHCP server inside the network
  • Design
    • Tries to create a full mesh of nodes
    • Multiple Supernodes for metadata
  • Added a few features from n2n
    • INI file, Help text, Tidied up the CLI options and reduced options
    • Tried to make the defaults work better
  • Built in web server
    • Status page, jsonRPC, Socket interfaces, Monitoring/Stats
  • Current State of fork
    • Still young. Another contributor
    • Only soft announced. Growing base of awareness
  • Plans
    • IPv6
    • Optimise encryption/compression
    • Improve packaging and submit to distros
    • Test coverage
    • Better NAT piercing
    • Continue improve config experience
    • Selectable tuntap drivers
    • Mobile phone support hoped for but probably some distance away
  • Speaker’s uses for software
    • Manage mothers computer
    • Management interface for various servers around the world
    • LAN Gaming using Windows 98 machines
    • Connect back to home network to avoid region blockinghttps://github.com/n42n/n3n
  • https://github.com/n42n/n3n

From the stone age to silicon: The Dwarf Axe guide to the evolution of technology by Steven Ellis

  • What is a “Dwarf Axe” ?
    • Snowflakes vs Dwarf Axes
    • It’s an Axe that handled down and consistently delivers a service
    • Both the head ( software ) and the handle ( hardware ) are maintained and upgraded separately and must be maintained. Treated like the same platform even though it is quite different from what it was originally. Delivers the same services though
  • Keeps a fairly similar services. Same box on a organisation diagram
  • Home IT
    • Phones handed down to family members. Often not getting security patches anymore
  • Enterprise IT
    • Systems kept long past their expected lifetime
    • Maintained via virtualisation
  • What is wrong with a Big Axe?
    • Too Big to Fail
    • Billion dollar projects fail.
  • Alternatives
    • Virtual Machines – Running on Axe somewhere,
    • Containers – Something big to orchestrate the containers
    • Microservices – Also needs orchestration
  • Redesign the Axe
    • The cloud – It’s just someone else Axe
  • Options
    • Everything as a service. 3rd party services
  • Re-use has an end-of-life
    • Modern hardware should have better )and longer) hardware support
  • Ephemeral Abstraction
    • Run anywhere
    • Scale out not up
    • Avoid single points of failure
    • Focus on the service (not the infra or the platform)
    • Use Open tools and approaches
  • Define your SOE
    • Not just your OS
Share

Audiobooks – March 2024

Surely You Can’t Be Serious: The True Story of Airplane! by David Zucker, Jim Abrahams, Jerry Zucker

Covers the career of the makers (ZAZ) and the long path to writing, pitching, pre-production and making of the classic movie. As well as reactions to it. 4/5

All these worlds by Dennis E Taylor

3rd book in Bobverse trilogy . Worth it if you liked the others. The Bobs deal with deaths of their human friends and most plots are wrapped up. 3/5

The Two-Penny Bar by Georges Simenon

Following a cold case Maigret gatecrashes the weekend gathering of a group of friends when one is unexpectedly murdered. Felt a little unrealistic at times. 3/5

The Car: The Rise and Fall of the Machine That Made the Modern World by Bryan Appleyard

A general history of the car with a bit of speculation about it’s future. A smooth and interesting read. 4/5

My Scoring System

  • 5/5 = Brilliant, top 5 book of the year
  • 4/5 = Above average, strongly recommend
  • 3/5 = Average. in the middle 70% of books I read
  • 2/5 = Disappointing
  • 1/5 = Did not like at all
Share

The 10 Thickest Books I Own

Prompted by a comment from someone I present below the 10 thickest books in my personal library. I made no correction for hardcover vs softcover. Measured at center of book with mild compression

The Books in order. 10th thickest left, thickest on right

My top 10 books ended up being a bit of a mix

  • Three Fiction: 1 Science Fiction, 1 Fantasy, 1 annotated detective series
  • One giant book of Chess puzzles
  • Two books about lots of things. 500 Villages and 100 Museum Objects
  • Two biographic books about a National Leader during wartime
  • A book of social history
  • A book looking at big trends in all recorded history

The Countdown

10th – 58mm – The New Annotated Sherlock Holmes. Volume 2. 1878 pages. Softcover

The tallest, widest and book with the most pages. Has the original text in the centre with notes on the outside and lots of illustrations

9th – 60mm – Villages of Britain by Clive Aslet. 658 pages. Hardcover

1-2 pages on 500 English villages. Usually covers an interesting feature, event or person

8th – 61mm – Team of Rivals by Doris Kearns Goodwin. 916 pages. Softcover

A book on Abraham Lincoln’s Cabinet. Basis for the movie “Lincoln” and my book is a movie branded version

7th – 61mm – The Lord of the Rings by J R R Tolkien. 1192 pages. Softcover

My much battered single volume edition I’ve had since I was a kid.

6th – 63mm – Chess 5334 Problems, Combinations, and Games by Laszlo Polgar. 1104 pages. Softcover

Mostly pictures of chess positions (6 per page) and the solution. Almost no words

5th – 64mm – A History of the World in 100 Objects. 707 pages – Hardcover

Based on a Radio Series. Each object has a couple of very nice photos and then around 3 pages of text about it and where it came from. Very nice book.

4th – 65mm – Seveneves by Neal Stephenson. 867 pages. Hardcover

A Science Fiction book about what happens when the Moon blows up.

3rd – 66mm – Why the West Rules – For now. 750 pages. Hardcover

A big idea history book with speculation about the future.

2nd – 69mm – Road to Victory. Winston S. Churchill 1941-1945 by Martin Gilbert. 1416 pages. Hardcover

Part of the huge 8 volume official biography of Churchill. Covering Pearl Harbor to VE Day.

1st – 72mm – Family Britain 1951-57 by David Kynaston. 776 pages. Hardcover

Part of an ongoing series of books about the social history of Britain from 1945 to 1979. Covers a lot of ordinary lives and major events are often seen via individual’s reactions rather than being covered directly.

Share