lca2019 – Simon Lyall's Blog

Linux.conf.au 2019 – Friday – Lightning talks and Conference Close

Closing Stuff

Special Badge given out

Projects from Junior Group from Club Penguin
Rusty Wrench award to Joshua Hesketh

Lightning Talks

3 minutes each
Martin Kraft
- Digital trust and trust in the real world
- In real world it is wired into our brains
- Aunt’s Laptop. Has Trusted CAs, Debian.
- Doesn’t know what lock on browser means
- Imaging that trust is a competition that happens in real time, that takes interactions, mood, random stuff.
- Maybe when you visit a good vs bad website the browser looks visably different
- Machine Learning
Brimly Collings-Stone
- Maori language not available on AAC outputs
- Need a device that speaks Maori and represents Maori grammar accurately
Mathew Sherborn
- RSI
- Got it in the past, tried various keyboards
- Type-Matrix but it broke
- ErgoDox – open source keyboard
- Mascot.com – Keyboard in batch orders
- Like the ErgoDox-E – $500 but good – web app to program
- Change the Dvorak keyboard with special keyboard
Emma Springkmeier
- What do I do when it all goes wrong
- Potentially stressful situations – phone calls, meetings.
- eg last year’s lightning talk
- What I do to cope
- Talk to friends, explain how I feel to others, listening to calming music, breathing techniques ( 4s in, 4s hold, 4 out, 4 holding, repeat )
Karl Kitcher
- Secretary of the NZ Open Source Society
- Charity since 2008
- Reducing in interest in the recent years
- Open source is not so prevalent, people not really caring, trying to maintain the momentium
- Open vs Fauxpem
- nzoss.nz – signup to the mailing list
- Various services to projects
Leon Wright
- About Leon’s badge
- Twitter bot hooked to hug detector in his badge
- 2017 badge detects hugs
- 2018 version 2 . So good twitter shadow banned his account
- 2019 – Docker containers and other crazy stuff
Talia White
- At LCA since 2018 – Was only 8. Now 12
- Ordered a robot kit for ardiano
- Made various projects
- Don’t give up, struggled to start with coding, got better
Brenda Wallace
- Works for the NZ Govt
- Sometimes abigious
- Going to publish for some legislation as python rules
- rules.nz
- eg Social welfare rules,
- Unit tests
Paul Gunn Stephen
- GDP per km of coastline
- %coastline length for area
- Means hard to get Tsunami warning systems
- Cheaper
- ETC Lali system approach
- Every Village has a local warning system
- Redundant system
E Dunham
- You should speak at conferences
- 54th talk in 5 years
- Promotes your company
- Intersection: What you know, what conference needs and what the attendees needs
- Find conference want to attend
- Write abstract
- Submit a lot, get rejected a lot
- Each reject is how you dodged a bullet
Charell
- CVE-2019-3462
- Bug in apt that allows injection of bad content
- Why https
- Attestation
- apt-transport-https – enable
Jen Zajac
- Project scaffolding eg Cookiecutter, yoeman
- Lots of generating options
- Creates templates for a project
Hugh Blemmings
- Ardionu and Beagleboard
- Cool but not high performance
- A True open and HP computer
- Open Hardware, Open software stack, no bin blobs, No unexpected software, No cost/perf penality
- openpowerfoundation.org
Benno Rice
- Cobol
- Over 50 years old
- Not used much
- What Language is the new Cobol?
- PHP is the new COBOL
- Perl is the new COBOL
- Python2 ?
- Javascript ?
- C ?
- Y2K – Maybe the real Cobol is the maintenance we incurred along the way
- Maybe you should support software before it bites you back

Closing Stuff

652 people attended
2.4TB transferred over the SSID
3113 Coffee vouchers

Lots of sponsors and suppliers and staff thanked

Linux.conf.au 2020 is in …. Gold Coast

Linux.conf.au 21st birthday!
Gold Coast convention and Exhibition centre
13 – 17th January 2020

Linux.conf.au 2019 – Friday – Session 2

OpenLI: Lawful Intercept Without the Massive Price Tag
– Shane Alcock

Police get Warrent to ISP
ISP Obligations
- Can’t tip off person being intercepted
- Both current and past intercepts must be private
- Can’t accept other people’s communications
- Must accept all communications
NZ Lawful Intercept
- All Providers with more than 4000 customers must be LI capable
- Must be streamed live
- TCP/IP over tunnel
- Higher level agencies have extra requirements
- 2 seperate handovers – IRI metadata for calls, IP sessions. CC= data packets
Open LI
- $10,000s – $100,000s costs to impliment and license from vendors
- WAND had some experise in packet collection
- Known my NZ Network Operator community
- Voluntary contributions from NZ Network Operators
- $10k+ each
- Buys 50% of my time to work on it for a year.
- Avoiding Free Rider problem
  - Early access for supporters
  - Dev assistence with deployment
  - Priority support for bugs and features
Building Blocks
- Developed and tested on Debian
- Should work on other Linux flavours
- Written in C – fast and likes writing C
- Use libtrace from WAND
- Data Plane Develop Kit
Provisioner
- Interface for operators
- Not very busy
Collector
- Comms from Provisioner
- Intercept instructions
- Recommended run on bare-metal
- 1RU Server with 10G interface with DPDK support
- Supports multiple collectors
Mediator
- Gets data from Collector
- Forwards to Agency based on instructions from Provisioner
Target Identification
- Nothing on the packets linked to target user
- People get dynamic IPs, can change
- For VOIP calls need to know RDP port
- SIP for VIP , Radius to IP to ID the user’s IPs/Ports
- Deriving caller identities from SIP packets can be tricky. Other headers can be used, depends on various factors
Performance Matters
- 1Gb/s plans are available to residential customers
- ISP may have multiple customers being intercepted. Collector must not drop packets
- Aim to support multiple Gb/s of data
- libtrace lets use spread load across multiple interfaces, cpus etc
- But packets now be in multiple threads
- Lots of threads to keep things all in sync
Status
- All done and deployed by at least one ISP
- Core fucntionally in place
- 500k PPS with DPDK
- On https://github.com/wanduow/openli
Future
- Build user-driver community around the software
Questions
- Can it handle a hotel? – maybe
- ISPs or police contributing? – Not yet
- What have people been doing so far? – They have been gettign away with saying they will use this
- What about bad guys using? – This probably doesn’t give them any more functionality
- Larger Operators? – Gone with Vendor Solutions
- Overseas Interest? – One from Khazakstan , but targetted at small operators
- Why not Rust, worry about parsing data – Didn’t have time to learn Rust

But Mummy I don’t want to use CUDA – Open source GPU compute
Dave Airlie

Use Cases
- AI/ML – Tensorflow
- HPC – On big supercomputers
- Scientific – Big datasets, maybe not on big clusters
What APIs Exist
- CUDA
  - NVIDIA defined
  - Closed Source
  - C++ Based single source
  - Lots of support libraries ( BLAS, CiDNN ) from NVIDIA
- API – HIP
  - AMD Defined
  - Sourcecode released on github
  - C++ based single source
- OPenCL
  - Khronos Standard
  - Open and Closed implimentations
  - 1.2 v 2.0
  - OpenCL C/C++ Not single source (GPU and CPU code separate)
  - Online vs offline compilation (Online means final compilation at run time)
  - SPIR-V kernel
- SYCL
  - Khronos Standard
  - C++ Single source
  - CPU Launch via OpenMP
  - GPU launch via OpenCL
  - Closed (codeplay) vs Open(triSYS)
  - Opening of implementation in Progress (from Intel – Jan 2019)
- Others
  - C++AMP – MS
  - OPenMP – Gettign better for GPUs
  - OpenACC
  - Vulkan Compute
    - Low level submission API
    - Maybe
- Future
  - C++ standard
  - C++ ISO standards body, ongoing input from everybody
  - Implementations must be tested
  - Still needs execution environment
Components of GPU stack
- Source -> Compiler
- Output of GPU and CPU code
IR
- Intermediate representation
- Between source and final binary
- NVIDIA PTX – liek assemble
OpenCL Stacks
- Vendor Specific
- LLVM Forks
Open Source
- Development vs Release Model
- Vendors don’t want to support ports to competitors hardware
- Distro challenges
  - No idea on future directions
  - Large bodies of code
  - Very little common code
  - Forked llvm/clang everywhere in code
Proposed Stack
- Needs reference implementation
- vendor neutral, runs on multiple vendors
- Shared Code based (eg one copy of clang, llvm)
- Standards based
- Common API for runtime
- Common IR as much as possible
- Common Tooling – eg single debugger
- SPIR-V in executable -> NIR -> HW Finaliser
- Maybe Intel’s implementation will do this
Questions
- Vulkan on top of Metal/Molten ? – Don’t know
- Lots of other questions I didn’t understand enough to write

Linux.conf.au 2019 – Friday – Session 1

Preventing the IoT Dystopia with Copyleft- Bradley M. Kuhn

The S in IoT stands for Security
Many stories of people hacking into baby monitors and home cameras
IoT Devices often phone home to manufactorers website in order that you can access then remotely. “I suppose there are Chinese hackers watching my Dogs all day, I hope they will call me if they need water etc”
Open source people have historically worked to get around problems like this.
1992 – If you wanted Linux, you downloaded the software onto floppies and installed it yourself. And Often had to work hard to make it work.
Today only a small percentage of laptops sold have Linux on it.
But Linux is commonly installed on IoT devices – 90% odd
But
- No [easy] way to reinstall it yourself
- Much worse than laptops
- GPL includes “The scripts used to control the compilation and install of the executable”
- “Freedom to Study” is not enough
Linksys Wifi router
- OpenWRT Project
- Release forced from Linksys and Cisco
- “Source as received from Linksys from GPL enforcement”
- Is OpenWRT a Unicorn
  - Few projects with serious alternative firmware project
- Still sold new after 20 years
BusyBox Lawsuits
- Before IoT was even a term
- At least one model of Samsung TV -> samygo.tv
- “Baffles me as to why do the manufactorers want us to buy more hardware”
Linux focuses to much on big corp users and ignores hobbyist users
- Kernel peopel only care about the .c files. Don’t care about the install scripts etc.
- People at top of Linux now got their start hacking on the devices in front of them.
- The next generation of developers will be those hackers not from IBM and other big companies
- You didn’t need anything but a computer and an internet connection to become and upstream developer in those days. This is becoming less true.
- If the only thing you can install Linux on is a rackmount server, a cloud server or maybe a laptop and none of the IoT devices around you then things don’t look good….
Linux was successful because users could install it on their own devices
Linux won’t remain the most important GPL program if users can’t install their modifications. Tinkering is what makes Free software great.
Upstream matters of course, but downstream matters more.
- There may be 1000s of Linux developers
- Put 2 billion people have Linux on their phone – Which is locked down and they can’t reinstall
We don’t need a revolution to liberate IoT devices
- because the words are already there in the GPL
- We just have to take up our rights
What you can do.
- Request Linux sources on every device you own – Companies have figured out people almost never ask
- Try to build and install them. If you can’t ask a friend or ask Conservancy for help
- If it doesn’t build/install it is a GPL violation, report it Conservancy
- Step up as a leader of a project devices that matter to you.
Why this will work
- The problem seems insurmountable now, only because we have been led astray
- First and absolutely necessary step towards privacy and scurity on those devices
- When the user controls the OS again, the balance of power can be restored
Questions
- Best way to ask for source code? Try email, the manual should say.
- How to get the new code on the device? Needs some push onto industry
- What if writing requires expensive equipment? Fairly rare, many devices allow over-the-air upgrades, we should be able to go the same way.
- Is there a list of compliant devices? – Proposed in past. Want to go softly at first in many cases
- Am I exposed to liability if I modify and distribute code I receive? – Almost certainly note, contact Conservatory if you are threatened.

Web Security 2019 – James Bromberger

History of browser
- No images
- Images
- Netscape with crappy ‘International Security”
- https takeup is growing
- Chrome is hitting 60-70%
- 82% of browser are “modern”, crossover of chrome users to new version is about 3 months.
PCI
- Remove early TLS in mid 2018
- TLS 1.1 and higher allowed
The legacy browser has gone in the real world
- Some envs still behind, but moving ahead
What can we do with as little changes as possible?
0. Don’t use http, use https
- Use letsencrypt
- Stds reducing max length of certs from 5 years
1. TLS protocols
- 7 versions out there (old ones SSL).
- Most over 10+ years old
- Only 6 in the wild
- 3 not-known to be comprimised ( 1.1 1.2 1.3 )
- Very few clients only support 1.1 and not 1.2 (small gap in 2006-2008 ). IE supports 1.2. So maybe disable 1.1
- Log the protocol being used so you have data on your users
- OTOH not much supports 1.3 yet
- Use 1.2 and 1.3
- Turn off on the Browsers to
- Looks at which libraries you are using in code that makes https connections
2. Cypher Suite Optimisation
- New EC certs for key exchange
- New certs getting changed to ECDSA
- AES is standard for bulk encryption. GCM mode is best although windows 9 can’t do (Upgrade to 10!)
- MAC/Cehecksum – remove MD5, SHA1, remove SHA2-256+ , New ones coming
Security Header
- Content-Security-Policy
- Referer-Policy – Usually locked down
- Feature-Policy – lots of stuff
- ” X-Content-Type-Options: no-sniff ” – don’t guess content type

4. CAA
- Around 200 Cert Authorities
- Authorized record type (CAA record) lists what CAs are allowed to issue certs for you.
- DNS Sec is useful – But during US Govt shutdown DNS keys are expring
5. Sub Resource Integrity
- Scripts included by html
- Can include checksums in html calling to varify
6. Cookies
- Secure – httpsonly
- “SameSite=Strict” – Reduces cross site request forgery
7. Http2
- Binary wire protocol
- Apache 2.4 on debian
- Forces better protocols
8. Lots more
- New compression algorithms
- Network error logs

Linux.conf.au 2019 – Friday – Keynote: A linux.conf.au Story – Rusty Russell

Bitcoin Billionare
1992
- The days of SunOS
- Read the GNU Manafesto
1995
- Using GPP compiler at work
- First patch accepted on November 1995
1997
- USENIX Conference in california
- UseLinux – Had a track for Linux
- Hung around a bunch of top guys in Linux talked about added SMP to Linux
- Talk on porting Linux to Sparc by David Miller & Miguel de Icaza. Going into improvements and showing how Linux port to sparc bet Solaris in the Lmbench benchmarks on same hardware.
- Relaized lived in a world where students could create and port OS that bet the original OS from the vendor
1997 – 1998
- Wrote (with another guy) and got ipchains added to Linux
- “I woke up one morning and I was kernel firewall maintainer”
- Got job people paid to work on Linux firewall code
1998
- Decided needed an Australian Linux conference
- Oct-Nov visited a bunch of LUGS to invite people and find person to collect money.
- People not sure what they wanted to go to a Linux conference ( $380 bucks)
- Invited John Maddog Hall
- Created and ran a slashdot ad
- Created card got into $14k negative
- Last session of the 3rd day, reran the 3 best talks
Three stories from 1998
- Tutorial Books for each of the tutorials- Couldn’t get photocopies from commercial facility, so had to make 400 copies of books via 4 coin operated photocopiers
- Tridge bought up a triple-CD burner. People ran it in relays
- Somebody said. “I can’t believe you don’t have conference tshirts”. He bought white tshirts, got them screen printed and sold them.
End of conference Tridge organised a gift from the Speakers to Rusty. Pewter Beer mug
Linux.conf.au after 1999
- 2001 scheduled 3 talks from Rusty. At the same time
- Met Tridge at LCA – Moved to Canberra they did AusLabs
How Great Projects
- Smart and Capable enough to complete them
- They are Dumb enough to try
- When somebody tells you about a project?
  - That sounds Great, Tell me more
  - What can I do to help
- Enable people’s enthusiasms
- Collaboration is a super Power
- Get along with people is a skill
- “Constructive absenteeism”
Headwinds to collaboration
- Signs are welcoming to some people
- Other people get signs that they are not so welcoming
- Good are seeing them when they are aimed at them, not so good are even seeing they exist when they are not aimed at them.

Linux.conf.au 2019 – Thursday – Session 3

Open Source Tools for Publishing and Processing Earth Observation Imagery – Paul Haesler

Golden age of satelite imagery
Geostationary – One area – Good for weather
Circum-polar orbits – all over earth every 10-16 days
Data processing Chain
- Level 0 – Raw
- Level 1 – Geo rectify – Measure surface radience
- Level 2 – Curroect for sun, sat angle, atmostphere – ARD – Records surface reflectance
Landsat-8 (25-30m , 8-16 day cycle, data since 1982 ) Sentinel-2 ( better, 5 day cycle, 10m resolution)
Digital Earth Australia
The Problem
- Open Data Cube
- Python, based on xarry
- Postgres for metadata
- Actual satellite sata from local or network repo (transparently)
- GUI maintained by CEOS
Nationalmap.govt.au
- Aims to publish all Aus OpenData that can be mapped
- Based on TerriaJS
- Some DEA data was already being publish but need for additional stuff
COGs – Cloud Optimised GeoTIFFs
DataCube_ows
- Lightweight web application server
- Developed by Datda61 for GA
- WMS
  - OGC Web Map service
  - Good for general-usage web apps
  - Returns standard images (eg png)
  - Support 1.3 well, works with most clients
  - Styles for band-mapping
  - on-the-fly solar angle correction
- WCS
  - Version 1 supported
  - Works well with TerriaJS , works okay with QGIS or ArcGIS
Next Steps
- WPS for on-the-fly processing is regularly discussed
- Better ingegration with datacube-core
- More recent WCS versions inc WCS-2EO
- Sparse Data problems

The Tragedy of systemd – Benno Rice

References to Contempt Culture
Ancestry of Systemd
- Unix: Happy accident, place, time, reaction to the previous thing
- housekeeping functions – “mounting filesystems and starting daemons”
- inetd – Super Daemon for all sockets – “worked well until The Internet Happened”
Then the Internet happened
- forking a process per connection doesn’t scale
- Lots of persistent state for things like databases
- Service
  - Might be a bunch of processors
  - Init starts but doesn’t manage
  - initab can restart things in SystemV
System Config vs Service bootstrap
- Mixed in togeather
- Service management needs more
Windows NT
- Service model there from beginning
MacOs
- Application Model means lot richer interaction with the host
- Application delegate
- launchd
The Idea of Systemd
- launchd
  - Service handling in MacOS
  - Took over init, inetd, cron
  - Can listen on ports. Start stuff. doesn’t need to start on boot, boot gets faster, power reduced, security improved
  - Move system services to daemons, then start daemons as needed
- From Launchd to systemd
  - upstart
    - event driven
    - shell based
  - Rethinking PID 1 – Lennart
  - “Start less” , “Start more in parallel” , “listen to hardware and software changes”
  - cites launchd
System management
- Everything is a lot more dynamic
- Hotplug , DHCP , etc
- Don’t install 15 different packages that all behave differently
- But systemd will have to do things in a different way to those 15 other things
The reality of systemd
- Widely adopted ( 2011 – 2015 )
- Arguments
  - Violates the unix philosphy – actually systemd actually is many binaries
  - It is bloated and monolithic – Well it does do a lot of thing
  - It is buggy – So is all software, actually a good failure mode
  - I can’t stand Lennart Poettering – He’s delivered. “I won’t defend his community interaction”
  - It is not portable – UNIX is dead – Posix isn’t really a thing anymore, there are not a bunch of crazy Linux variations. “These days you have Linux and some rounding errors”
- cgroups
- User-Level units
Change – System is a lot of disruptive change
The Tragedy of Change
- Nerds love change as long as we are the ones doing it
- System boot ups using shell script interaction is like the old blanky we should of got rid of 20 years ago
- The Knee-jerk – Abuse is not Cool
The Next Generation
- They See a lot more APIs
- Thinking in Containers is different from thinking in not-containers
What does Systemd have that FreeBSD (or even future Linux) could use, or could do better
- Message Transport
- RPC Framework
- Kernel and Use-space services should look similarly to the services above them
- Service Lifescycle
- Automation via API – Easier for vendors to write appliances
- Containers
- The System Layer
  - Doesn’t have to be the only implementation of theis
- Consistent Device Naming
- Better Log/Event/Audit Handling
- A new model of an application ( a bunch of things managed as a Unit, See the MacOS model)
Questions
- Launchd option – Too MacOS specific
- Dynamic Libraries = DLL Hell – Containers avoid, different problems
- Is reaction to systemd scaring other big changes off – Possible, hard to write, very hard to handle the social issues to push though
- Where is FreeBSD at? – A long way away, no consensus this sort of change needed
- Should everything have been swallowed up – Thought experiment, If systemd had instead defined an API for separate projects instead of writing them itself, would that have worked? And now we do know what is needed could we switch to a separate model with APIs?
- Enbeded Devices need systemd – Anything Dynamic needs it
- What Push back from FreeBSD – Something like that but not systemd. Some like launchd
- What needs to change in community and systemd team to make things better – See Adam Harvey’s talk on language changes. Hard since everythign is asking for different stuff, systemd people.
- What should systemd go further into – Messageing and RPC stuff more pervasive and more thought about. Something into the kernel.

Linux.conf.au 2019 – Thursday – Session 2

Go All-In! – Bdale Garbee

How to get companies involved in FOSS
First contribution of source code was almost 40 years ago
Used Vax BSD at CMU – HAd the deal with an obscure priesthood
KA9Q TCP/IP stack for amateur radio in the 80s
Appearance of RMS in my world
- GNU Manifesto
- 4 Freedoms
- GPL
Debian
Worked as LInux CTO from one of the largest IT companies in the world
Collaborative Development Model
- Spread out besides just making FOSS
- No one company in charge
- Diverse range of contributors, massively different motivations
- We get a software commons we all get to benifit from
Free Software means Freedom of Choice
- Reduced barriers between users and producers of software
- Any user can be a dev, or pay someone to dev
- If upsteams goes bad, things can be forked
What it means to be successful when you are operating in an open and collaborative model?
- The goal of a trad company is for investment to yeld technological control points
- First mover advantage
- Differentiated features, preferably patentable
- Collaborative dev model allows us to recognise the benifit of collaroation on all the non-differentialting elements. Leavign more value to the users / customers
- Thinks less about control points, more about points of affinity. What is it that would make a customer want to user you products or services?
Innovation these days largely takes place in the open space
Wrights/Goddard – They didn’t get told to to the next new thing, they just started it as a hobby
Free Software enables people who we don’t know exist to create innovation and invent things we can’t imagine
Long Tail of Contribution
- Example: People who did one Linux Kernel contribution, often to fix on specific thing that was causing them problems.
- No company on earth that can hire that resource
- Needs to be easy for people to access the code and make contributions
Attributed of Successful Communities
- Active contribution and collaboration
- Diverse participation
- Solid core of code
- Recognizable mainline trunk
- Unified, cohesive structure
- Low barriers to entry
Choosing the right license
- Businesses can only be successful with permissive licenses
- The most successful projects seems to be communities built around open contribution
- Share-alike licenses stop possible problem of Closed Corporate fork while the original project withers

Beach Wreck Ignition: Challenges in open source voice –
Kathy Reid

MycroftAI – One of the few open source voice stacks
Introduction to a Voice Stack
- Wake Word – eg “Hey Alexie”
- Utterance – Phrase of command
- Speech2text processor
- Looks for keywords etc
- Runs a command
- Dialog – acknowledger + response
Wake Word
- PocketPhinx, Snowbox, Mycroft Ai Precise
- Some use Phonemes (smallest units of sounce in a language)
- Hard to tell differences between all words
- Always listening, connected to internet
- Some use Use Neural networks
- Low accuracy can cause frustration
  - Bias towards male speaker (10:1 male:female in dataset). Also more with American than other accents
  - To unbias the sample had to tag the samples with ethnicity, gender etc. Which was a problem with ethics of taggign samples/speakers
Speech to Test
- Kaldi – no network needed, compute heavy
- Deep Speech – From mozilla
- Challenges
  - Lots of accents out there. Hard
  - Only trained for most common accents
  - Also problem with regional slang
  - Need to train on individual speaker
  - But need lots of data to understand a speaker
Endangered Languages
- No commercial imperative to cover them
- Mycroft Translate using Pootle to translate command words to 40 languages
- Issues for gendered languages, formality
Intent Parsers
- Rasa, Mycroft Adapt, Mycroft Padatious
- Intent Collisions – Use confidence scoring depending on how explicit the request is.
Text to Speech
- Mary TTS, Espeak, Mycroft Mimic, Mycroft Mimic 2
- Mimic recording studio, Need 40-60 hours audio
- Challenges
  - Natural sounding voice – making the voice sounds not robotics
  - Pronunciation – often requires after creation

Linux.conf.au 2019 – Thursday – Session 1

A Long Day’s Journey Into Backups – Rachel Kelly

A journey in four stages
Version 1 – State of Backups Aug 2017
- Needed to look though old logs to see how far back problem wentDaily diffing from duply/duplicity – But where was the original?
- Tried to restore data from old backups
- Couldn’t restore from backup since original was too old
- Couldn’t get it to work, needed something new fast
Version 2
- Created tarball, uploaded to AWS S3, via daily cron
- Done quickly
- Not reliable sending to s3
- Needed ongoing work
- Big Win: But at least complient, and we could get data back out
Try 3 : Shiny EFS
- EFS is AWS’s NFS solution
- tarball created on local EFS disk, easypeasy
- Big Win: Reliable backups, incorporated into infrastructure, retrievable data. 8 weeks off backups
- Miscalculated cost: About 10x original estimate
Try 4: Tarball to s3 redux
- Tarball to s3. PLUS infrastructure
- Would work going forward
- S3 approx 1/10 the cost of EFS
- Big Win: Reliable, inexpensive, functional, dependable
- Discovered that EFS not encrypted
- Able to manage well
Current Solution
- cron job calls a bash script
- Safety in bash: set -euo pipefail
- tar up the filesystem (with exclusions)
- Set it to s3 with aws cli (with hosts’s hostname)
- After 56 days is sent to glacier
- restore script to get a file (uses AWS credientials of current instance)
What’s Next?
- I could work on backups forever and they will never be perfect
- Ability to restore to a new instance
- Want to be able to get files from anywhere
- Microservice backups
- Deglacierisation
- What we need
- Better CLI tool – safer
What I really really want
- A decent enterprise solution
- NIH is a dangerous habit
- Speaker knows no one using a hosted enterprise solution
- Vendor solutions seem to be crazy overkill
Feedbacks
Amanda Recomended
Every morning restore dev enviroment from anon prodouting one. Comet Backup
Wasabi compatable with s3
Recoment “Retrospec”
tarsnap
bacula
rsnapshot
Looking at Borg Back
rsync to zfs, zfs send
Personal backups, using duplicity
Industry 4.0
- After mechanization, mass production and automation
- The machines run everything and we reap all the benefits – maybe
Robot Hype
- Post AI winter
Implementoer’s bias (top down design of neutral network setup, choose number of layers etc)

Linux.conf.au 2019 – Thursday – Keynote: Shannon Morse

Personal Branding for the Security Conscious – Shannon Morse

Who am I
- Youtube videos on Infosec, Travel
Imposter Syndrome
- Work hard to beat it.
- Say Yes to offers
- Work hard to make somethign I am proud of
- Surround yourself with positive people
Accomplishment
- Keep a list of them, be proud of them
Backstory
- No background in Linux, hacking, infosec
- Mom and Dad supported me
RTFM Sucks
- Lots of egos and elitism in forums and community online
- Decided to become the resource for learners
Starting your career
- What do companies need, what hiring for
- How has industry changed?
- Diversity numbers?
- Can you change industry in a positive way
- Review CERT holders numbers vs Openings looking for those certs
- Look at job titles being advertised
- Industry growing -> lots of beginners
How can you get good at it
- Understand what is your best way to learn
- Read books, classes, videos, whatever
- Compile your list of passions
- Get list of influencers / thought leaders / speakers in the area
- Follow them on social media
- Learn from your role models
- you might end up being a thought leader in their eyes
- Follow people in other areas too
Keep learning
- Do it every single day
- Make it become a habit
- Make it a routine
Resume
- Create a one-pager
- Business cards
- Dropped out of college put “Huitus”
Build you platform
- Youtube, write articles, videos, whatever
- If you can afford it, offer free classes for under-represented groups
Personal brand
- Develop the blocks
  - skillsets, values, what does it mean for you to succeed
  - What obstacles have you overcome
  - what are you passions
  - what makes you unique
  - write and live by your vision statement
- If you don’t control and manage your brand others will do it for you
  - Where do you draw that privacy line?
  - Quiz yourself
  - Eg how public are you on you income?
- Resources
  - Joindeleteme.com
  - password managers
  - 2FA
  - Guest vs home Networks
- Clean up your social media accounts, delete old junk
- Smart sharing
  - Share stuff but not barcodes, addresses
  - Have a plan
- Be ready to deal with targeted harassment
  - Keep notes, screenshots, know who to contact
  - Trolls? Block; banhammer
  - Troll back (YMMV)
Why I don’t quit
- Do it because you love it

Linux.conf.au 2019 – Wednesday – Session 3

Why linux.conf.au continues to amaze 20 years on!
Steven Hanley, Angus Lees, Hugh Blemings

Three people who have attended every Linux.conf.au
What has happened to keep it relivant and how it has evolved
Focus on Open Source Software , These days hardware too
Pre-history
- Australia big in early Linux
- late 90s Linux and LUGS growing in Aus
CALU
- In 1998 Rusty toured LUGS and organised CALU in July 99 in Monash
- 9 July – 11 July
- 6 tutorials and 16 talks
Post-CALU
- Lots of excitement
- Height of dotcom boom, Big Linuxcare Expo in Darling Harbour in 2000. Very commercial
- Push for another one
2001
- UNSW interested in Lions memorial conference
- CALU type conference would be good
- Sydney big due to Olympics
- Domain picked
Challenge of early events
- Everythign was new
- CFP wasn’t formalised
- Linux Aus still in infancy
- Open Source was new and Few conferences
- Few people worked fulltime in FOSS
- sceduled over a weekend cause people didn’t get holidays
Early objectives we hold on to
- Community rather than commercial, modest size
- Easier for students and hobbists, low price
- tech/non-tech balance to encource interesting delegates
- Miniconfs
- Speakers treated well
- Timeing to make conference attractive (Summer!)
- University venue, dorms, communal accom, Holidays
Miniconfs
- First in Brs in 2002. more in 03, formalised in 2004
- Open to all delegates
- Incubate possible future conferences
- Fill the week without adding more effort
- Try out nice topics, extended BOFS
- Practice ground for new speakers
Growth and Roaming
- Change of location helps
- New team help avoids burnout and bring fresh ideas
- Allows more people to contribute
- Repeat city visits with new people involved
- Allows people who are less-technical to help out
- Bid process introduced, overseen by LA
- Specifics of venue and location help teams structure their event
- New locations add tourism aspect, encourages aspect
- Positive experience shared by work-of-mounth
Expansion
- Will anyone go to Perth? NZ?
- Infinite growth is not a goal (complexity, conference atmosphere)
- New activities and events are continuously evaluated
Call for papers
- Originally ad-hoc
- Seperate Panel since 2005
- See Mary’s blog post in 2006
- 4-5 times proposals to slots
- Process
  - Actively solicit hot speakers / topics
  - Review submissions individually
  - face-to-face to reach final as group
- Conference organisers set overall theme, choose keynote speakers
Conference MNGT software
- Much NIH
- New software in 2004, 2005, 2007, 2017
- Mainly for CFP has hooks to other conference components (scheduling, badges, website)
Ghosts of conferences Past
- Mailing list to ask
- Visit conference organisers
- Some people have remained over multiple years
Giving Back
- Donation to a charity, action off a shirt singned by speakers
Event Style and Flow
- The current event is typical
- Moved from Weekend to week as people get paid by employer
- Less Beer, More Food
Giving back to the technical commons
- Regional delegate program ( 2004 for a few years )
- Stories of laptops being fixed by the guy who dev’d it
- git is an outcome of “no more bitkeeper” in 2005
- Encouraging work done once talk has been accepted
- The Debian couch had no back so he had to sit around and support each other
Sponsorship
- Some Very long term
- Some years had to educate sponsors on how to participate
- Local vendors and sponsors
- Nurturing sponsosrs takes consious effort
Learning lessons
- Floods (Brisbane 2011)
- Budget issues (where LA comes into play)
- CFP feedback loop has sought to tweak technical/non-tech focus
- Code of conduct has made conf better
- People step up to make things happen
Resources
- Mirror
- Mary Gardiner’s post on getting a talk accepted
- Simon Lyall’s guide

Right to Not Broadcast – Karen Sandler

Thought the problem with the propitiatory device in her heart was about transparency, now feels it is more about control
Got a new device where the programmer (controller) only worked when it was touching, rather than over a distance.
Research team bought a device on ebay that had patients data still in it.
On the other hand the Keynote Speaker this morning had to hack into her own device to get the info.
Sleep Apnea machines transmitting data to both doctors and insurance companies
Smart TVs
- Listening for wake words all the time
- Sending viewing data to 3rd parties
Various Legislation
- HIPAA , NZ Health Information Privacy Code, Aus Privacy Act, GDPR
- GDPR – Europe’s gift to the rest of the world
” Incorporating connectivity means we can never be totally in control of our critical information “
The environment/reason we provide the data in now may change
Often the non-connected option is marginal or doesn’t even exist.
[ Laptop ran out of battery here ]
Things need to be worked on
- Raising the awareness of the non-networking, privacy-first issue
- Even among the LCA-type crowd

Linux.conf.au 2019 – Wednesday – Session 2

Around the world in 80 Microamps: ESP32 and LoRa for low-power IoT – Christopher Biggs

Promise of IOT
- Control everything
- Sensors everywhere
- Reduce cognitive load
Problem
- Computers everywhere = wires everywhere
- Can’t be done every time
- But em if you got em
- eg Power over ethernet, Ethernet over power, Ethernet over coax
- Wifi is great for connections. But what about power?
Batteries are bad
- Lead acid – obsolete just about everywhere
- Single-use dry cells – leak
- Nickel Metal-hydride – Some stuff
- Lithium – Everything else
- Sample battery
  - Chap-stick battery = 2 Amp/hours
  - 3.7 volts
  - Labels on batteries often lie – you need to always verify
  - Energy capacity is quoted for 20h discharge, not linear relationship
- But they are geting better due to phones, scooters, drones pushing
Off the shelf solutions for packs
- Smart ones may turn themselves off if draw very low
- Cell plus simple system works
- Cell with “Battery Managmnet system” is a bit more complex
- Solar panels are useless, needs to be a4, a4, a5 size at least
- Linux systems too much draw for non-wired sensors, need to be used as hubs
Computers can spend most of time asleep
- Config one or more wake-interupts
- Arduino deep sleep – Sleep consumption as low as 6 micro amps. ( 38 years with Chapstick battery)
- Watch out for stock voltage regulators (eats 10 mA)
ESP 8266 Sleep modes
- Several levels off sleep modes
- Wake up every 5 minutes = 1 week battery life
ESP 32
- Sleepier modes
- Complex sleep patterns.
- Ultra-low coprocessor
  - 4 register, 10 instructions, 16-bit, special slow memory
  - Can be configed to wake up at intervals
  - Can go back to sleep or wake the main cores
- ULP in practice
  - Write code, load into the ULP processor
  - Enough code to decide to go back to sleep or wake up main processes
Aim for efficiency
- Sleep as much as possible
- Use interrupts not polling where possible
Nasty Surprises
- Simple resistors ladders leak power
- Linear regulators leak power
- Poor antennas cost watts
- Beware: USB programming bridges that are always on
- Almost all the off-the-shelf IoT boards are no good for permanent installation
Solutions
- Turn of everything you are not using
- radio turn off when not in use, receiver turn it on now and then. Do store-and-forward
- slow down the cpu, turn off bluetooth
- Reduce brightness of lights
- BE careful about cutting out safety features
Case Study – Smart water meter in multi-tenant building
- Existing meter has a physical rotating dial, can count rotations
- In cellar with no power
- Create own
  - ESP-32
  - Wifi for setup or maint
  - LoRA for comms every 15 minutes
  - ULP monitors 4 sensors
  - ULP wakes CPU after number of elasped minutes and/or pulse
  - Transmits to Linux-based hub covers building
  - 150mA WiFi
  - 100mA over LoRA
  - 50mA when idle with radio on
  - 40mA when idle with radio off
  - 80uA in deep sleep
  - Average under 1mA , lifetime = 1-5 years
Recap
- Wires are hard
- Measure and understand usage
- The basics off deep-sleep
- ESP32 Ultra-low-Power co-processor
- Design your own battery-friendly systems (see Arts Miniconf presentation)
- Project and monitor your battery lifetime
- Website

Deep Learning, Not Deep Creepy – Jack Moffitt

What is machine learning
- Make decisions based on statistics
How is deep learning different?
- Many layers of neurons each learning more sophisticated representations of features in the data
- Transfer learning – reuse N-network for similar task where less training data
- Generative Adversarial networks
The dark side of deep learning
- Works better with more data. Incentive for companies to get a huge amount of data
- Computationally very expensive – Creates incentive to move things to large clouds
- Inaccessible to smaller players
- Hard to debug, black boxes.
- Amplify biases in training data, somemays to fix but not generally fixable
- Data may be low quality
Machine learning @mozilla
- Deepspeech and common voice
- Deepspeech – state-of-art speech detection
  - Existing solutions owned by big companies. Costs $ and in cloud
  - Opening up models and train data will allow innovation
  - Based on baidu’s deep speech paper
  - pre-trained models for english
  - runs real-time on mobile
  - word error rate of 6.48% on librivox
  - streaming support
- Common voice
  - Crowd source voice data for new applications
  - 20 languages launched
  - 1800 hours collected so far
- Deepproof – spelling and grammar checker
  - Existing one is basically a keylogger ( Grammarly )
  - Needs to be small enough to run on device
  - Learn by example, rather than few rules
    - Less language-specific tuning
    - More scaleable
  - Local interface to avoid sending private text to several
  - 12 million 300-character chunks from wikipedia
  - Inject plausible mistakes
  - Real-life data
  - Maybe improve with federated learning without disclosing text
- Lpcnet
  - lots of test-to-speech are end-to-end
  - a separate network converts spectrograms to audio
  - GRiffin-Lim sounds bad
  - WaveNet / WaveRNN needs 10s GFLOPS
  - needs something for efficient for on-device
  - Currently 1.5-6 GFLOPS
  - real-time on mobile
  - Works okay
  - Other applications
    - Speech compression
    - noise compression
Questions:
- Does the audio slow-down work on non-speech? Not really
- How do you deal with region variations of speech and grammar – Common voice is collecting