Linux.conf.au 2019 – Friday – Keynote: A linux.conf.au Story – Rusty Russell

Rusty Russell
  • Bitcoin Billionare
  • 1992
    • The days of SunOS
    • Read the GNU Manafesto
  • 1995
    • Using GPP compiler at work
    • First patch accepted on November 1995
  • 1997
    • USENIX Conference in california
    • UseLinux – Had a track for Linux
    • Hung around a bunch of top guys in Linux talked about added SMP to Linux
    • Talk on porting Linux to Sparc by David Miller & Miguel de Icaza. Going into improvements and showing how Linux port to sparc bet Solaris in the Lmbench benchmarks on same hardware.
    • Relaized lived in a world where students could create and port OS that bet the original OS from the vendor
  • 1997 – 1998
    • Wrote (with another guy) and got ipchains added to Linux
    • “I woke up one morning and I was kernel firewall maintainer”
    • Got job people paid to work on Linux firewall code
  • 1998
    • Decided needed an Australian Linux conference
    • Oct-Nov visited a bunch of LUGS to invite people and find person to collect money.
    • People not sure what they wanted to go to a Linux conference ( $380 bucks)
    • Invited John Maddog Hall
    • Created and ran a slashdot ad
    • Created card got into $14k negative
    • Last session of the 3rd day, reran the 3 best talks
  • Three stories from 1998
    • Tutorial Books for each of the tutorials- Couldn’t get photocopies from commercial facility, so had to make 400 copies of books via 4 coin operated photocopiers
    • Tridge bought up a triple-CD burner. People ran it in relays
    • Somebody said. “I can’t believe you don’t have conference tshirts”. He bought white tshirts, got them screen printed and sold them.
  • End of conference Tridge organised a gift from the Speakers to Rusty. Pewter Beer mug
  • Linux.conf.au after 1999
    • 2001 scheduled 3 talks from Rusty. At the same time
    • Met Tridge at LCA – Moved to Canberra they did AusLabs
  • How Great Projects
    • Smart and Capable enough to complete them
    • They are Dumb enough to try
    • When somebody tells you about a project?
      • That sounds Great, Tell me more
      • What can I do to help
    • Enable people’s enthusiasms
    • Collaboration is a super Power
    • Get along with people is a skill
    • “Constructive absenteeism”
  • Headwinds to collaboration
    • Signs are welcoming to some people
    • Other people get signs that they are not so welcoming
    • Good are seeing them when they are aimed at them, not so good are even seeing they exist when they are not aimed at them.

Share

Linux.conf.au 2019 – Thursday – Session 3

Open Source Tools for Publishing and Processing Earth Observation Imagery – Paul Haesler

Paul Haesler
  • Golden age of satelite imagery
  • Geostationary – One area – Good for weather
  • Circum-polar orbits – all over earth every 10-16 days
  • Data processing Chain
    • Level 0 – Raw
    • Level 1 – Geo rectify – Measure surface radience
    • Level 2 – Curroect for sun, sat angle, atmostphere – ARD – Records surface reflectance
  • Landsat-8 (25-30m , 8-16 day cycle, data since 1982 ) Sentinel-2 ( better, 5 day cycle, 10m resolution)
  • Digital Earth Australia
  • The Problem
    • Open Data Cube
    • Python, based on xarry
    • Postgres for metadata
    • Actual satellite sata from local or network repo (transparently)
    • GUI maintained by CEOS
  • Nationalmap.govt.au
    • Aims to publish all Aus OpenData that can be mapped
    • Based on TerriaJS
    • Some DEA data was already being publish but need for additional stuff
  • COGs – Cloud Optimised GeoTIFFs
  • DataCube_ows
    • Lightweight web application server
    • Developed by Datda61 for GA
    • WMS
      • OGC Web Map service
      • Good for general-usage web apps
      • Returns standard images (eg png)
      • Support 1.3 well, works with most clients
      • Styles for band-mapping
      • on-the-fly solar angle correction
    • WCS
      • Version 1 supported
      • Works well with TerriaJS , works okay with QGIS or ArcGIS
  • Next Steps
    • WPS for on-the-fly processing is regularly discussed
    • Better ingegration with datacube-core
    • More recent WCS versions inc WCS-2EO
    • Sparse Data problems

The Tragedy of systemd – Benno Rice

  • References to Contempt Culture
  • Ancestry of Systemd
    • Unix: Happy accident, place, time, reaction to the previous thing
    • housekeeping functions – “mounting filesystems and starting daemons”
    • inetd – Super Daemon for all sockets – “worked well until The Internet Happened”
  • Then the Internet happened
    • forking a process per connection doesn’t scale
    • Lots of persistent state for things like databases
    • Service
      • Might be a bunch of processors
      • Init starts but doesn’t manage
      • initab can restart things in SystemV
  • System Config vs Service bootstrap
    • Mixed in togeather
    • Service management needs more
  • Windows NT
    • Service model there from beginning
  • MacOs
    • Application Model means lot richer interaction with the host
    • Application delegate
    • launchd
  • The Idea of Systemd
    • launchd
      • Service handling in MacOS
      • Took over init, inetd, cron
      • Can listen on ports. Start stuff. doesn’t need to start on boot, boot gets faster, power reduced, security improved
      • Move system services to daemons, then start daemons as needed
    • From Launchd to systemd
      • upstart
        • event driven
        • shell based
      • Rethinking PID 1 – Lennart
      • “Start less” , “Start more in parallel” , “listen to hardware and software changes”
      • cites launchd
  • System management
    • Everything is a lot more dynamic
    • Hotplug , DHCP , etc
    • Don’t install 15 different packages that all behave differently
    • But systemd will have to do things in a different way to those 15 other things
  • The reality of systemd
    • Widely adopted ( 2011 – 2015 )
    • Arguments
      • Violates the unix philosphy – actually systemd actually is many binaries
      • It is bloated and monolithic – Well it does do a lot of thing
      • It is buggy – So is all software, actually a good failure mode
      • I can’t stand Lennart Poettering – He’s delivered. “I won’t defend his community interaction”
      • It is not portable – UNIX is dead – Posix isn’t really a thing anymore, there are not a bunch of crazy Linux variations. “These days you have Linux and some rounding errors”
    • cgroups
    • User-Level units
  • Change – System is a lot of disruptive change
  • The Tragedy of Change
    • Nerds love change as long as we are the ones doing it
    • System boot ups using shell script interaction is like the old blanky we should of got rid of 20 years ago
    • The Knee-jerk – Abuse is not Cool
  • The Next Generation
    • They See a lot more APIs
    • Thinking in Containers is different from thinking in not-containers
  • What does Systemd have that FreeBSD (or even future Linux) could use, or could do better
    • Message Transport
    • RPC Framework
    • Kernel and Use-space services should look similarly to the services above them
    • Service Lifescycle
    • Automation via API – Easier for vendors to write appliances
    • Containers
    • The System Layer
      • Doesn’t have to be the only implementation of theis
    • Consistent Device Naming
    • Better Log/Event/Audit Handling
    • A new model of an application ( a bunch of things managed as a Unit, See the MacOS model)
  • Questions
    • Launchd option – Too MacOS specific
    • Dynamic Libraries = DLL Hell – Containers avoid, different problems
    • Is reaction to systemd scaring other big changes off – Possible, hard to write, very hard to handle the social issues to push though
    • Where is FreeBSD at? – A long way away, no consensus this sort of change needed
    • Should everything have been swallowed up – Thought experiment, If systemd had instead defined an API for separate projects instead of writing them itself, would that have worked? And now we do know what is needed could we switch to a separate model with APIs?
    • Enbeded Devices need systemd – Anything Dynamic needs it
    • What Push back from FreeBSD – Something like that but not systemd. Some like launchd
    • What needs to change in community and systemd team to make things better – See Adam Harvey’s talk on language changes. Hard since everythign is asking for different stuff, systemd people.
    • What should systemd go further into – Messageing and RPC stuff more pervasive and more thought about. Something into the kernel.

Share

Linux.conf.au 2019 – Thursday – Session 2

Go All-In! – Bdale Garbee

Bdale Garbee
  • How to get companies involved in FOSS
  • First contribution of source code was almost 40 years ago
  • Used Vax BSD at CMU – HAd the deal with an obscure priesthood
  • KA9Q TCP/IP stack for amateur radio in the 80s
  • Appearance of RMS in my world
    • GNU Manifesto
    • 4 Freedoms
    • GPL
  • Debian
  • Worked as LInux CTO from one of the largest IT companies in the world
  • Collaborative Development Model
    • Spread out besides just making FOSS
    • No one company in charge
    • Diverse range of contributors, massively different motivations
    • We get a software commons we all get to benifit from
  • Free Software means Freedom of Choice
    • Reduced barriers between users and producers of software
    • Any user can be a dev, or pay someone to dev
    • If upsteams goes bad, things can be forked
  • What it means to be successful when you are operating in an open and collaborative model?
    • The goal of a trad company is for investment to yeld technological control points
    • First mover advantage
    • Differentiated features, preferably patentable
    • Collaborative dev model allows us to recognise the benifit of collaroation on all the non-differentialting elements. Leavign more value to the users / customers
    • Thinks less about control points, more about points of affinity. What is it that would make a customer want to user you products or services?
  • Innovation these days largely takes place in the open space
  • Wrights/Goddard – They didn’t get told to to the next new thing, they just started it as a hobby
  • Free Software enables people who we don’t know exist to create innovation and invent things we can’t imagine
  • Long Tail of Contribution
    • Example: People who did one Linux Kernel contribution, often to fix on specific thing that was causing them problems.
    • No company on earth that can hire that resource
    • Needs to be easy for people to access the code and make contributions
  • Attributed of Successful Communities
    • Active contribution and collaboration
    • Diverse participation
    • Solid core of code
    • Recognizable mainline trunk
    • Unified, cohesive structure
    • Low barriers to entry
  • Choosing the right license
    • Businesses can only be successful with permissive licenses
    • The most successful projects seems to be communities built around open contribution
    • Share-alike licenses stop possible problem of Closed Corporate fork while the original project withers

Beach Wreck Ignition: Challenges in open source voice –
Kathy Reid

Kathy Reid
  • MycroftAI – One of the few open source voice stacks
  • Introduction to a Voice Stack
    • Wake Word – eg “Hey Alexie”
    • Utterance – Phrase of command
    • Speech2text processor
    • Looks for keywords etc
    • Runs a command
    • Dialog – acknowledger + response
  • Wake Word
    • PocketPhinx, Snowbox, Mycroft Ai Precise
    • Some use Phonemes (smallest units of sounce in a language)
    • Hard to tell differences between all words
    • Always listening, connected to internet
    • Some use Use Neural networks
    • Low accuracy can cause frustration
      • Bias towards male speaker (10:1 male:female in dataset). Also more with American than other accents
      • To unbias the sample had to tag the samples with ethnicity, gender etc. Which was a problem with ethics of taggign samples/speakers
  • Speech to Test
    • Kaldi – no network needed, compute heavy
    • Deep Speech – From mozilla
    • Challenges
      • Lots of accents out there. Hard
      • Only trained for most common accents
      • Also problem with regional slang
      • Need to train on individual speaker
      • But need lots of data to understand a speaker
  • Endangered Languages
    • No commercial imperative to cover them
    • Mycroft Translate using Pootle to translate command words to 40 languages
    • Issues for gendered languages, formality
  • Intent Parsers
    • Rasa, Mycroft Adapt, Mycroft Padatious
    • Intent Collisions – Use confidence scoring depending on how explicit the request is.
  • Text to Speech
    • Mary TTS, Espeak, Mycroft Mimic, Mycroft Mimic 2
    • Mimic recording studio, Need 40-60 hours audio
    • Challenges
      • Natural sounding voice – making the voice sounds not robotics
      • Pronunciation – often requires after creation

Share

Linux.conf.au 2019 – Thursday – Session 1

A Long Day’s Journey Into Backups – Rachel Kelly

Rachael Kelly
  • A journey in four stages
  • Version 1 – State of Backups Aug 2017
    • Needed to look though old logs to see how far back problem wentDaily diffing from duply/duplicity – But where was the original?
    • Tried to restore data from old backups
    • Couldn’t restore from backup since original was too old
    • Couldn’t get it to work, needed something new fast
  • Version 2
    • Created tarball, uploaded to AWS S3, via daily cron
    • Done quickly
    • Not reliable sending to s3
    • Needed ongoing work
    • Big Win: But at least complient, and we could get data back out
  • Try 3 : Shiny EFS
    • EFS is AWS’s NFS solution
    • tarball created on local EFS disk, easypeasy
    • Big Win: Reliable backups, incorporated into infrastructure, retrievable data. 8 weeks off backups
    • Miscalculated cost: About 10x original estimate
  • Try 4: Tarball to s3 redux
    • Tarball to s3. PLUS infrastructure
    • Would work going forward
    • S3 approx 1/10 the cost of EFS
    • Big Win: Reliable, inexpensive, functional, dependable
    • Discovered that EFS not encrypted
    • Able to manage well
  • Current Solution
    • cron job calls a bash script
    • Safety in bash: set -euo pipefail
    • tar up the filesystem (with exclusions)
    • Set it to s3 with aws cli (with hosts’s hostname)
    • After 56 days is sent to glacier
    • restore script to get a file (uses AWS credientials of current instance)
  • What’s Next?
    • I could work on backups forever and they will never be perfect
    • Ability to restore to a new instance
    • Want to be able to get files from anywhere
    • Microservice backups
    • Deglacierisation
    • What we need
    • Better CLI tool – safer
  • What I really really want
    • A decent enterprise solution
    • NIH is a dangerous habit
    • Speaker knows no one using a hosted enterprise solution
    • Vendor solutions seem to be crazy overkill
  • Feedbacks
  • Amanda Recomended
  • Every morning restore dev enviroment from anon prodouting one. Comet Backup
  • Wasabi compatable with s3
  • Recoment “Retrospec”
  • tarsnap
  • bacula
  • rsnapshot
  • Looking at Borg Back
  • rsync to zfs, zfs send
  • Personal backups, using duplicity
  • Industry 4.0
    • After mechanization, mass production and automation
    • The machines run everything and we reap all the benefits – maybe
  • Robot Hype
    • Post AI winter
  • Implementoer’s bias (top down design of neutral network setup, choose number of layers etc)

Share

Linux.conf.au 2019 – Thursday – Keynote: Shannon Morse

Personal Branding for the Security Conscious – Shannon Morse

Shannon Morse
  • Who am I
    • Youtube videos on Infosec, Travel
  • Imposter Syndrome
    • Work hard to beat it.
    • Say Yes to offers
    • Work hard to make somethign I am proud of
    • Surround yourself with positive people
  • Accomplishment
    • Keep a list of them, be proud of them
  • Backstory
    • No background in Linux, hacking, infosec
    • Mom and Dad supported me
  • RTFM Sucks
    • Lots of egos and elitism in forums and community online
    • Decided to become the resource for learners
  • Starting your career
    • What do companies need, what hiring for
    • How has industry changed?
    • Diversity numbers?
    • Can you change industry in a positive way
    • Review CERT holders numbers vs Openings looking for those certs
    • Look at job titles being advertised
    • Industry growing -> lots of beginners
  • How can you get good at it
    • Understand what is your best way to learn
    • Read books, classes, videos, whatever
    • Compile your list of passions
    • Get list of influencers / thought leaders / speakers in the area
    • Follow them on social media
    • Learn from your role models
    • you might end up being a thought leader in their eyes
    • Follow people in other areas too
  • Keep learning
    • Do it every single day
    • Make it become a habit
    • Make it a routine
  • Resume
    • Create a one-pager
    • Business cards
    • Dropped out of college put “Huitus”
  • Build you platform
    • Youtube, write articles, videos, whatever
    • If you can afford it, offer free classes for under-represented groups
  • Personal brand
    • Develop the blocks
      • skillsets, values, what does it mean for you to succeed
      • What obstacles have you overcome
      • what are you passions
      • what makes you unique
      • write and live by your vision statement
    • If you don’t control and manage your brand others will do it for you
      • Where do you draw that privacy line?
      • Quiz yourself
      • Eg how public are you on you income?
    • Resources
      • Joindeleteme.com
      • password managers
      • 2FA
      • Guest vs home Networks
    • Clean up your social media accounts, delete old junk
    • Smart sharing
      • Share stuff but not barcodes, addresses
      • Have a plan
    • Be ready to deal with targeted harassment
      • Keep notes, screenshots, know who to contact
      • Trolls? Block; banhammer
      • Troll back (YMMV)
  • Why I don’t quit
    • Do it because you love it

Share

Linux.conf.au 2019 – Wednesday – Session 3

Why linux.conf.au continues to amaze 20 years on!
Steven Hanley, Angus Lees, Hugh Blemings

Steven, Angus, Hugh
  • Three people who have attended every Linux.conf.au
  • What has happened to keep it relivant and how it has evolved
  • Focus on Open Source Software , These days hardware too
  • Pre-history
    • Australia big in early Linux
    • late 90s Linux and LUGS growing in Aus
  • CALU
    • In 1998 Rusty toured LUGS and organised CALU in July 99 in Monash
    • 9 July – 11 July
    • 6 tutorials and 16 talks
  • Post-CALU
    • Lots of excitement
    • Height of dotcom boom, Big Linuxcare Expo in Darling Harbour in 2000. Very commercial
    • Push for another one
  • 2001
    • UNSW interested in Lions memorial conference
    • CALU type conference would be good
    • Sydney big due to Olympics
    • Domain picked
  • Challenge of early events
    • Everythign was new
    • CFP wasn’t formalised
    • Linux Aus still in infancy
    • Open Source was new and Few conferences
    • Few people worked fulltime in FOSS
    • sceduled over a weekend cause people didn’t get holidays
  • Early objectives we hold on to
    • Community rather than commercial, modest size
    • Easier for students and hobbists, low price
    • tech/non-tech balance to encource interesting delegates
    • Miniconfs
    • Speakers treated well
    • Timeing to make conference attractive (Summer!)
    • University venue, dorms, communal accom, Holidays
  • Miniconfs
    • First in Brs in 2002. more in 03, formalised in 2004
    • Open to all delegates
    • Incubate possible future conferences
    • Fill the week without adding more effort
    • Try out nice topics, extended BOFS
    • Practice ground for new speakers
  • Growth and Roaming
    • Change of location helps
    • New team help avoids burnout and bring fresh ideas
    • Allows more people to contribute
    • Repeat city visits with new people involved
    • Allows people who are less-technical to help out
    • Bid process introduced, overseen by LA
    • Specifics of venue and location help teams structure their event
    • New locations add tourism aspect, encourages aspect
    • Positive experience shared by work-of-mounth
  • Expansion
    • Will anyone go to Perth? NZ?
    • Infinite growth is not a goal (complexity, conference atmosphere)
    • New activities and events are continuously evaluated
  • Call for papers
    • Originally ad-hoc
    • Seperate Panel since 2005
    • See Mary’s blog post in 2006
    • 4-5 times proposals to slots
    • Process
      • Actively solicit hot speakers / topics
      • Review submissions individually
      • face-to-face to reach final as group
    • Conference organisers set overall theme, choose keynote speakers
  • Conference MNGT software
    • Much NIH
    • New software in 2004, 2005, 2007, 2017
    • Mainly for CFP has hooks to other conference components (scheduling, badges, website)
  • Ghosts of conferences Past
    • Mailing list to ask
    • Visit conference organisers
    • Some people have remained over multiple years
  • Giving Back
    • Donation to a charity, action off a shirt singned by speakers
  • Event Style and Flow
    • The current event is typical
    • Moved from Weekend to week as people get paid by employer
    • Less Beer, More Food
  • Giving back to the technical commons
    • Regional delegate program ( 2004 for a few years )
    • Stories of laptops being fixed by the guy who dev’d it
    • git is an outcome of “no more bitkeeper” in 2005
    • Encouraging work done once talk has been accepted
    • The Debian couch had no back so he had to sit around and support each other
  • Sponsorship
    • Some Very long term
    • Some years had to educate sponsors on how to participate
    • Local vendors and sponsors
    • Nurturing sponsosrs takes consious effort
  • Learning lessons
    • Floods (Brisbane 2011)
    • Budget issues (where LA comes into play)
    • CFP feedback loop has sought to tweak technical/non-tech focus
    • Code of conduct has made conf better
    • People step up to make things happen
  • Resources
    • Mirror
    • Mary Gardiner’s post on getting a talk accepted
    • Simon Lyall’s guide

Right to Not Broadcast – Karen Sandler

Karen Sandler
  • Thought the problem with the propitiatory device in her heart was about transparency, now feels it is more about control
  • Got a new device where the programmer (controller) only worked when it was touching, rather than over a distance.
  • Research team bought a device on ebay that had patients data still in it.
  • On the other hand the Keynote Speaker this morning had to hack into her own device to get the info.
  • Sleep Apnea machines transmitting data to both doctors and insurance companies
  • Smart TVs
    • Listening for wake words all the time
    • Sending viewing data to 3rd parties
  • Various Legislation
    • HIPAA , NZ Health Information Privacy Code, Aus Privacy Act, GDPR
    • GDPR – Europe’s gift to the rest of the world
  • ” Incorporating connectivity means we can never be totally in control of our critical information “
  • The environment/reason we provide the data in now may change
  • Often the non-connected option is marginal or doesn’t even exist.
  • [ Laptop ran out of battery here ]
  • Things need to be worked on
    • Raising the awareness of the non-networking, privacy-first issue
    • Even among the LCA-type crowd

Share

Linux.conf.au 2019 – Wednesday – Session 2

Around the world in 80 Microamps: ESP32 and LoRa for low-power IoT – Christopher Biggs

Christopher Biggs
  • Promise of IOT
    • Control everything
    • Sensors everywhere
    • Reduce cognitive load
  • Problem
    • Computers everywhere = wires everywhere
    • Can’t be done every time
    • But em if you got em
    • eg Power over ethernet, Ethernet over power, Ethernet over coax
    • Wifi is great for connections. But what about power?
  • Batteries are bad
    • Lead acid – obsolete just about everywhere
    • Single-use dry cells – leak
    • Nickel Metal-hydride – Some stuff
    • Lithium – Everything else
    • Sample battery
      • Chap-stick battery = 2 Amp/hours
      • 3.7 volts
      • Labels on batteries often lie – you need to always verify
      • Energy capacity is quoted for 20h discharge, not linear relationship
    • But they are geting better due to phones, scooters, drones pushing
  • Off the shelf solutions for packs
    • Smart ones may turn themselves off if draw very low
    • Cell plus simple system works
    • Cell with “Battery Managmnet system” is a bit more complex
    • Solar panels are useless, needs to be a4, a4, a5 size at least
    • Linux systems too much draw for non-wired sensors, need to be used as hubs
  • Computers can spend most of time asleep
    • Config one or more wake-interupts
    • Arduino deep sleep – Sleep consumption as low as 6 micro amps. ( 38 years with Chapstick battery)
    • Watch out for stock voltage regulators (eats 10 mA)
  • ESP 8266 Sleep modes
    • Several levels off sleep modes
    • Wake up every 5 minutes = 1 week battery life
  • ESP 32
    • Sleepier modes
    • Complex sleep patterns.
    • Ultra-low coprocessor
      • 4 register, 10 instructions, 16-bit, special slow memory
      • Can be configed to wake up at intervals
      • Can go back to sleep or wake the main cores
    • ULP in practice
      • Write code, load into the ULP processor
      • Enough code to decide to go back to sleep or wake up main processes
  • Aim for efficiency
    • Sleep as much as possible
    • Use interrupts not polling where possible
  • Nasty Surprises
    • Simple resistors ladders leak power
    • Linear regulators leak power
    • Poor antennas cost watts
    • Beware: USB programming bridges that are always on
    • Almost all the off-the-shelf IoT boards are no good for permanent installation
  • Solutions
    • Turn of everything you are not using
    • radio turn off when not in use, receiver turn it on now and then. Do store-and-forward
    • slow down the cpu, turn off bluetooth
    • Reduce brightness of lights
    • BE careful about cutting out safety features
  • Case Study – Smart water meter in multi-tenant building
    • Existing meter has a physical rotating dial, can count rotations
    • In cellar with no power
    • Create own
      • ESP-32
      • Wifi for setup or maint
      • LoRA for comms every 15 minutes
      • ULP monitors 4 sensors
      • ULP wakes CPU after number of elasped minutes and/or pulse
      • Transmits to Linux-based hub covers building
      • 150mA WiFi
      • 100mA over LoRA
      • 50mA when idle with radio on
      • 40mA when idle with radio off
      • 80uA in deep sleep
      • Average under 1mA , lifetime = 1-5 years
  • Recap
    • Wires are hard
    • Measure and understand usage
    • The basics off deep-sleep
    • ESP32 Ultra-low-Power co-processor
    • Design your own battery-friendly systems (see Arts Miniconf presentation)
    • Project and monitor your battery lifetime
    • Website

Deep Learning, Not Deep Creepy – Jack Moffitt

Jack Moffitt
  • What is machine learning
    • Make decisions based on statistics
  • How is deep learning different?
    • Many layers of neurons each learning more sophisticated representations of features in the data
    • Transfer learning – reuse N-network for similar task where less training data
    • Generative Adversarial networks
  • The dark side of deep learning
    • Works better with more data. Incentive for companies to get a huge amount of data
    • Computationally very expensive – Creates incentive to move things to large clouds
    • Inaccessible to smaller players
    • Hard to debug, black boxes.
    • Amplify biases in training data, somemays to fix but not generally fixable
    • Data may be low quality
  • Machine learning @mozilla
    • Deepspeech and common voice
    • Deepspeech – state-of-art speech detection
      • Existing solutions owned by big companies. Costs $ and in cloud
      • Opening up models and train data will allow innovation
      • Based on baidu’s deep speech paper
      • pre-trained models for english
      • runs real-time on mobile
      • word error rate of 6.48% on librivox
      • streaming support
    • Common voice
      • Crowd source voice data for new applications
      • 20 languages launched
      • 1800 hours collected so far
    • Deepproof – spelling and grammar checker
      • Existing one is basically a keylogger ( Grammarly )
      • Needs to be small enough to run on device
      • Learn by example, rather than few rules
        • Less language-specific tuning
        • More scaleable
      • Local interface to avoid sending private text to several
      • 12 million 300-character chunks from wikipedia
      • Inject plausible mistakes
      • Real-life data
      • Maybe improve with federated learning without disclosing text
    • Lpcnet
      • lots of test-to-speech are end-to-end
      • a separate network converts spectrograms to audio
      • GRiffin-Lim sounds bad
      • WaveNet / WaveRNN needs 10s GFLOPS
      • needs something for efficient for on-device
      • Currently 1.5-6 GFLOPS
      • real-time on mobile
      • Works okay
      • Other applications
        • Speech compression
        • noise compression
  • Questions:
    • Does the audio slow-down work on non-speech? Not really
    • How do you deal with region variations of speech and grammar – Common voice is collecting

Share

Linux.conf.au 2019 – Wednesday – Session 1

Filesender: Sending large files across facilities – Ben Martin

Dr Ben Martin
  • 10 Year old project
  • Web based File Sharing
    • Quick 1-to-few file sharing between people
    • Files go away after a month by default
    • Simpler to run than anon-ftp etc
    • Stats of downloads available to sharer
    • User only needs web browser
    • Upload resume, important with TB sized files
    • Notifications
    • share with explicit groups
    • Browser-to-browser encryption of data AES-256
    • SAML for auth scale
    • GDPR by default, about privacy page
  • Overview
    • Server side is PHP
    • Client side JS with light widgets
    • MariaDB
  • Server Storage
    • Chunked 5MB files
    • Cept used at aarnet
  • Downloading
    • On the fly zip64 archive creation
    • One of more files listed per transfer
    • Links for console download if needed
  • Dragons
    • Auto Downloads and fast uploads cross browser is HARD
    • Mixed browsers
    • Long uploads can exceed auth sessions times
    • Web crypto support w3c
    • People use ancient versions of databases
  • Lots of details on the Database and Encryption. Sounds like both have improved to a good state
  • Future
    • UI refresh
    • Mobile App
    • E2E Encryption
    • Docker image for easy setup
    • More SAML info, apache config?
    • Integration of Endpoints ( auto youtube etc )
    • Session Clone to investigate problems (but privacy?)
    • Run the whole thing in the cloud
  • Questions
    • Command line? – REST API (php and client)
    • RSYNC for slightly changing files – Being investigated

Hot Potato – James Forman and Callum Dickinson

James and Callum
  • What is Hot potato
    • Not a monitoring System
    • monitoring System -> Hot potato -> On-call person
    • Web app in python and flask
    • Tells you things and stays out of the way
  • Why ?
    • Spark shutdown paging Network
    • Needed quick version
  • Goals
    • Don’t get in the way
    • Alert reduction
    • Highly available
    • Support any System – Nagios family now, Prometheus later
    • Support methods – Pushover, SMS, Paging
  • What else can it do
    • Failure notifications when contacts are not working
    • Heartbeats so know when monitoring system is down
  • Planning stuff to add to it
    • Teams – put everyone on call
    • Team escalations
    • Planned work ( go to person working before oncall, extend windows )
    • Support Hotline integration
    • Mobile App
    • Adding German and Italian
  • How it works
    • Flask App
    • RabbitMQ
    • Database (cockroachDB)
    • Apps talk via the Databases
    • Alert -> Object in DB -> Put on Queue -> Worker
    • Worker -> Get details to send to -> Try to send -> Store result in DB
    • If Failure fails then work it’s way though the list.
  • Questions:
    • ACK can use pushover so don’t have to login to app
    • Looking at teams functions
    • CockroachDB picked since it seems very reliable
    • Not sure about restoring/calendaring features going in since need to make it generic?
    • Endpoints fairly modular so should be extendable to new ones.
Share

Linux.conf.au 2019 – Wednesday – Keynote

#WeAreNotWaiting: how open source is changing healthcare
– Dana Lewis

Dana Lewis
  • Getting diagnosed with a chronic disease is like being struck by lighting.
  • Insulin takes a while to kick in
  • Manual diabetes
    • Have to check level over and over again
    • Have to judge trend and decide more insulin, food, exercise, etc
    • Constantly
  • Device
    • Windows-only interface software to access
    • Alarm not great, various other limitations
  • Idea
    • Pull data off the device, create smarter app
    • Hard to do
  • First version
    • Device -> WinPC -> dropbox -> app -> pushover -> phone
  • 2nd Version
    • Press button to indicate what she is doing (eating, sleeping) when get alert
  • 3rd version
    • Hook into Insulin pump to do it automatically
    • Replace the human do they same thing over-and-over again in the loop.
    • all portable
    • Person doesn’t have to wake up to adjust, petter sleep
  • OpenAPS
    • Open Source
    • Created a list of ways it could fail (battery fails, wires come out)
    • Focusing on safety
    • Limiting dosing ability in hardware and software
    • Failing back safely to standard device operation
  • Approx 1000 users
    • 9 million+ hours of DIY closed loop experience
    • Anonymized dataset available
  • Sample child user
    • Before: 4.5 manual interventions per day by parents
    • After: 0.7 per day
  • School Child ( 5 vs 6th Grade )
    • 420 visits to school nurse ( 2.3 /day ) – 66 visits for events
    • 5 visits with OpenAPS – 3 visits (Gym related)
  • Intel Edison Platform
    • Smaller than Raspberry Pi
    • But discontinued (looking for old ones to replace)
  • Going back to Pi, now smaller
    • Has built in display with “Explorer Hat”
  • Outsiders are building stuff because they can and the traditional companies are not meeting the need. Innovate with small solution and build it up
  • Ask what little things you can do for people with similar problems. This started with just “Make a louder alarm”.

Share

Linux.conf.au 2019 – Tuesday – Session 3 – Docs Down Under

How to Avoid Meetings – Maia Sauren

Maia Sauren
  • What Distributed/International Teams involves
    • Lots of late meetings
    • Not up on the inclusive languages
    • We’d language quirks from ESL speakers
    • Taxonomy changes between fields
    • Stereotypes are incomplete
    • Ask Culture vs Guess Culture
    • Micro-cultures , down to schools, extra years.
    • When you make a private joke without context, someone is left out
    • What governance model wins (who do they raise barriers for)
    • It’s harder to change a relationship over the phone than maintain one
    • A relationship with a CoC is less fragile (shouldn’t be figured out in the fly during conflict)
    • “How do you want to have arguments?”
    • Set standards early and resolve swiftly
    • Normalise conflict resolution
    • Adulting: it’s for people who don’t want to cry even more later
    • What have you done this week you are proud of?

Disaster Recovery Book – Svetlana Marina

Svetlana Marina
  • Software Development Process
    • Speaning all time “firefighting” so no time to improve processes
  • Operation Support Model
  • Incident Model
    • Alert Raise
    • Initial Response
      • Runbook should have: Alert type -> Investigation help, exact queries into log search, links
    • Impact, escalations, SLA
      • Send email to stakeholders, keep their trust
      • Include “communication required” in runbook
    • Investigation and Damage control
      • Do what is required to fix problem, not fix the root cause
      • Message to stakeholders again
    • Through Investigation (depends on situation)
    • Fix root cause
    • Post Incident Review

The Bus Plan: Junior Staff Training – Andrew Jeffree

Andrew Jeffree
  • Staff coming into industries with lots of automation, but what when automation fails?
  • Staff only know about automation, can only run a playbook, don’t understand what it does
  • Disclaimer: Automation isn’t bad.
    • Config Managment, Custom tooling, CICD, Infrastructure as code
  • Buttons
    • Need to understand what they do
    • Sometimes they are not there
    • Can be Hard to implement a button that doesn’t exist
  • Do it the manual way
  • Training
    • Do something manually (install wordpress) and document
    • Expand what you have done
    • Tweak it
    • Break and fix it
    • Don’t be afraid to change something manually
  • Questions
    • Split for new people – 30% training , 70% work
    • Manual -> ansible -> puppet
    • Don’t want them to be stuck in the mindset that everthing we do is the best way

When Agile Doesn’t Work Anymore: Managing a Large Documentation Project – Lana Brindley

Lana Brindley
  • We all age
    • Sometimes old docs should just be binned
  • Old docs base, massive changes needed, but wanted to save it
    • Changes to toolchain too
  • Proof of concept
    • Define how big the job will be, what is involved
    • Get buy-in
  • Plan, Plan, Plan
    • Create a real timeline
    • Advertise your plan, tell everybody about it
    • Do a presentation to team for each phase
  • Research
    • Who is your audience?
    • No, really, who is your audience? – eg Sales and support may use docs more than customers
    • Lets the customers know that “somebody cares about these docs”
  • User/task analysis
    • Where do we need to focus work
    • What are the big tasks
  • Do the thing
    • You have to sit down and write it
    • You are breaking the agile system
    • You need to track your work and who doing what
    • eg this case, moving from big chapters to small topic-based docs.
    • Agree ahead of time on process, have buy-in
  • Review
    • What went on?
    • WHo not to stuff it up next time
    • Try not to blame people too much
  • Agile vs … something else
    • Sometimes Agile is not the best model
  • Tips for working outside Agile
    • Outreach
      • Especially product owners.
      • Get people on board
    • Track your work
      • Create Epics of sprints
      • Don’t go overboard
    • Should about it
      • outreach never stops
      • Present at sprint reviews
      • Brownbags

Share