Category: Linux.conf.au

Dealing with Contributor Overload Holden Karau

• Developer Advocate at Google
• Apache Spark, Contributor to BEAM

Some people from big projects, some from projects hoping to get big

• Remember it’s okay to not fix it all
• The fun of a small project
  • Simple communication
  • Aligned incentives
  • Easy to tell who knows what
  • Tight community
• The fun of a parge project
  • More people to do the work
  • More impact and people thanking you
  • Lots of ideas and experiences
  • If $s then fun conferences
  • Get paid to work on it.
• Is my project on Fire? or just lots of people on it.
  • Measurable
    • User questions spike
    • issue spike
  • Lesss measurable
    • Non-explicit stuff not being passed on
• Classic Pipeline
  • Users -> contributors -> committers _> PMC
  • Each stage takes times
  • Very leaky pipeline, perhaps it leaks too much
• With hyper-growth project can quickly go south
  • Committer:user ration can’t get too far out.
• Even without hyper-growth: sadness
  • Same thing happens, but slower
• Overload – Mitigation
  • You don’t have to answer everyone, this can be hard
  • Stackoverflow
  • Are your answers easily searchable
  • Knowledge base – “do you mean”
  • Take time and look for patterns in questions
  • Find people who like writing and get to to write a book
    • Don’t to for core committers, they will have no time for anything else
• Issue overload
  • Try and get rid of duplicate tickets
  • Autoclose tickets – mixed results
• How to deal with a spike
  • Raise the bar
  • Make it easier
  • Get Perl to solve the problem
• Raising the bar
  • Reject trivial changes – reduces the onramp
  • Add weird system – more posts on how to contribute
• What can Perl solve
  • Style guide
  • bot bot bots
  • make it faster to merge
  • Improve PR + reviewer notice
  • Can increase productivity
• Add more committers
  • Takes time and effort
  • People can be shy
  • Make a guide for new folks to follow
  • Have a safe space for people to ask questions
• Reduce overhead for contributing well
  • Have doc on how to contribute next to the code, not elsewhere that people have to search for.

The Open Sourcing of Infrastructure Elizabeth K. Joseph

The recent history of infrastructure

• 1998
  • To make a server use Solaris or NT. But off a shelf
  • Linux seen as Cheap Unix
  • Lots of FUD

Got a Junior Sysadmin Job

• 2004
  • Had to tell people the basics “What is free software?”  , “Using Open Source Web Applications to Produce Business Results”
  • Turning point LAMP stack
  • Flood of changes on how customers interacted with software over last
    • Reluctance to be locked-in by a vendor
    • Concerns of security
    • Ability to fix bugs ourselves
    • Innovation stifled when software developed in isloation

Last 10 years

• Changes in how peopel interacted with software
  • Downtime un-acceptable
  • Reliance of scaling and automation
  • Servers as Pets -> cattle
  • Large focus on data

Open Source is now Ubiquitous

• Even Microsoft is using it a lot and interacting with the community

Operations tools were not as Open Sourced

• Configuration Management
  • puppet modules, chef playbooks
• Open application definitions – juhu charms, DC?OS Universe Catalog
• Full disk images
  • Dockerhub

The Cloud

• Cloud is the new propriatory
• EC2-only infrastructure
• Questions you should ask beforehand
  • Is your service adhering to open standards or am I locked in?
  • Recourse if the company goes out of business
  • Does vendor have a history of communicating about downtime and security problems?
  • Does vendor responds to bugs and feature requests?
  • Will the vendor use data in a way I’m not comfortable with?
  • Initial costs may be low, but do you have a plan to handle long term, growing costs
• Alternatives
  • Openstack, Kubernetes, Docker Swarm, DC/OS with Apache Mesos

Hybrid Cloud

• Tooling can be platform agnostic
• Hard but can be done

How to run Kubernetes on your spare hardware at home, and save the world Angus Lees

• Mainframe ->
• PC ->
• Rackmount PC
  • Back the rackmount PC even with built-in redundancy will still fail. Or the location will go offline, or your data spreads across multiple machines
• Since you need to have distributed/redundancy anyway. New model (2005). Grid computing. Clever software, dumb hardware. Loosely coupled servers
  • Libraries > RPC / Microservices
  • Threadpool -> hadoop
  • SQL -> key/store
  • NFS -> Object store
  • In-place upgrades -> “Immutable” image-based build from scratch
• Computers in clouds
  • No cases. No redundant Power, journaling on filesystems turned off, etc
• Everything is in clouds – Secondary effects
  • Corperate driven
  • Apache license over GPL
  • Centralised services rather than federated protocols
  • Profit-driven rather than scrating itches
• Summary
  • Problem
    • Distributed Systems hard to configure
    • Solutions scale down poorly
    • Most homes don’t have racks of servers
  • Implication
    • Home Free Software “stuck” at single-machine architecture
• Kubernetes (lots of stuff, but I use it already so just doing unique bits)
  • “Unix Process as a service”
  • Inverts the stack. Data is important then app. Kernel and Hardware unimportant.
  • Easy upgrades, everything is an upgrade
  • Declarative API , command line interface
• “We’ve conducted this experiment for decades now, and I have news for you, Hardware fails”

Hardware at Home

• Raid used to be “enterprise” now normal for home
• Elastic compute for home too
• Kubernetes for Home
  • Budget $100
    • ARM master nodes
    • Mixed architecture
  • Assume single layer-2 home ethernet
  • Worker nodes – old $500 laptops
    • x86-64
    • CoreOS
    • Broken screens, dead batteries
  • 3 * $30 Banana pis
    • Raspberry Pi2
    • armv7a
    • containOS
  • Persistentvolumes
    • NFS mount from RAID server
  • Service – keepalived-vip
  • Ingress
    • keepalived and nginx-ingress , letsEncrypt
    • Wildcard DNS
  • Status
    • Works!
    • Printing works
    • Install: PXE boot and run coreos-install
  • Status – ungood
    • Banana PIs a bit too slow.
  • github.com/anguslees/k8s-home

Is the 370 the worst bus route in Sydney? Katie Bell

• The 370 bus
  • Goes UNSW and Sydney University. Goes around the city
• If bus runs every 15 minutes, you should not be able to see 3 at once
• Newspaper articles and Facebook group about how bad it is.
• Two Questions
  • Bus privitisation better or worse
  • Is the 370 really the worst
• Data provided
  • Lots of stuff but nothing the reliability
  • But they do have realtime data eg for the Tripetime app (done via a 3rd party)
  • They have a API and Key with standard format via GTFS
• But they only publish “realtime” data, not the old data
  • So collected the realtime data, once a minute for 4 months
  • 557 GB
• Format
  • zipfile of csv files
  • IDs sometimes ephemeral
  • Had to match timetable data and realtime data
  • Data had to be tidied up – lots
• Processing realtime data
  • Download 1 minute
  • Parse
  • Match each of around ~7000 trips in timetable (across all of NSW)
  • Write ~20000 realtime updates to the DB
  • Running 5 EC2 instances at leak
  • Writing up to 40MB/s to the DB
• Is the 370 the worst?
  • Define “worst”
  • Found NSW definition of what an on-time bus is.
  • Now more than 5:59 late or 1:59 early. Measured start/middle/end
  • Victoria definition strictor
  • She defined:
    • Early: more than 2min early
    • On time: 2m early – 5 min late
    • late more than 5m late
    • Very late – more thna 20m late
  • Across all trips
    • 3.7 million trips
    • On time 31%
    • More than 20m late 2.86%
  • Best routes
    • Nightime buses
    • Outside of Sydney
    • Shorter routes
    • 86% – 97% or better
  • Worst
    • Less than 5% on time
    • Longer routes
    • 370 is the 22nd worst
      • 8.79% on time
  • Worst routes ( percent > 20 min late)
    • 23% of 370 trips (6th worst)
    • Lots of Wollongong
  • Worst agencies
    • No obvious difference between agencies and private companies
  • Conclusion
    • Privatisation could go either way
    • 370 is close to the worst (277 could be worse) in Sydney
  • bus-shaming.com
  • github.com/katharosada/bus-shaming

Questions

• Used Spot instances to keep cost down
• $200 month on AWS
• Buses better/worse according to time? Now checked yet
• Wanted to calculate the “wait time” , not done yet.
• Another feed of bus locations and some other data out there too.
• Lots of other questions

Executive director of Software Freedom Conservancy

Previously spoke that LCA 2012 about closed-source software on her heart implant. Since then has pivoted career to more open-source advocacy in career.

• DMCA exemption for medical device research
• When you ask your doctor about safety of devices you sound like a conspiracy theorist
• Various problems have been highlighted, some progress
• Some companies addressing them

Initially published paper highlighting problem without saying she had the device

• Got pushback from groups who thought she was scaremongering
• Companies thinking about liability issues
• After told story in 2012 things improved

Had to get new device recently.

• Needed this disabled since her jobs pisses off hackers sometimes
• All manufacturers said they could not disable wireless access
• Finally found a single model that could be disabled made by a European manufacturer

 

Note: This is a quick summary, Lots more covered but hard to cover. Video should be good. Her slides were broken though much of the talk be she still delivered great talk.

Collaborating with Everybody: Open Source Drug Discovery

• Term used is a bit undefined. Open Source, Free Drugs?
• First Open Source Project – Praziquantel
  • Molecule has 2 mirror image forms. One does the job, other tastes awful. Pills were previously a mix
  • Project to just have pill with the single form
    • Created discussion
    • Online Lab Notebook
    • 75% of contributions were from private sector (especially Syncom)
    • Ended up finding a approach that worked, different from what was originally proposed from feedback.
    • Similar method found by private company that was also doing the work
• Conventional Drug discovery
  • Find drug that kills something bad – Hit
  • Test it and see if it is suitable – Led
  • 13,500 molecules in public domain that kill maleria parasite
• 6 Laws of Open Scrience
  • All data is open and all ideas are shared
  • Anyone can take part at any level of the project
• Openness increasing seen as a key
• Open Source Maleria
  • 4 campaigns
  • Work on a molecule, park it when doesn’t seem promising
  • But all data is still public
• What it actually is
  • Electronic lab book (80% of scientists still use paper)
  • Using Labtrove, changing to labarchives
  • Everything you do goes up every day
  • Todo list
    • Tried stuff, ended up using issue list on github
    • Not using most other github stuff
  • Data on a Google Sheet
  • Light Website, twitter feed
• Lab vs Code
• Have a promising molecule – works well in mice
  • Would probably be a patentable state
  • Not sure yet exactly how it works
• Competition – Predictive model
  • Lots of solutions submitted, not good enough to use
  • Hopeful a model will be created
• Tried a a known-working molecule from elsewhere, but couldn’t get it to work
  • This is out in the open. Lots of discussion
• School group able to recreate Daraprim, a high-priced US drug
• Public Domain science is now accepted for publications
• Need to to make computers understand molecule digram and convert to representative format which can then be search one.
• Missing
  • Automated links to databases in tickets
  • Basic web page stuff, auto-porting of data, newsletter, become non-profit, stickers
  • Stuff is not folded back into the Wiki
• OS Mycetoma – New Project
  • Fungus with no treatment
  • Working on possible molecule to treat
• Some ideas on how to get products created this way to market – eg “data exclusivity”

 

Beyond Web 2.0 Russell Keith-Magee

• Django guy
• Back in 2005 when Django first came out
  • Web was fairly simple, click something and something happened
  • model, views, templates, forms, url routing
• The web c 2016
  • Rich client
  • API
  • mobile clients, native apps
  • realtime channels
• Rich client frameworks
  • reponse to increased complexity that is required
  • Complex client-side and complex server-side code
• Isomorphic Javascript development
  • Same code on both client and server
  • Only works with javascript really
  • hacks to work with other languages but not great
• Isomorphic javascript development
  • Requirements
  • Need something in-between server and browser
  • Was once done with Java based web clients
  • model, view, controller
• API-first development
• How does it work with high-latency or no-connection?
• Part of the controller and some of the model needed in the client
  • If you have python on the server you need python on the client
  • brython, skulp, pypy.js
  • <script type=”text/pyton”>
  • Note: Not phyton being compiled into javascript. Python is run in the browser
  • Need to download full python interpreter though (500k-15M)
  • Fairly fast
• Do we need a full python interpreter?
  • Maybe something just to run the bytecode
  • Batavia
  • Javascript implementation of python virtual machine
  • 10KB
  • Downside – slower than cpython on the same machine
• WASM
  • Like assembly but for the web
  • Benefits from 70y of experience with assembly languages
  • Close to Cpython speed
  • But
    • Not quite on browsers
    • No garbage collection
    • Cannot manipulate DOM
    • But both coming soon
• Example: http://bit.ly/covered-in-bees
• But “possible isn’t enough”
• pybee.org
• pybee.org/bee/join

Using “old skool” Free tools to easily publish API documentation – Alec Clew

• https://github.com/alecthegeek/doc-api-old-skool
• You API is successful if people are using it
• High Quality and easy to use
• Provide great docs (might cut down on support tickets)
• Who are you writing for?
  • Might not have english as first language
  • New to the API
  • Might have different tech expertise (different languages)
  • Different tooling
• Can be hard work
• Make better docs
  • Use diagrams
  • Show real code (complete and working)
• Keep your sentence simple
• Keep the docs current
• Treat documentation like code
  • Fix bugs
  • add features
  • refactor
  • automatic builds
  • Cross platform support
  • “Everything” is text and under version control
• Demo using pandoc
• Tools
• pandoc, plantuml, Graphviz, M4, make, base/sed/python/etc

 

Lightning Talks

• Nic – Alt attribute
  • need to be added to images
  • Don’t have alts when images as links
  • http://bit.ly/Nic-slides
• Vaibhav Sager – Travis-CI
  • Builds codes
  • Can build websites
  • Uses to build Resume
  • Build presentations
• Steve Ellis
  • Openshift Origin Demo
• Alec Clews
  • Python vs C vs PHP vs Java vs Go for small case study
  • Implemented simple xmlrpc client in 5 languages
  • Python and Go were straightforward, each had one simple trick (40-50 lines)
  • C was 100 lines. A lot harder. Conversions, etc all manual
  • PHP wasn’t too hard. easier in modern vs older PHP
• Daurn
  • Lua
  • Fengari.io – Lua in the browser
• Alistair
  • How not to docker ( don’t trust the Internet)
  • Don’t run privileged
  • Don’t expose your docker socket
  • Don’t use host network mode
  • Don’t where your code is FROM
  • Make sure your kernel on your host is secure
• Daniel
  • Put proxy in front of the docker socket
  • You can use it to limit what no-priv users with socket access to docker port can do

 

Manage all your tasks with TaskWarrior Paul ‘@pjf’ Fenwick

• Lots of task management software out there
  • Tried lots
  • Doesn’t like proprietary ones, but unable to add features he wants
  • Likes command line
• Disclaimer: “Most systems do not work for most people”
• TaskWarrior
  • Lots of features
  • Learning cliff

Intro to TaskWarrior

• Command line
• Simple level can be just a todo list
• Can add tags
  • unstructured many to many
  • Added just put putting “+whatever” on command
  • Great for searching
  • Can put all people or all types of jobs togeather
• Meta Tags
  • Automatic date related (eg due this week or today)
• Project
  • A bunch of tasks
  • Can be strung togeather
  • eg Travel project, projects for each trip inside them
• Contexts (show only some projects and tasks)
  • Work tasks
  • Tasks for just a client
  • Home stuff
• Annotation (Taking notes)
  • $ task 31 annotate “extra stuff”
  • has an auto timestamp
  • show by default, or just show a count of them
• Tasks associated with dates
  • “wait”
  • Don’t show task until a date (approx)
  • Hid a task for an amount of time
  • Scheduled tasks urgency boasted at specific date
• Until
  • delete a task after a certain date
• Relative to other tasks
  • eg book flights 30 days before a conference
  • good for scripting, create a whole bunch of related tasks for a project
• due dates
  • All sorts of things give (see above) gives tasks higher priority
  • Tasks can be manually changed
• Tools and plugins
  • Taskopen – Opens resources in annotations (eg website, editor)
• Working with others
  • Bugworrier – interfaces with github trello, gmail, jira, trac, bugzilla and lots of things
  • Lots of settings
  • Keeps all in sync
• Lots of extra stuff
  • Paul updates his shell prompt to remind him things are busy
• Also has
  • Graphical reports: burndown, calendar
  • Hooks: Eg hooks to run all sort of stuff
  • Online Sync
  • Android client
  • Web client
• Reminder it has a steep learning curve.

Love thy future self: making your systems ops-friendly Matt Palmer

• Instrumentation
  • Good article
• Instrumenting incoming requests
  • Count of the total number of requests (broken down by requestor)
  • Count of reponses (broken down by request/error)
  • How long it took (broken down by sucess/errors
  • How many right now
• Get number of in-progress requests, average time etc
• Instrumenting outgoing requests
  • For each downstream component
  • Number of request sent
  • how many reponses we’ve received (broken down by success/err)
  • How long it too to get the response (broken down by request/ error)
  • How many right now
• Gives you
  • incoming/outgoing ratio
  • error rate = problem is downstream
• Logs
  • Logs cost tends to be more than instrumentation
• Three Log priorities
  • Error
    • Need a full stack trace
    • Add info don’t replace it
    • Capture all the relivant variables
    • Structure
  • Information
    • Startup messages
    • Basic request info
    • Sampling
  • Debug
    • printf debugging at webcale
    • tag with module/method
    • unique id for each request
    • late-bind log data if possible.
    • Allow selective activation at runtime (feature flag, special url, signals)
  • Summary
    • Visbility required
    • Fault isolation

 

Look out for what’s in the security pipeline – Casey Schaufler

Old Protocols

• SeLinux
  • No much changing
• Smack
  • Network configuration improvements and catchup with how the netlable code wants things to be done.
• AppArmor
  • Labeled objects
  • Networking
  • Policy stacking

New Security Modules

• Some peopel think existing security modules don’t work well with what they are doing
• Landlock
  • eBPF extension to SECMARK
  • Kills processes when it goes outside of what it should be doing
• PTAGS
  • General purpose process tags
  • Fro application use ( app can decide what it wants based on tags, not something external to the process enforcing things )
• HardChroot
  • Limits on chroot jail
  • mount restrictions
• Safename
  • Prevents creation of unsafe files names
  • start, middle or end characters
• SimpleFlow
  • Tracks tainted data

Security Module Stacking

• Problems with incompatibility of module labeling
• People want different security policy and mechanism in containers than from the base OS
• Netfilter problems between smack and Apparmor

Container

• Containers are a little bit undefined right now. Not a kernel construct
• But while not kernel constructs, need to work with and support them

Hardening

• Printing pointers (eg in syslog)
• Usercopy

 

Code of Consult and Safety

• Badge
  • Putting prefered pronoun
  • Emoji
• Free Childcare
  • Sponsored by Github
  • Approx 10 kids
• Assistance Grants
• Attendees
  • Breakdown by gender etc
  • Roughly 25% of attendees and speakers not men
• More numbers
  • 104 Matrix chat users
  • 554 attendees
  • 2900 coffee cups
  • Network claimed to 7.5Gb/s
  • 1.6 TB over the week, 200Mb/s max
  • 30 Session Chairs
  • 12 Miniconfs
  • 491 Proposals (130 more than the others)
  • 6 Tutorials, 75 talks, 80 speakers
  • 4 Keynote speakers
  • 21 Sponsors

Linux.conf.au 2018 – Sydney

• A little bit of history repeating
• 2001, 2007, 2018
• Venue is UTS
• 5 minutes to food, train station
• https://lca2018.org
• @lca2018 on twitter
• Looking for a few extra helpers

Raffle

• In support of Outreachy
• 3 interns funded

Final Bit

• Thanks to team members

 

 

Use #lcapapers to tell Linux.conf.au what you want to see in 2018

Michael Still and Michael Davies get the Rusty Wrench award

Karaoke – Jack Skinner

• Talk with random slides

Martin Krafft

• Matrix
• End to end encrypted communication system
• No entity owns your conversations
• Bridge between walled gardens (eg IRC and Slack)
• In Very late Beta, 450K user accounts
• Run or Write your own servers or services or client

Cooked – Pete the Pirate

• How to get into Sous Vide cooking
• Create home kit
• Beaglebone Black
• Rice cooker, fish tank air pump.
• Also use to germinate seeds
• Also use this system to brew beer

Emoji Archeology 101 – Russell Keith-Magee

• 1963 Happy face created
• 🙂 invented
• later 🙁 invented
• Only those emotions imposed by the Unicode consortium can now be expressed

The NTPsec Project – Mark Atwood

• Since 2014
• For and git in 2015 from parent ntp project
• 1.0.0 release soon
• Removed 73% of lines from classic
  • Removed commandline tools
  • Got write of stuff for old OSes
  • Changed to POSIX and modern coding
  • removed experiments
• Switch to git and bugzilla etc
• Fun not painful
• Welcoming community, not angry
• ntpsec.org

National Computer Science Summer School – Katie Bell

• Running for 22 years
• Web stream, Embedded Stream
• Using BBC Microbit
• Lots of projects
• Students in grade 10-11
• Happens in January
• Also 5 week long online programming competition NCSS Competition.

Blockchain – Rusty Russell

• Blockchain
• Blockchain
• Blockchain

Go to Antarctica – Jucinter Richardson

• Went Twice
• Go by ship
• No rain
• Nice and cool
• Join the government
• Positions close
• Go while it is still there

Cool and Awesome projects you should help with – Tim Ansell

• Tomu Boards
• MicroPython on FPGAs
• Python Devicetree – needs a good library
• QEMU for LiteX / MiSoC
• NuttX for LiteX / MiSoC
• QEMU for Tomu
• Improving LiteX / MiSoc
• Sypress FX2
• Linux to LiteX / MiSoC
• DMMI2USB
• j.mp/timpro-lca2017

LoRa TAS – Paul Neumeyer

• long range (2-3km urban 10km rural)
• low power (batter ~5 years)
• Unlicensed radio spectrum 915-928 Mhz BAnd (AUS)
• LoRaWAN is an open standard
• Ideal for IoT applications (sensing, preventative maintenance, smart)

Roan Kattatow

• Different languages mix dots and commas and spaces etc to write numbers

ZeroSkip – Ron Gondwana

• Crash safe embeded database
• Not fast enough
• Zeroskip
• Append only database file
• Switch files now and then
• Repack old files togeather

PyCon Au – Richard Jones

• Python Conference Australia
• 7th in Melbourne in Aug 2016 – 650 people, 96 presentation
• In Melb on 308 of August on 2016
• 2017.pycon-au.org

Buying a Laptop built for Linux – Paul Wayper

• Bought from System76
• Designed for Linux

openQA – Aleksa Sarai

• Life is too short for manual testing
• Perl based framework that lets you emulate a user
• Runs from console, emulates keyboard and mouse
• Has screenshots
• Used by SUSE and openSUSE and fedora
• Fuzzy comparison, using regular expressions
• open.qa

South Coast Track – Bec, Clinton and Richard

• What I did in the Holidays
• 6 day walk in southern tasmania
• Lots of pretty photos

Continuously Delivering Security in the Cloud – Casey West

• This is a talk about operation excellence
• Why are system attacked? Because they exist
• Resisting Change to Mitigate Risk – It’s a trap!
• You have a choice
  • Going fast with unbounded risk
  • Going slow to mitigate risk
• Advanced Persistent Threat (ATP) – The breach that lasts for months
• Successful attacks have
  • Time
  • Leaked or misused creditials
  • Miconfigured or unpatched software
• Changing very little slowly helps all three of the above
• A moving target is harder to hit
• Cloud-native operability lets platforms move faster
  • Composable architecture (serverless, microservices)
  • Automated Processes (CD)
  • Collaborative Culture (DevOps)
  • Production Environment (Structured Platform)
• The 3 Rs
  • Rotate
    • Rotate credentials every few minutes or hours
    • Credentials will leak, Humans are weak
    • “If a human being generates a password for you then you should reject it”
    • Computers should generate it, every few hours
  • Repave
    • Repave every server and application every few minutes/hours
    • Implies you have things like LBs that can handle servers adding and leaving
    • Container lifecycle
      • Built
      • Deploy
      • Run
      • Stop
      • Note: No “change “step
    • A Server that doesn’t exist isn’t being cromprimised
    • Regularly blow away running containers
    • Repave ≠ Patch
    • uptime <= 3600
  • Repair
    • Repair vulnerable runtime environments every few minutes or hours
    • What stuff will need repair?
      • Applications
      • Runtime Environments (eg rails)
      • Servers
      • Operating Systems
    • The Future of security is build pipelines
    • Try to put in credential rotation and upsteam imports into your builds
• Embracing Change to Mitigate Risk
• Less of a Trap (in the cloud)