Wellington Open 2016

Over Easter 2016 (March 25th – 27th) I played in the Wellington Open Chess Tournament. I play in the tournament about half of the time. This year it was again being played at the CQ Hotel in Cuba street so I was able to stay at the venue and also visit my favorite Wellington cafes.

There were 43 players entered (the highest for several years) with around 9 coming down from Auckland. I was ranked 16th with a rating of 1988 and the top 4 Wellington players ( Dive, Wastney, Ker & Croad) who are all ranked in the Top 10 in NZ were playing.

See the Tournament’s page for details and downloads for the games. Photos by Lin Nah and me are also up on Flickr for Days one, two and three.

Round 1 – White vs Dominic Leman (unrated) – Result win

This game was over fairly quickly after my opponents 5th Move (Nf6) which let me win a free Bishop after ( 5.. Nf6 6.Nxc6 bxc6 7.Bxc5 ) and then they played (7.. Nxe4) to take the pawn which loses the Night since I just pin it again the King with Qe2 and pick it up a move or two later.

 

 

Round 2 – Black vs Michael Steadman ( 2338) – Result lose

Mike plays at my club and is rated well above me. However I put on a pretty poor show and made a mistake early in the Opening (which was one of my lines rather than something Mike usually plays). Error on move 5 lost me a pawn and left my position poor. I failed to improve and resigned on move 21.

Round 3 – White vs Kate Song (1701) – Result win

After 6. ..a5

I was very keen on beating Kate. While she is rated almost 200 points lower than me she improving faster and beat me in the last round of the Major Open at the NZ Champs at the start of this year.

We were the same colours as our game in January so I spent some time prepping the opening to avoid my previous mistakes.

In that game Black played 6.. a5  (see diagram) and I replied with the inaccurate Be2 and got tied into knots on the Queen side. This time I played 7. Bd3 which is a better line. However after 7. ..Nh6 8. dxc5 Bxc5 9. O-O black plays Ng4 which gives me some problems. After some back and forth Black ended up with a bit of a mid-game advantage with a developed bishop pair. and control of the open C file.

 

27. Bg5 and I offer a draw

However on move 27 after the rooks had been swapped I was able to play Bg5 which threaten to swap Black’s good Bishop or push it backwards. I offered a draw.

Luckily for me Kate picked to swap the Bishops and Queens with 27. ..Bxg5 28.Nxg5 Qd1+ 29.Qxd1 Bxd1 which left me with almost all my pawns on black squares and pretty safe from her white squared bishop. I then was able to march my King over to the Queenside while my Kingside was safe from the Bishop. After picking up a the a-pawn when the Knight and Bishops swapped I was left with a King plus A&B pawns vs King an b-pawn with around 3 tempo in reserve for pushing back the Black king.

Round 3 – Michael Nyberg vs Leighton Nicholls

Position after 71. Kxg4

Another game during round 3 went very long. This was the position after move 71 , White has just taken blacks last pawn. The game kept going till move 125! White kept try to force black to the edge of the board while black kept his king close to the centre and the Knight nearby (keeping the king away with checks and fork threats).

At move 125 Black (Nicholls) claimed a draw under the 50-move rule at which point Michael Nyberg asked “are you sure” and “are you prepared for any penalties?”. After Leighton confirmed he wanted to go ahead with the claim Michael claimed that the draw rules were changed a couple of years ago and that King+Rook vs King+Knight was allowed 75 moves. And that since the draw claim was incorrect Leighton should lose.

However a check of the Official FIDE rules online showed that there was no such special limited for the material, the rule is always 50 moves (Rule 9.3) . The penalty for incorrectly claiming a draw would also have been 2 minutes added to Michael’s time not Leighton losing the game (Rule 9.5b).

The Arbiter checked the rules and declared the game a draw while Michael grumbled about appealing it (which did not happen). Not a good way to end the game since I thought Leighton defended very well. Especially the way Michael was very aggressive while being completely in the wrong.

There have been exceptions to the 50-move draw rule in the past but it has been a flat 50 moves since at least 2001 since while some positions take longer in theory no human would actually be able to play them perfectly.

Round 4 – Black vs David Paul – Result win

Another game against somebody close to my rating but a little below. So while I should win it could be hard. I didn’t play the opening right however and ended up in a slightly poor position a couple of tempo down.

After 32 Re4 draw offered

After some maneuvering (and the odd missed move by both sizes) white offered a draw after move 32. I decided to press on with f6 and was rewarded when after 32. ..f6 33.Kf2 Kf7 White played 34.b4? which allowed me to play Nc3 and bounce my Night to b5 and then take the Bishop on d6 along with an extra pawn.

 

After 44. ..Kd6

A few moves later I’m a pawn up and with a clear path to the win although I made a mistake at the ended it wasn’t bad enough to be fatal.

 

 

 

Round 5 – White vs Russell Dive – Game lost

After getting onto 3 points after 6 rounds I was rewarded with playing the top seed. As often happens with stronger players he just seemed to make 2 threats with every move and my position slowly (well not that slowly) got worse and worse as I couldn’t counter them all (let alone make my own threats).

Eventually I resigned 3 pawns down with no play (computer assessed my position as -5.0)

Round 6 – Black vs Brian Nijman – Game Lost

Last round was once again against a higher rater play but one I had a reasonable chance against.

After 10. ..Bg6

I prepped a bit of the opening but he played something different and we anded up in a messy position with White better developed but not a huge advantage.

We both had bishops cutting though the position and Queens stuck to the side but it would be hard for me to develop my pieces. I was goign to have to work hard at getting them out into good positions

 

After 23. d5

After some swaps white ended up charging though my centre and with lots of threats. I spent a lot of time looking at this position workign out what to do.

White has the Bishop ready to take the pawn on b5 and offer check, possibly grab the Knight or pin the rook. While th Knight can also attack the rook. and the pawns can even promote.

I ended up giving up the exchange for a pawn but promptly lost a pawn when white castled and took on f7.

After 32. Ne2

I decided to push forward hoping to generate some threats and managed to when I threated to mate with two Knights or win a rook after 32. Ne2

34.Rxc5+ Kxc5 35.Be1 Rd8 36.Rc7+ followed but I played 36. ..Kd4 and blocked by Rook rather than Kb6 giving myself a tempo to move my rook to d1. This would have probably picked up another exchange and should have been enough for the win.

 

After 47. g6

And then I found another win. All I had to do was push the pawn. On move 47 I just have to put a piece on f2 to block the bishop from taking my pawn on g1. If 47. ..Nf2 48. Bxf2 Rxf2 49. g1=Q leaves me a Queen vs a rook and I can take the pawn on g6 straight away.

But instead I got Chess Blindness and just  swapped the pawn for the Bishop. I then tried to mate (or perpetual check) the King instead of trying to stop the pawns (the computer says 50. ..Nf4 is just in time). A few moves later I ran out of King-chasing moves and resigned. At which point everybody told me the move I missed 🙂

So I ended up with 3/6 or 50% in the tournament. I Losts to the players better than me and beat the lower rated ones. I’m a little disappointed with the last game and the games against Russell Dive and Mike Steadman but happy with the others. Definitely need to keep working on things though.

Share

Linux.conf.au 2016 – Friday – Session 3

Lighting talks

  • New Zealand Open Source Society
    • nzoss.org.nz
  • LCA 2015 give-aways of ARM chromebooks
    • Linux on ARM chellenge
    • github/steven-ellis
  • Call to Arms
    • x86 != Linux
    • Please consider other archetectures
  • StackPtr
    • Open Source GPS and MAP sharing
    • Android client and IOS to come
    • Create a group, Add placemaps, Share location with a group
    • Also run OpenStreetmaps tileserver
    • stackptr.com/registration  – Invite code LCA2016
  • Hat Rack
    • code is in githug, but what about everything else?
    • How to ack stuff that isn’t code?
    • bit.do/LABHR    #LABHR
    • Recommend people, especially people not like you
    • github.com/LABHR/octohatrack
  • Pycon
    • Melbourne 12-16 August
    • DjangoCon Au, Science and Data Miniconf, Python in Education plus more on 1st day
    • CPF open in mid-March
    • Financial assistence programme
    • pycon-au.org
  • Kiwi PyCon
    • 2016 in dunedin
    • Town Hall
    • 9-11 September
    • kiwi.pycon.org
  • GovHack
    • Have fun
    • Open up the government data
    • 29-31 July across Aus and NZ
  • JMAP: a better way to email
    • Lots of email standards, all aweful
    • $Company API
    • json over https
    • Single API for email/cal/contacts
    • Mobile/battery/network friendly
    • Working now at fastmail
    • Support friendly (only uses http, just one port for everything).
    • Batches commands, uses OOB notification
    • Effecient
    • Upgrade path – JMAP proxy
    • http://jmap.io  , https://proxy.jmap.io/
  • Tools
    • “Devops is just a name for a Sysadmin without any experience”
    • Lets get back to unix principals with tools
  • Machine Learning Demo
  • Filk of technical – Lied about being technical/gadget type.
  • ChaosKey
    • Randomness at 1MB/s
    • Copied from OneRNG
    • 4x4mm QFN package attached to USB key
    • Driver in Linux 4.1 (good in 4.3)
    • Just works!
    • Building up smaller batches to test
    • Hoping around $30

Closing

  • Thanks to Speakers
  • Clarification about the Speaker Gifts
  • Thanks to Sponsors
  • Raffle – $9680 raised
  • SFC donations with “lcabythebay” in the comment field will be matched (twice) in next week or two.
  • Thanks to Main Organisers from LCA President
  • Linux.conf.au 2017
    • Hobart
    • January 16th-20th 2017
    • At the Wrest Point casino convention centre. Accommodation on site and at Student accommodation
    • hobart.lca2017.org
  • Thanks to various people
  • hdmi2usb.tv is the video setup
Share

Linux.conf.au 2016 – Friday – Session 2

Free as in cheap gadgets: the ESP8266 by Angus Gratton

  • I missed the start of the talk but he was giving a history of the release and getting software support for it.
  • Arduino for ESP8266 very popular
  • 2015-2016 maturing
  • Lots of development boards
    • Sparkfun ESP8266 thing, Adafruid Hazaah, WeMOS D1
  • Common Projects
    • Lots of lighting projects, addressable LED strips
    • Wireless power monitoing projects
    • Copy of common projects. Smoke alarm project
    • ESPlant – speakers project built in Open Hardware Miniconf – solar powered gardening sensor
    • Moodlight kickstarter
  • Shortcomings
    • Not a lot of documentation compared to other micro-controllers. 1/10 that of similar products
    • Weird hardware behaviour. Unusual output
    • Default baud rate 74880 bps
    • Bad TLS – TLS v1.0, 1.1 only , RSA 512/1024 . 2048 might work
    • Other examples
  • FOSS in ESP8266
    • GCC , Lua , Arduino, Micro Python
    • axTLS , LWIP, max80211, wpa_supplicant
    • Wrapped APIs, almost no source, mostly missing attribution
    • Weird licenses on stuff
  • Does this source matter?
    • Anecdote: TLS random key same every time due to bad random function (later fixed). But still didn’t initially use the built-in random number generator.
  • Reverse Engineering
    • Wiki , Tools: foogod/xtobjdis , ScratchABit , radara2 (soon)
    • esp-open-rtos – based on the old version that was under MIT
    • mbedTLS – TLS 1.2 (and older) , RSA to 4096 and other stuff. Audited and maintained
    • Working on a testing setup for regression tests
  • For beginners
    • Start with Ardino
    • Look at dev board
  • Future
    • Hopefully other companies will see success and will bring their own products out
    • but with a more open licenses
    • ESP32 is coming, probably 1y away from being good and ready

secretd – another take on securely storing credentials by Tollef Fog Heen

  • Works for fastly
  • What is the problem?
    • Code can be secret
    • Configuration can be secret
    • Credentials are secret
  • Secrets start in the following and move to the next..
    • directly code
    • then a configuration file
    • then an pre-encrypted store
    • then an online store
  • Problems with stores
    • Complex or insecure
    • Manual work to re-encrypt
    • Updating is hard
    • Not support for dev/prod split
  • Requirements for a fix
    • Dynamic environment support
    • Central storage
    • Policy based access controls, live
    • APIs for updating
  • Use Case
    • Hardware (re)bootstrapping
    • Hands-of/live handling
    • PCI: auditing
    • Machine might have no persistent storage
  • Options
    • pwstore – pre-encrypted
    • chef-vault – pre-encrypted
    • Hashicorp Vault – distributed, complex, TTL on secrets
    • etcd – x509
  • Secretd
    • go
    • SQL
    • ssh
    • tree structure, keys are just strings
    • positive ACLs
    • PostgressSQL backend
    • Apache Licensed
  • Client -> json over ssh -> secret-shell -> unix socket ->  secretd -> postgressSQL
  • Missing
    • Encrypting secrets on disk
    • Admin tools/other UIs
    • Auditing
    • Tool integration
    • Enrolment key support
  • Demo
  • Questions:
    • Why not sqlite? – Cause  I wanted at database. Postgres more directly supported the data structure I wanted, also type support
    • Why do just use built-in postgress security stuff? – Features didn’t exist a year ago, also requires all users must exist as DB users.

 

Share

Linux.conf.au 2016 – Friday – Session 1

Keynote – Genevieve Bell

  • Building the Future
  • Lots of rolls as an Anthropologist at Intel over last 15 years or so
  • Vision of future from 1957 shows what the problems are in 1957 that the future would solve
  • Visions of the future seem very clean and linear, in reality it is messy and myriad.
  • ATM machine told her “Happy Birthday”
  • Imagining “Have you tried turning it off and on again?” at smart city scale is kind of terrifying.
  • Connectivity
    • Many people function well when they are offline, some people used to holiday in places with no cell reception
    • Social structures like Sabbath to give people time offline, but devices want us to be always online
    • Don’t want to always have seamless between devices, context matters. Want work/home/etc split
  • IOT
    • Technology lays bare domestic habits that were previously hidden
    • Who is else knows what you household habits are -> Gossip
  • Big Data
    • Messy , incomplete, inaccurate
    • Average human tells 6-200 lies per day
    • 100% of Americans lie in online profiles
      • Men lie about height, Women lie about weight
    • More data does not equal more truth. More data just means more data
  • Algorithms
    • My optimise for the wrong things (from the user’s point of view)
  • Security and Privacy
    • Conversation entwined with conversation about National Security
    • Concepts different from around the world
    • What is it like to release data under one circumstance and then to realise you have released it under several others
  • Memory
    • Cost of memory down to zero, we should just store everything
    • What are the usage models
    • What if everything you ever did and said was just there, what if you can never get away from it. There are mental illnesses based on this problem
  • Innovation
    • What is changing? to whose advantage and disadvantage? what does this mean to related areas?
    • Our solutions need to be human
    • We are the architects of our future
  • Question
    • Explain engineers to the world? – Treated first year at Intel like it was Anthropology fieldwork. Disconnect between what people imagine technologists think/do and what they really do. Need to explain what we do better

Helicopters and rocket-planes by Andrew Tridgell

  • The wonderful and crazy world of Open Autopilots
  • Outback Challenge
    • 90km/h for 45 minutes
    • Search pattern for a lost bushwalker with UAV
    • Drop them a rescue package
    • 2016 is much harder VTOL, get blood sample. Most do takeoff and landing remotely (30km from team).
    • “Not allowed to get blood sample using a propeller”
  • VTOL solutions – Helicopters and Quadplanes – tried both solutions
    • Communication 15km away, 2nd aircraft as a relay
    • Pure electric doesn’t have range. 100km/h for 1h
  • Helicopters
    • “Flying vibration generators with rotating swords at the top”
    • Hard to scale up which is needed in this case. 15cc motor, 2m blades, 12-14kg loaded
    • Petrol engines efficient VTOL and high energy density
    • Very precise control, good in high wind (competition can have ground wind up to 25 knots)
    • Normal stable flight vibrates at 6G , show example where in a couple of seconds flight goes bad and starts vibrating at 30+ G in a few seconds due to control problem (when pitch controller was adjusted and then started feedback loop)
  • Quadplanes
    • Normal Plane with wings but 4 virtually pointing propellers added
    • Long range, less vibration
    • initially two autopilots plus one more co-ordinating
    • electric for takeoff, petrol engine for for long range forward flight.
    • Hard to scale
    • crashed
  • Quadplane v2
    • Single auto-pilot
    • avoid turning off quad motors before enough speed from forward motor
    • Pure electric for all motors
    • Forward flight with wings much more efficient.
    • Options with scale-up to have forward motor as petrol
  • Rockets
    • Lohan rocket plane – Offshoot of The Register website
    • Mission hasn’t happened yet
    • Balloon takes plane to 20km, drops rocket and goes to Mach 2 in 8 seconds. Rocket glides back to each under autopilot and lands at SpacePort USA
    • 3d printed rocket. Needs to wiggle controls during ascent to stop them freezing up.
    • This will be it’s first flight so has autotune mode to hopefully learn how to fly for the first time on the way down
  • Hardware running Ardupilot
    • Bebop drone and 3DR solo runs open autopilot software
    • BBBmini fully open source kit
    • Qualcom flight more locked down
    • PXFMini for smaller ones
  • Sites
    • ardupilot.com
    • dronecode.org
    • canberrauav.org.au

The world of 100G networking by Christopher Lameter

  • Why not?
    • Capacity needed
    • Machines are pushing 100G to memory
    • Everything reqires more Bandwidth
  • Technologies
    • Was 10 * 10G standards CFP Cxx
    • New standard is 4 * 28Gs QSFP28 . compact and designed to replace 10G and 40G networking
    • Inifiband (EDR)
      • Most mature to date, switches and NICs available
    • Ethernet
      • Hopefully available in 2016
      • NICS under dev, can reuse EDR adapter
    • OmniPath
      • Redesigned to try replace infiband
    • Comparison connectors
      • QSFP28 smaller
    • QSFP idea with spliter into 4 * 25G links for some places
      • Standard complete in 2016 , 50G out there but standard doesn’t exist yet.
      • QSFP is 4 cables
  • 100G switches
    • 100G x 32 or 50G x64 or 25G x 128
    • Models being released this year, hopefully
    • Keeping up
  • 100G is just 0.01ns per bit , 150ns for 1500MTU packet, 100M packets/second, 50 packets per 10 us
  • Hardware distributed packets between cores. will need 60 cores to handle 100G in CPU, need to offload
  • Having multiple servers (say 4) sharing a Nic using PCIe!
  • How do you interface with these?
    • Socket API
  • Looking Ahead
    • 100G is going to be a major link speed in data centers soon
    • Software needs to mature especially the OS stack to handle bottlenecks

 

Share

Linux.conf.au 2016 – Thursday – Session 3

Law and technology: impedance mismatch by Michael Cordover

  • IP lawyer
  • Known as the EasyCount guy
  • Lawyers and Politicians don’t get it
    • Governing behaviour that is not well understood (especially by lawyers) is hard
    • Some laws are passed under assumption that they won’t always be enforced (eg Jaywalking, Speeding limits). Pervasive monitoring may make this assumption obsolete
  • Technology people don’t get the law either
    • Good reasons for complexity of the law
    • Technology isn’t neutral
  • Legal detailed programmatic specifically
    • Construction
    • Food
    • Civil aviation
    • Broadcasting
  • Anonymous Data
    • Personal information – info from which id can be worked out
  • 100s of examples where law is vague and doesn’t well map to technology
    • Encryption
    • Unauthorised access
    • Copyright
    • Evidence
  • The obvious, easy solution:
    • Everybody must know about technology
    • NEVER going to happen
  • Just make a lot of contracts
    • Copyright – works fairly well, eg copyleft
    • TOS – works to restrict liability of service providers so services can actually be safely provided
    • EULAs
    • P3P – Privacy protection protocol
    • But doesn’t work well in multiple jurisdictions, small ppl against big companies, etc
  • Laws that are fit for purpose
    • An ISP is not an IRC server
    • VOIP isn’t PSTN
    • Focus on the outcome, sometimes
  • A somewhat radical shift in legal approach
    • It turns out the Internet is (sometimes) different
    • United States vs Causby – 1946 case that said people don’t work air above their property to infinity. Airplanes could fly above it.
  • You can help
    • Don’t ignore they law
    • Don’t be too technical
    • Don’t expect a technical solution
    • Think about policy solutions
    • Talk to everybody

 

Share

Linux.conf.au 2016 – Thursday – Session 2

Machine Ethics and Emerging Technologies by Paul ‘@pjf’ Fenwick

  • Arrived late
  • Autonomous cars
    • Little private ownership of autonomous vehicles
    • 250k driving Taxis
    • 3.5 million truck drivers + plus more that depend on them
    • Most of the cost is the end-to-end on a highway. Humans could do the hard last-mile
  • Industrial revolution
    • Lots of people put out of jobs
    • Capital offence to harm machines
    • We still have tailors
    • But some jobs have been eliminated – eg Water bearer in cities
  • Replacing humans with small amounts of code
  • White collar jobs now being replaced
  • If more and more people are getting put out of jobs and we live in a society that expects people to have jobs what can we do?
    • Education to retrain
  • We *are* working less 1870=70h work week , 1988=40h work week
  • Leisure has much increased 44k hours -> 122k hours (shorter week + live longer)
  • What do people do with more leisure?
    • Pictures of cats!
    • Increase in innovation
  • How would the future work if machines are doing the vast majority of jobs?
    • Technological dividend
    • Basic income
  • Drones
    • “Drones have really taken off in the last few years”
    • Delivery drones
    • Disaster relief
    • Military drones – If autonomous then radio silent
    • Solar powered drones with multi-day/week duration
      • Good for environmental monitoring
      • Have anonymous warfare, somebody launches it, and it kills some people, but you don’t know who to blame
  • Machine Intelligence
    • Watson getting better at cancer diagnosis and treatments plan than many doctors
  • Questions:
    • Please focus on the upsides of lethal autonomous robots – Okay with robots, less happy with taking the machine out of the loop.
    • Why work week at 40 hours – Conjecture by Paul – Culture says humans must work and work gives you value and part time work is seen as much less important

Open Source Tools for Distributed Systems Administration by Elizabeth K. Joseph

  • Tools that enable distributed teams to work
  • Works day to day on Openstack
  • How most projects do infrastructure
    • Team or company manges do it or they just use github
    • Requests via mailing list or bug/ticketing system
    • Priority determined by the core team
  • Is there a better way – How Openstack is different – Openstack infrastructure team
    • Host own git, wiki, ircbots, mailing lists, web servers and run them themselves
    • All configs are open source and tracked in git
    • Anyone can submit changes to our project.
    • We all work remotely
  • Openstack CI system
    • 800+ projects
    • All projects must work togeather
    • changes can’t break master branch
    • code must be clean
    • testing must be completely automated
  • Tools for CI (* is they own tools)
    • Launchpad for Auth
    • git
    • gerrit
    • zuul* – gatekeep
    • Geaman
    • jenkins
    • nodepool*
  • Automated Test for infrastructure
    • flake8
    • puppet parser validate, puppet lint, puppet application tests
    • XML checkers
    • Alphabetized files ( cause people forget the alphabet)
    • Permissions on IRC channels
  • Peer review means
    • Multiple eyes on changes prior to merging
    • Good infrastructure for developing new solutions
    • No special process to go through commit access
    • Trains us to be collaborative by default
    • Since anyone can contribute, anyone can devote resources to it
  • Gerrit in-line comments
  • Automated deployments. Either puppet directly or via vcsrepo
  • Can you really manage infrastructure via git commits
    • Cacti – cacti.openstack.org
      • Cacti are public so anybody can check them
      • No active monitoring
    • Puppetboard
      • so you can watch changes happening
      • Had to change a little so secret stuff not public
    • Documentation
      • Fairly good since distributed team
    • Not quiet everything
      • Need to look at logs
      • Some stuff is manual
      • Passwords need to be privately managed (but in private git repo)
      • Some complicated migrations are manual
  • Maintenance collaboration on Etherpad
  • Collaboration
    • Via IRC various channels
    • main + incident + sprint + weekly meetings
    • channel/meeting logs
    • pastebin
    • In-person collaboration at Openstack design summit every 6 months
  • And then there are timezones
    • The first/root member in a particular region struggles to feel cohesion with the team
    • Increased reluctance to land changes into production
    • makes slower on-boarding
    • Only solved by increasing coverage in that time-zone so they’re not alone
  • Questions
    • Reason why no audio/video? – Not recorded or even hard to access if they are
    • How to dev “write documentation” culture – Make that person responsible to write docs so others can still handle it. Helps if it it really easy to do. Wikis never seem to work in practice, goes though same process as everything else (common workflow)
    • Task visibility – was bugzilla + launchpad – trying storyboard but not working well.
Share

Linux.conf.au 2016 – Thursday – Session 1

Jono Bacon Keynote

  • Community 1.0 (ca 1998)
    • Observational – Now book on how to do it
    • Organic – people just created them
    • Technical Enviroment – Had to know C (or LaTex)
  • Community 2.0 (ca 2004, 2005)
    • Wikipedia, Redhat, Openstack, Github
    • Renaissance – Stuff got written down on how to do it
    • Self Organising groups – Gnome, Kde, Apache foundation – push creation of tech and community
    • Diversity – including of skills , non-technical people had a seat at the table and a role.
    • Company Engagement – Starting hiring community managers, sometimes didn’t work very well
  • Community 3.0 ?
  • Why?
    • “Thoughtful and productive communities make us as species better
  • Access and power is growing exponentally
  • But stuff around is changing
    • Cellphones are access method for most
    • Cloud computering
    • CD-printers, drones, cloud, crowdfunding, Ardinino
    • Lots for channels to get things to everybody and everybody can participate
  • “We need to empower diversity of both people and talent”
  • Human brain has not had a upgrade in a long time
  • Bold and Audacious Goals
    • Openness is at the heart of all of these
    • Open source in the middle of many
  • Eg Drone
    • Runs linux
    • Open API
  • “Open Source is where Society innovates”
  • “Need to make great community leadership accessible to everybody”
  • “Predictable collaboration – an aspirational goal where we won’t *need* community managers”
  • Not just about technology
    • We are all human.
  • Tangible value vs Intangible value
    • Tangible can be measured and driven to fix the numbers
    • Intangible – trust, dignety
  • System 1 thinking vs System 2 thinking
    • Instant vs considered
  • SCARF Model of thinking
    • Status – clarity of relative importance, need people to be able to flow between them
    • Certainty – Security and predictability
    • Autonomy – People really want choices
    • R – I got distracted by twitter, I’m sure it was important
    • Fair – fairness
  • Two Golden Rules
    • We accomplish our goals indirectly
    • We influence behaviour with small actions
  • We need to concentrate to building an experience for people to who join the community
  • Community Workflow
    • Communication – formal, inclormal? Coc? Tech to use?
    • Release sceduled, support?
    • How to participate, tech, hackthons
    • Government structure
  • Paths for different people
    • New developers
    • Core Developers
    • Consumers
    • Downstream Cosutomers
    • Organizations
  • Opportunity vs Belonging
  • Questions
    • Increasing Signal to Noise ratio – Trolls are easy[er], harder for people who are just no deft in communication. Mentorship can help
    • Destructive communities (like 4chan) , how can technology be used to work against these – Leaders need to set examples. Make clear abusive behavour towards others. Won’t be able to build tools that will completely remove bad behaviour. Had to tell destructive vs direct automatically but they can augmented.
    • What about Linus type people? – View is that even though it works for him and it is okay with people he knows. Viewed inwards by others it sets a bad example.

Using Persistent Memory for Fun and Profit by Matthew Wilcox

  • What is it?
    • Retains data without power
    • NV-DIMMs available – often copy DRAM to flash when power lost
    • Intel 3D X-point shipping in 2017. will become more a standard feature
  • How do we could use it
    • Total System persistence
      • But the CPU cache is not backed up, so pending writes vanish
    • Application level persistence
      • Boot new kernel be keep the running apps
      • CPU cache still
    • Completely redesigned operating system to use
      • But we want to use in 2017
    • A special purpose filesystem
      • Implementation not that great
    • A very fast block device
      • Usaged as very fast cache for apps really need it. Not really general purpose
    • Small modifications to existing file systems
      • On top of ext2 (xip)
      • DAX
  • How do we actually use it
    • New CPU instructions ( mostly to make sure encourage that things are flushed from the CPU cache)
    • Special purpose programming language shouldn’t be needed for interpreted languages. But for compiled code libraries might be needed
  • NVML library
  • Stuff built on NVML library so far.
    • Red-Black tree, B-tree, other data-structures
    • Key-value store
    • Fuse file system
    • Example MySQL storage engine
  • Resources
  • Questions
    • In 2017 will we have mix of persistent and non-persistent RAM? – Yes . New Layer in the storage hierarchy
    • Performance of 3d will be slower a little slow than DRAM but within ballpark, various trade-offs with other characteristics
    • Probably won’t have native crypto

Dropbox Database Infrastructure by Tammy Butow

  • Dropbox for last 4 months, previously Digital Ocean, prev National Australia Bank
  • Using MySQL for last 10 years. Now doing it FT.
  • 400 Million customers
  • Petabytes of data across thousands of servers
  • In 2012 Dropbox just had 1 DBA, but was huge then.
  • In 2016 it has grown to 9 people
  • 6000 DB servers -> DB Proxy -> DB as a service (edgestore) -> memcache -> Web Servers (nginx)
  • Talk – Go at Dropbox, Zviad Metreveli on Youtube
  • Applications talk directly to edgestore not directly to database
  • vitess is mysql proxy (by youtube) similar to what dropbox wrote. Might move to that
  • Details
    • Percona 5.6
    • Constantly upgrading (4 times in last year)
    • DBmanager – service we manage mysql via
  • Each Cluster is proiamry + 2 replicas
  • Use xtrabackup ( to hdfs locally and s3)
  • Tools
    • Tasks grow and take time
    • DBmanager
      • Automating DB operations
      • Web interface with standard operations and status of servers
      • Cloning Screen
      • Promotion Screen
      • Create and restore backups
      • WebUI gives you feedback and you can see how things are going. Don’t need magic command lines. Good for other teams to see stuff and do stuff (options right in front of them).
      • Benchmarking
      • Database job scheduling and prioritization. Promotion will take priority over anything else.
      • Common logging, centralized server and nice gui that everyone can see
    • HERMES
      • Availbale on dropbox github
      • Visable all quests and actions that need to be done by the team
    • Monitoring
      • Grafana
  • Performance
    • Improving backup and restore speed.
      • LZOP
      • xtrabackup
  • Auto-remediation (naoru) – up on github at some point
  • Inventory Management
    • Machine Database (MDB)
    • Has tags for things like kernel versions
  • Diognostics
    • Automated periodic tcpdump
    • Tools to kill long running transactions
    • List current queries running
    • atop
  • The Future
    • Reliabilty, performance and cost improvements
    • Config management
    • Love the “Go Programming Language” by Kernighan
    • List of Papers they love
  • Questions
    • Using percona not mariadb. They also shard not cluster DBs
    • Big Culture change from Back to Dropbox – At Bank tried to decom old systems, reduce risk. At Dropbox everyone is very Brave and pushing boundarys
    • machine database automatically built largely
    • Predictive Analysis on hardware – Do some , lots of dashboards for hardware team, lifecycle management of hardware. Don’t hug servers. Hug the hardware class instead.
    • Rollbacks are okay and should be easy. Always be able to rollback a change to get to back to a good stack.
Share

Linux.conf.au 2016 – Wednesday – Session 3

The future belongs to unikernels. Linux will soon no longer be used in Internet facing production systems. by Andrew Stuart

  • Stripped down OS running a single application
  • Startup time only a few milli-seconds
  • Many of the current ones are language specific
  • The Unikernel Zoo
    • MirageOS – Must be written in OCaml
    • Rump –  Able to run general purpose software, run compiled posix applications, largely unmodified. Can have threading but not forking
    • HalVM – Must be coded in Haskell
    • Ling – Erlang
    • Drawbridge – Microsoft research project
    • OSv – More general purpose
    • “Something about Unikernels seems to attract the fans of the ‘less common’ languages”
    • plus a bunch more..
  • Unikernels and security
  • Bunch of people point out problems and alternative solutions the unikernel are trying to solve.

 

An introduction to monitoring and alerting with timeseries at scale, with Prometheus by Jamie Wilkinson

  • prometheus.io
  • SRE ultimately responsible for the reliability of google.com , less that 50% of time on ops
  • History of monitoring, Nagios doesn’t scale, hard to configure
  • Black-box monitoring for alerts
  • White-box monitoring for charts
  • Borgmon at Google, same tool used my many teams at google
  • Borgmon not Open Source, but instead we’ll look at Prometheus
  • Several alternatives alternatives
  • Borgman
  • Alert design
    • SLI – a measurment
    • SLO – a goal
    • SLA – economic incentives
  • Philosopy
    • Every time you get paged you should react with sense of urgency
    • Those that are not important shouldn’t be paged on, perhaps just to console
  • Instrumentation
    • Client exports a interface usually http , prometheus polls /metrics on this server gets plain page with numbers
    • Metrics are numbers not strings
    • Don’t need timestamps into data
  • Tell prometheus where the targets are in the “scrape_configs”
    • All sorts of ways to find targets (DNS, etc)
  • Variables all have labels, name, things like localtions
  • Rule evaluation
    • recording rules
    • tasks run built in fuctions like sum up data by label (eg all machines with the same region label), find rate of change etc
  • Pretty graphs shown in demo
  • https://github.com/jaqx0r/blts
  • Questions
    • Prometheus exporting daemon/proxy
    • Language ability to support things like flapping detection/ignore
    • Grafana support for Prometheus exists
Share

Linux.conf.au 2016 – Wednesday – Session 2

Welcoming Everyone: Five Years of Inclusion and Outreach Programmes at PyCon Australia by Christopher Neugebauer

  • How to bring more people to community run events
  • Talk is not about diversity in tech
  • Talk is about “Outreach and Inclusion in Events”
  • Outreach = getting them in , Inclusion = making them feel welcome
  • About funding programmes for events
  • FOSS happens over the Internet , face-to-face is less common than in other areas/communities
  • Events are where you can see the community
  • BUT: Going to a conference costs money – travel, rego, parking, leave from job
  • Events have equality of access problem
  • Inequity of access is  a problem with diversity
  • Solution: Run outreach programmes
  • Money can reduce the barriers, just spending money can help solve the problem
  • Pycon Australia has had outreach for last 5 years
  • FOSS vs other outreach programmes
    • Events have easy goals, define ppl/numbers to target, exact things to spend on, time period defined
    • Similar every year, similar result each year
    • Long-term results are ill-defined
    • Engagement is hard to track
  • Pycon Australia
    • Fairly independent of Python software foundation
    • Biggest Pycon within 9 hours of flying
    • Pycon US – 2500 attendees, $200k on financial attendance
    • Pycon Aus 2015 – 450 attendees , 5-8% of budget on funding
  • 2011
    • Harassment and Codes of Conduct were a big thing
    • Gender diversity policy, code of conduct, 20% speaks were women, First Gender diversity grants
    • 2 Grants, – 1 ticket and 1 Ticket + $500 funded out of general conf budget
    • 7 strong applicants at time when numbers were looking low (later picked up)
    • Sponsor found and funded all 7 applicants
  • 2012
    • 1st of 2 years running conf in Hobart
    • Moving from Sydney is hard. Australia big and people have to fly between cities (especially to Hobart)
    • Hobart long way away for many people and small number of locals
    • Sponsor increased funding to $700, funded 10 people for $500 + ticket
    • Previous grant recipient from 2011 was speaking in 2012
  • 2013
    • Finding more speakers from more places
    • Outreach and Speaker support run out of the same budget, cap removed on grants so International travel possible.
    • Anyone could apply removed purely on gender limit. So other people who needed funding could apply. Eg Students, teachers, geographic minorities
    • $12,500 allocated
    • As more signups and more money came in more could go to the assistance budget
    • If remove gender targeting then then what happens to diversity
    • Got groups like GeekGirlDinners to target people that needed grants rather than directly chasing people to apply.
    • Over half aid budget going to women
    • Teachers good force multiplies
  • 2014
    • Lost previous diversity Sponsor
    • Previously $5k from Sponsor + $7k from general fund.
    • Pycon US – Everybody pays to attend ( See Essay by Jesse Noller – Everybody Pays )
    • Most speakers have FOSS-friendly employers or can claim money
    • Argument: Some confs make everybody pay no matter their ability.
    • Told speakers that by default they would be charged, but by charge they weive it by just asking. Also said where the money was going and prioritised speakers to assistance. Also all organisers paid
    • Extra money from about $7000
    • Simplified structure of grants, less paperwork, just gave people a budget. Worked well since many people went with good deals.
    • Caters better for diverse needs
    • Also had Education Miniconf, covered under teacher traning budget. Offered to underwrite costs of substitute teachers for schools since that is not covered by normal school professional-dev budget
  •  Results
    • Every time at least one funding recipient has spoken at next conference
    • Many fundees come back when get professional jobs
    • Evangelize to the friends
  • Discovery
    • expanding fund gets people you might not expect
    • Diverse people have diverse needs
    • Avoid making people do paperwork, just give them money
    • Sponsors can make boot-strapping starting a programme easier
    • Don’t expect 100% success
    • Budget liberally, disburse conservatively
    • Watch out for immigration scams
    • Decline requests compassionately
  • Questions
    • Weekend hard for Childcare – Not heavily targeted
    • Targeting Speakers for funding rather than giving all of them means it gets to go a lot further. Better Bang for buck

Sentrifarm – open hardware telemetry system for Australian farming conditions by Andrew McDonnell

  • Great time to be a maker, everybody is able to make something
  • Neighbour had problem with having to measure grass fire danger in each paddock before going out with machinery during summer
  • Needs Wind Speed, temperature, humidity
  • Sentrifarm
    • Low power, solar
    • distributed
    • Works in area with slow internet, sim card expense adds up however
    • Easy to use for farmer, access via their farm.
    • Data should not be owned by cloud provider
  • Hackerday Prize
    • Build “something that matters”
    • Prizes just for participating
    • Document progress, produce a video
  • Our Goals
    • Cheap and Cheerful
    • Aussie “bush mechanic” ehtos
    • Enjoy the adventure
  • Used stuff from 24+ other opensource projects
  • Prototyping
    • Tried out various micro-controllers an other equipment
    • Most you could only buy for a few dollars
    • Tools – Bus Pirate
  • Radio links
    • ISM-band radio module “Lora” technology
    • SPI interface, well documented SX1276
    • $20 for the module
    • Propriety radio protocol, long rang low power, but open interface on top of it
  • Eagle used (alt is KiCAD) to design circuit
    • Build own shields to plug sensors and various controllers into
  • playformio.org – run one command, creates a arduino project and builds with one command for multiple micro-controllers
  • MQTT-SN – communications protocol for low-bw links.
  • Breakdown of his stack, see his slides for details
  • Backend Software
    • Ubuntu
    • Docker
    • Carbon + Whisper + Graphite, Grafana
    • “Great time to be a hacker, using who knows how many lines of code and only had to write 7 to get it to work together”
  • Grafana hard to setup but found a nice docker container
  • Data kept separately from the container
  • Goal to get power down
  • Used 3D-printer to create some parts from mounting bits.
    • OpenSCAD – Language to design the parts
  • Range of Lori of 5km un-evalated , 9km up a tower with sinple home-built antenna
  • Won a top-100 prize at Hackaday of a t-shirt
  • You can do it
  • Questions
    • Ask home survives weather? – Not a lot of experience yet, some options
    • Home likely others to use? – Maybe but main gaol was to building it
Share

Linux.conf.au 2016 – Wednesday – Session 1

Going Faster: Continuous Delivery for Firefox by Laura Thomson

  • Works for Cloud services web operations team
  • Web Dev and Contious delivery lover
  • “Continuous delivery is for webapps” – Maybe not just Webapps? Maybe Firefox too
  • But Firefox is complicated
  • Process very complicated – “down from 5 source control systems to 3”
  • But plenty of web apps are very complicated (eg Netflix)
  • How do we continuous deliver Firefox
  • How it works currently
    • Release every 6 weeks
    • 4 channels – Nightly -> Aurora -> Beta -> release
    • Mercurial Repo for each channel
  • Release Models
    • Critical Mass – When enough is done and it is stable
    • Single Hard deadline – eg for games being mass released
    • Train Model – fixed intervals
    • Continuous Delivery
  • Deployment Maturity Model
  • Updates
    • New Build -> Generate  a diff -> FF calls back -> downloads and updates
    • Hotfixs
    • Addons automatically updated
  • Currently pipeline around 12 hours long, lots of tests and gatekeeping
  • “Go Faster”
    • System add-ons
    • Test Pilot
    • Data Separate from code
    • Downloadable content
    • Features delivered as web apps
  • System addons
    • Part of core FF, modularized into an add-on
    • Build/test against existing FF build, a lot smaller test
    • Updated up to daily(for now) on any release channel
    • signed and trusted
    • Restartless updates
      • install or update without a browser restart
      • Restarts suck
      • Restartsless coming soon for system add-ons
    • Good for rapid iteration, particularly on the front-end
    • Wrappers for services
    • Replacing hotfixes
  • Problems with add-ons
    • Localalisation
    • Optimizing UX : Better browser faster vs update fatigue
    • Upfront telemetry requirements
    • Dependency mngt on firefox
    • Dependency management between system add-ons (coming soon)
  • Add-ons in flights
    • Firefox hello is already an add-on
    • Currently in beta in 45
    • First beta updates before 46
  • Test Pilot
    • Release channel users opt in to new features
    • Release channel users different from pre-release ones
    • Developed as regular ad-ons (not system add ons)
    • Can graduate to system add-ons by flipping a bit
  • Data should be seperate from code
    • Sec policy
    • blocklists
    • tracking protection list
    • dictionaries
    • fonts
  • Many times Data update == release , this is broken
  • Also some have their own updaters
  • Kinto
    • Lightweight JSON storage with sync, sharing, signing
    • Natice JSON over http
    • niceties of couchDB backed by postgressDB
  • How Kinto Works
    • pings for updates
    • balrog supplies link to kinto
    • signed data downloaded, checked, applied
  • Kinto good for
    • Add-ons block list
  • Downloadable Content
    • Some parts of the browser may not need frequently
    • May not be needed on startup
    • eg languages packs, fonts for Firefox on Android
  • Features delivered remotely
    • Browser features delivered as web apps
    • Pull in content from the server
    • in a early stage
  • Futures
    • Easy for projects to impliment
    • Better “knobs and dials” (canaries A?B, data viz)
    • Pushed based updates
    • Simpler localisation
  • Questions
    • They support rollbacks
    • Worst case: Firefox has a startup crash
    • Not sure sure ice weasel would fit in.
    • How will effect ESR channel? – Won’t change, they will stay security-only
    • Bad Addons – Hate ones that reporting user-data, crashers (eg skype toolbar at one point), Highjack your browser and change settings
    • There is much collaboration between [open source] browsers
    • You are avoiding the release cycle, planning to speed it up – Lots of tests that can’t get rid of all, working on it but not a simple thing to solve.
Share