In the old days
– Shipping used to be hard and expensive, eg on physical media
– High cost of release
– but everybody else was the same.
– Lots of QA and red tape, no second chances
Then we got the Internet
– Speed became everything
– You just shipped enough
But Hardware still was a limiting factor
– Virtual machines
– IaaS
– Containers
This led to complacency
– Still had a physical server under it all
Birth of devops
– Software got faster but still had to have hardware under their somewhere
– Disparity between operations cadence and devs cadence
– things got better
– But we didn’t free ourselves from hardware
– Now everything is much more complex
Developers are now divorced from the platform
– Everything is abstracted
– It is leaky buckets all the way down
Solutions
– Education of developers as to what happens below the hood
– Stop reinventing the where
– Harmony is much more productive
– Lots of tools means that you don’t have enough expertise on each
– Reduce fiefdoms
– Push responsibility but not ownership (you own it but the devs makes some of the changes)
– Live with the code
– Pit of success, easy ways to fail that don’t break stuff (eg test environments, by default it will do the right thing)
– Be Happy. Everybody needs to be a bit devops and know a bit of everything.
Martina Iglesias – Automatic Discovery of Service metadata for systems at scale
Backend developer at Spotify
Spotify Scale
– 100m active users
– 800+ tech employees
– 120 teams
– Microservices architecture
Walk though Sample artist’s page
– each component ( playlist, play count, discgraphy) is a seperate service
– Aggregated to send result back to client
Hard to co-ordinate between services as scale grows
– 1000+ services
– Each need to use each others APIs
– Dev teams all around the world
Previous Solution
– Teams had docs in different places
– Some in Wiki, Readme, markdown, all different
Current Solution – System Z
– Centralise in one place, as automated as possible
– Internal application
– Web app, catalog of all systems and its parts
– Well integrated with Apollo service
Web Page for each service
– Various tabs
– Configuration (showing versions of build and uptimes)
– API – list of all endpoints for service, scheme, errors codes, etc (automatically populated)
– System tab – Overview on how service is connected to other services, dependencies (generated automatically)
Registration
– System Z gets information from Apollo and prod servers about each service that has been registered
Apollo
– Java libs for writing microservices
– Open source
Apollo-meta
– Metadata module
– Exposes endpoint with metadata for each service
– Exposes
– instance info – versions, uptime
– configuration – currently loaded config of the service
– endpoints –
– call information – monitors service and learns and returns what incoming and outgoing services the service actually does and to/from what other services.
– Automatically builds dependencies
Situation Now
– Quicker access to relevant information
– Automated boring stuff
– All in one place
Learnings
– Think about growth and scaling at the start of the project
Q: How to handle breaking APIs
A: We create new version of API endpoint and encourage people to move over.
Bridget Cowie – The story of a performance outage, and how we could have prevented it
– Works for Datacom
– Consultant in Application performance management team
Story from Start of 2015
– Friday night phone calls from your boss are never good.
– Dropped in application monitoring tools (Dynatrace) on Friday night, watch over weekend
– Prev team pretty sure problem is a memory leak but had not been able to find it (for two weeks)
– If somebody tells you they know what is wrong but can’t find it, give details or fix it then be suspicious
Book: Java Enterprise performance
– Monday prod load goes up and app starts crashing
– Told ops team but since crash wasn’t visable yet, was not believed. waited
Tech Stack
– Java App, Jboss on Linux
– Multiple JVMs
– Oracle DBs, Mulesoft ESB, ActiveMQ, HornetQ
Ah Ha moment
– Had a look at import process
– 2.3 million DB queries per half hour
– With max of 260 users, seems way more than what is needed
– Happens even when nobody is logged in
Tip: Typically 80% of all issues can be detected in dev or test if you look for them.
Where did this code come from?
– Process to import a csv into the database
– 1 call mule -> 12 calls to AMQ -> 12 calls to App -> 102 db queries
– Passes all the tests… But
– Still shows huge growth in queries as we go through layers
– DB queries grow bigger with each run
Tip: Know how your code behaves and track how this behavour changes with each code change (or even with no code change)
Q: Why Dynatrace?
A: Quick to deploy, useful info back in only a couple of hours
Ken Mugrage – What we’re learning from burnout and how DevOps culture can help
Originally in the Marines, environment where burnout not tolerated
Works for Thoughtworks – not a mental health professional
Devops could make this worse
Some clichéd places say: “Teach the devs puppet and fire all the Ops people”
Why should we address burnout?
– Google found psychological safety was the number 1 indicator of an effective team
– Not just a negative, people do better job when feeling good.
What is burnout
– The Truth about burnout – Maslach and Leiter
– The Dimensions of Burnout
– Exhaustion
– Cynicism
– Mismatch between work and the person
– Work overload
– Lack of control
– Insufficient reward
– Breakdown of communication
Work overload
– Various prioritisation methods
– More load sharing
– Less deploy marathons
– Some orgs see devops as a cost saving
– There is no such thing as a full stack engineer
– team has skills, not a person
Lack of Control
– Team is ultimately for the decissions
– Use the right technolgy and tools for the team
– This doesnt mean a “Devops team” contolling what others do
Insufficient Reward
– Actually not a great motivator
Breakdown in communication
– Walls between teams are bad
– Everybody involved with product should be on the same team
– 2 pizza team
– Pairs with different skill sets are common
– Swarming can be done when required ( one on keyboard, everybody else watching and talking and helping on big screen)
– Blameless retrospectives are held
– No “Devops team”, creating a silo is not a solution for silos
Absence of Fairness
– You build it, you run it
– Everybody is responsible for quality
– Everybody is measured in the same way
– example Expedia – *everything* deployed has A/B tesing
– everybody goes to release party
Conflicting Values
– In the broadest possible sense
– eg Company industry and values should match your own
Reminder: it is about you and how you fit in with the above
Pay attention to how you feel
– Increase your self awareness
– Maslach Burnout inventory
– Try not to focus on the negative.
Pay attention to work/life balance
– Ask for it, company might not know your needs
– If you can’t get it then quit
Talk to somebody
– Professional help is the best
– Trained to identify cause and effect
– can recommend treatment
– You’d call them if you broke your arm
Friends and family
– People who care, that you haven’t even meet
– Empathy is great , but you aren’t a professional
– Don’t guess cause and effect
– Don’t recommend treatment if not a professional
Q: Is it Gender specific for men (since IT is male dominated) ?
– The “absence of fairness” problem is huge for women in IT
Q: How to promote Psychological safety?
– Blameless post-mortems
Damian Brady – Just let me do my job
After working in govt, went to work for new company and hoped to get stuff done
But whole dev team was unhappy
– Random work assigned
– All deadlines missed
– Lots of waste of time meetings
But 2 years later
– Hitting all deadlines
– Useful meetings
What changes were made?
New boss, protect devs for MUD ( Meetings, uncertainty, distractions )
Meetings
– In board sense, 1-1, all hands, normal meetings
– People are averaging 7.5 hours/week in meetings
– On average 37% of meeting time is not relevant to person ( ~ $8,000 / year )
– Do meetings have goals and do they achieve those goals?
– 38% without goals
– only half of remaining meet those goals
– around 40% of meetings have and achieve goals
– Might not be wasted. Look at “What has changed as result of this meeting?”
Meetings fixes
– New Boss went to meetings for us (didn’t need everybody) as a representative
– Set a clear goal and agenda
– Avoid gimmicks
– don’t default to 30min or 1h
Distractions
– 60% of people interrupted 10 or more times per day
– Good to stay in a “flow state”
– 40% people say they are regularly focussed in their work. but all are sometimes
– 35% of time loss focus when interrupted
– Study shows people can take up to 23mins to get focus back after interruption
– $25,000/year wasting according to interruptions
Distraction Fixes
– Allowing headphones, rule not to interrupt people wearing headphones
– “Do not disturb” times
– Little Signs
– Had “the finger” so that you could tell somebody your were busy right now and would come back to them
– Let devs go to meeting rooms or cafes to hide from interruptions
– All “go dark” where email and chat turned off
Uncertainty
– 82% in survey were clear
– nearly 60% of people their top priority changes before they can finish it.
– Autonomy, mastery, purpose
Uncertainty Fixes
– Tried to let people get clear runs at work
– Helped people acknowledge the unexpected work, add to Sprint board
– Established a gate – Business person would have to go through the manager
– Make the requester responsible – made the requester decide what stuff didn’t get done by physically removing stuff from the sprint board to add their own
I’ve been playing with Prometheus monitoring lately. It is fairly new software that is getting popular. Prometheus works using a pull architecture. A central server connects to each thing you want to monitor every few seconds and grabs stats from it.
In the simplest case you run the node_exporter on each machine which gathers about 600-800 (!) metrics such as load, disk space and interface stats. This exporter listens on port 9100 and effectively works as an http server that responds to “GET /metrics HTTP/1.1” and spits several hundred lines of:
Other exporters listen on different ports and export stats for apache or mysql while more complicated ones will act as proxies for outgoing tests (via snmp, icmp, http). The full list of them is on the Prometheus website.
So my problem was that I wanted to check my virtual machine that is on Linode. The machine only has a public IP and I didn’t want to:
Allow random people to check my servers stats
Have to setup some sort of VPN.
So I decided that the best way was to just use put a user/password on the exporter.
However the node_exporter does not implement authentication itself since the authors wanted the avoid maintaining lots of security code. So I decided to put it behind a reverse proxy using apache mod_proxy.
Step 1 – Install node_exporter
Node_exporter is a single binary that I started via an upstart script. As part of the upstart script I told it to listen on localhost port 19100 instead of port 9100 on all interfaces
# cat /etc/init/prometheus_node_exporter.conf
description "Prometheus Node Exporter"
start on startup
chdir /home/prometheus/
script
/home/prometheus/node_exporter -web.listen-address 127.0.0.1:19100
end script
Once I start the exporter a simple “curl 127.0.0.1:19100/metrics” makes sure it is working and returning data.
Step 2 – Add Apache proxy entry
First make sure apache is listening on port 9100 . On Ubuntu edit the /etc/apache2/ports.conf file and add the line:
Listen 9100
Next create a simple apache proxy without authentication (don’t forget to enable mod_proxy too):
# more /etc/apache2/sites-available/prometheus.conf
<VirtualHost *:9100>
ServerName prometheus
CustomLog /var/log/apache2/prometheus_access.log combined
ErrorLog /var/log/apache2/prometheus_error.log
ProxyRequests Off
<Proxy *>
Allow from all
</Proxy>
ProxyErrorOverride On
ProxyPass / http://127.0.0.1:19100/
ProxyPassReverse / http://127.0.0.1:19100/
</VirtualHost>
This simply takes requests on port 9100 and forwards them to localhost port 19100 . Now reload apache and test via curl to port 9100. You can also use netstat to see what is listening on which ports:
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:19100 0.0.0.0:* LISTEN 8416/node_exporter
tcp6 0 0 :::9100 :::* LISTEN 8725/apache2
Step 3 – Get Prometheus working
I’ll assume at this point you have other servers working. What you need to do now is add the following entries for you server in you prometheus.yml file.
First add basic_auth into your scape config for “node” and then add your servers, eg:
Now restart Prometheus and make sure it is working. You should see the following lines in your apache logs plus stats for the server should start appearing:
Notice that connections are 15 seconds apart, get http code 200 and are 11k in size. The Prometheus server is using Authentication but apache doesn’t need it yet.
Note that the “prom” in field 3 indicates that we are logging in for each connection. If you try to connect to the port without authentication you will get:
Unauthorized
This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.
That is pretty much it. Note that will need to add additional Virtualhost entries for more ports if you run other exporters on the server.
Maybe try and engage with the system we have for a start before writing it off.
You disrupt the system yourself or you hold the system accountable
Nick McFarlane
He wrote a book
Rock Stars are dicks to work with
So you want to Start a Business
Hosted by Reuben and Justin (the accountant)
Things you need to know in your first year of business
How serious is the business, what sort of structure
If you are serious, you have to do things properly
Have you got paying customers yet
Could just be an idea or a hobby
Sole Trader vs Incorporated company vs Trust vs Partnership
Incorperated
Directors and Shareholders needed to be decided on
Can take just half an hour
when to get a GST number?
If over $60k turnover a year
If you have lots of stuff you plan to claim back.
Have an accounting System from Day 1 – Xero Pretty good
Get an advisor or mentor that is not emotionally invested in your company
If partnership then split up responsibilities so you can hold each other accountable for specific items
If you are using Xero then your accountant should be using Xero directly not copying it into a different system.
Remuneration
Should have a shareholders agreement
PAYE possibility from drawings or put 30% aside
Even if only a small hobby company you will need to declare income to IRD especially non-trivial level.
What Level to start at Xero?
Probably from the start if the business is intended to be serious
A bit of pain to switch over later
Don’t forget about ACC
Remember you are due provisional tax once you get over the the $2500 for the previous year.
Home Office expense claim – claim percentage of home rent, power etc
Get in professionals to help
Diversity in Tech
Diversity is important
Why is it important?
Does it mean the same for everyone
Have people with different “ways of thinking” then we will have a diverse views then wider and better solutions
example “Polish engineer could analysis a Polish specific character input error”
example “Controlling a robot in Samoan”, robots are not just in english
Stereotypes for some groups to specific jobs, eg “Indians in tech support”
Example: All hires went though University of Auckland so had done the same courses etc
How do you fix it when people innocently hire everyone from the same background? How do you break the pattern? No be the first different-hire represent everybody in that group?
I didn’t want to be a trail-blazer
Wow’ed out at “Women in tech” event, first time saw “majority of people are like me” in a bar.
“If he is a white male and I’m going to hire him on the team that is already full of white men he better be exception”
Worried about implication that “diversity” vs “Meritocracy” and that diverse candidates are not as good
Usual over-representation of white-males in the discussion even in topics like this.
Notion that somebody was only hired to represent diversity is very harmful especially for that person
If you are hiring for a tech position then 90% of your candidates will be white-males, try place your diversity in getting more diverse group applying for the jobs not tilt in the actual hiring.
Even in maker spaces where anyone is welcome, there are a lot fewer women. Blames mens mags having things unfinished, women’s mags everything is perfect so women don’t want to show off something that is unfinished.
Need to make the workforce diverse now to match the younger people coming into it
Need to cover “power income” people who are not exposed to tech
Even a small number are role models for the future for the young people today
Also need to address the problem of women dropping out of tech in the 30s and 40s. We can’t push girls into an “environment filled with acid”
Example taking out “cocky arrogant males” from classes into “advanced stream” and the remaining class saw women graduating and staying in at a much higher rate.
Podcasting
Paul Spain from Podcast New Zealand organising
Easiest to listen to when doing manual stuff or in car or bus
Need to avoid overload of commercials, eg interview people from the company about the topic of interest rather than about their product
Big firms putting money into podcasting
In the US 21% of the market are listening every single month. In NZ perhaps more like 5% since not a lot of awareness or local content
Some radios shows are re-cutting and publishing them
Not a good directory of NZ podcasts
Advise people use proper equipment if possible if more than a once-off. Bad sound quality is very noticeable.
One person: 5 part series on immigration and immigrants in NZ
Making the charts is a big exposure
Apples “new and noteworthy” list
Domination by traditional personalities and existing broadcasters at present. But that only helps traction within New Zealand
At the Gather Conference again for about the 6th time. It is a 1-day tech-orientated unconference held in Auckland every year.
The day is split into seven streamed sessions each 40 minutes long (of about 8 parallel rooms of events that are each scheduled and run by attendees) plus and opening and a keynote session.
How to Steer your own career – Shirley Tricker
Asked people hands up on their current job situation, FT vs PT, sinmgle v multiple jobs
Alternatives to traditional careers of work. possible to craft your career
Have somebody to report to, eg meet with friend/adviser monthly to chat and bounce stuff off
Cultivate Women’s mentoring group
This doesn’t seem to filter through to young people, they feel they have to pick a career at 18 and go to university to prep for that.
Give advice to people and this helps you define
Try and make the world a better place: enjoy the work you are doing, be happy and proud of the outcome of what you are doing and be happy that it is making the world a bit better
How to I “motivate myself” without a push from your employer?
Do something that you really want to do so you won’t need external motivation
Find someone who is doing something write and see what they did
Awesome for introverts
If you want to start a startup then work for one to see what it is like and learn skills
You don’t have to have a startup in your 20s, you can learn your skills first.
Sometimes you have to do a crappy job at the start to get onto the cool stuff later. You have to look at the goal or path sometimes
Books and Podcasts – Tanya Johnson
Stuff people recommend
Intelligent disobedience – Ira
Hamilton the revolution – based on the musical
Never Split the difference – Chris Voss (ex hostage negotiator)
The Three Body Problem – Lia CiXin – Sci Fi series
Lucky Peach – Food and fiction
Unlimited Memory
The Black Swan and Fooled by Randomness
The Setup (usesthis.com) website
Tim Ferris Podcast
Freakonomics Podcast
Moonwalking with Einstein
Clothes, Music, Boy – Viv Albertine
TIP: Amazon Whispersync for Kindle App (audiobook across various platforms)
TIP: Blinkist – 15 minute summaries of books
An Intimate History of Humanity – Theodore Zenden
How to Live – Sarah Bakewell
TIP: Pocketcasts is a good podcast app for Android.
Tested Podcast from Mythbusters people
Trumpcast podcast from Slate
A Fighting Chance – Elizabeth Warren
The Choice – Og Mandino
The Good life project Podcast
The Ted Radio Hour Podcast (on 1.5 speed)
This American Life
How to be a Woman by Caitlin Moran
The Hard thing about Hard things books
Flashboys
The Changelog Podcast – Interview people doing Open Source software
The Art of Oppertunity Roseland Zander
Red Rising Trilogy by Piers Brown
On the Rag podcast by the Spinoff
Hamish and Andy podcast
Radiolab podcast
Hardcore History podcast
Car Talk podcast
Ametora – Story of Japanese menswear since WW2
.net rocks podcast
How not to be wrong
Savage Love Podcast
Friday Night Comedy from the BBC (especially the News Quiz)
Answer me this Podcast
Back to work podcast
Reply All podcast
The Moth
Serial
American Blood
The Productivity podcast
Keeping it 1600
Ruby Rogues Podcast
Game Change – John Heilemann
The Road less Travelled – M Scott Peck
The Power of Now
Snow Crash – Neil Stevensen
My Journey to becoming a Change Agent – Suki Xiao
Start of 2015 was a policy adviser at Ministry
Didn’t feel connected to job and people making policies for
Outside of work was a Youthline counsellor
Wanted to make a difference, organised some internal talks
Wanted to make changes, got told had to be a manager to make changes (10 years away)
Found out about R9 accelerator. Startup accelerator looking at Govt/Business interaction and pain points
Get seconded to it
First month was very hard.
Speed of change was difficult, “Lean into the discomfort” – Team motto
Be married to the problem
Specific problem was making sure enough seasonal workers, came up with solution but customers didn’t like it. Was not solving the actual problem customers had.
Team was married to the problem, not the married to the solution
When went back to old job, found slower pace hard to adjust back
Got offered a job back at the accelerator, coaching up to 7 teams.
Very hard work, lots of work, burnt out
50% pay cut
Worked out wasn’t “Agile” herself
Started doing personal Kanban boards
Cut back number of teams coaching, higher quality
Spring Board
Place can work at sustainable pace
Working at Nomad 8 as an independent Agile consultant
Work on separate companies but some support from colleges
Find my place
Joined Xero as a Agile Team Facilitator
Takeaways
Anybody can be a change agent
An environment that supports and empowers
Look for support
Conversation on how you overcome the “Everest” big huge goal
Hard to get past the first step for some – speaker found she tended to do first think later. Others over-thought beforehand
It seems hard but think of the hard things you have done in your life and it is usually not as bad
Motivate yourself by having no money and having no choice
Point all the bad things out in the open, visualise them all and feel better cause they will rarely happen
Learn to recognise your bad patterns of thoughts
“The Way of Art” Steven Pressfield (skip the Angels chapter)
Are places Serious about Agile instead of just placing lip-service?
Questioner was older and found places wanted younger Agile coaches
Companies had to completely change into organisation, eg replace project managers
eg CEO is still waterfall but people lower down are into Agile. Not enough management buy-in.
Speaker left on client that wasn’t serious about changing
Went though an Agile process, made “Putting Agile into the Org” as the product
Show customers what the value is
Certification advice, all sorts of options. Nomad8 course is recomended
Over Easter 2016 (March 25th – 27th) I played in the Wellington Open Chess Tournament. I play in the tournament about half of the time. This year it was again being played at the CQ Hotel in Cuba street so I was able to stay at the venue and also visit my favorite Wellington cafes.
There were 43 players entered (the highest for several years) with around 9 coming down from Auckland. I was ranked 16th with a rating of 1988 and the top 4 Wellington players ( Dive, Wastney, Ker & Croad) who are all ranked in the Top 10 in NZ were playing.
See the Tournament’s page for details and downloads for the games. Photos by Lin Nah and me are also up on Flickr for Days one, two and three.
Round 1 – White vs Dominic Leman (unrated) – Result win
This game was over fairly quickly after my opponents 5th Move (Nf6) which let me win a free Bishop after ( 5.. Nf6 6.Nxc6 bxc6 7.Bxc5 ) and then they played (7.. Nxe4) to take the pawn which loses the Night since I just pin it again the King with Qe2 and pick it up a move or two later.
Round 2 – Black vs Michael Steadman ( 2338) – Result lose
Mike plays at my club and is rated well above me. However I put on a pretty poor show and made a mistake early in the Opening (which was one of my lines rather than something Mike usually plays). Error on move 5 lost me a pawn and left my position poor. I failed to improve and resigned on move 21.
Round 3 – White vs Kate Song (1701) – Result win
After 6. ..a5
I was very keen on beating Kate. While she is rated almost 200 points lower than me she improving faster and beat me in the last round of the Major Open at the NZ Champs at the start of this year.
We were the same colours as our game in January so I spent some time prepping the opening to avoid my previous mistakes.
In that game Black played 6.. a5 (see diagram) and I replied with the inaccurate Be2 and got tied into knots on the Queen side. This time I played 7. Bd3 which is a better line. However after 7. ..Nh6 8. dxc5 Bxc5 9. O-O black plays Ng4 which gives me some problems. After some back and forth Black ended up with a bit of a mid-game advantage with a developed bishop pair. and control of the open C file.
27. Bg5 and I offer a draw
However on move 27 after the rooks had been swapped I was able to play Bg5 which threaten to swap Black’s good Bishop or push it backwards. I offered a draw.
Luckily for me Kate picked to swap the Bishops and Queens with 27. ..Bxg5 28.Nxg5 Qd1+ 29.Qxd1 Bxd1 which left me with almost all my pawns on black squares and pretty safe from her white squared bishop. I then was able to march my King over to the Queenside while my Kingside was safe from the Bishop. After picking up a the a-pawn when the Knight and Bishops swapped I was left with a King plus A&B pawns vs King an b-pawn with around 3 tempo in reserve for pushing back the Black king.
Round 3 – Michael Nyberg vs Leighton Nicholls
Position after 71. Kxg4
Another game during round 3 went very long. This was the position after move 71 , White has just taken blacks last pawn. The game kept going till move 125! White kept try to force black to the edge of the board while black kept his king close to the centre and the Knight nearby (keeping the king away with checks and fork threats).
At move 125 Black (Nicholls) claimed a draw under the 50-move rule at which point Michael Nyberg asked “are you sure” and “are you prepared for any penalties?”. After Leighton confirmed he wanted to go ahead with the claim Michael claimed that the draw rules were changed a couple of years ago and that King+Rook vs King+Knight was allowed 75 moves. And that since the draw claim was incorrect Leighton should lose.
However a check of the Official FIDE rules online showed that there was no such special limited for the material, the rule is always 50 moves (Rule 9.3) . The penalty for incorrectly claiming a draw would also have been 2 minutes added to Michael’s time not Leighton losing the game (Rule 9.5b).
The Arbiter checked the rules and declared the game a draw while Michael grumbled about appealing it (which did not happen). Not a good way to end the game since I thought Leighton defended very well. Especially the way Michael was very aggressive while being completely in the wrong.
There have been exceptions to the 50-move draw rule in the past but it has been a flat 50 moves since at least 2001 since while some positions take longer in theory no human would actually be able to play them perfectly.
Round 4 – Black vs David Paul – Result win
Another game against somebody close to my rating but a little below. So while I should win it could be hard. I didn’t play the opening right however and ended up in a slightly poor position a couple of tempo down.
After 32 Re4 draw offered
After some maneuvering (and the odd missed move by both sizes) white offered a draw after move 32. I decided to press on with f6 and was rewarded when after 32. ..f6 33.Kf2 Kf7 White played 34.b4? which allowed me to play Nc3 and bounce my Night to b5 and then take the Bishop on d6 along with an extra pawn.
After 44. ..Kd6
A few moves later I’m a pawn up and with a clear path to the win although I made a mistake at the ended it wasn’t bad enough to be fatal.
Round 5 – White vs Russell Dive – Game lost
After getting onto 3 points after 6 rounds I was rewarded with playing the top seed. As often happens with stronger players he just seemed to make 2 threats with every move and my position slowly (well not that slowly) got worse and worse as I couldn’t counter them all (let alone make my own threats).
Eventually I resigned 3 pawns down with no play (computer assessed my position as -5.0)
Round 6 – Black vs Brian Nijman – Game Lost
Last round was once again against a higher rater play but one I had a reasonable chance against.
After 10. ..Bg6
I prepped a bit of the opening but he played something different and we anded up in a messy position with White better developed but not a huge advantage.
We both had bishops cutting though the position and Queens stuck to the side but it would be hard for me to develop my pieces. I was goign to have to work hard at getting them out into good positions
After 23. d5
After some swaps white ended up charging though my centre and with lots of threats. I spent a lot of time looking at this position workign out what to do.
White has the Bishop ready to take the pawn on b5 and offer check, possibly grab the Knight or pin the rook. While th Knight can also attack the rook. and the pawns can even promote.
I ended up giving up the exchange for a pawn but promptly lost a pawn when white castled and took on f7.
After 32. Ne2
I decided to push forward hoping to generate some threats and managed to when I threated to mate with two Knights or win a rook after 32. Ne2
34.Rxc5+ Kxc5 35.Be1 Rd8 36.Rc7+ followed but I played 36. ..Kd4 and blocked by Rook rather than Kb6 giving myself a tempo to move my rook to d1. This would have probably picked up another exchange and should have been enough for the win.
After 47. g6
And then I found another win. All I had to do was push the pawn. On move 47 I just have to put a piece on f2 to block the bishop from taking my pawn on g1. If 47. ..Nf2 48. Bxf2 Rxf2 49. g1=Q leaves me a Queen vs a rook and I can take the pawn on g6 straight away.
But instead I got Chess Blindness and just swapped the pawn for the Bishop. I then tried to mate (or perpetual check) the King instead of trying to stop the pawns (the computer says 50. ..Nf4 is just in time). A few moves later I ran out of King-chasing moves and resigned. At which point everybody told me the move I missed 🙂
So I ended up with 3/6 or 50% in the tournament. I Losts to the players better than me and beat the lower rated ones. I’m a little disappointed with the last game and the games against Russell Dive and Mike Steadman but happy with the others. Definitely need to keep working on things though.
Free as in cheap gadgets: the ESP8266 by Angus Gratton
I missed the start of the talk but he was giving a history of the release and getting software support for it.
Arduino for ESP8266 very popular
2015-2016 maturing
Lots of development boards
Sparkfun ESP8266 thing, Adafruid Hazaah, WeMOS D1
Common Projects
Lots of lighting projects, addressable LED strips
Wireless power monitoing projects
Copy of common projects. Smoke alarm project
ESPlant – speakers project built in Open Hardware Miniconf – solar powered gardening sensor
Moodlight kickstarter
Shortcomings
Not a lot of documentation compared to other micro-controllers. 1/10 that of similar products
Weird hardware behaviour. Unusual output
Default baud rate 74880 bps
Bad TLS – TLS v1.0, 1.1 only , RSA 512/1024 . 2048 might work
Other examples
FOSS in ESP8266
GCC , Lua , Arduino, Micro Python
axTLS , LWIP, max80211, wpa_supplicant
Wrapped APIs, almost no source, mostly missing attribution
Weird licenses on stuff
Does this source matter?
Anecdote: TLS random key same every time due to bad random function (later fixed). But still didn’t initially use the built-in random number generator.
Reverse Engineering
Wiki , Tools: foogod/xtobjdis , ScratchABit , radara2 (soon)
esp-open-rtos – based on the old version that was under MIT
mbedTLS – TLS 1.2 (and older) , RSA to 4096 and other stuff. Audited and maintained
Working on a testing setup for regression tests
For beginners
Start with Ardino
Look at dev board
Future
Hopefully other companies will see success and will bring their own products out
but with a more open licenses
ESP32 is coming, probably 1y away from being good and ready
secretd – another take on securely storing credentials by Tollef Fog Heen
Works for fastly
What is the problem?
Code can be secret
Configuration can be secret
Credentials are secret
Secrets start in the following and move to the next..
directly code
then a configuration file
then an pre-encrypted store
then an online store
Problems with stores
Complex or insecure
Manual work to re-encrypt
Updating is hard
Not support for dev/prod split
Requirements for a fix
Dynamic environment support
Central storage
Policy based access controls, live
APIs for updating
Use Case
Hardware (re)bootstrapping
Hands-of/live handling
PCI: auditing
Machine might have no persistent storage
Options
pwstore – pre-encrypted
chef-vault – pre-encrypted
Hashicorp Vault – distributed, complex, TTL on secrets
Lots of rolls as an Anthropologist at Intel over last 15 years or so
Vision of future from 1957 shows what the problems are in 1957 that the future would solve
Visions of the future seem very clean and linear, in reality it is messy and myriad.
ATM machine told her “Happy Birthday”
Imagining “Have you tried turning it off and on again?” at smart city scale is kind of terrifying.
Connectivity
Many people function well when they are offline, some people used to holiday in places with no cell reception
Social structures like Sabbath to give people time offline, but devices want us to be always online
Don’t want to always have seamless between devices, context matters. Want work/home/etc split
IOT
Technology lays bare domestic habits that were previously hidden
Who is else knows what you household habits are -> Gossip
Big Data
Messy , incomplete, inaccurate
Average human tells 6-200 lies per day
100% of Americans lie in online profiles
Men lie about height, Women lie about weight
More data does not equal more truth. More data just means more data
Algorithms
My optimise for the wrong things (from the user’s point of view)
Security and Privacy
Conversation entwined with conversation about National Security
Concepts different from around the world
What is it like to release data under one circumstance and then to realise you have released it under several others
Memory
Cost of memory down to zero, we should just store everything
What are the usage models
What if everything you ever did and said was just there, what if you can never get away from it. There are mental illnesses based on this problem
Innovation
What is changing? to whose advantage and disadvantage? what does this mean to related areas?
Our solutions need to be human
We are the architects of our future
Question
Explain engineers to the world? – Treated first year at Intel like it was Anthropology fieldwork. Disconnect between what people imagine technologists think/do and what they really do. Need to explain what we do better
Helicopters and rocket-planes by Andrew Tridgell
The wonderful and crazy world of Open Autopilots
Outback Challenge
90km/h for 45 minutes
Search pattern for a lost bushwalker with UAV
Drop them a rescue package
2016 is much harder VTOL, get blood sample. Most do takeoff and landing remotely (30km from team).
“Not allowed to get blood sample using a propeller”
VTOL solutions – Helicopters and Quadplanes – tried both solutions
Communication 15km away, 2nd aircraft as a relay
Pure electric doesn’t have range. 100km/h for 1h
Helicopters
“Flying vibration generators with rotating swords at the top”
Hard to scale up which is needed in this case. 15cc motor, 2m blades, 12-14kg loaded
Petrol engines efficient VTOL and high energy density
Very precise control, good in high wind (competition can have ground wind up to 25 knots)
Normal stable flight vibrates at 6G , show example where in a couple of seconds flight goes bad and starts vibrating at 30+ G in a few seconds due to control problem (when pitch controller was adjusted and then started feedback loop)
Quadplanes
Normal Plane with wings but 4 virtually pointing propellers added
Long range, less vibration
initially two autopilots plus one more co-ordinating
electric for takeoff, petrol engine for for long range forward flight.
Hard to scale
crashed
Quadplane v2
Single auto-pilot
avoid turning off quad motors before enough speed from forward motor
Pure electric for all motors
Forward flight with wings much more efficient.
Options with scale-up to have forward motor as petrol
Rockets
Lohan rocket plane – Offshoot of The Register website
Mission hasn’t happened yet
Balloon takes plane to 20km, drops rocket and goes to Mach 2 in 8 seconds. Rocket glides back to each under autopilot and lands at SpacePort USA
3d printed rocket. Needs to wiggle controls during ascent to stop them freezing up.
This will be it’s first flight so has autotune mode to hopefully learn how to fly for the first time on the way down
Hardware running Ardupilot
Bebop drone and 3DR solo runs open autopilot software
BBBmini fully open source kit
Qualcom flight more locked down
PXFMini for smaller ones
Sites
ardupilot.com
dronecode.org
canberrauav.org.au
The world of 100G networking by Christopher Lameter
Why not?
Capacity needed
Machines are pushing 100G to memory
Everything reqires more Bandwidth
Technologies
Was 10 * 10G standards CFP Cxx
New standard is 4 * 28Gs QSFP28 . compact and designed to replace 10G and 40G networking
Inifiband (EDR)
Most mature to date, switches and NICs available
Ethernet
Hopefully available in 2016
NICS under dev, can reuse EDR adapter
OmniPath
Redesigned to try replace infiband
Comparison connectors
QSFP28 smaller
QSFP idea with spliter into 4 * 25G links for some places
Standard complete in 2016 , 50G out there but standard doesn’t exist yet.
QSFP is 4 cables
100G switches
100G x 32 or 50G x64 or 25G x 128
Models being released this year, hopefully
Keeping up
100G is just 0.01ns per bit , 150ns for 1500MTU packet, 100M packets/second, 50 packets per 10 us
Hardware distributed packets between cores. will need 60 cores to handle 100G in CPU, need to offload
Having multiple servers (say 4) sharing a Nic using PCIe!
How do you interface with these?
Socket API
Looking Ahead
100G is going to be a major link speed in data centers soon
Software needs to mature especially the OS stack to handle bottlenecks
