NetHui 2015 – Thursday Afternoon

Domains: growth, change, transition

  • Transition of .nz to second level domains
  • Some stuff re moving root zone control away from the US
  • Problem with non-ascii domains (IDNs). They work okay, but not 3rd party apps or apps in Organisations. Eg can’t register on Facebook or other websites.
  • 60% of Government Depts don’t accept IDNs as email addresses, lots of other orgs
  • 1/3 of all new .nz domains created at second level
  • Around 95k or 600k .nz domains now at second level (about 2/3s of these from rights are 3LD holder)
  • Some people when you give them your address.nz change it into address.co.nz
  • 1st principles of .nz whois public policy.
  • People are in danger if they address is published
  • But what the ability to contact the real owner of a domain
  • 4 people in room with signed domains
  • 300 signed .nz domains. 150 with DS record
  • Around 3 people in room with new TLDs. See ntldstats.com for current stats

Internet of Things

  • Where does the data from your house appliances go?
  • Forwarded to other companies
  • Issues need to be understandable by ordinary citizens especially terms and conditions
  • Choose the data that you choose to share with the company rather than company choosing what it shares with you (and others)
  • In health care area people worried about sharing data if it will affect their insurance premiums or coverage
  • Many people don’t understand what their data is, they don’t understand that if every time they do something (on a device) it is stored and can be used later. How to educate people without sounding paranoid?
  • “IoT is connecting things whose primary purpose is not connecting to the Internet”
  • “The cost of sharing is bearable, because the sharing is valuable.”
  • More granularities of trust. No current standards or experience or feeling for this since such a new area and rapidly evolving
  • NZ law should override overly aggressive agreements (by overseas companies)
  • Some discussion about standards, lots of them, full stack, piecemeal, rapidly changing
  • Will the IoT make everything useless after the zombie apocalypse?
  • “Denial of Service attack on your IoT pill bottle would be bad!”
  • Concern that something like a pill bottle failing can put life in danger. Very high level of reliability needed which is rare and hard in software

Panel: Parliamentary Internet Forum

  •  With Gareth Hughes (Green Party), Clare Curran (Labour Party), Brett Hudson (National Party), Ria Bond (NZ First), Karen Melhuish Spencer (Core Education), Nigel Robertson (University of Waikato)
  • What roles does the Education system play in the Internet
    • National guy mostly talked about UFB and RBI programmes, computers in homes
    • Gareth Hughes adopts the “I went out to XYZ School” story. Pushes Teachers not trained and 1 in 4 homes don’t have Internet access.
    • Claire – Got distracted about discussion re her pants. But she said 40% of jobs at risk over next 10-15 years due to impact of technology
    • Karen – I got distracted about another clothing related discussion on twitter
    • Nigel – 1. Use the Internet to do what we already do better. Help people to use the Internet better (digital literacy)
  • Lots of discussion about retraining older people to handle jobs in the future as their present jobs go away
  • How much should government be leading vs getting out of the way and just funding it?
    • Nigel – Government should provide direction. Different in tertiary and other sectors
    • Karen – Collaborative and connected but not mandating
  • “We need to prepare people not just for the jobs of the future, but also to create the companies of the future” – Martin Danner
  • Lots of other stuff but I got distracted.
Share

NetHui 2015 – Thursday Morning

Ministerial address: Hon. Amy Adams, Minister for Communications

  • Mentions she was at community group meeting where people were “shocked” when it was suggested that minutes be sent via email
  • Talk up of the UFB rollout. Various stats about how it is going
  • Also mentioned that Mobile build is part of UFB, better cellular connectivity in rural regions
  • Notes that this will never be 100% complete. The bar keeps moving
  • Very different takeup in different regions. 2% in some 19% in others. Local organisations pushing
  • Good Internet is especially important for remote countries like New Zealand
  • Talk about getting better access in common areas (eg shared driveways) for network builds
  • Notes how Broadcasting and Communications as well as other areas are converging. Previously they were separate silos. Similar for other areas.
  • Harmful Digital Communications Act.
    • Says new framework, adjustment may be needed and bedding down the courts.
    • Says that majority of cases will go to mediation
    • Similar Act in Australia very few things going to courts
    • Gave similar silly literal readings of others acts ( RMA requires a permit to sneeze )
  • 5 “Questions” to minister. 2 on TPP, 1 on Captions, 1 pushing some project and one actual question that she got to answer.
  • Maybe they should look at this idea for the Questions

Keynote: Kathy Brown, ISOC CEO

  • GDP of a National is highly correlated with the growth of the Internet
  • 75% of the benefit of the Internet goes to existing businesses
  • ISOC Global Internet Report 2015
  • Huge growth in Mobile Internet
  • “94% of the global population is covered by mobile networks. Mobile broadband covers 48% of global population”
  • Huge gap between developed and developing counties
  • Report is Online and “Interactional”
  • Challenges
    • Openness of the Internet means information is out there, exposed and gettable by the wrong people sometimes
    • Generational divide in attitude to privacy
  • Privacy is a matter of personal choice. The tools should be available should you wish to use them

Govt 2.0: Digital by default

  • Rachel Prosser and David Farrar facilitating.
  • Room full
  • Result 10 programme background
  • NZ Government Web toolkit
  • 50,000 registered with NZ Realme site
  • Shared rules between local governments, problems with same rules everywhere. Some limitations,. Perhaps at least similar technical standards
  • People don’t care about governments structure, they just want a service, don’t care how depts are arranged.
Share

NetHui 2015 – InTac afternoon

Building an access network for demand and scale – new challengesKurt Rogers, Chorus

  • Over 1 million broadband connections on access network
  • 70-80% of BB connections
  • Average connection sped now near 20Mb/s due to VDSL and Fibre
  • Busiest 15 minute period (around 9pm Thursday) of week averaging 0.5Mb/s per user ( up from 100kb/s just 3 years ago )
  • Jump in mid-2013 when Netflix and Lightbox launched
  • Average bandwidth per user growing 50%/year. Grown that much in 1st half of 2015
  • Quite a few people still on ADSL1 modems when ADSL2 would work
  • Same a lot of people can get VDSL that don’t realize
  • Lots of people on 30Meg fibre plan at the start, now most going for 100Mb/s
  • Rural broadband (RBI)
    • 85k lines upgraded to FTTN
    • Average speed jumped 5.6Mb/s to 15Mb/s after a single rural cabinet upgraded cause everybody could now use ADSL2 and faster uplink. One fibre guy got 48Mb/s on VDSL, other 37Mb/s
    • More speed out there than some people realize
  • VDSL bandplan moving from 997 to 998. Trail average speed increases were from 32 to 46Mb/s for downstream. Minimal change on upstream speed.
  • Capacity
    • Aggregation link bandwidth. Alert threshold at 70%, Max threshold at 90%
  • Technology down the road to speed up aggregation links with Next Generation PON technology

The new smart ISPColin Brown, GM of Networks at Spark

  • Working on caching infrastructure, bigger and closer to their edge
  • Big traffic growth this year
  • Big growth in mobile traffic especially upload
  • 60% of phones in stores are 4G capable
  • Providers investing a lot of money , profits lower. Less like banks, more like airlines
  • Technology refresh every 5 years rather than every 10
Share

NetHui 2015 – InTac morning

IntroductionDean Pemberton, InternetNZ

Dean was going to do an intro but got cock-blocked by some guy in a High-Vis vest.

The People Factor: what users wantPaul Brislen, ex-CEO of TUANZ

  • Working from home since 1999, 30kb/s at first. Made it work
  • Currently has 10Mb/s shared with busy family, often congested, not using much TV yet
  • Television driving demand.
  • Some infrastructure showing the strain
  • Southern cross replacement will be via Sydney. A couple of thousand km in the wrong direction when going to the US
  • Rural broadband still to deliver on the promise, no uptake stats, not great service level
  • Internet access critical path for economic development. lack of political will
  • Dean got to do his intro talk now.
  • Will Internet be priced on peak usage? A: Already offpeak discounts, some ISPs manage home/biz customer ratio to keep traffic balanced
  • Average usage per customer is 5Mb/s for ISP with streaming orientated ISP (acct sold with device).
  • 60% of International traffic going to Aus (to CDNS)
  • Consumers don’t accept buffering, high quality video (bitrate and production quality). Want TV to just-work.
  • NZ doesn’t want to be a “rural” level of internet access, equiv to a farm in more connected countries
  • Could multicast work for live events like sport?
  • Hard to get overage to work to work when people leave TV on all day
  • Plenty of people in Auckland not getting UFB till 2017 (or later)

The connected home and the Internet of ThingsAmber Craig, ANZ

  • At top of Hype cycle
  • Has home Switches on Wemo (have to get upgraded)
  • Lots of devices generating a lot of data
  • Video Blogging – 10GB of raw data, 1GB of finished for just 5 minutes. Uploading to shared drives, sending back and forth through multiple edits
  • Network capacity if probably not much for IoT compared to video, but home will be a source of a lot more uploads
  • With IPv6 maybe less NAT, harder to manage (since people are not used to it).
  • Whose responsibility is it to ensure that Internet works in every room
  • Building standards, what are customers, government, ISP each prepared to pay for?
  • What about medical dependency people who need Internet. A lot of this goes over GSM since that is more “reliable”

Lightbox – content delivery in New ZealandKym Nyblock, Chief Executive of Lightbox

  • Lightbox is part of Spark ventures, morepork, skinny, bigpipe
  • Lighbox – On line TV service, $12.99/month thousands of hours of online content
  • 40% of US household have SVOD, but pay-TV only down 25%
  • Many providers around the world, multiple providers in many countries. Youtube also bit player in the corner
  • SVOD have some impact on piracy, especially those who only pirate cause they want content same day as programme airs in the US
  • Lots of screens now in the house, TV not only viewed on TVs
  • Lightbox challenges
    • Rights issues, lots of competition with other providers, some with fuzzy launch dates
    • NZ Internet not too bad
    • Had to work within an existing company
  • Existing providers
    • Sky – 850k homes, announced own product, has most sports
    • Netflix – approx 30k homes, coming to NZ soon
  • From Biz plan to launch in 12 months
  • Marketing job to be very simple – “Grandma Rule” ( can be explained to Grandma, used by her)
  • Express service delivers content right after views in the US. Lots of views for the episodes that are brand new. One new episode can be 10% of days total views
  • Very agile company, plans changed a lot.
  • Future
    • Customers will have several providers and change often
    • Multiple providers in the market, more to come
    • Premium and exclusive content will drive, simple interface will keep it
    • Rights issues are a problem but locked into the studio system
    • Try to “grow the category”, majority on consumers still using linear, scheduled TV
    • Try to address local rights ownership. This is the bit where they dug at US based providers and people using them.
    • Working on a Sports offering
    • and then she showed a Lightbox ad 🙁
    • Question costs of other ISPs of getting good lightbox due to charges from Spark-Wholesale for bandwidth exchanged. Not really answered

Quickflix – another view of content delivery in New ZealandPaddy Buckley, MD of Quickflix NZ

  • 1st service to launch in March 2012
  • Subscription service for movies and TV shows and Standalone pay-per-view service for new-release movies and some TV shows
  • Across lots of devices, Smart TVs, phones, computers, games consoles, tablets, tivo, chromecast. No Linux Client 🙁
  • Just 15% of views via the website now
  • Content: New release movies, subscriptions content movies, TV shows
  • Uses Akamai for delivery. Hosting Centers in Sydney and Perth. AWS/Azure
  • Unwritten 5 second rule. Content should play within 5 seconds of pressing play
  • The future
    • Multiple Models, Not just SVOD, eg TVOD, AVOD, EVOD, EST
    • More fibre, fast home wifi and better hardware
    • VOD content getting nearer to the viewer. HbbTV combines broadcast and on-demand being done by freeview
    • Android TV
    • Viewing levels to increase (volume and frequency), people will pick and mix between providers
    • Aiming at 50% of households, 1 million is quite a lots for any scale.
  • Coming soon
    • 1080p/4K , 5.1 surround sound
    • Fewer device limits. All services and all devices
    • More streams
    • Changing release windows
    • Live streaming
    • PPV options to compliment
    • Download now, view later
  • What we need from ISPs
    • Significant bandwidth
    • Mooorrreee bandwidth
    • People will change ISPs if the ISP can’t provide the level of service
    • Netflix is naming and shaming. Netflix best/worst list
  • Prediction that NZ could hit 50% SVOD within a couple of years
  • Asked if they will be going broke in next few months. Says he’s done deal with Presto in Aus and will ease funding problems but business as normal in the NZ
  • SVOD has evolved from back-catalog TV shows a few years ago to first-run now. Will probably keep going forward with individual shows being provider-exclusive for now, especially since services are fairly low cost per month
  • A few questions about subtitles. Usually available (although can cost extra) but not good support with end devices to turn on/off .
Share

LInks: WW1 Maps, Shawshank, Microservices, Dev Interviewing

Share

Static networking in Ansible the quick and dirty way

I’m in the process of setting up a server at home to replace an old one. I’m maintaining the new one via Ansible to try and get keep as tidy as possible. Part of the setup involves setting up a bridge interface so that I can run kvm virtual machines on the box.

In order to make the box a little more stable I decided to make the ethernet settings static rather than via DHCP. Unfortunately ansible doesn’t really have a nice standard way of setting up network ports (there are a few modules around but none in the main distribution).  After looking around I decided just to make a simple ansible role to handle the files.

The machine is running centos7. The networking initially looked like:

/etc/sysconfig/network-scripts/ifcfg-enp2s0
::::::::::::::
HWADDR=9C:B6:54:07:E8:49
TYPE=Ethernet
BOOTPROTO=dhcp
NAME=enp2s0
ONBOOT=yes
#

I decided the easiest way was to just manually create and copy the files. So I created a static_networking role.

roles/static_networking/handlers/main.yml
roles/static_networking/files/grey/ifcfg-enp2s0
roles/static_networking/files/grey/ifcfg-bridge0
roles/static_networking/tasks/main.yml
roles/static_networking/tasks/setup-redhat.yml

Inside the tasks the main.yml just loads up the setup-redhat.yml which is:

---
- name: copy files if they are listed in var
  copy: src={{ ansible_hostname }}/ifcfg-{{ item }} dest=/etc/sysconfig/network-scripts/ owner=root mode=0644
  with_items: static_interfaces
  notify:
  - restart network

Which is fairly simple. It just goes though a list of “static_interfaces” for a host and copies these files from the local machine to the machine I am setting up. If the copy makes any changes it sends a notify.

For the machine “grey” I just create some entries in hosts_vars/grey.yml

static_interfaces:
 - enp2s0
 - bridge0

and then the files themselves:

roles/static_networking/files/grey/ifcfg-bridge0
::::::::::::::
DEVICE="bridge0"
ONBOOT="yes"
TYPE=Bridge
BOOTPROTO=static
IPADDR=10.1.1.28
NETMASK=255.255.255.0
GATEWAY=10.1.1.1
::::::::::::::
roles/static_networking/files/grey/ifcfg-enp2s0
::::::::::::::
DEVICE="enp2s0"
ONBOOT="yes"
NM_CONTROLLED="no"
BOOTPROTO="none"
BRIDGE=bridge0
HWADDR="9c:b6:54:07:e8:49"

which are the actual files to be copied. If any files are actually updated the handler will be triggered

roles/static_networking/handlers/main.yml 
---
# Called by "name" when network config files are changed
- name: restart network
  service: name=network state=restarted

Overall it seems to work and I only broke networking once (the ip on enp2s0 keep getting re-added until I forced network manager to forget about it). I wouldn’t really recommend this sort of thing for non-trivial sites though. Keeping per-site configs in roles isn’t really the best way to do things.

Share

Linux.conf.au 2015 – Day 5 – Session 3

NoOps with Ansible and Puppet – Monty Taylor

  • NoOps
    • didn’t know it was a contentious term
    • “devs can code and let a service deploy, manage and scale their code”
    • I want to change the system by landing commits. don’t want to “do ops”
    • if I have to use my root access it is a bug
  • Cloud Native
    • Ephemeral Compute
    • Data services
    • Design your applications to be resilient via scale out
    • Cloud scale out, forget HA for one system, forget long-lived system, shared-nothing for everything. Cloud provides the hard scale-out/HA/9s stuff
    • Great for new applications
  • OpenStack Infra
    • Tooling, automation, and CI for the openstack project
    • 2000 devs
    • every commit is fully tested.
    • each test runs on a single use cloud slave
    • 1.7 million test jobs in the last 6 months. 18 TB of log data
    • all runs in HP and rackspace public clouds
  • Create Servers manually at 1st
  • Step 1 – Puppet
    • extra hipster because it is in ruby
    • If you like ruby it is awesome. If don’t is it less-awesome
    • collaboration from non-root users
    • code review
    • problem that it blows up when you try and install the same thing in two different places
    • 3 ways to run. masterless puppet apply. master + puppet agent daemon . master + puppet agent non-daemons
  • Secret stuff that you don’t want into you puppet git repo
    • hiera
  • Step 2 – Ansible for orchestration
    • Control the puppet agent so it runs it nicely and in schedule and on correct hosts first
    • Open source system management tool
    • Sequence of steps not description of state like puppet
    • ad-hoc operation. run random commands
    • easy to slowly grow over time till it takes over puppet
    • yaml syntax of config files
  • Step 3 – Ansible for cloud management
  • Ansible config currently mixed in with puppet under – http://git.openstack.org/cgit/openstack-infra/system-config/

 

Conference Closing

  • Steve Walsh wins Rusty Wrench award
  • Preview of Linux.conf.au 2016 in Geelong
    • Much flatter than Auckland
    • Deakin University – Waterfront Campus
    • Waurn Ponds student accomadation 15 minutes with shuttles
    • Feb 8th – 12th 2016
    • CFP 1st of June 2015
    • Theme “life is better with linux”
    • 4 keynotes confirmed or in final stages of discussion, 2 female, 2 male
    • NFS keytags
    • lcabythebay.org.au
  • Announcement for Linux.conf.au 2017 will be in Hobart

 

Share

Linux.conf.au 2015 – Day 5 – Session 2

When Everything Falls Apart: Stories of Version Control System Scaling – Ben Kero

  • Sysadmin at Mozilla looking after VCS
  • Primarily covering mercurial
  • Background
    • Primarily mercurial
    • 3445 repos (1223 unique)
    • 32 million commits
    • 2TB+ transfer per day
    • 1000+ clones per day
    • Biggest customer = ourselves
    • tested platforms > 12
  • Also use  git (a lot) and a bit of:  subversion, CVS, Bazaar, RCS
  • 2 * ssh servers, 10 machines mirror http traffic behind load balancer
  • 1st story – know what you are hosting
    • Big git repo 1.7G somebody asked to move off github
    • Turned out to be mozilla git mirror, so important to move
    • plenty of spare resources
    • But high load straight away
    • turned out to be mercurial->git converter, huge load
    • Ran garbage collection – took several hours
    • tweaked some other settings
  • 2nd story
    • 2003 . “Try” CI system
    • Simple CI system (before the term existed or they were common)
    • flicks off to build server, sends status back to dev
    • mercurial had history being immutable up until v2.1 and mozilla was stuck on old version
    • ended up with 29,000 brashes in repo
    • Around 10,000 heads some operations just start to fail
    • Wait times for pushes over 45 minutes. Manual fixes for this
    • process was “hg serve” only just freezein gup, not any debug info
    • had to attached debugging. trying to update the cache.
    • cache got nuked by cached push, long process to rebuild it.
    • mercurial bug 4255 in process of being looked at, no fix yet
  • The new system
    • More web-scalable to replace old the system
    • Closer to the pull-request model
    • multi-homing
    • leverage mercurial bundles
    • stores bundles in scalable object store
    • hopefully minimal retooling from other groups (lots of weird systems supported)
  • Planet release engineering @ mozilla

SL[AUO]B: Kernel memory allocator design and philosophy – Christopher Lameter

  • NOTE: I don’t do kernel stuff so much of this is over my head.
  • Role of the allocator
    • page allocator only works in full page size (4k) and is fairly slow
    • slab allocator for smaller allocation
    • SLAB is one of the “slab allocators”
  • kmeme_cache , numa aware, etc
  • History
    • SLOB: K&R 1991-1999 . compact
    • SLAB: Solaris 199-2008 . cache friendly, benchmark friendly
    • SLUB: 2008-today , simple and instruction costs count, better debugging, defrag, execution time friendly
  • 2013 – work to split out common code for allocators
  • SOLB
    • manages list of free objects with the space of free objects
    • have to traverse list to find object of sufficient size
    • rapid fragmentation of memory
  • SLAB
    • queues per cpu and per node to track cache hotness
    • queues for each remote node
    • complete data structures
    • cold object expiration every 2 seconds on each CPU
    • large systems with LOTS of CPUs have huge amount of memory trapped, spending lots of time cleaning cache
  • SLUB
    • A lot less queuing
    • Pages associated with per-cpu. increased locality
    • page based policies and interleave
    • de-fragmentation on multiple levels
    • current default in the kernel
  • slabinfo tool for SLUB. tune, modify, query, control objects and settings
  • can be asked to go into debug mode even when debugging not enabled with rest of the kernel
  • Comparing
    • SLUB faster (SLAB good for benchmarks)
    • SLOB slow
    • SLOB less memory overhead for small/simple systems (only, doesn’t handle lots of reallocations that fragment)
  • Roadmap
    • More common framework
    • Various other speedups and features

 

Share

Linux.conf.au 2015 – Day 5 – Session 1

How to get one of those Open Source jobs – Mark Atwood

  • Warns talk might still have some US-centric stuff still in it
  • “Open Source Job” – most important word is “Job”
    • The Open Source bit means you are a bit more transferable than a closed-source programmer
    • Don’t have to move to major tech city
  • Communication skills
    • Have to learn to Write clearly in English
    • Heave to learn how to speak, including in meetings and give some talks
    • Reachable – Have a public email address
    • Don’t be a jerk, reputation very important
  • Technical skills
    • Learn how to program
    • Start with python and javascript
    • Learn other languages eg scale, erlang, clojure, c, C++
    • How to use debugger and IDE
    • Learn to use git well
    • Learn how to code test (especially to work with CI testers like jenkins)
    • Idea: Do lots of simple practise problems in programming using specific technique or language
  • Relationships & Peers
    • Work with people remote and nearby
    • stackoverflow
    • Don’t be a jerk
  • Work
    • Have to “do the work” then “get the job”
    • Start by fixing bugs on a project
    • Your skills will improve and others will see you have those skills
  • Collaborate
    • Many projects use IRC
    • Most projects have bug tracker
    • Learn how to use the non-basic stuff in git
    • Peer programming
  • Reputation
    • Portfolio vs resume
    • github account is your portfolio
    • Need to be on social media, at least a little bit, most be reachable
  • Getting the Job
    • If you have a good enough a rep the jobs will seek you out
    • Keywords on github and linkedin will attract recruiters
    • People will suggest you that apply
    • Conferences like linux.conf.au
    • Remember to counter-offer the offer letter
    • Once you are working for them, work out what is job related an the company might have a claim on. make sure you list in your agreement any projects you are already working on
  • Health
    • Don’t work longer than 40h a week regularly
    • 60h weeks can only be sustained for a couple of weeks
    • Just eat junk-food
    • Don’t work for jerks
  • Money
    • Startups – bad for your health. Do not kill yourself for a nickle, have real equity
  • Keep Learning
  • 3 books to read
    • Oh the palces you will go – Dr Seuss
    • Getting things Done – David Allen
    • How to fail at almost everything and still win big – Scott Adams

 

Pettycoin: Towards 1.0 – Rusty Russell

  • Problem it bitcoining mining is expensive, places lower limit on transaction fees
  • Took 6 months of to mostly work on pettycoin
  • Petty coin
    • Simple
    • gateway to bitcoin
    • small amounts
    • partial knowledge, don’t need to know everything
    • fast block times
  • Altcoins – bitcoin like things that are not bitcoin
    • 2 million posts to altcoin announce forum
    • lots of noise to talk to people
  • review
    • Paper released saying how it should have been done
    • hash functions
    • bitcoin blocks
    • Bitcoin transactions
  • Sidechain
    • alternative chains that use real bitcoins
    • Lots of wasted work? – bitcoin miners can mine other chains at the same time
    • too fast to keep notes
    • Compact CVP Proofs (reduce length of block header to go all the way back )

 

Share

Linux.conf.au 2015 – Day 5 – Keynote/Panel

  • Everybody Sung Happy birthday to Baale
  • Bdale said he has a new house and FreedomBox 0.3 release this week
  • Rusty also on the panel
  • Questions:
    • Why is Linus so mean
    • Unified Storage/Memory machines – from HP
    • Young people getting into community
    • systemd ( I asked this)
    • Year of the Linux Desktop
    • Documentation & training material
    • Predict the security problems in next 12 month
    • Does NZ and Australia need a joint space agency
    • Will you be remembered more for Linux or Git?
Share