LCA2012 – Friday after lunch

Codec 2 – David Rowe

  • Open speech Codec. Low bitrate 2400 b/s down to 1400 b/s
  • Applcations for digital radio
  • Fills <5000 b/s gap
  • http://rowetel.com/codec2.html
  • Not a DSP talk
  • Can send 45 calls inside 64 kb/s chanel
  • Not useful for VOIP due to IP/UDP overhead of 8kb/s on 1400b/s data
  • Main use radio spectrum. Less data = less power required since your power gets concentrate on less bits
  • doesn’t matter too much if odd packet dropped
  • proprietary codecs slowing digital voice over radio
  • Proprietary codes: hardware or licensed software form, difficult to distribute, can’t modify
  • Example g729 license $40k. Doesn’t believe closed source codecs benefit society
  • Authors of propriety/patented codecs borrowed heavily from public domain. perhaps 5% is original. Good news is only 5% needs to be replaced
  • Speech coding: eg 16bit samples at 8kHz, comprss to 1400-2400 b/s . What can we thrown away, retain intelligible speech, retain natural speech. Use a model of speech, send model parameters, for effecient than coding waveform
  • Model: example is pitch, humans 50-500 Hz , can be represented with 7 bits, updated every 20ms 7/0.02 = 350b/s to represent pitch
  • Codec 2 uses Sinusoidal speech coding. Multiple Sine waves added togeather
  • Bit allocation: 56bits every 40ms. Of these: Amplitude 32 , Frame energy 10 , voicing 4, pitch 10
  • Developing Codecs: complex DSP algorithms, run codecs in non-realtime, dump values from codecs every “frame” ( 80 samples, 10 ms of speech) . Gnu Octave
  • Banned exports list includes ” Speech codecs below 2400 b/s ” . Have been advised by DECO that Codec 2 has “assessed as not controlled” but waiting for certificate

 

UEFI and Linux – Matthew Garrett

  • Replacement for PC BIOS
  • BSD licensed core
  • Adds standardized support for new hardware features
  • Platform init
  • EFI image load – loaded drivers
  • EFI OS loader load – oot from ordered list of EFIOS loaders
  • Boot services terminate -> OS handover
  • Boot services – memory allocation, timers, image loading, GUIDs.
  • Runtime services – non-volatile variable store, boot data, system information, crash dumps (already in Linux 3.2)
  • Able to update firmware by reset and grab new firmware out of variables on bootup
  • GPT – GUID partition table – no practical restrictions on size and number – more metadata about partition type and service
  • That all sounds good …. but ….
  • TianoCore – Open Intel reference UEFI reference implementation, 7061 files, >100MB of code, 10% of size of Linux kernel. Bigger than Linux core kernel
  • Large codebase, some bugs
  • UEFI is poorly tested in the real world. UEFI contains a lot of code. UEFI contains a lot of bugs
  • Some problems with secure boot 🙂
Share

LCA2012 – Friday Morning

Bloat: How and Why UNIX Grew Up (and Out) – Matt Evans and Rusty Russell

  • Cool projects: spark, plover, Homebrew Cray-1A
  • Compare PDP-11 Unix vs Modern Ubuntu 11.10
  • Binary sizes: cat 152 bytes vs  531k KB
  • grep command: 2176 bytes vs 687 KB
  • ls command: 4904 bytes vs 628 KB
  • V6 cat command just 12 lines of assembler, 2 * 512bytes buffers, a.out 16 bytes overhead
  • Binaries 30% because we chose speed over size. ~9% speed gain
  • V6 Runtime coverage: cat 99% , grep 78%, ls 85%
  • V7 has reduced coverage. some commands converted from assembler to C
  • x86 runtime with dietlibc coverage: cat 11% , grep 23% , ls 39%
  • x86 static cat has 700k of libc dependencies, 17% of libc, 313 objects it depends on
  • libc 1.7M but widely shared among hundreds of processes
  • dynamic ls accesses 90k of libc but 476kB paged in.
  • For sample system. libwebkit 8.5MB , 5MB wasted, 33MB wasted real RAM
  • What about a 64Bit version of a PDP-11 – a PDP-11 . Various assumptions on how binary size would increase
  • PDP-44 – binaries around 50% larger
  • 32 bit ubuntu binaries are 9% smaller than 64 bit ubuntu
  • Forward port V6 binaries to x86 . V6 cat almost same size as dietlibc version
  • More work to forward port V6 ls, lots of assumptions not longer true. Code tricks no longer work. 20% larger cause of ELF and nmap. 120% penalty due to modern infrastructure (eg malloc realloc)
  • Backport x86 “ls” and “cat” to V6. Only backport some options
  • cat: remove old options and error reporting. Kept some features.
  • ls: remove lots of options.
  • Binaries 60% larger due to flexibility
  • 440% bloat due to new features
  • Asmutls – reimplementation of current Linux utils in x86 assembler.
  • The talk is online, hard to do notes since it jumped around a lot and graphs hard to read

 

Open vSwitch – Simon Horman

  • Switch contains ports, ports has one of more interface, packets are forwarded by flow
  • Flows may be identified by lots of combos, address, vlan, ports, TOS
  • 1st packet in flow gets sent to userspace controller, controller makes decision, tells datapath what to do with future packets, resends first packet back to datapath. Later packets the datapath knows what to do (from hashtable lookup) and handles itself
  • Configured by JSON database, persists across restarts
  • database controlled via Unix socket or via TCP. Change action won’t return until database update performed
  • cute ” –may-exist ” options when creating stuff that does nothing if what you are requested already exists
  • He did some demos of standard sort of stuff, truck interfaces, port mirroring, fairly simple commands to do
  • Does VXLAN and GRE tunnels
  • Oracle looking to put in Oracle Linux soon to replace current bridging code
  • Can do millions of packets per second. Some bottlenecks in tunneling code
Share

LCA2012 – Thursday last session

Challenges for the Linux plumbing community – Jonathan Corbet

  • Good news is boring, so how about some “high quality problems”
  • Security
  • Stuxnet , kernel.org , RSA hack , DigiNotar
  • Scary ones are there must be others we haven’t heard about
  • The bad guys are: motivated, capable, well funded. Not just script kiddies
  • Not just about money anymore, with governments hacking lives are at stake
  • We are on the front line. Not just security software, all code security critical
  • Is your code secure? Who reviews it? What sort of testing? Plans for dealing with vulnerabilities?
  • Is your infrastructure secure? – Who has access, who can change files? Are security updates applied? What are your plans in case of a breach?
  • Are your processes secure? Who can commit? What can sign releases? Can you detect tampering? What do they know about the codes provenance?
  • Tools
  • Lockdep, valgrind, fault injection, sparse, smatch
  • GCC python plugin, MELT, LLVM static analyzer
  •  and need to actually use the tools that exist
  • Hardware
  • hardware complexity leads to software complexity
  • Complex interfaces: example V4L3 media controller interface.
  • Control over our hardware
  • Life is okay (could be better, could be worse)
  • What is our influence over manufacturer?
  • Example: Chasing tablet manufactures , no influence on design, have to port after device launched
  • Example: By the time “Rock Box” runs on a device device is obsolete and not in shops
  • How can we be more involved in conception and design of hardware in the first place?
  • Linux Only
  • Once upon a time we depended heavily on portability
  • The DRM tree deemphasized BSD support, This hurt BSD but… would we rather do without kernel mode settings
  • Might be inevitable but try not to be too arrogant
  • The platform problem
  • Code you control vs Black box
  • The kernel’s ARM subtree (re-implements stuff from elsewhere in kernel)
  • XFree86 (tried to keep everything in user space)
  • Opportunistic suspend (Andriod decided “too hard” to fix rest of kernel)
  • Async I/O (implemented multiple times, no comprehensive implementation)
  • Example: wireless devices had own 80211 implementation. replaced with max80211
  • Example: PowerTop used to find wide range a random things causing high power usage in laptops
  • Ongoing examples: Bufferbloat, marvell-cam drivers, User-Space TCP, Control groups, Andriod

 

 

What is in a tiny Linux installation? by Malcolm Tredinnick

  • Skipping bootloader portion
  • Kernel is big – 9.6M lines of C, 250k lines of assembler
  • Booting the kernel
  • “make allnoconfig” , smallish, 222 “y” ‘s. 842KB bzimage, build time under 15s, no file systems, no fancy hardware, ISA, no PCI
  • “make allyesconfig” , 5177 y options. 39MB bzimage, over 1h to build, includes drivers/staging
  • booting allnoconfig via qemu-kvm . Gets to “unable to mount root file system”
  • Kernel components – hardware arch, drivers, subsystems, others
  • need roto filesystem in memory, initrd / initramfs . init process just in cpio archive, can just be hello world
  • need initrd, initramfs , RAM disk block device, ELF binary support. 889KB bzimage (up 50kb)
  • Now boots, use “rdinit=/hello” option in qemu , just prints out hello world
  • Transition to userspace
  • initrd loads some modules etc, runs pivot_root , run startup scripts
  • Userspace
  • Why are you doing this? Single purpose system, usb stick (rescue, puppy linix, Damn Small Linux) , tiny memory, tiny storage usage, fast power on. Trade-off of options
  • We have to run something, need some binaries, shared libraries, large binaries with multiple purposes (busybox)
  • Busybox – one binary – acts differently depending on calling name, installed as symlinks
  • Busybox: fairly small, default utilities, 2MB without networking, easy to test
  • C libraries – glibc (probably not a good idea), eglibc (easier to build, binary compatible with glibc, can take things out), uClibc (alternative, very small, some overlap with busybox, source code compatible with glibc)
  • Device and Proc mngt will need: procfs, sysfs, tmpfs, udev, cgroups
  • Build environments: you are cross compiling (build root), binutils, C libraries & cross compiling, Test, x86 instead x86 is harder
  • See links in slides for some help
  • mdebian is something to look at
Share

LCA2012 – Thursday after lunch

Women in Open Technology & Culture – Valerie Aurora and Mary Gardiner

  • Very umbrella term including fan fiction, open data, wikipedia, open access
  • Why – important areas – women’s participation (especially in charge) very low
  • Important for women to be in charge, creating, designing, building, not just as users
  • 5 kinds of groups – project specific (debian women), feminist activism, teaching technical skill, networking, majority women projects.
  • Community / project specific
  • Linuxchix, owoot, pyladies, wikichix, etc ( linuxchix spawned several)
  • low participation, poor replacement rate of leaders (often after they get FT jobs), low communication between, sometimes tension between.
  • Feminist advocacy
  • geek feminism, ada initiative, mind the gap
  • growing and active – the new hotness, sharing best practices, paid work more common, some conferences
  • Teaching women technical skills
  • usually one day or evening courses.
  • Growing hugely, vary widely in topics and skills, sharing best practises
  • In person networking socialization
  • Women in code, girl geek coffees, girl geek dinner
  • try not to be dominated by marketing women ( use of “geek” term helps)
  • Growing, easy to start local chapters
  • Majority Women Groups
  • Dreamwidth, Organisation for transformative works
  • Often fan-fiction support, protect against takedown, let author control commercialisation
  • Survey
  • In person vs Online
  • Activist vs non-activist
  • Community vs technical
  • Focussed vs broad topics
  • Projects with broad focus within a narrow group seem not to work
  • Projects with very technical focus but accoess different technologies seem not to work either (lack common language)
  • Why Start – recruit and retain, networking, role models, safe space, feel normal
  • Lessons on starting
  • Don’t – join an existing one
  • If you are a man, don’t do on behalf – “Nothing about us without us”
  • Don’t expect women to start a group
  • Find 3 or more women to start a group
  • Don’t use girl/chix/ladies – use women
  • Go broad instead of narrow on topic
  • have clear defined goals and scope
  • Start small, be realistic about work
  • Consider one-off event rather than group
  • Avoid NIH , reuse best practices
  • be prepared to moderate any public forums you create
  • Failure modes
  • Become “the nice place” that everybody goes to
  • Loses focus on women
  • Safe Space moderation too many hours
  • Ran low on time, slides will be online

 

Hacking Everything – Matt Evans

  • Reuse things , not just hacking things like audrino that are supposed to be hacked
  • reuse, need, art & design
  • Gambiarra – brazilian art of an improvised fix
  • 1940s radios and TV owners could fix their gear. today people are more passive
  • wants people to tinker with things.
  • Save resources
  • Save money
  • take apart things, learn by example
  • Low cost manufacturing makes hacking hard ( solid state everything )
  • Cheap development makes hacking easier ( reuses common technology, extra bits on devices unused )
  • Some products are open hardware designs
  • Things to look for – similar to ref design, debug code left in, unused features, factory test points/ports
  • Ports that are wired up but unused often serial ports
  • “My CD player has a serial port” , common on many devices
  • Acquire a “logic level”USB-serial cable
  • Other ports – JTAB , In-System programming
  • Example: Picture frame, derived from sample board for camera, serial interface, built in CLI
  • Old Wifi, ADSL boxes good with OpenWRT
  • Don’t just consume – re-consume
  • Teach others and tell the world
  • Collaborate at a local hackerspace
  • support companies that make things hackable

 

Share

LCA2012 – Thursday Morning

Desktop Home hacks – Allison Randal

  • Just a hobby product, must be fun, open,
  • Wanted computer to be available when away from desktop, but not be disruptive and uncomfortable
  • Affordable, approachable to hobbyist, there are more expensive alternatives
  • nodes – jeenode, audrino clone, tiny, easy to hid, cheap ( $20 including wireless RFM12B vs $20-30 + 30 for wifi for more conventional audrino )
  • RFM12B – 66 bytes
  • Server – pandaboard ~$200 . ARM processor with full Ubuntu install
  • Server – aggregates data from all notes and sends commands to all it’s nodes. Contains services for system, json data feeds from nodes, web client interface, interface for sending commands to system
  • Client(s) – several machines in house used from
  • client – jquery mobile , small applet , on desktop , chromium app mode
  • Inputs: Temperature ,  Humidity , motion sensor (lights on/off)
  • Input: RFID reader (disappointed at short range, <1m , one was in doorways to trace path of wallet)
  • Input: touch sensor ( simple controls, in pillow ) , small keyboard (bluetooth, usb)
  • Output: Glowing egg ( multi colour, hand sized ) , Power Tail (power extension, turns on/off)
  • Output: Hollowed out candle with LEDs inside , Instamorph & super Sculpey to create “solid” objects
  • Tools: Soldering iron, misc tools, wax carving kits
  • Lesson: Need better camera with macro lens and better light to document 🙂
  • software not currently release, not really in releasable format, thinking of making some as audrino shields
  • Resources: adafruit.com , sparkfun , parallax.com , digikey.com (bad interface) , optopart.com , makershed.com , freetronics.com (Australia) ,
  • Tools: Talk in inkscape/sozi  , Also use: vala (webserver) , jquery mobile

 

Cheap Tabloid tricks – Angus Kidman

  • Journalists have secret lawyer fantasies, they have wrung the information out of the sources
  • Is the IT media biased?
  • no coverage of lca2012 in main Aus IT press
  • more coverage in 2nd tier
  • Only 3% pageviews on lifehacker use Linux
  • Not many IT journalists these days, One on FT on lifehacker, more to do (websites, blogs, video, podcasts)
  • Freelancers – can be an option. Freelance rates not good, getting worse, competition from bloggers, etc. Hard to convince editor that story is worthwhile.
  • IT news driven by fashion. In early 2000s belief among publications that Linux stories would drive traffic from slashdot etc
  • Current fashion is facebook and Apple. Stories about them in demand
  • Eg Lifehacker did apple angle on LCA keynote
  • Open source Projects lack definitive spokesperson
  • Media obsessed with cult of Trivia, Celebrity . eg Linus
  • Media not influenced by advertisers
  • Does Linux need media? Yes if want to reach more than the 3%
  • Identify the Audience. right publication
  • Be Fashionable. eg mention facebook or apple
  • Be Concise.
  • Be contactable. email, phone if in a hurry
Share

LCA2012 – Wednesday last session

The Samba tour of scripting languages – Amitay Isaacs and Andrew Bartlett

  • Samba is C based
  • But seems to have a lot of scripting
  • Has to be portable no non-gnu systems like solaris, reply on POSIX sh, make (not gun make), awk, m4 and a c compiler
  • shell scripts for first testing. Over 10,000 lines if shell in building
  • Python , TCL and Lua bindings all added but unpopular and eventually removed
  • Perl over 20,00 lines
  • IDL build initially in awk. Switched to perl based PIDL
  • javascript before it was cool – embedded javascript engine
  • But the cool kids were using python so switched from Javascript to python
  • Exception based languages things cleaner
  • waf is python based build system
  • python bindings for most things, or via C hook
  • Lots of other stuff being written into python, called directly by samba for small tasks
  • 90,000 lines of python
  • Example: Samba3 upgrade –  python based tool in 3 weeks. Business login in python, exceptions test for bad input
  • At build time python checks to see if ABI has changed from previously and alert developer if it has.
  • Test frameworks – unit tests on standalone components. Environmental tests with everything running, different types on server setups, fake ips and tests made. All runnable as non-root
  • 9000 test in 1300 test suites. Mosts test in C, some in python or shell

 

 

Running Python in Grub – Josh Triplett

  •  ” I ported python to grub “
  • perception that Linux doesn’t need BIOS
  • Involves programming hardware in functional/safe/optismise configuration
  • Lots of stuff to support, a few decades of compatibility, very bare-metal programming, small number of people working on it
  • What can go wrong: Broken or disabled CPU features, missing or broken memory, sub-optimal power mngt, delays & latency, USB bios handoff. Undocumented customer interfaces
  • Why not test under Linux? Linux gets in the way, no direct BIOS access for tests
  • Grub2 – 32 bit addresses, written in C, can read files, menus, single thread, no OS to disturb, only uses a bit of BIOS itself
  • ORigin: Replacing DOS test programs, test for power mngt, new grub commands for command-line.
  • Grub script language – bashish, no expressions, can just glue things together in menus, have to write lots of C
  • Ported CPython 2.7 to GRUB
  • Wrote a C/Posix compatibility layer for GRUB, floating point functionality via fdlibm , ported much of python standard library
  • Build C extension modules, added “bits” module to access platform functionality
  • ACPICA already in Linux for parsing ACPI. ported as a grub module with python module to access it.
  • bits passes grub command line to python for python to parse
  • FUSE for Python and GRUB.  added a python device reading python/foo invokes a python callback.
  • logging to in-memory buffer and sends to log in FUSE via system. Save to ACPI table and then OS can grab it later
  • SMP support in grub, pyton scripting in ring0, python modules for platform interfaces CPU, PCI, PCI-e, ACPI including decoding and method evaluation, logging, test suite evaluation
  • BITS test: power mngt configuration, perf optimisation, CPU config registers, SMI frequency/latency , USB handoff from BIOS to OS including effects on SMI and C states.
  • Used by BIOS vendors before shipping boards, BIOS problems actually get fixed!
  • http://biosbits.org/
  • Freakiest stuff I’ve seen so far this week
Share

LCA2012 – Wednesday after Lunch

Ubuntu ARM – David Mandala

  • Targeted ARM v7 – first release April 2009
  • 7 different kernels in 10.10 . Improving with Linaro
  • OMAP 3 100% in main Linux kernel so easy to support.
  • Someday unified kernel onto ARM
  • Toshiba AC-100 netbook
  • 11.10 preview release of ARM server
  • Lots of work to get SMP and now 64 bit to work. Some code assumed they would never exists
  • Virtualisation support soon for server space
  • Main sense of ARM in server space is 10x saving in power eg ~50W vs ~5W
  • Lots of other stuff this guy was going too fast for me to keep up

 

 Helping your Audience learn – Jacinta Richardson

  • Conferences let you vary your level of intensity according to your energy
  • Conferences – no assessment
  • Training is different, all day, 6-8 hours, several days in row, builds on previous days
  • can’t afford to get lost.
  • Cognitive load – how much effort somebody has to apply to learn a new thing.
  • Intrinsic – how hard actually thing is.
  • Extraneous – how harder trainer makes it than it needs to be
  • Germaine – how well concepts build on what we already understand
  • Building framework takes time. Scaffolding has to be well designed. Lots of simply examples
  • Mind map of material -> what you are teaching to build foundation
  • Be realistic what you can fit into day (including breaks, people late)
  • 6 – 6.5 hours optimistic. Seems to be max most people can handle
  • 90 minutes then break ( eg 90min on, 30min off repeat 4 times )
  • Documents – short prose sections, short examples, short chapters (work though in 90 minutes)
  • Key info at start of the day. Dont do lots of extra stuff like class rules
  • Easy stuff, unimportant stuff at the end of the day.
  • Essentials at the start of the course
  • First 90 minutes or first day is most important
  • Options extras at end of course or end of each day
  • If you have lots of stuff -> create another course, make more money
  • Use diagrams, code, pictures, comics
  • reduce germane cognitive load:
  • Order carefully
  • group similar concepts
  • Put import stuff in bold
  • 10:10 – 10 minutes instructions, 10 minutes of student exercise. Sometimes 10:20 . Occasionally 10:30
  • 1-3 concepts in that 10 minutes. But try to balnce
  • Spare time = more examples
  • 90 minutes = 3 x 10:20 + 4.5 x 10:10
  • Target exercises at each key point. Doesn’t have to be real-world
  • 1 point = 1 exercise
  • Easy to advanced exercises. Additional exercises to really advanced people
  • NO answer files. Cause everybody will cheat
  • Minimise cross-chapter reliance
  • Sometimes you have to rely on previous stuff (should have been at start on day one). Try and avoid since people will have missed or not picked up previous concepts
  • New topic = clean slate
  • Good, through course notes
  • Not slides, write a book, should be readable later, good advertising
  • A few other ideas:
  • Keep room cold, keep it fresh. 21-23 degrees
  • Bell curse applies to student ability.  Students not slow, but have less foundation or experience in topics
  • Target average student. Offer extra help for ones behind. Don’t slow down for slowest student.
Share

LCA2012 – Wednesday Morning

A Tour of btrfs – Avi Miller

  • Now
  • All data and metadata is copy-on-write
  • CRC all metadata and data
  • Writable snapshots
  • multi-device support ( raid0 , raid1 , raid 10 )
  • online resize and defrag, online device replace
  • transparent compressions, efficient storage for small files
  • Soon
  • Fixes for perf and stability
  • background scrubbing, LZO compression, batched discard, file defrag options, per-inode flags
  • Larger block sizes for ( especially for metadata, to provide perf improvements in some cases)
  • Scrubbing uses CRC to varify data on disk, fixes bad ones with good copy on another disk which has okay CRC
  • “df ” gives completely wrong values on how full the disk is or it’s size
  • discard/trim supported both real time and batched
  • Drive Swapping – current raid rebuilds via balance code, can also restripe between RAID levels
  • btrs send/receive in development
  • Embedded – friendly to small machines, not as friendly to small disks (being worked on) Works well with low-end flash drives
  • RAID 5/6 – MErge pending completion of fsck work. will also add triple mirroring
  • beta read-only filesystem recovery tool – copies data out of corrupt FS
  • tree root-history log lets us recovery from many hardware errors. “mount -recover”
  • New fsck release on the way. May be announced very soon
  • yum-plugin-fs-snapshot . yum plug to trigger a snapshot on package install/upgrades

 

The Web as an application development platform – Shane Stephens and Mike Lawther

  • When and how to move from native to web/cloud apps
  • Examples
  • Text based like email (pine, mutt, outlook). 15 years ago web-based email started. access anywhere, no install required, easy to start using, use anywhere
  • Desktop publishing. Hard to collaborate with other people using desktop based software by emailing docs around to other people. On the web it’s often native multi-user. Send Links rather than emailing whole doc.
  • On web everybody always running latest version
  • Github gives you one-stop shop for projects. wiki, forums etc. Native in app rather than bolt on to desktop version
  • Graphics
  • Flickr – has native sharing, backups etc
  • Online editing of images etc now possible
  • Games
  • Farmville – no install, easy to share links
  • Angry Birds – has web version, html5 , flash for sound, 60fps
  • Not at stage where First person shooters going to happen yet. Users have high end hardware
  • Overall Benefits – No install, universal access to data, always using latest version, collaboration and sharing built in. Simple text layout, Web as IDE, open and modular enviroment
  • Drawbacks – Layout more involved than desktops apps, distributed code makes debugging hard, cross browser compatibility, security limits flexibility.
  • Useful web technologies (see also HTML5Rocks website) :
  • Display / Rendering
  • HTML , SVG , Canvas – All fairly easy to combine
  • WebGL , flexbox and grid <- future
  • Communications
  • Standard http requests, AJAX / XHR , Websockets / Browser Channel , libraries like Faye etc
  • Storage
  • Traditionally just cookies, Session Storage, Local Storage, indexDb, AppCache
  • Environment Enhancements
  • jQuery , CoffeeScript , NaCl
Share

LCA2012 – Tuesday after Lunch

Mistakes were made by Selena Deckelmann

  • Misc management
  • Prepare for failure, Failure is an option (it will happen)
  • Book: “Everything is Obvious”
  • 2 examples (NZ and Scotland) of rats gnawing through cables and taking out country
  • Document -> Test -> Verify
  • Failure to Document
  • Write Docs, Update Documentation, Make documentation a step with your written processes, assign some time to that step.
  • Doc Tools: Graphic Designers, wikis, sphinx, diagrams  – timelines – bug tracking – ordered todo lists
  • Failure to test
  • Verify success criteria – Write tests – test with buddy – have a plan
  • testing frameworks, staging environment, repeatable shell scripts
  • Failure to verify
  • Have a plan for things going wrong – have staging environment – test rollback plan, not just implementation plan
  • Tools – People, staging environment
  • Failure to imagine
  • share stories of failure – talk to people are different from yourself – act out implementation scenario
  • Failure to Implement
  • reflection ( post-mortem )
  • Plan to do  post-mortem, document the plan with numbered steps and a timeline – test plan & rollback plan – Identify point of no return
  • During – screen sharing – chatroom – Voice – Headsets – Designated time-keeper

Scaling Openstack by  James Blair and Monty Taylor

  • 6 projects in openstack.
  • collection of related repositories
  • Most contributors paid to work on it by their companies
  • number , quality and area or contributors varies
  • 6 monthly releases – design summits – continuously open truck – dev on master – Monthly milestones – stable branches post release
  • Vision – consistent tooling and process on all projects -> Consistent Product -> Multiplier effect.
  • Minimize meta-development, Standard tools
  • Gerrit – code review
  • Jenkins – Testing (pre and post merge)
  • Orchestra (bare metal deployment)
  • Lanchpad, documentation servers, planet, repos
  • Environment: Ubuntu, Everything in Python (pep8 standard, openstack.common ). virtualenv/pip
  • Gated truck – ensure quality – auto tests – means devs always start from working code – keeps bad code out of tree – process same for everybody, transparent, automated.
  • Gerrit – stand-alone patch review system – lots of integration hooks – lots of review categories
  • SSI using openid for all of project sites
  • Git review is implemented as git sub-command to submit things to gerrit. zero-config <- looks cute
  • Vendors can have labs and tests and code can be automatically submitted and tested on it

 

Share

LCA2012 Tuesday – Before Lunch

I’ll be posting some updates from Linux.conf.au in Ballarat in the next few days. I arrived on Sunday but this is my first post. I’ll see if I do an out-of-order blog post on that later.

The keynote from Bruce Perens was really good. He talked a lot about the failure of open source to engage the public and policy makers. He also covered a bunch of cool hardware projects and cheap (and often open) small boards and other electronic products.

Smashing a square peg into a round hole – David Basden and Chris Collins

  • Automation your automation.
  • Anchor – Hosting provider, doing built solutions, non-standard requirements
  • Puppet is one step, PXE & dbootstrap just another, hundreds of others
  • A “simple” build used to take a day, down to 10 minutes
  • instead of defining all steps define dependencies to get a “partial ordering”
  • figure out what has to be done, in what steps, keep track of what has been done
  • Lack of security in many queuing systems, bad agents can grab tasks they aren’t supposed to etc
  • Outline of “Audience” job control system. Design goals and decisions
  • Code on github.net/anchor

 Extracting metrics from logs for realtime trending and alerting by Jamie Wilkinson

  • passive system, query application about it’s metrics. However sometimes hard to parse
  • However most apps log to system or other logs
  • emtail – exporting modular tail
  • plugins – on event X do Y – usually regex
  • Metrics are values, times ( name, value, when, type (counter, gauge) , string, tag
  • metrics are exported over a common protocol ( google protocol in G version, json in open source version )
  • exported over http using json or CSV,  sent to something slse to save and do something with
  • Written in Go. Old version in python but too google specific
  • 20 minutes talking about the source code (I browsed the web during this bit)
  • I and a few others in the audience seemed to think this duplicated a lot of other tools with no obvious huge advantage over them. Bit of google NIH.

 

 

Share