Author: simon

Git For Ages 4 And Up by Michael Schwern

It is not you, is really is complicated

It is easier to understand git from inside out, cause the interface is so far…

Getting started – init and clone

• Entire repo in .git directory after “git init”

Getting stuff done add and commit

• add writes to the repo
• commit creates a commit object and puts labels on it
• take a new copy this file.
• Add creates a new node, commit attaches a commit-object to it and moves the head and master

Branching

• git branch feature – added feature label to current node
• git checkout feature – next commit will move feature label
• git commit – moves the feature label and the head
• head always points to what you have checked out

Commits

• Every one is unique and has unique id
• They never change
• content, author, date

Staging area / Index / Cache

• Place to build up stuff before a commit
• eg what is in “add”

Workflow

• Isolate -> work <-> update -> share
• git checkout master – move back to master branch, can’t see stuff in branch
• get merge feature – merge in the branch “feature”
• git log –graph –decorate all
• reset – “arbitary move labels around command”
• Merge master back into feature
  • git checkout feature
  • git merge master
• Unlike cvs “commit does not meet share”

Working with others

• git clone remote work – clones a remote into work folder
• Normally “remote” will be the URL of the remote repo
• origin/master – label to commit where git thinks remote currently is.
• pull == fetch + merge
• git checkout -b bugfix  – checkout and branch
• git push origin bugfix – push to the branch “bugfix” on the remote “origin”
• git push -u origin bufix – the -u sets the default branch to push to on the remote “origin” for next time

Tags

• git tag v1.0 b08234
• Add any old random tag to a commit, defaults to current commit you are on
• cannot move tags except with force
• can branch from tags

Rebase

• git rebase -i  – interactive mode
• squash
• creates a new commit ( hanging off parent of commit your are rebasing ) and move branch to it, leaves previous branch hanging detached
• Don’t rebase after you have pushed

Bunnie – Linux in the Flesh: Adventures Embedding Linux in Hardware 

As CPU speeds growth has stall mobile CPUs have caught up with Desktop CPUs

Cost of Mobile CPUs $20 on intro vs $X00 on intro for desktop CPUs

Time spend making a product

• 11% on Hardware design – schematic and PCB layout
• 20% on product and software dev
• 33% in mass production ramp

Giving Linux a body

• Consumers don’t like bare boards
• Comprehensive 3D models
• Build injection molds – Steel cut, EDM tidy, Hand polish
• Around $20k to create tool
• People like Apple will start multiple tool runs and keep the best
• General rule – $99 product you want to sell wholesale for $30
• Magic prices products sell at $99, $149, $199
• 90% of hardware sales between Black Friday and Christmas. Can’t ship just after Christmas

Chellenges of Embedding Linux

• Embedded CPUs have lots of weird bugs
• Bootloaders
  • CPU-Specific media loaders
  • Firmware has raw Iron – NO MMU, No DRAM, etc
• Small memory footprint
  • Linux runs just a few MB of memory
  • Every $ counts, lots of time fixing code
• Flash File Systems
  • Few 1000 read/write cycles
  • No write g’teed to succeed
  • Power-down corruption is a fact of life, need backup or read-only copies
• Power Management
  • Reduce frequency and voltage when CPU is lightly loaded
  • Need to adjust for high and low load
  • Rouge processes can chew CPU/Power
  • Lots of auditing needed
  • User apps are hard
• Suspended / Sleep hard
  • Drivers might be written with this in mind
• Huge number of devices that can be plugged in, lots of drivers to verify
• Dev community
  • Smaller pop that desktop
• Updates and patches and rollout and distribution hard to do, must do yourself
• Software is the major bottleneck – cause it is easy, you can do anything
• System architect must create a useful cage for the software dev

Case Study: Robot Vision

• Kovan
• Needs to do image processing
• Lots of choices
  • Using FPGA on PCIe uses up all of memory bandwidth in small systems
  • Connect via USB means it is just a “camera driver”
  • Connect between Soc and the Screen
  • Or between SoC and the DRAM
• Ended up between SoC and LCD
• Ends up as /dev/fpga
• Handle multiple types by probing

Case: NeTV

• Goes between device and screen
• Overlay stuff over video
• But signal is all encrypted
• Sync Pixels and replace one by one using Man-in-the-middle attack on encrypted signal!!!
• Very hard to get timing, drift of 100s of pixels per frame
• Source clock from the incoming signal
• A few other tricks

Dev Enviroment

• Created Amazon AMI to help people get up and going
• But hardware devs not used to using these
• Not a 100% fit to normal dev model

users: delighted; (better UX using CSS 3 in particular and “HTML5” in general) by Adam Harvey

User Experience

• “Used by meatbags sitting in keyboards”
• UI & UX
• ” The only thing that numbs the pain of dealing with IE6 “
• How to put yourself in the user’s shoes? Personas
• Simplify Interface
  •  Three Click rule
  • 4 Clicks to get to most wanted page of sample website
  • Other study says people keep clicking until they get the page they want
• Have to find balancer between what you want and what you can do

Example: PHP website, new version

• No real metrics
• Dropdown menus, hard to decide what to put on the documentation menu, guessed what people might want

People expect web interfaces to be smooth these days

Example @font-face

• Icon fonts
  • Put things that were in sprites into icon-fonts
  • Positives – scalable, bandwidth efficient, text effects are available eg blink
  • Negatives – Lack of Accessibility, No colours
  • Very compatible with browsers
• Alternative SVG
  • Positives – colour, scriptibility
  • Negatives – Accessibility, slower, IE 9+ only, Andriod 3+ only
• Consider sprites, Use icon fonts most of the time, SVG for specific needs

Transitions

• Smoothly make a change in an attribute
• Gotchas
  • Image transition support variable in browsers
  • Still vendor prefix
  • But effect does degrade well
• Good browser support except IE
• Use when you can, especially if you don’t care about IE

Keyframe Animation

• Can do simple animations of objects of CSS

 

Droids that talk: Pairing Codec2 and Android by Joel Stanley

Sophisticated DSP and SDR are within the reach of the LCA attendee skill see

FOSS Speech Codecs

• Post to any platform
• Possibilities

Digital Voice over Radio

• Naarrow bandwidth
• Low bitrate
• Carried by VHF, HF
• Not GSM or VOIP
• 1000 times more efficient than voice over WIFI
• 1/3 of bandwidth used by commercial Am Radio

Software Radio

• Software and be free
• CPU cycles can be nearly free

How Radio works?

• Antenna
• Pre-Amp – boast signal
• Mixer – Takes pre-amp signal  + local oscillator signal
• Another Audio Amplifier

FreeDV on Andriod

• libusb-andriod
  • No lsoc support on Andriod
  • Integrates with Andriod permission framework
• fdmdv2 and codec2
  • Worked as-is from freeDV
• JNI Layer
  • Thread USB callback
• Audio Playback
  • Uses Audiotrack API
• Graphing hard- used GraphView
• Canvas class for drawing scatter plot
• Future – Waterfall plot

Links

• https://blog.jms.id.au

• http://www.rowetel.com/blog/codec2.html

• http://freedv.org

 

The future of non-volatile memory by Matthew Wilcox

NVM Express Standard

• Talking to NVM across the PCIe bus
• Various vendor standards
• Want to have one driver, be able to switch between vendors easily

PCIe Drives

• 2.5 and 3.5 Inc drives
• A common slot for SA / SATA / PCIe
• Support up to 4 lanes at 8Gb/s

Post-NAND Era

• Various technologies in dev to replace NAND
• Ferroelectric, Magnetoresistive, Phase-change, Racetrack
• Most promise DRAM-comparable speeds
• Only last a short time, not days/weeks of persistence with no power
• CPU will treat these the way it treats DRAM – just loads and stores to it directly, no API

Programming Model

• What if you allocate persistent memory?
• malloc() is the wrong model
  • CPU is not persistent, after reset out of sync with memory
• Block device also wrong model
• POSIX provides open(), mmap()
• So maybe a filesystem then?
  • Must bypass page cache
  • May be a simpler filesystem, not fully-featured, no page cache etc
  • Keith Packard has already done this for graphics memory
  • But writing files systems is hard, long time to get them stable

Linux Assumes page cache

• O_DIRECT and splice() are good examples. May be other corner cases

Linux really sucks at sync

• msync(MS_ASYNC) is a no-op
• msync(MS_SYNC) is an inefficient way of calling fsync()
• fsync() synchronises more than we need
• So does fdatasync()
• sync_file_range() doesn’t sync enough

Humans suck at sync

• tdb has contained some horrendous races and missing syncs
• If we expect normal human programmers to get it right we have to come up with something easy

Observability

• CPU A modifies a cacheline and starts syncing it to the persistent memory
• CPU B reads the same cacheline and observes CPU A’s Changes
• Machine loses power. On reboot are we guaranteed that CPU A’s changes are still visable?
• Difference between coherence and persistence
• Journalling is HARD

Reliability

• Current API insulates us
• Memory can become corrupted, as long as we crash before it hits storage, the corruption will never be observed

Improving error logs

• Write directly to logs from inside the kernel
• Snapshot kernel and state for recovery or debugging
• Future machines may not have DRAM at all. What happens if you have  a problem, hard to cold boot
• “Emails to 10TB system state to LKML” – Jeff Waugh
• Cosmic rays corrupting memory while running

 

 

So after a late night hustling people at foosball and swapping rumours about a certain person being ejected from the conference, I managed to leave my key in my room in the rush to get to the opening (although in the end I caught a bus).

The winner of The Rusty Wrench award was Donna Benjamin this year, I liked the way she talked about each of the past winners and got everybody to acknowledge them ( Rusty, Pia, Mary and Kim ).

 Think, Create & Critique Design by Andy Fitzsimon

” I’m a drinker with a speaking problem ”

We are all designers

Fundamentals of design

• Elements and principals of design
• Like cooking – ingredients create flavours influencing a meal
• Elements are the raw tools:
  • Line – continious path between two points, process, plot
  • Shape – When I line joins around to cover an area – shapes used to explan something (pymrid scheme )
  • Space – positive and negative
  • Size
  • Colour
  • Value
  • Texture – structure and feel
• Principles
  • made with elements and with other priciples
  • proportion
  • Pattern – using same element multiple times
  • Graduation – incremental changes to one element over another
  • Balance / Harmony / Unity – One or more elements creating a cohesion
  • Contrast – abrupt difference between elements creating a compostion
  • Emphasis
  • Form – The “whole” that the sum of the parts make
  • Gestalt

Practices

• Visual Design
• The Swiss Won – “International Typographic Style”
  • Typographic
  • Famously minimalistic
  • easy to critique and easier to impliment
  • Baseline grid – can check with a ruler
  • Always follows a vertical rhythm
  • If you have a design you already have a grid
  • A varied scale – Robert Bringhurst
• Art Nouveau
  • Hard to do
• Style tiles and brand guides

Tools

• Patterns
• Wireframe
• Workflow
• Persona
• Analytics
• Instrumentation
• Surveys
• Reviews
• User Testing

Have some common sense

Interactive design

• Progressive disclosure
• Form follows function
• Affordance
• Hyper realism vs skeumorphism
  • Hyper realism – makes things look real
  • skeu – reminds you of something real

Experience design

• Deliberate differences
• think, make, become (take ownership, win with empathy!)

Nail the hierarchy of needs

• Lovable
• meaningful
• pleasurable
• convienient
• predictable
• purposeful
• They are easy to observe but hard to tell
• damned hard to hit them all

Good design is a process

• Design thinking
  • It is a quick workflow
  • define
  • find
  • guess
  • try
  • check
  • do
  • learn

Failing at life is helps you design

Design for hacker is a great book, if you can stomach apple worship and web 2.0

Bunch of other books..

ndftz.com/poster.pdf

 

Vampire Mice: How USB PM impacts you by Sarah Sharp

How USB power Management works

• 3 types of management
  • Device suspend
  • Host suspend
  • Link power management
• Devices suspended when inactive
• When all device on host you can suspend host
• 1 device keeps host awake which keeps CPU awake
• Device must to support suspend (according to spec)
  • But lots don’t
  • Drivers sometimes don’t
  • No USB transfers when suspended, so if userspace polling then can’t suspend

How USB power Mngt does not work

• Drivers missing auto-suspend support
• Impossible to get device to idle
• Userspace polls device
  • Has a SD card been inserted yet?
• USB suspend issues
  • Disconnect on resume
  • Unsafe suspend behaviour ( usb hard drive cut power to spinning disk without parking )
  • No remote wakeups ( mouse only wakes up when button pushed, not just when it is moved )
  • Event loss during resume
  • too risky to enable by default
• Hard to tell if firmware version is good or bad
• All sorts of weird issues with different platforms ( USB Hubs etc )

USB Device Suspend Issues

• Blacklist to big to keep in the kernel
• USB Device suspend off by default
• Can be turn on by user (per device) via powertop
• Powertop setting won’t persist across device unplugs or reboots
  • Solution: Create a udev rule

Takeaway:

• Try using powertop
• Create Udev rule to keep

Challenges with USB device suspend

• Users must turn on
• Require driver modification
• Timeout too course grained
• Devices can’t refuse to allow suspend

USB 3.0 Link Power Management

• Link Power Management states U1 and U2
• Hosts and hubs track idleness
• OS sets timeout once
• No driver modification
• Devices can refuse U1 and U2
• Some vendors don’t like Link PM – can be detected
• Some Hubs don’t support either
• USB 2.1
  • New L1 state
  • No changes to USB 2.0 hubs

New Intel stuff

• Panther Point vs Lynx Point chipsets
• Panther Point
  • Has xHCI controller
  • Only 4 points under xHCI
  • Supports USB 3.0 Link PM
• Lynx Point
  • All Ports
  • Supports 3.0 PM and 2.1 PM
  • Completely unused ports can be turned off completely

Summary

• Lot of broken devices
• But quiet a few that do work
• Try powertop
• github.com/sarahsharp/usb-pm-tools
• Link PM improvements
• New Intel USB PM improvements

Actual saving is Probably more than you think, especially if you get the whole chain to sleep.

Somebody said “about 4 watts” for SandyBridge

Servers can also save. Options in HP G7 servers. But problems

 

Getting your talk accepted: write a convincing talk proposal – Jacinta Richardson

Background

• On a lot of papers committees
• LCA, SAGE-AU, OSDC

Pick Conference you want to speak at

• Some easier to get into than others
• SAGE-AU 50%
• OSDC 50%
• YAPC easy
• OS Bridge – harder
• OSCON – 30-50% chance
• everywhere else – medium
• LCA – really hard. 5x the proposals received than accepted

Speaker rewards

• Free entry

Call for Proposals

• Not always widely distributed
• Join mailing lists, watch websites, ask

Write Abstract

• The hard bit
• Some confs narrow or wide on talk topics
• Audience 1 – The programme Comittee
  • Doesn’t know if you are a good speaker
  • Look for link to video of you speaking
  • If no video then assume if writing bad/good then speaking similar
  • Check spelling and writing style
  • Tell a story but not too long
  • Not academia , avoid insane amounts of jargon
  • Paragraphs good. Might only read the first
  • “read first sentence of each paragraph”
• Audience 2 – The attendees
  • Why your talk?
  • Against other options
  • Good title
  • Skip over – “X for fun and profit” , “making X sexy” , “What I did on my X holidays”
  • 5 or fewer words for title
  • Convincing first paragraph or even the first sentence

Ask for help

• From usergroups
• Or people you have met at LCA
• people on the papers comittee

Enabling Compute Clusters atop OpenStack – Enis Afgan

cloudman – usecloudman.org

• People want a ready to use service, something they can just sit down and start using
• Bridge between Saas and IaaS
• Allows somebody to create a pre-configured compute cluster

Deploy

• Start with Cloud account
• Start master instance
• Use Cloudman web interface
• Multiple types of clusters availabile

Galaxy Cluster

• Used for Genomic Science
• Web based platform

Value Added features

• Customise your instance, add tools, add image, snapshot images, share images
• Auto scaling
• Flexible architecture ( openstack, Amazon , etc)

Open Programming Lightning Talks

Adam Harvey

• Not all are sites are facebook
• Big frameworks are overkill for some people
• Microframework – “Under 1000 lines of code”
• Silex
  • autoloads in lots of extra code
  • 33,086 lines of php code being pulled in
  • Not very micro 🙁
• Slim
  • Autoload – 6000 lines
• Flight
  • Autoload – 800 lines of code
•  Maybe just use raw PHP instead of a framework

Paul L – The Poor Man’s SANbox

• Allow people to enter python code into program
• Way to stop them doing bad stuff was over my head

Dave Boucher – Yak Shaving

• Transactional memory – red/black tree insertion
• Graphically show how RB tree inserts something
• Use SVG library in python
• SVG has animations
• Pretty!

Tom Sutton – Safe Strings in Haskell APIs

• Turn on OverloadedStrings
• Create customer datatype
• Can put special string types that don’t do things like concatenation if you don’t want (eg for a special type with SQL commands)
• andhetalkedsofastatheendIcouldn’tundertsandhissolution

Roger Barnes – poker, packets, pipes, and python

• Wanted a poker buddy
• packet caputure between andriod app and server while playing online poker
• ngrep
• Hack your router to get Linux on it
• Grab stream of info – all plaintext!
• ipython notebook
• parsing game, map card values
• Need live capture data
• Solution: ssh + ngrep + pipes
• watch out for buffering
• grab poker value and hints into lookups tables

Benner Leslie

• Python and Haskell
• Embed Haskell code into python
• Wanted to keep writing python most of the time and only use Haskell where it was needed
• Combine using foreign C-types

Nico – LatProc and Clockwork

• Libary for tools process control
• Controls machine that gets wool samples from bails of wool
• latproc on github

Duncan Rowe – Some commands I’ve developed over the years

• pd – keeps recent dirs in stack , allows you to skip to them
• sfl – searchs for strings in multiple paths
• bak – backup a file, just renames to filename.bak , various other options

Russell Stuart – PAMPython

• PAM in python
• PAM modules normally require C
• Can do various PAM functions in python
• Good for one-shot commands
• Sneaks in under all the programs that depend on PAM

Peter Chubb – When Arduino is not enough

• Stellaris launchpad just $12
• RaspberryPi $35
• Odriod U: quad core 2G RAM, 3W – PC like performance $69

 

Open Govt Miniconf – Open data panel

• Cassie Findlay – State records – NSW
  • Mostly hard copies records but digital archives initiative
  • Started Open Data Project, making metadate as open data
  • API to catalogue
  • OpenGov NSW website
    • Mostly Annual Reports from Govt Agencies
    • Place for other docs to be released in future
    • Copies of last 20 years of Govt Gazettes
    • New API
• Christen Normal – ACT Govt
  • Even when govt data available, often in hard to use formats
  • Needs to be more in line (format wise) to what community expects. APIs not PDFs
  • Variety of reasons why people want data, journalists, public, many with short time
  • New website, web servers API, download data
  • Technology not problem, people just have different attitudes
  • Expensive, not my job, people will find something was done wrong
• John Billia – Aus Govt ICT management Office
  • Covers emerging technologies
  • Looking at Big Data over last 6 months
  • If used effectively will led to better govt
  • Concerns to pricacy, wishes of citizens as to if/who their data shared
  • Problems with governments of projects, better data might improve outcomes
  • Example was open source policy a few years ago
    • Checks every govt tender to make sure complies with Open Source policy
  • Also managing Whole of Govt transition to IPv6
    • Just under half Govt Agencies have already enabled IPv6 on external facing websites etc
    • More that half of rest will be up by end of Q1 2013 and rest by Q2 or Q3 2013
• Julian Carver – Christchurch Earthquake recovery agency
  • NZ Declaration on Open and Transparent data
    • Active programme of release of data
  • Example “Charities register” , data via API
  • Example “ASB Property Guide” – brings up property data
  • Example “Info Connect” , transport data, used by 12 apps, eg traffic delays, looking at traffic flows to predict economic data
  • Best way to happen was to built it ourselves and embarrass govt
  • Ask people what they want released
  • Compulsory for agencies to release data
  • data.govt.nz ahead of Aus Central and State Govt totals

 

Today I thought I’d go between miniconfs a little looking for talks I’m interested in. I was originally going to come mostly to openstack but I some of the talks seem a little specialised and there are some good talks elsewhere. Hopefully everybody will to close to schedule so switching will be easy.

Introduction to OpenStack – Joshua McKenty

Worked at NASA

Components increasing a rapid rate but most computer, networking and storage

Almost all commiters are paid to work on openstack

Quantum/Networking: Lots of paid plugins alongside free options

Generally a bit of an intro to the openstack organisation rather than the technology that I was expecting.

The WebKit Browser Engine – An Overview – Dirk Schulze

What is webkit?

• Just a browser engine
• Used in Safari, Chrome and others
• High market share on mobile, lower penetration on desktop

Components

• Webcore – triggering load pages, actually drawing, calculating layout,
• Javascript engine (alternative used in Chrome is V8)
• “Webkit” – platform dependent stuff, access graphic libraries (gtk, qt)

Webcore

• html document + CSS + Javascript
• Parse HTML docs, get dom elements, create a DOM Tree
• Everything in DOM tree can be accessed from javascript
• Render Tree just has stuff needed to render the page
• Eg <head> element doesn’t get into render tree but are in dom tree, same with “display: none” elements
• Some elements are in render tree but not Dom tree (eg anonymous blocks)
• Renderlayer – render elements above/below other elements ( eg using – style=”z-index: 1;” ), stacking context ( also via opacity, filter or mask )

Render Object

• Layout – dimension of the element ( height, width, plus borders, positioning )
  • So if updated only need to repaint affected area
• Paint – Draw element on screen
• Multiple paint phases called from RenderLayer
  • Background, borders
  • floating content
  • inline content
  • Based on CSS boxing model, code follows spec
• Hit testing
  • Pointer events – homer, onmouseover
• SVG – different from other renderers
  • One element – one renderer
  • No CSS boxing model, No anonymous blocks, different handling on transformations

Implementing new elements and Interfaces

• Look at the specification

 

Tuesdays Keynote was by Radia Perlman of Network Protocol Folklore

General protocol about protocol design

• Need more people in the field that hate computers
• Autoconfiguration
• Knobs if you want them
• Be evolutionary if possible

Networking is taught as if TCP/IP arrived from the sky. As if nothing else ever existed

She teaches by looking at a problem and looking at how different protocols solve it

Comparing technologies

• Nobody knows both of them
• Everybody is partisan
• Both moving targets
• Hard to compare via benchmarks since people are just comparing implementations rather than actual technology

The Story of Ethernet

• ISO 7 layer model
• Ethernet was intended to be layer 2, neighbour to neighbour, was are packets forwarded
• Ethernet physical was a new type of link, multiple nodes on single link
• But we haven’t done CSMA/CD networks for years
• No hopcount field in Ethernet since never occurred to designers that people would be forwarded the packets
• People started building networks layer-2 only without layer-3
• Needed to forwarded Ethernet between networks, but had to work with existing ethernet packets -> Bridge
• Spanning tree reduced created loop-free subset of the topology

Why is wrong with IP as L3 protcol

• Every link must have own address block
• Configuration intensive
• In 1992 Internet could have adopted CLNP but NIH
• Also advantages not obvious then since things like DHCP, NAT so advantages of CLNP not as obvious

TRILL

• Switches run routing protocol between themselves
• Replaces spanning tree (switch by switch basis)
• Wraps ethernet packets in trill headers, forwards to other trill switch and then unwraps
• Various ways to link which end devices are behind which trill switch
• Link state routing between trill switches to create shortest paths
• Can upgrade switches to trill one by one an “just starts working better”
• Anything can do the trill encapsulate/decapsulate

Similar to TRILL

• VXLAND / NVGRE
• Wrap IP rather than ethernet

Protocol Forklore

• Version number
  • What is the purpose?
  • What is the new protocol vs the old protocol?
  • Envelope says how to parse the header (how to parse the packet)
  • Need to define what node does when it sees a different version number
• Parameters
  • minimise these
• Latency
  • cut-through – forward before you have received the whole thing
  • Destination should be near the start of the header
  • tcp has checksum so need to see the whole header before you forward