Category: Tech

Smoke Signals – Mark Pesce

• 1 billion seconds since learned to code ( 1979 )
• Programmed for wide range of platforms
• Inspired to “Neuromancer” to get into VR, Setup Ono Sendai to create VR game console
• Companies failed cause they wanted whole market, locked down with patents etc
• Tried more open method, created 3D browser for VRML over www. 17 years ago
• “a resource shared is a resource squared”
• Tried to give away as much as possible, the richer I become, more ways than financially
• smoking habit picked up from our friends, spreads like a epidemic
• Similar affect for overweight people
• Similar for divorce
• What we choose to imitate is determined by the network of our peers
• Choose your friends carefully
• Digital social network can be bigger than 150 limit. Average facebook 30 though
• Facebook itself is part of your social graph
• facebook knows stuff about you that you would never tell any one friend
• We surrendered all our personal details to a closed source system
• wikileaks kicked off various commercial service, coincidental terms of service violations presumably under govt pressure
• “The Internet is fundamentally not free”
• The internetv is now hostile since it is controlled by govts etc
• Distribute everything – lesson of Napster, no central target, break up components, whole greater than parts, redundant parts, extension of unix progs, pipes, widely distributed
• Transport Independence – cellphone networks central org able to be controlled by govt, uucp and fidonet decentralized communication networks,. hierarchy efficient but vulnerable. Must look at alternatives, speed and convince might have to be sacrificed
• Secure everything – Security for simplicity is a false tradeoff. Personal data is valuable
• Open everything
• Resilience – above principles led to this. may be less efficient.
• Resilience , safe, fast – pick 2
• Plexus – Mange you own social graph
• Number of people with mobiles = Number of people earning more than $1.50 per day

Building RPMS – How Fedora’s Koji Works, and how you can use it to build your own software – Dennis Gilmore

• Koji – task scheduler, results tracker, used by fedora and redhat as a build system
• Not – no software release life cycle, not deployment, repo mngt, continuous integration
• Goals – build repoducability
• Components – mock, yum, rpmbuild, createrepo
• mock – creates a chroot for each build, executes commands, out->results location, ensures output is controlled and repeatable, builds on yum
• yum – reads software metadata, installs rpm packages
• rpmbuild – converts bare components into binary package , spec file + tarball + patch files – scrpm and binary rpm
• various koji components
• kojihub – xml-rpc – passive application – controls wring to fs – talks to DB
• kojira – manages repo
• kojid – runs mock to build rpms, createrepo to general repos, polls hub for new tasks
• koji-web view status of things, can sign up for notifications, cancel and resubmit tasks
• koji cli – users, tasks, permissions
• Other tools can use koji
• Other tasks  – live CDs – Appliance creation – Maven builds – windows builds(!)
• koji org – tags ( pack name and option additional arch) – targets ( build tag and destination ) – unique nvra ( name version release arch) for all rpms
• Live demo
• mash tool to get things out
• extend via koji-api

Changing the X server development process – Keith Packard

• Old Old model – Everybody commits to master, at some point only do “necessary patches” , release manger
• cuts release at some point – didn’t scales to lots of devs
• Old model – everybody to master – release branch made – additions by editing wiki page adding ids –
• release manger cherry picks
• lack of testing of release branch, a bit of overhead, 4-5 releases
• Idea to copy Linux kernel
• New Model –
• only one person commits to master
• Some people publish own trees
• Some people send patches to xorg-devel, easy entry
• All patches need to be reviewed before being merged
• Not quite the Linux model – only general agreement needed to merge – release management more
• mechanical, just checking patches have review, merge cleanly, testing after each patch

• Why Change?
• bad at hitting scheduled dates
• git master frequently unstable, sometimes not even build-able
• little discussion about even major changes
• limited release testing, largely caused by instability of master

• 6 month release period, aligned with fedora, ubuntu, meego
• last 2 releases 4 days slip and 0 day slip

• Context 1-5 – 1.7 very active dev period, 100,000s LOC removed
• all commits are supposed to be reviewed
• not to worried if reviewer comes from same company as long as they are credible and know that area.
• contributors 50-100 per release, dropping a bit in 1.8 and 1.9
• Big jump is patch related threads and other threads on mailing lists?

• Success?
• dramatic reduction in LOC changes
• Dramatic increases in communication
• Hitting releases on time
• Meet original objectives

• lack of useful test suite
• testing to just make sure basic build

Noirin Shirley – Baby steps in Open source development at apache

• 90 projects, starting with every letter but “y”
• various types of projects
• No corp members or contributers
• Decisions only on mailing list
• From inside – national building
• Not insistent on using specific technology
• How to do open source the apache way
• need to cooperate, follow directions, communicate,
• reading docs, look at existing coding style, read list for a while
• Mind you manners, careful of mis-understandings
• communicate clearly – email guidelines
• apache incubator – projects first entry, existing codebase, time to get up and running, legal and other
• formal procedures
• play well with others, consensus on change to code etc, can’t veto a release (just 3 members needed)
• Google summer of code.
• Apache mentoring program
• Accepts none pure “code” projects like documentation, testing
• Working with formal education, two classes of students being mentored
• Meetups encouraged, works well if you meet people in person

Use the force Linus – Jon Oxer

• Usability, computing falacies
• The best UI is no UI
• Things should just work, you should have to do anything.
• eg your door should just unlock for you, not for others
• eg your TV should just play what you are watching, stop when you move away
• Story about reverse engineering of Kinect scanner for linux
• Demo of controlling helicopter via gesture
• Demo of controlling curtains

Kernel Development – How things go wrong – Jonathan Corbet

• Kernel dev a success – ~4.5 releases/year
• Linux soon to your toothbrush
• Why talk about failure? Gives kernel process a bad name, scares people off, bad press, failure can teach us
• Note: The kernel community does not lack for clowns – won’t be covering them, names named here are people he respects
• Example 1 – tux3 file systems by Daniel Phillips –
• 2008/07 announced , 2008/11 booting , 2009/08 last commit – project now dead
• Do NOT add stuff to out-of-tree project, merge early
• Lesson out of tree code – for users, few contributers, invisible, have to keep sync’d with mainline, harder work
• Lesson – get in mainline early
• Example 2- em28xx video4linux driver
• 2005/11 merged , 2008/01 last patch by ori author, 2009/08 author leaves completely
• “People submitting code have to be aware they will lose control”
• eg 2004/05 Hans Reiser tries to block updates to reiser3 fs , wanted to keep control and concentrate on reiser3
• maintainership does NOT mean ownsership
• Example 3 – 2.5.x IDE
• 2002/02 Martin Dalecki’s IDE cleanups , 2002/03 IDE18 subsystem takeover , 2002/08 IDE115 merged, 2002/08 Martin’s quits all IDE work reverted
• What happened? – “Breakage is the price you have to pay for advancements” – martin Daleski .
• Kernel intolerant towards regressions, listen to people who complain
• Example 4 – Deadline Scheduler
• 2007/03/ first post, 2 days later Linux likes, 2 weeks later Linus get irritated (due to regressions), 2007/04 Molnar posts CFS, 2007/07 CFS merged, 2007/07 Con leaves
• Con leaves unhappily
• Lessons – Improve for everybody, not just a small set, at least don’t make it worse
• Lessons – Some parts of the kernel are hard to change
• Lessons – Participate in the wider kernel mailing list discussion, not just list that is for your changes, subsystem
• Lessons: Aim for the solution to the problem, not for the inclusion of specific code
• Example 4 – reiser4
• 2002/10 First code post, 2003/07 merge request, 2004/08 added to mm, 2005/09 push 2.5.14 , 2006/07 push 2.6.19 , 2006/10 arrested
• Problems – Non-POSIX behavior , numerous technical difficulties , hard-to-reproduce benchmarks, antagonistic approach to others, memories of reiser3
• lessons – Linux is not a research system
• Lesson – visionary brilliance will not excuse pore implementation
• lessons – it’s best not to access others of conspiring against you
• lessons – community remembers the past and thinks far into the future
• Example 5 – systemtap
• 2003/11 Dtrace debuts, 2005/10 rhel4 introduces systemtap, 2008/07 Ftrace merged, 2009/06 perfevents merged, 2009-09 systemtap 1.0 released, Not yet merged possibly never
• 2008 kernel summit, 50% had used systemTap, 20% had succeeded
• bad sign if even kernel devs can’t make it work
• lesson- if kernel dev community doesn’t see value, it won’t go in
• Example 6 – Talpa – provide hooks for anti-virus
• posted in Aug 2008, never merged as such
• Problems – Kernel devs didn’t like, why broken security model, badly expressed requirements
• But – fanotify is accepted, same authors, similar code
• What changed – cleaned up file event notification (replace inotify and dnotify)
• What changed – Enable virus scanners to hook into file ops without using rootkit techniques
• lesson – sell to developers not to mangers or customers
• lesson – user-space API matters, hard to change later
• Example 6-15 posted
• Why bother? – “Why not hack on CMSses or something else where the standards are a little lower”
• Fun!
• Elite club
• you will get job offers
• Influence – how the kernel meets your needs
• VFS filesystem patches by Nick Hagen(sp) – worked – intrusive and tricky code – good code – listened to people – didn’t behavior for people, all behind the scen

Sendmail revisited – Eric Allman

• sendmail more than 30y old, survived well, changed the world
• Start 1980 at UC Berkeley, no support, before Internet existed, built as matter of nessesity
• Then: CPUs 16bit, <1MIPS  . Disks <<1GB ,  Memory < 1MB  , Network <56k/s  ,  DBs research only. Phones huge and uncommon
• Today: CPUs 64bit, >50k MIPS , Disks 2TB , Memory >100 GB , Network Megabit to ~160GB , NOSQL hot, RDMBS boring
• 9.6k link to arpa link, only 2 ports for everybody to share, $5k for 1 tty port
• Observed people just wanted email. Solution forward messages between networks
• design principles:Only 1 programmer, don’t redesign user agents, don’t change local mail store, make delivermail adapt to the world
• Problems: compiled in config, no address translate,
• Transittion to sendmail – UCB awarded contract , adding SMTP support, needed to be build in, forced introduction of queue, queues harder than look
• released 1982 with 4.1a BSD – SMTP, Queueing, runtime config
• sendmail picked up my most unix vendors. Unix wars raged on – vendors added value and incompatible added up.
• Returned to Berkley, sendmail rewrite -> sendmail8
• Added in extensions developed elsewhere. Revision of SMTP, DSNsm other systems, 8bit mail, new configuration package
• sendmail bat book dramatic increased uptake of sendmail8
• Commercial sendmail – Sendmail Inc in 1998. 1st commercial open source/commercial hybrid companies
• Added by commercial company – encryption & auth, milter, virtual hosting, LDAP, lots more checking
• Part 2 Lessons Learned
• Changes in requirements over time:
  
• Reliability – deliver or bounce
• Functionality and performance
• Then became about protection
• Next legal/regulatory issues
• Increasingly controlling costs
• Design decisions and evolution
• Overly general: can do anything, reality world was in flux, easier to build tool than solution, still true today to extent, retrospect would do again
• rewrite rules: good idea, using tabs as active character was wrong. concept right but syntax and control flow could have been better
• Message munging – essential for interoperability at the time, still used – to you pass through transh, reject or fix
• Syntax of config – flat (no nesting) , to much used of single characters, ugly but not fundimentally flawed, would use something like apache todday
• smtp and queuing embedded in sendmail – good idea
• structure of queue – two files per message, requires lots of scans, ascii simplified debugging, correct for time, wouldn’t do now
• Using m4 for config – painful syntax, damn “dnl” ‘s aren’t even necessary. Retrospec some tool was needed unclear if m4 was the right one.
• Extending vs changing features. Example masquerading. Wrong first time, reimplemented but added new feature instead of fixing. Didn’t want to break existing sites. retrospec wrong and big part of why sendmail hard to configure, should have provided upgrade tools
• Accepting and fixing bogus input. permits broken software to exist. right at the time unclear if it’s the right thing today.
• What do diff today –
• Fix problems asap, v7 mailbox format, tabs in config files, bogus features
• use modern tools
• more privilege separation
• create string abstraction
• create string separation
• separate mailbox names from unix user ids
• cleaner config file
• What would I do the same –
• use C as the implementation language
• Bit things off in smallish chunks
• syslog
• rewriting rules (without tab char)
• Net rely too heavily on outside tools
• Takeaways –
• KISS actually works ( sendmail was a “2nd System” )
• If you don’t know what you are doing advance designs don’t help much
• The world is a messy place
• Flexibility trumps performance
• Fix things early – if you succeed installed base only gets bigger
• ASCII is great for internal files and protocols
• documentation is key to broad acceptance
• the design space is always changing
• Q: if you were not Eric and you were starting today which mail server would you use? A: “postfix”

Open source Documentation –  Lana Brindley

• User Interface + documentation = can get into program
• People don’t think where manuals come from
• Like a black box
• At Redhat use 5 phase waterfall model
• Phase 1 – Information plan – work out dates, audience, sources
• Tools – wiki and ticketing system
• Phase 2 – Decide what goes in – look like, title, abstract, where info, ID Subject matter expert. Google, wikipedia
• Tools – wiki , bug system , master tracker bug
• Phase 3 – create 1st draft of document – reviewed by legal, writers, technical review
• bugs raise against doc, fixed, Quantity Engineering
• Tools – publican, draft to svn,
• Phase 4 – translation and publication
• Up to 26 languages
• Don’t publish dead-tree books, single page html, pdf, paged html, epub
• Phase 5 – Review on process
• Geographically spread out, , phone meetings
• Advice:
• Keep *some* documentation, blogs, wikis, comment code, that tech writers can use
• Get organised
• Choose your voice
• Say what you mean
• Use Short sentences. 15-20 words on paper, 10-15 on a screen
• You’re not at University – there is not an exact word limit
• edit with knife
• each editing pass should make your document 20% shorter
• read what you write, walk away from it so you read what it says not what you think it says. Read it aloud. Read it backwards (v short docs), or each sentence backwards.

The latest and coolest with HTML5 video – Silvia Pfeiffer

• htttp://caniuse.com
• Video Manipulation
• Not just a 2 new tags, combining technologies
• htmlvideoguide.net – updates from Silvia’s book
• new websites should probably user mp4 and webm
• CSS3 transitions
• Javascript API and Media
• SVG – not uniform implementation across the browsers
• SVG filters for video only in FF4 for now
• Canvas better implemented across browsers
• Very interesting demos
• face detection in 5 lines of a web worker
• shared video and audio playing
• Demo of implementing audio visualisation of audio samples in realtime directly in htnl
• Generate music directly ( or via samples) . Part of proposed audio API
• Media accessibility – captions, subtitles, audio descriptions, external text file or in-band
• WebVTT, <track> element , TextTrack API

Linux Graphics Directions – Keith Packard

• Room was so full Linux almost got kicked out
• Linux runs everywhere.
• X runs on desktops, laptops but only on some of the mobile devices
• Open source community not engaged in any graphics that is not X
• Dominates desktop market mainly because GTK and QT use them
• Otherwise nobody runs X
• need to engage these people
• X ended up doing just about all graphics tasks
• Why X? – Lots of mature code for mode setting, have to duplicate
• Why X? – Clipping
• Why X? – Video Memory management
• Window Manager + X Server + compositing manager are all separate, more complicated
• How have things changed? – shared libraries, huge data increase, http/html does remote access, theme-able UI’s , making things fast-enough is easier, screens have colour now
• Re-Integrating the Window System
• Simplify Architecture – integrate compositing, in-app window management, synchronous operations
• What about remote apps? – HTTP, VNC/Rdesktop, WiDi, X as peer application
• Providing broad system architecture support
• mode setting – KMS
• Input device – Evdev
• Rendering/memory management – GEM
• Missing pieces – key mapping – libxkbcommon creates sharable interface
• Missing – Input devices – evdev isn’t sufficient – need user-mode touch devices
• NMissing – Accessibility APIs – mouse keys, sticky keys
• Reducing X specific code apps have to depend on