Linux.conf.au 2014 – Day 3 – Session 1

Systems Administration in the Open by Elizabeth Krumbach Joseph

  • Works for HP , paid to work on Openstack project
  • Normally infrasture is maintained by a team
    • You interact with them via tickets and email
    • Priority determined by the team
  • Infrastructure team at open stack
    • CI system
    • Wiki, website, IRC bots
  • Everything is in a public git repo
    • Puppet modules
    • git.openstack.org
    • Anyone can submit patches
  • Openstack code review and CI chelleganes
    • Lots of individual projects (infrastructure is just another one)
    • Syntax checks, testing automated, changes should never break the master
  • Using
    • launchpad
    • git
    • gerrit
    • zuul
    • gearman
    • jenkins
    • nodepool
  • Probably don’t need that much at most places, but already used by other projects
  • Anyone on the Internet can look at our changes and/or do code reviews
  • checks:
    • puppet pareser validate
    • puppet-lint
    • XML
    • Alphabetized project files
  • Peer review
    • Multiple eyes on changes prior to merging
    • Good infrastructure for developing new solutions (particularly for distributed teams)
    • Trains us to be collaborative by default
    • No special process to go through commit access
  • Changes get checked in
    • Either puppet master gets updated and applies change
    • Or vcsrepo module in puppet pulls in latest version of the project
  • Can you really manage via git commits
    • Cacti to keep eye on server usage
    • Puppet dashboards so you can watch your changes get applie(or not)
    • Thorough, specific documentation at http://ci.openstack.org
  • Sometimes you need to login to a server
    • More difficult for complicated migrations, upgrades
    • Passwords need to be more privately managed
    • Other stuff kept in hiera and maintained out of band
  • Conclusion
    • You have the tools you need to figure out and patch the infrastructure
    • Priority is largely determined by the patch submitter
    • No need to wait for the infrastructure team to figure out and write your change

 

Linux Filesystems: Where did they come from? by Dave Chinner

  • Motivation
    • Based on previous study that reviewed commits on file systems from git tree
    • Focus primary on bug fixes
    • Ignored feature work, didn’t look at the people
    • Only considered 2.6.x kernels
    • Listed Patch type, type of bug fixed
    • Only listed by number of commits, not the number of lines
    • Listed bugs fixed per release
    • But didn’t list “why” some of these things happened
  • Fire System Technologies
    • 1970s
      • Record based
      • small file name sizes
      • File control blocks rather than inodes
      • No Hierarchical namespaces – multiple “user regions” to segregate files
    • early 1980s
      • Sector based – complex Cylinder/head/sector mappings and optimisations
      • inode tables
      • bitmaps for free space
      • Resource forks
      • Hierarchical directory structure
        • simple and limited in size and depths
    • late 1980s
      • Extents and btrees in research
      • journalling first used
      • full 32 bit address space in use
      • Cylinder roup seek optimisation lead to multiple inode/data groups per file system
    • Early 1990s
      • maximising IO, > 4GB file systems. minimising seeks in reasearch.
      • RAID
      • log structured filesystems and copy-on-write
      • 5-10 year gap from research to production
    • late 1990s
      • soft updates to replace journaling
      • data transformations (compression, encryption)
      • wandering logs
      • async journalling
    • Current tech 00s
      • transperant error correction
      • direct device management (raid itself)
      • reference counted copy-on-write btrees
      • log structured merge trees
  • Linux file Systems
    • Minux FS – 16 bit, 64MB
    • 1992 ext File System – 2GB MAx, 255 char file names
    • 1993 – ext2 – 4TB max size, a/c/mtime support , group based bitmap allocator, extensible
    • 1998 – journaling proposed for ext2
    • 1999 – IBM release JFS under GPL
    • 2000 – SGI release XFS under the GPL
    • 2001 – Ext3 merged, reiser3 merged, JFFS2 merged, First JFS and XFS releases
    • 2002 – JFS and XFS merged
    • 2004 – Reiser4 released
    • 2005 – NILFS released
    • 2006 – ext4 first proposed, created
    • 2007 – BTRFS concenved
    • 2008 – ext4 declared stale, tux3 design first published
    • 2009 – BTRFS merged, NILFS2 merged
    • 2010 – LogFS merged in 2.6.34
    • 2013 – F2FS merged in 3.8
  • Linux history in git trees
    • Complete from 2.4.0
    • Older releases mostly intact
    • commit date issues, time based search fail
  • XFS History in a git tree
    • Complete from initial commit in 1993
    • Older commits mostly intact, some issues
  • Looking at extN / XFS / btrfs
    • When – diffstat
    • what – difss and commit messages
    • who – commits, mailing lists
  • EXT file system
    • 2500 lines added in 1991 to get it working
    • Removed in 2.1.21
  • ext2
    • 1994 – first commit
    • 1998 – Steven Tweedy did lots of work
    • 2002 – various stuff done
    • 2003 – extended attributes originally from XFS added by XFS team
    • 2008 – reservation based block allocation backported from ext3
  • ext3
    • 2002 – created
    • 2003 – journalling added , extended attributes
    • nothing much since then except maintenance
  • ext4
    • 2007- create
    • features steadily being added 2007 till now
  • btrfs
    • 2x the amount of code than ext4
    • Steady add of features since 2008
    • Stuff being added not removed
    • Sync’d to Linux merge and fix windows every 3 months
  • xfs
    • 1995 release
    • 2000 removed a tonne of stuff due to licenses pre-GPL (and it still worked)
    • Several other code removals
    • Code removals often mechanism being replaced by something better
  • Compare
    • xfs is as old as all others and has had more work done on it most years except recently with btrfs
    • btrfs due to overtake xfs as largest filesystem in 2014

 

Share