Systems Administration in the Open by Elizabeth Krumbach Joseph
- Works for HP , paid to work on Openstack project
- Normally infrasture is maintained by a team
- You interact with them via tickets and email
- Priority determined by the team
- Infrastructure team at open stack
- CI system
- Wiki, website, IRC bots
- Everything is in a public git repo
- Puppet modules
- git.openstack.org
- Anyone can submit patches
- Openstack code review and CI chelleganes
- Lots of individual projects (infrastructure is just another one)
- Syntax checks, testing automated, changes should never break the master
- Using
- launchpad
- git
- gerrit
- zuul
- gearman
- jenkins
- nodepool
- Probably don’t need that much at most places, but already used by other projects
- Anyone on the Internet can look at our changes and/or do code reviews
- checks:
- puppet pareser validate
- puppet-lint
- XML
- Alphabetized project files
- Peer review
- Multiple eyes on changes prior to merging
- Good infrastructure for developing new solutions (particularly for distributed teams)
- Trains us to be collaborative by default
- No special process to go through commit access
- Changes get checked in
- Either puppet master gets updated and applies change
- Or vcsrepo module in puppet pulls in latest version of the project
- Can you really manage via git commits
- Cacti to keep eye on server usage
- Puppet dashboards so you can watch your changes get applie(or not)
- Thorough, specific documentation at http://ci.openstack.org
- Sometimes you need to login to a server
- More difficult for complicated migrations, upgrades
- Passwords need to be more privately managed
- Other stuff kept in hiera and maintained out of band
- Conclusion
- You have the tools you need to figure out and patch the infrastructure
- Priority is largely determined by the patch submitter
- No need to wait for the infrastructure team to figure out and write your change
Linux Filesystems: Where did they come from? by Dave Chinner
- Motivation
- Based on previous study that reviewed commits on file systems from git tree
- Focus primary on bug fixes
- Ignored feature work, didn’t look at the people
- Only considered 2.6.x kernels
- Listed Patch type, type of bug fixed
- Only listed by number of commits, not the number of lines
- Listed bugs fixed per release
- But didn’t list “why” some of these things happened
- Fire System Technologies
- 1970s
- Record based
- small file name sizes
- File control blocks rather than inodes
- No Hierarchical namespaces – multiple “user regions” to segregate files
- early 1980s
- Sector based – complex Cylinder/head/sector mappings and optimisations
- inode tables
- bitmaps for free space
- Resource forks
- Hierarchical directory structure
- simple and limited in size and depths
- late 1980s
- Extents and btrees in research
- journalling first used
- full 32 bit address space in use
- Cylinder roup seek optimisation lead to multiple inode/data groups per file system
- Early 1990s
- maximising IO, > 4GB file systems. minimising seeks in reasearch.
- RAID
- log structured filesystems and copy-on-write
- 5-10 year gap from research to production
- late 1990s
- soft updates to replace journaling
- data transformations (compression, encryption)
- wandering logs
- async journalling
- Current tech 00s
- transperant error correction
- direct device management (raid itself)
- reference counted copy-on-write btrees
- log structured merge trees
- 1970s
- Linux file Systems
- Minux FS – 16 bit, 64MB
- 1992 ext File System – 2GB MAx, 255 char file names
- 1993 – ext2 – 4TB max size, a/c/mtime support , group based bitmap allocator, extensible
- 1998 – journaling proposed for ext2
- 1999 – IBM release JFS under GPL
- 2000 – SGI release XFS under the GPL
- 2001 – Ext3 merged, reiser3 merged, JFFS2 merged, First JFS and XFS releases
- 2002 – JFS and XFS merged
- 2004 – Reiser4 released
- 2005 – NILFS released
- 2006 – ext4 first proposed, created
- 2007 – BTRFS concenved
- 2008 – ext4 declared stale, tux3 design first published
- 2009 – BTRFS merged, NILFS2 merged
- 2010 – LogFS merged in 2.6.34
- 2013 – F2FS merged in 3.8
- Linux history in git trees
- Complete from 2.4.0
- Older releases mostly intact
- commit date issues, time based search fail
- XFS History in a git tree
- Complete from initial commit in 1993
- Older commits mostly intact, some issues
- Looking at extN / XFS / btrfs
- When – diffstat
- what – difss and commit messages
- who – commits, mailing lists
- EXT file system
- 2500 lines added in 1991 to get it working
- Removed in 2.1.21
- ext2
- 1994 – first commit
- 1998 – Steven Tweedy did lots of work
- 2002 – various stuff done
- 2003 – extended attributes originally from XFS added by XFS team
- 2008 – reservation based block allocation backported from ext3
- ext3
- 2002 – created
- 2003 – journalling added , extended attributes
- nothing much since then except maintenance
- ext4
- 2007- create
- features steadily being added 2007 till now
- btrfs
- 2x the amount of code than ext4
- Steady add of features since 2008
- Stuff being added not removed
- Sync’d to Linux merge and fix windows every 3 months
- xfs
- 1995 release
- 2000 removed a tonne of stuff due to licenses pre-GPL (and it still worked)
- Several other code removals
- Code removals often mechanism being replaced by something better
- Compare
- xfs is as old as all others and has had more work done on it most years except recently with btrfs
- btrfs due to overtake xfs as largest filesystem in 2014
