lca2011 – Thur Keynote – Sendmail revisted

Sendmail revisited – Eric Allman

  • sendmail more than 30y old, survived well, changed the world
  • Start 1980 at UC Berkeley, no support, before Internet existed, built as matter of nessesity
  • Then: CPUs 16bit, <1MIPS  . Disks <<1GB ,  Memory < 1MB  , Network <56k/s  ,  DBs research only. Phones huge and uncommon
  • Today: CPUs 64bit, >50k MIPS , Disks 2TB , Memory >100 GB , Network Megabit to ~160GB , NOSQL hot, RDMBS boring
  • 9.6k link to arpa link, only 2 ports for everybody to share, $5k for 1 tty port
  • Observed people just wanted email. Solution forward messages between networks
  • design principles:Only 1 programmer, don’t redesign user agents, don’t change local mail store, make delivermail adapt to the world
  • Problems: compiled in config, no address translate,
  • Transittion to sendmail – UCB awarded contract , adding SMTP support, needed to be build in, forced introduction of queue, queues harder than look
  • released 1982 with 4.1a BSD – SMTP, Queueing, runtime config
  • sendmail picked up my most unix vendors. Unix wars raged on – vendors added value and incompatible added up.
  • Returned to Berkley, sendmail rewrite -> sendmail8
  • Added in extensions developed elsewhere. Revision of SMTP, DSNsm other systems, 8bit mail, new configuration package
  • sendmail bat book dramatic increased uptake of sendmail8
  • Commercial sendmail – Sendmail Inc in 1998. 1st commercial open source/commercial hybrid companies
  • Added by commercial company – encryption & auth, milter, virtual hosting, LDAP, lots more checking
  • Part 2 Lessons Learned
  • Changes in requirements over time:
  • Reliability – deliver or bounce
  • Functionality and performance
  • Then became about protection
  • Next legal/regulatory issues
  • Increasingly controlling costs
  • Design decisions and evolution
  • Overly general: can do anything, reality world was in flux, easier to build tool than solution, still true today to extent, retrospect would do again
  • rewrite rules: good idea, using tabs as active character was wrong. concept right but syntax and control flow could have been better
  • Message munging – essential for interoperability at the time, still used – to you pass through transh, reject or fix
  • Syntax of config – flat (no nesting) , to much used of single characters, ugly but not fundimentally flawed, would use something like apache todday
  • smtp and queuing embedded in sendmail – good idea
  • structure of queue – two files per message, requires lots of scans, ascii simplified debugging, correct for time, wouldn’t do now
  • Using m4 for config – painful syntax, damn “dnl” ‘s aren’t even necessary. Retrospec some tool was needed unclear if m4 was the right one.
  • Extending vs changing features. Example masquerading. Wrong first time, reimplemented but added new feature instead of fixing. Didn’t want to break existing sites. retrospec wrong and big part of why sendmail hard to configure, should have provided upgrade tools
  • Accepting and fixing bogus input. permits broken software to exist. right at the time unclear if it’s the right thing today.
  • What do diff today –
  • Fix problems asap, v7 mailbox format, tabs in config files, bogus features
  • use modern tools
  • more privilege separation
  • create string abstraction
  • create string separation
  • separate mailbox names from unix user ids
  • cleaner config file
  • What would I do the same –
  • use C as the implementation language
  • Bit things off in smallish chunks
  • syslog
  • rewriting rules (without tab char)
  • Net rely too heavily on outside tools
  • Takeaways –
  • KISS actually works ( sendmail was a “2nd System” )
  • If you don’t know what you are doing advance designs don’t help much
  • The world is a messy place
  • Flexibility trumps performance
  • Fix things early – if you succeed installed base only gets bigger
  • ASCII is great for internal files and protocols
  • documentation is key to broad acceptance
  • the design space is always changing
  • Q: if you were not Eric and you were starting today which mail server would you use? A: “postfix”
Share