My rating for books I read. Note that I’m perfectly happy with anything scoring 3 or better.
5/5 = Brilliant, top 5 book of the year 4/5 = Above average, strongly recommend 3/5 = Average. in the middle 70% of books I read 2/5 = Disappointing 1/5 = Did not like at all
by Ben Rhodes The World As It is: Inside the Obama White House A memoir of a senior White House staffer, Speechwriter & Presidential adviser. Lots of interesting accounts with and behind the scenes information. 4/5
by John Scalzi Redshirts A Star Trek parody from the POV of five ensigns who realise something is very strange on their ship. Plot moves steadily and the humour and action mostly work. 3/5 by Laura Ingalls Wilder Little House on the Prairie The book covers less than a year as the Ingalls family build a cabin in Indian territory on the Kansas Prairie. Dangerous incidents and adventures throughout. 3/5 by Rich Houston Wheels Stop: The Tragedies and Triumphs of the Space Shuttle Program, 1986-2011 A book about the post-Challenger Shuttle missions. An overview of most of the missions and the astronauts on them. Lots of quotes mainly from the astronauts. Good for Spaceflight fans. 3/5
by Bina Venkataraman The Optimist’s Telescope: Thinking Ahead in a Reckless Age Ways that people, organisations and governments can start looking ahead at the long term rather than just the short and why they don’t already. Some good stuff 4/5
A Reminder of my rating System
5/5 = Brilliant, top 5 book of the year 4/5 = Above average, strongly recommend 3/5 = Average. in the middle 70% 2/5 = Disappointing 1/5 = Did not like at all
by Michael Burlingame Abraham Lincoln: A Life (Volume Two)
2nd volume covering Lincoln’s time as president. Lots of quotes from contemporary sources. Fairly good coverage of just about everything. 3/5
Capital City: Gentrification and the Real Estate State by Samuel Stein
Some interesting insights although everything being about New York and very left-wing politics of the author muddle the message. Worth a read if you are into the topic. 3/5
by Norman Maclean Young Men and Fire
The story of the 1949 Mann Gulch fire that killed 13 smoke jumpers. Misses a point due to lots of talking to maps/photographs but still a gripping story. 3/5
by Helen Fry The Walls Have Ears: The Greatest Intelligence Operation of World War II
The secret British operation to bug German POWs to obtain military intelligence. Only declassified in the late 1990s so very few personal recollections, but an interesting story. 3/5
I’ve decided to change my rating system
5/5 = Brilliant, top 5 book of the year 4/5 = Above average, strongly recomend 3/5 = Average. in the middle 70% 2/5 = Disappointing 1/5 = Did not like at all
edited by Gregory Benford Far Futures 5 Hard SF stories set it the distant (10,000 years+) future. I thought they were all pretty good. Would recommend 4/5 by Laura Ingalls Wilder Farmer Boy: Little House Series, Book 2
A year in a life of a 9 year old boy on a farm in 1860s New Year State. Lots of hard work and chores. His family is richer than Laura’s from the previous book. 3/5
by Neil DeGrasse Tyson Astrophysics for People in a Hurry A quick (4h) overview and introduction of our current understanding of the universe. A nice little introduction to the big stuff. 3/5
by David McCullough The Pioneers: The Heroic Story of the Settlers Who Brought the American Ideal West
The Story of five of the first settlers of Marietta, Ohio from 1788 and the early history of the town. Not a big book or wide scope but works okay within it’s limits. 4/5
by David Hepworth 1971, Never a Dull Moment: Rock’s Golden Year
A month by month walk though musical (and some other) history for 1971. Lots of gossip, backstories and history changing (or not) moments. 4/5
by Cal Newport Digital Minimalism: Choosing a Focused Life in a Noisy World A guide to cutting down electronic distrations (especially social media) to those that make your life better and help towards your goals. 3/5
The Fight to Keep the Watchers at Bay – Mark Nottingham
Disclaimer: I am not a security person, But in some sense we are all security people.
Why Secure the Internet
What has happened so far?
http -> https In 2010 even major services, demo of firesheep program to grab cookies and auth off Wifi Injecting cookies in http flows Needed to shift needle to https http/2 big push to make encrypted-only , isn’t actually though browsers only support https. “Secure Contexts” cool features only https Problem: Mixed Content “Upgrading Insecure Requests” allow ad-hoc by pages HTTPs is slow – istlsfastyet.com Improvement in speed of implimentations Let’s Encrypt Around 85-90% https as of Early 2020 Some people were unhappy Slow Satellite internet said they needed middle boxes to optimise http over slow links People who did http shared caching TLS 1.2 -> TLS 1.3 Complex old protocol Implementation monculture Outdated Crypto TLS 1.3 Simplify where possible encrypt most of handshake get good review of protocol At around 30% Lots of implementations Some unhappy. Financial institutions needed to sniff secure transactions (and had bought expensive appliances to do this) They ended up forkign their own protocol TCP -> QUIC TCP is unencrypted, lots of leaks and room for in-betweens to play around QUIC – all encrypted Spin Bit – single bit of data can be used by providers to estimate packet loss and delay. DNS -> DOH Lots of click data sold by ISPs Countries hijacking DNS by countries to block stuff DNS over https co be co-located by a popular website Some were unhappy Lots of pushback from governments and big companies Industry unhappy about concentration of DNS handling Have to decide who to trust SNI -> Encrypted SNI Working progress, very complex South Korea unhappy, was using it to block people Traffic Analysis Packet length, frequency, destinations TOR hard to tell. Looking at using multiplexing and fix-length records But the ends Customer compromised or provider compromised (or otherwise sharing data) Observations Cost and Control Cost: Big technology spends no obsolete Control: some people want to do stuff on the network We have to design tthe Internet to the pessimistic case You can’t expose application data to the path anymore Well-defined interfaces and counterbalanced roles Technology and Policy need to work togeather and keep each other in check Making some people unhappy means you need some guiding principles
LumoSQL – updating SQLite for the modern age – Dan Shearer
LumoSQL = SQLite + LMDB – WAL
” Is a replacement for fopen() “ Key/Value stores. Everyone used Sleepycat BDB – bought be Oracle and licensed changed Many switched to LMDB (approx 2010) Howard Chu 2013 SQLightning faster than SLQite but changes not adopted into SQLite
Funded by NLNet Foundation Dan Shearer and Keith Maxwell
What isn’t working with SQLite ?
Inappropriate/unsupported use cases Speed Corruption Encryption
What hasn’t been done so far
Located code, started on github.com/LumoSQL Benchmarking tool for versions matrix Mapped out how the keywords store works So different backend can be dropped in. Fixed bugs with the port and with lmdb
First Release Feb 2020 Add Multiple backends Implement two database advances
Open Source Won, but Software Freedom Hasn’t Yet: A Guide & Commiseration Session for FOSS activists by Bradley M. Kuhn, Karen Sandler
Larger Events elsewhere tend to be corperate sponsored so probably wouldn’t accept a talk like this
Free Software Purists
2010-era Laptops are some of the last that are fully free-software Later have firmware and other stuff that is all closed. HTC Dream – some firmware on phone bit but rest was free software
Coupons are all Digital. You need to run an app that tracks all you processors “As a Karen I sometimes ask the store to just ket me have the coupon, even though it is expired” Couldn’t install Disneyland App on older phones. So unable to bypass lines etc.
Proprietary dumping ground
Bradly had a device. Installed all the proprietary apps on it rather than his main phone But it’s a bad idea since all the tracking stuff can talk to each other.
Hypocrisy of tradition free software advocacy
Do not criticise people for use Proprietary software It is it is almost impossible to live your life without use it It should be an aspirational goal Person should not be seen as a failure if they use it Asking others to use it instead is worse than using it yourself Karen’s Laptop: It runs Debian but it is only “98% free”
Paradox: There more FOSS there is, the less software freedom we actually have in our technology
But there is less software freedom than there is in 2006 Because everything is computerized, a lot more than 15 years ago. More things in Linux that Big companies want in datacentres rather than tinkerers in their homes want.
What are the right choices?
Be mindful Try when you can to use free software. Make small choices that support software freedom Shine a light on the problem Don’t let the shame you feel about using proprietary software paralyze you and don’t let the problems we face overwhelm you into inaction Re-prioritize your FOSS development time. Is it going to give more people freedom in the world? Maybe try to do a bit in your free time. Support each other FAIF.us podcast
Advanced Stream Processing on the Edge by Eduardo Silva
Data is everywhere. We need to be able to extract value from it
Put it all in a database to extract value Challenge: Data comes from all sorts of places More data -> more bandwidth -> more resource required Delays as more data ingested Challenge: lots of different formats
Collect from different sources convert unstructured to structured enrichment and filtering multiple destinations like database or cloud services
Started in 2015 Origins lightweight log processor for embedded space Ended up being used in cloud space Written in C Low mem and CPU Plugable arch input -> parser -> filter -> buffer -> routing -> output
Unstructured to structured Metadata Can add tags to date on input, use it later for routing
Perform processing while the data is still in motion Faster data processing in Memory No tables No indexing Receive structured data, expose a query language Nomally done centrally
Doing this on the edge
Offload computation from servers to data collectors Only sends required data to the cloud Use a SQL-like language to write the queries Integrated with fluent core
Aggregation functions Time funtiocs Timeseries functions You can also write functions in Lua
Also exposed prometheus-type metrics
The History of X: Lessons for Software Freedom – Keith Packard
1984 – The Origins of X
Everything proprietary Brian Reid and Paul Asente: V Kernel -> VGTS -> W window system Ported to VAXstation 100 at Stanford 68k processor, 128k of VRAM B&W Bob Scheifler started hacking W -> X Ported to Unix , made more Unix Friendly (async) renamed X
Unix Workstation Market
Unix was closed source Vendor Unix based on BSD 4.x Sun, HP, Digital, Apollo, Tektronix, IBM this was when the configure program happened VAXstation II Color graphics 8bit accelerated Sun 3/60 CPU drew everything on the screen
Early Unix Window System – 85-86
SunView dominates (actual commerical apps, Ddesktop widgets) Digital VMS/US Apollo had Domain Tektronix demonstrated SmallTalk all only ran on their own hardware
X1 – X6
non-free software Used Internally at MIT Shared with friends informally
X10 – approx 1986
Almost usable Ported to various workstations Distribution was not all free software (had bin blobs) Sun port relied on SunView kernel API Digital provided binary rendering code IBM PC/RT Support completed in source form
Why X11 ?
X10 had warts rendering model was pretty terrible External Windows manager without borders Other vendors wanted to get involved Jim Gettys and Smokey Wallace Write X11, release under liberal terms Working against Sun Displace Sunview “Reset the market” Digital management agreed
X11 Development 1986-87
Protocol designed as croos-org team Sample implementation done mostly at DEC WRL, collaboration with people at MIT Internet not functional enough to property collaborate, done via mail Thus most of it happened at MIT
MIT X Consortium
Hired dev team at MIT Funded by consortium Members also voted on standards Members stopped their on develoment Stopped collaboration with non-members We knew Richard too well – The GPL’s worst sponsor Corp sponsors dedicated to non-free software
X Consortium Standards
XIE – X Imaging Extensions PIX – Phigs Extension for X LBX – Low Bandwidth X Xinput (version 1)
The workstation vendors were trying to differentiate. They wanted a minimal base to built their stuff on. Standard was frozen for around 15 years. That is why X fell behind other envs as hardware changed.
X11 , NeWs and Postscript
NeWS – Very slow but cool Adobe adapted PostScript interpreter for windows systems – Closed Source Merged X11/NeWS server – Closed Source
The Free Unix Desktop
All the toolkits were closed source Sunview -> XView OpenView – Xt based toolkit
X Stagnates – ~1992
Core protocol not allowed to change non-members pushed out market fragments
Collapse of Unix
Opening a treasure trove: The Historical Aerial Photography project by Paul Haesler
Geoscience Australia has inherated an extensive archive of hisorical photography 1.2 million images from 1920 – 1990s Full coverage of Aus and more (some places more than others)
Historical Archive Projects
Canonical source of truth is pieces of paper Multiple attempts at scanning/transscription. Duplication and compounding of errors Some errors in original data “Historian” role to sift through and collate into a machine-readable form – usually spreadsheets Data Model typically evolves over time – implementation must be flexible and open-minded
What we get
Flight Line Diagrams (metadata) Imagery (data) Lots scanned in early 1990s, but low resolution and missing data, some missed
Flight line diagram pipeline High resolution scans Georeferences Film pipeline Filmstock High Resolution scans Georeference images Georectified images Stitched mosaics + Elevation models
Only about 20% of film scanned. Lacking funding and film deteriorating
Other states have similar smaller archives (and other countries)
Many significantly more mature but may be locked in propitiatory platforms
Open Data ( Cc by 4.0) Open Standards (TESTful, GeoJSON, STAC) Open Source PostGreSQL/PostGIS Python3: Django REST Framework Current Status: API Only. Alpha/proof-of-concept
Search for Flight runs Output is GeoJSON
Scanning and georeferencing (need $$$) Data entry/management tools – no spreadsheets Refs to other archives, federated search Integration with TerriaJS/National Map Full STAC once standardized
Engineer tested, manager approved: Migrating Windows/.NET services to Linux – Katie Bell
Works at Campaign Monitor
sends email spam Company around since 2004
Software product generations
Originally a monolith Windows, C# .net framework, IIS, Monolithic SQLServer Went to microservices (called Reckless Microservices) Windows, C# .net , OWIN Hosting / Nancy , Modular databases
Gen 2 – “Reckless” Microservice
Easy to create a new microservices and deploy etc Runs in ec2
Wanted to go to a tools like dockers, kubernetes that were not well supported by microsoft tools
Gen 3 – Docker Services
Lots of ways to do stuff
3 different ways of doing everything Confusing and big tax on developers Losing knowledge about how the older Reckless stuff worked
A Crazy Idea
Run all the Reckless services in docker Get rid of one whole generation
What does it take?
Move from .NET Framework to .NET Core Framework very Windows specific – runtime installed at OS level Core more open and cross-platform – self contained executable apps But what about Mono? (Open Source .NET Framework) . Probably not worth the effort since Framework is the way forward But a lot of .NET Framework APIs not ported over to .NET Core. Some replaced by new APIs .Net Standard libraries support on both though, which is lots of them
What Doesn’t port to Core?
Libraries moved/renamed Some libs dropped IIS, ASP.NET replaced with ASP.NET Core + MVC WCF Server communication Old unmaintained libraries
Luckily Reckless not using ASP.NET so shouldn’t to too hard to do. Maybe not sure a crazy idea.
But most companies don’t let people spend lots of time on Tech Debt.
Asked for something small – 2 weeks of 3 people.
1 week: Hacky proof of concept (getting 1 service to run in .NET Core) 2nd week: Document and investigate what full project would require and have to do Last Day: Time estimates Found that Windows ec2 instance were 45% Cost saving alone of moving from Windows to Linux justied the project Pitching: Demo Detailed time estimates Proposal with multiple options Concrete benifits, cost savings, problems with rusty old infra
Microsoft Portability Analyzer Just run across app and gives very detailed output icanhasdot.net Good for external dependencies
Web Hosting differences
OWIN Hosting vs Kestrel ASP.NET Core DI
Libraries that Do support .NET Standard
Had to upgrade all our code to support the new versions Major changes in places
case-sensitive filenames Windows services, event logging
Libararies that did not support .net Standard
Magnum – unmaintained Topshelf
.NET Framework Libraries can be run under .NET Core using compatibility shim. Sometimes works but not really a good idea. Use with extreme caution
Took 6-8 months of 2-3 people Everything migrated over. Around 100 services 78 actually running 43 really needed to be migrated 31 actually needed in the end Estimated old hosting cost $145k/year Estimated new hosting costing $70k/year Actual hosting cost $15k/year Got rid of almost all the extra infrastructure that was used to support reckless. another $25k/year saved
Advice for cleanup projects
Ask for something small Test the idea Demonstrate the business case Build detailed time estimates
Collecting information with care by Opel Symes
People build systems for people without checking our assumptions about people are valid Be aware of my assumptions, this doesn’t cover all areas
Form “First Name” and “Last Name” -> “Dear John Smith” Fields Required – should be optional Should not do character checks ( blocking accents etc ) Check production support emoji.. everywhere MySQL Character Encodings. Only since 5.5 , default in MySQL 8 Every Database, table and text cloumn and defaults need to be changed to the new character set. Set connection options so things don’t get lost in transfer. Personal Names around the world Chinese names Names can be long Recommendation Ask for “Full name” (where a legal name is required) and “Greeting” Unicode all the way down – test with emoji No Length limits
Email addresses are quite complex Does it have an “@” Checked it is not a simple typo of a well-known email down Will it be accepted by the email sender? Look for an MX record Ask the SMTP server if this username is valid Simple checks for common errors Don’t roll your own checking, use you own mail server or the mail library that you will using to send.
Transgender vs Cisgender Non-binary – Gender that isn’t male or female Don’t just give the two options A 3rd “other” option isn’t ideal A freeform field is good. Gender Alternative from Nikki Stevens Instead ask if people make up an “under representated community”
What pronounces should we use to refer to you? ( he , she, they ) Works okay in English but may not in other languages Some lanugages lack gender-nutral pronoun Some languages lack gender pronouns pronoun.is
Ask for “None” but don’t actually print it “Dear None Smith” Ask for Mx Have a freeform field ( Dr, Count ) Maybe avoid titles if possible Don’t show people according to gender, ask specifically.
Gender – WGEA
The Act defines gender as male or female. Others are not reported. Have an explanation for people who don’t fit in the above
Make it simple to change Give users options if it isn’t (eg show preferred name)
Usernames are often options Changing them comes with some caveats Using UUIDS to links to users rather than usernames
There are security implications
Make it possible and no to hard
Keynote: Who cares about Democracy? by Vanessa Teague
The techniques for varifying electronic elections are probably to difficult for real voters to use.
The ones that have been deployed have lots of problems
Complex maths for end-to-end varifiable elections
– people can query their votes to varify it was recorded – votes are safely mixed so others can’t check.
– 2 bugs. One in the shuffling, one in decryption proof
End-to-end verifiable elections: limitations and criticism
Users need to do a lot of careful work to verify If you don’t do it properly you can be tricked You can ( usually ) prove how you voted Though not always, and usually not in a polling-place system Verification requires expertise Subtle bugs can undermine security properties
What does all this have to do with NSW iVote?
Used Closed source software Some software available under NDA afterwards Admitted it was affected by the first Swiss bug. This was when early voting was occuring Also so said 2nd Swiss bug wasn’t relevant. After code was available they found it was relevant, a patch had been applied but it didn’t fix the problem NSW law for election software is all about penalties for releasing information on problems.
Victoria has passed a bill that allows elections to be conducted via any method which is aimed at introducing electronic voting in future elections
Electronic Counting of Paper Records
Keynote: Who cares about Democracy? by Vanessa TeagueVarious areas have auditing software that runs against votes This only works on FPTP elections, not Instant-runoff elelctions Created some auditing software what should work, this was testing using some votes in San Francisco elections A sample of ballots is taken and the physical ballot should match what the electronic one said it is.
Australian Senate vote
Auditing not done, since not mandated in law
What can we do
Swiss has laws around transparency, privacy and varivication NSW Internet voting laws is orientated around protecting the vendors by keeping the code secret California has laws about Auditing Australian Senate scrutineering rules say nothing about computerised scanning and auting Aus Should Must be a meaningful statistical audit of the paper ballots with meaningful observation by scrutineers
Varifiable e-voting at polling place is feasible over the Internet is an unsolved problem The Senate count at present provides no evidence of accuracy but would if a rigorous statistical audit is mandated
How else to use verifiable voting technology?
Crowsourcing amendments to legislation with a chance to vote up or down Open input into parliamentary quesions A version for teenagers to practice debating what they choose
Open collaborations: leadership succession and leadership success – Anne Smith & Myk Dowling
Started playing Kerbal Space Program and using lots of mods to it.
Comprehensive Kerbal Archive Network 150k downloads of a previous release, 72k of last release 1035 starts on github 124 releases from 16 developers Written in C-sharp
Why was the project a success out of around 1.4 million projects?
FOSS projects are generally modular C and C-derived languages are predictive of success Portability predictor of success Layered Development
83% of FOSS Projects fail. 46% before and 37% after a stable release
How do projects organise?
First the founder 1-2 Then a belt of users emerages Then a periphery – active users A core of developer emerges Some formality emerges
Relying on self-motivated people limits the number of people who will join your team If you lose people by brushing them off you reduce your team diversity, team diversity gives increased likelihood of success From the core to the periphery. Order of magnitude decrease in activeity but order of magnitude increase in size. Therefore is 1:1 level or work. Which is about the same level of code:support work that is needed. Flat structures are not stable; FOSS teams self-organise into a complex of a dual-layer structure Leaders should prioritise the people on the periphery. Many join for a short term need, the leader has to give them other reasons to stick around.
Links to other Projects
Friction with Mod authors. Mods who though CKAN installed things the wrong way and caused problems got annoyed. Some authors of modules that were under FOSS asked for it to be removed, which CKAN resisted doing. CKAN was mostly orientated towards users and not so much towards the authors Significant group of mod authors considered opting out of CKAM Speaker proposed a policy that allowed mod authors to delete mod
Strong technical contributions Participatory behavior Organisation building behaviors
Leadership origin and style
Typically the initial leader/s are the founder/s Often shared Leaders may move from core to periphery without losing the position Organisation focus vs Product (technical) focus People with both skills are the ones selected for leadership
CKAN in Transition
Removed mods as requested Which broke things for some time Leadership got transfered over Original technical-orientated leader stepped back A more Organizational-orientated leader took over A clear and public succession is much better. Although some people still dropped out. But better and an acrimonious fork
Make speed and smooth Happen at the speed of military coups Limited participation from a predecessor assits in a smooth change Establishing succession rules helps
Review the state of your projects public-facing website from the POV of the peripheral people you want to attract.
Open Source Citizenship by Josh Simmons
Healthy Projects are vital which is why many companies are investing in projects
They don’t just need money
What are companies doing now?
Upsteaming contributions Contributing to the ecosystem Paid contributors on staff (full or part time) Hire out of the Project contributors Supporting with money, infrastructure etc. Both projects directly and other things Programs to help contributors get started. Sharing their experience
What companies provide is not always what communities want
What are Communities asking for?
Volunteer design, UX/UI Project management technical writing data science marketing/PR
and yet, code still dominates. These skills need onramps to contribute to your projects.
Contribute beyond what the company needs Projects want testing and QA resources Fund conference travel for contributors Event Space Open Source friendly contracts for employees who contribute to Open Source – See the “Contract Patch Program” Jobs the maintainers and contributors when heavily relying on their work If the maintainers are not getting paid that is a risk for the business Encourage Universities to give students credit for contributing to FLOSS Abide by community norms
Building a Culture of Open Source Citizenship
Enumerate and value your dependencies Raise internal aweness Incentivise your people to contribute to open source train, train and train Be Patient
For FLOSS Projects
Make it easy to learn about you project Have clear project government and licensing Say what you are looking for We want to know the invest we make in you is going to be used well and in a trasparant way Have a way to receive Money Look at being a member of a larger organisation like Software Conservatory See also open collective if you are just starting out Have a plan for how you are going to use the money Be prepared to work with corporate timelines Be prepared to onboard new contributors Contributor documentation