DevSecOps – Anthony Rees
“When Anthrax and Public Enemy came together, It was like Developers and Operations coming together”
- Everybody is trying to get things out fast, sometimes we forget about security
- Structural efficiency and optimised flow
- Compliance putting roadblock in flow of pipeline
- Even worse scanning in production after deployment
- Compliance guys using Excel, Security using Shell-scripts, Develops and Operations using Code
- Chef security compliance language – InSpec
- Insert Sales stuff here
- ispec.io
- Lots of pre-written configs available
Immutable SQL Server Clusters – John Bowker (from Xero)
- Problem
- Pet Based infrastructure
- Not in cloud, weeks to deploy new server
- Hard to update base infrastructure code
- 110 Prod Servers (2 regions).
- 1.9PB of Disk
- Octopus Deploy: SQL Schemas, Also server configs
- Half of team in NZ, Half in Denver
- Data Engineers, Infrastructure Engineers, Team Lead, Product Owner
- Where we were – The Burning Platform
- Changed mid-Migration from dedicated instances to dedicated Hosts in AWS
- Big saving on software licensing
- Advantages
- Already had Clustered HA
- Existing automation
- 6 day team, 15 hours/day due to multiple locations of team
- Migration had to have no downtime
- Went with node swaps in cluster
- Split team. Half doing migration, half creating code/system for the node swaps
- We learnt
- Dedicated hosts are cheap
- Dedicated host automation not so good for Windows
- Discovery service not so good.
- Syncing data took up to 24h due to large dataset
- Powershell debugging is hard (moving away from powershell a bit, but powershell has lots of SQL server stuff built in)
- AWS services can timeout, allow for this.
- Things we Built
- Lots Step Templates in Octopus Deploy
- Metadata Store for SQL servers – Dynamite (Python, Labda, Flask, DynamoDB) – Hope to Open source
- Lots of PowerShell Modules
- Node Swaps going forward
- Working towards making this completely automated
- New AMI -> Node swap onto that
- Avoid upgrade in place or running on old version
