Troy Cornwall & Alex Corkin – Health is hard: A Story about making healthcare less hard, and faster!
Maybe title should be “Culture is Hard”
@devtroy @4lexNZ
Working at HealthLink
– Windows running Java stuff
– Out of date and poorly managed
– Deployments manual, thrown over the wall by devs to ops
Team Death Star
– Destroy bad processes
– Change deployment process
Existing Stack
– VMware
– Windows
– Puppet
– PRTG
CD and CI Requirements
– Goal: Time to regression test under 2 mins, time to deploy under 2 mins (from 2 weeks each)
– Puppet too slow to deploy code in a minute or two. App deply vs Conf mngt
– Can’t use (then) containers on Windows so not an option
New Stack
– VMware
– Ubuntu
– Puppet for Server config
– Docker
– rancher
Smashed the 2 minute target!
But…
– We focused on the tech side and let the people side slip
– Windows shop, hard work even to get a Linux VM at the start
– Devs scared to run on Linux. Some initial deploy problems burnt people
– Lots of different new technologies at once all pushed to devs, no pull from them.
Blackout where we weren’t allowed to talk to them for four weeks
– Should have been a warning sign…
We thought we were ready.
– Ops was not ready
“5 dysfunctions of a team”
– Trust as at the bottom, we didn’t have that
Empathy
– We were aware of this, but didn’t follow though
– We were used to disruption but other teams were not
Note: I’m not sure how the story ended up, they sort of left it hanging.
Pavel Jelinek – Kubernetes in production
Works at Movio
– Software for Cinema chains (eg Loyalty cards)
– 100million emails per month. million of SMS and push notifications (less push cause ppl hate those)
Old Stack
– Started with mysql and php application
– AWS from the beginning
– On largest aws instance but still slow.
Decided to go with Microservices
– Put stuff in Docker
– Used Jenkins, puppet, own docker registery, rundeck (see blog post)
– Devs didn’t like writing puppet code and other manual setup
Decided to go to new container management at start of 2016
– Was pushing for Nomad but devs liked Kubernetes
Kubernetes
– Built in ports, HA, LB, Health-checks
Concepts in Kub
– POD – one or more containers
– Deployment, Daemon, Pet Set – Scaling of a POD
– Service- resolvable name, load balancing
– ConfigMap, Volume, Secret – Extended Docker Volume
Devs look after some kub config files
– Brings them closer to how stuff is really working
Demo
– Using kubectl to create pod in his work’s lab env
– Add load balancer in front of it
– Add a configmap to update the container’s nginx config
– Make it public
– LB replicas, Rolling updates
Best Practices
– lots of small containers are better
– log on container stdout, preferable via json
– Test and know your resource requirements (at movio devs teams specify, check and adjust)
– Be aware of the node sizes
– Stateless please
– if not stateless than clustered please
– Must handle unexpected immediate restarts
