Allen Geer, Michael Harrod – Kiwi Ingenuity – Kiwi’s can Overcome Tough Problems In DevOps
- Contrast – US vs NZ
- In the US companies are bigger, lots more people, lots more money to throw at problems.
- Contrast with Arial Topdressing pioneered in NZ using surplus WW2 aircraft
- Since the problems are up to 100x bigger in the US the tools are designed for that scale. ROI might not not be there for smaller companies.
- Dealing with Scale
- Avoid “Shinny new thing” syndrom, plan for keeping things for at least 5 years.
- Ramp up slowly with the tool, push it into other areas.
- Avoid Single Person Silo.
- Bring up some Kiwi Inginuity (Look at Open source, Use the Free Tiers or Cheap Tiers).
- Out-Innovate the US companies rather than trying to out-scale
- Infrastructure: Monetization of Toil
- Spending time and money on stuff you can automate
- Lots of manual creating of infrastructure, servers, firewalls.
- Lack of incentive for providers who charge for changes to automate stuff
- Other Providers will automate (especially overseas ones that will come into NZ)
- People take risks (eg no DR) in order to save money.
- Innovator’s Dilemma
- Solutions
- More vocal customers
- Providers should provider a platform, lots more self-service. Ahnd-holding for the hard stuff not the day-to-day
- Charge for outcomes not person-hours
- Begin Small
- It’s an experiment – Freedom to Fail
- Inattentive Customer Service
- Overseas companies have a lot more forums, helpdesks, quick responses.
- “Kiwis reluctant to make a fuss” , Companies not used to people making a fuss
- Apply “American Ingenuity” – Striving focus to increase customer satisfaction.
- Build a healthy community (eg online forums) around your service.
- Gather insights from customers
- Bezos – “When a customer contacts us, we see this as a defect” . Focus on the source of problems
- Evolving Kiwi Workforce
- NZ has older and aging workforce. 2nd oldest in the OECD
- Slightly Fewer peoples with degrees
- 11% of workforce 65+ by 2038
- Learning in the workplace
- Leverage senior Knowledge
- Telco – Older customers didn’t want to approach young workers in mall. Brought in retired engineers to work in stores.
- Mentoring and reverse-mentoring. Mentor learns insights from mentoree too (eg about younger people’s habits)
- Introducing people to DevOps
- Kiwi DevOps models
Craig Box – Teaching Old Servers New Tricks: extending the service mesh outside the cluster
- Service Mesh
- Managing a service is hard
- metrics, monitoring, logging, traceing
- AAA encryption, certs
- load balancing, routing, network policy
- quota
- Failure handling, fault inject
- Microservices
- Not just for hipsters
- Works best at scale. Lots of devs
- Now introduce a network in between everything. Lots of hard dtuff, distributed systems are hard
- Leaky abstractions
- Have to build stuff into microservice to deal with problems of the network
- In multiple libraries and languages
- Can we fix it?
- Sidecar Pattern
- The sidecar does all the hard stuff instead of making the microservice itself do it.
- Talks TCP. Able to work with all languages
- Proxies as sidecars
- SPOF
- Sidecar is attached to each MS
- Flexability and Power
- Single place where we can do everything
- Traffic going in: TLS termination, metrics, quota
- Traffic out of workloads: Authentication, TLS connections
- Istio
- Open platform
- Not always microservices
- Uniform observability
- Operational Agility
- Policy driven Security
- How istio works
- Proxies + control plane
- Pilot in control plane pushes config to proxies, keeps track of them, looks up stuff in k8s cluster
- Mixer – policy check and telemetry
- Citidel – cert authority to proxies
- Control plane has to run on k8s
- Proxies run using envoy
- Zipkin built in
- All done automatically for kubernetes environments ( admission controller adds sidecar )
- Adding a VM to a service mesh
- Enable the mesh expansion, connect the networks
- Add the gateway IP to the VM
- Get a cert and copy to the VM
- Install proxy and node agent
- Traffic from cluster -> VM .
- Add the service to DNS in the cluster,
- Create a ServiceEntry on the cluster
- Traffic from VM -> Cluster service
- Add Service and IP to /etc/host on the VM
- Sample Application – Hipster Shop
- productcatalogservice is outside of kubernetes
- headless service in kubernetes
- manually created service entry in k8s
- Experimental istio commands to simplify process to single command