Engineer tested, manager approved: Migrating Windows/.NET services to Linux – Katie Bell
Works at Campaign Monitor
- sends email spam
- Company around since 2004
Software product generations
- Originally a monolith
- Windows, C# .net framework, IIS, Monolithic SQLServer
- Went to microservices (called Reckless Microservices)
- Windows, C# .net , OWIN Hosting / Nancy , Modular databases
Gen 2 – “Reckless” Microservice
- Easy to create a new microservices
- and deploy etc
- Runs in ec2
Wanted to go to a tools like dockers, kubernetes that were not well supported by microsoft tools
Gen 3 – Docker Services
- Linux
- Java / Go
Lots of ways to do stuff
- 3 different ways of doing everything
- Confusing and big tax on developers
- Losing knowledge about how the older Reckless stuff worked
A Crazy Idea
- Run all the Reckless services in docker
- Get rid of one whole generation
What does it take?
- Move from .NET Framework to .NET Core
- Framework very Windows specific – runtime installed at OS level
- Core more open and cross-platform – self contained executable apps
- But what about Mono? (Open Source .NET Framework) .
- Probably not worth the effort since Framework is the way forward
- But a lot of .NET Framework APIs not ported over to .NET Core. Some replaced by new APIs
- .Net Standard libraries support on both though, which is lots of them
What Doesn’t port to Core?
- Libraries moved/renamed
- Some libs dropped
- IIS, ASP.NET replaced with ASP.NET Core + MVC
- WCF Server communication
- Old unmaintained libraries
Luckily Reckless not using ASP.NET so shouldn’t to too hard to do. Maybe not sure a crazy idea.
But most companies don’t let people spend lots of time on Tech Debt.
Asked for something small – 2 weeks of 3 people.
- 1 week: Hacky proof of concept (getting 1 service to run in .NET Core)
- 2nd week: Document and investigate what full project would require and have to do
- Last Day: Time estimates
- Found that Windows ec2 instance were 45%
- Cost saving alone of moving from Windows to Linux justied the project
- Pitching:
- Demo
- Detailed time estimates
- Proposal with multiple options
- Concrete benifits, cost savings, problems with rusty old infra
- Microsoft Portability Analyzer
- Just run across app and gives very detailed output
- icanhasdot.net
- Good for external dependencies
Web Hosting differences
- OWIN Hosting vs Kestrel
- ASP.NET Core DI
Libraries that Do support .NET Standard
- Had to upgrade all our code to support the new versions
- Major changes in places
OS Differences
- case-sensitive filenames
- Windows services, event logging
Libararies that did not support .net Standard
- Magnum – unmaintained
- Topshelf
.NET Framework Libraries can be run under .NET Core using compatibility shim. Sometimes works but not really a good idea. Use with extreme caution
Overall Result
- Took 6-8 months of 2-3 people
- Everything migrated over.
- Around 100 services
- 78 actually running
- 43 really needed to be migrated
- 31 actually needed in the end
- Estimated old hosting cost $145k/year
- Estimated new hosting costing $70k/year
- Actual hosting cost $15k/year
- Got rid of almost all the extra infrastructure that was used to support reckless. another $25k/year saved
Advice for cleanup projects
- Ask for something small
- Test the idea
- Demonstrate the business case
- Build detailed time estimates
Collecting information with care by Opel Symes
The Problem
- People build systems for people without checking our assumptions about people are valid
- Be aware of my assumptions, this doesn’t cover all areas
Names
- Form “First Name” and “Last Name” -> “Dear John Smith”
- Fields Required – should be optional
- Should not do character checks ( blocking accents etc )
- Check production support emoji.. everywhere
- MySQL Character Encodings. Only since 5.5 , default in MySQL 8
- Every Database, table and text cloumn and defaults need to be changed to the new character set. Set connection options so things don’t get lost in transfer.
- Personal Names around the world
- Chinese names
- Names can be long
- Recommendation
- Ask for “Full name” (where a legal name is required) and “Greeting”
- Unicode all the way down – test with emoji
- No Length limits
- Email addresses are quite complex
- Does it have an “@”
- Checked it is not a simple typo of a well-known email down
- Will it be accepted by the email sender?
- Look for an MX record
- Ask the SMTP server if this username is valid
- Simple checks for common errors
- Don’t roll your own checking, use you own mail server or the mail library that you will using to send.
Gender
- Transgender vs Cisgender
- Non-binary – Gender that isn’t male or female
- Don’t just give the two options
- A 3rd “other” option isn’t ideal
- A freeform field is good.
- Gender Alternative from Nikki Stevens
- Instead ask if people make up an “under representated community”
Pronouns
- What pronounces should we use to refer to you? ( he , she, they )
- Works okay in English but may not in other languages
- Some lanugages lack gender-nutral pronoun
- Some languages lack gender pronouns
- pronoun.is
Titles
- Ask for “None” but don’t actually print it “Dear None Smith”
- Ask for Mx
- Have a freeform field ( Dr, Count )
- Maybe avoid titles if possible
- Don’t show people according to gender, ask specifically.
Gender – WGEA
- The Act defines gender as male or female.
- Others are not reported.
- Have an explanation for people who don’t fit in the above
Data Retention
- Make it simple to change
- Give users options if it isn’t (eg show preferred name)
Changing Username
- Usernames are often options
- Changing them comes with some caveats
- Using UUIDS to links to users rather than usernames
Changing Emails
- There are security implications
Deleting Data
- Make it possible and no to hard
