Sunday, August 20, 2017

Tales from the Enterprise - Backward Compatibility

Backward compatibility - this is something you will hear over and over in various forms during your life as an enterprise developer. The arguments become even more strong when you consider someone who’s developing framework libraries for the organization. These are libraries that gets shared across the organization providing a standardised set of services - both business specific common operations and business agnostic operations like data access, logging, security etc...I can certainly see why these libraries need to sit at the organization level providing common functionality. However after few years down the track one realizes that there are lot of business critical application that have taken dependencies on these libraries. Now you realise that a change to a framework library that should not take more than couple of days, suddenly becomes a saga and involves dances between multiple teams and can get quite nasty. The underlying reason for this is that ‘Backward compatibility’.  

Even if the changes to the libraries themselves are tiny, the complexities around how to manage that change in the context of many applications that depended on these libraries can become quite complex.

So how does one keep upgrading these framework libraries all the while being backward compatible? Most of the business systems still want to get the benefit of these upgrades, but they don’t want to deal with the risks associated with the changes. What could be some of the tips dealing with cases like this.

  1. Version control is your friend. With a modern version control like git, concepts like branches are no longer difficult - it’s second nature. Use them to your advantage when you want to test out upgrades in isolation to a product that’s live and is potentially going through it’s own business critical changes.
    1. Create an ‘Upgrade’ branch to isolate framework library changes (Also absorb changes to the main branches in a controlled manner)
  2. Automated builds - Assuming your business critical application already has a CI build, (Don’t be surprise if there aren’t - then you gotto make one immediately) make another CI build so that the upgrade branch is continuously tested automatically.
    1. Make a copy of the existing CI build (TeamCity is great at this) and point to the upgrade branch created earlier
  3. Pick a few tests as critical - Your business project may have thousands of tests, but it might not be 100% reliable. You have to pick a subset of tests (you may have to write a few) that directly relates to the changes you are making. This is your test suite - you must make sure that they remain green during your upgrade project.
  4. Dependency Management - If your legacy projects are not using a proper dependency management framework like Nuget, you are in for a mess. If it’s not, it’s worthwhile investing your time to do so. In my case most of the shared libraries did actually had nugets published, however some of  the legacy apps were happily doing direct lib references like they did 5-8 years ago. It was well worth the time and effort to make them use nugets instead.
    1. Make the project use nuget (or paket or maven) instead of lib dependencies
    2. Make use of nuget versions to isolate from potential breaking changes
  5. Adapt semantic versioning - What goes hand in hand with dependency management is semantic versioning. With semantic versioning you can take a more or less confident approach towards adapting newer versions of a given internal shared library.
  6. Break it if you really want to - There may be times, when the friction from the past is too much, that you can’t really progress without making breaking changes. In these instances be brave and make it. If you don’t do it, it won’t be tried for a few more years and by that time the sheer weight of technical debt will break the systems down anyway/ But the caveat is with techniques like semantic versioning in place, you can do these in a safer manner.
  7. Pick a good time window for testing - Testing by the framework library developers themselves might not be sufficient. May be the merge back to main branch can wait till the project has a meaty enough change planned, so that you could leverage from a full regression test cycle. Being a bit business savvy in these decisions could be critical, since the project it self might not want to spend money in the framework library upgrade.

Post a Comment