My versioning system is broken. It's painful and I'm not entirely sure what to do about it. Time for a blog entry so!

Previously I wrote about my vision for the Ricebridge Versioning System. All Ricebridge components have three version numbers. For example, 1.2.3 means that you have major version 1, minor version 2, and build number 3.

Major versions allow for incompatible changes. Minor versions add new functionality but keep compatibility. Build numbers track bug fixes and minor changes. I think it works very nicely. It's pretty clear and easy to follow. And you know where you stand and what you can upgrade to. You know that a major version change will probably break your code and you'll have some extra work to do. But you also know that you can upgrade to a bug fix release or take advantage of a new method in a minor release by just dropping the jar in and continuing on your way.

I'm very attached to the three number versioning system. And so are my customers who've got very used to it. Changing versioning system is in itself a major version change really (given that you are breaking semantic compatibility), and I don't really feel like it at the moment!

So what's broken? Well I have new version of CSV Manager coming out soon. This will add a few new things, including Java Beans support. Nothing that will affect the existing code, so no problems there.

However, based on the experience gained building XML Manager, and my future plans for the product set, it looks like some of the interfaces in CSV Manager will have to change.

I want to create a coherent set of components that all work the same way. That's very important. There has to be interop, both at the code level and at the user level. That means that the APIs should be the same, Once you learn the API of one Ricebridge component, you can then apply it to all Ricebridge components (pretty much).

But the original CSV Manager API is not right for this. It needs to be modified. That breaks compatibility. So fine. We go to CSV Manager 2.0.

Except, not so fine. Not at all. You see, Ricebridge customers get an upgrade path. When you buy from us, you get the right to upgrade, for free, to a release that has the same major version. Right now CSV Manager is at major version 1. If I bump it up to major version 2, then all those customers will loose out. Ouch. Not very nice at all. Definitely not the right way to go. I want existing customers to get the full benefit of the new API as well.

It looks like we can’t bump the major version. So let's drop the compatibility restriction on minor versions. That could work. Except now, when you look at a version number, you can't tell right away whether it will work with your current setup. Not so good either.

How about using four version numbers? We can add an extra one for compatibility: major.release.minor.build. So we bump the release number every time there's an incompatibility change. Except this is not very user-friendly. Four version numbers is really pushing it. Three is just about as much as anyone can take. In any case, it's a change I don’t want, as noted above.

Another option is to stick with the old API until the real version 2.0. We can add the new stuff, but keep the old stuff in and deprecate it. This is the standard way of doing things and it is the approach that I use normally. It works especially well for adding and removing methods from API classes. But it doesn't work for interfaces that the user implements.

If you want to change an interface that a user has implemented, then you have to force the user to change their implementation class. There's no easy way round that (sadly we're in Javaland, so dynamic solutions are awkward). This happens because the interaction model between the interface and client code changes. The implementation class must provide the new services required by the client code. In some cases it might be possible to work around this, but mostly it's very messy and there is no way you can ask people to do this.

In the end, we are forced to choose between a new implementation interface and an old one. So if we want to wait until 2.0 for the big change, then we must stick with the old API for now. This works, but it increases the amount of people using the old API and makes the changeover much more painful for many more people. Microsoft has been badly stung by this many times: The long and sad story of the Shell Folders key. It seems better on the whole to take the hit now and impact the smallest number of people.

So where does this leave us? The next release of CSV Manager will include the new API. It will break old code. I'm really sorry to have to do this to my customers. It's very nasty. But the benefit will be the many ways in which you will be able to use Ricebridge components together to solve all sorts of tricky problems. It seems like a good tradeoff in the longterm.

Actually it's not even as bad as all that. The main CSV Manager methods will not be changing much and 90% of existing code should still work fine. The change will be occurring in a part of the code that is not even used by most customers (LineProviders). So maybe all this agonising isn't even necessary!

That still leaves the problem that version 1.1 and version 1.2 will be incompatible. We still need a way to communicate this and to allow users to control versioning in their own projects. It looks like an additional stream of version information is needed. We need to track API changes separately.

One idea is to have API revisions. This means publishing a detailed description of the API and assigned it a revision number. All CSV Manager releases with the same API revision number are going to be API compatible. That means you can copy in the new jar and things will still just work.

When incompatible changes have to be made to the API, then the revision number changes. A new API description is published, showing the changes, so that users can track what happened. So each version of CSV Manager has an API revision. You can show this in a table and it should then be easy for people to follow.

API revisions would not change very frequently. In fact, as we move further along with building Ricebridge components, things should start to settle down very solidly. I don't think that there will be many API revisions.

In this system, the minor version number then means: new functionality and possibly a new API revision. This creates extra work in that API revision changes have to be made clear to users, and handled carefully.

What really gets me about all this is that I had actually put in place a system to deal with it. After reading David Bau's posts about a theory of compatibility, I created a design for the user-implemented interfaces that could accommodate certain types of changes.

When you implement a Ricebridge interface, you are actually advised to extend a designated abstract support class. This class has the job of insulating you from future API changes by translating changes into the older version of what you expect. Except that this only works in some cases. To solve my existing problem it looks like I would have to write a lot of reflection code and that would probably have a bad effect on the performance of the system. It also means that new customers would find the support class to be a confusing mess. I have in fact used this technique already on one minor change to CSV Manager.

One problem that it does not solve however is that I made a mistake with the method signatures when I first released CSV Manager. Some *Impl methods which should be protected are actually public. Bugger!

So the way forward is still not clear. I am inclined to take the hit now and offer free support to all customers who need it to make the change.

What do you guys and gals think I should do?

In proving foresight may be vain;
The best-laid schemes o' mice an 'men
Gang aft agley,
An'lea'e us nought but grief an' pain,
For promis'd joy!

Robert Burns

tag-gen:Technorati Tags: software version system release compatibility Del.icio.us Tags: software version system release compatibility