Ok so it's been a while since I last post anything here, mainly because I've been working on a number of different big projects for the last couple of months and this left no time for blog posting.
So having come up for air for a few hours at least I thought it'd be a good idea to blog about them see if I can drum up some interest ;)
The major addition to my task list over the past few months has been the Sigil project. This is a set of tools to help developers build OSGi bundles. It started off as an eclipse plugin but about two or three months ago my team mate came along with some really great code to integrate it with Ivy.
I believe Sigil is the first tool out there to unify OSGi development in the IDE and server side build in a common configuration file (this being a good thing as it saves the messy job of keeping other systems in sync).
The IDE supports tools such as code complete, quick fixes and tools to analyse OSGi dependencies. I've also built in the idea of repositories (currently either file system or OBR but extensible via plugins) which allow the developer to download dependencies on the fly whilst developing by simply adding an import-package or require bundle statement to their code. Oh and the same repositories can be used in eclipse and ivy :)
The other big piece of code I've been working on is of course Newton. There are no big feature announcements for this release as we've been focussing on making the current platform more and more robust. But we've just made available the 1.3.1 release which has a number of internal improvements to reliability and scalability - I'd encourage you to take a look - one game we end up playing in the office is "Wak - A - Mole" with the containers when the fractal demo is deployed. You can take out a container (ctrl-c, ctrl-z, power cycle, cable unplug, etc) and the infrastructure will always recover :)
Anyway that seems like a good amount of detail for the time being. I'll try to blog some more about some of this stuff soon...
Laters,
Thursday 27 November 2008
Tuesday 22 July 2008
Is this an application which I see before me?
This post has been triggered by two interesting posts on the topic of what it is to be an application in an OSGi environment. This is something I felt I just had to get stuck in with as its exactly what we've been working on in the Newton project.
The way we've tackled this is very similar to the approach suggested by Mirko, except instead of using Spring-DM as the top level element in the model we've picked SCA.
SCA is a relatively new specification but it gives a vendor neutral way of describing the service dependencies and architecture of an application running in an enterprise environment.
The basic pattern is that a component which has a certain implementation (java, spring, bpel, c, etc) is able to publish a number of services (over a range of protocols - in-memory, RMI, WS, JMS, etc) and consume services from other components via references. The combined unit which is the component, it's services and it's references are then packaged in an entity called a composite.
In Newton we associate a composite with a top level (or root) bundle. This bundle then provides the class space for the composite to instantiate it's implementation, services and references. Importantly the bundle does not have to contain all of the classes that it needs to function but can use OSGi tools such as Import-Package to achieve modularity at the deployment level.
When an SCA composite is installed in a Newton runtime we go through a series of steps:
To achieve this we group composites together into an entity known as a System. A system is a group of composites which each specify a particular replication strategy and a set of contract requirements. A replication strategy is a dynamic pattern used by Newton to decide how many instances of a Composite should be installed at any given instant. A contract is a pattern used to restrict where a composite should be installed, i.e. a contract may specify a filter such as (&(machine.cpu.count>1)(machine.memory>1G)) i.e. only install on dual cpu or greater machines with more than 1GB of memory available.
So in the Newton world an application is actually a very dynamic entity. It is built from OSGi bundles and wired together on the fly. But it never the less has rules which are defined via a model (expressed in SCA plus our system extensions) which specify:
Laters,
The way we've tackled this is very similar to the approach suggested by Mirko, except instead of using Spring-DM as the top level element in the model we've picked SCA.
SCA is a relatively new specification but it gives a vendor neutral way of describing the service dependencies and architecture of an application running in an enterprise environment.
The basic pattern is that a component which has a certain implementation (java, spring, bpel, c, etc) is able to publish a number of services (over a range of protocols - in-memory, RMI, WS, JMS, etc) and consume services from other components via references. The combined unit which is the component, it's services and it's references are then packaged in an entity called a composite.
In Newton we associate a composite with a top level (or root) bundle. This bundle then provides the class space for the composite to instantiate it's implementation, services and references. Importantly the bundle does not have to contain all of the classes that it needs to function but can use OSGi tools such as Import-Package to achieve modularity at the deployment level.
When an SCA composite is installed in a Newton runtime we go through a series of steps:
- Resolve the root bundle that supplies the class space for the composite. If this is not the first time the root bundle has been resolved we increment an internal counter
- Resolve and optionally download the bundle dependencies required to satisfy the root bundle against a runtime repository (this includes ensuring that we reuse existing bundles within the runtime - if they were installed for other composites)
- Build a runtime model around the SCA document that controls the lifecycle of the component as references come and go
- Finally when all required references are satisfied (a dynamic process) we publish the services to be consumed by other components in the enterprise.
- Unpublish the services and release any references.
- Shut down the implementation and discard our runtime model.
- The bundle root counter is decremented, if the bundle root counter reaches zero then it is no longer required in the runtime and is marked as garbage.
- Finally garbage collect all bundles that are no longer in use, so clearing down the environment.
To achieve this we group composites together into an entity known as a System. A system is a group of composites which each specify a particular replication strategy and a set of contract requirements. A replication strategy is a dynamic pattern used by Newton to decide how many instances of a Composite should be installed at any given instant. A contract is a pattern used to restrict where a composite should be installed, i.e. a contract may specify a filter such as (&(machine.cpu.count>1)(machine.memory>1G)) i.e. only install on dual cpu or greater machines with more than 1GB of memory available.
So in the Newton world an application is actually a very dynamic entity. It is built from OSGi bundles and wired together on the fly. But it never the less has rules which are defined via a model (expressed in SCA plus our system extensions) which specify:
- how implementations and interfaces are connected
- how remote services should interact via bindings
- how they should scale
- where they should install
Laters,
Wednesday 2 July 2008
A New Hope
A long time ago in a galaxy far far away...
A group of rebels formed the OSGi Alliance, an alliance against the empire (or at least monolithic code empires). The alliance was created to champion an ideal (in CS terms a specification) which described a standardised way for modules of Java code in a single virtual machine to interoperate without requiring the JVM to be restarted.
The specification was simple to understand and many were drawn to it due to the elegance of its implementation. In time many who chose this new life found that despite the simplicity of the specification championed by the JEDi alliance they could nevertheless wield wondrous powers not seen before in the Java world.
Inevitably these powers drew the attention of those still living within the empire. Initially some sought to discredit the Jedi powers, either through misunderstanding or fear. But soon others became envious and wanted these powers for themselves.
Some within the empire chose to take the difficult road and started to convert their legacy methodologies to meet the requirements of the Jedi ideal. But others saw a different path, instead of converting their methodologies to conform to the Jedi ideals they decided to convert the Jedi ideals to fit their own methodologies.
Those in the Jedi council found themselves torn between the short term promises of wealth offered by the empire or sticking to their ideals and holding out for the long term riches of a truely flexible Java virtual machine architecture.
It is at this point in the story that we find our selves. Peter Kriens (Obi Wan?) has recently blogged about the choices facing the Jedi alliance and argues for the purist ideals to be upheld.
Myself I find myself acting as a trader (or possibly a smuggler - gonna argue for Han but you make your own judgements...) between these two worlds.
As a developer and architect of Infiniflow I have been directly involved with building a distributed computing infrastructure that seeks to embrace ways of the Force as championed by the Jedi alliance for a single JVM but across the enterprise.
Our driving goal that led us as a company to the Jedi way of life was acceptance within our architecture that things change, or to put it another way: there is no one rigid model of the enterprise but rather a series of iterations which encapsulate the functions required to service the current business requirements.
Whether you believe a word of this, having walked the boundary between the Jedi world and that of the Empire I am acutely aware of the problems associated with integrating tools built by these two communities.
My own gut feeling is that the Jedi should stick to their ideals and treat integration with the Empire as an edge case - i.e. certainly build up specifications but treat them as boundary conditions vs compromising the core ideals. In this way legacy methodologies can be incorporated but in such a way as to highlight the fact that it is just a stepping stone towards a new ideal.
When it is impossible to integrate a legacy pattern I'd argue that this is a point when we have to admit that Gödel was right - it is not always possible to please all of the people all of the time (I paraphrase but you get the point). You can always delegate legacy cases to a separate jvm and communicate remotely to the old world.
If the Jedi council compromise their core ideals for ill conceived or temporary solutions they risk sending out a mixed and confusing message to those who are new to this technology.
Padawan learning the ways of the Force for the first time are unlikely to appreciate the subtle differences between such abstract concepts as import-package and require-bundle. They will of course go for which ever is the simplest pattern to achieve their short term goals but this could well be at the expense of logical coherency within the OSGi specification.
Once stepping onto the path to the dark side it is very difficult to turn back and ultimately leads to ruin. (Or cool lightning powers - you decide)
I have to give credit to the fantastic blog posts of Hal Hildebrand for the Star Wars theme to this blog entry, whether this will become a common theme for my own posts I'm unsure but it was certainly fun to write.
Laters,
A group of rebels formed the OSGi Alliance, an alliance against the empire (or at least monolithic code empires). The alliance was created to champion an ideal (in CS terms a specification) which described a standardised way for modules of Java code in a single virtual machine to interoperate without requiring the JVM to be restarted.
The specification was simple to understand and many were drawn to it due to the elegance of its implementation. In time many who chose this new life found that despite the simplicity of the specification championed by the JEDi alliance they could nevertheless wield wondrous powers not seen before in the Java world.
Inevitably these powers drew the attention of those still living within the empire. Initially some sought to discredit the Jedi powers, either through misunderstanding or fear. But soon others became envious and wanted these powers for themselves.
Some within the empire chose to take the difficult road and started to convert their legacy methodologies to meet the requirements of the Jedi ideal. But others saw a different path, instead of converting their methodologies to conform to the Jedi ideals they decided to convert the Jedi ideals to fit their own methodologies.
Those in the Jedi council found themselves torn between the short term promises of wealth offered by the empire or sticking to their ideals and holding out for the long term riches of a truely flexible Java virtual machine architecture.
It is at this point in the story that we find our selves. Peter Kriens (Obi Wan?) has recently blogged about the choices facing the Jedi alliance and argues for the purist ideals to be upheld.
Myself I find myself acting as a trader (or possibly a smuggler - gonna argue for Han but you make your own judgements...) between these two worlds.
As a developer and architect of Infiniflow I have been directly involved with building a distributed computing infrastructure that seeks to embrace ways of the Force as championed by the Jedi alliance for a single JVM but across the enterprise.
Our driving goal that led us as a company to the Jedi way of life was acceptance within our architecture that things change, or to put it another way: there is no one rigid model of the enterprise but rather a series of iterations which encapsulate the functions required to service the current business requirements.
Whether you believe a word of this, having walked the boundary between the Jedi world and that of the Empire I am acutely aware of the problems associated with integrating tools built by these two communities.
My own gut feeling is that the Jedi should stick to their ideals and treat integration with the Empire as an edge case - i.e. certainly build up specifications but treat them as boundary conditions vs compromising the core ideals. In this way legacy methodologies can be incorporated but in such a way as to highlight the fact that it is just a stepping stone towards a new ideal.
When it is impossible to integrate a legacy pattern I'd argue that this is a point when we have to admit that Gödel was right - it is not always possible to please all of the people all of the time (I paraphrase but you get the point). You can always delegate legacy cases to a separate jvm and communicate remotely to the old world.
If the Jedi council compromise their core ideals for ill conceived or temporary solutions they risk sending out a mixed and confusing message to those who are new to this technology.
Padawan learning the ways of the Force for the first time are unlikely to appreciate the subtle differences between such abstract concepts as import-package and require-bundle. They will of course go for which ever is the simplest pattern to achieve their short term goals but this could well be at the expense of logical coherency within the OSGi specification.
Once stepping onto the path to the dark side it is very difficult to turn back and ultimately leads to ruin. (Or cool lightning powers - you decide)
I have to give credit to the fantastic blog posts of Hal Hildebrand for the Star Wars theme to this blog entry, whether this will become a common theme for my own posts I'm unsure but it was certainly fun to write.
Laters,
Tuesday 10 June 2008
To Be(*) Or Not To Be(*) That Is The Question
(*) Included in an API bundle.
There's been a lively debate on the OSGi mailing list over the past couple of weeks surrounding the issue of whether an API should be packaged with it's implementation in one bundle or whether it should be split up into two bundles (one for API and one for implementation).
I think it's fair to say there are a range of opinions on the subject. What is clear however is that there is no one size fits all answer to this question. As a developer or architect you need to consider the use case your API and implementation are going to be put to in order to make the "correct" decision.
The fundamental issue that everyone involved in the discussion agreed on is that using a bundle should be as simple as possible. The difference of opinions comes in depending on what measure of simplicity you take to be most important. I think there are three main themes that have so far been discussed:
I'll try to give a quick overview.
A bundle can contain classes (like any standard Java jar). However it can also import classes from other bundles via a number of different import mechanisms. Importing classes allows bundles to be modularized such that different bundles provided different sets of functionality but still declare their dependencies in order to run. In order to use classes from a bundle all import requirements must be satisfied. In order to satisfy an import requirement a bundle exporting that requirement must be installed in the OSGi runtime.
So if BundleA depends on classes exported from BundleB you have to install both BundleA and BundleB in order to use BundleA. If BundleB depends on classes from another bundle you need to install those bundles too. You can probably see that this problem can rapidly diverge from being a single bundle install to being a large complicated set of bundle installs.
Another important detail to be considered is that in OSGi it is possible for many bundles to export the same API package and only one will be picked by the runtime to actually provide the classes.
When we discuss splitting API and implementation we are saying that one bundle will export the API classes and another bundle will import that API and provide an implementation for it. When we talk about grouping API and implementation we mean that the API will be exported from the same bundle that provides the implementation. Many implementations may export the same API but this is OK in an OSGi framework.
My own advice would be to start by assuming that API and implementation are packaged in separate bundles. The reasoning behind this is based on the following criteria:
Hopefully that's helped some people understand the issues.
Laters
There's been a lively debate on the OSGi mailing list over the past couple of weeks surrounding the issue of whether an API should be packaged with it's implementation in one bundle or whether it should be split up into two bundles (one for API and one for implementation).
I think it's fair to say there are a range of opinions on the subject. What is clear however is that there is no one size fits all answer to this question. As a developer or architect you need to consider the use case your API and implementation are going to be put to in order to make the "correct" decision.
The fundamental issue that everyone involved in the discussion agreed on is that using a bundle should be as simple as possible. The difference of opinions comes in depending on what measure of simplicity you take to be most important. I think there are three main themes that have so far been discussed:
- installation simplicity - minimizing the number of bundles needed to run an OSGi application
- runtime simplicity - minimizing the connectivity between bundles in a running OSGi application.
- architectural simplicity - minimizing duplicated packages between bundles used to build an OSGi application
I'll try to give a quick overview.
A bundle can contain classes (like any standard Java jar). However it can also import classes from other bundles via a number of different import mechanisms. Importing classes allows bundles to be modularized such that different bundles provided different sets of functionality but still declare their dependencies in order to run. In order to use classes from a bundle all import requirements must be satisfied. In order to satisfy an import requirement a bundle exporting that requirement must be installed in the OSGi runtime.
So if BundleA depends on classes exported from BundleB you have to install both BundleA and BundleB in order to use BundleA. If BundleB depends on classes from another bundle you need to install those bundles too. You can probably see that this problem can rapidly diverge from being a single bundle install to being a large complicated set of bundle installs.
Another important detail to be considered is that in OSGi it is possible for many bundles to export the same API package and only one will be picked by the runtime to actually provide the classes.
When we discuss splitting API and implementation we are saying that one bundle will export the API classes and another bundle will import that API and provide an implementation for it. When we talk about grouping API and implementation we mean that the API will be exported from the same bundle that provides the implementation. Many implementations may export the same API but this is OK in an OSGi framework.
My own advice would be to start by assuming that API and implementation are packaged in separate bundles. The reasoning behind this is based on the following criteria:
- In general an implementation is likely to depend on more packages than it's API
- You can always collapse back to one bundle later if you use import-package vs require-bundle
- If you use a provisioning mechanism such as Newton or P2 (when it's ready) downloading two bundles vs one is handled automagically
- If you are considering making your application distributed or want it to run in a constrained environment you can install the API without having to resolve the implementation dependencies (possibly a big deal in a client/server architecture)
- If you want to restart or reinstall the implementation bundle then this doesn't automatically mean the restart of all client bundles that are using the API from that bundle
- OSGi export-package has a uses attribute - to specify classes that the exported API has gained from imports. It is possible for combinations of exports and imports to cause bundles to be mutually exclusive - such that it is impossible to close the graph and install all bundles in the same OSGi runtime at the same time. Limiting the connectivity in the graph via separating API from implementation reduces the risk of running into this problem.
- Start by designing your API to depend on as little as possible.
- Make your implementation depend on API packages and any other dependencies it needs to function.
- If after doing this the implementation depends only on API consider whether the implementation is ever likely to get more complicated.
- If it isn't then you can probably collapse back to one bundle.
Hopefully that's helped some people understand the issues.
Laters
Sunday 8 June 2008
Software Versioning Is Complex Or Am I Imagining It?
So it seems everyone and his dog are talking about versioning at the moment. Specifically the proposed version numbering systems used in OSGi and JSR 277 and their ability to coexist (or not).
For my own part I personally favor the OSGi scheme because to my mind it is simpler and better defined.
The OSGi scheme says that there are three numbers used in a version, major.minor.micro -> 1.2.1. The micro version is incremented when ever there are bug fixes. The minor version is incremented when there are backwards compatible additions and the major version is incremented when there are any non backwards compatible changes. It is also possible to add postfix elements to an OSGi version i.e. 1.2.1_20080608 (todays date appended). These postfix elements are considered lexicographically to mean increments/improvements which do not warrant a version update - i.e. nightly builds etc.
When a module consumer wishes to specify a version they wish to use they specify either a single version number.
1.0.0
or a version range
[1.0.0,2.0.0)
The first version means any version greater or equal to 1.0.0. The second means any version between 1.0.0 and any version prior to 2.0.0.
However anyone who has worked with software development for any non trivial amount of time will know there are still inherent problems in versioning schemes in that they require developers to correctly markup when changes to their code effect the compatibility of that code.
This is a big deal as depending on the consumer of the code even changes as innocuous as a java doc comment update can effect the version number. There's a really good discussion of the issues surrounding this here.
As we all know developers are mostly human and so; make mistakes, are lazy or are just plain forgetful so it is inevitable that version numbers will sometimes not be updated when they should have been.
After release these badly versioned modules (poison pills) will sit around in software repositories for potentially a very long time causing instabilities in the systems that use them.
Stanly Ho's suggestion for combating this issue is to allow consumers of modules to specify complex lists which select certain version ranges but exclude certain known bad versions.
Like many other aspects of his proposal I think this is going to be overly complex and unmanageable in any practical environment. It effectively means build managers have to rebuild their module every time they find a bad module which is a lot of extra work, let alone how the build manager then propagates his/her updated module to all clients.
OK so what's the point I'm getting to? Well during discussions with colleagues it occurred to me there is a relatively simple extension we could make to version numbering schemes that nips off a lot of the problems at the start of the development life cycle vs at the end.
This is a good thing as if it is easier for developers to handle version numbering there should be fewer bad versions.
I'm not saying there will be no bad versions as there are always the unintended API changes which slip past quality control etc but I would argue these are the exception vs the rule in a well managed project. Any bad versions that do slip out could be highlighted and deleted from the collective repositories by administrators. But given a smaller number of mistakes this problem should be tractable.
I'll prefix the rest of this blog entry by saying my idea is a little crazy so please bare with me, pat me on the head and say there, there back to the grind stone you'll get over it :)
So enough pontificating, the idea:
It occurred to me that software versions should really carry an imaginary coefficient (i.e. square root of minus one).
[Sound of various blog readers falling off their seats]
...
What the hell does that mean I here you ask.
I said it was crazy and I'm only half proposing it as a real (chuckle) solution. However to my mind it seems more natural to label versions intended as release candidates or as development code as having an imaginary coefficient.
The reasoning behind this is that whilst development code is tending towards a real API or implementation there are always going to be aspects to it that is not quite finished and so can be considered imaginary. I think a useful way of visualising this is that as software approaches release the imaginary coefficient gets smaller, i.e. there is a rotation in the software from the imaginary plane to the real.
Ok so just because it has a nice analogy doesn't make it valid, how does this help in real world software development?
Firstly this scheme does not effect existing versioning semantics, if you want to work purely in the real plane then that is perfectly acceptable. So it doesn't break existing OSGi bundle versions in the wild today. However, it does give developers a way to markup whether they want their code to interact with code that is still in development.
Imagine a case where software producer A (providing module A) is building release candidates and software consumer B is building module B that depends on module A:
ModuleA:1.0.0_RC1
ModuleA:1.0.0_RC2
ModuleA:1.0.0_RC3
etc.
ModuleB:import ModuleA:1.0.0
In the current scheme there is no way to distinguish release candidates from final code and incremental patches so when producer A builds his release candidate consumer B sees each release candidate as actually being more advanced than the final 1.0.0 release.
This is clearly wrong.
As software engineers we tend to manage this internally by having a succession of 0.9, 0.9.1, 0.9.2, etc releases prior to the 1.0.0 RC releases so minimizing the number of RC candidates that ever get released and making it relatively simple to tidy up any that do accidentally make it into live environments.
However this breaks the simple rules set up in the OSGi scheme in that there is a discontinuity between 0.8 and 0.9 that may be non backwards compatible. We then have to employ all sorts of artificial rules during development to say import [0.0.0,0.9.0) but this needs to be applied to all modules after we have decided to make 0.9 the non backwards compatible internal release.
Instead I propose that when producer A starts work on a set of changes to module A he increments the appropriate version element (depending on the scale of the changes he/she is making) early and adds an imaginary coefficient to mark it in development.
Therefore from the previous example we have
ModuleA:1.0.0_3i (RC1)
ModuleA:1.0.0_2i (RC2)
ModuleA:1.0.0_1i (RC3)
As an external consumer of module A we are then able to use [1.0.0,2.0.0) to mean all compatible increments. An internal consumer prior to release can say [1.0.0_2i,2.0.0)
to mean all releases of module A after RC2. Importantly this will continue to work after release with no need to update existing imports.
But (I here you say) how do we know how far away from zero to start when starting the imaginary coefficient. I would argue that this can be handled in the same way as real versions i.e. 1.0.0_0.1i is closer to release than 1.0.0_1i and 1.0.0_0.0.1i is closer still. So this issue is likely project or problem specific - start where ever you like.
We could come up with a scheme where by the major.minor.micro elements of the imaginary coefficient denote degrees of confidence as to how closely the code matches the proposed design - i.e. major = zero -> developer tested, minor = zero -> system tested, micro = zero -> beta tested etc.
The notion of complex version numbers applies equally to the JSR 277 four number version scheme which to my thinking is a completely pointless addition to a spec which does nothing to address the actual problem and breaks lots of existing code that was previously working fine.
I'd be very happy for someone to come along and state why imaginary version numbers are not needed as in general I prefer to reuse existing tools where possible and ideally the simplest tool that does the job.
So if nothing else this is a vote for OSGi versioning but with a couple of notes on how we may be able to improve it.
Laters
Update: 09/06/2008
Apologies I linked to the second part of the eclipse article on version updates by mistake the correct link is here.
For my own part I personally favor the OSGi scheme because to my mind it is simpler and better defined.
The OSGi scheme says that there are three numbers used in a version, major.minor.micro -> 1.2.1. The micro version is incremented when ever there are bug fixes. The minor version is incremented when there are backwards compatible additions and the major version is incremented when there are any non backwards compatible changes. It is also possible to add postfix elements to an OSGi version i.e. 1.2.1_20080608 (todays date appended). These postfix elements are considered lexicographically to mean increments/improvements which do not warrant a version update - i.e. nightly builds etc.
When a module consumer wishes to specify a version they wish to use they specify either a single version number.
1.0.0
or a version range
[1.0.0,2.0.0)
The first version means any version greater or equal to 1.0.0. The second means any version between 1.0.0 and any version prior to 2.0.0.
However anyone who has worked with software development for any non trivial amount of time will know there are still inherent problems in versioning schemes in that they require developers to correctly markup when changes to their code effect the compatibility of that code.
This is a big deal as depending on the consumer of the code even changes as innocuous as a java doc comment update can effect the version number. There's a really good discussion of the issues surrounding this here.
As we all know developers are mostly human and so; make mistakes, are lazy or are just plain forgetful so it is inevitable that version numbers will sometimes not be updated when they should have been.
After release these badly versioned modules (poison pills) will sit around in software repositories for potentially a very long time causing instabilities in the systems that use them.
Stanly Ho's suggestion for combating this issue is to allow consumers of modules to specify complex lists which select certain version ranges but exclude certain known bad versions.
Like many other aspects of his proposal I think this is going to be overly complex and unmanageable in any practical environment. It effectively means build managers have to rebuild their module every time they find a bad module which is a lot of extra work, let alone how the build manager then propagates his/her updated module to all clients.
OK so what's the point I'm getting to? Well during discussions with colleagues it occurred to me there is a relatively simple extension we could make to version numbering schemes that nips off a lot of the problems at the start of the development life cycle vs at the end.
This is a good thing as if it is easier for developers to handle version numbering there should be fewer bad versions.
I'm not saying there will be no bad versions as there are always the unintended API changes which slip past quality control etc but I would argue these are the exception vs the rule in a well managed project. Any bad versions that do slip out could be highlighted and deleted from the collective repositories by administrators. But given a smaller number of mistakes this problem should be tractable.
I'll prefix the rest of this blog entry by saying my idea is a little crazy so please bare with me, pat me on the head and say there, there back to the grind stone you'll get over it :)
So enough pontificating, the idea:
It occurred to me that software versions should really carry an imaginary coefficient (i.e. square root of minus one).
[Sound of various blog readers falling off their seats]
...
What the hell does that mean I here you ask.
I said it was crazy and I'm only half proposing it as a real (chuckle) solution. However to my mind it seems more natural to label versions intended as release candidates or as development code as having an imaginary coefficient.
The reasoning behind this is that whilst development code is tending towards a real API or implementation there are always going to be aspects to it that is not quite finished and so can be considered imaginary. I think a useful way of visualising this is that as software approaches release the imaginary coefficient gets smaller, i.e. there is a rotation in the software from the imaginary plane to the real.
Ok so just because it has a nice analogy doesn't make it valid, how does this help in real world software development?
Firstly this scheme does not effect existing versioning semantics, if you want to work purely in the real plane then that is perfectly acceptable. So it doesn't break existing OSGi bundle versions in the wild today. However, it does give developers a way to markup whether they want their code to interact with code that is still in development.
Imagine a case where software producer A (providing module A) is building release candidates and software consumer B is building module B that depends on module A:
ModuleA:1.0.0_RC1
ModuleA:1.0.0_RC2
ModuleA:1.0.0_RC3
etc.
ModuleB:import ModuleA:1.0.0
In the current scheme there is no way to distinguish release candidates from final code and incremental patches so when producer A builds his release candidate consumer B sees each release candidate as actually being more advanced than the final 1.0.0 release.
This is clearly wrong.
As software engineers we tend to manage this internally by having a succession of 0.9, 0.9.1, 0.9.2, etc releases prior to the 1.0.0 RC releases so minimizing the number of RC candidates that ever get released and making it relatively simple to tidy up any that do accidentally make it into live environments.
However this breaks the simple rules set up in the OSGi scheme in that there is a discontinuity between 0.8 and 0.9 that may be non backwards compatible. We then have to employ all sorts of artificial rules during development to say import [0.0.0,0.9.0) but this needs to be applied to all modules after we have decided to make 0.9 the non backwards compatible internal release.
Instead I propose that when producer A starts work on a set of changes to module A he increments the appropriate version element (depending on the scale of the changes he/she is making) early and adds an imaginary coefficient to mark it in development.
Therefore from the previous example we have
ModuleA:1.0.0_3i (RC1)
ModuleA:1.0.0_2i (RC2)
ModuleA:1.0.0_1i (RC3)
As an external consumer of module A we are then able to use [1.0.0,2.0.0) to mean all compatible increments. An internal consumer prior to release can say [1.0.0_2i,2.0.0)
to mean all releases of module A after RC2. Importantly this will continue to work after release with no need to update existing imports.
But (I here you say) how do we know how far away from zero to start when starting the imaginary coefficient. I would argue that this can be handled in the same way as real versions i.e. 1.0.0_0.1i is closer to release than 1.0.0_1i and 1.0.0_0.0.1i is closer still. So this issue is likely project or problem specific - start where ever you like.
We could come up with a scheme where by the major.minor.micro elements of the imaginary coefficient denote degrees of confidence as to how closely the code matches the proposed design - i.e. major = zero -> developer tested, minor = zero -> system tested, micro = zero -> beta tested etc.
The notion of complex version numbers applies equally to the JSR 277 four number version scheme which to my thinking is a completely pointless addition to a spec which does nothing to address the actual problem and breaks lots of existing code that was previously working fine.
I'd be very happy for someone to come along and state why imaginary version numbers are not needed as in general I prefer to reuse existing tools where possible and ideally the simplest tool that does the job.
So if nothing else this is a vote for OSGi versioning but with a couple of notes on how we may be able to improve it.
Laters
Update: 09/06/2008
Apologies I linked to the second part of the eclipse article on version updates by mistake the correct link is here.
Saturday 7 June 2008
A JSR To Far?
So there's a lot of conversation going on around the both the politics and the technological issues surrounding JSR 277. Personally I think the spec is doomed if it doesn't start working much more closely with the OSGi community - and here's why.
The OSGi community now comprises almost all the major software vendors who have been expending significant effort working on the OSGi specifications and OSGi enabling their applications over a number of years. The JSR is currently stalled and taking advice predominantly from Sun internal researchers.
If JSR 277 makes it into the Java 7 release then it seems entirely plausible that the major vendors could choose not to certify their products on Java 7 (especially if the JSR gets in the way of their existing investment in OSGi).
Where would this leave Sun if the likes of Websphere, Weblogic, Spring, etc, etc are not certified to run in an enterprise environment? Also where does it leave Java?
Disclaimer, these are my own interpretations of the situation and I have had no conversations with any of the protagonists leading me to this assessment. I'm just concerned that politics and egos seem to be getting in the way of one of the most important evolutions of the Java eco system which could be a disaster for all involved in Java.
Please guys, set egos aside and come to a sensible decision that allows us all to get on with our day jobs.
The OSGi community now comprises almost all the major software vendors who have been expending significant effort working on the OSGi specifications and OSGi enabling their applications over a number of years. The JSR is currently stalled and taking advice predominantly from Sun internal researchers.
If JSR 277 makes it into the Java 7 release then it seems entirely plausible that the major vendors could choose not to certify their products on Java 7 (especially if the JSR gets in the way of their existing investment in OSGi).
Where would this leave Sun if the likes of Websphere, Weblogic, Spring, etc, etc are not certified to run in an enterprise environment? Also where does it leave Java?
Disclaimer, these are my own interpretations of the situation and I have had no conversations with any of the protagonists leading me to this assessment. I'm just concerned that politics and egos seem to be getting in the way of one of the most important evolutions of the Java eco system which could be a disaster for all involved in Java.
Please guys, set egos aside and come to a sensible decision that allows us all to get on with our day jobs.
Subscribe to:
Posts (Atom)