OSGi Dev Con 2009 & OSGi Tooling Summit Roundup

So I'm currently sitting in my hotel lobby waiting till we jump in the hire car off to the airport for our flight home. It's been a long week - the time zone difference between San Francisco and London is always exhausting and the mental effort of networking, presenting, and talking to customers is always pretty intensive too.

I think the conference was definitely quieter this year - you couldn't help but feel the shadow of the economic climate hanging over us all - however though volume was down compared to last year, quality seemed to be much higher. My rough market assessment based on talking to people on our exhibition stand was that awareness of OSGi is much improved compared to last year.

Even better for me people really seemed to get what we (Paremus) are about. Last year we were the "RMI guys". This year people we talked to seemed to get genuinely excited about what our product is a really about: a flexible, scalable solution to provisioning and managing dynamic distributed OSGi based applications in enterprise environments.

Pretty much everyone we talked to was very positive on OSGi as a runtime solution to Java modularisation. However, the problem for its adoption in the enterprise is the backwards compatability issues in moving to an OSGi environment (things like static references and Class.forName are a really bad idea in OSGi) and the fact that OSGi development relies on a reasonable amount of domain specific knowledge.

I think good tooling solutions that work right the way though the stack are crucial to help new developers though the pitfalls of the new classloader space. Unfortunately, tooling support for new developers is pretty disjointed.

Hence, the next part of my post...

We (Paremus) recognised that it does us no good if customers are having difficulty building OSGi based solutions. Therefore, my team mate Derek Baum and I have been working on a solution to developing OSGi bundles which works in both an IDE and headless build environment called Sigil. Prior to OSGi Dev Con Paremus chose to licence Sigil under the Apache licence, as we recognise that tooling is an area where we need support from the community in order to help the community as a whole.

On the Friday, after the end of the conference, I and a number of other representatives with interests in the area of development tooling met at an OSGi Tooling Summit hosted by Yan Pujante at LinkedIn's Mountain View offices. The group was pretty large and diverse (as you can see here). The OSGi Alliance's typical role in this respect is to foster communication (via summits, conference calls etc), then generate one or more RFPs which list a set of requirements for the problem domain, and finally put together a set of RFCs which provide a spec to allow different vendors to work together.

The format of the meeting was an initial intro and positioning statements (i.e. what we were there to achieve). We then moved into a blue sky discussion where we tried to think of a perfect world solution to the OSGi development lifecycle - this stimulated some really interesting conversation which I'll doubtless follow up on in the next couple of weeks in future blog posts. Finally, we put together a series of use cases headers that captured the major tasks undertaken by developers when building, maintaining and releasing OSGi applications into the wild.

I had a number of really encouraging conversations with Chris Aniszczyk and Peter Kriens who work on PDE and BND respectively, both of which have a lot of cross over with the work I've been doing on Sigil.

Chris has just twittered that he's thinking of changing the name of PDE to BDE which I think is a great idea - perhaps some merging of BDE, Sigil and BND? I think there is likely some common core classes that are useful to all OSGi developers. We're obviously going to need some domain specific extensions for plugin development, newton development, spring development, etc. but I see these as addons on top of a common OSGi layer.

I guess in a perfect world I'd like to be able to support Maven, Netbeans and IntelliJ users as well. Hopefully I'll be able to update you in the next couple of months on progress in this area.

Laters,

Flocking Behaviours

Just a quick note to say I'll be running a BOF session with Peter Kriens and Chris Aniszczyk on OSGi development tooling at EclipseCon this year as part of the OSGi track. You can find details here.

I guess my main focus for calling the BOF is that I'm very interested in talking to other OSGi developers to see what it is we on the tooling side can do to make our collective jobs easier.

What is it about OSGi development that really frustrates you - and what can tools do to make it easier?

It'll also be really interesting to meet up with other tools developers to see if we can get a bit of reuse going on - likely we won't solve that in one BOF but if we can just put names to faces and get a bit of shared understanding going on we're likely to be more productive in the long term.

Hope to see you there.

Laters,

OSGi tooling

There's a really interesting conversation going on at TSS about OSGi and future directions for Enterprise Java.

I've posted a reply which I thought it was worth reposting here:

I think there are two issues with [the approach of repackaging existing modules as OSGi bundles and simply importing/exporting all packages] which really cause headaches going forward; Module vs API dependencies and complex "Uses" graphs.

Firstly module vs api; in most dependency tools such as Maven and Ivy the developer specifies dependencies at the module layer - i.e.

<dependency org="org.apache.log4j" name="org.apache.log4j" rev="1.2.15" />

But then spring have added an OSGi version of the module which has a different module id.

<dependency org="org.apache.log4j" name="com.springsource.org.apache.log4j" rev="1.2.15" />

How does the tooling know that these two modules are actually compatible? It can't and this tends to lead to false dependency failures. This is the same problem as using module dependencies in OSGi at runtime. Actually the code is not dependent on the module but on the API that the module supplies.

Having been working with OSGi as part of our [1] distributed OSGi platform for almost 4 years now, I came to the conclusion that the tooling should use the notion of package import at build time - just as OSGi does at runtime. This is the approach I've taken in Sigil [2] which we're now using to build our next product release.

Having taken the jump to use API dependencies our build system is now much simpler to manage. Instead of having to manually lookup which module supplies a package, I just type the package into the IDE and our tooling goes off and trawls the OSGi manifest information in various repositories to find a module that supports it (seemless). We've also used this on a number of third party applications (the record being a project with over 600 modules - which we managed to convert to use sigil in less than a couple of hours based on pretty much a simple grep of the java code for java import statements).

Secondly the major thorn in the approach of naively exporting all packages in a module is the complex "uses" information it generates. "Uses" is a flag provided by OSGi on exports to assert that the class space is coherent across multiple bundle import/export hops.

It is currently believed that the "uses" information makes the problem of resolving bundle dependencies NP-complete. As the number of bundles and import/exports increases the number of calculations the OSGi runtime has to do to check that all bundles can sit together goes up at a terrifying rate.

I've referred to this as placing barbed wire around sand castles (in most cases). If the modules were more sensibly designed i.e. only exporting the "really" public code then this problem is much reduced.

I'm also attending the summit Jason refers to and I've called a BOF "OSGi Development Tooling" with Peter Kriens and Chris Aniszczyk at the OSGiDevCon conference just prior to the summit to discuss tooling efforts w.r.t. OSGi. The BOF is open to the general public and it would be great to hear opinions before we sit down at the summit to figure out a path forwards.

[1] http://www.paremus.com
[2] http://sigil.codecauldron.org