OSGi & The Cloud (Part 2)

This is the second blog entry in a series documenting the underlying points I made in my recent talk at the OSGi Community Event in London.  Entitled "OSGi And Private Cloud",  the slides are available here and the agenda is as follows:

• Where is Cloud computing today? (Part 1)
• Where does OSGi fit in the Cloud architecture?
• What are the challenges of using OSGi in the Cloud?
• What does an OSGi Cloud platform look like?

In this section of the talk I look at where OSGi fits into the Cloud architecture. However, as the community event was co-hosted with JAX London it wasn't a given that everyone at my talk would know OSGi. This is also possibly true for others reading this blog, so to make sure we're all starting from a similar page, I'll briefly explain the basics of what OSGi is about for those who have not come across it before.

OSGi A Quick Review

I've been working with Richard Hall, Karl Pauls and Stuart McCulloch on writing OSGi In Action which explains OSGi from first principles to advanced use cases, so if you want to know more that's a good place to look. However, here I'd like to give my elevator pitch for OSGi which would be something like as follows...

OSGi first became a standard in 1999 and provides a set of specifications for building dynamic modular Java applications. It has success stories in every area of Java development from embedded devices, though desktop applications to enterprise applications. The core features that OSGi provides a Java application are:

• Modules - the building blocks from which to create applications
• Life cycle - control when modules are installed or uninstalled and customise their behaviour when they are activated
• Services - minimal coupling between modules

You might say that none of these are new ideas, so why is OSGi important? The key is in the standardisation of these fundamental axioms of Java applications. Instead of every software stack having a new and inventive way of wiring classloaders together, booting components, or connecting component A to component B, OSGi provides a minimal flexible specification that allows us to get interoperability between modules and let developers get on with the interesting part of building applications.

An Uncomfortable Truth

To see where OSGi fits into the Cloud story it's worth taking a brief segue to consider a point made by Kirk Knoernschild at the OSGi community event in February this year. Namely that we are generating more and more code with every passing day:

• Lines of code double every 7 years
• 50% of development time spent understanding code
• 90% of software cost is maintenance and evolution

By 2017, we'll have written not only double the amount of code written in the past 7 years but more than the total amount of code ever written combined! Object Orientation has helped in encapsulating our code so that changes in private implementation details do not effect consumers. But in fact OO turns out to be just a stop gap and it is reaching the limits of its capabilities. If you refactor public objects or methods you still need to worry about who is consuming these and without modules this can be a hard question to answer.

Eric Newcomer of Credit Suisse gave another good talk at the recent community event on the scale of software development at the bank. The message I took away from this presentation is that within any large organisation, one can probably locate an example of virtually any computer algorithm ever conceived (in fact if you look hard enough you will more than likely find two). If we look out to small and medium sized organisations, sure they won't have pre-canned examples but any developer worth his salt can probably; knock up some approximation within a couple of days, find an open source library to do the same job, or part with some money to a vendor to get the job done.

The message from these two presentations is, that as we move into the era of Cloud computing the real problem is not how to author code but how to manage and reuse code and to do so at scale. As businesses grow and Cloud makes hardware cheaper and cheaper to use, market competition is driving computer software to larger and larger scales to cope with increased processing, network and storage volumes. This scaling tends to lead to more complexity and often in an exponential relationship.  But what do I mean by scale when talking about software in the Cloud? And how do we tame the complexity versus scale curve?

Types of Scale

There are three measures of scale that I think are of relevance to this discussion of OSGi and the Cloud:

• Operational scale - the number of processors, network interfaces, storage options required to perform a function
• Architectural scale - the number and diversity of software components required to make up a system to perform a function
• Administrative scale - the number of configuration options that our architectures and our algorithms generate

In fact, I think we've got pretty good patterns by now for dealing with the operational scale. As we increase the number of physical resources at our disposal, this drives the class of software algorithms required to perform a function. To pick a random selection Actors, CEP, DHTs and Grid are just some of the useful software patterns for use in the Cloud. However, I think architectural and administrative scale is often less well managed.

In terms of architectural scale, think about all the libraries we have to perform similar functions; logging, data access layers, RPC frameworks, web frameworks. How many of the millions of lines of code that we are generating are boilerplate copies of each other? Redundant architecture is a major problem in the growth of software. Which parts are really providing value to the business? Which parts are harmless clones of each other? Which parts are a maintenance cost that should be replaced? We employ abstractions to protect ourselves from underlying implementations but these abstractions can themselves become maintenance costs. I would argue that as software engineers we are suffering from the paradox of choice.

When managing code we need to worry about updating code, as code is very rarely a static entity; bugs are fixed, new APIs are created, old ones are deprecated and removed. With the volume of code in existence, we need mechanisms to manage the complexity created by the constant churn of logic that makes up our business systems. This leads us onto the problems of administrative scale.
 
Administrative scale hampers our ability to reason about and evolve deployed systems. The human brain has evolved to deal with relatively small connected graphs. But software today consists of multiple configuration options - libraries that implement APIs, network configurations, storage configurations, queue depths, the list is endless. When we look at the interconnected nature of many software architectures, do we really know what the impact of changing parts of the configuration will have?

All this brings me to...

OSGi Cloud Benefits

In Part 1 of this series of blogs I mentioned that the Nist definition of a cloud includes the statement that: "Cloud software takes full advantage of the cloud paradigm by being service oriented with a focus on statelessness, low coupling, modularity and semantic interoperability", to my mind OSGi has these bases covered.

OSGi is a specification for modular Java that encourages low coupling via the use of services and it certainly allows you to build stateless applications. OSGi also promotes semantic interoperability via the fact that the code runs in a JVM and is abstracted from the underlying platform. Higher order levels of interoperability can easily be enabled using API and implementation modules that provide service abstractions around common platform functions, such as accessing data or scheduling tasks.

But why should cloud software have these features?

Returning to my theme of scale and complexity from the previous section, modularity and service orientated architectures enable encapsulation of coherent components to help reduce architectural complexity. Semantic interoperability aids in the war against administrative complexity as the same code can run no matter what hardware or network environment it is deployed in. Finally, stateless architectures are just a good design goal for dealing with production scale.

OK interesting, but you might say that "TechnologyX (pick your favourite) can also provide these features, so really sell me on the OSGi cloud benefits". In which case I propose that there are four additional benefits of OSGi with respect to Cloud software which I'll deal with in turn:

Dynamic: Clouds can be tempestuous environments with latency and contention being major factors. In these sorts of environments software that is designed to cope with runtime dependencies coming and going is more robust than static architectures. Consider the analogy with civil engineering and bridge or aeroplane wing design - rigid architectures are more fragile than those that incorporate degrees of flexibility. The Remote Services chapter from OSGi 4.2 specification promotes a discovery based services API. Thus, if a service dependency is lost due to movements in the Cloud then client code does not go into a spin making socket connect timeouts, it just gracefully moves into what ever state makes most sense and the rest of the processing can continue.

Extensible: Clouds are all about expansion so following good XP principles when you start a new project you should only develop the parts of the application you actually know are needed. However, versioned module dependencies and service interfaces mean that you can easily abstract or update simple implementations as the application demands grow.

Lightweight: Clouds are meant be light, right? OSGi promotes modular design.  Modular designs in turn allow you to tune a software deployment to the actual task at hand. OSGi lifecycle and dynamic services patterns even allow us to extend an application at runtime. This enables all sorts of interesting new use cases, for example:

• if you need to get diagnostics information out of the software, only deploy the diagnostics components for the time that they are needed - for the rest of the time run lean
• if you need to scale up a certain component's processing power, swap an in-memory job queue for a distributed processing queue and when you're done swap it back again.

Self describing: OSGi bundles contain a description of the module in their jar manifest files. This helps in the war against administrative complexity, notably via automation and audit. Just as an OSGi framework can validate that a bundle has been deployed with all its necessary dependencies, it is also possible to reverse this process and download required dependencies automatically. There are several implementations of this pattern already in use in OSGi today; Nimble, OBR and P2. This simplifies deployments by allowing software engineers to focus on what they want to deploy instead of what they need to deploy.

In terms of audit, OSGi bundles have a number of standardised headers to describe meta features such as name, description, license and documentation, so if you want to find out about a piece of software in a bundle just look at its manifest information. Once you get into this frame of thinking, other meta data such as author, build date, business unit also make sense to be embedded into the bundle. This sort of meta information can greatly benefit system admins and system builders in the future.

OSGi Cloud Services

To conclude this post, assuming I've managed to convince you of the benefits of OSGi in Cloud architectures, here are some ideas for potential cloud OSGi services (definitely non exhaustive):

• MapReduce services - Hadoop or Bigtable implementations?
• Batch services - Plug and play Grids?
• NoSQL services - Scalable data for the Cloud!
• Communications services - Email, Calendars, IM, Twitter?
• Social networking services - Cross platform widgets?
• Billing services - Making money in the Cloud!
• AJAX/HTML 5.0 services - Pluggable UI architectures?

These would enable developers to start building modular, dynamic, scalable applications for the Cloud and are in fact pretty simple to achieve if there's the will power to make it happen.

I think OSGi provides an excellent foundation for building Cloud software. There are things it doesn't do but its extensible nature means that it is very easy to build additional tools on top of it and really start addressing the problems of scale. I'll look at some of tools I've been working on in this area in the final post.

So all good right? Well there are still of course challenges, so in the next post I'll look at some of these and discuss how to overcome these. In the meantime, I'm very interested in any feedback on the ideas found in this post.

Laters

OSGi & The Cloud (Part 1)

I recently attended the OSGi Community Event where I gave a talk entitled "OSGi And Private Cloud" the slides for which are available here. However as has been pointed out, if you watch the slide deck they're a little on the zen side, so if you weren't at the event then it's a bit difficult to guess the underlying points I was trying to make.

To address this I've decided to create a couple of blog entries that discuss the ideas I was trying to get across. Hopefully this will be of interest to others.

In the talk the agenda was as follows:

• Where is Cloud computing today?
• Where does OSGi fit in the Cloud architecture? (Part 2)
• What are the challenges of using OSGi in the Cloud?
• What does an OSGi cloud platform look like?

I'll stick to this flow but break these sections up into separate blog entries. So here goes with the first section...

Where is Cloud computing today?

Ironically Cloud computing is viewed by many as a pretty nebulous technology so before even describing where Cloud computing is, it's possibly useful to define what Cloud computing is.

• Wikipedia defines a Cloud as: "Internet-based computing, whereby shared resources, software, and information are provided to computers and other devices on demand, like the electricity grid".
• InfoWorld defines Cloud as: "[sic] a way to increase capacity or add capabilities on the fly without investing in new infrastructure, training new personnel, or licensing new software. Cloud computing encompasses any subscription-based or pay-per-use service that, in real time over the Internet, extends IT's existing capabilities".
• NIST defines (see the "NIST Definition of Cloud Computing" link) a Cloud as: "a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction"

For me all of these definitions seem pretty similar to Utility computing. Again wikipedia defines Utility computing as "[sic] the packaging of computing resources, such as computation, storage and services, as a metered service similar to a traditional public utility (such as electricity, water, natural gas or telephone network)". So what is the boundary between Utility computing and Cloud computing? Others have attempted to define Cloud by what it is not. I tend to agree with some of these points but not others.

So where does this leave us…?

Actually I think the NIST definition I referred to above does a good job of describing what Cloud is as long as you read past the first sentence. For me the summary of this document is that:

Cloud is computation that is: on demand; easily accessed from a network; with pooled resources; and rapid elasticity.

As a final foot note in the NIST document it mentions that:

"Cloud software takes full advantage of the cloud paradigm by being service oriented with a focus on statelessness, low coupling, modularity and semantic interoperability".

This last sentence for me is of fundamental importance when considering the relevance OSGi to the Cloud.

I'm tempted to break off from the flow of the slides here and leap to the conclusion but I've set my self a goal of explaining my presentation. So before I explore this point further I'll continue with the slide deck as presented but rest assured this point will be returned to.

Why Cloud?

So as the previous discussion suggests, the reason for using a Cloud model is it gives users just in time:

• Processing power
• Storage capacity
• Network capacity.

Clouds models are great for small, medium and large organisations.

• Small organisations benefit from the reduced startup costs of Clouds compared with setting up and provisioning home grown infrastructure for web sites, email, accounting software, etc.
• Medium sized organisations benefit from the on demand nature of Clouds - as their business grows so can their infrastructure
• Large organisations benefit from Cloud due to their shared resources - instead of having to maintain silos of computing infrastructure for different departments they can get cost savings via economy of scale.

There are a large number of vendors touting Cloud products, including Amazon, Google, Salesforce, Rackspace, Microsoft, IBM, VMware and Paremus. These products fit into various categories of Cloud, IAAS (Infrastructure as a Service), PAAS (Platform as a Service), SAAS (Software as a Service) and Public or Private Cloud.

Cloud Realities

So Cloud seems pretty utopian right? In fact despite promise of Cloud the realities it delivers are somewhat different.

• As there are so many vendors, there are also multiple APIs that developers need to code to for simple things they used to do like loading resources
• Depending on the vendor the sort of things you can do in a Cloud are often limited (in Google App Engine you can't create Threads for example)
• Some vendors package Clouds as VM images and for simple deployments this is perfectly ok. However, as the number and variation of applications changes the course grained nature of VM images is a prohibitive factor. The system administrator needs to build and manage lots of different VM image configurations.

Finally, and this is a factor that effects all Clouds, they are not - despite marketing - infinite resources (pdf). Contention and latency are real problems in Cloud environments. The the shared nature of Cloud architectures means that SLAs can be severely impacted by seemingly random processing spikes by other tenants. Cloud providers employ many different tactics to minimise these problems but running an application in the Cloud and running it on dedicated hardware is not a seamless transition.

Why Private Cloud?

If Clouds are about shared resources and mimic utility services like electricity, gas or water, why would organisations choose to host their own Clouds? Well, it basically comes down to a healthy dose of paranoia. For large organisations the risks associated with Public Cloud are just too great for many of their business processes. Concerns often touted in the industry are:

• Data ownership risks – A bank for example is often extremely reluctant to host private customer details on infrastructure they don't own. This can be for legal/regulatory reasons or business intelligence reasons
• Data inertia – I've heard one horror story at a previous Cloud Camp in London about a Web2.0 portal that had started their business in the Cloud (I forget which one). However once it got successful its users were creating data in the Cloud faster than they could offload it, meaning they had a hell of a job moving providers once they started. Not a good position to be in from a price/negotiation perspective
• API lock-in – The requirement to code to vendor specific APIs is a major problem in terms of vendor lock-in. As a small company the cost of starting out is pretty minimal but once the business starts to scale, finding you are locked into an uncompetitive pricing plan is obviously not good for business
• SLA – The contention and latency issues of Clouds can mean that for those businesses that are are in a competitive compute-intensive business then any downtime or latency outside of your control can have a major effect on your bottom line.

Private Cloud implies all of the on-demand, dynamic, network accessible goodness, but in a controlled environment where the business has direct control of the cloud tentants, so can better control their SLA. A bit like owning a well, or growing your own food, there are costs but also benefits.

This pretty much takes me to the end of the beginning. I left this section with a question posed in the form of a set of images (which I've used in this blog entry). The Cloud vision is certainly an enticing one and there are already a lot of commercial benefits to using Cloud in day-to-day business. However, there are definitely still problems. The final question is...

How Do We Get Here?

This is M51a “The whirlpool galaxy” discovered by Charles Messier in 1774 (and its companion galaxy NGC 5195).

I came at computing from the physics angle and when I think of computer software/architecture I tend to think in terms of patterns. A galaxy is just a cloud of gas after all - but there is structure, dynamicity and mechanics that describe their overall behaviour!

• Homogeneous Cloud deployments can be inefficient – sometimes local processing is just faster than distributed algorithms. We need fidelity in our Cloud deployments, but with fidelity comes administrative cost. Dealing with fine grained structure on the scale of millions (or even tens of thousands) of nodes requires new models
• Clouds are dynamic, resources come and go, their can be gaps in communication caused by latency, their can even be large scale events like data centre collapse. Software that is deployed on them must be able to cope with these dynamics
• As anyone who has worked on any large or even medium sized software project knows, software is often a tangled web of interdependencies. Changing the processing laws in local parts of an architecture can have drastic effects on distant components. We need ways of encapsulating and segmenting Cloud software to allow for updates.

There we go, I'll try to write up the next section "Where does OSGi fit in the cloud" soon. But to look briefly into the future (or the past - depending on your perspective), I think we need to use modularisation and abstraction to help tame the complexity/scale curve and this is where OSGi fits into the Cloud model.

In the mean time, I'm very interested in any comments or feedback on any of the ideas discussed here.

Laters,

Hello Tatooine

So in my last couple of posts I've been showing the power of Nimble. You will have noticed that it is primarily a console environment. As such you may be wondering how you can provide your own commands to execute in the Nimble shell - Posh (Paremus OSGi Shell).

Posh is an implementation of the command line interface specified in RFC-147 from the OSGi alliance. If you are familiar with OSGi development you will know that to date every framework implementation has defined it's own particular API for providing command line utilities within an OSGi runtime. This has meant that there is significant duplication of effort when writing commands to work in the various environments.

To this end the alliance proposed RFC 147 in order to provide a common standard that different frameworks could implement such that a command that worked in one framework could work unchanged in another. The initial implementation of RFC 147 was developed primarily by Peter Kriens and donated to the Felix project earlier this year. Since then there have been a number of maintenance releases and it has been included as a component of the Felix Karaf container and the Nimble container from Paremus.

This gives you some background, so now the standard thing for me to do would be to write a trivial hello world application. But that's no fun, so instead of conforming to the norm I thought it would be more interesting to port the Starwars Asciimation work to run in OSGi as an RFC 147 command line interface.

Yep this is very probably the geekiest post you will ever see... :)

I hasten to add that I did not undertake the core task of this myself, instead I took the liberty of contacting Simon Jansen (the author of Asciimation) to ask his permission to "borrow" the ascii text from his player. This he was very kind to do.

The first thing we need to do to define our cli is define a class that implements the core functionality as shown below:

package org.chronologicalthought;

import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.PrintStream;
import java.io.Reader;
import java.net.URL;

public class Starwars {
 public void starwars() throws IOException, InterruptedException {
   play(67);
 }

 public void starwars(int frameLength) throws IOException, InterruptedException {
   URL res = Starwars.class.getResource("/starwars.txt");
   if (res == null)
     throw new IllegalStateException("Missing resource");
   InputStream in = res.openStream();
   try {
     InputStreamReader reader = new InputStreamReader(new BufferedInputStream(in));
     render(reader, System.out, frameLength);
   } finally {
     in.close();
   }
 }

 private void render(Reader reader, PrintStream out, int frameLength) {
   // ...
 }
}

Here the command provides two methods, play and play(int) and prints the individual frames from the "starwars.txt" file embedded in our bundle to System.out.

Wait a minute you might be thinking. Where's the API to the CLI? Well this is one of the neat things about RFC 147 you don't need to write your code to any API. The specification provides a clever utility in the form of a ThreadIO service that multiplexes the references to System.in, System.out, and System.err so the command can interact with the user. It also calls methods on the class reflectively so there is no need to implement a defined interface. Simply declare a method and Posh will attempt to convert arguements supplied from the command line to match the method signature.

The next step is to define an activator that publishes our cli class to the OSGi bundle context.

package org.chronologicalthought;

import java.util.Hashtable;
import org.osgi.service.command.CommandProcessor;

import org.osgi.framework.BundleActivator;
import org.osgi.framework.BundleContext;

public class Activator implements BundleActivator {

  public void start(BundleContext ctx) throws Exception {
    Hashtable props = new Hashtable();
    props.put(CommandProcessor.COMMAND_SCOPE, "ct");
    props.put(CommandProcessor.COMMAND_FUNCTION, new String[] { "starwars" });
    ctx.registerService(Starwars.class.getName(), new Starwars(), props);
  }

  public void stop(BundleContext ctx) throws Exception {
  }
}

This activator publishes the Starwars class with two attributes:

• CommandProcessor.COMMAND_SCOPE - a unique namespace for our command
• CommandProcessor.COMMAND_FUNCTION - the names of the methods to expose as commands in the cli

The code is available from here for those who want to take a look around:

Now the final stage is to load our command within nimble. Since my cli has such a trivial set of dependencies I will defer the usage of the full nimble resolution to another blog post. So instead I will use a set of trivial commands which I've defined here. So finally let the show commence:

$ svn co http://chronological-thought.googlecode.com/svn/trunk/starwars
$ cd starwars
$ ant
$ posh
Paremus Nimble 30-day license, expires Wed Dec 30 23:59:59 GMT 2009.
________________________________________
Welcome to Paremus Nimble!
Type 'help' for help.
[feynman.local.0]% source http://chronological-thought.googlecode.com/svn/trunk/nimble-examples/basic-commands.osh
[feynman.local.0]% installAndStart file:build/lib/org.chronologicalthought.starwars.jar
[feynman.local.0]% starwars





                       WWW.ASCIIMATION.CO.NZ


                             presents

The final piece of Nimble functionality that it would be fun to demonstrate is stopping the movie. Simply hit Ctrl-C. The nimble shell then sends a Thread.interrupt to the currently running command. For those who want to see the movie to the end and don't want to wait try running:

% starwars 20

To set the frame length as 20 milliseconds.

Enjoy the show.

Laters.

And for my next trick

Just for fun and to demonstrate the power of the Posh (sh)ell environment I decided to knock together the following trivial script to do a "traditional" OSGi bundle file install from a directory:

// create a temporary array for storing ids
array = new java.util.ArrayList;

// iterate over the files passed 
// in as arguement 1 to this script
each (glob $1/*) {

  // use the BundleContext.installBundle 
  // method to install each bundle
  id=osgi:installBundle $it;

  // store the bundle id for start later
  $array add $id;
};

// iterate over our installed bundles
each ($array) {
  // use the BundleContext.start method
  //to start it
  osgi:start $it;
};

To try this out for yourself or to find out more about Nimble you look here once installed you can run the above script using the following command:

posh -k http://chronological-thought.googlecode.com/svn/trunk/nimble-examples/file-install.osh <your bundles dir>

Where you should replace <your bundles dir> with a path to a directory on your local file system that contains bundles.

Hmmm what to blog next...ponders...

Laters,

Nimble OSGi

So I just sent a rather cryptic twitter message with the instructions:

posh -kc "repos -l springdm;add org.springframework.osgi.samples.simplewebapp@active"

I figure it's probably worth a short note to explain what this is doing given the narrowband aspect of twitter communications.

This command is running an instance of the posh (sh)ell which ships with Nimble. There are two switch parameters parsed to the shell:

-c: Tells posh to execute the command passed in from the unix shell in the posh (sh)ell environment
-k: Tells posh to remain running after the command has completed and open a tty session for user input

Now we come to the actual commands:

repos -l springdm: tells posh to load the spring dm repository index into the nimble resolver

add org.springframework.osgi.samples.simplewebapp@active: tells nimble to resolve all dependencies for the spring simplewebapp from it's configured repositories.

The interesting thing about nimble resolution is that it doesn't just figure out the bundles that need to be installed. It also figures out what state these bundles should be in. If you look at the bundles in the nimble container using the command lsb you will see that not only are all the bundles installed but certain key bundles have also been activated:

lsb
*nimble/com.paremus.util.cmds-1.0.4.jar                           00:00    59Kb
0  ACTIVE    org.eclipse.osgi:3.5.1.R35x_v20090827
1  ACTIVE    com.paremus.posh.runtime:1.0.4
2  ACTIVE    com.paremus.posh.shell:1.0.4
3  RESOLVED  com.paremus.util.types:1.0.4
4  ACTIVE    com.paremus.nimble.core:1.0.4
5  ACTIVE    com.paremus.nimble.repos:1.0.4
6  ACTIVE    com.paremus.nimble.cli:1.0.4
7  RESOLVED  javax.servlet:2.5.0.v200806031605
8  RESOLVED  com.springsource.slf4j.api:1.5.6
9  RESOLVED  com.springsource.slf4j.nop:1.5.6
10  RESOLVED  com.springsource.net.sf.cglib:2.1.3
11  RESOLVED  com.springsource.edu.emory.mathcs.backport:3.1.0
12  RESOLVED  org.springframework.osgi.log4j.osgi:1.2.15.SNAPSHOT
13  RESOLVED  com.springsource.org.aopalliance:1.0.0
14  RESOLVED  org.springframework.osgi.jsp-api.osgi:2.0.0.SNAPSHOT
15  RESOLVED  com.springsource.slf4j.org.apache.commons.logging:1.5.6
16  RESOLVED  osgi.cmpn:4.2.0.200908310645
17  RESOLVED  org.mortbay.jetty.util:6.1.9
18  RESOLVED  org.springframework.osgi.jstl.osgi:1.1.2.SNAPSHOT
19  RESOLVED  org.springframework.core:2.5.6.A
20  RESOLVED  org.springframework.osgi.commons-el.osgi:1.0.0.SNAPSHOT
21  RESOLVED  org.mortbay.jetty.server:6.1.9
22  ACTIVE    org.springframework.osgi.samples.simplewebapp:0.0.0
23  RESOLVED  org.springframework.beans:2.5.6.A
24  RESOLVED  org.springframework.osgi.io:1.2.0
25  RESOLVED  org.springframework.osgi.jasper.osgi:5.5.23.SNAPSHOT
26  RESOLVED  org.springframework.aop:2.5.6.A
27  RESOLVED  org.springframework.osgi.catalina.osgi:5.5.23.SNAPSHOT
28  RESOLVED  org.springframework.context:2.5.6.A
29  ACTIVE    org.springframework.osgi.catalina.start.osgi:1.0.0
30  RESOLVED  org.springframework.osgi.core:1.2.0
31  RESOLVED  org.springframework.web:2.5.6.A
32  RESOLVED  org.springframework.osgi.web:1.2.0
33  ACTIVE    org.springframework.osgi.web.extender:1.2.0
34  ACTIVE    com.paremus.posh.readline:1.0.4
35  ACTIVE    com.paremus.util.cmds:1.0.4

This listing also demonstates another key feature of nimble. Typing lsb resulted in the following log line:

*nimble/com.paremus.util.cmds-1.0.4.jar                           00:00    59Kb

This demonstrates that the nimble container resolved the lsb command from its repository index and installed it on the fly. In fact if you look at the Nimble download it is only 55K in size. All of the extra functionality is automatically downloaded based on information provided via the nimble index files and traversing package and service level dependencies!

To complete this blog post you can browse the simple web app running from nimble by opening:

http://localhost:8080/simple-web-app/

Nimble is available for download here.

OSGi Dev Con 2009 & OSGi Tooling Summit Roundup

So I'm currently sitting in my hotel lobby waiting till we jump in the hire car off to the airport for our flight home. It's been a long week - the time zone difference between San Francisco and London is always exhausting and the mental effort of networking, presenting, and talking to customers is always pretty intensive too.

I think the conference was definitely quieter this year - you couldn't help but feel the shadow of the economic climate hanging over us all - however though volume was down compared to last year, quality seemed to be much higher. My rough market assessment based on talking to people on our exhibition stand was that awareness of OSGi is much improved compared to last year.

Even better for me people really seemed to get what we (Paremus) are about. Last year we were the "RMI guys". This year people we talked to seemed to get genuinely excited about what our product is a really about: a flexible, scalable solution to provisioning and managing dynamic distributed OSGi based applications in enterprise environments.

Pretty much everyone we talked to was very positive on OSGi as a runtime solution to Java modularisation. However, the problem for its adoption in the enterprise is the backwards compatability issues in moving to an OSGi environment (things like static references and Class.forName are a really bad idea in OSGi) and the fact that OSGi development relies on a reasonable amount of domain specific knowledge.

I think good tooling solutions that work right the way though the stack are crucial to help new developers though the pitfalls of the new classloader space. Unfortunately, tooling support for new developers is pretty disjointed.

Hence, the next part of my post...

We (Paremus) recognised that it does us no good if customers are having difficulty building OSGi based solutions. Therefore, my team mate Derek Baum and I have been working on a solution to developing OSGi bundles which works in both an IDE and headless build environment called Sigil. Prior to OSGi Dev Con Paremus chose to licence Sigil under the Apache licence, as we recognise that tooling is an area where we need support from the community in order to help the community as a whole.

On the Friday, after the end of the conference, I and a number of other representatives with interests in the area of development tooling met at an OSGi Tooling Summit hosted by Yan Pujante at LinkedIn's Mountain View offices. The group was pretty large and diverse (as you can see here). The OSGi Alliance's typical role in this respect is to foster communication (via summits, conference calls etc), then generate one or more RFPs which list a set of requirements for the problem domain, and finally put together a set of RFCs which provide a spec to allow different vendors to work together.

The format of the meeting was an initial intro and positioning statements (i.e. what we were there to achieve). We then moved into a blue sky discussion where we tried to think of a perfect world solution to the OSGi development lifecycle - this stimulated some really interesting conversation which I'll doubtless follow up on in the next couple of weeks in future blog posts. Finally, we put together a series of use cases headers that captured the major tasks undertaken by developers when building, maintaining and releasing OSGi applications into the wild.

I had a number of really encouraging conversations with Chris Aniszczyk and Peter Kriens who work on PDE and BND respectively, both of which have a lot of cross over with the work I've been doing on Sigil.

Chris has just twittered that he's thinking of changing the name of PDE to BDE which I think is a great idea - perhaps some merging of BDE, Sigil and BND? I think there is likely some common core classes that are useful to all OSGi developers. We're obviously going to need some domain specific extensions for plugin development, newton development, spring development, etc. but I see these as addons on top of a common OSGi layer.

I guess in a perfect world I'd like to be able to support Maven, Netbeans and IntelliJ users as well. Hopefully I'll be able to update you in the next couple of months on progress in this area.

Laters,

Flocking Behaviours

Just a quick note to say I'll be running a BOF session with Peter Kriens and Chris Aniszczyk on OSGi development tooling at EclipseCon this year as part of the OSGi track. You can find details here.

I guess my main focus for calling the BOF is that I'm very interested in talking to other OSGi developers to see what it is we on the tooling side can do to make our collective jobs easier.

What is it about OSGi development that really frustrates you - and what can tools do to make it easier?

It'll also be really interesting to meet up with other tools developers to see if we can get a bit of reuse going on - likely we won't solve that in one BOF but if we can just put names to faces and get a bit of shared understanding going on we're likely to be more productive in the long term.

Hope to see you there.

Laters,