All the small things

I was actually planing on writing something about this topic for months now, but I never found time to do so. Now that Peter Kriens picked up the topic in his current post [1], I just felt the need to write about my thoughts on this issue.

What is it all about?

When working on computer programs for a while, you pretty soon realize that you don’t want to invent the wheel all over again. So, people came up with concepts driving software reuse further and further. Starting from functional programming, finding its way to the fine world of objects and in our days driving more and more towards component based programming. Nothing surprising - though. With the help of OSGi I’d like to say Java has a pretty good standing among its competitors and could already be considered component ready.

However, there is one problem, at least as far as I am concerned. Looking at software product lines (I know, I know, they are more considered a fancy dream than a real world solution, but please hang in there for a moment and let me explain) the core concept - as I would call it - is the idea of divide and conquer. In a complex domain as computer science, it is not feasible to believe one can manage everything down to the finest granularity. So what you end up doing is creating the big picture and drill down on a “as needed” basis - a typical top down approach. Of course, this assumes you already have your main building blocks to work with. OSGi by its nature in general supports you in doing so. You only pick your functionality in terms of simple bundles and throw them into an OSGi container and everything just works - well, almost.

Unfortunately, that’s just the simplest scenario when things like this are working. Peter is saying in his blog that applications (bundles belonging to one domain and providing a certain functionality with their collective interaction, as I would define it) are simple to manage, because they express their dependencies. Tools like the OBR[2] make installation issues disappear and all the remaining requirements for an “application” concept to solve their problems are actually derived f.i. from the use of Class.forName() in legacy code to extend their applications and the sharing of implementation classes rather than interfaces. Here I have to disagree. Of course, the whole Class.forName() ideology is a problem within OSGi, but I don’t think this is the major reason for the demand for an application model.

Why is the concept of an application worth while?

Let’s assume you’re building a pretty complex application with … 400+ different bundles or so. Referring to software product line design you’re no longer managing all of these bundles, but you’re managing some sort of an application stack. If your interested in (see figure 1) creating a new application with “Domain Logic 1″ and “Domain Logic 3″, you just pick those and all the underlying plumbing should be taken as well. Of course (and here it becomes tricky) this involves loose coupling, which doesn’t include dependencies on a specific implementation or even worse implementer on the domain logic stack. Everything is abstracted from the implementation and everything just depends on interfaces and services.

One of my core problem here is that there is no distinct bundle representing an application. There is no syntax, convention, anything. I need to know which bundle in the application is my “core” (and I am not even talking about finding out, what my applications are, I can choose from). Having figured out which one to use, we are still not in a safe harbor. The problem continues with the separation of API and implementation. Ones both are separated by design (as shown in figure 2), I can happily resolve all my bundles, but nothing works, when solely relying on Package-Import statement scoping (which I - again! - highly recommend). Automatic tools dependency discovery tools just can’t find a path to the implementation. In figure 2 you can see the problem show by the arrow. Starting from the “Core Bundle”, I only have a dependency on the API. The API doesn’t have any dependency on the implementation. The Implementation bundle does have a dependency on the API, but for the dependency resolution tool, it is not clear, if the bundle is just using the API or implementing it.

As you now see, there is a certain problem and I wasn’t actually surprised when I heard about Springs approach to this problem. Although Springs way is a fast solution for the immanent problem, it introduces dependencies one might not want. Bundling components together in the Spring way ties the interface to the implementation, limiting the flexibility of the whole design.

So, basically we are doomed, right?

Well, the question is, what do we REALLY need. As I mentioned before, in my opinion, it is a set of just a few features:

1. an application identifier (this is great to figure out what functionality I actually have)
2. a way to express simple dependencies (with Import-Package, I think we pretty much have that)
3. some way to express dependencies on our environment, like the existence of a certain service implementation.

Looking closely at the core spec, we can actually find almost everything we need. Unfortunately, we can’t use this right away. Feature number one for instance can very easily be achieved by just introducing a new property in the Manifest.mf file - even a version can be introduced, although I am not sure this is a good idea yet.

The tricky part comes with feature number 3. Looking at the latest official OSGi core spec version 4.1, there is no meta data concept reflecting services*. Fortunately, there are still the old specs available for download [3] and here we can find in version 3 the “Export-Service” and “Import-Service” headers removed from the current specification. Assuming all bundles are correctly tagged with the right meta data, tools like the OBR can easily create a dependency tree and figure out which bundles to pick. Even better, such tools are now able to pick the best implementation based on their other dependencies and potential version conflicts on the imports of each implementation bundle. I think such a dynamic solution is way better fitting the OSGi way of doing things than any static one.

Ok, I know R3 is so “old school” and we are living in a brave new world. Swimming back is not an option in a buzz, hype and in general fast pace driven world. Fortunately, we don’t have to. With Spring DM [4], there is a way to express dependencies on services in a declarative way. Here you can define which services you are consuming and which ones you are providing, thus giving 3rd party tools the ability to analyze the dependencies way more subtle. If the Spring DM approach is used widely - as a new “official” standard, tools like OBR can now work again. They are able to assemble the right product configuration and deploy a runnable application based on the additional meta data from the Spring DM configuration. The only drawback so far, only if everyone is using this configuration, we will get the best results. If partially used, the results may not be providing any significant improvement based on the very bundle constellation.

Something I intentionally left out till now is the new idea of providing nested frameworks. I haven’t thought that entirely through though, I have to admit. In general I like the idea, but for other reasons. The main reason, why I don’t like it for this particular problem of the lack of the notion of applications is that when looking at figure 1, you might end up with several layers of applications, which would then ultimately end-up in requiring multiple layer nesting of frameworks. And speaking about complexity I am not sure, if a framework in a framework in a framework in a …. makes life so much easier. Besides, this still doesn’t solve the problem with discovering the necessary dependencies for an application - not counting manually configuration of such a sub framework of course.

Conclusion

For me there is a clear need for an application like notion in OSGi. As shown above, there are valid use cases even now, which make the use of OSGi kind of limited in the long run. However, I actually don’t see the need for an entirely new concept, above or beyond a bundle. The only thing we really need is a clear statement, how to provide and obtain the required information to do the described extended dependency analysis. With these information at hand, tools by 3rd parties can be developed and make us all equally happy. At the end, we should keep in mind that OSGi is (currently) about defining a component standard, which should be open and not enforcing any lock in. This is in contrast to most of the adapters wishes of having a simple technology, solving their immanent problems. I think, now there is time for great tools to support working with OSGi more easily, helping developer to adapt and embrase the functionality and freedom offered by OSGi.

Cheers,
Mirko

*) actually I was cheating a little bit here. OSGi R4 compendium contains the Declarative Service (DS) definition, which in fact defines valuable meta data. The reason, why I didn’t mention it here is basically related to the fact that Spring DM is superior to DS in many ways and DS is not part of the core specification, so I favored Spring DM for the sake of functionality.
–

[1] OSGi Blog: http://www.osgi.org/blog/2008/07/osgi-application-grouping.html
[2] OBR: www.osgi.org/Repository
[3] OSGi Spec Download Site: http://www.osgi.org/Specifications/HomePage
[4] Spring DM: http://www.springframework.org/osgi