Currently, versions can only be considered as simple timestamps with a sophisticated syntax. You might be able to read intentions from the provider but further than that they are useless. The syntax of a version theme might indicate some sort of intend to express compatibility, but in fact this is never enforced, so you can’t really trust it! You ALWAYS have to check it yourself (manually or with integration tests) to figure out if it really fulfills the contract. As a result you can only be sure to some extend that some bits changed, if the version got increased – what ever this means. For me, this is the ultimate problem with dynamism in applications. You can’t test all possible installation scenarios of your clients as soon as you open yourself to third parties. Someone will break your perfect little world, better anticipate it!

In OSGi, things get even harder from a testers perspective (and developer for that matter). As the only system, I know of, it defines a way to describe future versions a component (aka bundle) can be compatible against. For instance you can specify that you require a certain version of a package your module depends up-on. Therefore you just add the required dependency in your Manifest.mf file as a range:

Import-Package: dictionary;version="[1.0.0,2.0.0)"

A typical usage scenario in OSGi might look like this:

OSGi dependency versioning

Here the dictionary “user” (cl.client) defines a usage dependency of the dictionary API (dictionary.api). As you can see, the dependency definition differs between a “user” and an “implementer”, because the implementer of the functionality (dictionary.impl) not only has to be binary compatible, but also source code compatible (like implementing a new method of an interface). Fortunately the OSGi allows to at least express such a dependency. The distinct version scheme of OSGi imposes a specific usage. If you’re not binary compatible bump up the major number, if you’re binary compatible bump the minor number. If you do bug fixing, not affecting the API go for the build number and last but not least, if you want to indicate a distinct notion like a release or an iteration, use the quantifier. That’s what it should look like:

[major[.minor[.build[.quantifier]]]]"

You may think this is a need approach and you’re probably right. It is great compared to everything else we have right now. Unfortunately this is not the silver bullet. Not at all, but it is a scheme which is explicit and well defined to some extend. Someone reading this, will at least have an idea what the intention of the component author was.

Now, what is the catch. Plain simple, this versioning scheme is not enforced. Everyone can version their OSGi artifacts however they want. There is no enforcement whatsoever. When you go to any bundle repository, none of them are verified and frankly, I don’t know how they ever will be. Of course, we can go all fancy and do API checks as Eclipse[1] does (and by the way, which is doing a pretty good job so far) or use an approach introduced by Premek Brada[2] at the last OSGi DevCon in Zurich. There is even the possibility to rerun all the former unit tests of a bundle that should be compatible with the latest version to check for semantic inconsistencies, but who is doing this and who guarantees that this covers all edge cases? No one can! Well, at least yet. Maybe in some years with fancy self learning AI on quantum computers, but for now we have to stick what we got.

I understand, my post is quite dark and negative up to now – nothing will work, we will ultimately fail, software reuse as the epic fail, blah blah blah. It’s true, we are not going to be a 100% perfect, but maybe, we don’t need to. Maybe we should just take what we got and try to make the best out of it. Evolve slowly but consistently and fail gracefully…

For one, testing is inevitable. For instance take the small and innocent build number that indicates just a bugfix. Now what is a bugfix? It can be a performance improvement – ok, that fine. It can also be a miscalculation or returning true instead of false in some odd case. When developing against such an behavior it is certainly not a compatible change, when you expect such a behavior and it gets fixed in a build number increased release. On the other hand, bumping the major number, because it in fact affects the client is a pretty heavy step, rendering all users incompatible unless they bump the version range to a higher number. The only thing in my opinion is creating test code to check if this behavior continues to be there and implement ways in your code to handle the fact that at some point it is different. This is painful, but there will always be this one edge case screwing with you.

Another thing we can do is rethinking of the actual version scheme. For one, this bump in the minor or major version, does it actually affect the implementer of the API? Well, not necessarily – not in all cases. So why do we start from 0 every time a higher number gets increased? Aren’t they distinct from each other? Well, maybe they have to get increased by one, but why reset them back to zero?

2.4.2 becomes 3.5.3 or 3.4.2 instead of 3.0.0.

The same holds true to version ranges. Why is the consumer in charge of defining the range? Ultimately, the provider knows what version the current library is still compatible with (from a client point of view). Now, having both, the client and provider version range can allow for a very flexible system.

Inverse Versioning

Last but not least, the ranges we’re defining, if a version segment gets increased independently from each other, as explained before, should version ranges then incorporate this as well? Allowing ranges on all version segments, not only as a whole?

Inverse versioning with distinct version segments

I know, I am describing a maintenance nightmare. How could one stay sane with so many variants? Short answer, humans can’t! Fortunately, there are tools and best practices that can help. With tooling you can get the version right you can depend upon. This is already done. With approaches like running old unit tests against the new API that specifically aim at the API level can verify the (most) of the behavior didn’t change. It is not easy – no doubt, but it is a chance we should take. At least when we want to get to a point where we actually can reuse 3rd party components.

Truth to be told, most of the things I said are just idea’s I picked up over the last few years and there is currently a lot of discussion going on.

• Peter Kriens: http://www.osgi.org/blog/2009/12/versions.html
• Chris Aniszczyk: http://aniszczyk.org/2009/11/23/jigsaw-versioning-is-ridiculous/
• Neil Bartlett: http://neilbartlett.name/blog/2009/12/11/reasons-not-to-mourn-jsr-294/
• “Sun” Versioning: http://wikis.sun.com/display/IpsBestPractices/Packaging+Best+Practices+-+Versioning
• Alex Blewitt: http://alblue.blogspot.com/2008/05/version-numbers-and-jsr277.html and http://alblue.blogspot.com/2009/12/jsr294-is-dead-long-live-osgi.html
• Mirko Jahn (myself – sorry): http://osgi.mjahn.net/2009/04/02/the-myth-of-software-reuse/

If you want to get a full picture, I urge you to read through all of those sources, because ultimately, the worst thing we can do is redoing the mistakes of our past!

Cheers,
Mirko

References:

[1] PDE API Tools: http://www.eclipse.org/pde/pde-api-tools/
[2] Safe Bundle Updates: http://www.slideshare.net/pbrada/safe-bundle-updates

Update[1]: I added the links Alex mentioned. I certainly read his blog, which eventually influenced me as well. None of these concepts are coming from me personally unfortunately. I can’t take any credit for it.  Sorry for not giving you the credit you deserved in the first place, Alex. Btw., the order is arbitrary coming from the top of my head. No semantics are applied whatsoever! I hope I haven’t forgotten someone else as well!

I thought a lot about the resistance of parts of the Java community in reusing the OSGi standard and starting from what we already have and in my conclusion the problem is two fold. First, it has come from a source outside the JSR initially. Well, I know this is debatable, because OSGi was one of the first JSRs and now with JSR 291 it in fact is one, but it is not “developed” within the JSR process, so one can say this is a valid point. Second modularization is in fact a tough call. When looking around, there is no other language or standard (even beyond Java itself) that has entirely tackled the problem and boy since David Parna’s introduction of the concept of modularity in 1972 quite some time has passed. A “fan” or not one has to admit that OSGi has already has gained a name in terms of modularizing the JVM and looking at the time required, well it took over 10 years to get that far. So I think it is fair to say that modularization is not easy to accomplish and to get right. In fact I had my problems (and partially still have) with OSGi as well. Let me explain…

My first contact with OSGi

When I first encountered OSGi I was working at IBM Research trying to explore ways to improve the modularization approach of [UIMA]. UIMA itself already had the notion of components or modules if you prefer already, but in a way it left certain features to get improved. In particular the isolation of each module wasn’t enforced, so introducing 3rd party modules into the runtime was a risky business. Especially when giving the fact that these modules had access to basically all resources. You never know what they are using or how they behave in the system. That, for instance was a reason to separate the processing into an other Java process. If that one fails, not the entire system is affected. OSGi in this context wasn’t THE savior, but it helped to make it more robust. No Java based model can provide real isolation of modules on JVM level (without the need to create a separate JVM instance is still something entirely missing in Java – one can ALWAYS create an OutOfMemoryError if he or she intends to). The great benefit of OSGi was simple. Hiding internal APIs, making dependencies explicit and potentially create a repository of reusable artifacts being usable without pages of manuals describing on how to set-up a system suitable for these modules.

How OSGi is received (on first encounter)

As you may imagine, one of the most important goals was to hide OSGi as much as possible. No services, no Import-Package (but Require-Bundle on one aggregation bundle), basically reduce OSGi to its minimum. Well, the appreciation was… limited. All people I was working with were exceptional bright and determined researchers in MLP. They developed highly sophisticated algorithms on how to analyze unstructured data, but only a few were software engineers. So the code was… working, but not production ready. Enforcing them to apply rules that ultimately made it harder for them getting things done and as a result slowing them down wasn’t something they welcomed very much. This is something I can totally understand and relate to! What is a system worth, if you don’t gain anything! Well, the problem here is that they didn’t gain anything, because they just used their code and to a limited extend the code of others, so the overhead of integrating with others was rather limited. The real benefit is visible when you start reusing multiple artifacts and potentially in different versions. This is something we (as a community) are trying to achieve for decades but failed so far, if you ask me.

Some reflections about our history

Looking at the history of Java it feels like it is a common problem. One tends to address the immediate problems. In principle there is nothing wrong with that, but in often after introducing a new standard you realize that the actual problem is way more complicated or you haven’t anticipated all vital use cases and the trivial approach looking so attractive at the beginning won’t work in the long run. The problem then is only that you don’t want to brake backwards compatibility and eventually this will drive the design of your solution, ultimately removing many otherwise possible ways to go. Talking about history, when we look back, jars where introduced as a distribution format for applets. Surely solving the immediate problem. After a while, Java on the server emerges and yet another problem arose. Separation of different web applications. Well, we know how it ended, eventually we got different class loading solutions for each J2EE vendor, because this part of the specification was left out.

My point is…

Now we have Jigsaw addressing a subset of OSGi’s modularization capabilities, which by itself is perfectly fine (even if it is not OSGi what they are going to use). The problem I fear is only that soon there will be more requirements altering the current approach unfitting. In particular, the great benefit of modules should be to be able to reuse them, without the need to understand all their internals. It was stated that OSGi’s restrictive class loading approach is not suitable for many applications. I tend to agree. I experienced this pain and it wasn’t pleasant. I heard that not allowing split packages is not acceptable, because it is a necessity. Why? Is it, because it is a good design or just because current systems are using it? If you want to create modules, you want to make them robust, so except for the exposed (public) API, it shouldn’t matter how it is implemented. If modules are not enforced to have their own class loader, you can never be sure whether or not you’ll have collisions. Yes, this is a pain in the **** to rethink and rewrite existing code! And I also have better things to do, than to retrofit some working libraries – no doubt! The question we should ask ourselves however is what do we really want? A system seamlessly integrating with existing code or provide a new way of thinking which might not be as simple as hoped, but give us what we need in the long run? Do we want reusable entities or a better provisioning system as plain jars? Don’t get me wrong here, I am not saying OSGi is the solution or should be used at all! Just saying that the requirements are dictating what you should get and if you intend to create “real” modules, the lessons OSGi learned are most valuable, even if you are eventually taking a completely different approach. In fact it has its short comings and flaws as any other system has.

OSGi is not perfect either

Working with OSGi for a while, I came across several places, where I noticed the need for improvement. For instance, OSGi provides a pretty strict module system, which when running is well defined. Unfortunately, there is a problem with how to get to this state or even knowing when this state is achieved. The idea behind its model is that using services, there should not be any dependency on the start order, because everything can change at any time. This is a nice idea, but in real world it is impossible to achieve. For instance, I am currently working with embedded devices and due to the limited hardware the  start-up can take minutes. Based on the dynamics however, a user might already see a web interface popping up, which seems perfectly usable. However, when using OSGi’s ConfigAdmin service to gather all configured configurations, it can happen that not yet started bundles don’t provide the required information, so the resulting configuration is incomplete. Similar things apply to the start order (no start levels are not sufficient) when using security. It is just not standardized how to ensure that certain bundles have to be loaded before everything else in order to start a particular runtime with security on. OSGi limits itself to define the runtime behavior, leaving out configuration issues when moving from one container to another. Basically the configuration for the start levels of each bundle getting loaded is an implementation detail – doesn’t this look familiar to you when using JEE . Also providing tools to inspect the runtime behavior (how the container resolves dependencies is not sufficiently defined, so you have to jump through a lot of hoops or use implementation details of a particular implementation. Not really helpful/desirable from a tooling provider perspective. I could go one and on, like with every other technology. Work in progress is written over everyone of them. So, you see, there is so much that can still be done to improve our situation. No one is perfect and will ever be. We can just try to do our best to progress and learn from our faults.

Properties of a true module system

Well, after talking so much about what’s not right and what can be improved, I guess it is just fair to also draw a rough picture of a system providing the features of true – meaning reusable – modules. To help me out a little, I took the liberty in quoting one of the experts of componentization/modularization (which is not quite the same). Clemens Szyperski is currently working for Microsoft (I think) and so has no real relation to Java at all, but his observations and remarks hold true even in our space.

„A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be deployed independently and is subject to composition by third parties.“ [Szyperski et. al. 2002, p.41]

… and a more brief definition of required properties.

„The characteristic properties of a component are that it:
• is a unit of independent deployment;
• is a unit of third-party composition;
• has no (externally) observable state.“ [Szyperski et. al. 2002, p.36]

I think this sets the floor for more further investigations. In particular, the citations above are nice, but pretty vague and open to interpretation. For instance, what is “independent” when we’re talking about deployment? What does composition mean in this context? Is it just a descriptor, is it a zip or rpm file containing everything? I don’t know and it doesn’t actually matter I think. What matters is that the resulting system is compelling, concise and valid within itself. Of course, some features seem better than others, but we also need freedom to explore new ways of thinking.

So while thinking about the core features of a (potentially) new module system, for me I identify the following as most important:

• Isolation:  When talking about isolation, I am basically talking about trust. Having a module system defining an isolation on module level (which currently can only achieved by having a custom class loader), gives me as the module designer the trust that I can create something without the fear of breaking something on the user side, just because of an implementation detail that is not part of the public API.I can trust that when I have the isolation on module level, I know what will have to deal with. As they say: Good fences make for good neighbors.
• Information hiding: Yes this is as old as it can get in software engineering, but this is the most critical part in so many ways. First of all, I understand modules as a way of abstraction. Zoom out of the implementation details and just focus on the API. That’s what I want! Black Boxes, only with interfaces (or maybe factories visible). Of course, API contracts purely defined in Java Interfaces are not precise enough to provide all required information (like ranges, value lists, limits,…), but is a starting point. The right documentation should or for now has to do the rest.
• Enabling reuse: Currently the silver bullet to decouple is dependency injection. Although a great way of doing so, it is not enforced. Anyone can just programmatically wire up classes. As a result you get bound to an implementation, which again makes reuse and updates way harder. If I know I have to use a certain API, I usually don’t care who is providing the implementation and that’s key for every robust system.
• Predictability: Well, usually even in JEE we assume we know how an application will behave, once deployed, but in reality, it is just not true. Resolution is based on the class path which can contain, depending on where you deploy your application, basically any classes and libraries. Now you have many not easy to manage factors affecting what gets loaded when. For instance there can be multiple logging frameworks in different versions pressent interfeering with the one provided by your application. Depending when they are found in the class path, they might cause problems or not. A deterministic system, that declaratively defines its dependencies will only see the required ones. Everything else is just hidden and serve other applications if they need it – no interference with each other!
• flexible binding, yet safe binding: This is something increasingly important and also not accomplished satisfyingly by any module system I know of – so far. Basically what one wants is to create an application based on the dependencies known and being able to fix later in time appearing problems without the need to redeploy and change the whole application. For instance if a security vulnerability in one module is detected. The latest version should be deployable and tell the runtime that it is a fixed version of some of the existing ones within the runtime, so those versions can be replaced.
• robustness: Currently no in-JVM approach can guarantee any runtime behavior like the consumed memory or the allocated cpu cycles. If you get a malicious module, it can bring down the entire JVM. There are already research projects out there providing such features, so in theory it should be possible to achieve this on JVM level.

Of course, there is more, but I think the list is contains the most important parts. You might have noticed, there is no mentioning of OSGi or the way OSGi is doing it. I believe there is always more than one way to accomplish your goals, so maybe the OSGi community overlooked a possibility – I honestly don’t know. So if you’re able to come up with any solution that fulfills these requirements, I would be more than happy. Maybe and only maybe you can consider looking at some of the approaches the OSGi was taking to tackle parts of these problems.

Yours,
Mirko

References:

[UIMA]: http://incubator.apache.org/uima/
[Componentization Wars Part II]: http://osgi.mjahn.net/2008/12/04/componentization-wars-part-ii-guerrilla-tactics/
[dmeuima]: http://www.alphaworks.ibm.com/tech/dmeuima/
[Szyperski et. al. 2002]: Szyperski, Clemens ; Gruntz, Dominik ; Murer, Stephan: Component Software. Beyond Object-Oriented Programming. Addison-Wesley Professional, 2002 – ISBN 0201745720

The buzz, what is it about?

Before the extender hit the light, people were advertising the white board pattern as THE OSGi way of doing things. A way of inverting control, taking the burden of house keeping from the consumer like it is required with the well known listeners. On the OSGi website you can even find a good paper on “why listeners are considered harmful“. Well, things change or better progress and with the extender another problem got addressed. The inconvenient with the white board pattern is that the client still has to take care of some stuff ending up in boilerplate code. Here is where the extender has its strengths. The core idea is to have a designated bundle that scans all already deployed bundles as well as all bundles that are being deployed during runtime for specific features, properties, interfaces, annotations, what ever you want in order to take appropriate actions on behave of this bundle. These actions can be something simple as registering a service for every implementation of a certain interface or as complex as doing some xml transformations of configuration files that will be loaded in your system triggering a whole cascade of events. The key point is that it is completely up to the extender what to do and how during deploy time.

Imagine you have your application extensively using the white board pattern without an extender. All “consumers” of your service have to publish a specific service in the registry in order to get recognized. You have to exactly specify what the signature of this service has to be. This can be any number of interfaces or just properties you register along with the service. All this is just fine, but when it comes to changes of your approach you have to let all you’re clients know about the changes and wait till they have adapted/ migrated or provide a runtime compatibility layer that checks for the old and the new syntax. With the extender, you can still use the white board pattern, but this time you take care of registering the service. Now, everything is defined in your extender, which interfaces are important, what properties to add. As soon as it comes to a change in your design, chances are good that you can just adapt the extender in order to register the client services with new properties, use different interfaces (if possible of course) or even use something advanced as a generic proxy that wraps around the old interface and registers itself with the new API. You can now intercept service calls, if you like or need and transform the transfered data as you please. But the best thing about it, despite the mentioned is that all this happens during deploy time of the bundle so it just happens once without the need for your clients to touch any of their code. This makes the interfaces between service providers and consumers more solid and less likely to break.

Pros and Cons – abridge

Of course, there is no such thing as free lunch. The extender also has its limitations, you should well be aware of. Here is a short list with the compiled pros and cons. Of course, depending on the scenario you might find more to either ones. If you think I forgot something important or don’t agree, don’t hesitate to comment. Constructive feedback is always appreciated.

disadvantages:

- One of the first things that pops into ones mind is the problem of disabling the extender for a specific bundle. This might be interesting, because you don’t want to start/enable all bundles at start-up or just don’t want to provide all the functionality at once (f.i. to increase start-up speed). There are ways, like white/ black listing bundles, lazy activation triggered by properties or manual selection based user input. Although all this is possible, it is somewhat against the idea of the extender, which is automatically processing and removing repetitive tasks.
- The second difficulty is the customization. Of course you can define a specific format that enables you to customize the way your extender works and what it does, but there are scenarios where you can’t upfront decide what to do. A common way to work around this is defining your own DSL describing your “intent” in an abstract way, like in your own DSL. However, this introduces another layer of complexity and can make your application/design even more complex – depending on your scenario of course. This is not a problem of the extender, but a general problem one usually faces when working in OSGi. Everything is intended to be generic, but at some point you have to be specific on what you actually wonna do. So, I guess it is not a real drawback, but something to consider.
- Last but not least every automatic mechanism requires awareness of developers. Everyone using your bundles should be aware that something is happening behind the curtains. Otherwise chances are high the mechanism will be circumvented or miss used. Again, I think this holds true for many mechanism or patterns, but is well worth mentioning here.

advantages (assuming done right):

- First of all an extender means less burden for the developer. The overall goal is to remove boilerplate code, typical copy & paste use cases and deterministic, error prone tasks. For most developers this is not only boring, but also a major pain point when fixing a bug or misconfiguration. By eliminating or at least minimizing such repeating tasks, you’re doing the average Joe and ultimately yourself a big favor.
- You might have guessed it. Eliminating c&p from your code base also makes your application more robust to changes. I mentioned that already above, but just let me emphasize that the ability to express the intend, rather then the actual how is one of the greatest advantages.
- Inversion of Control. Yes, I know it is a buzz word and it is usually used grasp attention, but here it really makes sense. Like the whiteboard pattern one does remove the burden of the consumer. What kind of burden this is, depends on the extender. In many cases it will be registering the services the right way, indicating a specific functionality or setting a configuration. The beauty about this is similar to the white board pattern. Less burden for the “extendee” (the one being extended by an extender) and a centralized management logic, which can be updated, improved by the “extender”. I think it is important to note here that the extender can provide or consume, depending on your design, unlike the white board pattern.
- Of course, one advantage generates others like less source code for instance. Unless you’re getting paid by lines of code you’re producing (just a joke -I honestly hope no one is evaluated on that one!), having a smaller code base is always a good thing. Other advantages would be less complex extendee bundles or simpler adaption to new requirements.

Conclusion and outlook

As you could see, extenders are the next evolutional step in the OSGi world. There are great new ways of designing your applications that can make your application even more robust and simpler to use. However, there also comes the responsibility to use the technology wisely and not just for the sake of it. As seen with Eclipse and now Spring, more and more advanced extender concepts are popping up and we certainly haven’t seen everything possible yet. In a future blog post, I am going to look deeper in how to create an actual extender, what to keep in mind, how the up-coming OSGi R4.2 spec will help us here, what kind of extenders are there already and of course common pitfalls one may face implementing your own. Stay tuned and have fun experimenting with extenders.

Cheers
Mirko

Yesterday I read about the latest news concerning Sun’s plans about the future of JSR 277, JSR 294 and its new plans on inventing componentization for its JVM called Jigsaw. I hate to say it, but for me, it seems like Sun behaves like a small child trying to insist to be the one driving all development and not letting someone else play with its toys. It’s anything but the behavior of a well establish and industry leading company or one who wants to become one. First they completely ignored the problem of componentization in Java while persuing enterprise development with J(2)EE and missed their chance to actually set the foundation for real software reuse what you would expect from and enterprise ready language specification. Later, when OSGi began to rise they started the JSR 277 and tried to create a competing standard in secret, which didn’t work out too well as we all know. Now, because of the pressure of the community and the lack of showing a better solution they are abandoning the JSR and start developing their own “internal” componentization approach with the Jigsaw project. Of course they are claiming it is not intended to be used outside their own use case for componentizing the JVM, but it is hard for me to believe this. Call me paranoid, but for me it sounds much more like an attempt to develop another system, which after completion is suddenly moved into an official standard. Sun’s statement that they are going to revive JSR 294 and are inviting even the OSGi Alliance to participate to work on it feels more like a distraction so that Sun is able to pursue its plans with Jigsaw in private that without public notice they suddenly can come up with a self made de-facto standard. Again, I might be to paranoid, but it just doesn’t feel right. Is all that just a coincident? I don’t think so. Why can’t they embrace the work done already and see it as a great chance to propel their Language and create a true reusable software stack, no other vendor can offer? Hal Hildebrand just blogged about Sun’s attempt to introduce this new project and the way they are trying to persuade big industry players about their great intents… Well, you should really read his post about it and please tell me, if they have such humble goals, why does everyone they consult have to sign a NDA? I strongly believe that if those ideas are so great, why not share them and let the community decide and participate?

To wrap things up, I ask all of you who feel like me, share you’re opinion! Comment on Hal’s post, blog about it, link to it, spread the word. I believe we – as a community – have to stand up and say what we think to show Sun, that those kind of guerrilla tactics, especially when so easily to look through are not working. Keep your eyes open and don’t fall for the dark side

Cheers,
Mirko

So, what do I mean with that. In short, the common APIs you are usually faced with when dealing with plain Java applications. You have a class which represents your API, you may have some configuration files to set your system up and that’s it – plain simple. Logging is a perfect example for that. Let’s take any of your preferred, non OSGi logging APIs. You have a configuration file or system properties, which set the basic configuration and then you are good to go. Just call a static factory, obtain you logging object and start what ever you wanna do with it.

Now, what is the problem with this in an OSGi context? At first glance, nothing. You can set-up your system and it behaves like in plain Java. Problems only occur when you start trying to use it in a dynamic, typical OSGi-way. Changing a configuration during run-time, providing the configuration in a different bundle or fragment will require you to take more things like the start order into account. This in return makes your bundles no longer following core principles of OSGi, which say that as soon as a bundle gets resolved, it can be used.

Solving this seems hard. You can’t change a public API. Not like usual, just extract interfaces and make the implementation available as a service. Well, in some cases this might be possible, sure it won’t be in all cases. Looking at this issue, what is the core problem? The export of the API packages! Assuming, you only depend on packages (not using Require-Bundle) bundles, depending on a certain API become resolved, as soon as this API becomes available as an exported package.

A possible solution to this dilemma is treating export packages like services. During run-time, a bundle can explicitly add or remove exported packages. With this, you would be able to only export a certain API when you are certain, it will work as expected. Of course, this needs more thinking, because this imposes several problems (Security is just one of them), but the potential is great. To be honest, I am currently not sure if it is worth the risk or even the effort, but I am definitely sure it is worth exploring. Just think of the possibility to actually eliminate the dynamic import header with all of its drawbacks and removing proprietary workarounds like buddy class loading. So, what do you think?

Cheers,
Mirko

Now, despite the existence of Java or even OSGi on an iPhone, what are these new, cool and so desired applications, we can’t wait to see on our hand held devices? I gave it some thought and here is the list of “killer apps” I would love to see on the iPhone sooner or later.

• Google Maps with GPS: No need to explain that much, this is THE application I am waiting for on the iPhone. No more expensive navigation software extensions for different countries will be needed anymore. You always have it at hand! (Ideally with offline capabilities to enable the use even withouth internet access.)
• Instant Messaging: Something like fring is due for a long time. You never need to be offline, no need for expensive “Text Messages”, just write your Messages, without length limitations and even with pictures and other gimics. Not to forget features like Voice over IP via Skype for instance.
• Environment controller: Let’s call it the ubiquitous remote. You can not only control all your HIFI and TV equipment with it (ok, we need blue tooth here, but maybe in 3-5 years…), but also open your garage door, the car, dim the light in the room (the one where you are, based on location information) or your favorite computer game with the gyroscope in the iPhone. Why having tens of different applications, if one can do everything for you?
• Black box: Ok, this can be a good and a bad thing, I have to admit, but looking at it from a purely exploratory possitiv perspective, an iPhone can be used as a sort of a black box in case of accidents. Later, officials can be enabled to reconstruct the precise movements of the iPhone. A little bit like a flight recorder in airplanes. One can record where, with which speed, which movements the fon was moved.
• Emergency tool: Not only a phone, which calls 911, 110 or what ever the emergency number of the country you are in is, but also a live safer for others – with applications helping people to apply first aid step by step, with pictures and animated videos to show how it is done properly. An application capable of automatically submitting emergency data to the doctor nearby in an ad-hoc network as soon as the doctor authorizes himself.
• Interactive Games: Games are nothing new – no big deal about it. Something, which is new, are the kind of games introduced by the Nintendo WII. Equipment like a gyroscope or a position tracking system are new and animate people to actually play together. The iPhone can push this even further. With a mobile device, you can play anywhere with everyone, thus making playing a social experience. Just imagine a game you can play on your way to work, with someone in your train. Based on an adhoc-network, you probably don’t even now the one, but still can play against that individual. Even better are big sport events. Here thousands of people can be enabled to play against each other in some sort of a parallel game. A very interesting field, I have to admit. So many things are involved or have to thought about here: how to handle ad-hoc networks, how to negotiate the right score between two parties and avoid cheating, how to handle online/ offline behavior, how to secure the communication and manage trust in spontaneous networks,… Maybe I have to elaborate on this one in a later post, this topic just fascinates me, although I actually don’t play games.
• My Personal Buddy: An application, I am imagining for years now. An “intelligent” agent helping me to organize my life, by proposing appointments according to my daily behavior, my favorites and so on. I think Peter Kriens once wrote something more detailed about some of the functionalities I would expect.

As you probably noticed, I am convinced of the success of the new iPhone and I already have some ideas, what might come in the future. However, the iPhone is surely not the answer to all questions and still has several drawbacks, but it is certainly a move in the right direction. Let’s seen what’s coming up next.

Stay tuned,
Mirko

Fragments, like bundles are just simple jar files with additional manifest headers; actually they are almost the same as the bundle manifest headers. At first glance, bundles and fragments look alike. Both are jars by definition and specify additional manifest headers. The huge difference is that a Fragment is like virus. It needs to be attached to a host in order to function. During the attaching process of a fragment to a exactly specified host, the manifest files were logically merged if not conflicting and the dependencies are being resolved, if possible (if not possible, the fragment will just not be resolved and attached). After that, the resources in the fragments are loadable with the host ClassLoader, thus the Fragment does NEVER have its own ClassLoader (but its own ProtectionDomain if not an extension fragment, but we are not going there). Actually, there are quiet a few more things worth mentioning, but to understand what I am training to explain here, it should be fine. If something beyond that is needed, I am going to explain it as we go.

Now, that we know, what a Fragment is, what is it actually for where is the problem. Well, the purpose as described in the spec is pretty straight forward and logic. For customization of your bundle depending on the environment, often only very little, yet delicate changes are sometimes necessary. This can be a simple localization issue of your translations or the packaging of system dependant libraries like “dll”, “.so” or “.dmg” you are invoking in your bundle code. Depending on system properties like “os.name” or “lang” you can let the container decide, which bundles/fragments suit your environment – so far so good.

Now, where might be the problem with that? To motivate this, one has to keep in mind that one of the strengths of OSGi is its encapsulation model. Based on this declarative, strong and rigid component definition, OSGi became a very robust deployment format/ environment. A bundle or component is very expressive about its requirements and features, which makes it easy to handle them. Now, with Fragments attached to a host, the behavior of the host might change, without any changes of the version information within the bundle. The former strong formalism is now softened. Especially if you look into Software Product Lines, where you have thousands of bundles in many different versions, dynamically wired together, you can’t test every combination effectively; you have to be sure, that you can rely on a certain behavior, enforced by the component contract, which is in this case no longer possible.

The question is how a fragment potentially manages to violate the component contract. Concerning this question, there has been a discussion on the OSGi-Dev list this week. Just imagine you have a bundle X with three packages a, b and c. Fragment F defines one Import-Package: b. Another Bundle Z provides the package b as an export and has c internally available (see figure).

We further assume that bundle X defines in package a only a BundleActivater loading a class from package b. Now we have everything set, to begin our small example. Starting the bundle X (with the attached fragment F) will result in loading the BundleActivator. The activator then will try to load the class from package b. Depending on the OSGi class loading strategy, the class loader first looks into the import statements of the host, if there are no entries (like in our case), the fragment is queried for import statements. This time we’re successful and the ClassLoader is trying to obtain the class from another bundle. Here we have our problem. The behavior of X can be altered by providing another implementation of package b as an export of bundle Z in the container (it has to be a different bundle – import your exports in the fragment won’t work in that case). The problem here is not, that the package is replaced with an alternate implementation; this is pretty common in OSGi with the best practice of importing your exports. The crux here is that there is no statement of imports and the actual developer of the bundle might not have intended to replace the package at all.

Of course this behavior has advantages as well. Consider you have a OSGi-efied bundle, which unfortunately isn’t converted perfectly, so you still have some Class.forName() calls in one minor package. The license however doesn’t allow you to touch the bundle at all, but now with the help of a fragment, you can inject your own code as described above. Not sure, if one really needs that, but you can.

Something else you can do with fragments is to emulate a global class space. I was actually working in on a project, where we had a pretty monolithic software in terms of the core DTOs. Although “Extensions” were easy exchangeable and loadable with different class loaders, the core data transfer objects were not, unless you serialize them, which of course was out of discussion. Now imagine, you have one core bundle, where every extension contributes its DTOs in form of a fragment, you end up with a global class space for all your DTOs. Nice on first glace, it becomes nasty soon, when you think about the implications. First of all, versioning is now only very limited available. Second, every new deployed fragment lets your whole application refresh, because everything depends on this core bundle and every fragment is attached to it – at least for me, no desirable solution.

The finish this (actually pretty long) post today, I can think of one last case, where it can help. Imagine, you have multiple versions of a package available and your bundle requires a distinct version, because of strange constraints, but can’t make these imports explicit, but dynamic with DynamicImport-Package, normally you don’t have a chance to provide a distinct version. With a fragment, you are able to inject a distinct version before the DynamicImport-Package definition is used in the class loading strategy.

However, I tried to show different ways of using fragments, besides the usual ones of providing translations or system dependant libraries. Everyone can of course do what ever he or she wants, but in my opinion, all these alternative usages are extremely dangerous and I can’t think of any sane reason to use them in real projects. Just because you can do something doesn’t mean it is go to do so. I gave it quiet a thought and I actually came to the conclusion, that the class loading behavior should be changed in a way that, no matter what happens, the bundle class path lookup should be performed entirely, before any attached fragment is asked. This would be a valid solution to prevent violations of the bundle contract in my opinion, but well… these are just my 2 cents. Take it or leave it

–
Mirko

The iPhone is way more powerful than regular cell phones and provides a perfect platform for next generation mobile applications with its big display and the new intuitive user interface. Even better, with the eRCP project, Java already has a simple framework foundation for mobile applications and together with native GUI support thanks to eSWT, performance and usability should no longer be an issue. I haven’t heard about any plans to provide eSWT support for the iPhone yet though, but as soon as Sun provides a JVM at the end of June, I guess it is just a matter of time – at least I really hope for it!

Going back in time one can say that Sun missed the opportunity to conquer the mobile market. Java is deployed on millions of devices, but real business only a few companies managed to do with it. Now, with a powerful phone, technologies like OSGi, eRCP and native eSWT widgets, I am really curious how Java competes against Google’s Android. The newcomer against the old-timer, does a wealth of experience and a lively community stand against a giant with the wind of fortune and the hype of the brand in its sails? I guess, there is a good chance for Java and I really hope to see my Java applications running on an iPhone, but that’s a different story. Maybe in a few years my dream comes true and mobile phones no longer need a Java light aka J2ME any more, but will ship with a full blown JSE. So every jPhone ehm, I mean iPhone is sold with Java on board.