All the small things

Usually when I am blogging, I am talking about the latest technology, standards or general trends. This time however, I wonna talk about politics. No, not about the elections in US, but about politics in software development practiced by the big players to achieve their business goals. Don’t get me wrong from the start. I think this is completely normal in general. We all are trying to achieve our goals the one way or the other, but something just doesn’t feel right…

Yesterday I read about the latest news concerning Sun’s plans about the future of JSR 277, JSR 294 and its new plans on inventing componentization for its JVM called Jigsaw. I hate to say it, but for me, it seems like Sun behaves like a small child trying to insist to be the one driving all development and not letting someone else play with its toys. It’s anything but the behavior of a well establish and industry leading company or one who wants to become one. First they completely ignored the problem of componentization in Java while persuing enterprise development with J(2)EE and missed their chance to actually set the foundation for real software reuse what you would expect from and enterprise ready language specification. Later, when OSGi began to rise they started the JSR 277 and tried to create a competing standard in secret, which didn’t work out too well as we all know. Now, because of the pressure of the community and the lack of showing a better solution they are abandoning the JSR and start developing their own “internal” componentization approach with the Jigsaw project. Of course they are claiming it is not intended to be used outside their own use case for componentizing the JVM, but it is hard for me to believe this. Call me paranoid, but for me it sounds much more like an attempt to develop another system, which after completion is suddenly moved into an official standard. Sun’s statement that they are going to revive JSR 294 and are inviting even the OSGi Alliance to participate to work on it feels more like a distraction so that Sun is able to pursue its plans with Jigsaw in private that without public notice they suddenly can come up with a self made de-facto standard. Again, I might be to paranoid, but it just doesn’t feel right. Is all that just a coincident? I don’t think so. Why can’t they embrace the work done already and see it as a great chance to propel their Language and create a true reusable software stack, no other vendor can offer? Hal Hildebrand just blogged about Sun’s attempt to introduce this new project and the way they are trying to persuade big industry players about their great intents… Well, you should really read his post about it and please tell me, if they have such humble goals, why does everyone they consult have to sign a NDA? I strongly believe that if those ideas are so great, why not share them and let the community decide and participate?

To wrap things up, I ask all of you who feel like me, share you’re opinion! Comment on Hal’s post, blog about it, link to it, spread the word. I believe we – as a community – have to stand up and say what we think to show Sun, that those kind of guerrilla tactics, especially when so easily to look through are not working. Keep your eyes open and don’t fall for the dark side

Cheers,
Mirko

A lot of people are following the buzz about OSGi now and are excited about all the things it can do for you. A great world of new opportunities. Now, after the first draft of OSGi 4.2 it very much looks like we’re getting even more as an official OSGi standard (with RFC 124). The formally called Spring Dynamic Modules (in short SpringDM) is going to be OSGi-efied. Today, I want to take a short peak into what this can actually mean and how it could affect the software/ component reuse in your software stack even today.

First things first, when I talk to people about OSGi the most common complain is nothing OSGi specific at all. Basically it is the problem of every API out there. If you implement against an open or proprietary API you’re bound to it to some extend. Ok, this is besides the issue of being to complex. When done right, which includes using all the versioning features OSGi offers it is unfortunately kinda true. We’re still lacking the right tooling support here. Anyway, let’s stick to the first one first (I’ll cover the tooling problem in a later post). Speaking of proprietary, it is true you have to add custom headers to your jar files, but this alone doesn’t hurt you. You can still just use the jar as – yeah well – a simple jar in your web container or any other Java application without any problems at all. This was simple, but there is more of course – the OSGi doesn’t only specify manifest headers. There is also an API you use and program against. Activators for instance can be used to do something like setting up or shutting down your bundle and opening or closing connection pools to databases, registering services signaling that you’re bundle is there and ready to get its work done. Assuming you are not using any of the compendium APIs the imposed layer is rather small and can be encapsulated in a single package per bundle and more often just one single class.

However, as soon as you start using those few classes they become part of your jar and thus you’re bound to them. So, we’re still stuck right? Well, not really. First of all, every OSGi expert (and probably any other sane software engineer) will tell you not to intermingle your domain logic with middleware APIs of any kind. As already mentioned, with OSGi, you can usually keep this layer pretty thin. With Spring DM we can make it now even thinner and separate domain concerns from the container entirely. So, how can we achieve this. Of course, Spring is not doing any magic, although it does a pretty good job. All it does is actually providing an IoC framework, which declaratively wires beans together. Plain simple speaking, that’s about it. You could also use other IoC frameworks like Google Guice or pico container if like, but the integration in OSGi with Spring is pretty unique. The big question remaining is how we can best separate these two concerns.

To start with, you design your domain logic in a bean driven way. By that you not only achieve a well testable domain code, you also remove hard wired dependencies not relevant for your core needs – a very common practice in Domain Driven Design, to throw in some buzz words Package all that code in one or more bundles depending on your needs. What you got is the code base you can reuse in any environment. Now, we need the glue code to actually make it work in OSGi – the package with the few OSGi specific classes. Those you now have to put into a different bundle or fragment (more on this later). By this separation you no longer deliver container specific code, if you don’t need to. Even more, depending on your deployment scenario, you can change and adjust the configuration to your specific needs, all separated in different bundles/fragments. Removing configurations from your domain bundles has the advantage of independent versioning of you actual code and configuration. The domain code, if used f.i. in a Software Product Line is now no longer related to the configuration of your products based on this code and can be versioned independently – enabling you to see what really changes in your application.

Fig.1 - Sample Application Architecture

This approach is shown in figure 1. You have two applications using the same library with different configurations. You can fix bugs or add features in the library without touching the configurations and vice versa. Interesting now is the difference between fragments and bundles. When should I use what? The advantage of a bundle is that it is the way OSGi intends to deploy code. You can use Activators and define your dependencies. The fragment in contrast was actually intended to only carry optional data like i18n information and alike. The advantage however is that the fragment and its host are sharing the same class loader(, which is also the biggest disadvantage and should be considered really carefully!). This distinction basically sets when to use what. When ever the configuration is loaded by the library and can’t be injected by another bundle, you need to have access to the bundles class path in order to provide the required resource. But wait a minute, you can’t use Activators in fragments. How could we now do the initialization? Here Spring DM comes in. You can just provide a Spring configuration in your fragment and as soon as the fragment is attached, Spring will recognize it and just initializes your beans defined in the descriptor and you have your Activator again. You even have access to the BundleContext if required. However, this approach is only intended for code that otherwise wouldn’t work or can’t be changed for some reasons. The best way to do it is for sure using bundles for that kinda thing.

The main point, I was trying to make here is that with the help of Spring and the up coming manifestation in OSGi R4.2 (if accepted), we have even more opportunities to build reusable and coherent artifacts, perfect suitable for software product lines. However, all the presented possibilities also impose potential for misuse and you should be careful by choosing your way. In many cases the simplest way on first sight eventually becomes a dead end, so be careful and farsighted when making your decisions.

Cheers
Mirko

OSGi provides developers with plenty of configuration and selection mechanisms to adapt your application perfectly to a certain environment. One of these features is the usage of native code. Other than plain Java in OSGi you can simply define a header like the following:

Bundle-NativeCode: lib/superDuper.dll;
osname = WindowsNT;
processor = x86;
selection-filter="(net.mjahn.sample.windowing=win32)";
language = en;
language = de

Now doing this in a fragment is a very nice feature. If your environmental requirements are met, your fragment gets attached to the host and the correct library will be loaded by your OS. On the other hand, if you deploy that fragment on another machine with let’s say a Solaris OS, it simply won’t get resolved. The advantage is you can just create an assembly with all native libraries for the different OS versions and systems in different fragments. Depending on your environment, the container will figure out, which fragment to choose and attach to the host. Piece of cake, I would say.

Recently I was faced with a similar, but unfortunately more difficult problem. I wanted to provide two different bundles containing a distinct implementation of a specific API as fragments; depending on the OS it is running on. On first sight, it is the problem, I already discussed, but on a second view, I realized that it is actually plain Java. The API, I am providing in my fragments is OS dependent, but doesn’t contain native code. I thought, I could work around it, by providing something like a selection-filter or anything similar to the mechanisms of the Bundle-NativeCode header, but there isn’t. (If I am wrong here, please enlighten me, but I haven’t found anything in the 4.1 OSGi core specification!).

Of course, if you have the possibility, you can just implement your own mechanisms to do what OSGi does for native code, just with plain old Java code. Check the environment and load the correct libraries or even simpler, just only provide the correct fragment versions. This is all true; it definitely would work that way. The question, I am asking myself is more, shouldn’t this actually be an infrastructural concern? Shouldn’t the infrastructure be able to decide what is necessary and suits best? In my opinion, one of the big advantages of OSGi is the expressiveness of constraints. Everything is explicit, so from my point of view, it is logical to enable such declarative dependencies without native code as well.

One question remains: Where should this mechanism be added? Like with singleton bundles together with the “Bundle-SymbolicName”? I wouldn’t say so. I think the “Bundle-RequiredExecutionEnvironment” would be a perfect spot for that. A selection based on System properties is flexible, yet distinct enough, so the already introduced “selection-filter” could be applied here. The execution environment in general fits the scope of this property way better than any other, already defined in OSGi.

Well, unfortunately, I am not an OSGi member and the next spec will take a while to come, so I guess this is nothing to be expected any time soon, if at all Maybe I am wrong about all that, because I am missing something, but I actually haven’t found any evidence stating the opposite. Till then, I hope for the best – a more compatible (IT) world.

– Mirko

When first diving into OSGi, you probably don’t notice fragments at all. After a while, once you become familiar with the concept and the main parts of the spec, you start looking into ways to accomplish things, which were fairly simple done in plain Java, but might be really painful in OSGi. By then the latest, you’ll find them in the spec and they are really tempting in so many ways. Before I elaborate on that, let me just summarize the main features of fragments in order to give everyone a chance to follow.

Fragments, like bundles are just simple jar files with additional manifest headers; actually they are almost the same as the bundle manifest headers. At first glance, bundles and fragments look alike. Both are jars by definition and specify additional manifest headers. The huge difference is that a Fragment is like virus. It needs to be attached to a host in order to function. During the attaching process of a fragment to a exactly specified host, the manifest files were logically merged if not conflicting and the dependencies are being resolved, if possible (if not possible, the fragment will just not be resolved and attached). After that, the resources in the fragments are loadable with the host ClassLoader, thus the Fragment does NEVER have its own ClassLoader (but its own ProtectionDomain if not an extension fragment, but we are not going there). Actually, there are quiet a few more things worth mentioning, but to understand what I am training to explain here, it should be fine. If something beyond that is needed, I am going to explain it as we go.

Now, that we know, what a Fragment is, what is it actually for where is the problem. Well, the purpose as described in the spec is pretty straight forward and logic. For customization of your bundle depending on the environment, often only very little, yet delicate changes are sometimes necessary. This can be a simple localization issue of your translations or the packaging of system dependant libraries like “dll”, “.so” or “.dmg” you are invoking in your bundle code. Depending on system properties like “os.name” or “lang” you can let the container decide, which bundles/fragments suit your environment – so far so good.

Now, where might be the problem with that? To motivate this, one has to keep in mind that one of the strengths of OSGi is its encapsulation model. Based on this declarative, strong and rigid component definition, OSGi became a very robust deployment format/ environment. A bundle or component is very expressive about its requirements and features, which makes it easy to handle them. Now, with Fragments attached to a host, the behavior of the host might change, without any changes of the version information within the bundle. The former strong formalism is now softened. Especially if you look into Software Product Lines, where you have thousands of bundles in many different versions, dynamically wired together, you can’t test every combination effectively; you have to be sure, that you can rely on a certain behavior, enforced by the component contract, which is in this case no longer possible.

The question is how a fragment potentially manages to violate the component contract. Concerning this question, there has been a discussion on the OSGi-Dev list this week. Just imagine you have a bundle X with three packages a, b and c. Fragment F defines one Import-Package: b. Another Bundle Z provides the package b as an export and has c internally available (see figure).

We further assume that bundle X defines in package a only a BundleActivater loading a class from package b. Now we have everything set, to begin our small example. Starting the bundle X (with the attached fragment F) will result in loading the BundleActivator. The activator then will try to load the class from package b. Depending on the OSGi class loading strategy, the class loader first looks into the import statements of the host, if there are no entries (like in our case), the fragment is queried for import statements. This time we’re successful and the ClassLoader is trying to obtain the class from another bundle. Here we have our problem. The behavior of X can be altered by providing another implementation of package b as an export of bundle Z in the container (it has to be a different bundle – import your exports in the fragment won’t work in that case). The problem here is not, that the package is replaced with an alternate implementation; this is pretty common in OSGi with the best practice of importing your exports. The crux here is that there is no statement of imports and the actual developer of the bundle might not have intended to replace the package at all.

Of course this behavior has advantages as well. Consider you have a OSGi-efied bundle, which unfortunately isn’t converted perfectly, so you still have some Class.forName() calls in one minor package. The license however doesn’t allow you to touch the bundle at all, but now with the help of a fragment, you can inject your own code as described above. Not sure, if one really needs that, but you can.

Something else you can do with fragments is to emulate a global class space. I was actually working in on a project, where we had a pretty monolithic software in terms of the core DTOs. Although “Extensions” were easy exchangeable and loadable with different class loaders, the core data transfer objects were not, unless you serialize them, which of course was out of discussion. Now imagine, you have one core bundle, where every extension contributes its DTOs in form of a fragment, you end up with a global class space for all your DTOs. Nice on first glace, it becomes nasty soon, when you think about the implications. First of all, versioning is now only very limited available. Second, every new deployed fragment lets your whole application refresh, because everything depends on this core bundle and every fragment is attached to it – at least for me, no desirable solution.

The finish this (actually pretty long) post today, I can think of one last case, where it can help. Imagine, you have multiple versions of a package available and your bundle requires a distinct version, because of strange constraints, but can’t make these imports explicit, but dynamic with DynamicImport-Package, normally you don’t have a chance to provide a distinct version. With a fragment, you are able to inject a distinct version before the DynamicImport-Package definition is used in the class loading strategy.

However, I tried to show different ways of using fragments, besides the usual ones of providing translations or system dependant libraries. Everyone can of course do what ever he or she wants, but in my opinion, all these alternative usages are extremely dangerous and I can’t think of any sane reason to use them in real projects. Just because you can do something doesn’t mean it is go to do so. I gave it quiet a thought and I actually came to the conclusion, that the class loading behavior should be changed in a way that, no matter what happens, the bundle class path lookup should be performed entirely, before any attached fragment is asked. This would be a valid solution to prevent violations of the bundle contract in my opinion, but well… these are just my 2 cents. Take it or leave it

–
Mirko