Why OSGi Qualifiers Aren’t Working

13. April, 2012

If you don’t understand how OSGi bundles get versions: You’re not alone.

On paper, the rules are pretty simple and straightforward.

In reality, the rules are broken by many Eclipse bundles because the tools don’t help to enforce them (Alex Blewitt wrote two great posts about that: “Why OSGi qualifiers aren’t working” and “Using Humans to solve a Tooling problem“). It’s not a rare problem either. Alex found 10% of the bundles got a new qualifier but didn’t actually change. That doesn’t take bundles into account which did change but the version wasn’t bumped.

When I started on an automated converter to turn Eclipse bundles to Maven artifacts, I hit the same problems. Some bundles get rebuild for no apparent reason, some have changes but the version wasn’t bumped.

This causes some problems. First of all: Which of those two qualifiers is “bigger”? “v20120119-1537″ or “xx-20120301-1000-e37-RELEASE”?

And if you think that’s probably a mistake: That’s the qualifier for org.eclipse.jdt.core.source. It’s one of the core bundles for Eclipse. If even the JDT people don’t get it right, there isn’t much hope.

When  building something with Maven, you have something similar: SNAPSHOT versions. But unlike Eclipse,

  • Maven forces you to drop the SNAPSHOT when you build a release
  • Maven replaces the string “SNAPSHOT” in the version with a build timestamp. This gives a consistent version scheme.
  • There are tools that check for SNAPSHOT versions
  • Maven can’t mix SNAPSHOT and releases in a repository (so you’re less likely to accidentally pollute your build or, worse, the build of someone else).

Unfortunately, OSGi have abandoned -SNAPSHOT versions for R5.

But maybe we can fix the problem on the Eclipse side. If you care, support Bug 376718 - Strip qualifiers for release builds.

How I Came to Hate Orbit

29. February, 2012

Orbit is an Eclipse project which contains IP clean OSS code to be used in Eclipse projects.

Why is that important? Well, IP clean means that big companies who consume Eclipse code, can use it safely (their lawyers have to check the EPL once and after that, their developers can use any code they can find on eclipse.org).

Now, Eclipse writes a lot of source code but not everything. commons-io, for example, contains a lot of code which would be really cool to have for Eclipse projects.

Orbit was created to have just that: Copies of OSS code from all kinds of places relicensed under the EPL. Sweet.

Enter stage: maven.eclipse.org

The idea behind maven.eclipse.org is to provide projects outside of the Eclipse foundation a place where they can find Eclipse’s OSGi bundles as Maven artifacts (which are easier to consume when you build your projects with Maven). Sweet 2.

To make the Eclipse bundles available on maven.eclipse.org, I need to convert them. Part of that process is to assign each Eclipse bundle (which has bundle ID and a version) a Maven coordinate (which consists of group ID, artifact ID and version). That’s not always simple but I’ve found a rule that works pretty well.

If it wasn’t for Orbit.

Orbit contains bundles which you can also find on Maven central. One example is commons-io.

And Orbit didn’t simply copy the JAR from Maven Central – they changed it. In this case, the changes are purely for legal reasons (EPL license, moved a couple of files around). Not a big problem here. I could map the two to the same coordinate – except that you would sometimes build software which contains code licensed by Apache and sometimes code which is licensed by EPL. Worse, some other code might be under the viral GPL on Maven Central – linking that could turn your project into open source! Not a problem for most developers but it could be a problem for your legal department and guess which head is going to roll?

But there is more.

Let’s have a look at … org.apache.batik.pdf. This is a fragment of the Batik’s rasterizer JAR from Maven Central. In this case, we have several issues:

  • Orbit has split a single artifact from Maven Central into several Orbit bundles. Which bundle should get the same coordinate as the original Maven Artifact?
  • The Orbit bundle also contains code from commons-io and commons-logging. So it pollutes your classpath with classes that you don’t expect. Worse, someone (not the Eclipse guys) removed “unnecessary” methods from some classes copied from commons-io. So if the compiler sees this JAR on the classpath, you will get errors for many methods in the class IOUtils. Bad.
  • Some Orbit bundles have bugs fixed but the version hasn’t been changed – only the qualifier changed which works if you use OSGi to build your classpath but Maven doesn’t. A couple of bundles from org.apache.batik are among them (bug 329376).

While it might make sense from Orbit’s point of view to do this, it makes my life a bit complicated.

Building RCP Apps With OSGi

16. March, 2011

Dave Orme wrote a really interesting article about building blocks of a RCP application with OSGiThe OSGi Building Block Pattern: An Invitation

I agree with him: The RCP wizard should really create projects to build a p2 repo and to package the bundles and features into something that a user can download and install.

Right now one of the major stumbling blocks when starting with Eclipse projects is that they either don’t build at all or that I fail to bundle/package them into some “output.”

The typical situation is that I’ve managed to import the project into my workspace. Now I get a lot of compile errors because bundles are missing in my IDE. Problem: I see the names but I have no idea at all where to download them. (See bug 340014 – “Offer a quick fix to install missing dependencies from p2 repositories”)

After manually googling for bundle names, trying to find the p2 repo which might contain them (in former times, p2 repositories offered a way to quickly browse them with a web browser – that doesn’t work anymore, so it’s poking in the dark). After a couple of restarts, the compile errors are gone.

At long last, I can start to fix my problem.

But now what? How can I create the “thing” that I need? (where “thing” can be a RCP app, a p2 repo, a bundle, a feature). Eclipse doesn’t allow to save the final after-build-step anywhere. Users must remember the steps: Export…, select the correct tool out 500, fill out the 100+ options in the little dialogs that pop up, rinse, repeat.

Welcome bug 340018 – “Allow to save export actions in a “launch” config”

Debugging BIRT: How do I enable logging for OSGi/Equinox?

12. July, 2010

If you ever tried to enable logging for OSGi (Equinox) because starting the BIRT engine fails for mysterious reasons, you will have noticed that BIRT removes all osgi.* options from the System.properties before it launches (see ).

Instead, it expects these options in config.ini (which must be in the current folder):

# Specify the file with the debug options. See the .options file in the org.eclipse.osgi*.jar for examples
# Change the classloader. Possible values are: "app", "fwk", "boot" (default)
# app: Use the current SystemClassLoader
# boot: Use the boot classloader
# fwk: Use the classloader which was used to load OSGi.

Use fwk if you see errors because of missing XML parser classes. The Java runtime has a private static field which contains the XML parser factory and if you touch any XML code before you start OSGi, then that field will be set and OSGi will be forced to use this XML parser — only the default boot classloader can’t see the parser. Bummer.

Jazoon 2010, day 2

7. June, 2010

This is my report of day 2 (see my posts about day 1).

Total Cost of Ownership by Ken Schwaber

This talk was basically about Scrum and the fact that you can’t get something of value for free. Or to put it another way around: If you save some time today by quickly hacking up a feature, you’re gonna pay in the future. There is even an interest on this, so the later you have to pay, the more expensive it will get.

So the next time your boss asks you to do something “quick”, ask him whether (s)he is aware of the total cost and whether (s)he is really willing to pay it.

Unleash your processor(s) by Václav Pech

We all know that CPU’s don’t get faster, they just reproduce faster. PCs sold today have 4 cores (and each core can execute two threads at the same time). In 2012, intel is planning to release a 50 core chip and that’s just peanuts to what you can find on your graphics card (which can have roughly 600 very simple CPUs on a single chip).

The main problem here is that we, as humans, are extremely good at parallel processing at the hardware level (most of our body continues to works while we talk, think, eat, etc.) but we’re extremely bad when thinking about parallel problems.

Concepts from the HPC world and mainframes come to the rescue: Actors, Fork/Join, Parallel Arrays, Agents and Dataflow.

The idea is to get away from the tedious synchronization and use data structures which are already thread-safe and then write simple algorithms which are invoked by a framework on an as-needed basis. Imagine you have a huge amount of images to scale to thumbnails. The algorithm is always the same and it works independent of the input. So you can allocate a number of generic worker threads. Each of them gets a copy of the algorithm at runtime plus the arguments (image and thumbnail file name).

Then you have an algorithm to traverse the directory tree which produces the input and output file names. Instead of doing everything yourself, you take a parallel array and add file names as your tree workers discover them. This will trigger the thumbnail workers.

The interesting thing here: No synchronization. You don’t even write the threads. All you do is a single call:

inParallel (filenames, thumbnailer);

The thumbnailer is just as simple:

public class Thumbnailer extends Actor {
    public void act (Object item) {
        File input = ((File[])item)[0];
        File output = ((File[])item)[0];
        ... insert favorite scaler here ...

See? No synchronized, volatile or extends Thread. Can’t wait? Check out JSR-166y

JavaFX: Designer developer workflow by Martin Gunnarsson and Pär Sikö

Tough one since I couldn’t decide where to go. HTML 5 with WebSockets? Maven 3.0?

Mouth-wetting talk about what you could do with software if everyone was just a little bit more open. In the talk, they showed how you could draw something in Photoshop and then export the design and use it directly in JavaFX.

It also showed some of the new features of JavaFX 1.3 which seems to follow the historical model of Java: 1.2 is the first version which is really usable. But it’s nice to see some progress in the Java world at last. I just wished we’d have got these five years ago when it would have mattered :-)

Lunch break. :-)

JavaFX – The condemned live longer by Andreas Fürer, David Sauter and Daniel Seiler

Along the same lines as the previous talk but this time, it shows some of the dark sides of JavaFX. Mostly boils down to: If you want to do fancy graphics in JavaFX, just do it. If you want to use it for more traditional UIs , then think again. Everything but the most simple components are still missing and those which exist sometimes have ugly bugs. :-(

My conclusion: Immature, oversold technology (or in managese: Sun’s bold response to the threads imposed by RCP and Flex/AIR).

Patterns and Practices in Hibernate by Patrycja Wegrzynowicz

Hibernate might be the most successful OR mapper for Java but it’s not the most simple (which is partly because of the documentation and that the problem itself isn’t very simple). I own the standard book about Hibernate, too, and I can agree that it explains in detail all the great features of Hibernate but it doesn’t answer “Why would I use that?”

Patrycja did a code review of the examples in the book and came up with lots of small problems and a couple of major ones (like code which doesn’t lock the rows in the correct order leading to an illegal state in the database). I talked to her and she wants to put the results of her code review online. This would mean we’d get a project with correct examples for using Hibernate.

She also mentioned SOLID which is an acronym made up from acronyms and stands for:

S – SRP (Single responsibility principle),
O – OCP (Open/closed principle),
L – LSP (Liskov substitution principle),
I – ISP (Interface segregation principle),
D – DIP (Dependency inversion principle)

This blog also explains it very well.

Migration to JPA – real life experience by Jan Sliwa

It’s always interesting when marketing hits the real world and all those buzz words are stripped to the bones. Jan talked about how to build a Java application which connects data centers all over Europe which contain sensitive data (medical records). To make the data secure, they applied a simple solution: The personal data is stored on the computers of the responsible doctor and only the medical files are saved on the servers. This means that the medical data itself is anonymous.

Two of the problems they encountered were:

  1. Creating an EntityManagerFactory is expensive. When do I open/close one? Is one enough for the whole application? Do I need a pool?
  2. How do I know whether an object is detached from the session?

He also talked about problems during testing. Maybe he should read my blog more often :-)

Managed JPA in an OSGi framework – getting the best of both worlds by Tim Ward

OSGi is a framework for the paranoid. By default, it hides everything. So how do you expose your model to both the JPA framework and all the other places where it is used?

Tim explains the problems they encountered and how they solved them. My conclusion: For your problem, OSGi is not the solution. Spring and similar frameworks have shown how to do DI properly and Maven has shown how to handle dependencies. OSGi more and more feels like a remnant from the cold war where no one trusted anyone.

That’s for all for day 2. Next: day 3.

Jazoon 2010 Day 1

2. June, 2010

So, this is the great wrap-up of Jazoon 2010, day 1. What did I have?

The keynote by Danny Coward

Java SE and JavaFX: The Road Ahead. After the acquisition by Oracle, everyone was curious as to what happens to Java. Unfortunately, the slides aren’t online, yet but from my faint memory, we might get closures after all and with a sane syntax, too. Plus all the stuff mentioned on Sun’s JDK 7 page. ATM, this stuff is a bit fluent and it’s hard to get a definitive list but something is moving at least.

From my point of view, closures and all the other language features are too late for the Java language (important companies won’t upgrade to Java 7 and time soon, some of them even cling to 1.4!) but the implementation in the main language of the Java VM will allow to build better and faster non-Java languages on top of the VM. Now if the VM would include a compiler API to build JNI code for native libraries on the fly, we would have a worthy challenger for .NET. Yeah … I know. A man can have dreams, okay?

And there was some talk about JavaFX. It seems that the technology is starting to reach its beta-phase, soon (see my notes for the second day). He showed one demo: Geo View of Vancouver 2010. It’s a world map with which country won how many medals and when you open one of the blobs, you get the names of the athletes in a fan-out widget. You can click on the name to get more information (like the photo) or you can compare the results against countries with the same number of athletes or population or closest GDP or just closest geographically. It gives a nice example how to visualize a lot of data and wade through them intuitively.

Client / Server 2.0 with Java and Flex by James Ward

James showed how you can use Flash and a Java server to build really nice web apps. He showed several examples: A few lines of code to build a UI which runs on an Android mobile phone, in the web browser and on the desktop. All with really nice performance. One was the insurance company demo. Just enter some arbitrary data until you come to the damage details and incident report. They show new ways to enter information which make the tool usable to anyone who can recognize a car and a top-view of a street.

If you like what you see, you should probably take the Tour de Flex. It shows off a whole lot of stuff. Also try the Tour de Flex Dashboard. It shows you in real time who looks at what part of the TdF right now.

Blueprint – Modern Dependency Injection for OSGi by Costin Leau

Another DI system, this time tied to OSGi. Nothing really exciting here. The talk was okay but the speaker soon lost my interest.

One thing to note: Eclipse 4 comes with a different DI system. I wonder if they will drop that in favor of the new OSGi standard in 4.1.

Patterns and Best Practices for building large GWT applications by Heiko Braun

I went to see this but quickly realized that I’ve heard the talk before at the JUGS. Here is the link to the slides. As a result of his experience he started project errai which collects best practices to build large GWT applications.

Objects of Value by Kevlin Henney

One of the main weak points on software development is that we don’t know what we’re talking about. When my project manager comes to me and asks “When are you done?” my answer is “Soon” … Right ;-) Or think about strings. Everyone else on the planet calls it “text”.

Obviously, Kevlin had a lot of fun on stage and so had we. In essence, “Objects of Value” or “Value Objects” are even more simple than POJOs (think Integer class). The main reason to use them is to make your code more expressive and readable. Instead of

public User (String name, String firstName, int age, String zipCode, String city)

you (can) create a couple of value objects:

public User (Name name, FirstName firstName, Age age, ZipCode zipCode, City city)

This may sound ridiculous (and it is in this example) but in a lot of places, using String is just a form of bad laziness (the kind of laziness which leads to maintenance problems later). One of the advantages of the approach above is that you notice when you mix last and first name because the compiler will tell you. The major disadvantage is that it leads to a class explosion. Not to an instance explosion since we just replace a String value object with something that tells us what we have, though.

In addition to that, Java isn’t really meant for these kinds of objects. There is a lot of boiler plate code to define value objects and to use them. But if you have a system that is sufficiently complex and you use a value with a unit in many places (think of a currency value), you should really consider to replace the String+BigDecimal combination with a value object.

Many important points of his talk can be found in the paper Objects of Value on his homepage.

This concludes the first part of my Jazoon 2010 report. Go on with part 2.


Get every new post delivered to your Inbox.

Join 315 other followers