The other day, Paul Withers wrote a blog post about RunJava, which is a very-old and very-undocumented mechanism for running arbitrary Java tasks in a manner similar to a C-based addin. I had vaguely known this was there for a long time, but for some reason I had never looked into it. So, for both my sake and general knowledge, I'll frame it in a time line.

History

I'm guessing that RunJava was added in the R5 era, presumably to allow IBM to use existing Java code or programmers for writing server addins (with ISpy being the main known one), and possibly as a side effect of the early push for "Java everywhere" in Domino that fell prey to strategy tax.

Years later, David Taib made the JAVADDIN project as a "grown up" version of this sort of thing, bringing the structure of OSGi to the idea. Eventually, that morphed into DOTS, which became more-or-less supported in the "Social Edition" days before meeting a quiet death in Domino 11.

The main distinction between RunJava and DOTS (other than RunJava still shipping with Domino) is the thickness of the layer above C. DOTS loads an Equinox OSGi runtime very similar to the XPages environment, bringing in all of the framework support and dependencies, as well as services of its own for scheduled task and other options. RunJava, on the other hand, is an extremely-thin layer over what writing an addin in C is like: you use the public static void main structure from runnable Java classes and you're given a runNotes method that are directly equivalent to the main and AddinMain function used by C/C++ addins.

Utility

Reading back up on RunJava got my brain ticking, and it primarily made me realize that this could be a perfect fit for the Open Liberty Runtime project. That project uses the XPages runtime's HttpService class to load immediately at HTTP start and remain resident for the duration of the lifecycle, but it's really a parasite: other than an authentication-helper servlet, the fact that it's running in nHTTP is just because that's the easiest way to run complicated, long-running Java code. For a while, I considered DOTS for this task, but it was never a high priority and has aged out of usefulness.

So I decided to roll up my sleeves and give RunJava a shot. Fortunately, I was pretty well-prepared: I've been doing a lot of C-level stuff lately, so the concepts and functions are familiar. The main run loop uses a message queue, for which Notes.jar provides an extremely-thin wrapper in the form of lotus.notes.internal.MessageQueue. And, as Paul reminded me, I had actually done basically this same thing before, years ago, when I wrote a RunJava addin to maintain a Minecraft server alongside Domino. I'd forgotten about that thing.

Lessons

Getting to the thrust of this post, I think it's worth sharing some of the steps I took and lessons I learned writing this, since RunJava is in a lot of ways much more hostile a place for code than the cozy embrace of Equinox.

#1: Don't Do This

The main lesson to learn is that you probably don't want to write a RunJava task. It was already the case that DOTS was too esoteric to use except for those with particular talent and needs, and that one at least had the advantage of being kind-of documented and kind-of open source. RunJava gives you almost no affordances and imposes severe restrictions, so it's really just meant for a situation where you were otherwise going to write an addin in C but don't want to have to set up a half-dozen compiler toolchains.

#2: Lower Your Dependencies Dramatically

The first big general thing to keep in mind is that RunJava tasks, if they're not just a single Java class file, are deployed right to the main domino JRE, either in jvm/lib/ext or in ndext. What this means is that any class you include in your package will be present in absolutely everything Java-related on Domino, which means you're in a minefield if you want to bring in any logging packages or third-party frameworks that could conflict with something present in the XPages stack or in your own higher-level Java code.

This is a fiddlier problem than you'd think. A release or so ago, IBM or HCL added a version of Guava to the ndext folder and it wreaked havoc on the version my client's app was using (which I think came along for the ride from ODA). You can easily get into situations where one class for a library is loaded from XPages-level code and another is loaded from this low level, and you'll end up with mysterious errors.

Ideally, you want no possible class conflicts at all. I took the approach of outright white-labeling some (compatibly-licensed) code from Apache and IBM Commons to avoid any possibility of butting heads with other code on the server. I was also originally going to use the Darwino NAPI or Domino JNA for a nicer Message Queue implementation, but scuttled that idea for this reason. It's Notes.jar or bust for safe API access, unfortunately.

#3: Use the maven-shade-plugin

This goes along with the above, but it's more a good tool than a dire warning. The maven-shade-plugin is a standard plugin for a Maven build that lets you blend together the contents of multiple JARs into one, so you don't have to have a big pool of JARs to copy around. That on its own is handy for deployment, but the plugin also lets you rename classes and aggregate and transform resources, which can be indispensable capabilities when making a safe project.

#4: Make Sure Static Initializers and Constructors are Clean

What I mean by this one is that you should make sure that your JavaServerAddin subclass does very little during class loading and instantiation. The reason I say this is that, until your class is actually loaded and running, the only diagnostic information you'll get is that RunJava will say that it can't find your class by name - a message indistinguishable from the case of your class not even being on the server at all. So if, for example, your class references another class that's missing or unresolvable at load time (say, pointing at a class that implements org.osgi.framework.BundleActivator, to pick one I hit), RunJava will act like your code isn't even there. That can make it extremely difficult to tell what you're doing wrong. So I found it best to make very little static other than JVM-provided classes and to delay creation/lookup of other objects and resources (say, translation bundles) until it was in the runNotes method. Once the code reaches that point, you'll be able to get stack traces on failure, so debugging becomes okay again.

#5: Take Care With Threads When Terminating

The Open Liberty runtime makes good use of java.util.concurrent.ExecutorServices to run NotesThread code asynchronously, and I'll periodically execute even a synchronous task in there to make sure I'm working with a properly-initialized thread.

However, when terminating, these services will start to shut down and reject new tasks. So if, for example, you had code that executes on a separate thread and might be run during shutdown, that will fail likely-silently and can cause your addin to choke the server.

#6: That Said, It's a Good Idea to Use Threads

A habit I picked up from writing Darwino's cluster replicator is to make your addin's main Message Queue loop very simple and to send messages off to a worker thread to handle. Doing this means that, for complex operations, the server console and the user won't sit waiting on a reply while your code churns through an individual message.

In my case, I created a single-thread ExecutorService and have my main loop immediately pass along all incoming commands to it. That way, the command runner is itself essentially synchronous, but your queue watcher can resume polling immediately. This keeps things responsive and avoids the potential case of the message queue filling up if there's a very-long-running task (though that's less likely here than if you're drinking from the EM fire hose).

#7: Really, Don't Do This

My final tip is that you should scroll back up and heed my advice from #1: it's almost definitely not worth writing a RunJava addin. This is a special case because a) the goal of the project is to essentially be a server addin anyway and b) I was curious, but normally it's best to use the HttpService route if you need a persistent task.

It's kind of fun, though.

In my earlier post, I went over the tack I took when developing a couple extension features for Open Liberty, and specifically the way I came at it with Tycho.

Shortly after I posted that, Alasdair Nottingham, the project lead for Open Liberty, dropped me a line to mention how programmatic service registration isn't preferred, and instead the idiomatic way is to use Declarative Services. I had encountered DS while fumbling my way through to getting these things working, but I had run into some bit of trouble or another, and I ended up settling on what I got working and not revisiting it.

Concepts

This was a perfect opportunity to go back and do things the right way, though, so I set out to do that this morning. In my initial reading up, I ran across a blog post from the always-helpful Vogella Blog that talks about coming at OSGi DS from essentially the same perspective I have: namely, having been used to Equinox and the Eclipse plugin/extension mechanism. As it turns out, when it comes to generic OSGi, Equinox can kind of poison your brain. The whole term "plug-in" instead of "bundle" comes from earlier Eclipse; "features", "update sites", and all of p2 are entirely Equinox-specific; and the "plugin.xml" extension mechanism is of a similar vintage. However, unlike some other vestiges that were tossed aside, "plugin.xml" is still in active use.

At its core, the Declarative Services system is generally similar to that route, in that you write classes to implement a given interface and then declare that your bundle provides that using XML files. The specifics are different - DS is more type-safe and it uses individual XML files in the "OSGI-INF" directory for each service - but the concept is similar. DS also has an annotation-based mechanism for this, which allows you to annotate your service classes directly and not worry about maintaining XML files. It's something of a compiler trick: the XML files still exist, but your tooling of choice (PDE, bnd, etc.) will generate the files based on your annotations. It took a bit for Eclipse to get on board with this, but, as of Neon, you can enable this processing in the preferences.

Implementation

Fortunately for me, my needs are simple enough that making the change was pretty straightforward. The first step was to delete the Activator class outright, as I won't need it for this. The second was to add an optional import for the org.osgi.service.component.annotations package in my Liberty extension bundle. I suspect that this is a bit of a PDE-ism: the annotations aren't even retained at runtime (and the package isn't present in the Liberty server), but this is the only mechanism Eclipse has to add a dependency for a plug-in project.

The annotation for the user registry was as straightforward as can be, needing a single line in this heavily-clipped version of the class:

1
2
3

@Component(service=UserRegistry.class, configurationPid="dominoUserRegistry")
public class DominoUserRegistry implements UserRegistry {
}

With that, Eclipse started generating the associated XML file for me, and the registry showed up at runtime just as it had before.

The TrustAssociationInterceptor was slightly more complicated because it had some extra initialization properties set, in particular the one to mark it as executing before normal SSO. This was a little tricky in two ways: Java annotations don't have any mechanism for specifying a literal Map for properties, and the before-SSO property is a boolean, but I could only write a string. It turned out that the property, uh, property on the annotation has a little mini-DSL where you can mark a property with its type. The result was:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

@Component(
	service=TrustAssociationInterceptor.class,
	configurationPid=DominoTAI.CONFIG_PID,
	property={
		"invokeBeforeSSO:Boolean=true",
		"id=org.openntf.openliberty.wlp.userregistry.DominoTAI"
	}
)
public class DominoTAI implements TrustAssociationInterceptor {
}

Further Features

This is proving to be a pretty fun side project within a side project, and I think I'll take a crack at developing some more features when I have a chance. In particular, I'd like to try developing some API-contribution features so that they can be deployed to the server once and then used by web apps without having to package them (similar in concept to XPages Libraries). This is how Liberty implements its Jakarta EE specifications, and I could see making some extra ones. That's also exactly what CrossWorlds does, and so I imagine I'll crib a bunch of that work.

In my last post, I put something of a stick in the ground and announced a multi-blog-post project to discuss the process of making an XPages app portable for the future. In true season-cliffhanger fashion, though, I'm not going to start that immediately, but instead have a one-off entry about something almost entirely unrelated.

Specifically, I'm going to talk about developing a custom UserRegistry and TrustAssociationInterceptor for Open Liberty/WebSphere Liberty. IBM provides documentation for this process, and it's alright enough, but I had to learn some specific things coming at it from a Domino perspective.

What These Services Are

Before I get in to the specifics, it's worth discussing what specifically these services are, especially TrustAssociationInterceptor with its ominous-sounding name.

A UserRegistry class is a mechanism to provide a Liberty server with authentication and user info services. Liberty has a couple of these built-in, and the prototypical ones are the basic and LDAP registries. Essentially, these do the job of the Directory and Directory Assistance on Domino.

A TrustAssociationInterceptor class is related. What it does is take an incoming HTTP request and look for any credentials it understands. If present, it tells Liberty that the request can be considered authenticated for a given user name. The classic mechanisms for this are HTTP Basic and form-cookie authentication, but this can also cover mechanisms like OAuth. In Domino, this maps to the built-in authentication mechanisms and, more particularly, to DSAPI filters.

How I Used Them

My desire to implement these developed when I was working on the Domino Open Liberty Runtime. I wanted to allow Liberty to use the containing Domino server as a user registry without having to enable LDAP and, as a stretch goal, I wanted to have some sort of implicit SSO without having to configure LTPA.

So I ended up devising something of an ad-hoc directory API exposed as a servlet on Domino, which Liberty could use to make the needed queries. To pair with that, I wrote a TrustAssociationInterceptor implementation that looks for Domino auth cookies in incoming requests, make a call to a small servlet with that cookie, and grabs the associated username. That provides only one-way SSO, but that's good enough for now.

The Easy Part

The good part was that my assumption that my comfort with Tycho going in would help was generally correct. Since the final output I wanted was a bundle, I was able to just add it to my project structure like any other, and work with it in Eclipse's PDE normally. Tycho and PDE didn't necessarily help much - I still had to track down the Liberty API plugins and make a local update site out of them, but that was old hat by this point.

What Made Development Weird

I went into the project in high spirits: the interfaces required weren't bad, and Liberty uses OSGi internally. I figured that, with my years of OSGi experience, this would be a piece of cake.

And, admittedly, it kind of was. The core concepts are the same: building with Tycho, bundle activators, MANIFEST.MF, and all that. However, Liberty's use of OSGi is, I believe, much more modern than Domino's, and certainly much less focused on Equinox specifically.

For one, though Liberty is indeed OSGi-based, it doesn't use Maven Tycho for its build process. Instead, it uses Gradle and the often-friendlier bnd tooling to handle its OSGi composition. That's not too huge of a difference, and the build process doesn't really affect the final built feature. The full differences are a whole big topic on their own, but the way they shake out for this purpose is essentially a difference in philosophy, and the different build mechanism was something of a herald of the downstream distinctions.

One big way this shows is in service registration. Coming from an Eclipse heritage, Equinox-based apps tend to use "plugin.xml" to register services, Liberty (and most others, I assume) favors programmatic registration of services inside the bundle activator. While this does indeed work on Equinox (including on Domino), this was the first time I'd encountered it, and it took some getting used to.

The other oddity was how you encapsulate your bundle as a feature in Liberty parlance. Liberty uses the term "feature" to refer to individual components that make up the server, and which you can configure in the "server.xml" file. These are declared using files similar to MANIFEST.MF with specialized headers to declare the name of the feature, the bundles that make it up, and any APIs it provides to the server and apps. In my case, I wrote a generic mechanism to deploy these features when a server is established, which writes the manifest files to the server's feature directory. Once they're deployed, they become available to the server as a feature with the "usr" prefix, like "usr:dominoUserRegistry-1.0" for my case.

In The Future

I have some ideas for additional features I'd like to develop - providing implicit APIs for Darwino and Jakarta NoSQL/JNoSQL would be handy, for example. This way went pretty smoothly, but I'll probably develop non-Domino ones using either Gradle or Maven with the maven-bundle-plugin. Either way, it ended up fairly pleasant once I discarded my old assumptions, and it's another good entry in the "pros" column for Liberty.

...yet here we are. It's been a very interesting week in ODA circles. It started with Daniele Vistalli being curious if it was possible to run the OpenNTF Domino API on WebSphere, specifically the surprisingly-friendly Liberty profile. And not just curious: it turned out he had the chops to make it happen. Before too long, with the help of the ODA team, he wrote an extension to the WebSphere server that spins up native (lotus.domino.local) Notes RPC connections using Notes.jar and thus provides a functioning entrypoint to using ODA.

The conversation and possibilities spun out from there. It's too early to talk about too many specifics, but it's very exciting. After the initial bulk of the work, I was able to pitch in a bit yesterday and help figure out how to make this happen (click to view the full size):

That's a JSF app being built and deployed to a WebSphere Liberty server managed in Eclipse on the local machine, iterating over the contents of a DbDirectory pointed at a remote server. And it all happens to be running natively on the Mac - no VM involved.

So, like I said: it's been an interesting week. There are more possibilities than just this case, but even this alone holds tremendous promise: connecting to NSF data with the true, full API (brought into the modern day by ODA) but with the full array of J2EE support available.

Happy New Year!