The Myriad Idioms For Finding Implementations In Java

Oct 18, 2022, 10:25 AM

Tags: jakartaee java
  1. Java Services (Not the RESTful Kind)
  2. Java ClassLoaders
  3. Managed Beans to CDI
  4. The Myriad Idioms For Finding Implementations In Java

A few years ago, I wrote a post about Java service location, which covered things like META-INF/services and OSGI extensions. Today, I'd like to discuss a similar concept: code in a top-level API that finds a specific implementation. For reasons that will become clear shortly, I'll call this the "FactoryFinder pattern".

Background

Not all Java code uses this kind of thing and, while service loading is related, the overlap isn't complete. Where this does come up a lot is in a framework like Jakarta EE, which is very intentionally split between vendor-neutral specification classes/interfaces (the ones starting with jakarta.*) and specific implementations.

For example, the Jakarta REST (née JAX-RS) specification only defines various classes and interfaces within the jakarta.ws.rs package space, but doesn't include any actual implementation. That's left to various vendors. The number of implementations varies by spec, and JAX-RS is particularly prolific on this front. In the XPages Jakarta EE project, we use RESTEasy, whose classes are all in the org.jboss.resteasy package space.

There's a (usually) hard wall between these layers: the spec declares an API that programmers can use, and then the implementation has to allow itself to be called by those class names and obey the specification's rules. When writing JAX-RS resources in an NSF, the fact that it's using RESTEasy does not enter into your experience. That raises the question, though, of how this works. How does the vendor-neutral specification locate the implementation classes to hand off the work? Well, that question has a number of different answers.

Entrypoint Classes and Locating Implementations

In general, each spec accomplishes this using one or more entrypoint classes. For example, JAX-RS uses RuntimeDelegate and its static getInstance() method to locate server implementations and ClientBuilder and its newBuilder() method to load client implementations. Outwardly, these methods just promise that they'll find and provide an implementation, but the actual way that specs do this varies.

One of the most common ways to coordinate this loading is to have a class named FactoryFinder. This idiom and specific name proved very popular over at Sun as they built up the JEE specs:

Eclipse Open Type dialog for FactoryFinder

Despite their identical names, each of these classes is a different implementation, and they have different characteristics. There are routines in common, and each spec uses a subset of these. I'll go over the common ones here, in no particular order other than that I'll start with the ones found in the JAX-RS API first.

ServiceLoader

This one is used in basically every spec up until the latest era. This uses the java.util.ServiceLoader class to find implementations by way of text files in META-INF/services named after the spec class and containing implementation class names. For example, RESTEasy contains a file named META-INF/services/jakarta.ws.rs.ext.RuntimeDelegate that references the class org.jboss.resteasy.core.providerfactory.ResteasyProviderFactoryImpl. That looks like this:

1
2
3
4
5
Iterator<T> iterator = ServiceLoader.load(service, FactoryFinder.getContextClassLoader()).iterator();

if(iterator.hasNext()) {
	return iterator.next();
}

FactoryFinder.getContextClassLoader() there is a utility method that just uses an AccessController block to work with Java policy limitations like we see on Domino all the time.

This is simple enough in the normal case, but can get a little tricky when you add in something like OSGi. By default, ServiceLoader will look in the thread-context class loader, which will usually be where your application code lives. Inside an app container, like an NSF, the implementation class may not actually be visible, though. Accordingly, many of these finders fall back to looking using the class loader of the spec class, which has a higher chance of seeing the implementation. That looks similar:

1
2
3
4
5
Iterator<T> iterator = ServiceLoader.load(service, FactoryFinder.class.getClassLoader()).iterator();

if(iterator.hasNext()) {
	return iterator.next();
}

In the XPages Jakarta EE project, neither of these calls will tend to work by default, since neither the app nor the API bundle won't see the implementation bundle by default. In some cases, I deal with this via the methods below, but in others I will do so by re-packaging the implementation as an OSGi fragment bundle. Fragment bundles attach themselves onto their host's classloader fully, and this allows ServiceLoader to find the implementation.

Configuration Properties

A handful of these specs, JAX-RS included, will also look for the name of an implementation class using an external properties file. The placement of this in the priority order - as a fallback after ServiceLoader - and the classes used in the implementation make me figure that these are quite often relics of earlier habits.

JAX-RS, for its part, will look within the java.home system property, which points to the JVM's installation directory. In there, it looks for a properties file named lib/jaxrs.properties:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
String javah = System.getProperty("java.home");
configFile = javah + File.separator + "lib" + File.separator + "jaxrs.properties";
File f = new File(configFile);
if (f.exists()) {
	Properties props = new Properties();
	inputStream = new FileInputStream(f);
	props.load(inputStream);
	String factoryClassName = props.getProperty(factoryId);
	return newInstance(factoryClassName, classLoader);
}

This tries to use the thread-context class loader only, so it wouldn't work for a complex app server situation. Likely, it's meant for either an older type of application or a standalone special-purpose JAR.

System Properties

Similar to reading a designated properties file, these specs will often then fall back to looking for a Java system property of a given name. These properties may be dynamically set at runtime or may be set during the JVM launch. Often, this property will be the name of the interface/abstract class being looked up, like so:

1
2
3
4
String systemProp = System.getProperty(factoryId);
if (systemProp != null) {
	return newInstance(systemProp, classLoader);
}

This one can actually come in handy sometimes - though not ideal, I've used similar cases where I set the name of an implementation or delegation class in a property before initializing the spec. It's best to avoid that when possible, but I'm often glad it's there.

OSGi Escape

Next up is one that JAX-RS doesn't use, but shows up periodically. Though Jakarta EE isn't based around OSGi, a good number of the implementations historically have used (and still use) it, and OSGi always sits in a "not standard, but too popular to consistently ignore" limbo.

To account for this, there's a similarly semi-standard library called the OSGi resource locator. This library provides a class named org.glassfish.hk2.osgiresourcelocator.ServiceLoader that does its own search and loading for ServiceLoader-compatible META-INF/services files within OSGi bundles in the current platform. The idea is that, if you have an OSGi-based platform that you want to work with this type of loading, you will provide the Resource Locator class and let any loaders written to use it fall back to it.

Because this class is not normally present even when actually in OSGi, APIs that make use of it have to be careful and indirect about trying to load it at all. We'll use JAX-B as our example here. They'll generally try to load the bridge class reflectively, which avoids having OSGi-wrapping tools like bnd create a potentially-undesired dependency on the presence of the bridge. Then, they'll reflectively ask it to load service implementations. That tends to look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
// Use reflection to avoid having any dependency on ServiceLoader class
Class serviceClass = Class.forName(factoryId);
Class target = Class.forName(OSGI_SERVICE_LOADER_CLASS_NAME);
Method m = target.getMethod(OSGI_SERVICE_LOADER_METHOD_NAME, Class.class);
Iterator iter = ((Iterable) m.invoke(null, serviceClass)).iterator();
if (iter.hasNext()) {
	Object next = iter.next();
	logger.fine("Found implementation using OSGi facility; returning object [" +
		next.getClass().getName() + "].");
	return next;
} else {
	return null;
}

That's also generally wrapped in a big try/catch block to avoid gumming up the works if any pieces are missing.

The XPages Jakarta EE project actually contains a reimplementation of this that avoids some hurdle or other that I found with the stock version. I avoided doing something like that for a while, but it ended up being the most practical way to get some of these specs working.

Default Implementation

Back outside the realm of OSGi, a handful of these specifications will also include a hard-coded default provider class name. These are generally the classes from what used to be dubbed reference implementations and which are largely components of GlassFish by virtue of that being Sun's version.

For example, the JSON-P API has a final fallback of trying to look for org.glassfish.json.JsonProviderImpl by name:

1
2
Class<?> clazz = Class.forName(DEFAULT_PROVIDER);
return (JsonProvider) clazz.getConstructor().newInstance();

Though these implementations generally also declare themselves via ServiceLoader files, this is presumably useful in historical or edge cases where there's still a decent chance that the RI will be available. This does have an unfortunate effect on error messages, though, where the case of "I can't find any implementation at all" ends up being reported as e.g. "Provider org.glassfish.json.JsonProviderImpl not found". That's not really a problem with the approach as such, though, but rather just the way it shakes out in practice.

Manually-Set Implementation or Locator

The final mechanism I'm going to discuss is sort of a final escape hatch. Sometimes, the provider class will have a method that lets you set an arbitrary implementation yourself, without having the API do any of these lookups at all. Some, like MicroProfile Config and CDI even go one step further and provide a method that configures not just a specific implementation but rather an implementation locator. These APIs are my friends and I love them.

This mechanism works well for my needs in the XPages Jakarta EE project, where either it's easier to just set one implementation for the whole server or, like with CDI, there's complex logic that requires inspecting the active Servlet request to see what NSF I'm in.

APIs of this style will usually have a method named like setInstance or setProvider on either their core entrypoint class or on the provider locator. For example, MicroProfile Config provides the former on its ConfigProviderResolver class:

1
2
3
public static void setInstance(ConfigProviderResolver resolver) {
	instance = resolver;
}

instance here is a static property. Once it's set - either by this method or by a dynamic lookup - the main instance() method will use it:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
if (instance == null) {
	synchronized (ConfigProviderResolver.class) {
		if (instance != null) {
			return instance;
		}
		instance = loadSpi(ConfigProviderResolver.class.getClassLoader());
	}
}

return instance;

The XPages JEE project makes use of this method at HTTP start, setting a provider resolver that includes some stock config sources as well as some classes that know how to read properties from the Notes environment and from the xsp.properties file.

Though this mechanism seems like the crudest out of the bunch, I'm extremely happy whenever it's there.

Conclusion

That was a lot! And there's not really a lesson to be learned here, but rather more that it's often useful to know about all these different mechanisms. When working in the XPages JEE project, I've had to use almost all of them at one time or another, and I've had to familiarize myself with which APIs use which and adapt them individually. For some, I've altered the implementation to be a fragment bundle; for others, I've created my own fragment to provide services and implementations; and so forth. It's a bit of a shame that there's no grand unified system for this, but at least it can be interesting to see the messy path that these specs have taken as Java technologies and the ecosystem evolved.

Upcoming Sessions at CollabSphere 2022

Oct 11, 2022, 2:07 PM

It's CollabSphere time again, and I'm delighted to be involved in a few sessions this time. Since I just very recently did an OpenNTF webinar covering the Jakarta EE Support project, these sessions take the form of roundtables I'm helping lead.

One is about Java generally with a tint of Jakarta specifically on Domino:

DEV103 - Java and Jakarta EE on Domino Roundtable
Wednesday at 3 PM Eastern
The story of Java on Domino can be complex, but there are tools and strategies available to keep your development maintainable and consistent with the wider world. This roundtable will focus on discussing the current array of best options for development on Domino, with a specific focus towards discussing how the XPages Jakarta EE Support project can be used to improve your development experience and the capabilities of your apps.

Another is being led by Graham Acres and Heather Hottenstein and includes me, Justin Hill, and Karsten Lehmann in a panel discussion about open-source projects in the Domino sphere:

COL119 - Community Open Source Initiatives and Contributors
Wednesday at 1 PM Eastern
The story of Java on Domino can be complex, but there are tools and strategies available to keep your development maintainable and consistent with the wider world. This roundtable will focus on discussing the current array of best options for development on Domino, with a specific focus towards discussing how the XPages Jakarta EE Support project can be used to improve your development experience and the capabilities of your apps.

There's also a companion session to that one - COL120 - on Thursday at the same time about specifically HCL's open-source initiatives.

Finally, there's OpenNTF's now-annual roundtable hosted by Graham and me:

COL107 - OpenNTF Roundtable: Have Your Say
Thursday at 12 PM Eastern
OpenNTF has been supporting the Notes and Domino community for two decades. Yes, two decades! In that time there have been volumes of projects, code, documentation and other contributions from many people in our community. Over the last few years, we have also been able to bring a number of experts to speak on a wide range of topics in our monthly webinar series. We have expanded our focus from pure development on Domino to additional topics around the wider HCL Collaboration portfolio and system administration too. Once again, we would like to hear from you. Where else can we bring value to the community? Are there webinar topics you would like to see? Are there other ways people would like to be involved?

Beyond the sessions I'm actively participating in, I noticed that this one from Graham and Heiko Voigt name-drops me in the description, so I would be remiss if I didn't mention it:

COL112 - The HCL Domino AppDev State of the Nation - 2022 Edition
Wednesday at 4 PM Eastern
The Domino AppDev landscape has changed recently with new investment from HCL in VoltMX and VoltMX Go. Domino Volt (now soon to be Domino Leap), the AppDev Pack, the Domino REST APIs, Volt MX and the Nomad Clients have added a lot of opportunities but also a lot of confusion. In the Open Source area, Jesse Gallagher build a lot of new capabilities in the Java space. And that doesn’t include investment in old friends like LotusScript. What can be used and when? What apps can I build with which part of the stack? And what about XPages? In this session, we present an overview of the dev tool ecosystem as well as some roads that seem to be stable. Whether you're a Low-Coder, pro-Coder or a CFA in the Domino AppDev World (Come From Away), we will give you a matrix to determine your way forward, be it that you are building new apps, enhancing or modernizing existing apps or migrating apps from one path to another. And we will also try to tell you what seems to be future proven and what - well, not so much.

That one's right after my earlier roundtable on Java/Jakarta and in the same track, so you should be able to just kick back and enjoy a nice Wednesday afternoon there.

Jakarta NoSQL Driver for Keep

Oct 9, 2022, 3:41 PM

Tags: jakartaee java
  1. Jakarta NoSQL Driver for the AppDev Pack, Part 1
  2. Jakarta NoSQL Driver for the AppDev Pack, Part 2
  3. Jakarta NoSQL Driver for Keep

In what has surreptitiously turned into something of a series, I followed up my recent tinkering with Jakarta NoSQL, the AppDev Pack, and Keycloak with doing something similar with Keep.

Keep, like the AppDev Pack's Proton task, provides a remote API for Domino data. It differs from the ADP in a couple notable ways:

  1. It uses REST endpoints instead of gRPC. The pros and cons of comparing the two are well beyond the scope of this post, but you can say that a REST API is easier to work with using any old HTTP client, while gRPC has higher performance when thrashed. For the purposes of a JNoSQL driver, this is entirely an implementation detail.
  2. Keep imposes rules about data access on top of what you'd normally do with Domino. While Proton and DAS give you direct document access, Keep requires configuring individual forms and views, as well as defining specific types and access levels for fields. It's a focused REST API builder of its own, suitable for providing access to Domino data to clients directly.

The Client

The AppDev Pack ships with a Java client library presumably generated from the HCL-internal spec. Since Keep is REST-based, there's less need for a specific generated client, but it can still be tremendously useful. Keep includes (and is indeed based around) an OpenAPI spec file. The neat thing with those files is that, since they're so common, there are plenty of tools to work with them. One I've used a few times now is OpenAPI Generator, which will take such a spec file and emit bindings for a bunch of languages.

I've used the Java generator before and have gotten familiar with it. Because Java lacked a good standard HTTP client until Java 9 and still doesn't have a core-API standard type-safe client or JSON library, this generator provides options for a good number of common choices. Since this driver is targeting a Jakarta EE environment and it's fair to assume MicroProfile will be around, I went with that: JSON access is done via JSON-B and REST access is done via the ever-delightful MicroProfile Rest Client. This pair ended up producing much-cleaner client code than the previous generation I had done, which used the Apache HttpClient library directly and Jackson for JSON. I think that, post-generation, I had to go in and change some javax imports to jakarta, but otherwise it worked smoothly.

The generator emitted interfaces marked with @RegisterProvider (making them accessible via CDI) and then using JAX-RS annotations to define method signatures. For example (trimmed and reformatted):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
@RegisterProvider(ApiExceptionMapper.class)
@Path("")
public interface DataApi  {
	/* snip */

    /**
     * Send a DQL query and get JSON documents back
     *
     */
    @POST
    @Path("/query")
    @Consumes({ "application/json" })
    @Produces({ "application/json" })
    public List<Map<String, Object>> query(
		@QueryParam("dataSource") @NotNull String dataSource,
		@QueryParam("action") @NotNull String action,
		@Valid QueryRequest queryRequest,
		@QueryParam("richTextAs") RichTextRepresentation richTextAs,
		@QueryParam("count") Integer count,
		@QueryParam("start") Integer start
	) throws ApiException, ProcessingException;
}

It's possible that I had to change an earlier DominoDocument type to Map<String, Object> to account for the ever-varying content of Domino documents.

In any event, each of the operations from Keep's OpenAPI spec got a method like this, grouped into interfaces like DataApi, CodeApi, etc., also based on values in the spec.

Authentication

Lucky for me, a lot of the stuff I did to set up Keycloak authentication with the AppDev Pack carried over here. Keep, for the most part, uses JWT authentication, with tokens often coming from its /auth endpoint. It can also work with external JWT providers like Keycloak. To do that, you can configure Keep with your provider's public key, after which Keep will trust tokens issued by it.

For my setup, I configured Keycloak to emit Keep-friendly tokens, adding in a Domino-format DN and Keep-friendly scopes like $DATA. Once I did that, Keep started trusting tokens I acquired from Keycloak, which I passed to it from my Liberty login the same way I had with my ADP testing.

Implementation and Conclusion

After this point, the implementation itself is actually the least interesting. I copied the code for the Proton driver and mostly swapped out method calls and object types. Since the documents in both cases are treated as JSON data, most of my utility code was already just fine.

Keep does provide view access, which is something that Proton doesn't, so I may implement that. Though I was originally hostile to the idea of adding views to the driver, it's an unfortunate necessity in Domino work, especially when building on top of existing applications.

As it is, the driver isn't in a proper state for release, but I expect that I'll have cause to return to it and implement the missing pieces - views, attachments, and so forth. It's also just a good exercise in the mean time for working more heavily with things like the MP Rest Client, and it's good to see how well it holds up.