Though one day I'd really like to sit down and work on expanding and categorizing the documentation for the XPages JEE project, in the mean time I can at least put together some scattered info in the form of blog posts, webinars, and example apps. Add this post to the pile! Some of it will be a rehash of previous posts, but it doesn't hurt to see it rephrased.

One of the main tentpoles of a pure XPages JEE app (I also wish I had a catchier name for it) is the data-access layer, which uses Jakarta Data and NoSQL to have an abstraction layer over the specifics of Domino data. We as Domino developers are conditioned by privation to do our data access in a fairly primitive way: though the lotus.domino classes technically sit a couple layers of abstraction over the base, they're still stuck at the level of dealing with documents, views, items, and other constructs specifically. If you want to read a the "first name" value from a document representing a person, you always have to do doc.getItemValueString("FirstName") - then the same for last name, job title, and so forth. If you're reading from a view, you have to have an alternate version of your code that loops through that - using the same loop style you've written a million times in more- and less-efficient ways - and reads from column values. In a simple case, fine, but this ends up being a lot of boilerplate code.

Anyway, without further ado, I'll get in to how it should be done.

How It Should Be Done

A basic class for representing a Person would look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

package model;

import java.util.stream.Stream;

import org.openntf.xsp.jakarta.nosql.mapping.extension.DominoRepository;

import jakarta.nosql.Column;
import jakarta.nosql.Entity;
import jakarta.nosql.Id;

@Entity
public class Person {
	public interface Repository extends DominoRepository<Person, String> {
		Stream<Person> findAll();
		
		Stream<Person> findByLastName(String lastName);
	}
	
	@Id
	private String documentId;
	@Column
	private String firstName;
	@Column
	private String lastName;
	
	/* snip: getDocumentId(), setDocumentId(), getFirstName(), etc.) */
}

There's a bit of a nod to being Domino-specific there, which we'll cover later, but there's no business where you have to do doc.getUniversalID(), doc.getItemValueString("FirstName"), and all that. You don't even need to implement any code that does the document lookups - the framework provides that for you. Thanks to some CDI magic, the you can use the repository as-is in a bean or REST service:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

package rest;

import java.util.Comparator;
import java.util.List;

import jakarta.inject.Inject;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.QueryParam;
import jakarta.ws.rs.core.MediaType;
import model.Person;

@Path("person")
public class PersonResource {
	
	@Inject
	private Person.Repository personRepository;
	
	@Path("byLastName")
	@Produces(MediaType.APPLICATION_JSON)
	public List<Person> getByLastName(@QueryParam("lastName") String lastName) {
		return personRepository.findByLastName(lastName)
			.sorted(Comparator.comparing(Person::getFirstName))
			.toList();
	}
}

Under the hood, the JNoSQL driver translates the call to personRepository.findByLastName(lastName) to a DQL query like Form = "Person" and LastName = "Fooson", reads the "FirstName", "LastName", and UNID from each document, and then returns Person objects in a stream.

Adding A Bit Of Finesse

In that example, I have the bit where the REST service sorts the users it found by first name - that's fine, but it's really the database's job. Fortunately, that's easy to add. First, we can change our repository to add a jakarta.data.Sort method:

1
2
3
4

public interface Repository extends DominoRepository<Person, String> {
	// ...	
	Stream<Person> findByLastName(String lastName, Sort<Person> sortOrder);
}

Then, we can simplify the REST service:

1
2
3
4
5

@Path("byLastName")
@Produces(MediaType.APPLICATION_JSON)
public List<Person> getByLastName(@QueryParam("lastName") String lastName) {
	return personRepository.findByLastName(lastName, Sort.asc("firstName")).toList();
}

The result will be the same (or actually probably more what you want, since this will be case-insensitive), but now the DB is doing the work. Specifically, the work it's doing is that this will cause the driver to make a temporary NSF that will house a QueryResultsProcessor view that lists those documents sorted by the FirstName item. It will also be mildly intelligent about this: if the data documents in the DB haven't changed, it'll re-use the existing index, which means that subsequent calls to the same method with the same parameters (and user) will be very fast.

Domino Optimizations

Using that default DQL+QRP behavior is sufficiently performant in a lot of cases (moreso than you might think, too), but it's still potentially prone to the usual pitfalls we have with Domino data access. If you have more documents or complex selections, you'll want to fall back to views or folders.

Say you want to juice your quarterly numbers so you can buy another house, and so you want to get rid of a bunch of employees and you're triaging them via a folder. First, let's add a column to show their salary to our list of fields in the Person class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

@Entity
public class Person {
	// ...
	
	@Id
	private String documentId;
	@Column
	private String firstName;
	@Column
	private String lastName;
	@Column
	private int salary;

	// snip
}

Then, we can add another method to the repository class for this:

1
2
3
4
5

public interface Repository extends DominoRepository<Person, String> {
	// ...
	@ViewEntries("Vacation-House Triage")
	Stream<Person> findDoomedEmployees(Sort<Person> sortOrder);
}

The @ViewEntries annotation there will override the default DQL-and-QRP behavior, and instead open and traverse the view or folder with a ViewNavigator. That also means altering the behavior of the Sort parameter: if you pass Sort.desc("salary"), instead of creating a sorted column in the temporary QRP view, now it will call view.resortView(...) to make use of the "Click on column header to sort" functionality in the folder's design, making the lookup as fast as possible.

Creating And Modifying Documents

So far, these examples have only been to read existing data, but other parts of CRUD similarly work with high-level objects.

If we want to create a new user, there's no more work to do in the repository - we can just add an endpoint to our REST service:

1
2
3
4
5
6

@POST
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Person createPerson(Person newPerson) {
	return personRepository.save(newPerson);
}

The save method is provided by the base repository interface, so that's all that's needed. The framework will take a value like:

1
2
3
4

{
	"firstName": "Foo",
	"lastName": "Fooson"
}

...convert the JSON to a Java object, write the values into a document with the "Person" form value, retrieve a new version of the object with the UNID filled in, and return:

1
2
3
4
5

{
	"documentId": "020322A6F79CB02C85258C140058A1FF",
	"firstName": "Foo",
	"lastName": "Fooson"
}

Similar to the Domino-specific support for views, there's another version of the save method that you can call to compute with form: personRepository.save(newPerson, true).

To update an existing document, you do basically the same thing, but you put the UNID into the object (in practice, the method above should strip the UNID just in case), which will be in a document retrieved from. For example:

1
2
3
4
5
6
7
8

@Path("{documentId}")
@PUT
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Person updatePerson(@PathParam("documentId") String documentId, Person newPerson) {
	newPerson.setDocumentId(documentId);
	return personRepository.save(newPerson);
}

You might want to check if the document by UNID already exists for business-logic purposes, but the underlying behavior is mostly the same anyway.

What We Didn't Have To Do

The primary thing that I want to emphasize is how much work the app developer doesn't have to do. Anything you can do with the JEE framework is doable with the traditional XPages toolkit, but, by the same token, it'd also all be doable with assembly language - you definitely shouldn't do that, though.

This takes care of a tremendous amount of busywork: opening and processing documents, traversing view efficiently, efficient DQL and QRP use, converting to and from JSON, and building REST endpoints. I didn't get into other normal problems that could be solved here: model validation via annotations, sending 404s via meaningful exceptions, modifying data-reading behavior via custom reader classes (critical with weird old Notes data), calling external REST services by interface, and so forth.

Whenever I work with Domino apps not designed with this sort of thing, it's a depressing experience. It's the same mess of varyingly-performant view reading, manually mapping data to intermediate objects, manually writing or reading JSON objects, manually doing HTTP calls for REST services, and all of that over and over and over. It's annoying to write, annoying to read, easy to get wrong, and extremely difficult for a new developer (even an experienced one) to maintain. If you haven't tried the framework and you're writing web apps for Domino, I highly recommend giving it a shot.

When I was getting familiar with modern Java server development, one of the biggest conceptual stumbling blocks by far was CDI. Part of the trouble was that I kind of jumped in the deep end, by way of JNoSQL's examples. JNoSQL is a CDI citizen through and through, and so the docs would just toss out things like how you "create a repository" by just making an interface with no implementation.

Moreover, CDI has a bit of the "Maven" problem, where, once you do the work of getting familiar with it, the parts that are completely baffling to newcomers become more and more difficult to remember as being unusual.

Fortunately, like how coming to Maven by way of Tycho OSGi projects is "hard mode", coming to CDI by way of a toolkit that uses auto-created proxy objects is a more difficult path than necessary. Even better, XPages developers have a clean segue into it: managed beans.

JSF Managed Beans

XPages inherited the original JSF concept of managed beans, where you put definitions for your beans in faces-config.xml like so:

1
2
3
4
5
6
7
8
9

<managed-bean>
	<managed-bean-name>someBean</managed-bean-name>
	<managed-bean-class>com.example.SomeBeanClass</managed-bean-class>
	<managed-bean-scope>application</managed-bean-scope>
	<managed-property>
		<property-name>database</property-name>
		<value>#{database}</value>
	</managed-property>
</managed-bean>

Though the syntax isn't Faces-specific, the fact that it is defined in faces-config.xml demonstrates what a JSF-ism it is. Newer versions of JSF (not XPages) let you declare your beans inline in the class, skipping the XML part:

1
2
3
4
5
6
7
8

package com.example;
// ...
@ManagedBean(name="someBean")
@ApplicationScoped
public class SomeBeanClass {
	@ManagedProperty(value="#{database}")
	private Database someProp;
}

These annotations were initially within the javax.faces package, highlighting that, while they're a new developer convenience, it's still basically the same JSF-specific thing.

While all this was going on (and before it, really), the Enterprise JavaBeans (EJB) spec was chugging along, serving some similar concepts but it really is kind of its own, all-consuming beast. I won't talk about it much here, in large part because I've never used it, but it has an important part in this history, especially when we get to the "dependency injection" parts.

Move to CDI

Since it turns out that managed beans are a terrifically-useful concept beyond just JSF, Java EE siphoned concepts from JSF and EJB to make the obtusely named Contexts and Dependency Injection spec, or CDI. CDI is paired with some associated specs like Common Annotations and Inject to make a new bean system. With a switch to CDI, the bean above can be tweaked to something like:

1
2
3
4
5
6
7
8

package com.example;
// ...
@Named(name="someBean")
@ApplicationScoped
public class SomeBeanClass {
	@Inject @Named("database")
	private Database someProp;
}

Not wildly different - some same-named annotations in a different package, and some semantic switches, but the same basic idea. The difference here is that this is entirely divorced from JSF, and indeed from web apps in general. CDI specifically has a mode that works outside of a JEE/Servlet container and could work in e.g. a command-line program.

Newer versions of JSF (and other UI engines) deprecated their own version of this to allow for CDI to be the consistent pool of variable resolution and creation for the UI and for the business logic.

The Conceptual Leap

One of the things blocking me from properly grasping CDI at first was that @Inject annotation on a property. If it's just some Java object, how would that property ever be set? Certainly, CDI couldn't be so magical that I could just do new SomeBeanClass() and have someProp populated, right? Well, yes, that's right. No matter how gussied up your class definition is with CDI annotations, constructing an instance with new will pay no attention to any of it.

What got me over the hurdle is realizing that, in a modern web app in particular, almost everything you do runs through CDI. JSP request? That can resolve CDI. JAX-RS resource? That's managed by CDI. Filters? CDI. And, because those objects are all being instantiated by CDI, the CDI runtime can do whatever the heck it wants with them. That's why the managed property in the original example is so critical: it's the same idea, just managed by the JSF runtime instead of CDI.

That's how you can get to a class like the controller that manages the posts in this blog. It's annotated with all sorts of stuff: the JAX-RS @Path, the MVC spec @Controller, the CDI @RequestScoped, and, importantly, the @Inject'ed properties. Because the JAX-RS environment instantiates its resource classes through CDI in a JEE container, those will be populated from various sources. HttpServletRequest comes from the servlet environment itself, CommentRepository comes from JNoSQL as based on an interface in my non-JEE project (more on that in a bit), and UserInfoBean is a by-the-numbers managed bean in the CDI style.

There's certainly more indirect "magic" going on here than in the faces-config.xml starting point, but it's a clear line from there to here.

The Weird Stuff

CDI covers more ground, though, and this is the sort of thing that tripped me up when I saw the JNoSQL examples. Among CDI's toolset is the creation of "proxy" objects, which are dynamic objects that intercept normal method calls with new behavior. This is a language-level Java feature that I didn't even know this was a thing in this way, but it's been there since 1.3.

Dynamic scripting languages do this sort of thing as their bread and butter. In Ruby, you can define method_missing to be called when code calls a method that wasn't already defined, and that can respond however you'd like. Years ago, I used this to let you do doc.foo to get a document item value, for example. In Java, you get a mildly-less-loosey-goosey version of this kind of behavior with a proxy's InvocationHandler.

CDI does this extensively, even when you might think it's not. With CDI, all instances are dynamic proxy objects, which allows it to not only inject field values, but also add wrapper code around method calls. This allows tools like MicroProfile Metrics to do things like count invocations, measure timings, and so forth without requiring explicit code beyond the annotations.

And then there are the whole-cloth new objects, like the JNoSQL repositories. To take one of the examples from jnosql.org, here's a full definition of a JNoSQL repository as far as the app developer is concerned:

1
2
3
4
5
6

public interface PersonRepository extends Repository<Person, Long> {

  List<Person> findByName(String name);

  Stream<Person> findByPhones(String phone);
}

Without knowledge of CDI, this is absolute madness. How could it possibly work? There's no code! The trick to it is that CDI ends up creating a dynamic proxy implementation of the interface, which is in turn backed by an InvocationHandler instance. That instance receives the incoming method call as a string and array of parameters, parses the method to look for a concept it handles, and either generates a result or throws an exception. Once you see the capabilities the stack has, the process to get from a JAX-RS class using @Inject PersonRepository foo to having that actually work makes more sense:

• The JAX-RS servlet receives a request for the resource
• It asks the CDI environment to create a new instance of the resource class
• CDI runs through the fields and methods of the class to look for annotations it can handle, where it finds @Inject
• It looks through its contributed extensions and finds JNoSQL's ServiceLoader-provided extension
• One of the beans from that extension can handle creating Repository instances
• That bean creates a proxy object, which handles method calls via invoke

Still pretty weird, but at least there's a path to understanding.

The Overall Importance

The more I use modern JEE, the more I see CDI as the backbone of the whole development experience. It's even to the point where it feels unsafe to not have it present, managing objects, like everything is held together by shoestring. And its importance is further driven home by just how many specs depend on it. In addition to many existing technologies either switching to or otherwise supporting it, like JSF above, pretty much any new Jakarta EE or MicroProfile technology at least has it as the primary mechanism of interaction. Its importance can't be overstated, and it's worth taking some time either building an app with it or at least seeing some tutorials of it in action.