Last year, while developing Jakarta Modules, I realized that there's a path to accessing Java source files by path in normal NSF apps. This ended up being tracked as a Defect Article, but has unfortunately since been deferred to an unspecified future time. I think it's worth fixing before such a time, though, so I set about doing that.

TL;DR

If you have a file in Code/Java in Designer named "com.mycompany.MyClass", the source is accessible via "foo.nsf/xsp/com/mycompany/MyClass.java". There are a couple ways to patch this:

• Set up a web substitution rule for /*.nsf/xsp/*.java to /doesnotexist.html or the like. That won't cover non-Java files (like translation.properties, etc.), but it'd at least prevent your source code from being visible. It'd also be good to set one up for plugin.xml
• Install my new project Code/Java blocker

Further Explanation

The specific problem here is the way the "foo.nsf/xsp/*" URL namespace works. That is essentially the "webapp version" of your NSF, and URLs coming in here are processed as Servlets and, failing that, static resources. When a request comes in matching that, the XPages runtime looks for IServletFactory implementations to handle it first - if one of those matches, it hands the request off to the matched Servlet. If none matches, it then moves on to some code that handles static resources like a classic Java web app would. I'm not quite sure why it does this, but I'd guess it's probably a holdover from traditional web behavior and not actually really useful. Still, it may be load-bearing after so many years, so I can see why HCL wouldn't want to outright remove this.

Anyway, the problem arises from the fact that the virtual filesystem used by the "static resource" search includes all file-resource-like entities that don't start with /WEB-INF. Java classes stored as Code/Java resources are this type of file, and so their conceptual file path is included in the pool. Thus, if you know the name of a Java class inside an NSF and have web access to that NSF, you can get its source via URL. This is admittedly a bit of a stretch as an attack vector: it being a problem relies on someone a) knowing the full name of a Java class and b) that Java class containing damaging information. If you have your app set to show stack traces on errors (not the default), someone could potentially glean some local class names, and from there walk the tree by looking at imports and class references, but it's still a long walk.

Long walk or no, though, it's worth fixing.

The first fix - the web redirection rule - should cover the bulk of what you'd want. There's an off chance that it might run afoul of some edge case where there actually is a Servlet that's intentionally serving up files that end in ".java", but that's pretty unlikely.

Because I like writing code, I made a small project that does the job a little more specifically. The way the plugin works is that it contributes a IServletFactory to the XPages runtime for all NSFs that builds a collection of Code/Java files and then, when a request comes in that matches that or some other vetoed paths, returns a stock Servlet that returns 404 in the same way that other non-matched paths in that namespace do. That should make it so that the source files are not accessible and also don't "leak" their presence at all.

Either way, I suggest applying a fix for this until it's done in the product.

Earlier in the month, I wrote about some of my preliminary experiences dealing with the fallout of the switch from CKEditor to TinyMCE in Domino 14.5. In that post, I talked about some of the ways to deal with covering both, but I only casually mentioned one of the severe limitations in the TinyMCE Dijit implementation: unlike the CKEditor version, it doesn't let you provide anything other than strings for properties. That's fine for a lot of things, but that means that complicated toolbars aren't settable, nor are boolean properties like paste_as_text. If you've ever dealt with users and the sorts of things they want to put into rich text fields, you might understand why that parameter is useful.

I wanted a way to configure this in a way that is reasonably global and reasonably light-touch, ideally in a way that would leave older customizations working on pre-14.5 servers. I've come up with a tack that I'm mostly happy with, though it still has some limitations.

The Tack

The route I'm taking now is sort of similar to CKEditor's customConfig property, but uses the Dojo module system. I'm subclassing the ibm.xsp.widget.layout.CKEditorWrapper class (still the name when using TinyMCE) and using that as a way to set other init properties. There are a few steps for this.

Module Path

First off, we'll need to register a custom Dojo module path. Unfortunately, this isn't settable in themes for some reason, so I dropped this code in the common layout custom control:

1
2
3

<xp:this.resources>
	<xp:dojoModulePath url="example/widget" prefix="example.widget"/>
</xp:this.resources>

Theme

Then, I cracked open the app's theme and configured it to apply a dojoType to rich-text fields on Domino 14.5+:

1
2
3
4
5
6
7

<control>
	<name>InputField.RichText</name>
	<property>
		<name>dojoType</name>
		<value>#{javascript:parseInt(session.evaluate('@Version')[0], 10) &gt;= 495 ? 'example/widget/CustomTinyMCE' : ''}</value>
	</property>
</control>

This will work except in the uncommon case where you're already setting a dojoType for your rich-text fields... if so, you may have some more work to do, or you may just want to set <property mode="override"> there.

JavaScript

Finally, make a JavaScript file named "example/widget/CustomTinyMCE.js" in your NSF. I recommend making this a File Resource and then using the Generic Text Editor to edit it - that one supports newer syntax and is a bit less crash-prone than the default Designer JS editor. Set the contents thusly:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

define(
	'example/widget/CustomTinyMCE',
	['dojo/_base/declare', 'ibm/xsp/widget/layout/CKEditorWrapper'],
	function(declare) {
		return declare('example.widget.CustomTinyMCE', [ibm.xsp.widget.layout.CKEditorWrapper], {
			constructor: function () {
				this.opts.paste_as_text = true
				this.opts.toolbar = 'undo redo | cut copy pastetext'
			}
		})
	}
)

And customize at will. this.opts is set by the superconstructor to the contents of any dojoAttribute or attr properties on your component, so anything you set here will override what's set there. This can be useful in particular for things like code that sets the toolbar shorthands like "Large". Here, I just use a string for the toolbar, but, this being JavaScript, you are free to use the structured syntax.

Further Customization

The values you set in this.opts here will override anything set by dojoAttributes or in the default config HCL provides, so you have pretty free reign here. You could also use that for your own custom configurations if you have different needs on different pages - read the values in this.opts and then apply different configuration based on that. This would, admittedly, have been a nice touch in the case of the "Large"/"Medium"/"Small" toolbar shorthands.

Overall, I think this is a pretty reasonable approach. I wish that you could register dojoModulePaths in theme files, but the need for that could be worked around by moving the JavaScript to an OSGi plugin with more-predictable paths. This centralized route also leaves room for alternate customizations in the case that HCL switches the RT editor again in the future.

A few years back, when Twitter finally truly went over the edge into hell, I wrote a post about setting up my Mastodon instance, which uses Keycloak for OIDC auth backed by Domino LDAP for the directory.

In the intervening time, Domino sprouted OIDC provider capabilities and so it's been on my back-burner to switch Mastodon over to use it. Keycloak has been fine for me, but I only use it for OIDC needs, and it'll be all the better to cut down on the number of distinct servers I run. Plus, while Keycloak's Docker container makes it easy to run, I'm much more comfortable with how Domino works, and so it's always nice to use it when I can.

For the most part, slotting Domino into Keycloak's place here was a matter of following the directions in the Domino documentation to set up the provider, which I won't rehash here. There are only really a few things that make it special.

Dynamic Claims

As I mentioned in the original post, I use a special field in my person document called "mastodonusername" to store the name to use for Mastodon. In Keycloak, I mapped that value from LDAP to a claim in the generated JWT, and I'd need to do the same on Domino.

Though there's not a UI for it, towards the end of the 14.5 EA period, HCL added in the ability to specify one field to map from the person document to the token. This is done via notes.ini variables, so I put this in my server's Configuration doc:

1
2

OIDC_PROVIDER_DYNAMIC_CLAIM_FIELD=mastodonusername
OIDC_PROVIDER_DYNAMIC_CLAIM_NAME=mastodonusername

With that in place, the token gets the same "mastodonusername" claim that Keycloak was using, and it'll continue to map my login to the right user in Mastodon.

PKCE

The current proper way to do OIDC is to include PKCE, which was originally designed for mobile apps but can be used with all login flows. Domino's OIDC Provider lets you turn this off as a requirement on a per-client level, but it's still best to include it when possible.

Mastodon doesn't enable PKCE by default, but it made it in as an option a couple versions ago. Enabling it is done via an environment variable, so I added this to the .env file that my Mastodon container uses:

1

OIDC_USE_PKCE=true

And, well, that's it. Now it uses PKCE and all is well.

That's About It

And that's about it! I didn't expect any specific problem, since OIDC is a nice open standard and Domino's implementation hews very closely to best practices, but it's always nice to see things go smoothly in action.

At the top of the update notes for Designer 14.5 is this bit:

My guess is the reason for this change is that CKEditor 4 hit end-of-life in June 2023 and CKEditor 5 dropped the option to license as MPL, meaning the only way to use it in Domino would be to pay for a presumably-onerous enterprise license^†. Fair enough.

However, you may notice that the documentation for this change is a little... thin. Specifically, the screenshot above is the documentation. Considering that customizing CKEditor has been a popular and long-documented feature of XPages, it's possible you're sitting on some code that used to customize CKEditor to your needs but does not work in 14.5. Since CKEditor no longer ships with Domino, there's no opt-in switch, so we have to deal with it.

Well, I don't have all the answers for how to do this properly, but I did recently have a need to at least start digging into it, so I figured I'd share a preliminary tack to start making cross-version-compatible code.

Customizing TinyMCE

Like CKEditor, TinyMCE has mechanisms to customize its appearance in both simple and programmatic ways. The way TinyMCE is initialized is that it's passed a configuration object during creation, like in this example from their docs:

1
2
3
4
5

tinymce.init({
  selector: 'textarea',
  skin: 'oxide-dark',
  content_css: 'dark'
});

As it happens, the Dojo plugin XPages uses to instantiate TinyMCE (still called "ibm/xsp/widget/layout/CKEditorWrapper" for some reason) takes all of your HTML attributes and passes them to this object, vaguely similar to the way CKEditor did it. That means you can get the above behavior like so (we don't need the selector, since XPages does that for us):

1
2
3
4
5
6

<xp:inputRichText>
	<xp:this.attrs>
		<xp:attr name="skin" value="oxide-dark"/>
		<xp:attr name="content_css" value="dark"/>
	</xp:this.attrs>
</xp:inputRichText>

I say "vaguely similar" because, while CKEditor will interpret applicable attributes (like toolbar) as JavaScript by running them through eval(...), the same is not the case for the TinyMCE code. So, for example, this snippet, trying to replicate the next example from the TinyMCE page, will not work:

1
2
3
4
5
6

<xp:inputRichText>
	<xp:this.attrs>
		<xp:attr name="skin" value="(window.matchMedia('(prefers-color-scheme: dark)').matches ? 'oxide-dark' : 'oxide')"/>
		<xp:attr name="content_css" value="(window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'default')"/>
	</xp:this.attrs>
</xp:inputRichText>

That will just cause the page to try to load the URL "/yourapp.nsf/(window.matchMedia('(prefers-color-scheme:%20dark)').matches%20? 'dark' : 'default')=", which will, uh, not work. I assume you can deal with this via TinyMCE plugins, but I haven't looked into those yet.

When The Twain Meet

If you're migrating from older versions to 14.5+, your task is to modify all of your code that customizes CKEditor. The most common "good" case for this is that your customizations break in innocuous ways. However, it's also possible that your customizations will result in a broken editor, particularly around toolbars. For example, the afore-linked documentation from 8.5.2 includes the GUI version of this:

1
2
3
4
5

<xp:inputRichText>
	<xp:this.dojoAttributes>
		<xp:dojoAttribute name="toolbarType" value="Large"/>
	</xp:this.dojoAttributes>
</xp:inputRichText>

(for the record, xp:this.dojoAttributes is effectively interchangeable with xp:this.attrs for this use)

This causes CKEditor to display its "large"-type suite of toolbar icons, as opposed to "medium" or "slim". However, this causes TinyMCE to display no toolbar icons at all. The reason for this is that CKEditorWrapper.js maps "toolbarType" to just "toolbar" and, while CKEditor treated "toolbar" as either a shorthand name or as JavaScript, TinyMCE treats the same attribute as a space-delimited list of action names, and "Large" is not one.

This creates a bit of a pickle! You'll definitely need to open up your app's code to change this. If you're doing a one-way migration, you can change this to be a list of the buttons you'd like, but things will get messier if you want to have multiple versions of Domino deployed, or you want to prep your apps for the migration before making the switch.

There are a lot of ways one could go about doing this and I'm not sure I love the provisional mechanism I'm going to share here, but this is at least one way to do it. Since the one way we can know that we're using a different editor is by the build version of Domino, we can make a switch and only load attributes based on the running server version:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

<xp:view xmlns:xp="http://www.ibm.com/xsp/core">
	<xp:this.dataContexts>
		<xp:dataContext var="isTinyMce" value="${javascript:parseInt(session.evaluate('@Version')[0], 10) &gt;= 495}"/>
	</xp:this.dataContexts>
	<xp:inputRichText>
		<xp:this.attrs>
			<!-- CKEditor attributes -->
			<xp:attr name="toolbar" value="Slim" loaded="${not isTinyMce}"/>

			<!-- TinyMCE attributes -->
			<xp:attr name="toolbar" value="undo redo styles" loaded="${isTinyMce}"/>
		</xp:this.attrs>
	</xp:inputRichText>
</xp:view>

If you have a lot of CKEditor customizations, your best bet will be to use a customConfig script to consolidate them if you don't already. TinyMCE will ignore that, so at least it won't do any harm.

Like I said, I don't love this approach, since it'd be a real pain if you have a lot of rich-text fields in your apps, but at least it's a starting point. It could be more scalable (if very fiddly) to write a renderer to replace RichTextRenderer that checks the server version and then does its own heuristics to change the output values for the HTML. It might also be possible to dredge up the old CKEditor files from older Domino and write your own Dojo plugin to re-wrap them to keep things chugging for a bit, but that'd be only a temporary fix.

Anyway, it's a messy situation and I didn't see a lot of documentation for it, so I figured I'd jot this down. Good luck!

^† After posting this, I noticed that TinyMCE went through a similar permissive-or-GPL switch in the move from 6.8 to 7.0. That explains why Domino ships with version 6.8.4 and makes me suspect that we'll face another similar move before long.

When Notes 14 FP2 came out, I made a blog post detailing a change that HCL made to the JRE classpaths that Designer uses for compilation. Specifically, since jvm/lib/ext is not present in newer Java versions, people had found that they manually had to add JARs from ndext to the JRE to get the same sort of "local JAR" behavior from previous versions. HCL "fixed" this by making it so that Designer adds all of the JARs in ndext to the compiling JRE, which sort of fixed the problem. It had the unfortunate side effect, though, of adding the various contaminants present in the Notes JVM to the classpath, particularly Poi and (in that post's case) the ancient Servlet spec.

In that post, I explained how to fix it. However, I hadn't yet noticed that it's not just that Designer does this once, but rather that it does it every time you launch, re-breaking compilation if your code runs afoul of this. Since then, I found a workaround that seems to stick, so it's worth documenting here.

Whatever code re-mangles the classpath does it specifically to the one named "jvm", so what I do now is go to Preferences - "Java" - "Installed JREs" and add a new definition for the same path but with only the core runtime package and a distinct name:

If you have any local JARs you do want to include, add them here.

Then, I mark that JRE as the default:

With this so marked, it will (at least in my experience) stick as your default across launches and you will be blessedly free from this particular excessive zeal on Designer's part.

Earlier today, I released version 3.6.0 of the XPages JEE project, which has a number of handy improvements.

NoSQL Driver Improvements

In continuing with my goal to remove as many instances of needing to use the lotus.domino API (or others, like ODA or JNX) in app dev, I've added a couple more useful features to NoSQL DominoRepository objects:

• The queryEffectiveAccess() method gives you a summary of the current user's access rights to the underlying database: the user's DN, the access level (Author, Editor, etc.), their privileges (delete docs, etc.), and their roles
• The send(...) method lets you send the underlying document as mail
• The queryLastModified() method will tell you the last time the database was modified in a speedy way, which can be useful for cache invalidation

There are also a handful of fixes and performance improvements, such as a better ETag generation routine that brings it more in line with the standard way of doing these.

sessionAsSigner in Faces Requests

Originally, Faces (JSF) requests didn't have access to sessionAsSigner due to the way the Servlet had to work around the XPages environment without colliding. Now, though, these sessions will be available during requests for Faces files, allowing them to be used as dominoSessionAsSigner on a page itself and also injected normally in managed beans.

Annotated Async Methods

This release contains version bumps for a number of upstream implementations, and one is of particular note: Concurro, the Jakarta Concurrency implementation, added support for the runAt property of the @Asynchronous.

In case you're unfamiliar with it, the Concurrency spec provides for a couple handy ways to do asynchronous programming, and the glue code in this project makes it so that works on Domino and carries the DB and user context into the other threads. One of the neat ways you can do that is via annotations on CDI beans that make things implicitly async. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

@Asynchronous
public CompletableFuture<String> doAsyncOp() {
	// Do some complex work here
	return Asynchronous.Result.complete("job's done");
}

public void kickOffSomeWork() {
	// Do basic work here
	CompletableFuture<String> future = doAsyncOp();
	// Continue on while the async operation does its thing
}

The CompletableFuture there is a standard Java class that you can use to wait in the current thread until it's done (future.get()) or compose other behavior. It's quite handy, and generally much easier than trying to get this stuff to work without this project.

The new addition in Concurro is support for runAt, which lets you provide a repeating schedule for the task, looping until the app ends or the method returns a complete result. For example:

1
2
3
4
5
6
7
8

private int scheduleRan;

@Asynchronous(runAt = @Schedule(cron="* * * * * *"))
public void runScheduled() {
	if(++scheduleRan == 5) {
		Asynchronous.Result.complete(null);
	}
}

Calling runScheduled() will run that code every second (thanks to that cron pattern) until it's done so five times, at which it will stop. While it was previously possible to do this by providing a Trigger implementation passed to the ManagedScheduledExecutorService, it can potentially be much nicer and clearer to do it in annotation code like this.

Misc. Improvements

There are also just generally a number of bug fixes here and there, hopefully avoiding the ServiceLoader trouble that cropped up recently.

I also did some fun tinkering and expansion of the XPages Build Stubs project to be able to build this one without requiring the real proprietary dependencies. In practice, it's still better to use a generated update site from the real thing, since the stubs don't include every class, but this made it easier to set up automated builds on GitHub Actions without yet having to worry about how to keep secrets from leaking. This doesn't include the ability to build the NSFs from the ODPs or run the integration test suite, since both of those require true Domino containers, but it's a nice step in making the build more portable.

Recently, the estimable Heiko Voigt has been writing a blog series on htmx with Domino. Since this post will build on the current state of that directly, I recommend you read each of those posts, which will give you an explanation of what htmx is and some examples of using it with Domino data.

I've been meaning to kick the tires on htmx for a while now, and this was a good excuse. It's a neat tool and just so happens to dovetail perfectly with the strengths of the XPages Jakarta EE project. So: let's see how that shakes out!

For starters, I made an NSF much like the state in part 4 of Heiko's blog series - a trip tracker with the same fields and a view. The implementation after that diverges, though.

Data Access

To get to the data, I created a NoSQL entity to represent the trips, using the convenient syntax of records:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

package model;

import java.time.LocalDate;
import java.util.stream.Stream;

import org.openntf.xsp.jakarta.nosql.mapping.extension.DominoRepository;
import org.openntf.xsp.jakarta.nosql.mapping.extension.ViewEntries;

import jakarta.nosql.Column;
import jakarta.nosql.Entity;
import jakarta.nosql.Id;

@Entity
public record Trip(
	@Id String unid,
	@Column("Date") LocalDate date,
	@Column("User") String user,
	@Column("Start") String start,
	@Column("Destination") String destination,
	@Column("KM") int km,
	@Column("Expenses") double expenses,
	@Column("Description") String description
) {
	public interface Repository extends DominoRepository<Trip, String> {
		@ViewEntries("Trips")
		Stream<Trip> listTrips();
	}
	
}

With this, the listTrips() method is pretty equivalent to the view reading that Heiko's example does, just handled by the framework.

Pages

For the front end, we'll start with a basic Jakarta Pages (JSP) page. This could really be plain HTML, since it has almost no logic to it, but it's nice to stick with the same tool as we'll use later. It's pretty thin, with the CSS and JS being the same ones from Heiko's second post.

This file is "WebContent/WEB-INF/views/home.jsp":

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

<%@page contentType="text/html" pageEncoding="UTF-8" trimDirectiveWhitespaces="true" %>
<!DOCTYPE html>
<html>
	<head>
		<title>htmx Tester</title>
		
		<link rel="stylesheet" href="${pageContext.request.contextPath}/htmx-main.css" />
		<script type="text/javascript" src="${pageContext.request.contextPath}/htmx.min.js"></script>
	</head>
	<body>
		<header id="main-header">
			<h1>htmx and HCL Domino w/ Jakarta EE</h1>
		</header>
		
		<main>
			<div hx-get="${mvc.basePath}/apihandler/trips" hx-trigger="load">
			</div>
		</main>
	</body>
</html>

There are a few things to note that differ from the original:

• ${pageContext.request.contextPath} is used to get the path to the app. In XPages, a preceding "/" in an element like <xp:styleSheet/> will imply this, but we need to bring our own here. It'll be something like "/dev/htmx.nsf" here.
• ${mvc.basePath} is a handy way to reference the base path of the REST app. It'll be something like "/dev/htmx.nsf/xsp/app" here.
• Since we'll have a REST endpoint specifically for the Trip objects instead of an adaptable any-view one, the hx-get URL doesn't include the replica ID.

In Heiko's post, he uses a JSON extension to htmx to allow calling REST endpoints that emit JSON and smoothly translating those to HTML on the client side. Since Jakarta Pages is a perfectly-good HTML-templating engine in its own right, though, we'll do that work on the server and skip the extension.

This file is "WebContent/WEB-INF/views/trips-table.jsp":

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

<%@page contentType="text/html" pageEncoding="UTF-8" trimDirectiveWhitespaces="true" %>
<%@taglib prefix="c" uri="jakarta.tags.core" %>
<table id="mytable">
	<thead>
		<tr>
			<th>Date</th>
			<th>User ID</th>
			<th>Start</th>
			<th>Destination</th>
			<th>KM</th>
			<th>Expenses</th>
			<th>Description</th>
		</tr>
	</thead>
	<tbody id="tbody">
	<c:forEach items="${trips}" var="trip">
		<tr>
			<td><c:out value="${trip.date}"/></td>
			<td><c:out value="${trip.user}"/></td>
			<td><c:out value="${trip.start}"/></td>
			<td><c:out value="${trip.destination}"/></td>
			<td><c:out value="${trip.km}"/></td>
			<td><c:out value="${trip.expenses}"/></td>
			<td><c:out value="${trip.description}"/></td>
		</tr>
	</c:forEach>
	</tbody>
</table>

You can see that the concept is basically the same. Instead of having htmx do the interpolation, we'll use Pages's loops. That <c:forEach/> is basically <xp:repeat/> from XPages, while <c:out/> is basically <xp:text/>.

Controller Glue

Now that we have the data model and the HTML/htmx view, we'll tie them together with some controller classes. The first is the one that serves up the home page, and it's lean:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

package controller;

import jakarta.mvc.Controller;
import jakarta.mvc.View;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;

@Path("/")
@Controller
public class HomeController {
	@GET
	@View("home.jsp")
	public void get() {
	}
}

With this in place, visiting "dev/htmx.nsf/xsp/app" will load up index.jsp from above.

The controller class for our API is a bit more complicated:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

package controller;

import jakarta.inject.Inject;
import jakarta.mvc.Controller;
import jakarta.mvc.Models;
import jakarta.mvc.View;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import model.Trip;

@Path("apihandler")
@Controller
public class ApiController {
	@Inject
	private Trip.Repository tripRepository;
	
	@Inject
	private Models models;
	
	@Path("trips")
	@GET
	@View("trips-table.jsp")
	public void listTrips() {
		models.put("trips", tripRepository.listTrips().toList());
	}
}

Here, we use CDI to @Inject the repository for our Trip entity. The Models object comes from MVC, and it's what we use to populate the data that the .jsp file will need for its loop. You can then see the method where we call "dev/htmx.nsf/xsp/app/apihandler/trips" - it reads in the Trip objects and puts them in the "trips" Models field. That's how the <c:forEach/> in the Pages file gets its hands on it.

And with that, we're done! The result looks just the same as Heiko's example, and that was the goal. This was good to prove to myself that htmx is a great match here, and I'm tempted to try it in a larger project now.

About a year and a half ago, I wrote a post talking about adding JaCoCo code coverage to the integration-test suite of the XPages JEE project.

Today, I happened to notice that, though unheralded, Domino 14.5 FP1 bumped the JVM from Semeru 21.0.6 to 21.0.8. Normally, these little patch-level bumps are pretty irrelevant, but this one was a big deal: Java Flight Recorder support came to Semeru 21 in 21.0.7, so now we have it.

JFR Basics

Java Flight Recorder, in turn, is a built-in capability to monitor and profile your Java applications, similar to things like YourKit. The prime advantage that JFR brings is that it's very low-overhead, enough that it's possible to just run passively in all but particularly-performance-sensitive situations. Even if you don't keep it on all the time, though, it's possible to turn it on and off at will for a running application, whether or not you configured profiling ahead of time. That makes it very intriguing for production environments where you would be disinclined to install, say, the YourKit agent ahead of time.

With HotSpot JVMs, you can automatically enable JFR via a JVM argument, but Semeru doesn't currently implement that. Instead, Semeru supports just the manual way (which is also in HotSpot), using the jcmd tool, which comes with the JDK.

Working with JFR in Domino

The way jcmd works is that you give it the PID of a running compatible Java application and execute one of a number of commands. The tool predates JFR, but I hadn't had a need to know about it until now.

Domino doesn't ship with jcmd, but the one in a download of a Semeru JDK will work. I don't know if it has to be the same patch version or not, but I got the matching one for my tests. Then, you can run jcmd -l to get a list of running JVMs, like:

C:\java\jdk-21.0.8+9\bin>jcmd -l
10904
8884 <no information available>

Those are... well, they're just the PIDs of Java apps. Here, I happen to know that Domino's nhttp.exe is 10904 because it's the one that doesn't say "no information available", but in practice you'll want to do something proper to get your desired PID. In a Domino container, pgrep http will likely give you the number you want.

PID in hand, you can run execute a "JFR.start" command to start recording:

C:\java\jdk-21.0.8+9\bin>jcmd 10904 JFR.start filename=C:\out.jfr
Start JFR recording to C:\out.jfr

I recommend specifying a filename like that, since otherwise it'll always be the same stock one based on a JFR profile. If you're running automated tools, you'll want to avoid collisions.

Once you do whatever it is you want to profile, you can stop the recording:

C:\java\jdk-21.0.8+9\bin>jcmd 10904 JFR.stop
Stop JFR recording, and dump all Java threads to C:\out.jfr

That's about it, at least for the normal case. That file can be read by a number of tools. I use the aforementioned YourKit, since I have and like it, but you can also use the jfr command-line tool, the official JDK Mission Control app, IntelliJ, and likely other things.

Hooking It Into The Test Suite

Doing manual sampling like this is very useful on its own, and I do this sort of thing a lot when getting into the weeds of performance tuning. The low overhead, though, means it's a perfect addition to a full test suite as a "may as well" sort of thing.

Getting this going took a couple steps, but it wasn't too bad once I figured it out.

The first hurdle is the lack of jcmd in the Domino container. Fortunately, this is a straightforward fix: just grab it from the Semeru Docker image during build.

1

COPY --from=ibm-semeru-runtimes:open-21-jdk /opt/java/openjdk/bin/jcmd /opt/hcl/domino/notes/latest/linux/jvm/bin

Next, I want to kick off JFR before any tests run. Initially, I put this in the containerIsStarted method of the Testcontainer class, but the timing didn't work out there. Instead, it ended up in the PostInstallFactory Java class I have to do post-HTTP-init tasks:

1
2
3
4
5
6
7

Path jcmd = Paths.get("/opt/hcl/domino/notes/latest/linux/jvm/bin/jcmd");
if(Files.isExecutable(jcmd)) {
	long pid = ProcessHandle.current().pid();
	new ProcessBuilder(jcmd.toString(), Long.toString(pid), "JFR.start", "filename=/tmp/flight.jfr")
		.start()
		.waitFor();
}

This command will fail on Domino before 14.5FP1, but that's fine.

Finally, when the test runtime sees it's on a high-even version, it grabs the file on container exit:

1
2
3
4
5
6
7
8

if(useJfr()) {  // Does some checks based on container label
	// Find the PID for http to pass to jcmd
	String pid = this.execInContainer("pgrep", "http").getStdout();
	if(pid != null) {
		this.execInContainer("/opt/hcl/domino/notes/latest/linux/jvm/bin/jcmd", pid, "JFR.dump");
		this.copyFileFromContainer("/tmp/flight.jfr", target.resolve("flight-" + System.currentTimeMillis() + ".jfr").toString());
	}
}

With that in place, running the test suite will drop - alongside the log files and JaCoCo report - the JFR results in the "target" directory. Et voilà:

The information isn't as fine-grained as full profiling may get, but it'll give you a good comparison of relative expense of various calls, and that's often all you really need.

This is very neat and should be a handy addition. I'm looking forward to seeing what other good uses I can get out of JFR now that we have access to it.

Among the many classes in the lotus.domino API, DateTime has always been particularly cursed. Not quite as cursed as its brother DateRange, which has never been - and still is not - readable from views and is only barely reconstructable in documents, but still rather cursed.

The Curse

For one, because it shares the trait with most other objects of having a backing C++ object that needs to be explicitly freed, it's important to make sure they're recycled regularly. But they immediately throw you a curveball in that they're children of Session and not the Document or ViewEntry they're pulled from, making it highly likely that you're going to leave them floating in memory for longer than expected when you do a loop. Moreover, they're likely to show up in viewEntry.getColumnValues() even if you don't care, so you have to make sure to studiously call, say, viewEntry.recycle(columnValues) in your loop too - and how many of us remember to do that? And all that hassle just for 64 bits of data total.

To make matters worse, DateTime had the bad luck of being paired with the famously bad java.util.Date class, which is itself a weird wrapper around essentially just an Epoch timestamp and riddled with early-Java-era bad decisions. It's also a very poor representation for a Notes time value, since it can only represent a full date+time pair and has no concept of a time zone or offset. The remaining Notes-isms are left to methods like getDateOnly() and getTimeOnly() that return Strings. While an American may not worry about what "A string representation of the date part of the time-date" implies, that vagueness should send a shiver down the spine of anyone more worldly.

Better Ways

The thing is, though, that the Notes time format actually matches up pretty closely with three of the java.time classes added in Java 8: LocalDate, LocalTime, and OffsetDateTime. Though these classes are now over a decade old, IBM and now HCL have not added support for them in the lotus.domino API. JNX does, though, and the results are much nicer than dealing with DateTime. The trouble is that JNX can do this by way of accessing the two 32-bit integers that make up the "Innards" property of the TIMEDATE struct, which isn't accessible on DateTime.

Well, not properly accessible, anyway. I got a wild hair to improve the use in the NoSQL driver yesterday, so I started digging. Immediately, I noticed that the lotus.domino.local.DateTime concrete class - the one used when not using CORBA or whatever "cso" references - has two int instance members named "mInnards0" and "mInnards1". Well, that looked promising! My first attempt was to just get those values using reflection, but they were still set as 0 - that makes sense, since all the actual methods in that class go to native code that presumably accesses the C-side value.

Those properties are referenced in some methods to do with "restoring" the object, which is not really a concept in the API in general, but I figured it could be like how there seems to be some auto-GC code floating around in Notes.jar that doesn't really come into play. And, by gum, that's exactly what it is: for some reason, these values are written when you call recycle() on the object. I don't know if this restoration mechanism is actually used or useful, but I don't really care; those two innards values are everything I need.

Putting It To Work

So I snagged the code from JNX that handles this and dropped it into the NoSQL driver, and now I can have a path that handles lotus.domino.local.DateTime objects specially, recycling them and then reading the now-set innards values to convert to near-perfect java.time representations.

For the driver, this has some big benefits. Under profiling, when fetching 1000 view entries with a couple time values, calls to toTemporal took up 560 ms (about 20% of the total time spent), while now it takes 64 ms (now 2% of the total time). Moreover, not only have I now eliminated on of the bigger performance sinks, the result is better: now you can get an OffsetDateTime that actually represents the offset of the stored time. While 3 AM Eastern Daylight Time is technically the same moment as 7 AM UTC, the meaning of a document being created at 3 AM local time compared to 7 AM is potentially very significant.

I'm pleased as punch about this. Normally, relying on a weird side-effect and reflectively accessing non-public implementation instance methods is not a good idea, but I think it's fair to assume that this behavior is not going to change any time soon. I'll still likely convert the NoSQL driver to use JNX eventually, but this honestly really lowered the priority: I'm able to get to most other things I want in a fast-enough way, and these date/time values were the main thing where it was outright deficient.

Whenever I want to explain what CDI is to somebody who is unfamiliar with it, I start with "managed beans but better". It definitely is that - just using annotations to define scopes and @Inject to reference beans in code makes them much better than the ancient mechanism in XPages.

However, CDI definitely goes far beyond that, and I had use of one of its capabilities today: the event publish/subscribe system.

For a good while now, I had a task to add DQL "explain" logging to the NoSQL driver in the XPages JEE project. This would be useful in troubleshooting performance of the queries generated by NoSQL's mapping of method names like "findByFirstName" to the actual calls. The explain results could be used to determine where adding index views could speed things up.

The trouble with this was that it's never been clear how it should be logged. I could just use a normal java.util.Logger instance, but that would make the developer have to check the XML-based logs on the filesystem, which would be annoying for several reasons. I also didn't want to restrict it to just being something to go to a log, so I decided to instead use CDI events to publish the data when configured to do so. The mechanism to do this is in the project README, but I figure it's also a good example of using CDI's extended capabilities.

Publishing Events

Events themselves are arbitrary Java objects of any type and are published using the confusingly-named jakarta.enterprise.event.Event interface. For this purpose, we'll use a Java record as our event type:

1
2

public record MyEventClass(String payload) {
}

In a normal bean, you can get a handle on these emitters by declaring @Inject Event<MyEventClass> emitter in one of your beans. If your code is outside an existing bean (as is the case in my driver), you can call CDI.current().select(new TypeLiteral<Event<MyEventClass>>() {}).get() to get the same thing.

Once you have this, you can emit events by passing event objects to it:

1

emitter.fire(new MyEventClass("this is my payload"));

Receiving Events

Once you have code that emits events, the other side is to listen for them. This can be done by having any bean that has a method where a parameter is annotated with jakarta.enterprise.event.Observes. For example:

1
2
3
4
5
6

@ApplicationScoped
public class ListenerBean {
	public void eventHandler(@Observes MyEventClass event) {
		System.out.println("I received an event: " + event);
	}
}

And that's it - when an event of this type is fired anywhere, your bean will hear about it without needing any knowledge of the sender. In the case of the NoSQL driver, this means that developers can listen for ExplainEvents and log or otherwise process them as desired. That also means that my code doesn't have to make any assumptions about the results are to be used.

As is often the case, a publish/subscribe system like this can be a very potent tool, and you can go a lot further with this, writing your applications to be a lot more event-based. You could go TOO far with it, of course, but judicious use can make your code a lot cleaner, more explicit about what's going on, and more extensible for future needs.