One of my main goals in the slow-moving OpenNTF home-page revamp project I'm doing (which I recently moved to a public repo, by the way) is to, like on this blog, keep things extremely simple. There's almost no JavaScript - just Hotwire Turbo so far - and the UI is done with very-low-key JSP pages and tags.

The Library

This also extends to the server side, where I use the XPages Jakarta EE project as the baseline but otherwise don't yet have any other dependencies to distribute. I have had a few Java dependencies kept as JARs within the project, though, and they're always a bit annoying. The other day, when Designer died again trying to sync one of them to the ODP, I decided to try to remove PrettyTime in favor of something slimmer.

PrettyTime is a handy little library that does the job of turning a moment in time into something human-friendly relative to the current time. OpenNTF uses this for, for example, the list of recent releases, where it'll say things like "19 hours ago" or "5 months ago". Technically the same information as if it showed the raw date, but it's a little nicer. It's also svelte, with the JAR coming in at about 160k. Still, nice as it is, it's a binary blob and a third-party dependency, so it was on the chopping block.

The Strategy

My antipathy for using JavaScript isn't so much about an objection to the language itself but to the sort of bloated monstosities it gives rise to. Using it in the "old" way - progressive enhancement - is great. Same goes for Web Components: I think they're the right tool a lot less frequently than a lot of JavaScript UI toolkits do, but they have their place, and this is one such place.

What I want is a way to send a "raw" ISO time value to the client and have it actually display something nice. Conveniently, just the other week, Stephan Wissel tipped me off to the existence of the Intl object in JavaScript, which handles the fiddly business of time formating, pluralization, and other locale-related needs on behalf of the user. Using this, I could write code that translates an ISO date into something friendly without also then being on the hook for coming up with translations for any future non-English languages that the OpenNTF app may support.

The Component

In the original form, when I was using PrettyTime, the code in JSP to emit relative times tended to look like this:

1

<c:out value="${temporalBean.timeAgo(release.releaseDate)}"/>

I probably could have turned that into a JSP tag like <t:timeAgo value="${release.releaseDate}"/>, but it was already svelte enough in the normal case. Either way, this already looks very close to what it would be with a web component, just happening server-side instead of on the client.

As it happens, Web Components aren't actually terribly difficult to write. A lot of JS toolkits optimize for the complex case - Shadow DOM and all that - but something like this can be readily written by hand. It can start with some normal JavaScript (since I'm writing this in Designer, I made this a File resource, since then the JS editor doesn't die on modern syntax):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

class TimeAgo extends HTMLElement {
	constructor() {
		super();
	}

	connectedCallback() {
		/* Do the formatting */
	}
}

customElements.define("time-ago", TimeAgo);

Put that in a .js file included in the page and then you can use <time-ago/> at will! It won't do anything yet, but it'll be legal.

While the Intl library will help with this task, it doesn't inherently have all the same functionality as PrettyTime. Fortunately, there's a pretty reasonable idiom that basically everybody who has sought to solve this problem has ended up on. Plugging that into our component, we can get this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

class TimeAgo extends HTMLElement {
	static units = {
		year: 24 * 60 * 60 * 1000 * 365,
		month: 24 * 60 * 60 * 1000 * 365 / 12,
		day: 24 * 60 * 60 * 1000,
		hour: 60 * 60 * 1000,
		minute: 60 * 1000,
		second: 1000
	};
	static relativeFormat = new Intl.RelativeTimeFormat(document.documentElement.lang);
	static dateTimeFormat = new Intl.DateTimeFormat(document.documentElement.lang, { dateStyle: 'short', timeStyle: 'short' });
	static dateFormat = new Intl.DateTimeFormat(document.documentElement.lang, { dateStyle: 'medium' });

	constructor() {
		super();
	}

	connectedCallback() {
		let date = new Date(this.getAttribute("value"));

		this.innerText = this._relativize(date);
		if (this.getAttribute("value").indexOf("T") > -1) {
			this.title = TimeAgo.dateTimeFormat.format(date);
		} else {
			this.title = TimeAgo.dateFormat.format(date)
		}
	}

	_relativize(d1) {
		var elapsed = d1 - new Date();

		for (var u in TimeAgo.units) {
			if (Math.abs(elapsed) > TimeAgo.units[u] || u == 'second') {
				return TimeAgo.relativeFormat.format(Math.round(elapsed / TimeAgo.units[u]), u)
			}
		}
		return TimeAgo.dateTimeFormat.format(d1);
	}
}

customElements.define("time-ago", TimeAgo);

The document.documentElement.lang bit there means that it'll hew to being the professed language of the page - that could also use the default language of the user, but it'd probably be disconcerting if only the times were in one's native language.

With that in place, I could replace the server-side version with the Web Component:

1

<time-ago value="${release.releaseDate}" />

When that loads on the page, the browser will display basically the same thing as before, but the client side is doing the lifting here.

It's not perfect yet, though. Ideally, I'd want this to extend a standard element like <time/>, so you'd be able to write like <time is="time-ago" datetime="${release.releaseDate}">${release.releaseDate}</span> - that way, if the browser doesn't have JavaScript enabled or something goes awry, there'd at least be an ISO date visible on the page. That'd be the real progressive-enhancement path. When I tried that route, I wasn't able to properly remove the original text from the DOM like I'd like, but that's presumably something that I could do with some more investigation.

In the mean time, I'm pretty pleased with the switch. One fewer binary blob in the NSF, a bit of new knowledge of a browser technology, and the app's code remains clean and meaningful. I'll take it.

Yesterday, I wrote a post detailing my recent switch to Apple Silicon, and in it I noted that the main remaining problem I was butting heads with was running Domino in Docker containers, specifically in the context of working with Java.

While my general solution of connecting to Docker running remotely is good, it's all the better to have this stuff working locally. I gave it another shot today and found that the solution is in the post: disable the JIT.

Java uses the JIT - its just-in-time compiler - to speed up execution dynamically when running a program, and it's part of what makes Java surprisingly fast in practice. It's a whole rabbit hole of performance implications and comparisons, but it will suffice to say "JIT = good". However, as I found when running test cases that load libnotes.dylib directly, J9's JIT doesn't play well with the x64-to-ARM JIT that these Macs use, either Rosetta's or qemu's.

So, long story short, the solution was to modify my testing container to disable the JIT when in such an environment (the referenced commit has since been followed up a few times).

First off, I added a bit in my Domino One-Touch Config file to point to a Java options file in the notes.ini:

1
2
3
4
5
6
7
8

{
	/* ... */
	"notesINI": {
		/* ... */
		"JavaUserOptionsFile": "/local/JavaOptionsFile.txt"
	}
	/* ... */
}

I modified the Dockerfile to bring this in from the build environment:

1
2

# ...
COPY --chown=notes:notes staging/JavaOptionsFile.txt /local/

Finally, I modified the class I use to build the container to optionally populate this file with the flag to disable the JIT:

1
2
3
4
5
6

String arch = DockerClientFactory.instance().getInfo().getArchitecture();
if(!"x86_64".equals(arch)) {
	withFileFromTransferable("staging/JavaOptionsFile.txt", Transferable.of("-Djava.compiler=NONE"));
} else {
	withFileFromTransferable("staging/JavaOptionsFile.txt", Transferable.of(""));
}

(Transferable is a Testcontainers-ism for any available source of a file to put into the build environment. Often, this will be something from the project classpath, but it can also be arbitrary data like this.)

With that in place, the test suite works! Eventually!

The thing to know about this is that it's slow. It's going to be naturally slow due to emulation, and I imagine the lack of Java's JIT doesn't help anything. Running the XPages JEE test suite takes about 520 seconds in this setup, as compared to 180 seconds when run via my remote native VM. Such a dramatic drop in speed is enough that I'm likely to continue using the remote VM for most things, and only use local emulation for things that significantly benefit from filesystem binds. For what it's worth, it seems like multithreading is what really kills it: most tests are slower by roughly 50%, while a big multithread test balloons from 20s to 160s. That's consistent with my experience with local unit tests, where the multithread ones were the ones that crashed the process.

For the record, I found that this currently only works at all with Docker Desktop's default (qemu-based) emulation. When I switch the emulator to Rosetta, I hit a StackOverflowError from the HTTP JVM during init without any useful other information. That's fair enough, since that emulation type is still flagged as Experimental in Docker Desktop anyway.

Anyway, it's good to know that there's a way to make it work. It's still no replacement for a native container, but at least it works at all. Similar techniques should work with other tasks, like setting the above option in the JAVA_OPTS environment variable.

Almost exactly two years ago now, I wrote a post describing using Apple Silicon for work while my iMac was in the shop. That was an interesting experience, but too thorny for me to want to really stick with. Some of that was due to the limitations of the hardware - it was a pretty low-end MacBook Air - but most of that was due to the very-much-in-progress world of Java tooling for ARM Macs.

Since then, I'd been itching to find a machine to replace the iMac Pro and the recently-released Mac mini finally fit the bill. It, now dubbed Tethys, arrived on Friday, and I've been going through the process of re-carving-out my working environment, this time starting clean.

Eclipse

Compared to my first dive into ARM Macs, things have gotten a lot smoother. The biggest one for me is the stability of Java generally and Eclipse specifically. Last time, I was on the bleeding edge of that, and I was fortunate enough to at least help diagnose and report some of the things barring the port from being complete. The experience was very janky but the sheer snappiness of Eclipse on ARM really stuck with me, and it's still there in spades.

And, really, there's not even a lot of special things to know. If you download the aarch64 build of Eclipse, it just works like you'd want it to. I had to make sure to download some x64 builds of Java for when I'm running legacy native code, but that wasn't really different from my current collection of different Java versions anyway.

This is all a huge distinction from before, where I was going to weird lengths to try to run Eclipse via X11 remotely just to get something responsive.

Notes and libnotes

For my needs, Notes works fine. It's a shame that it's not native, but, as long as I run my test suites with an x64 JVM, things work about as well as they do on x64.

One immediate problem I ran into, though, was that some test cases that push multithreading would hard crash with a native error to do with the x64-to-ARM translation and JIT. Running with an OpenJ9-based JVM, I found that specifying -Djava.compiler=NONE in the launch args avoided this trouble. I'm sure that the tests run a bit slower now, but they were already emulated anyway, so it's not noticeable.

Domino and Docker

One of the sticking points with my half-transition years ago was the need to run my Domino Docker containers in emulation. Though HCL will have to port Domino to ARM eventually, they haven't yet released such a thing, and so that situation remains the same.

And, unfortunately, the segfault I encounter during my use remains. I had hoped that changed in qemu would fix it, but it appears to be unchanged. Recent versions of Docker can also take advantage of Apple's support for Rosetta in Linux, but for my needs that just creates a different segfault earlier in the process.

So, on this front, I'm doing the same thing I started doing before: running Docker on a VM on actual x64 hardware and using the DOCKER_HOST environment variable to point to it. Fortunately, I've taken steps in the intervening years to make this more practical. The main thing to work around with such a setup is the inability to use filesystem binds, and so I've changed a lot of my Dockerfiles and Testcontainers setups to instead copy more into the image at build. I still haven't solved every problem there, in particular a huge in-container build I'd like to do that points at a giant pool of dependencies that would be onerous to put into the image, but I'm working on that.

Conclusion

Overall, I'm pleased as punch with this thing. It's zippy all around, I haven't once heard the fan even when really hitting the CPU and GPU, and the state of the software ecosystem is such that I only use a very few x64-compiled processes during my daily work. I'd expected to have more of a checklist to work down to clean them all up, but it's really just the for-now-required bits. Not bad at all.

For a good long time now, XAgents have been one of the common ways to do non-HTML output in an XPages environment - JSON, mostly. I think the technique was codified and the term coined by Stephan Wissel back in 2008 and the idea has been the same since.

Effectively, an XAgent lets you write a Servlet but with a bit more scaffolding. Though XPages has a path to use Servlets officially, that method is more out-of-the-way than XAgents and doesn't (without further hoop jumping) give you some niceties like sessionAsSigner.

However, though they're venerable and sort of convenient, XAgents are lacking in a number of ways. For one, stuffing them inside an XPage is ungainly: even if the XPage just calls out to some Java code, you're polluting your UI-element space with something kind of unrelated, forcing you to name your XPages stuff like "apiEmployees.xsp". More importantly, though, it doesn't provide a lot of affordances for the sort of work you'd actually want to do when writing a REST API. Though XPages has a couple mechanisms for generating JSON, little of that is exposed by the XPage editor environment, and I suspect that many or most XAgents writing JSON commit the cardinal sin of doing so through direct string concatenation, likely often without much escaping. Further, there's no built-in mechanism for picking apart path segments or multi-branched routing.

So I figured today is a good opportunity to talk a bit about one of the XPages Jakarta EE Support project's flagship features: writing REST services that consume and emit JSON. This post covers some of the same ground I've talked about before, but sometimes it's useful to re-contextualize this sort of thing.

The XAgent Way

To level-set the discussion, I'll give a starting example in an XAgent, and we'll keep it simple. The idea here will be that you have Person documents in a database and you want to write an API that will take a UNID and emit the corresponding person's first and last name. This is very similar to the Employees example in the "Code-First REST API" example project from the JEE repo, but without diving into more-complex parts or the Jakarta NoSQL data layer.

In an XAgent, you might do something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" viewState="nostate" rendered="false">
	<xp:this.afterRenderResponse><![CDATA[#{javascript:
		var resp = facesContext.getExternalContext().getResponse();
		resp.setContentType("application/json");
		
		var writer = facesContext.getResponseWriter();
		
		var doc = database.getDocumentByUNID(param.unid);
		var result = toJson({
			"firstName": doc.getItemValueString("FirstName"),
			"lastName": doc.getItemValueString("LastName")
		});
		writer.write(result);
		
		writer.endDocument();
		facesContext.responseComplete();
	}]]&gt;</xp:this.afterRenderResponse>
</xp:view>

You'd put that in an XPage and call it like:

1
2

$ curl http://your.server/someapp.nsf/apiPeople.xsp?unid=68D1CAA80F65780D8525894D006B1CE7
{"lastName":"Fooson","firstName":"Foo"}

That does the job well enough. However, this will get gangly very fast. If you want to expand on the types of data the Person document contains, add more operations to the API, add other query parameters, or so forth, you'll have to either have a giant blob of code here, spin it off to an SSJS script library, or move it to Java classes. And, all along the way, the dev environment won't be giving you any help with the process - it knows nothing about writing REST APIs, and so it's all "manual". Better than with a classic agent, but not great.

Jakarta Way, Take 1

So let's move this over to Jakarta EE. We'll start by doing basically a concept-for-concept move: just take the above and make a REST class out of it.

Such a class could look something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

package rest;

import com.ibm.xsp.extlib.util.ExtLibUtil;

import jakarta.json.Json;
import jakarta.json.JsonObject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;

import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.NotesException;

@Path("v1/person")
public class PersonResourceV1 {
	
	@Path("{unid}")
	@GET
	@Produces(MediaType.APPLICATION_JSON)
	public JsonObject get(@PathParam("unid") String unid) throws NotesException {
		Database database = ExtLibUtil.getCurrentDatabase();
		Document doc = database.getDocumentByUNID(unid);
		
		return Json.createObjectBuilder()
			.add("firstName", doc.getItemValueString("FirstName"))
			.add("lastName", doc.getItemValueString("LastName"))
			.build();
	}
}

This one you'd call in a similar way, and get similar results:

1
2

$ curl http://your.server/someapp.nsf/xsp/app/v1/person/68D1CAA80F65780D8525894D006B1CE7
{"firstName":"Foo","lastName":"Fooson"}

There's a bit more boilerplate - it is Java - but you can already see some of the benefits. The URL is a little more REST-like and the code is much more task-focused. Instead of disabling all the normal features of the XPage and manually emitting text, a large amount of the code is explicitly describing what you intend to do in a REST service. The @Path annotations describe the components of the URL, and the use of @Path("{unid}") lets us name one of those parts without having to do a query string. Similarly, the @GET and @Produces(MediaType.APPLICATION_JSON) lines directly say what's going on: you can do a GET request to the URL and get JSON back.

These annotations pay off in a couple ways. First of all, the code will be a lot easier to understand when you come back later. Imagine if your API also handled new documents, deletions, and modifications - with an XAgent, you'd have branching paths and would have to rely on either good naming or commenting to mentally traverse it. With this, you would have other methods with similar annotations - @PUT, @POST, @DELETE - and would see exactly where requests are going to go.

And it's not just for you the programmer (or for the next human replacing you when you retire to a beach somewhere): these annotations mean something to tools that process it as well. One such tool comes with the XPages JEE project: the MicroProfile OpenAPI generator. With this class present, you automatically get a useful OpenAPI spec:

1

$ curl http://your.server/someapp.nsf/xsp/app/openapi.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

---
openapi: 3.0.3
info:
  title: XAgent Comparison
servers:
- url: http://your.server/someapp.nsf/xsp/app
paths:
  /v1/person/{unid}:
    get:
      parameters:
      - name: unid
        in: path
        required: true
        schema:
          type: string
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                type: object

This sort of thing pays off tremendously once you start working with a client JS app, especially with a larger development team.

Jakarta Way, Take 2

But we can do better than this. As it is, the direct translation from the XAgent still left the actual output pretty vague. We at least know it's emitting a JSON object, but that's the extent of it. Moreover, the app code wastes some conceptual time actually building the JSON object, which is okay but unnecessary. Let's bring JSON Binding into the mix. This will increase the size of the code, but will pay off in conceptual cleanliness and in future work.

That could look like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

package rest;

import com.ibm.xsp.extlib.util.ExtLibUtil;

import jakarta.validation.constraints.NotEmpty;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;

import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.NotesException;

@Path("v2/person")
public class PersonResourceV2 {
	
	// Normally, this class would be in a separate file
	public static class Person {
		private String firstName;
		private @NotEmpty String lastName;
		
		public String getFirstName() {
			return firstName;
		}
		public void setFirstName(String firstName) {
			this.firstName = firstName;
		}
		public String getLastName() {
			return lastName;
		}
		public void setLastName(String lastName) {
			this.lastName = lastName;
		}
	}
	
	@Path("{unid}")
	@GET
	@Produces(MediaType.APPLICATION_JSON)
	public Person get(@PathParam("unid") String unid) throws NotesException {
		Database database = ExtLibUtil.getCurrentDatabase();
		Document doc = database.getDocumentByUNID(unid);
		
		Person result = new Person();
		result.setFirstName(doc.getItemValueString("FirstName"));
		result.setLastName(doc.getItemValueString("LastName"));
		return result;
	}
}

Calling this will work the same way as last time, but with a "v2" to indicate our new-and-improved back end:

1
2

$ curl http://your.server/someapp.nsf/xsp/app/v2/person/68D1CAA80F65780D8525894D006B1CE7
{"firstName":"Foo","lastName":"Fooson"}

What does this get us? Well, for one, we're no longer explicitly working with JSON, which is nice. We could change the media type to XML and the output type will automatically adapt, and the same will go for any future formats we write an adapter for. Additionally, this will work better with a more-structured database access layer. I'll leave that part out here, but the aforementioned "Code-First REST API" example shows that.

Some more of the payoff shows up when we check the generated OpenAPI spec. If we make the same call as above, the output will be more descriptive now:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

---
openapi: 3.0.3
info:
  title: XAgent Comparison
servers:
- url: http://your.server/someapp.nsf/xsp/app
paths:
  /v2/person/{unid}:
    get:
      parameters:
      - name: unid
        in: path
        required: true
        schema:
          type: string
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Person'
components:
  schemas:
    Person:
      required:
      - lastName
      type: object
      properties:
        firstName:
          type: string
        lastName:
          minLength: 1
          type: string

For someone consuming this spec, this is starting to get really good. No longer does the output just say it's a generic JSON object: now it describes it as "Person", shows the two properties that will be output, and declares that lastName will always be at least one character both for input and output. As you add more operations and types, this spec will continue to grow, naturally piggybacking on the annotations and types you write.

Conclusion

Strictly speaking, you can do the same stuff with an XAgent that you can with the XPages JEE project. You could manage the internal complexity and you could manually write the OpenAPI spec, but it would be much, much harder to do, and I think it'd be a safe bet to say that very few XAgent-based APIs have specs to go with them anyway.

Especially as the scope of your app grows, the development and maintenance experience with the JEE approach is night-and-day compared to classical mechanisms like XAgents. If you're still writing APIs that way or similar, I definitely recommend you give this path a try.

When I remember, I like to do a bit of a year-in-review post, which generally revolves around my open-source work. For that purpose, I realized that my GitHub commit graph can set the stage nicely:

Apparently, April and October were busy months for the client work I do that isn't on github.com.

Projects

XPages JEE Support

My biggest project for the year was definitely the XPages Jakarta EE Support project. At the end of last year, I made the move to Jakarta EE 9 as the baseline, with its associated package-name switch from javax.* to jakarta.*. This move really let the project flourish. Since I'm no longer regularly fighting with the ancient versions of JEE specs that ship as part of the XPages stack, I've been able to add a bunch of new features, such as JSF, Concurrency, Transactions, and improved Servlets. I've also been able to make huge strides on the Jakarta NoSQL support, adding the necessary evil of view-based operations and provisional drivers for the AppDev Pack and Keep.

Though that project is starting to get very constrained by Jakarta EE 10's move to Java 11 as a base requirement, there's still a lot of work I can do with the EE 9 specs. I have some larger ideas in mind and I have a stack of issues in the list of various sizes to look at. This project is the baseline for one of my larger client projects, and it's my current best idea for what Java development with Domino should look like.

NSF ODP

Next up would probably be my workhorse, the NSF ODP Tooling. Though I put out a good few releases this year, they've primarily been about compatibility with new Eclipse and macOS releases. macOS compilation in particular is a real bugbear: the OS's tightening restrictions and Notes's varying ways of adapting to them have made it a real moving target.

I've had it in my mind for a while now to add a specific Docker-based compilation option. It's always been possible to run the compilation inside Docker by building your own container and using the tooling within it, but it'd be useful to have that as a standardized thing controlled by the Maven plugin. That way, you'd still do the normal Maven commands for your environment and, if configured, it would build an image internally based on the Domino container image and do the build in there. That should be more reliable, particularly for macOS and Windows, but I'll have some fiddly details to iron out. Should be doable, though - it's on my hopeful list for this year.

Domino Open Liberty Runtime

The Open Liberty Runtime project is in a weird spot. It's been progressing in fits and spurts, and it's gradually moving towards a good future. My general notion for it is to make it an easier way to run tasks attached to a Domino server. So far, those "tasks" have been specifically Liberty running webapps, but the core concept is just that it's managing executables that can cooperate with the Domino server. I'd like to make this more generic, smoothing the process of running other Java-based tasks with an arbitrary JVM and potentially managing non-Java executables as well. There have been some rocky days in there dealing with the changing landscape for how open-source JVM builds are located and deployed, but the growth of the Adoptium Marketplace and its stable API looks to have settled things down for me.

Other Stuff

Beyond specific projects, I can think of a few notable public things for me this past year.

Local Library Work

Since shortly after I moved to my current house, I've been involved with the Friends of the Library group for the local branch of our library system. This past year, I also joined the overall board for the system, as its bylaws reserve a seat for a member of each of the Friends groups. This has been interesting, since, while it's still a comparatively-low-key thing, it's a bit more structured of a group than other organizations I've been involved with. It's also been a good way to keep a bit more informed about local politics without actually getting into local politics.

Also, we just officially went fine-free today, which is neat. That's one of those things where you originally think that fines are essentially required, but statistics show that they're effectively just a tax on poor kids and systems are better off dropping basic fines. I'm pretty pleased with the move.

Mastodon

I remain thoroughly pleased with my mostly-move to Mastodon. I still check Twitter, but I've been cutting down on follows over there, mostly from service accounts and those who have also made the switch. It feels like a loose tooth now, and is mostly waiting for replacement by a good Mac client. I got in the Ivory beta a while back, and slotted that right in place where Tweetbot used to be on my phone.

With any luck, Twitter will continue to be afflicted by catastrophe after catastrophe and more people will make the jump. If you haven't yet, I certainly suggest you give it a shot. I like Action Retro's video for a primer.

GitHub Sponsors

A few months ago, I set up a GitHub Sponsors profile. I figure this may be a good way to handle some cases where something I'm doing doesn't rise to the level of a whole contract or to essentially "vote" in favor of me doing the sort of open-source work I do. Admittedly, I expect I'll keep doing that regardless, but the contributions I've received so far are very gratifying.

Resolutions

I haven't been the sort of person to do New Year's Resolutions as such, but I still figure it's a nice time of year to make general plans and goals for the year. For example, I found that my gaming habits got into a bit of a rut last year - lots of Stardew Valley and Terraria, and fewer games that I haven't already played to death. I've got a growing stack of acclaimed games to play, and I'd like to work down the list, even when the games are older than dirt. Same goes for reading: considering I'm on the board of the library system, I sure don't do a lot of book reading lately. I should fix that!

Beyond that, we'll see. There are some neat things going on at OpenNTF that could set the tone for the year - in the short term, we're planning on running some "repair caf?s" this month as a way to do another kind of community interaction. I wouldn't mind doing that sort of thing more often; I've done a few "let's talk about Java" calls in the past and always enjoy them. I hope those take off enough for us to do them regularly.

In any event: Happy New Year!

As part of moving more of my attention over to ActivityPub/Mastodon, I've seen a couple posts like this one describing how you can configure your primary domain to contain information that can lead searches to find your Mastodon account.

Most often, this is useful for people who have their own blog but use one of the larger existing Mastodon instances for their social activity. I'm not quite in that circumstance, but I still wouldn't mind it if you could look up my name as "@jesse@frostillic.us" and have it resolve to my actual account of "@jesse@pub.frostillic.us". Not a huge thing, but also seemingly not difficult to solve.

So, this morning, I set out to solve it. Before I get to the project I made, I'll make this clear: the reasonable way to do this would be to put a static JSON file up and have the "query" URL point to it. I didn't do it the reasonable way, but I didn't do it entirely unreasonably either.

How It's Done

The main mechanic in this sort of redirect lookup is WebFinger, a historically-but-disturbingly-named RFC from 2013. The idea of that is to be a HTTP-and-JSON-based equivalent for the sort of thing one used to use finger for. As is often the case, it's generalized beyond the specific need here, and it'd be interesting to implement more of its options, but for now the useful part is to be able to lookup a "resource" like acct:jesse@frostillic.us to get account info.

Mastodon provides this and related services for users in its directory, but that wouldn't help for trying to alias a user that doesn't exist in there. Since I'm already using Domino for my user store, I decided to implement a simple version of this to look up this info from Domino itself.

WebFinger For Domino

Thus was born the WebFinger For Domino project, which I expect to be a minor project with few releases. This project includes a single Servlet, listening at "/.well-known/webfinger", and it only supports lookups of account resources in the format that Mastodon uses. If you visit, for example, "https://frostillic.us/.well-known/webfinger?resource=acct:jesse@frostillic.us", you'll get output like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

{
  "aliases": [
    "https://frostillic.us"
  ],
  "subject": "acct:jesse@frostillic.us",
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "href": "https://pub.frostillic.us/@jesse"
    },
    {
      "rel": "self",
      "href": "https://pub.frostillic.us/users/jesse",
      "type": "application/activity+json"
    },
    {
      "template": "https://pub.frostillic.us/authorize_interaction?uri={uri}",
      "rel": "http://ostatus.org/schema/1.0/subscribe"
    }
  ]
}

If you try to look up a user that doesn't exist or isn't explicitly configured to participate in this, you'll get a 404 Not Found response.

As when I added the "MastodonUsername" field to begin with, I implemented this by adding some new fields to my Person document:

These fields are included directly or used to compose the URLs above. The specifics are documented in the project README.

It seems to hold together well enough, in that I was able to successfully search for my user by "@jesse@frostillic.us", and that's all I could ask. If you're using Mastodon and have your own Domino-backed domain, give the project a look.

Keeping with my productive week off, today I release version 2.9.0 of the XPages Jakarta EE Support project. Similar to the previous release, this one contains new features primarily related to Jakarta NoSQL, but also has some improvements for JSF and a bunch of bug fixes and compatibility improvements.

Jakarta NoSQL

The improvements to the JNoSQL driver come from some needs I came across when moving older lotus.domino/ODA-based code to using JNoSQL repositories. In particular, I added the remaining applicable view entry properties as available fields to map, added better support for reading note IDs, and fetching documents by note ID.

JSF

While JSF support remains limited by not having a proper way to add in third-party component libraries like PrimeFaces, it's still a potentially-compelling tool in an NSF as an alternative to XPages in some cases. Accordingly, I fixed a few bugs I had run into when loading pages after modifying the NSF design. Additionally, I fixed up support for JSF as an MVC view engine. It now properly joins JSP as a mechanism for rendering your output with an MVC structure, and I think there's some real potential there.

Bug Fixes and Compatibility

Most of the other closed issues deal with a few bugs here and there, and in particular involve some improvements for running apps in XPiNC and on a server with Domino Leap also installed. I don't use XPiNC anymore and haven't tried Leap, so I greatly appreciate bug reports specific to these and the assistance in tracking down the trouble.

The Future and Next Steps

I'm pondering now what the next release of the project will focus on. I have no shortage of feature ideas, and there are a few potentially-disruptive changes I'd like to make.

Unfortunately, those changes will be largely confined to improving the support for the specs that are already present and not advancing to new versions. The predicted Java-version wall arrived: Jakarta EE 10 is out and requires Java 11 and above. Since Domino remains mired in Java 8, that means that new versions of the specs and implementations are hard-incompatible until that changes.

On the plus side, there's still a lot of improvements I can make with Jakarta EE 9 as the baseline.

Reorganization

One big one I've been thinking about is a reorganization of the individual libraries that make up the project. The way it's been designed, almost every spec has its own Equinox Feature and XPage Library to go with it. This was fine early on when it was just CDI, EL, and JAX-RS, but it's grown annoying: installing the project in Designer is a seemingly-endless process of approving each plug-in at a time and the list of libraries to check in Xsp Properties is interminable. More critically, being able to selectively turn on and off specs like this doesn't make sense anymore. CDI has grown so important to Jakarta EE in general and this spec in particular that it doesn't make sense to not have it present if you're going to use this project at all. It's a foundational component of so many other parts and is essentially The Way to do Jakarta-based development.

So I'm thinking I'm going to reorganize the projects into fewer features and libraries, which will be a breaking change that will necessitate a bump to 3.0 - fortunately, the numbers line up well for that. I have a few potential options here:

1. Just lump them all into one. You'd have basically one big switch to say "this is a JEE project" and everything would be on. The virtue here is that this is how I already work and is essentially the recommended way to do things. Additionally, as far as I know, while having additional components may slow first load (though not as much as other parts), I don't think they have a significant impact if enabled but unused during runtime.
2. Try to line the specs up with one of the existing Jakarta Profiles. Those profiles are meant to be curated selections of useful specs, and this project has enough to implement what in newer versions is deemed the Core Profile. The trouble with this, though, is that the Core Profile is very much geared to be the shared subset with MicroProfile and similar and is a bit thin for Domino's monolith-focused development style. The Web and Full profiles, on the other hand, require "traditional" APIs like EJB that are not present in this project.
3. Break them apart into my own "core" and "optional" features. For example, it doesn't make sense to use this without JAX-RS, CDI, and Bean Validation enabled, but JSF is entirely independent of the other specs and is among the least likely to be used in practice for now. This would also allow me to establish a running flow where "experimental" features start out as optional add-ins and then eventually make their way to core.

I'm currently waffling between #1 and #3, with a slight lean towards #1. If I can be sure that either everything or nothing is present, I could get rid of some weird hedges and workarounds, like how the JAX-RS implementation doesn't "officially" know about the CDI library yet references CDI classes explicitly by name.

New Application Types

Currently, to use this project, you can either put your code into an NSF and use the automatic behavior of the libraries or you can put your code in OSGi-based webapps or Servlets and then manually manage integration with these specs.

Both of these are limited by their reliance on the many assumptions IBM built in to how these apps should work. In-NSF apps require that all Jakarta code come from a request including "xsp" in the URL or to a file ending in ".jsp", ".jsf", or ".xhtml". If you're writing, say, an MVC-based app, all of your URLs are going to have to start with something like "foo.nsf/xsp/app/...", which is okay but ugly. Additionally, the way these apps are implemented - NSFComponentModule - severely limits my hooks for listening for things like application and session expiration, which hampers CDI's lifecycle handling a bit.

For a good while, I've pondered the notion of adding another ComponentModule type to handle the case where you want to go all-in on Jakarta EE. With this idea, the new module implementation would have full control over incoming requests, allowing URLs without the xsp/app bit in there, and would have better handling of lifecycles. In this way, I could make it so that your could would look more like (or be identical to) a "normal" .war-based webapp, with fewer workarounds for the existing XPages stuff. This would also allow me to do things like lessen the amount of Servlet 2.5-to-5.0 bridging and could assist tremendously in improving JSF support.

Along similar lines, I've been considering doing something similar for OSGi-based webapps, and I've made some progress along those lines in a feature branch. The idea here would be to do something similar to how you can deploy web.xml-based webapps via OSGi now, but with built-in support for Jakarta EE 9 features (with web.xml then being optional). With this setup, you'd be able to write an app that does an Import-Package for the various jakarta.* packages you want and add a bit in your MANIFEST.MF to signal to this project that it should participate. This could either be a variant of the extension point used by the existing OSGi webapp support or using the Web-ContextPath directive from the OSGi spec. One of the goals here would be to make it so that you would be able to write a Jakarta EE 9 app using normal development tools - Eclipse/IntelliJ/VS Code, Maven, etc. - and then just use maven-bundle-plugin to add the OSGi info you need without having any specific dependencies on Domino bits, especially the nightmare of depending on the non-redistributable XPages OSGi artifacts.

Other Options

And, in the mean time, I have a bunch of other tasks I could work on. Slowly converting my client project to Jakarta NoSQL instead of direct ODA use has turned up a whole slew of things that would be useful to add (for example, stampAll support), so I can slowly burn down that feature-request list.

There's also the notion of documentation! While a lot of the behavior of this project is in theory documented by virtue of the upstream specs and the general world of Jakarta blogs, videos, and courses, there's enough to know about the specifics of the interactions with Domino that more documentation is in order. Historically, I've just done this by expanding the README, but it's gotten pretty unwieldy at this point. It would probably make sense to break the specifics and examples out into at least wiki pages, if not a format that can be built into a PDF/etc. and included in the distribution.

So yep, I'll have my hands busy with this thing for a good while more, I figure.

The other day, I talked about some changes/workarounds for Notes/Domino 12.0.2. Today, I made a few updates to some of the open-source projects I maintain, including another update to the generate-domino-update-site Maven plugin.

Domino Update Site Generator

In the 4.2.0 release, I added code to (mostly, as it turns out) account for HCL moving the NAPI implementation JAR down to jvm/lib/ext. In subsequent use, I found that, while that will suffice for building applications that use the OSGi dependencies, it didn't work for launching applications using it as a baseline - namely, the NSF ODP Tooling Maven plugin.

Today, I released a 4.2.1 version that improves this behavior by re-adding dependencies in the implementation bundle.

I also created a project page for it on OpenNTF. Though the project has always been hosted at the OpenNTF org on GitHub, I hadn't created a project page for it due to it just being a standalone Maven plugin. I figured it'd be useful to create an official page there for it, though.

NSF ODP Tooling

Speaking of the NSF ODP Tooling, I also found that local operations once again started crashing on macOS. Due to changes in macOS and the very weird ways that Notes works, local operations on there are a very moving target, and I have to do a lot of work in the project to account for changes to the embedded JVM and whether specific Notes versions work better with HotSpot or OpenJ9 JVMs.

Long story short, I release 3.10.0 today to account for this. Though I've found that the spawned JVM will still sometimes crash, it's after completing its work, so I considered that fine for now.

OpenNTF Domino API

Finally, I came to the OpenNTF Domino API. This project has admittedly been neglected for a little while: I'm the only active maintainer, and the client project I use it in targets Domino 11.0.1, so the 12.x builds have remained in an incomplete state for a while.

With the release of 12.0.2, I decided I should finish the wrappers for the new classes added in 12.x, so I did so and uploaded a build for 12.0.2. This primarily adds those wrappers, but also included a contributed fix and changes the distribution packaging to combine the XSP and non-XSP versions.

Notes and Domino 12.0.2 came out today. Generally, there are some neat features in development and on the server, but there are also a couple things you may run into depending on your workflow and installation type.

Update Site NSF

The update site NSF that ships with Domino uses SWT for some of its GUI elements when importing contents. This still works fine in the 32-bit client, but is broken in the 64-bit client. My guess on that front is that the 64-bit client doesn't come with a 64-bit native SWT JAR, probably because the SWT version used for this likely pre-dates x64's popularity on the desktop.

For now, the workaround is to use a 32-bit client if you're working with the Update Site NSF. Karsten said he's going to patch the OpenNTF version of the NSF to deal with this, so you can also wait for that one.

Domino Update Site Generator

I maintain the generate-domino-update-site Maven plugin that can be used to generate update sites for OSGi development against the Domino stack. These sites are replacements for the IBM Domino Update Site for Build Management, which was released for 9.0.1 and never updated since. Only HCL can make a new distributable version of that, so my tool lets you generate one for yourself from a Notes or Domino installation.

In 12.0.2, HCL shunted the NAPI implementation JAR down to jvm/lib/ext to support the shared JARs between XPages and agents feature. As a side effect, existing versions of my plugin would lack the NAPI classes.

Today, I released version 4.2.0, which fixes this and also contains improvements to let the plugin work on current Java versions and generate sites based on 12+ macOS clients.

Along these lines, I made Aha idea DDXP-I-352 a couple years ago to request that HCL provide such sites themselves or give OpenNTF permission to provide them, so I'd appreciate it if you voted for that.

The Target Platform Bug

This isn't a new thing, but it's worth mentioning here since it comes up frequently: the target platform bug from 9.0.1FP10 remains. As of earlier this year, defect article KB0086688 mentions this, though the status is "Deferred". If this afflicts you, it may help to bring it up with Support and reference that article.

Because of what I'll euphemistically call the current historical moment on Twitter, I (like a lot of people) decided to give another look at Mastodon. The normal way one would use it would be to sign up at mastodon.social and be on one's merry way, treating it just like a slightly-different Twitter.

However, Mastodon is intentionally designed to be federated in a way similar to email, and the software is available on GitHub complete with scripts for Docker Compose, Vagrant, and so forth. So I went and did that, setting up my currently-barely-used account at @jesse@pub.frostillic.us.

That on its own isn't particularly notable, nor are the specifics of how I set it up (it was a hodgepodge of a couple posts you can find by looking for "mastodon docker compose"). What I found neat for our purposes here was the way I could piggyback authentication onto stuff I had recently done with Keycloak. Keycloak, incidentally, was the topic of today's OpenNTF webinar, so, if you didn't see it, check back there for the replay when it's posted.

Having done the legwork for setting up Keycloak backed by Domino LDAP for my earlier tinkering, the setup to work with Mastodon was pretty straightforward (as far as these things go). I did the professional thing and took the basic config from a StackOverflow post, tweaking it to suit my needs.

The main Domino-y thing I wanted to tweak here was the username that I ended up with on Mastodon. Internally, the Domino short name for my account is "jgallagh", but I like to go by "jesse" when in an environment small enough to get away with it. So I cracked open the names.nsf subform I had added years ago for POSIX and SSH pubkey purposes and added a Mastodon section:

(apologies for how bad the new-era fonts look in my poor old Windows VM)

Then, I told my Mastodon config about that field for the UID:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

OIDC_ENABLED=true
OIDC_DISPLAY_NAME=frostillic.us Auth
OIDC_DISCOVERY=true
OIDC_ISSUER=https://<keycloak_url>/auth/realms/<real>
OIDC_AUTH_ENDPOINT=https://<keycloak_url>/auth/realms/<real>/.well-known/openid-configuration
OIDC_SCOPE=openid,profile,email
OIDC_UID_FIELD=mastodonusername
OIDC_CLIENT_ID=<client id>
OIDC_CLIENT_SECRET=<client secret>
OIDC_REDIRECT_URI=https://<mastodon URL>/auth/auth/openid_connect/callback
OIDC_SECURITY_ASSUME_EMAIL_IS_VERIFIED=true

On Keycloak, I made a new realm to cover this sort of "personal" setup to be able to split the user pool and then added a Client definition for Mastodon. I set it up as "Access Type" "confidential" and grabbed the client ID and secret for the config above and then configured the Redirect URI. To get the custom username field over from LDAP, I added a "user-attribute-ldap-mapper" Mapper in the LDAP User Federation definition to bring it in. Then, back in the Client definition, I added a "User attribute" token mapper to the config to bring this in as well so it's added to the JWT.

That covered the auth config, and it's been working well since. When you have OIDC configured in your Mastodon config, it sprouts a button below the main login mechanically labeled "OPENID_CONNECT":

Clicking that will send you to the configured Keycloak page to do the OIDC dance and, when all goes well, back to a freshly-configured Mastodon account.

Now, technically, this doesn't really gain me much that I couldn't have gotten by configuring the users separately in the Mastodon instance, but the experience is useful. I'm gradually getting really sold on the idea of having a multi-purpose Keycloak instance to handle authentication and authorization. Most of the time, it's a thin layer over what you could get by pointing to Domino LDAP from these disparate apps themselves. However, there are some benefits in that Keycloak is now the only one that has to deal with Domino's weird LDAP and also this gives me a lot of room for fine-grained control and federation with other providers. It's just neat.