I've been working on my workshop for this year's CollabSphere, and one of the main decisions I have to make is what I'm going to focus on. The idea of the workshop is to give a bit more brass-tacks information about how to use the project: rather than just a list of features, it'll be about the specific business of building an app using it.

But how does one build an app in it? There's certainly no lack of tools available, but that leads to the opposite problem: what's the right one for your project? What's likely to be the most common path people take?

The Types

As I've been working on it, I've grouped things into four main categories, and I figured it'd be useful to enumerate them here to coordinate my thoughts and provide some general information. There aren't hard lines between these: you can use any mixture of some or all of the parts in an app, and do different mixes in different apps. These are just what I expect to be the main groupings:

• "XPages Plus", using some new capabilities in existing or new apps with XPages-based UIs
• REST services, focusing on providing REST endpoints for JavaScript-based apps or other servers
• MVC and JSP, focusing on clean, lightweight UIs for document-based apps, but less ideal for complex business logic
• JSF, building the same sorts of apps XPages is adept at, but using newer technology

"XPages Plus"

The first route is how the project got started: you keep building XPages apps but sprinkle in a few new capabilities to improve them.

For example, you could replace your managed beans defined in faces-config.xml with CDI beans, allowing you to get the quick benefit of annotation-based definitions and then the bigger benefits of @Inject, producer methods, and interceptors.

You could also start using newer EL features, like the long-desired ability to pass parameters to methods.

This path wouldn't necessarily require a lot of reworking of your app or changing the way you think about XPages development, but would still be something of a minor development refresh and can set you up well for future improvements.

Your data access will likely still be through the traditional xp:dominoDocument and xp:dominoView components, but you could also write beans that access data with lotus.domino or ODA, or switch to using the NoSQL driver.

REST Services

Alternatively, you could decide you want to focus your apps around REST services with either a JavaScript app in, for example, React as the front end, or providing services to remote servers.

With this, you'd largely stop using XPages design elements entirely, instead defining your services in Java classes with JAX-RS annotations. This brings huge advantages over other ways to write REST services on Domino, with the JAX-RS annotations allowing for clear, logical definition of services, their parameters, and their output. Moreover, the ancillary tooling brings things like automatic OpenAPI definitions, which would be annoying to maintain using things like the XPages-side REST controls.

This path is good if you're specifically aiming to build a JavaScript-based app, either because you just like it, because your organization decided to go that route, or if you have a larger team that splits the duties of front-end and back-end developers. It can also naturally blend into the next one.

Your data access here won't be through the XPages components, but you could still use lotus.domino or ODA classes, or switch to the NoSQL driver. That actually goes for the next two, too, so we'll just count that as assumed.

MVC and JSP

I'll admit that part of the reason I want to consider this a top-tier route is because I just personally really like it. I've had a blast writing apps like this blog and the OpenNTF site using this path, with its much-cleaner code and back-to-basics approach to HTML.

Regardless of my personal enjoyment of it, though, this has some nice advantages. The fact that MVC builds on top of JAX-RS means that it melds well with the REST-services approach above. For example, you might primarily write REST services for a JS app, but then do a set of "admin" pages with MVC. Or you might use this as part of the prototype phase: structure your app the same way you will when you expand to a multi-tier team, but start out by doing a quick UI with MVC on top of the same or related endpoints.

With this path, your app will start with Java classes with JAX-RS annotations, and then you'd mix it with JSP files inside WebContent/WEB-INF. One down side to this approach is that Designer doesn't provide much help for writing JSP files. In the tooling, I bind .jsp and .tag files to the HTML editor, so you at least get normal HTML assistance, but that won't help you with specific JSP tags and EL. Fortunately, the set of tools you'll likely use in JSP is comparatively small, so you'll eventually memorize things like <c:forEach items="..." var="...">...</c:forEach> in much the same way that you could eventually write out an <xp:repeat/> in your sleep in XPages.

JSF

This one, technically tricky though it may be, is conceptually straightforward: write the same sort of apps you do with XPages, but do it with modern JSF instead. This makes a lot of sense, since JSF shares XPages's acumen with complicated forms with partial refreshes and changing state data, but has benefited from some development that didn't happen on the XPages side.

It's not a direct replacement: in particular, JSF has no knowledge of Domino data sources, so there's no xp:dominoDocument or xp:dominoView. You'd still need to do your data access via beans, as in the previous two options, likely using either lotus.domino/ODA or the NoSQL driver. Additionally, Designer really doesn't help you here - again, I map .xhtml and .jsf files to the HTML editor, but JSF components have a lot of properties to set, and so you'll be spending a lot of time referencing documentation.

Still, it's clear why this is proving to be a popular path. The development model is the same as in XPages, while the JSF stack (especially including PrimeFaces) brings a lot of amenities that aren't in XPages and are also more portable to other environments.

Conclusion

So, for now, I'm thinking of splitting up the workshop to cover each of these paths a bit. That runs the risk of feeling like too much of a grab bag, but I don't want to give the opposite impression, that the project only allows for some specific path. It's a broad platform update, accommodating many development approaches, and I want to keep that clear. Fortunately, each path has a pretty-clean pitch, and the shared components (CDI, bean validation, the REST client, etc.) build on each other well, so the idea that it's a pool of features that you can swim in is, I think, compelling.

Today, I released version 2.13.0 of the XPages Jakarta EE Support project. Though there's not a single big banner feature, this one brings a number of good enhancements in a bunch of areas.

Domino 14

The first thing it brings is compatibility with Domino 14 EAP1. The goal here is to just bring the same features to that version - it doesn't bump the individual components to their Jakarta EE 10 versions yet, since that will come with breaking changes and prevent use on 12.0.2 and before.

There remains a caveat here, which is that EAP1 doesn't include a Java compiler, and so JSP doesn't work unless you shim in parts of a JDK into a Domino installation. If you're not using JSP, though, you should be able to run your apps on 14 using this new build.

JSF

It turns out that Faces support is a popular feature, which makes sense: it's the most direct analogue to writing XPages, while bringing in a lot of new features. While Faces has always been tricky to keep working, this build includes some fixes for stability and usability. I'd still consider this route to be the least-proven way to do UIs with this project, but it's shaping up really nicely.

JavaSapi

Speaking of experimental features, this release comes with a new feature that builds on the JavaSapi bridge I added a bit ago: you can now specify extensions within an NSF that will participate in JavaSapi pre-processing of requests.

To do this, you can make a file named META-INF/services/org.openntf.xsp.jakartaee.jasapi.JavaSapiExtension in your NSF's Java classpath (e.g. the Code/Java folder) and have it name a JavaSapiExtension class. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

package javasapi;

import org.openntf.xsp.jakartaee.jasapi.JavaSapiContext;
import org.openntf.xsp.jakartaee.jasapi.JavaSapiExtension;

public class TestJavaSapiExtension implements JavaSapiExtension {
	@Override
	public Result rawRequest(JavaSapiContext context) {
		// Add a custom header to all responses
		context.getResponse().setHeader("X-InNSFCustomHeader", "Hello from NSF");
		return Result.SUCCESS;
	}
	
	@Override
	public Result authenticate(JavaSapiContext context) {
		// Custom authentication mechanism. If you use this, do it more securely!
		String overrideName = context.getRequest().getHeader("X-OverrideName");
		if(overrideName != null && !overrideName.isEmpty()) {
			context.getRequest().setAuthenticatedUserName(overrideName, "Basic");
			return Result.REQUEST_AUTHENTICATED;
		}
		return JavaSapiExtension.super.authenticate(context);
	}
}

As with any time I do anything with JavaSapi, I can't stress enough how unsupported this is. It's not even officially a feature of Domino, and I've found it fairly easy to crash the server by doing the wrong thing here. On the other hand, it's neat and fun, so... feel free to tinker with it.

NoSQL

Finally, I added some methods to DominoRepository instances to access profile and named notes:

1
2
3
4

SomeEntity profile = repository.findProfileDocument("SomeProfile", "Your Username")
	.orElseThrow(() -> new NotFoundException("Could not find profile for user"));
SomeEntity named = repository.findNamedDocument("Some Name", "Your Username")
	.orElseThrow(() -> new NotFoundException("Could not find named doc for user"));

I made them return Optional for safety's sake - I think they'll in general create the documents if they don't exist, but I wanted to leave room in the API for a future ability to only return them if they haven't previously been explicitly created.

Anyway, that's one more step in making the driver useful as a general-purpose Domino access mechanism. My goal is to make it so that you'd only need lotus.domino, ODA, or another Domino-specific API in specific edge cases. I can already do almost everything I need to, and now I'm just working down the list of less-critical features.

Next Steps

As I've been working on 2.13.0, I've also been working on the 3.0 branch, including an early beta last month. That's the branch that breaks pre-Domino-14 compatibility and bumps most components up to their Jakarta EE 10 versions. Since I can't realistically have a proper release of that until Domino 14 is out, my plan is to keep tinkering with the side branch and releasing betas from time to time.

In the mean time, I wouldn't be surprised if there's a 2.14.0. There are some tweaks and efficiency improvements I want to make particularly for JSF, so I expect I'll have enough on my plate before Domino 14's release to get another current-line release out.

CollabSphere 2023 is coming up at the end of next month, from August 29th through the 31st. It's in-person again this year, and, after quite a bit of hemming and hawing on that point, I decided to go. Moreover, I'll be hosting and participating in a couple sessions, including a workshop on the 29th. Specifically, that one is:

Developing Applications With XPages Jakarta EE
Tuesday at 1 PM Central, Linnaeus Room
The XPages Jakarta EE project adds a swath of new capabilities to Domino development. This workshop will give an overview of some of the major features and will include a follow-along demonstration of developing a representative application. There will also be room for questions and discussion about the project and how to integrate it with existing apps and workflows.

Attendees who wish to follow along with development should bring a laptop with a local Domino server and Designer (12.0.2 recommended) with the XPages Jakarta EE project from openntf.org installed in both.

As with the other workshops, these require pre-registration, so I encourage you to do so. If you'd like to attend and end up hitting trouble with the pre-setup process, head on to the OpenNTF Discord - there's a thread in the #specific-projects channel for the project that's a perfect spot for questions.

Beyond the workshop, I'll be running a roundtable similar to the one I did at last year's online CollabSphere:

DEV107 - Java and Jakarta EE on Domino Roundtable
Wednesday at 4 PM Central, Burnstein Room
The story of Java on Domino can be complex, but there are tools and strategies available to keep your development maintainable and consistent with the wider world. This roundtable will focus on discussing the current array of best options for development on Domino, with a secondary focus towards discussing how the XPages Jakarta EE Support project can be used to improve your development experience and the capabilities of your apps.

Immediately afterward, in the same room, is OpenNTF's roundtable, with the theme of a "live Repair Café" in the vein of the events we've been hosting in Discord:

COL104 - OpenNTF Live Repair Café: REST Practices
Wednesday at 5 PM Central, Burnstein Room
Join us for an interactive discussion at our OpenNTF Live Repair Café as we troubleshoot and repair together. Building on the success of our previous virtual events, this live event offers a unique opportunity to engage with experts and peers in the field face-to-face. Don't miss out on this chance to learn, share and grow together.

Finally, I'll be running a session on Thursday to discuss the new features we'll have access to in the move from Java 8 to Java 17:

DEV106 - Java 8 to Java 17: The New Goodies
Thursday at 9 AM Central, Linnaeus Room
Domino 14 is slated to include Java 17, an upgrade of two Long-Term-Service Java versions that brings a slew of improvements and new features. While most of the APIs we use will likely remain the same, recent versions of Java have focused heavily on developer-friendly niceties. This presentation will go over some of the more-useful ones for Domino developers, such as the new HttpClient, text blocks, records, convenience methods, and cleaner syntax.

That one should be neat. It's been a long time between releases we've used, and Java's been getting pretty good lately.

In any event, if you're attending CollabSphere, I hope to see you at these sessions.

I mentioned the other week that I've been pondering ways to rearrange the servers I have in the basement here, which I'm presumptuously calling a "homelab".

After some fits and starts getting the right parts, I took the main step over the weekend: reworking terminus, my gaming PC and work VM host. In my earlier post, my choice of OS was a bit up in the air, but I did indeed end up going with the frontrunner, Proxmox. While the other candidates had their charms, I figured it'd be the least hassle long-term to go with the VM-focused utility OS when my main goal was to run VMs. Moreover, Proxmox, while still in the category of single-vendor-run OSes, is nonetheless still open-source in a way that should be reliable.

The Base Drive Setup

terminus, being a Theseus-style evolution of the desktop PC I've had since high school, is composed generally of consumer-grade parts, so the brands don't matter too much for this purpose. The pertinent part is how I reworked the storage: previously, I had had three NVMe drives: one 256GB one for the system and then two distinct 2TB drives formatted NTFS, with one mounted in a folder in the other that had grown too big for its britches. It was a very ad-hoc approach, having evolved from earlier setups with other drives, and it was due for a revamp.

For this task, I ended up getting two more 2TB NVMe drives (now that they're getting cheap) and some PCIe adapters to hold them beyond the base capacity of the motherboard. After installing Proxmox on the 256GB one previously housing Windows, I decided to join the other four in a RAID-Z with ZFS, allowing for one to crap out at a time. I hit a minor hitch here: though they're all 2TB on paper, one reported itself as being some tiny sliver larger than the other three, and so the command to create the pool failed in the Proxmox GUI. Fortunately, the fix is straightforward enough: the log entry in the UI shows the command, so I copied that, added "-f" to force creation based on the smallest common size, and ran the command in the system shell. That worked just fine. This was a useful pace-setting experience too: while other utility OSes like pfSense and TrueNAS allow you to use the command line, it seems to be more of a regular part of the experience with Proxmox. That's fine, and good to know.

Quick Note On Repositories

Proxmox, like the other commercial+open utility OSes, has its "community"-type variant for free and the "enterprise" one for money. While I may end up subscribing to the latter one day, it'd be overkill for this use for now. By default, a Proxmox installation is configured to use the enterprise update repositories, which won't work if you don't set up a license key. To get on the community train, you can configure your apt sources. Specifically, I commented out the enterprise lines in the two pre-existing files in /etc/apt/sources.list.d/ and then added my own "pve-ce.list" file with the source from the wiki:

deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription

Importing Old Windows VMs

My first task was to make sure I'd be able to do work on Monday, so I set out to import the Hyper-V Windows VMs I use for Designer for a couple clients. Before destroying Windows, I had copied the .vhdx files to my NAS, so I set up a CIFS connection to that in the "Storage" section of the Proxmox GUI, basically a Proxmox-managed variant of adding an automatic mount to fstab.

From what I can tell, there's not a quick "just import this Hyper-V drive to my VM" process in the Proxmox GUI, so I did some searching to find the right way. In general, the tack is this:

• Make sure you have a local storage location set to house VM Disk Images
• Create a new VM with a Windows type in Proxmox and general settings for what you'd like
• On the tab where you can add disks, delete the one it auto-creates and leave it empty
• On the command line, go to the directory housing your disk image and run a command in the format qm importdisk <VMID> <imagename>.vhdx <poolname> --format qcow2. For example: qm importdisk 101 Designer.vhdx images --format qcow2
• Back in the GUI (or the command line if you're inclined - qm is a general tool for this), go to your VM, find the imported-but-unattached drive in "Hardware", and give it an index other than 0. I set mine to be ide1, since I had told the VM in Hyper-V that it was an IDE drive
• In "Options", find the Boot Order and add your newly-attached disk to the list
• Download the drivers ISO to attach to your VM. Depending on how old your Windows version is, you may have to go back a bit to find one that works (say, 0.1.141 for Windows 7). Upload that ISO to a local storage location set to house ISOs and attach it to your VM
• Boot Windows (hopefully), let it do its thing to realize its new home, and install drivers from the "CD drive" as needed

If all goes well, you should be able to boot your VM and get cracking. If it doesn't go well, uh... search around online for your symptoms. This path is about the extent of my knowledge so far.

The Windows Gaming Side

My next task was to set up a Windows VM with PCIe passthrough for my video card. This one was a potential dealbreaker if it didn't work - based on my hardware and what I read, I figured it should work, but there's always a chance that consumer-grade stuff doesn't do what it hypothetically should.

The first step here was to make a normal Windows VM without passthrough, so that I'd have a baseline to work with. I decided to take the plunge and install Windows 11, so I made sure to use a UEFI BIOS for the VM and to enable TPM support. I ran into a minor hitch in the setup process in that I had picked the "virtio" network adapter type, which Windows doesn't have driver support for in the installer unless you slipstream it in (which I didn't). Windows is extremely annoyed by not having a network connection at launch, dropping me into a "Let's connect you to a network" screen with no options and no way to skip. Fortunately, there's a workaround: type Shift+F10 to get a command prompt, then run "OOBE\BYPASSNRO", which re-launches the installer and sprouts a "skip" button on this phase. Once I got through the installer, I was able to connect the driver ISO, install everything, and have Windows be as happy as Windows ever gets. I made sure to set up remote access at this point, since I won't be able to use the Proxmox console view with the real video card.

Then, I set out to connecting the real video card. The documentation covers this well, but it's still kind of a fiddly process, sending you back to the command line for most of it. The general gist is that you have to explicitly enable IOMMU in general and opt in your device specifically. As a note, I had to enable the flags that the documentation says wouldn't be necessary in recent kernel versions, so keep an eye out for that. Before more specifics, I'll say that my GRUB_CMDLINE_LINUX_DEFAULT line ended up looking like this:

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt intel_iommu=on pcie_acs_override=downstream"

This enables IOMMU in general and for Intel CPUs specifically (the part noted as obsolete in the docs). I'll get to that last bit later. In short, it's an unfortunate concession to my current hardware.

Anyway, back to the process. I went through the instructions, which involved locating the vendor and device identifiers using lspci. For my card (a GeForce 3060), that ended up being "10de:2414" and "01:00.0", respectively. I made a file named /etc/modprobe.d/geforce-passthrough.conf with the following lines (doing a "belt and suspenders" approach to pass through the device and block the drivers, an artifact of troubleshooting):

options vfio-pci ids=10de:2414,01:00.0
blacklist nvidiafb
blacklist nvidia
blacklist nouveau

The host graphics are the integrated Intel graphics on the CPU, so I didn't need to worry about needing the drivers otherwise.

With this set, I was able to reboot, run lspci -nnk again, and see that the GPU was set to use "vfio-pci" as the driver, exactly as needed.

So I went to the VM config, mapped this device, launched the VM, and... everything started crapping out. The OS was still up, but the VM never started, and then no VMs could start, nor could I do anything with the ZFS drive. Looking at the pool listing, I saw that two of the NVMe drives had disappeared from the listing, which was... alarming. I hard-rebooted the system, tried the same thing, and got the same results. I started to worry that the trouble was the PCIe->NVMe adapter I got: the two missing drives were attached to the same new card, and so I thought that it could be that it doesn't work well under pressure. Still, this was odd: booting the VM was far less taxing on those drives specifically than all the work I had done copying files over and working with them, and the fact that it consistently happened when starting it made me think that wasn't related.

That led me to that mildly-unfortunate workaround above. The specific trouble is that PCIe devices are controlled in groups, and my GPU is in the same group as the afflicted PCIe-> NVMe adapter. The general fix for this is to move the cards around, so that they're no longer pooled. However, I only have three PCIe ports of suitable size, two filled with NVMe adapters and one with a video card, so I'm SOL on that front.

This is where the "pcie_acs_override=downstream" kernel flag works. This is something that used to be a special kernel patch, but is present in the stock one nowadays. From what I gather, it's a "don't do this, but it'll work" sort of thing, tricking the kernel into treating same-grouped PCIe devices separately. I think most of the trouble comes in when multiple grouped devices are performing similar tasks (such as two identical video cards), which could lead two OSes to route confusing commands to them. Since the two involved here are wholly distinct, it seems okay. But certainly, I don't love it, and it's something I'll look forward to doing without when it comes time to upgrade the motherboard in this thing.

As a small note, I initially noticed some odd audio trouble when switching away from an active game to a different app within Windows. This seems to be improved by added a dummy audio device to the VM at the Proxmox level, but that could also be a placebo.

But, hackiness aside, it works! I was able to RDP into the VM and install the Nvidia drivers normally. Then, I set up Parsec for low-latency connections, installed some games, and was able to play with basically the same performance I had when Windows was the main OS. Neat! This was one of the main goals, demoting Windows to just VM duty.

Next Steps: Linux Containers and New VMs

Now that I have my vital VMs set up, I have some more work to do for other tasks. A few of these tasks should be doable with Linux Containers, like the VM I had previously used to coordinate cloud backups. Linux Containers - the proper noun - differ from Docker in implementation and idioms, and are closer to FreeBSD jails in practice. Rather than being "immutable image base + mutable volumes" in how you use them, they're more like setting up a lightweight VM, where you pick an OS base and its contents are persistent for the container. My plan is to use this for the backup coordinator and (hopefully) for a direct-install Domino installation I use for some work.

Beyond that, I plan to set up a Linux VM for Docker use. While I could probably hypothetically install Docker on the top-level OS, that gives me the willies for a utility OS. Yes, Proxmox is basically normal old Debian with some additions, but I still figure it's best to keep the installation light on bigger-ticket modifications and, while I'm not too worried about the security implications with my type of use, I don't need to push my luck. I tinkered a little with installing Docker inside a Linux Container, but ran into exactly the sort of hurdles that any search for the concept warn you about. So, sadly, a VM will be the best bet. Fortunately, a slim little Debian VM shouldn't be too much worse than a Container, especially with performance tweaks and memory ballooning.

So, in short: so far, so good. I'll be keeping an eye on the PCIe-passthrough hackiness, and I always have an option to give up and switch to a "Windows 11 host with Hyper-V VMs" setup. Hopefully I won't have to, though, and things seem promising so far. Plus, it's all good experience, regardless.