Since its inception, Darwino has had two-way replication between it and Domino, and it's evolved over the years in fidelity and configurability. Recently, I was able to check an item off the to-do list that I've wanted for a while: "cluster-style" replication from Domino, where a document change immediately kicks off replication to Darwino.

Component #1: Extension Manager

Fortunately, the implementation is straightforward in concept: the Extension Manager has been in there since 4.0 and provides exactly the hooks one would need to implement this. The trouble with the Extension Manager, though, is that you can only subscribe to events from within an ExtMgr addin, and that means native code. You can't just hook into it from, say, an OSGi plugin.

My first thought was to use DOTS, which created this exact sort of bridge years ago. However, it's critically limited: the EM ferrying was never really fleshed out that much, and nor will it ever be, since it's not a supported project. It was kind-of-sort-of supported in the 9.x era for "social" purposes, but those days are behind us. Moreover, its separate OSGi environment wouldn't suit Darwino's needs particularly well.

The DOTS dynamic library, though, could still be potentially useful. Nathan Freeman came across this a couple of years ago: due to the way the DOTS dylib ferries the events from the ExtMgr world to DOTS, the channel is actually consumable by anything, not just DOTS specifically. My initial implementation did exactly this: it fed from the fountain of messages produced by the DOTS dylib for its own ends.

However, the core trouble still remains that DOTS isn't supported as such, and it has too many moving parts for us to want to take on as a dependency. Moreover, the "siphoning" only works if there's only one subscriber listening - on a server that also runs ODA (which Darwino does not use), you have the two consumers contending for messages, which is a recipe for missed events.

So I decided to write a custom-made ExtMgr addin, which would have the advantages of being much smaller and easier to maintain, feeding a different queue, and being a fun learning opportunity for me. The last part isn't as important to Darwino-the-product per se, but I always like when it lines up like that.

Component #2: Message Queues

The way DOTS and this new addin do their things is to use Message Queues, another technology presumably-not-coincidentally added to Domino in R4. The way these work is that you create a named queue (DOTS's, for example, is named MQ$DOTS) and then feed it strings, which are then consumed by anything running in the same Notes/Domino environment by requesting the queue of the same name and waiting for messages. It's pretty simple both in theory and in execution, with the minor problem that the API isn't officially available from Java.

Fortunately for ODA's (and presumably DOTS's) use, though it's not part of the official API, there is a lotus.notes.internal.MessageQueue class that is a (shockingly-thin) wrapper around the MQ* functions in the C API. It's functional, and the thinness of the wrapper means that the method parameters, though unnamed in the bytecode as usual, are clear matches for the equivalent C parameters.

I initially started using this, but ended up writing a nicer wrapper in Darwino's NAPI that implements BlockingQueue, making consumption in Java much clearer.

Component #3: The Listener

The final piece was the most comfortable, since it's entirely back in the warm embrace of Java: I wrote a class to listen for events coming through this queue, extract the database name, check for replicators configured for that NSF, and immediately kick any applicable ones off.

The End Result

Since the overhead of the work in a small replication is so minor, the end result is effectively the same as cluster replication between Domino servers: the change/creation/deletion is propagated over within a couple milliseconds (for small documents, due to, you know, physics). It's pretty satisfying to see in action.

Pub/Sub

If you've been following HCL's announcements lately and feel like this sounds very similar to the pub/sub support they've slated for V11, you're right. My guess is that they wanted to get Elastic Search working with low latency and (kindly and wisely) decided to turn the work required into a nicer interface for the same EM events we're using. It's a good feature and, assuming it's consumable from local Java, would have made my work here easier, but I didn't want to wait, and we target a couple Domino releases back anyway.

Over the years, initially thanks to Peter Tanner's diligent work as the OpenNTF IP Manager and now my own occupation of that seat, I've learned to really appreciate the virtues of dotting your "i"s and crossing your "t"s when it comes to making an open-source project legally clean and clear.

It's definitely something I underrated early on, though - caring about the specific differences between licenses and, in particular, maintaining things like per-file license/copyright headers felt like annoying busywork. For a project that only you will ever use, it technically is, but the hope of open source is that you'll get other people using your work and, ideally, contributing back in turn, and that's when it's important to make sure you have everything sorted out.

Why Bother?

Well, for one, you or your users could theoretically be sued or otherwise legally entangled if you don't keep track of this stuff. Admittedly, it's fairly unlikely, but the consequence of, for example, unknowingly including GPL software in your proprietary product is potentially significant.

It's for that sort of reason that it's important to make sure everything is clean before some large corporations will risk even looking at your project. IBM is particularly good about this because they were significantly burned in the past, and came out of it with extremely-strict view, and that rubbed off on OpenNTF both culturally and with their gracious technical assistance along the way.

And, since large consumers require this sort of vetting, it's also important to know how to do it if you want to contribute code to an open-source organization like OpenNTF, Eclipse, or Apache.

It's also surprisingly satisfying once you get into the swing of it, I've found.

The Example Project

Since I've been spending a lot of time recently with the NSF ODP Tooling project, we'll look at its GitHub repository.

Common Files

There are a couple common features that tend to show up, and which both people and tooling (like GitHub's license identifier) look for:

• The LICENSE file, which is the most critical. This contains the text of the license you're using, as well as one of the declarations of the copyright year (though, admittedly, it's easy to forget to include that part). This is what declares the effective license for the code in the repository that you own the copyright to, and should be included right from the start if possible.

• The NOTICE file, which is vital if you're including any code from any sources not covered under the main copyright. This file should list all of the third-party code you have included in the repository, its license type, and, if possible, where to acquire it. If your project's distributable form includes additional third-party code not included in the repository (such as Maven or npm dependencies), these should be enumerated here as well

  • Writing this file has an important side effect in that it forces you to account for the licenses of your dependencies. More than once, I've run into a situation where I found that a common dependency had an incompatible license (such as the pure GPL). In some cases, this has meant abandoning the dependency outright, while in others it has meant finding a better-licensed alternative. Eclipse Orbit exists in large part for this purpose.

• A legal directory containing any additional license/redistribution information not covered by the NOTICE. This can also sometimes take the form of files like NOTICE-Weld in the root of a project, and is useful for mass-including copyright/notice information from third parties in their original form.

In addition to including these files in the project repository, you should also make sure to include them in any binary distributions you make. In my projects, this takes the shape of inclusions in a Maven Assembly Plugin packaging file.

File Headers

I originally chafed against the idea of per-file copyright/license headers. They're not strictly necessary when the files are included in the original repository, they're redundant, you end up with massive commits touching hundreds of files just to change a year, and they can dwarf the size of the actual code they're copyrighting.

However, I've really come around to the practice of including them, and the main reason is that it makes the files easier for others to copy and use legally. It's one thing when someone finds their way to the root of your repository or distribution package, but it's another when they find an individual class by doing a web search or hitting F3 in Eclipse. In those cases, they can find their way up to the license (assuming your source package includes it), but it's much easier if it's just declared right up front.

It's also easier to clearly distinguish the third-party code you're including. When each file has its copyright information clearly noted, you can easily tell the difference between a sui-generis project file and an included third-party file without having to parse through the NOTICE every time.

And, fortunately, it doesn't have to be a huge hassle to maintain. In each of my Maven projects, I include a license plugin configuration to declare copyright information, any special data types, and which files to not include. Then, whenever I add new files or make a change after the turn of a year, I can run mvn license:format and it'll keep everything tidy for me.

pom.xml Configuration

Maven (and it's not alone in this) provides a lot of pom.xml-level elements to declare all sorts of metadata about your project, like its SCM repository, issue tracker, and, critically, license and developers. I like to declare the inception year, the license, and the <developers> block:

	<inceptionYear>2018</inceptionYear>

	<licenses>
		<license>
			<name>The Apache Software License, Version 2.0</name>
			<url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
		</license>
	</licenses>

	<developers>
		<developer>
			<name>Jesse Gallagher</name>
			<email>jesse@frostillic.us</email>
		</developer>
	</developers>

I use that <inceptionYear/> value as part of the license-header file to keep track of the copyright range, at least when there's contiguous multi-year development.

OSGi Stuff

Since most of my projects are still OSGi, I've been aiming to improve my licensing setup there too. The main place where this comes into play is in feature.xml files, which have required elements to specify the copyright and license. It's not terribly unusual for these to end up as their "[Enter Copyright Description here.]" defaults, but it's important to fill these in. They're included in the "accept the licenses" dialog when installing into Eclipse/Designer, and are available in the "Installed software" descriptions in the UI.

But What License to Use to Begin With?

I'll finish off this post with what is actually the most important part of the process, but which can usually be answered simply. There are a lot of open-source licenses out there, and you could theoretically make up your own, but for our purposes the choice tends to follow some basic rules:

• If you're contributing to an established OS organization, use theirs - for example, if you're contributing to Eclipse, use the EPL.

• If you want your code to be mixed other OS projects and (potentially) proprietary ones, pick Apache or something like it.

  • At OpenNTF, we have a preference for Apache over other similar licenses, because it's well-established and makes copyright handling clearer than the equivalents, something that is critical for large companies. Let past lawyers do your legwork on this one.

• If you want to require that users of your code keep the code open source, consider the GPL.

  • Be extremely wary of this, however: the GPL is intentionally "infectious" and limits how the code can be used. Various projects carve out little exceptions to the GPL to allow use in otherwise-non-GPL products, but it's still something of a minefield.
  • The GPL is one of the approved licenses for OpenNTF, but we kind of discourage it except in cases where a project is GPL because it's derived from previously-GPL'd code.

• If you don't want to be bothered too much by copyright and just want the code out there, consider Public Domain. In practice, it's usually best for you to retain copyright, but explicitly declaring Public Domain is certainly an effective way of allowing any use.

For projects in our community, the quick answer is "use Apache". It's permissive, covers copyright, and is known and trusted by pretty much everyone.

More Work Than It's Worth?

Both the earlier parts of this post and Betteridge's Law contribute to making it clear that my answer is "no, it's not more work than it's worth", but I can certainly see why it'd feel that way. The first couple times I submitted projects to OpenNTF and got a "here's some stuff to fix" email from Peter Tanner, part of me definitely chafed at the whole thing. That can be particularly the case for Notes-based contributions - sometimes, you just want to plunk an NTF on the project page and be done with it, and Notes certainly doesn't have a "wrap this NSF copy in a ZIP with LICENSE and NOTICE files" checkbox.

However, as I learned more about the legal importance of having licenses correct and got more practice at doing this stuff from the start, I started to appreciate the whole process. It also turned out to be really helpful to sort this stuff out on smaller projects before working on larger ones, especially ones with established teams and procedures.

In all, it's worth it both to allow you to contribute to larger projects and, regardless of project size, it's worth it for anyone consuming your code.

IBM and HCL held a webcast today to detail some plans for Notes/Domino V11. There were some interesting tidbits elaborating on things like the pub/sub support, and it'll be worth tracking down a recording of the event when it's available.

What's important for this series, though, is that this event served as the long-promised "roadmap" announcement for XPages. The roadmap is, in effect, option three: HCL plans to look into ways to reuse some existing XPages code, but in general you should be aiming to write your UIs in something else, either consuming REST services from an XPages container or accessing Domino data via another route (like the domino-db Node.js module and hypothetical Java gRPC client).

So we know the end of the path: not XPages. However, it's not like we're all just going to throw away our existing apps, so there's work to do determining how we're going to get there. The options remain pretty much what they were after CollabSphere last year, albeit now with the doubt removed. The first two options - returning to LotusScript or going to Node - have their advantages and disadvantages, and you could make a reasonable case for either. Personally, I'm not interested in going down those roads, though, and I think it's better for any app of reasonable complexity to dive into Java. Other members of the community and I have developed tools over the years to make it easier, and now's the time to take some of these steps if you haven't already.

Do Not Use Server JavaScript

Sever JavaScript was always something of a trap for app architecture. There's nothing inherently wrong with having a scripting language on your UI pages, and it certainly helped bridge some gaps, but the way it and Designer intertwined encouraged developers to create non-portable messes. If you're still writing SSJS, stop immediately.

Learn Proper Java

Java has been around for a long time, and the way to right "good" Java code has changed over time and varies greatly by your environment. Some aspects, though, apply generally, and it's useful to stay up-to-date on current practices. I don't know a better resource for this than Effective Java, which has been updated for Java 7-9 since I last read it.

Speaking of which, you should learn about Java 8 streams and lambdas - they're great. Julian Robichaux did a presentation on this topic back at Connect 2017, and the slide deck is very elucidating.

Adopt Standard Java Technologies

Last year, I created a project to bring some modern JEE technologies to XPages. These are some of the same technologies I've been talking about in my "XPages to Java EE" series and, while that project can't bring the full JEE development experience to XPages, using those tools will help you write code that, in some cases, could be directly dropped into a Java EE app with no modifications at all. There's a big asterisk when it comes to actually accessing Domino data, but that's a solvable problem as well (with some more development).

In particular, you should start writing JAX-RS services. Not only is JAX-RS an excellent and very-capable spec, but REST services are portable to absolutely any front end.

Adopt Automated Builds

Maven has been something of a bugaboo for XPages developers for a while, but doesn't have to be. Node development (server- or client-side) revolves around npm and various build plugins, and Maven is much the same thing. One of the biggest improvements I've made lately to all of my active XPages apps is to wrap the on-disk project for them inside a Maven artifact, using the NSF ODP Tooling. That project allows you to automatically build your NSFs alongside other parts of the project (such as OSGi plugins) without having Designer involved.

Check the example project in that repo, and stay tuned for a 2.0 release (probably) imminently.

Learn Other Toolkits

If you're just starting the process of figuring out what to do after XPages, it doesn't particularly matter which other toolkit you learn, as long as it's reasonably modern. If you take some time to learn how to make, say, a React app but end up going with something else down the line, the lessons you learn will apply very closely. A particularly-comfortable option could be to learn JSF, which has a common ancestry with XPages but has up-to-date capabilities.

Whatever it is, though, just learn some other toolkit.

Follow Channels and Accounts for Other Tech

Over the last couple months, I've started following a lot of Jakarta-related blogs and Twitter luminaries. This applies elsewhere - even if you're not using other toolkits yet, it's very helpful to start immersing yourself in the news and culture.

Don't Stay Still

The primary thing to take to heart is the importance of doing something. Unless you're planning to change careers or retire in the short term, you'll have to make one decision or another. XPages is not going to get meaningfully better, and even existing apps will get worse with time as browsers and technology change.

Other environments, though, are already leagues ahead and are constantly improving. Dive in; the water's fine!

In the introductory post to this series, I started by saying that I've come around to the idea that Java EE (and its kindred technologies) is the future for Java development with Domino, and I think it's worth taking some time to make the case that you should think so too.

A lot of it starts with a nagging question:

Is XPages Dead?

Speaking very strictly, no. XPages remains a component of a commercial product and so bug reports receive attention and fixes as they occur, and presumably all of the many problems the platform is bound to run into as the world evolves will also be addressed.

However, I think the more important question is:

Does It Even Matter If XPages Is Technically Not Dead?

At this point, XPages has, for almost its entire existence inside Domino, been "living" the peculiar kind of undeath that afflicts enterprise software. This is the sort of undeath you could see in, for example, IRIX, which had its last major release in 1998 but was "supported" with maintenance releases for another eight years. XPages has similarly had a few new features since the initial ExtLib release in the 8.5.2 era era - such as the Bootstrap renderkit and some quality-of-life bits - but it's clearly been in maintenance mode for a while. It's been a reactive type of maintenance, too: since XPages doesn't have the Notes-client-app advantage of existing in a controlled environment, XPages development has been increasingly a story of something breaking in some browser and harassing IBM or HCL until there's a fix.

"But wait," you might say, "HCL is going to be in charge now, and look how they're revitalizing the core of Domino! Maybe they'll do the same for XPages!" And, well, maybe they will. I doubt it, though - the core team they brought over isn't involved with XPages, and in general IBM and HCL seem to think so little of it that they don't capitalize it correctly in slides. HCL's development strategy seems to (defensibly) focus on porting the Notes client to other platforms and encouraging developers to either crawl back to LotusScript or to use development stacks that HCL isn't on the hook for, like Node.

And, even if they did staff up to enhance XPages, would it be wise to depend on that? It would still be a single-vendor stack using technologies (OSGi) and idioms (server-side page state) that work but aren't the way the wind is blowing. I would expect that any renewed push would be short-lived.

Okay, Grumpy Gus, Why Is Java EE Any Better?

There was a period in the recent past where Java EE was hitting some similar trouble: Oracle largely lost interest in Java EE (and Java in general, to a lesser extent) and development slowed. However, because so much was open source and (critically) the development pool was so much larger, things kept moving along, with the insurgent group of MicroProfile building new technologies. The transfer of Java EE to Eclipse has proven to be a blessing as well, with the community taking up the mantle splendidly.

As with anything, there's no absolute guarantee of a future path, but JEE has the critical mass that XPages doesn't, with community members and companies emotionally and financially invested in its future. There's no one company whose loss of interest would doom the whole thing, even the Java rump state at Oracle.

And, in the mean time, the technology is just better. You can use current versions of Java, new web technologies are adopted immediately, the IDEs are significantly more stable and featureful, the whole stack has source and Javadoc readily available, the surrounding tooling is better, the modern APIs are simpler and more descriptive, and so forth. Even though XPage's cousin, JSF, has lost some interest, it has evolved significantly past the point where XPages forked away - and is also just one of many good options for app development.

This Is The Bridge We Were Promised

When XPages was ported into Domino, one of the ancillary features for developers was that it would provide a path to the "real" development world, out of our abusive cell of Domino's legacy web stack. It meant being able to consume common Java libraries more practically than in Java agents, extending the capabilities of the platform ourselves, and using development practices that are closer to the rest of the world.

And, to a large extent, these promises came true. We've had the ability to work with the Servlet spec directly, we've been bringing in libraries like Poi with only medium-sized hurdles, and we've been able to (kind of) break the mental binding between the UI and data storage.

Now it's time to truly cross the bridge. There's still a case to be made for writing new XPages applications for now, but every line of XSP markup should be acknowledged as new technical debt. At this point, you should either start working towards Java EE or have a clear plan for how you're going to get there. It doesn't have to be Java EE: there's a lot to be said for taking up HCL on their suggestion to write Node apps, and there are other Java and non-Java frameworks out there that are actively advancing. Regardless of your choice, if you don't choose something, your development will remain stunted and you'll be at a near-guaranteed risk of some critical security problem showing up in an old Dojo version or some other part of the stack and having to clamor for an emergency fix that would otherwise be a one-line pom.xml or package.json tweak (if needed at all).

It's also just very pleasant - give it a try.