The second project I took on this past week was related to the first, and also relates to my ongoing struggles with Tycho.

While working on the NSF ODP Tooling, I figured that it could be a good candidate to move away from Tycho and to maven-bundle-plugin or bnd directly. Since I've been Mavenizing the Domino OSGi bundles for a good while now, and the tooling doesn't have any OSGi-dependent tests in it, it seemed like it could go smoothly. Unlike historical precedent, though, my enemy in this endeavor wasn't Domino, but rather Eclipse.

Repository Layouts

One of the big things that makes working with OSGi bundles - at least specifically ones in the Eclipse style - difficult with Maven is that they're generally provided using a repository layout called "p2". This is the evolution of the "site.xml" Update Site style and shares a lot of characteristics. In fact, a p2 repository will often have a "site.xml" file alongside its "artifacts.xml" and "contents.xml" (often Jar'd up) to provide backwards compatibility. It's how we package up XPages plugins and how once upon a time IBM provided [the XPages artifacts for Tycho use](https://openntf.org/main.nsf/project.xsp?r=project/IBM Domino Update Site for Build Management). As a live example, Eclipse 2019-12 is distributed via such a repository.

Maven has its own repository layout, variously called "Maven", "Maven2", "m2", or just "default". This serves a similar purpose, but is structured differently - whereas p2 just has the repo and its "features" and "plugins" directory (and, potentially, composite repositories) - Maven's repository system is organized like a conceptual folder tree based on translating a Group ID (like, say, "org.openntf.maven") into a successive series of subdirectories (like "org/openntf/maven"), followed by a directory for an Artifact ID, which in turn contains directories for each version, and finally within there are any of the actual files that make up a given named "artifact". As a live example, Maven Central is browsable in this manner.

Translating Between Them

Though the two layouts share a common core job - hosting Jar files (mainly) - they diverge enough in how the tools expect the metadata to be laid out that they're difficult to mesh. Tools like bnd can often work with whatever, and even Tycho can try to find OSGi bundles via Maven dependencies, but it's not smooth.

Over time, a pseudo-standard of adding p2 repositories to Maven has emerged, but it's only actually used as a marker to pass along to true Eclipse tools. The most common in our sphere is this construct, seen in projects like ODA:

1
2
3
4
5

<repository>
	<id>notes</id>
	<layout>p2</layout>
	<url>${notes-platform}</url>
</repository>

There, you reference the XPages update site somewhere, and then Tycho can use that to resolve dependencies for things like Require-Bundle: com.ibm.xsp.core. However, it's used only for Tycho's specific OSGi needs. You can't bring in the Tycho plugins and then have a non-Tycho module in your tree declare a dependency like that. Tycho has an implementation class for this, but it's intentionally stubbed out.

My Needs

The reason why tools that can work with both are so heavily slanted to the specific task of generating a true OSGi environment is that that's usually what you want. If you're designing, say, an Eclipse plugin or Eclipse-derived product, you want all of your tooling to know about OSGi from top to bottom, and that's where Tycho excels. It makes sure that all of your dependencies are correct and everything is OSGi-friendly.

This is as opposed to something like maven-bundle-plugin, which is most typically used to put an OSGi coat of paint on a project that isn't primarily geared towards OSGi.

I kind of want a middle ground, though. A project like the NSF ODP Tooling has grown into a sprawling hydra, with heads for Maven, Eclipse, Domino, and now Visual Studio Code. While that works, putting Tycho at the front of it ends up feeling needlessly proscriptive, and I'd love to toss it aside. However, I was blocked in my desire by a small thing: though Eclipse publishes their core bundles on Maven nowadays, Wild Web Developer is currently p2-only.

The Project

So I set out to solve this problem for myself and learn something in the process. As indicated by the <layout>p2</layout> option on the repository above, Maven's repository system is intended to be extensible. Unfortunately, it seems like it hasn't been extended particularly often in practice, and most of what I could find about it was that it's possible to do, but only via references to people saying that one could.

Fortunately, it turns out that it's actually not too difficult to implement after all, and I did just that.

What this Maven plugin does is allow you to specify p2 repositories in any old Maven project, using the ID of the repository you add as the Group ID of dependencies and then the Symbolic Name as the artifact ID. Now, with the plugin added to the project, I'm able to reference the p2-only Wild Web Developer artifact I need:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

<repositories>
	<repository>
		<id>org.eclipse.wildwebdeveloper</id>
		<url>https://download.eclipse.org/wildwebdeveloper/releases/0.8.0/</url>
		<layout>p2</layout>
	</repository>
</repositories>

<dependencies>
	<dependency>
		<groupId>org.eclipse.wildwebdeveloper</groupId>
		<artifactId>org.eclipse.wildwebdeveloper.xml</artifactId>
		<version>[0.5.0,)</version>
	</dependency>
</dependencies>

And, just like that, the bundle and its explicit dependencies show up in my Maven Dependencies group in Eclipse:

As a side bonus, this obviates the need for the mavenizeBundles half of the generate-domino-update-site project, since now I can just reference the generated site directly and get the dependencies, including with better behavior for embedded Jars than I had there:

The Tiny Details

I think that this plugin is about where it needs to be to suit my needs, but there is still an array of [tiny details I've yet to contend with](https://github.com/OpenNTF/p2-layout-provider/issues?q=is:open is:issue). It'll never be quite a perfect match for an arbitrary OSGi bundle (though some also contain useful Maven metadata), and so there will always be rough edges with something like this, but I think that it will solve a lot of headaches I'd otherwise have to deal with down the line.

If you think it'd be useful for your projects, take a look and let me know if you run into any trouble.

The last couple weeks of the year are always a good time to work on some side projects or small utilities to scratch an itch, and this year definitely ended up that way, seeing me work on a couple interesting things over this Christmas week.

The Project

The first of these is an enhancement to the NSF ODP Tooling project. Among the various components that I've put in there over time is an Eclipse content assistance plugin that provides some autocomplete capabilities when working with XPages and Custom Controls within an ODP. Specifically, it knows about the stock and ExtLib controls that ship with Domino, as well as any Custom Controls within the same project, and it allows Eclipse to provide completion suggestions. The code in there is actually hairier than you might think, and that's because it's building on top of Eclipse's generic text completion system, with only a little assistance from an existing implementation I cribbed. Still, it does the job.

The Problem

However, most of my XPages development nowadays takes place first outside Domino and then only ends up back on Domino when I want to make sure it works.

The trouble there is that the content assistance plugin I wrote is tied to both the ODP nature (an Eclipse-ism meaning that it knows a given project is associated with a set of capabilities) and the specific layout of an on-disk project.

The Quick Route

I originally set out to just loosen up that association a bit - take the existing plugin and allow it to work with any .xsp file and to try to find custom controls in a more webapp-type layout of a project.

However, I figured this was a good option to look beyond that. Though the plugin does indeed do what I want, the trouble is that it's thoroughly tied to Eclipse specifically. All of the classes use Eclipse core and XML tooling classes extensively and none of that would be portable to any other IDE. I figured this would be a perfect time to jump into the world of the Language Server Protocol.

Aside: What is LSP?

The Language Server Protocol is a standard for a way to provide IDE-type support in a way that's not dependent on any specific IDE. It grew out of Visual Studio Code and is gradually seeping its way across the whole development landscape.

It allows for the specifics of handling a language - checking validity, resolving classes and other entities, identifying keywords, and so forth - to be separated from the IDE used for editing it. Using LSP, if an IDE wants to support editing, for example, JavaScript, its creator no longer needs to create and maintain a tool to handle all of the intricacies and changing rules of the language - instead, it can bring in the LSP implementation for it and then focus just on the specifics of what the IDE does differently from others.

Additionally, this decoupling means that the LSP implementation doesn't even have to be written in the same language as the IDE using it, and are often just written in the same language that they're implementing. If you look across the list of implementations, you can see that the Swift server is written in Swift, the Ruby one in Ruby, and the Java one is actually Eclipse's Java tooling extracted from the IDE.

Wild Web Developer

Since Eclipse long predates LSP, it's historically had its own implementations for any languages it supports. Though it's had enough clout to support a lot of languages, some of them have trailed behind. Like, really far behind. Prime among these have been its web-language-related editors, which do an okay-enough job editing basic HTML, CSS, and JavaScript, but pretty much missed the boat on newer features, transpiled languages like TypeScript, and project structures like NPM. While there have long been plugins for Angular and TypeScript, they never fully kept up and the whole thing ended up falling far, far behind other IDEs like VS Code.

Enter Wild Web Developer, the whimsically-named project to bring the fruits of the LSP development to bolster Eclipse's web-tech support. Though it's named for its web language implementations, what it really is is a combination of two things: a generic text editor backed by a small array of LSP implementations and a syntax-coloring system derived from TextMate, which itself became a pseudo-standard for syntax coloring.

LSP4XML

That brings us to the piece that ties together LSPs and my immediate desires: LSP4XML, the most-popular XML Language Server implementation, which is used by both VS Code and Eclipse, and just so happens to be written in Java and is designed to be extended.

Since LSP4XML is so smoothly extensible and Wild Web Developer just added a way to contribute these extensions in Eclipse, that meant I could accomplish what I want without having to worry about writing a whole LSP implementation just to support XSP and DXL.

XSP Completion Participant

Contributing to an LSP4XML server involves creating an extension class that then registers the individual capabilities you want to provide.

In this case, I contributed an ICompletionParticipant implementation. ICompletionParticipant has a delightfully-straightforward API, and all you have to do is provide tag, content, and attribute suggestions based on the context the user's cursor is in.

With this simpler API, I was able to significantly refactor down my earlier implementation, making it much more readable and focused.

DXL Schema Contributor

The other piece of XML completion that I added to the NSF ODP Tooling was to provide the (blessedly-redistributable) DXL schemas that ship with Domino to Eclipse. Unlike the XSP completion assistant, this plugin is entirely code-free, consisting solely of the schema file itself and an extension contribution in plugin.xml. The reason this works is that each DXL file declares its XML namespace at the top of the file, and so I can tell Eclipse to look for the schema file I'm providing when editing DXL.

LSP4XML also provides a way to provide schema files, but it's a little more complicated, involving a resolver class implementation. The idea is the same, though, mapping the namespace to the DTD file.

In both cases, the standardized and descriptive nature of XML schemas means that merely providing them to the IDE allows for all sorts of code assistance, even down to the level of suggesting and validating attribute values. It's pretty great.

Side Benefits

Making this switch to LSP4XML accomplished my original goal: by changing the XSP handling in the NSF ODP Tooling for Eclipse, I switched over to the Wild Web Developer editor and got editing in .xsp files anywhere (and a bit snappier to boot, since it's inherently heavily multithreaded).

But, like I mentioned, Language Servers are used across IDEs, most notably Visual Studio Code. Thanks to VS Code's equivalent LSP4XML extension mechanism, I was able to contribute the same extensions used for Eclipse there, and get the same type of results. That's a far cry from being able to get all of the NSF ODP Tooling capabilities outside of Eclipse, but it's a big start.

The Next Version

Currently, these additions are just in the develop branch of the tooling and haven't made their way to a proper release yet, but they've proven themselves so far in my use. My hope is to make a few more improvements, get the VS Code extension into shape, and make it part of a "3.0" release of the Tooling.