Before you dive into OSGi development, you'll need a proper development environment, and that usually means Eclipse. Domino Designer, being Eclipse-based, can technically do the job and has the advantage of some of the setup being done, but it's so out-of-date at this point that it's not worth it (believe me, I tried originally). Newer Eclipse versions have improved noticeably, so it's best to grab a recent version. Currently, that means Mars, or Eclipse 4.5.1.

Before Eclipse, you'll need to install Java if you haven't already. To do that, go to Oracle's site and download the latest release for your platform. Once you install it and find out how many devices run Java, you can move on to installing Eclipse.

Head to the Eclipse download page and download the "Eclipse IDE for Java EE Developers" for your platform. They've recently added an Installer at the top of the page, and that works as well, but adds some new wrinkles that haven't made it into common knowledge yet. For now, you may want to stick with the "...or download an Eclipse Package" section. The "bitness" (32 or 64) doesn't matter too much - you should generally match the bitness of your OS.

Once you have Eclipse installed and running ("installation" generally involves dragging the extracted ZIP file content somewhere), the next step is to teach Eclipse about the XPages artifacts. If you're running on Windows and have a local Domino server, the XPages SDK may be your friend, though it hasn't been fully tested for Mars. If you are running on a non-Windows OS or otherwise don't want to go the full SDK route, there's a quick route that lacks the automation and debugger integration.

Setting up the environment used to be something of a hassle, but it became much simpler ever since IBM released the Update Site for Build Management. Primarily made for Maven-development use (sort of), this Update Site also serves as a quick stop to pick up all of the dependencies you need for XPages development in one place.

To make use of the Update Site, download the file from OpenNTF, extract it, and then extract the UpdateSite.zip file contained therein. You should end up with a folder like this:

In Eclipse, go to the preferences ("Window" → "Preferences" on Windows or "Eclipse" → "Preferences" on a Mac), then go to "Plug-in Development" → "Target Platform":

Edit the existing platform and, in the "Locations" tab of the ensuing dialog, click "Add..." → "Directory" → "Next", browse to that extracted UpdateSite folder, and click "Next" to verify that it finds a bunch of plugins, and then "Finish". Now, you'll be able to reference XPages platform plugins in your projects.

At this point, Eclipse is set up for the essentials, but it may still be a good idea to set up an appropriate JVM for Domino development to make sure you have a consistent baseline and to avoid problems where newer Java releases added classes methods not avilable in Java 6. Fortunately, on Windows (and Linux?), a Notes installation comes with a JVM that does the job nicely. To add it, go back to preferences, then "Java" → "Installed JREs" → "Add...". Choose "Standard VM" → "Next", browse to the "jvm" folder in your Notes (or Domino) installation directory, name it something appropriate, and then click Finish:

On the Mac, you can get a similar Java release from Apple, which nowadays will end up in /Library/Java/JavaVirtualMachines (older installers put it in /System/Library/Java/JavaVirtualMachines).

Now that Eclipse is set up, the next step will be to break ground on making an actual XPages plugin.

OSGi once stood for "Open Services Gateway initiative", but that name slid from "impossibly vague" to "entirely obsolete" rather quickly. For our needs, OSGi is a mechanism for bringing sanity to the "big pile of Jars" that you might otherwise have in a large Java system. It provides a standardized way to describe the name of a library, its version, its dependencies, its capabilities, and its interactions with other libraries. In this way, rather than just "commons-io-2.1.jar", you can conceptually deal with "I require org.apache.commons.io, version 2.0 or above" and the environment can suss out what that means based on what is installed.

As Domino developers, we care about a subset of the OSGi concepts, primarily those related to creating and loading plugins. There is a handful of vital terms involved, some of which I've already been tossing around:

• JAR: a Java Archive file is a ZIP file aimed at Java, generally containing classes and other resources (text files, images, and so forth). There are variants like "War" (Web Application Archive) and "Ear" (Enterprise Archive), but they're all still ZIP files.
• Bundle: A "bundle" is, in effect, OSGi's word for a JAR. It expands on a contained file called META-INF/MANIFEST.MF (so named because Java likes to be difficult sometimes) to define the description of the bundle: its name, version, and so forth.
• Plug-in: This is basically the same thing as a bundle. Presumably, it technically refers to a specialized kind of bundle, but we can use the terms interchangeably. It also technically has that hyphen in it, but I often write it "plugin" anyway.
• Feature: This excessively-vague term is OSGi's way of grouping plugins together into a single conceptual installable unit. For example, the main feature included in the Extension Library references ten distinct plugins.
• Update Site: Update sites are to features what features are to plugins: a way to organize these features into a collection, either to install them all at once or to provide a pool of available software to install. Eclipse-the-IDE uses them extensively for the latter case. As Domino developers, we primarily use them to distribute libraries and install them into the server's Update Site NSF.
• Target Platform: A target platform refers to the set of plugins available in a given environment, which is used to determine whether a given plugin has the dependencies it will need to run. In our case, the target platform is usually the XPages runtime environment, containing OSGi base plugins, Java servlet support plugins, and the XPages runtime plugins themselves (among others). Plugin development environments use these heavily.
• Extension Point: An extension point is a way for a plugin to either declare that it can either provide or consume a given service. For example, you might have a plugin that says "I can make use of any plugin that provides a color" and then another that says "I can provide the color blue". By using extension points, the first plugin can find the other, even though it may have been developed entirely independently.
• Java EE: Java Enterprise Edition (sometimes written J2EE thanks to Java's weird relationship with versions) is basically the standard "web server stuff" for Domino (though it covers other aspects). XPages is sort of like a Java EE environment, but the experience is distinct. Notably, a Java EE server is a general platform on which other web-dev toolkits - JSP, JSF, Vaadin, JAX-RS, and so forth - can run. Java EE isn't inherently related to OSGi, but the two eventually bleed into each other with Domino and WebSphere.

OSGi in general goes beyond that, and some other aspects may be useful down the line (such as interacting with the OSGi console), but for now it's enough to think of it as a set of decorations for your projects. With some of the basic terms in line, next I will move on to setting up an Eclipse environment to start development.

Java has been a bugbear for the Domino community for a long time. Though Notes and Domino have presented a top-to-bottom (mostly) Java environment for years, the monumental inertia of the corporate development community, the platform's tortured history of insufficiently-fleshed-out Java hooks, and IBM's "pay no attention to that man behind the curtain" pitch with regard to the language created an environment where Java is often something for "other developers".

XPages represented a tremendous leap forward for development on the platform, ditching the completely-unique bizarre forms-based web-dev experience in favor of something much closer to modern web development. At the time, the unstructured, SSJS-heavy façade presented by Designer made a sort of sense: the switch to XPages was already a huge change for Domino developers, and making XPage development look sort of like classic Notes/Domino development was a spoonful of sugar with the medicine.

But the critical flaw of XPages development has never been fixed: there's no smooth path from "hello, world" to a well-structured complex application. SSJS, as implemented in XPages, is unsuitable to the task, while Designer's presentation of the Java layer ranges from "non-obvious" to "hostile". Still, IBM did a good job of presenting the next major tier for XPages developers: writing Java first in the NSF and then in OSGi plugins. The problem has always been in figuring out how to get there.

This hurdle started out as an inconvenience just for those who wanted to figure out how the machine worked, but has grown into a significant problem for all Domino developers. While the XPages stack has remained relatively stagnant, the rest of the development world has raced forward, with new technologies giving rise to new frameworks and transforming the old. This is no discredit to the XPages team: they have consistently put in tremendous work across the board, but it's quite an industry to keep up with.

So we're in a tough spot now. The XPages platform isn't going away any time soon, but it's important for us as developers to progress. One option is to abandon ship entirely: pack up and move to some unrelated platform, leaving Domino behind entirely. But most of us, out of personal affection, professional acumen, or (primarily) business requirements, don't want to do that. Unfortunately, the path to improvement with Domino has only gotten more complicated with time. It began with learning about Java, OSGi, and Eclipse proper, but has since expanded to include a whole rogues gallery of other technologies: Maven, Tycho, m2e, Jenkins, Git, Wink/JAX-RS, jQuery, Bootstrap, Angular, and on and on.

There's no time like the present, though. I've walked this path, and I want to help everyone else walk it too. To kick that off, this series is going to cover the process of creating a basic XPages plugin, making it a little more complex, converting it to Maven, and building it with Jenkins.

To kick things off, the next post will provide an introduction to the concept of OSGi and the parts of it that we need to know for Domino development. Things may seem weird along the way, but trust me: it's worthwhile.

The other day, Jesper Kiaer wrote an interesting two-part series of blog posts talking about the security implications of HTTPEnableConnectorHeaders, something I mention in my setup example. In it, he describes the ability to specify an arbitrary username as a security hole, which I don't agree with, but I see where he's coming from. It certainly has, shall we say, implications.

The $WSRU header comes along for the ride when you enable the HTTPEnableConnectorHeaders notes.ini property (off by default) and allows you to specify any user name and have the Domino server trust it completely, as if it had generated it itself. The most straightforward use of this is as a sort of lightweight SSO: if you're already authenticated in to the front-end web server (say, however WebSphere did it when using this), that server can fill in the $WSRU field and have it "count" as a Domino login without forcing the user to log in again. This is sort of what I did to replace my DSAPI filter with a Lua script in nginx.

But what immediately comes to mind is that this allows anyone with HTTP access to the server to type in any ol' username and have Domino accept it. This is why binding Domino's HTTP stack to a local-only (or LAN-only, if you have a nicely-divided server infrastructure) address is important: you wouldn't want a server with this setting facing the public Internet.

The extra good news there is that, when it's fronted by a proxy like nginx, the defaults prevent these $-headers from being passed through from remote clients and all is well, whether or not you specifically account for them.

Now, Jesper also pointed out that, even if you bind HTTP locally, you're still potentially vulnerable to this sort of privilege-escalation problem if you have code running on the server that can make local HTTP calls. And for that case, well... yep. That's true, and it's concerning. However, honestly, that boat already sailed.

Domino's security model has long been lauded by its proponents as thorough and strong, and it is - it's quite good. However, things have gotten a little murkier ever since server-side code has become the primary method of executing Domino apps. Between a Notes client and a Domino server, security is very locked down: the client authenticates via its cryptographic keypair and no lying about its username is going to get the server to treat it as another name (hence why "Run on behalf of" agents don't perform as such when run locally). Running on a server, or between a server and a trusted server ID "client", it becomes a matter of just making up and passing around strings. The permissions used for a given database or view are just whatever the server decided to use as the name when opening it - that's what the hNames parameter of NSFDbOpenExtended is for.

This is how all web user logins work for Domino: the server agrees for itself that, for high-level API calls during that web request, it's going to act like CN=Joe Schmoe/O=SomeOrg instead of CN=Super Trusted Server/O=SomeOrg. It's still the server, though, which is why session.getUserName() in a web-run agent or XPage returns the server name, but session.getEffectiveUserName() returns the current user or run-on-behalf-of value.

The thing is that the hooks to do this aren't any deeper than the HTTP call with $WSRU set - they're all over the place. The XPages runtime provides a very-convenient hook, but that's not new: the C API is where it all starts, and that's reachable readily from LotusScript and Java. Once you have the same sort of access you'd need to make an HTTP call from an agent, the doors are open pretty wide.

But this isn't even really a problem! To run into any situation like this, you have to have code executing on your server, whether it's inside Domino or not. And if you have that in the hands of a real attacker, then, honestly, this "run on behalf of" business isn't what should keep you up at night. Once you have a trusted developer executing malicious code on your server, you're already done.

Before anything else, I should mention that this post is entirely on the topic of building OSGi plugins with Maven. If you're not doing that yet, this probably won't be particularly useful.

For the most part, when building OSGi plugins for XPages, you can be fairly confident that the available plugins will be fairly similar between Notes and Domino. That's not quite always the case, though - there are a set of plugins that are available in Domino that aren't present in Designer's runtime. They're generally physically there in the Notes install, in the osgi folder, but are presumably only loaded when you (inadvisably) fire up the local web preview.

In this situation, you may want to make your XPages library depend on a server plugin, but doing a normal OSGi dependency will cause it to fail to load in Designer. OSGi provides a clean mechanism for this: optional dependencies. If you mark a dependency (on a plugin or on a Java package) as optional, its absence will not prevent the plugin from loading. As long as the code that requires those classes is never run (as would be the case for server code loaded in Designer), you're in the clear.

However, when working on plugins with Maven, I've found it useful in a couple cases to tell Tycho to ignore optional dependencies to prevent it from choking on unimportant cascading dependencies or other issues. The trouble is when you combine these two techniques: now Tycho won't bother loading your optionally-required plugin, and so it won't have the classes available when it goes to compile your code.

A solution to this is to force Tycho to include the plugin in its view of the world regardless of what the MANIFEST.MF files say. This is accomplished in the target-platform-configuration plugin entry that shows up in your average Tycho pom.xml file. If you've started from the same starting point as most, the required section will already be in there: in the configuration, there should be a node named dependency-resolution and then one within that named extraRequirements. This allows you to shoehorn in these types of extra plugins - it's used in the normal case here for the "shim" Notes API plugin to avoid a dependency on Notes.jar. The same can be done for these non-Designer plugins:

<plugin>
	<groupId>org.eclipse.tycho</groupId>
	<artifactId>target-platform-configuration</artifactId>
	<version>${tycho-version}</version>
	<configuration>
		<!-- snip -->
		<dependency-resolution>
			<optionalDependencies>ignore</optionalDependencies>
			
			<extraRequirements>
				<requirement>
					<type>eclipse-plugin</type>
					<id>com.ibm.notes.java.api.win32.linux</id>
					<versionRange>9.0.1</versionRange>
				</requirement>
				<requirement>
					<type>eclipse-plugin</type>
					<id>com.ibm.domino.xsp.bridge.http</id>
					<versionRange>9.0.1</versionRange>
				</requirement>
			</extraRequirements>
		</dependency-resolution>

	</configuration>
</plugin>

Once that's in place, Tycho will include the plugin in its build and will be able to compile properly.

Incidentally, this technique has also proven useful in executing test cases in a setup that makes heavy use of fragments. Tycho won't automatically pick up all fragments when constructing a test environment, but you can force it to include them by adding an extra requirement on a feature that references them. For example:

<!-- snip -->
<extraRequirements>
	<requirement>
		<type>eclipse-feature</type>
		<id>some.parent.feature</id>
		<versionRange>0.0.0</versionRange>
	</requirement>
</extraRequirements>

This is the sort of Maven/OSGi interaction that brings joy to my heart and grey to my hair.

Today, I put two long-overdue releases up on OpenNTF.

First and by-far-foremost is version 2.0.0 of the OpenNTF Domino API. The major version reflects not so much a major new architectural change over the 1.5.x release candidates as it does the fact that those releases were conservatively named and presaged a Java-style "1.x forever" future. Various development builds and release candidates have been used in production by the API team and others for a while now, and so this represents a mature release of changes such as the Maven conversion, revamped auto-recycling, and graph API.

Alongside it, I bumped my own framework up to version 1.1.0 to reflect improved stability and a clean dependency on ODA 2.0.0. I also improved its packaging and created a distributable along the lines of the new ODA version.

So: enjoy!

Over the years, I've accumulated a stable of podcasts I listen to, mostly tech-related (and then mostly Apple-related), and I realized that this may be a handy list for anyone looking to pick up some new listening material. I've uploaded the full list here, but here are some of the highlights:

1. Accidental Tech Podcast - This show is hosted by John Siracusa (of OS X review fame), Marco Arment (of Overcast and Peace fame), and Casey Liss (of, now, ATP fame). It's almost all about technology, primarily Apple, with minor divergences into other topics.
2. Hypercritical - This show ended several years ago, but is worth going back to if you have the time; it's John Siracusa's original podcast.
3. Reconcilable Differences - Rounding out the Siracusa top-heaviness of my list, this podcast is less about tech and more about growing up and outlooks on life.
4. Roundtable Podcast - This one not only doesn't involve John Siracusa, but also isn't about programming. Instead, it's about video games, hosted by a couple people I watch on YouTube and Twitch, and with a slant towards indie games.
5. The Talk Show - Back to form, this is John Gruber's Apple-tech-heavy show, which is a great companion if you follow Daring Fireball.
6. Random Trek - A podcast about a random episode of a Star Trek series each week. That description should tell you quickly whether or not you'd like it.
7. Debug - This is something of a long-form conversation/interview show with a tech personality or two each episode. The episodes with Don Melton and Nitin Ganatra are particularly good.
8. The Incomparable - This one covers geeky culture stuff generally: TV, movies, books, and so forth. I only listen to the episodes where they're talking about something I know, but those episodes are always worth listening to.
9. Developing Perspective - This is a 15-minute-an-episode podcast about being an iOS developer, and is a great window into that world.
10. Quit - This is about improving your career, including the potential benefits and perils of quitting a steady-but-crummy job for something else, and is often quite entertaining to boot.

There are a number of other good ones on my list that I didn't cover, but I didn't want to make the "highlights" summary TOO crazy. If you're interested in getting up to speed with the Apple world in particular, you can't go wrong with either ATP or The Talk Show.

So, Domino administrators: what are your feelings about SSL lately? Do they include, perhaps, stress? It's "oh crap, my servers are broken" season again, and this time the culprit is a change in Apple's operating systems. Fortunately, in this case, the problem isn't as severe as an outright security vulnerability like POODLE and, better still, there is a definitive statement from IBM indicating that they are going to bring their security stack up to snuff almost in time.

But this isn't the first time we've been in this position, nor will it be the last. The focus on cracking and hardening TLS, particularly in the context of HTTPS, is not going to let up any time soon, nor will the basic movement towards encryption everywhere. So I would like to reiterate my stance: Domino is not suitable for direct external exposure via HTTP. The other protocols are problematic as well, but HTTP is the big one and, fortunately, the easiest to solve.

Whenever I've made this exhortation, part of the response I get is that administrators "should not" have to take this step. That Domino should be fully modern in its security stack or, at least, that IBM should handle this problem for them in one way or another. Or that one of Domino's traditional strengths is its all-on-one nature, with a single easy installation that takes care of everything, and that installing a separate web server is a complicated step that administrators shouldn't have to take.

Well... tough.

The promise of an integrated server system that took care of everything is a great promise, but it's always been extremely difficult to achieve, even for a platform firing on all cylinders. No matter the ideal, Domino does not perform at this level, and I still maintain that it should not need to. Outside of Domino and PHP, the application server is not generally expected to also be a full-fledged front-end web server, for exactly this sort of reason. Domino's job with respect to the web is to generate and serve up HTML, JSON, and other content; it's something else's job to make sure that that leaves your company's network securely.

If you still maintain that this should be Domino's job due to how much you pay for licensing, then that's a conversation between you and your IBM sales rep. I, though, am entirely fine with a paid-for app server not covering this ground, and that's in large part because the products that do perform this task are superb and often open-source.

These other products – nginx, Apache, HAProxy, and so forth – are made for this job. This flurry of SSL/TLS features and bugs you've been hearing about? These are all implemented or fixed in dedicated products, sometimes years before they come to your attention. And when new problems crop up, they're fixed and talked about immediately across the web, with guides for what to do appearing as soon as the problem arises.

Is it easier to continue using Domino HTTP directly than to set up a reverse proxy? Sure! Well, sort of, when there's not an active disaster to mitigate. And, much like how keeping an XPages (or other web) app up to spec and working on all target devices is more complicated than a legacy Notes app, sometimes that's just how the world goes. Deciding that it's complexity you don't want, or that your company's policy doesn't allow for an additional server, is not a tenable stance. Unless you're Apple, your company's policy will not bend the arc of the industry.

So, I implore you, at least give this kind of setup a real look and a trial run. I think you'll find that the basic setup is not dramatically more complicated than just Domino alone and will also open the door to new non-security features like improving page load speeds on the fly. If you want, with eyes open, to maintain an externally-facing Domino HTTP stack, that's fine, but I'll see you when the next security apocalypse comes around.

When developing an XPages application of beyond-minimal complexity, you're likely to run into a problem where your app starts saying that a class is incompatible with itself in one way or another. The exception usually traces down to something like "foo.SomeClass is incompatible with foo.SomeClass" or "cannot assign instance of foo.SomeClass to field X..." where the field is that same class. This has cropped up since time immemorial.

It's actually, though, something that IBM sort-of fixed in 8.5.3 by adding an xsp.properties option of xsp.application.forcefullrefresh=true and then, in 9.0, a GUI option in Xsp Properties:

Basically, this checkbox amounts to "don't break my app periodically". From what I gather, the default behavior is in the interest of being clever with classloaders, but can lead to creeping problems in complicated apps, to the point where changing seemingly-innocuous things like the ACL breaks the app until you restart HTTP or "kick" the app by modifying certain design elements (namely, Java classes or faces-config.xml). Since that behavior is never desired, there is no reason to not check this box, and I enable it on every new NSF I create.

This post is partly to spread information publicly and partly a useful note to my future self for the next time I run into this trouble.

In OGSi, the primary type of entity you're dealing with is a "Bundle" or "Plug-in" (the two terms are effectively the same for our needs). However, there's a sort of specialized type that you may run into called a "Fragment". They're similar to a plug-in in that they're a contained unit of Java code and resources, but they have the special property that they're attached to another plug-in and automatically come along for the ride when the main plug-in is used. This is useful in a couple situations, such as code organization, serving platform-specialized native libraries, after-the-fact additions, or providing library dependencies.

In the basic case, the only requirement is to specify in the fragment what the "parent" plug-in is (Eclipse provides a field for this in its editor) and then including the fragment in the installable feature alongside the plug-in. However, there are a few situations where a bit more work is required if you want to access the classes in the fragment: when used as part of a Tycho build and when used as an XSP Library in Designer (which may also apply to Eclipse dependency use generally).

Tycho

When doing a full Tycho build, even if both the plug-in and its fragment(s) are part of the current build, another project won't automatically include the fragment when doing the compilation. This can lead to a situation where the projects will compile cleanly in Eclipse (which handles the fragment attachment) but fail in Tycho. The trick, though small, is non-obvious: you have to tell the project that is using the fragment code about the fragment in its build.properties.

So say you have three projects: the main plug-in (some.main.plugin), a fragment attached to it (some.main.plugin.fragment), and the project consuming them (some.dependent.plugin). The normal first step is to include the main plug-in in the dependent plug-in's MANIFEST.MF as usual:

Require-Bundle: some.main.plugin

In Eclipse, this will suffice: both the main plug-in and its fragment will show up in the "Plug-in Dependencies" library. For Tycho, though, you have to tip it off using a line like this in build.properties:

extra.. = platform:/fragment/some.main.plugin.fragment

Think of this as saying "hey, dummy, don't forget about the fragment". Once you have that line, the Tycho-enabled Maven build should be able to resolve the fragment's classes and all will be well.

Designer

When using the plug-in and its fragment in an XSP Library in Designer, there's a similar-seeming problem: though Designer will include any direct dependencies of your Library plug-in in the class path, it won't pick up on any fragments by default (though it seems that Domino does). The trick here is that the primary plug-in has to tell Designer that it accepts fragments, which is done by setting Eclipse-ExtensibleAPI in the MANIFEST.MF file for some.main.plugin, like so:

Eclipse-ExtensibleAPI: true

Once that's in place, the fragment should start showing up in your NSF's classpath when the library is enabled.