I gave two presentations at MWLUG 2013 in Indianapolis, and here are the slide decks and resources for those:

AD110: Write Cleaner, More-Manageable XPages Applications With Java

• Slide Deck
• My blog template on GitHub
• My portfolio template on GitHub - I didn't actually go over this, but it was original plan: it provides a simple example of using controllers and my new model classes
• XPages Scaffolding on GitHub - this is the template I use when creating each new project and is the proper home of my controller and model classes

AD108: The org.openntf.domino API: Modernized Domino Programming for Java and XPages

• Slide Deck
• The API on GitHub
• Nathan Freeman's webinar code snippets
• Declan Lynch's hosted Javadoc
• Paul Withers's Demo Database
• Paul Withers's JAR-deploying Update Site

Last time, I said that my next post would delve into operations and comparisons in EL, but I figured first it would be good to have a real-use example of what I talked about last time.

In several of my apps lately, I've taken to using properties files to store app configuration, primarily the server and file paths for the databases that store the actual data. The standard way of doing this is to use the platform's built-in bundle-handling abilities. However, that'd leave me stuck with either putting the bundle reference in the XSP markup somewhere or in a theme - the former would muddy things up and the latter is not available at page-load time. So instead, I wrote a quick class to use as an application-scoped bean, making it available everywhere via EL and Java:

package config;

import java.io.IOException;
import java.io.Serializable;
import java.util.Locale;
import java.util.ResourceBundle;
import javax.faces.context.FacesContext;
import com.ibm.xsp.application.ApplicationEx;
import com.ibm.xsp.model.DataObject;

public class AppConfig implements Serializable, DataObject {
	private static final long serialVersionUID = -5125445506949601097L;

	private transient ResourceBundle config;

	public Class<?> getType(final Object keyObject) {
		return String.class;
	}

	public Object getValue(final Object keyObject) {
		if(!(keyObject instanceof String)) { throw new IllegalArgumentException(); }
		return getConfig().getString((String)keyObject);
	}

	public boolean isReadOnly(final Object keyObject) { return true; }

	public void setValue(final Object keyObject, final Object value) { }

	private ResourceBundle getConfig() {
		if (config == null) {
			try {
				ApplicationEx app = (ApplicationEx) FacesContext.getCurrentInstance().getApplication();
				config = app.getResourceBundle("config", Locale.getDefault());
			} catch (IOException ioe) {
				ioe.printStackTrace();
			}
		}
		return config;
	}
}

This is an example of a simple read-only DataObject implementation. There are only four methods to implement: getType, which returns the class of the value for the key; getValue, which returns the actual value; isReadOnly, which determines whether a given key is settable (always false here); and setValue, which would set the value for the key if I wasn't making it read-only.

It's pretty straightforward to take this idea and apply it to any of your Map-like objects to ease their use in EL without having to implement or stub-out the dozen or so methods demanded by the actual Map interface.

One of the main aspects to cleaning up the code of an XPage is the heavy use of Expression Language (EL), which is XPages/JSF's shorthand language for binding properties to values and methods. The most common kind you'll run across is the kind you get for "free" when working with document data sources:

<xp:inputText value="#{doc.FirstName}"/>

This is the essence of EL in a nutshell: the actual code behind the scenes can be pretty complex, but EL handles that for you. It goes far beyond this, though, in two important ways: accessing custom and non-Domino data sources and writing complex operations.

Accessing Custom Data Sources

In addition to accessing "built-in" data types like Domino documents and the scope variables, EL is generic enough to deal with several kinds of data access (technically, it ONLY does these, and the standard bindings are examples of it):

• Lists (and probably arrays). EL can use bracketed-subscript notation to reference individual elements of a Java list/array, e.g. #{names[3]}. I've found this to be the rarest use, but it could be useful if repeating over two parallel lists.
• Maps. This is how EL accesses the scope variables (requestScope, viewScope, etc.). It translates code like #{sessionScope.cart} to sessionScope.get("cart") and sessionScope.put("cart", ...).
• DataObjects. This is basically a "Map-lite" interface from IBM that is meant for making generic key/value objects easily-accessible via EL - this is what Domino document objects implement. It translates code like #{doc.FirstName} to doc.getValue("FirstName") and doc.setValue("FirstName", ...).
• Generic bean-style methods. If (and only if) your object doesn't implement any of the above interfaces, EL falls back to seeking out bean-style getters and setters. For example, if you have a class that implements no special interfaces, EL will attempt to match #{myObject.foo} to myObject.getFoo() (or myObject.isFoo()) and myObject.setFoo(...).

The first important thing to remember is that EL applies these rules to your own and third-party objects just as much as it does to IBM's stock objects. If your code is just #{doc.foo}, you can switch "doc" from a Domino document to a HashMap, a custom DataObject class, a bean with getFoo, or an entirely-different third-party data source that implements one of those and it will continue to work smoothly. If you instead wrote it as #{javascript:doc.getItemValueString('foo')}, your experience would be... a bit rougher.

The second important thing to remember is EL's equivalence between dot and bracket notations. What I mean by this is that these three are all equivalent (assuming "barHolder" is a variable on the page containing the string "bar"):

• #{foo.bar}
• #{foo['bar']} or #{foo["bar"]}
• #{foo[barHolder]}

Note that those are equivalent for every supported object type, including generic beans (okay, but not Lists, because #{foo.1} is illegal). This equivalence allows you to use EL to reference keys that would not be legal in dot notation (e.g. #{someProperties['us.iksg.setting']}) and to use other EL expressions to determine the field name. Say you're keeping some user-specific info in the application scope; you could end up with a chain like this to access it via EL:

<xp:text value="#{applicationScope.userCache[context.user.name].ticketCount}"/>

The more you understand the different ways of accessing your objects via EL, the more you can remove SSJS from your XPage markup and make your apps cleaner, (probably) faster, and more portable.

In my next post, I will cover the second major topic: complex operations and comparisons in EL.

Since my initial tinkering with Couchbase, I've been idly thinking about improvements to the data sources and potential uses I may have for it.

As for the latter, I think I may give a shot to implementing IKSG's new project-tracking system in it. I'll lose out on reader and author fields, but otherwise it'd be a pretty direct translation between NSF or Couchbase data stores.

For the former, I decided to drop my development code in a Github repository and start tracking my to-dos there:

https://github.com/jesse-gallagher/Couchbase-Data-for-XPages

The test database stored in that repository is the same as the one from my previous post with two main exceptions:

1. I added a new parameter to XSPCouchbaseConnection to specify whether or not it should, when acting as a DataObject, treat the bucket as a document store. Since "documents" in Couchbase are just keys with JSON strings as their values, it's a tricky matter to detect intent... thus, I decided I may as well just make it a config option.
2. I added a second example connection in faces-config.xml: clusterScope. Since the Java API automatically serializes and deserializes objects to store in the bucket, it makes for a perfect backing store for a clustered XPage app's persistent clusterScope.

I can't say how much time I'll have to dedicate to the improvements I want to make in the short term, but I think even the current form should be enough for me to build apps upon. I'm very interested to see if I can come up with tricks to use that are more difficult or are very inefficient in an NSF.

The other day, I attended a Couchbase Developer Day in Philly. If you're not familiar with Couchbase, it's one of the young crop of NoSQL databases, and more specifically a document database and a conceptual cousin to Domino by way of Damien Katz. It's this similarity that makes it so interesting to me - Couchbase's still-living predecessor CouchDB started life as Notes built from the ground up for the web and the heritage is clear. Couchbase itself ends up being like if you melted down the cores of NSF/NIF and poured the result on top of a speedy key-value store.

That all makes Couchbase a perfect candidate for some XPage tinkering. Ignoring the foundational key-value aspects of Couchbase, the JSON document storage and views in 2.0 make for perfect analogues to the Domino data sources in XPages. To wit:

<xp:this.data>
    <cb:couchbaseView var="beersView" connectionName="couchbasePelias"
        designDoc="beer" viewName="by_name" />
    <cb:couchbaseDocument var="beer" connectionName="couchbasePelias"
        documentId="${param.documentId}"/>
</xp:this.data>

I've set up a demo app on one of my test servers (don't be surprised if that link is periodically unavailable). It demonstrates one of the views in the beer sample database that comes with Couchbase (it's like fakenames.nsf, but more intoxicating), with each row providing a link to edit the document in the DB, though editing is disabled for Anonymous users. The app also contains pages to show the applicable Java classes and support elements. The sources are nowhere near complete works - the view source lacks any knowledge of keys, grouping, etc., while the document source doesn't allow for the creation of new documents, nor is there any representation of non-document values.

Pretty much every concept you work with in Couchbase has a corresponding concept in Domino, but with a bit of slant, like a different dialect (for example, they call it a Royale with Cheese). A table may suit this explanation best:

I'm tempted to find an excuse to build something using Couchbase as a back-end, just to see how its performance stacks up. It seems like my years of built-up NSF habits would serve me well, as many of the programming models and tradeoffs are similar, except that Couchbase views are probably consistently fast without having to cheat.

If I come back to the data sources, I'll probably flesh them out to support creating new documents, properly handle dynamically-computed properties, provide some control over view parameters (maybe including the geoboxing stuff built in), support Couchbase's document-save-conflict detection, and add a data source for basic key/value pairs. And if I do, that could provide a good excuse for me to package them up as a proper OSGi plugin, which I've so far avoided bothering with. We shall see.

1. This is a lie. The Couchbase way is actually very similar to Domino: Domino view design notes contain one or more collations, which correspond to the main index and any click-to-sort column directions. However, each of Couchbase's views inside its design document can produce wildly different data, not just resorts of the same selection, and so the fact that they're similar technically isn't very useful.

Yesterday, I suggested that everyone enable finalizing parameters in their code cleanups and, better, to do the same in their Java save actions; later, I set Eclipse loose on the org.openntf.domino API to implement it there. Adding final keywords to method parameters (and elsewhere where possible) is one of the habits I picked up from reading Effective Java a bit ago, but it's kind of a subtle thing and the benefits aren't immediately obvious.

If you're not at all familiar with the final keyword, it has a couple meanings in Java - in the context of variable and parameter declarations, it means that you can't reassign the value of the variable after it's been assigned the first time. So, for example, this is illegal and will generate a compile error:

final int i = 1;
i = 2;

In that example, it seems like a "why the heck would I do that?" sort of thing, but it's not too much of a stretch to picture a block of code involving something like this:

void doSomething(Document doc) {
	System.out.println("doc is " + doc.getUniversalID());
	
	// a bunch of code
	doc = database.getDocumentByID("FFFF0002");
	// a bunch more code
	
	System.out.println("doc is " + doc.getUniversalID());
}

That will WORK and it won't damage the original doc that was passed in, but it'll get confusing, particularly the more lines of code or eyeballs involved. Functioning or not, it's poor code style, and changing the method signature to doSomething(final Document doc) means that the compiler will enforce good style.

In addition to being good style, final has some technical benefits. First off, it might be faster. Secondly, if you ever use anonymous classes, final variables are a necessity for accessing information in the surrounding block. If you get into the habit of finalizing your variables, you'll already be ready by the time you have a need for anonymous classes.

final is a component of a larger and more important topic: immutability. I should probably write a lengthy post on immutability in the future, but for now there's an important thing to remember: final does not make your objects immutable. It only prevents assignment - it doesn't prevent calling methods on an object that change the object's value (in fact, Java lacks any language construct or cultural convention for immutability). So even if your Document variable is declared final, you (or any code you pass it to) can still call doc.replaceItemValue("Form", "Ha ha"); with impunity.

I'm always looking to improve my programming/development/administration methods, and in the past couple months I've made a few more changes. To my chagrin, these changes are mostly things that I should have been doing for a while now and I've been sloppy to do them differently, but hey - fixing a problem is still improvement!

Serious About Source Control

I've been using source control for a while now, and I hope everyone else has as well. However, I've been lax in a couple ways, which bit me a while ago. For example, I had a couple actively-used projects tied to source control in Designer - a few NSFs, a few NTFs. That was fine NORMALLY, but then this sequence of events happened:

• I run Designer in a Parallels and have my repositories stored Mac-side, pointed to via a drive-letter mapping to my home folder
• I Remote-Desktop'ed into my Windows VM from another machine
• Remote Desktop remapped the drive letters to share local resources
• Designer, which was running at the time, could no longer find the repository folders
• Apparently, Designer/eGit/whatever decided that the rational thing to do was to "sync" the NSFs with the now-missing repositories... which is to say, delete all design elements from the NSFs

Imagine my surprise when a previously-fine DB I hadn't intentionally touched in a while was suddenly naked, consisting of a single view that somehow survived the purge. Fortunately, the nature of the problem meant that the design was still stored in the repository and thus recoverable, but that's still not a problem you want to have.

So I've been shifting my development process another few steps closer to the ideal. The way I see it, that ideal is:

• Never open a production DB in Designer
• Do all development in entirely-distinct NSFs created either from scratch (for a new project) or via the "Associate With New NSF" action from the source-controled on-disk project. These NSFs should be worked on by a single person and should either be in an individual-user dev folder on the server or, better, on a local development server
• Create deployment "builds" as NTFs, either as design-only copies of a source-associated NSF or, again, from the "New NSF" action from an on-disk project (but it should then be disassociated immediately)
• "Deploy" the "builds" using Replace Design on the production NSF/NTF from the Notes client or, better, Administrator
• Ideally, the "build" NSFs should closely match a tag or branch in the source repository, but I can't be bothered to do that for my needs yet

That process goes the same for "classic" and XPages apps. Since source control became viable for Domino devs around the same time as XPages hit the scene, the two are often conceptually tied together, but in my experience the classic binary-DXL handling is plenty good enough to use it in projects without a line of Java or XSP.

Separate Application and Data

The standard way to develop a Notes app - classic or XPages - is to bundle the design and data together. This has historically had a lot of benefits and still makes sense in some cases (say, customizing individual mail files for different uses). However, particularly in the case of a standalone "app" (a help-desk application, a project-tracking system, etc.), this tight joining makes little sense and would be considered a bizarre aberration by anyone using most other systems.

Fortunately, as Domino developers, even if we split up the design and data, we still have it easier than most, since we don't have to worry about drivers and authentication credentials for the back-end DB (unless we want to). In the simplest case, which I've adopted, you have your "app" NSF and a "data" NSF next to it. The "app" NSF contains no forms or views, which the "data" NSF contains no XPages, Java, or other "executable" entities. I've derived a couple benefits from this already:

• The app and data NSFs can have different ACLs
• The data NSF can be set to disallow URL open. This means you don't have to worry about, say, clever users trying ?ReadViewEntries to peek into more than they should see
• You can easily switch between "dev", "production", and "test" data NSFs, just like normal developers do. You don't have to worry about copying data around between different dev versions of your app
• Multiple apps can point to the same data with impunity
• The data NSF can exist on another server... or cluster of servers, which you could switch between programmatically
• Data sharding becomes easier. Depending on the type of app, each user/project/department/whatever could have a different backing NSF fronted by the same single app

Admittedly, this makes some things moderately more complicated. Any Domino data sources will need to have a databaseName property specified, which will need to get its value from somewhere (worst case, you could hard-code it to start with, then "upgrade" to a managed bean). You also won't be able to use "app.nsf/view/key"-style URLs, since the app won't have any views to search by - you'll have to use query parameters or cobble together your own good-looking URLs with the assistance of Web Rules. You might not want to make your structure too document-focused anyway.

These changes have been bringing my development closer to classic desktop-app development, where these are less "techniques" and more requirements due to the nature of development, but they've proven to be valuable. Domino's ease of getting started and fast-and-loose development methods have distinct advantages, but it's always important to question which ones are better than other environments and which are traps.

A while ago, I set up Apache as a reverse proxy in front of Domino. Initially, that was in order to serve a WordPress site on the same server, but since then it has more than proven its worth in other ways, mostly in the areas of fault tolerance and SSL support.

IBM recognizes those benefits, which is why Domino 9 on Windows comes bundled with IHS; however, I am a responsible admin, so I don't run Windows on my main servers. Fortunately, stock Apache does the job nicely... or it does usually. Filled with enthusiasm for this kind of setup, I rolled out a small Linode VM dedicated entirely to running Apache as a proxy (sort of like their NodeBalancers, but manual). Unfortunately, I started running into a problem where sometimes sites fronted by it (such as this one) would not properly include their host information and would instead load the main I Know Some Guys site, which is the default on Domino. I wasn't able to find a fix that actually worked, so I decided to use that as an excuse to switch to a cute little number named Nginx.

So far, my experience with Nginx has been fantastic. The config files are like a cleaned-up version of Apache's and it matches it for each feature I've needed (namely, load balancing, easy config, and multiple SSL certificates). As a nice bonus, I didn't have to do any of the config massaging I had to for Apache in order to get XSP's funky resource-aggregation rules to work. If you have the means, I highly recommend it.

My latest foray into proxying also gave me an opportunity to look back into my main bugbears with the setup: Domino tracking the proxy server's IP instead of the original requester's and its lack of knowledge of SSL (which causes it to redirect from an SSL login page to a non-SSL one). Fortunately, it turns out that these problems have been sort-of-solved for years via a notes.ini setting added as part of Domino's terminal WebSphere infection: HTTPEnableConnectorHeaders.

By enabling that on my Domino server, I was able to start providing some of those headers. The remote-host headers ("$WSRA" and "$WSRH") work perfectly: setting that to the incoming host causes Domino to act just like that was the original requester, namely filling that in for the REMOTE_HOST field in classic and facesContext.getExternalContext().getRequest().getRemoteAddr() in XPages.

Unfortunately, I was stymied when I set "$WSIS" to True. Though it does indeed cause Domino to acknowledge that the incoming request is via SSL, it does it TOO well: Domino appears to revert to its behavior of only acknowledging a single SSL site, so it caused requests to essentially ignore the Host (and "$WSSN") headers. So that problem remains unsolved.

Still, I feel pretty good about my switch to Nginx and my abuse of the HTTP connector headers and look forward to tinkering some more. For reference, here is the config I use for the standard HTTP proxy ("arcturus" is the short name I gave for the main upstream target):

server {
    listen 80;
	
    location / {
        proxy_pass http://arcturus;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
        proxy_redirect off;
        proxy_buffering off;
        proxy_set_header    Host            $host;
        proxy_set_header    X-Real-IP       $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header    $WSRA           $remote_addr;
        proxy_set_header    $WSRH           $remote_addr;
        proxy_set_header    $WSSN           $host;
    }
}

Among the new features in Domino 9 is this little guy, found on the Launch pane of a database's properties when you set it to open an XPage:

If you've ever used an XPiNC application before, you'll know this is a godsend, promising the vast performance benefits of running an app on a server combined with the "the users are stuck using the Notes client" benefits of XPiNC.

I turned this on for a new app on one of my client's servers (let's say the server name is ClientName-2/ClientName) and took it out for a spin. However, I immediately noticed something was amiss: the status bar declared that it was unable to find the server "www.clientname-2.com". With a bit of searching, I found I wasn't alone: Notes assumes that the Domino server is available as its common name in local DNS for some reason, and fails in various ways depending on your network's DNS failover behavior and whether or not www.clientname-2.com is a real web site.

So, okay, as the last comment on that linked question indicates, you can "fix" this behavior by manually adding entries to the "XPages Performance" preferences pane (I have two because it's on two clustered servers, you see):

(Though it's blurred here, I note that the "Server" column stores only the common name of the server, not its full hierarchy. Let's hope all your common names are distinct!)

Alright, now that that is sorted out, Notes should use the http://whatever.com URL you typed in rather than its own ludicrously-naïve guess. Open up the app again and bam:

Oh. Okay. Here's where things got fun. At this point, the Notes client is essentially using an embedded web browser to point to http://whatever.com/admin/vr.nsf/Page.xsp and carrying with it no authentication information. The documentation cheekily suggests that this means that your "configuration is not correctly set up" and basically leaves it at that.

I'll spare you the intervening days of frustration and cut to the chase: I needed to create an Account document in the client, set up SSO on the server (I had been using normal session auth), and disable loading Internet configurations from Server\Internet Sites documents (seriously).

Because I wasn't about to have my users use the Preferences -> Accounts pane if I could avoid it, I discovered that you can apply Accounts via policies. I went into the "Accounts" view of the "Configuration" tab of Administrator and created an account like so:

(That bit about "PreferredUsernameField" was because I stole the instructions from something about the social stuff in 9. I don't know if it's required, but I don't dare remove it now.)

Once I had that account document created, I went to the active Desktop policy document, then the "Accounts" tab, clicked "Update Links", and chose "All Supported" (because I only had the one anyway):

So at this point, I had it so that the Notes client was properly picking up the Account document, but it still wasn't signing in via SSO properly. That's when I had flashbacks to setting up a Sametime server, I felt a dread in the pit of my stomach, and I disabled Internet Sites for the server. Once I generated a new SSO configuration (with no "Organization" specified, so it was choosable in the server document) and restarted HTTP, it started to work. Hooray-ish!

Fortunately, in my case, I'm lucky: there's a spare server available that doesn't need to handle normal web requests (at least barring a catastrophe), so I can afford to disable Internet Sites configuration for it. However, long-term, I would be delighted to be wrong in my diagnosis - so if anybody knows a way to get this working while still supporting Internet Sites, please let me know. I would also love to know if there's a way to either avoid using those "run these apps on the server" entries in the client prefs or distribute those via policy as well.

David Leedy has run into some trouble with NotesIn9.com being down and he asked if I could help him get the word out about it, which I'm more than happy to do:

Hi - Just wanted to drop a note out there about my NotesIn9.com website.  Currently it's redirecting to someplace else for some unknown reason.  I assume it's been php hacked but I don't know.