For the Design package of the OpenNTF Domino API, one of my goals was to add the ability to access the supporting structures around XPages: file resources, images, CSS, and, crucially, Java classes and the XPages themselves. My weapon of choice for the whole API is DXL, and this covered the job nicely for most of those aspects. DXL has clean representations for the first three in the list: specialized XML formats containing the pertinent metadata and a single BASE64-encoded block of the actual file data.

That's not the case for Java classes and XPages, however. For those, DXL falls back to the "note" format (inaptly referred to as "Binary DXL" in the Designer prefs). What sets this format apart from the element-specific DXL is that it doesn't attempt to present the data in a particularly friendly way - no specialized file- or image-resource support, no converting of agent code to a readable format, and no converting of rich text to the HTML-like structure you've seen if you've worked with DXL. Not everything is stored as binary data in this format - text, numbers, (some) formulas, and (most) date/time values are still converted to a human-friendly form. But the point remains: this format is closer to the data you would get by using the C API. In fact, for the "rawitemdata" entries, it is the data you would get by using the C API. This came up originally in the Design API and again the last couple days in the "nsfdata" package I'm working on to replace the design innards.

The APIs

I'd like to step back a moment from the notion of getting the C API data to go over the various APIs of Domino. The normal Java API we work with - the lotus.domino.local one, not the OpenNTF one - is a third level of abstraction (or maybe fourth - I'm not sure if there's an extra layer of C++ stuff in here). The Java API is a fairly-thin wrapper around the same back-end classes used in LotusScript ("LSXBE"). Other than presenting some data types as Java native types like String and int and providing a few convenience methods like DateTime#toJavaDate, it doesn't bring too much to the table.

The "LSXBE" objects have a more-complicated job (assuming there's not another layer): they bring object-oriented order to the lowest level, the C API. The C API is basically a big soup of data structures, constants, type aliases (e.g. DWORD = 32-bit integer), and functions to operate on these. As is usually the case with a C API, dealing with it is onerous, and so the higher-level APIs wrap these capabilities into the objects you know and love. This wrapping has a number of significant limitations in functionality and speed, but overall the capabilities you see in the objects have direct analogues at the C level.

Those limitations, incidentally, are laid bare by the XPage environment's cheating-like-crazy "napi" classes.

But the point is that using the higher-level APIs generally shields you from the harsh realities of the C structures, but they leak through in the "rawitemdata" elements in DXL. As it turns out, data in these elements, when BASE64-decoded, can be accessed according to the specifications in the C API struct documentation.

Composite Data

Most of the item data structures are both pretty straightforward and not actually presented in DXL anyway (numbers-list items, for example, are basically a count of entries and then a series of 64-bit floating-point values (oddly, the API supports the concept of a number range like a date range, but NSF does not)). There are a couple types that take some more understanding, though, and foremost among them is Composite Data.

Composite Data is primarily the format that NSF uses to store rich text data, though it's full of components to store non-RT data like file resources. A Composite Data item is essentially an array of "records", which consist of a few things:

1. A number indicating the type of the record (potentially when combined with the next part). A record type is something like "TEXT", "TABLEBEGIN", or "LINK": one of the core components that make up Composite Data. You can think of these sort of like XML start and end tags.
2. A number indicating either the stated size of the record or, if it's 0 or 255, the size of the number following this that indicates the stated size of the record. C APIs are full of crap like that.
3. If the size number was 0 or 255, a 16- or 32-bit integer indicating the stated size of the record.
4. Any fixed-size structure data stored in the record. For example, CDFILEHEADER, one of the records used in file resources, contains a number of fields indicating things like the length of the file extension (I don't know why), the number of CDFILESEGMENTs that make up the file, and so forth.
5. Any variable-length data stored in this record. Not all record types contain any special data beyond the basic struct, but many do. This usually consists of any associated strings (such as the aforementioned file extension) or raw binary data like that of a file segment. There may also be padding bytes attached to strings when the number of bytes used to store them is non-even.

This structure is why RichTextNavigator acts the awkward way it does.

The nice thing about understanding this is that, once you do, it opens a window into all sorts of low-level operations. You're no longer restricted to working with just the elements that "friendly" DXL presents: while you still have to do the work of understanding the pertinent records, the work is straightforward. I originally used this as a way to read and write file-resource-type notes, but my new classes are meant to be more generic, opening the door to arbitrary Composite Data manipulation.

So... okay... why?

As I mentioned at the start, the impetus to my delving this past week was to clean up the backend of the Design API to work more generically, rather than consisting of one-offs to deal with the items just in file resources. The side effects are intriguing in a "mad scentist" sort of way, though. In effect, what I've been building so far is a mechanism for accessing and manipulating native NSF data structures without the presence of an actual NSF (or, in fact, any Notes/Domino dependency at all), in the process completing the utility of DXL. The implications of a fully-functional non-NSF store of NSF data are fascinating.

It's also important foundational work if I deal with the C API more directly in the future. Years back, I dabbled in a Ruby wrapper around the API, and the concept has never quite left my brain. As the XPage NAPI demonstrates, there's a tremendous amount of speed and functionality benefit to be had in bypassing the legacy Java API completely, and I may travel more fully down that path one day.

But the final reason why is "because it's there". Learning more about these underlying structures provides tremendous insight into why Domino does the things it does. It's also just good for my brain to deal with an API with conventions other than the legacy Java API or Java-standard semantics.

So I'm interested to see where this will go in the long term. It's possible its life will be primarily as a slightly-better back end for the existing Design API functionality, but it opens up numerous potential roads for future capabilities.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Add sorting to the view

This step in the framework example is very similar to the second in that there's nothing particularly Framework-y about it.

Technically, the work in the title of this step is already done: I had already enabled click-to-sort and set sortable="true" in the XSP source. That on its own is enough to, as with a normal view, enable both available sort directions in the view panel:

So... that was easy! While we're at it, we'll go back in and add a categorized column. This, too, has no gotchas:

In my case, Designer gave this category a programmatic name of "$4", so we'll use that in the modified XSP for the view panel:

<xp:viewPanel value="#{Notes.all}" pageName="/note.xsp">
	<xp:this.facets>
		<xp:pager xp:key="headerPager" id="pager1" partialRefresh="true" layout="Previous Group Next" />
	</xp:this.facets>
	
	<xp:viewColumn columnName="$4"/>
	<xp:viewColumn columnName="Title" displayAs="link">
		<xp:viewColumnHeader value="Title" sortable="true"/>
	</xp:viewColumn>
	<xp:viewColumn columnName="Body">
		<xp:viewColumnHeader value="Body"/>
	</xp:viewColumn>
</xp:viewPanel>

This gives us the categorized column we expect, including the ability to collapse the categories like normal:

And, well, that's it too. Because of my extensive use of view features on the back end, Framework collections inherit the same features and (most of) the performance of the existing view data sources. That's a lot of the point: taking these model collections and using them in the ways you're used to using views in XPages apps is meant to be easy and not something you have to think too much about.

Fair warning: this post isn't technical. It's also full of sweeping generalities.

When I attended middle and high school, it was as part of the last group of geeks and nerds that should face systemic ostracization. Naturally, high school will still be terrible for most people, just because it's full of high schoolers, but the world at large has seen a near-complete triumph for geeks and nerds generally. And this has had some very positive side effects: because we had to deal with exclusion and diminishment, we could solve those problems in our own larger culture. We'd be the Good Guys.

Or at least we were supposed to be.

Truth be told, the tech industry HAS significantly advanced inclusive social values. It's much easier to fall outside of the mainstream and do fine in the tech community than in many others. However, for all this social advancement, we've still brought along an undercurrent of sexism.

This isn't some small, two-bit sexism, either. The trouble is that it's easy for those unaffected (say, men) to assume it's not there. That was my problem for a while: since I'm male and haven't done anything particularly sexist, I was content to assume things were hunky-dory. They're not.

Fortunately for me, others have written about this much more ably than I can. I'd like to ask anyone reading this to go and read at least some of these excellent posts/videos/podcasts on the topic from the last few months, in no particular order. Many are about associated communities like gaming and comics, but it's all related.

• No skin thick enough: The daily harassment of women in the game industry
• Your Princess Is in Another Castle: Misogyny, Entitlement, and Nerds
• Fake Geek Guys: A Message to Men About Sexual Harassment
• Gaming While Male: A 'Privilege' Few Men Recognize
• Brianna Wu - Nine ways to stop hurting and start helping women in tech
• Debug 34: Sexism in tech

There were also a great many outstanding articles posted about women's experiences during the heyday of YesAllWomen, but I can't track down the ones I was thinking of at the moment. They're worth searching for.

I've been mulling over the best ways to deal with it. The core part is easy: be a decent human being. Facing the reality of what goes on itself goes a long way towards this, since it's made me more cognizant of what half the population has to put up with. Another step I've taken today is to put my money where my mouth is and start donating to App Camp For Girls, a truly outstanding organization dedicated to giving girls an opportunity to be exposed to development while they're growing up. When I was a CS student, there were never more than two females in any of my normal-sized classes, and that's as good a place to start as anywhere else.

Beyond that? My plan is to keep being upset about this. Given that it's been a pretty much constant condition throughout human history, I don't expect it to be solved soon, but it's important to try to keep on the right track.

In yesterday's Framework-series post, I offhandedly mentioned that the "flashMessage" routine I use could easily be paired with or replaced by "toaster"-style notifications. This turned out to be very coincidental, as just today a NotesIn9 episode went up featuring Brad Balassaitis talking about using dGrowl for this purpose, along with an accompanying blog post.

Whenever I've used something like this, I've run into situations where I want to generate the message from the server, say as part of a partial refresh event to save an object. One option in that situation is to attach JavaScript to the "onComplete" and "onError" properties of the event handler (the latter of which is something I should do more regularly), but that doesn't give you access to all the information on the server that you may want to use in the message.

What I really want to do is send customized JavaScript code from the server back to the client, just for that event. Fortunately, there's an out-of-the-way method in the XPages stack to allow you to do this: UIViewRootEx#postScript. Though the documentation on that page is copious and detailed, it may not quite give you an indication of what that method does. What it does is to take a string of (client) JavaScript code that is then wrapped up and included in the currently-being-generated response to the browser, and which will be executed at the end of the current event (the partial refresh or, probably, initial page load). You can get to the view root object by resolving the "view" variable and casting it to an appropriate class:

UIViewRootEx2 view = (UIViewRootEx2)ExtLibUtil.resolveVariable(FacesContext.getCurrentInstance(), "view");

So I've used this in a couple situations, and here's an example of a method I have in my utils class for a current project. The project uses a WrapBootstrap theme that came with Gritter, which is a jQuery plugin for a similar effect, but it could be adapted for either Dojo module.

public static void toaster(final String summary, final String detail, final boolean sticky, final String styleClass) {
	StringBuilder result = new StringBuilder();
	result.append("jQuery.gritter.add({\n");
	if(StringUtil.isNotEmpty(summary)) {
		result.append("\ttitle: ");
		JSUtil.addString(result, summary);
		result.append(",\n");
	}
	if(StringUtil.isNotEmpty(detail)) {
		result.append("\ttext: ");
		JSUtil.addString(result, detail);
		result.append(",\n");
	}
	result.append("\tsticky: " + sticky + ",\n");
	
	String effectiveStyleClass = (styleClass == null ? "gritter-info" : styleClass);
	result.append("\tclass_name: ");
	JSUtil.addString(result, effectiveStyleClass);
	result.append("\n");
	
	result.append("})");

	FrameworkUtils.getViewRoot().postScript(result.toString());
}

So, first off: yikes. This is an example of why Java is awful at string handling and generating other languages. However, this ugliness is in service of a vital lesson that goes beyond this task:

Never, ever, ever generate code without proper escaping.

This is why it's so problematic to generate JSON or XML via view columns: formula language lacks escaping functions for those languages, and most code I've seen neglects to include its own. If you don't escape strings you're adding into another language, you're asking for parsing bugs at best and code-injection attacks at worst.

But back to the code at hand. I'm using the com.ibm.xsp.util.JSUtil class that comes with the XPages framework to assist in writing out JavaScript code. The result is that you can write something like this in Java:

toaster("Some alert!", "These are \"alert\" details", false, "gritter-error");

...and end up with code like this sent to the browser:

jQuery.gritter.add({
	title: "Some alert!",
	text: "These are \"alert\" details",
	sticky: false,
	class_name: "gritter-error"
})

In my example from yesterday, if I were using a partial refresh instead of a page redirection, I could use this method to alert the user of a successful save.

Because my model framework publishes FacesMessages for event conditions in a Faces environment, I've also written some companion methods to translate them into a toaster events and to drain the queue, useful during a partial refresh:

public static void toastMessage(final FacesMessage message) {
	String styleClass = null;
	if(FacesMessage.SEVERITY_ERROR.equals(message.getSeverity())) {
		styleClass = "gritter-error";
	} else if(FacesMessage.SEVERITY_FATAL.equals(message.getSeverity())) {
		styleClass = "gritter-error";
	} else if(FacesMessage.SEVERITY_INFO.equals(message.getSeverity())) {
		styleClass = "gritter-info";
	} else if(FacesMessage.SEVERITY_WARN.equals(message.getSeverity())) {
		styleClass = "gritter-warning";
	} else {
		styleClass = "gritter-success";
	}
	toaster(message.getSummary(), message.getDetail(), false, styleClass);
}
@SuppressWarnings("unchecked")
public static void toastMessages() {
	Iterator<FacesMessage> messages = FacesContext.getCurrentInstance().getMessages();
	while(messages.hasNext()) {
		FacesMessage message = messages.next();
		toastMessage(message);
	}
}

This sort of thing can be a useful way to bridge the client and server sides of the app without giving up client UI niceties.

When you use any binding in XPages - anything with "#{...}" or "${...}", regardless of whether it's EL, SSJS, or other - you're creating what is called in JSF-land either a ValueBinding or a MethodBinding, and the distinction is important but can be subtle, particularly when dealing with SSJS.

Value Bindings

The simplest case of value bindings are sprinkled throughout even the most basic XPage app, created when you bind a control to a document's field:

<xp:inputText value="#{doc.Title}"/>

The meaning is clear: "this text box connects to the 'Title' property of the 'doc' object". Though simple in concept and syntax, there's a lot of cleverness built into this, both in how many types of objects it can deal with to figure out what "Title" means and also in that it supports both read and write operations with the same syntax. This ability applies to your own Java beans as well. In the absence of any supported interface, EL will map something like #{bean.foo} to both bean.getFoo() and bean.setFoo(...) as appropriate.

The same does not apply to SSJS, however. You may have tried something like this once, only to find your text box became read-only:

<xp:inputText value="#{javascript:doc.getItemValueString('Title')}"/>

The reason for this is that, though you are still creating a value binding, SSJS is only capable of producing read-only bindings. This is because the XPage has no idea that the code inside the brackets is getting the value of a Notes document item; anything in between #{javascript: and } may as well be static as far as the page is concerned. It knows that the code can produce a value, but that's it. Custom binding languages like this CAN allow value writing - the little-used XPath binding does - but SSJS does not, for good reason.

Method Bindings

So that's a value binding - what about method bindings? Well, you're writing these all the time, too. Here's an example of one:

<xp:this.beforePageLoad><![CDATA[#{javascript:
	print("hello")
}]]></xp:this.beforePageLoad>

Method bindings written in SSJS are effectively the same as SSJS value bindings: they just execute the code contained in them and optionally return a value (for example, in button actions).

EL is also capable of creating method bindings, but it behaves slightly differently than it does with value bindings. Take the same binding as before, #{bean.foo}, except now it's attached to the action property of an event handler instead of the value property of a text box. In this context, it is a method binding, and EL no longer cares about supported interfaces or bean conventions. Instead of converting the call into getValue(...) on a DataObject or getFoo()/setFoo(...) on a normal object, EL looks for a method named foo() explicitly. There's no translation and there's no "read mode" or "write mode".

You can see this at play in the stub home page in XPages Scaffolding. The beforePageLoad and afterPageLoad events are method bindings, and so they are bound to methods on the controller with the exact names specified.

There's is a weird gotcha, but it's rare: some method bindings pass parameters along. If you look at the additional event methods on the basic controller class, you see that they expect a PhaseEvent object to be passed in. If you write these same methods in SSJS, you don't have to care about that - in this case, it appears that the page just throws the argument away. However, if you bind an event with arguments to a method without, you'll get an exception like java.lang.IllegalArgumentException: wrong number of arguments. And unfortunately, even adding the parameter doesn't solve it on its own... you have to manually bind the event for some reason. The good news is that those are the only cases I know of where that happens - everything else expects an argument-less method.

When you're writing in primarily SSJS, the distinction between the two only matters for write-mode value bindings, but the more you deal with Java in XPages, the more it pays to know a little about how the XPages framework perceives the binding types.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Add notification to the model

The next step in building our Framework app is a simple one: adding some basic visual notification when user saves a note. We'll leave the navigation rule to send the user to the home page in place and use flashScope (included in the Framework) and a "messages" custom control included in the scaffolding DB to add a message.

There are a few places where I could add this code, primarily either the model or the controller. Since it makes sense in this context, I'll add it to the model, via this additional code in the model.Note class:

@Override
protected void postSave() {
	super.postSave();

	FrameworkUtils.flashMessage("confirmation", "Note saved successfully.");
}

The flashMessage method in FrameworkUtils is a convenience method for exactly this sort of thing. It adds the message string to a List in the flashScope's property with the provided first parameter plus "Messages". So, in this case, #{flashScope.confirmationMessages} ends up containing ["Note saved successfully."].

These values are read by the xc:messages control included in the standard xc:layout control in the scaffolding DB. It outputs both the normal xp:messages notifications as well as these flash-scoped ones - currently, the classes for the messages are hard-coded in the control and should be tweaked for use with OneUI, Bootstrap, or another UI framework, but eventually I'll shift those out to themes. Since I'm using OneUIv3 for this demo, I set the classes in the control to the appropriate values. The result is a small message that appears on the first load of a subsequent page and disappears thereafter:

Fancy? No, but it doesn't need to be. This same postSave hook could be used to do much more complex things whenever a note is saved in any context, such as sending a notification email, writing a log entry, or Toaster notification (forgive the missing images on that link).

As happens periodically, I've been thinking about Domino views lately. When I get into one of these moods, I find it helps to take a step back to look at what an NSF is.

An NSF is, in its heart of hearts, a key/value store. Each entry has several keys of which the useful ones are Note ID and the UNID, which are 32-bit and 128-bit integers, respectively, and where the Note ID is fixed and the UNID is mutable. Each entry's value is a multimap with string keys and values that are either effectively blobs or multi-value strings, numbers, or date/times+ranges, plus metadata.

What it does not intrinsically have (conceptually) is an ability to collect, index, and query documents other than "all" or by key (db.search can be thought of as a specialized instance of indexing). Layered on top (again: conceptually) of the NSF are two IBM-supported indexing schemes: the view indexer (NIF) and the full-text index. Though these services are baked into Domino at an API level (including data transit over the same wire), they are in many ways no different from LDAP, the RnR manager, or third-party Domino addins: they are independent services that provide additional capabilities to the server.

This is a long-winded way of saying: there's nothing stopping anyone from doing their own indexer, particularly now that the primary use would be within an XPage running on the server directly, not a legacy client. So what would such an indexer need? The way I see it, there are four conceptual parts: document selection, index entry creation, index updating, and querying. Both of the two built-in methods handle these tasks in their own way. The full-text index's answers are:

• Document selection: all of them.
• Index entry creation: all processable string data in the document, with variations configured at setup time. There is no user-defined index-stored metadata and the resultant index is effectively a black box.
• Index updating: "immediately", by which it means "eventually". Specifically, it's handled by an updater task that I'll charitably assume batches changes for group processing. Can also be done on schedule or manually.
• Querying: a custom string-based DSL that allows for selection of documents and sorting by "relevance" or creation date. It also attempts to provide location/highlight information for matched data in the document, but it's best not to think about that.

NIF's answers are:

• Document selection: formula language, with the limitation that it can select only based on summary data.
• Index entry creation: a combination of formula language (also with the summary limitation) and column and view configuration, resulting in a combined array+tree structure of individual entries, each of which is a combined array+map structure. This also involves specifying categories and keys for later querying.
• Index updating: this is a bit more reliably quick than the FT update, and operates on similar lines: by default, it's triggered by DB change events, but can be updated manually and set to update less frequently.
• Querying: querying is done via a series of operations to read the index structure. These operations focus on getting a single column's values, selecting entries/documents by key, and traversing entries sequentially with some hierarchical operations. The additional information included during entry creation can be used to eliminate the need to access the actual documents later in some situations.

Click-to-sort columns in views are effectively separate indexes that share much of their configuration information. NIF can also be combined with the full-text search index to insersect the pre-selected contents of the view with the FT query result.

When I dabbled with Fancy Views years ago, I focused primarily on the first two components. For selection, I allowed either the standard formula-based selection or an FT-search query, and for the entry creation I took the framework established by NIF and expanded it to allow any JSR-223 language to return a value, to work transparently with MIMEBean values, and to allow storing any Serializable value. The updating was skeletal - basically whenever I ran the agent - and the querying was half-assed, being limited to a single sort value and then just iteration. Still, this concept has promise: because the relative expense of generating the index is dwarfed on modern systems by the value of having a better resultant index, allowing complex operations like non-summary/MIMEBean access and alternative languages is very worthwhile.

The last couple days, I've been taking a look at CQEngine. CQEngine focuses on the final step - querying. It operates on Java Collections, which could range from an ArrayList of HashMaps to an arbitrarily-complex database index that implements Collection and for which the user provides key/value adapters. Where CQEngine shines is being able to build complex queries across multiple attributes and ordering the results, much like you would do in SQL.

I'm not 100% sold on the notion of CQEngine being a building block of a new view indexer, but it has some promise - and any new indexer doesn't have to be a full NIF replacement or even the only new indexer. The lack of a string-based query syntax makes it a bit awkward (would it be represented as a tree of XSP components?) and the fact that the built indexes aren't meant to be serialized means that they'd have to be rebuilt once per session (though the backing index itself wouldn't be). Combined with an initial indexer that takes the Couch* approach of a JavaScript/JSR-223 function to select documents and emit entry values and an index-update task, it could provide some interesting capabilities while being potentially much faster and more flexible than NIF for many operations.

Though this is currently all speculation, it's satisfying to know that, like with the OpenNTF Domino API, there's nothing standing in between speculation and a real system other than doing a bit of programming. It's also just one of many potential non-exclusive paths. One of the coolest aspects of the Cambrian explosion of NoSQL technology in recent years is that each system comes with its own take on indexing/querying and associated support systems have arisen that can be used side-by-side with a document store like Domino. The latter systems also have the side effect of further opening the window to the outside world.

So will I actually try to fully build out one of these index-replacement ideas? Eh, maybe. I get the itch every once in a while, either for performance concerns or my desire to index on MIMEBean data, and having a working index replacement could open up a world of new possibilities. So we'll see. For now, I put my CQEngine tinkering up on GitHub and I expect I'll keep the concept floating around in the back of my brain for the next couple days at least.

A post on Christian Güdemann's blog and a followup on Nathan Freeman's made me figure it could be useful to discuss how I deal with the job of working with document collections in the frostillic.us Framework.

First of all: I am not solving the same original problem Christian had. His post discussed strategies for actually opening and processing every document and secondarily building a collection using inadvisable selection formulas for views (namely, @Today). I don't generally do that stuff, so our paths diverge.

The collection code in the Framework is very focused on dancing on top of existing view indexes: using them for collecting documents, for sorting/categorization, and accessing summary data. In many ways, the main Framework collections can be thought of as a re-implementation of the Domino view data source in XPages, except without the same cheating they do and with some side benefits.

Core

The core essence of a Framework collection - a DominoModelList - is that it stores information about how to access the underlying view, which category filter to use, and which model class to use to create objects. This can be seen in its constructor; it grabs important metadata information about the view and that's about it. It's not until it's required that the list does the dirty work of actually fetching a ViewNavigator (or re-using one in the current request). As much as possible, I aim to use ViewNavigators - which is to say, whenever there's not an FT search involved. Navigators are (for Domino) highly efficient, particularly compared to the shockingly-bad performance characteristics of ViewEntryCollection.

The primary consumer of the navigator is the get(int) method (among other things, collections implement List). If you'll kindly ignore some workaround code for a bug I haven't reliably been able to pin on either my code or the OpenNTF API yet, you can see the navigator use in action: unless there's a search in place (in which case it falls back to VEC), the method fetches an active navigator, skips to the appropriate requested entry, and uses getCurrent() to retrieve it. Though the skip/retrieve mix is odd when the average case will be iterating over successive entries, the performance is speedy enough that I haven't felt the need to try to cover both random and sequential access differently.

Sorting

Since 8.5.0, we have the ability to use click-to-sort columns in the back-end API, and I make use of this to expose sortable columns though TabularDataModel's sorting methods. Once a sorted column is chosen, I pass that along to the underlying view when fetching it. If there's an FT seach query specified, I use the surfaced-in-8.5.3 FTSearchSorted method to retain the sorting.

Collapsible Categories

In 9.0.1, IBM added the ability to collapse categories in navigators via the setAutoExpandGuidance method paired with the view's setEnableNoteIDsForCategories. Using TabularDataModel's expand/collapse methods, I maintain a Set of the faux category note IDs generated when setEnableNoteIDsForCategories is enabled and pass them to the navigator when appropriate. This allows my code to deal with arbitrarily-collapsed categories without containing any other special code - considering how uselessly buggy my implementation of this was before 9.0.1, I'm quite happy those methods are there.

The combination of the sorting and categorization capabilities means that my collections are able to support the same xp:viewPanel UI features that standard xp:dominoView data sources are.

Deferred Data Access

The final key concept in the framework is the deferral of actually accessing a document until it's necessary. Each model object can be constructed in one of three ways: as a new document in a database, as a wrapper around an existing document, and as a wrapper around a view entry. In the view entry case, the model object doesn't touch the underlying document. Instead, it makes a note of the database path and the UNID (if a non-category) for if it DOES need to access it later and then stores the entry's column values in a map. While the model objects don't make a user-side distinction between view entries and documents (you can request any item value whether or not it's in the view), it DOES use these cached column values first. So if your code only requests values that are present in the view, the underlying document is never accessed at all. This leads to exceedingly-efficient (comparatively) data access without making the user of the objects worry about manually accessing the document if the value isn't in the view.

---

The result of all this is that the Framework collections share all the advantages and pitfalls of the underlying views. Some things are easy and fast (categorization, sorting, multi-entry documents, summary data) while some are still impractical or slow (Rich Text, MIMEBeans, arbitrary queries). But you go to production with the database indexer you have, and so far this method has been serving me well.

If you were a Mac user in the 90s and early 2000s, life could be tough. Though you loved your platform of choice and could see its advantages plain as day, you were in a distinct minority. The world was full of people ready to line up to explain why you're an idiot on a sinking ship. And the part that stung was that they weren't always wrong: for every person talking out of their ass about how great having a menu bar in every window was or the joys of running ancient line-of-business DOS apps, someone else pointed out the very real stability and performance issues that plagued classic Mac OS and OS X respectively, the fifth-class-citizen status among game developers, or the very-real possibility that Apple would join Netscape and Be under the treads of Microsoft's then-indomitable war machine.

The recourse for users was reflexive defensiveness. Mac market share is down to 2%? Well, that's the good 2%. Windows 2000 is actually stable and fast now? Well, the UI is still second-rate. Schools are switching en masse to PCs? Boy, they'll regret that when their cheap Dells give out! Much like the slings and arrows thrown at the Mac community, these defenses contained just enough truth to soothe the wounds, but it didn't matter. Maybe the community circling the wagons staunched the bleeding a bit, but the thing that really mattered was Apple breaking its own malaise and making great products again.

Unsurprisingly, what has me thinking about this is Domino. Specifically, the popularity of a post on Chris Miller's blog about Denny's apparently switching away from Notes. Now, I don't have a gripe with the post or the specifics of the situation - for all I know, Denny's is making the worst decision of its corporate-IT life. What bothered me is the paint-by-numbers way the story echoes through the Domino community. We've seen these things go around before, and the components are familiar when Company X decides to ditch Notes/Domino for Competitor Y:

• "Just wait until they see the REAL cost of Y! Then they'll be sorry!"
• "They must not be thinking about all their crucial apps! Y doesn't do that!"
• "Their stated reasons are wrong! They said Domino doesn't do Feature Z, but Domino is actually the BEST at Feature Z!" (nowadays, replace with "Use some other IBM product to get Feature Z!")
• "This is the work of some clueless IT manager who's just following the trends!"
• "They must be using an ancient version of Notes! Modern Notes clients are exemplars of memory efficiency and clean design!" (alternative for server admins: "They must not have configured their mail environment properly!")
• "I betcha they'll still be using Notes for their crucial apps in ten years, 'migration' or no!"

Much like the defenses of classic Mac OS, these all contain kernels of truth, and maybe Company X would indeed be better off sticking with Domino. But are these really claims you want to hang your hat on? The last one in particular is telling. Does it really fill your heart with pride that companies are going to be saddled with ancient, un-migratable code until the sun goes nova? There's a reason why "modernization" is such a big topic for Domino developers: the unfathomable mass of legacy Notes client apps is a severe issue. Whether you attempt to deal with it via clever en-masse approaches, by using remote-desktop workarounds, or by dragging apps one-by-one into the present, there's no getting around the fact that depending on Notes client apps is now an undesirable condition for a company.

So what's to be done about it? Well, for the most part, that's IBM's job. But for an individual developer, it's better to focus on why you should build a Domino app today, not decades ago. It builds on the past, sure - an XPages app I've been building for a client brings together data from dozens of their existing apps built over years in ways that would be much more difficult on other platforms. But even better is to acknowledge reality with clear eyes. Google Apps are really good, and they work on everything! Exchange provides a better mail/contacts/calendar experience for varied clients than Domino does. Non-XPages web-dev environments are brimming with surprising features and deployment has gotten good in recent years. The advantages of other platforms are not necessarily enough to be worth a switch, but they still exist.

Personally, yes, I care whether Domino ends up flourishing, but I profit more from addressing reality as it is rather than immediately dismissing bad news as illegitimate.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Add validation and translation to the model

Now that we have a page to edit the model objects, it's time to spruce them up a bit by adding some type information, an additional field, and some basic translation. When we first set up the Note model, it didn't contain any information about the fields it expects, instead leaving everything up to the pass-through defaults. Now, we'll codify the two fields we added last time, add validation to the title, and add a date/time field:

package model;

import java.util.Date;
import javax.persistence.Table;
import org.hibernate.validator.constraints.NotEmpty;
import frostillicus.xsp.model.domino.AbstractDominoModel;

@Table(name="Note")
public class Note extends AbstractDominoModel {
	private static final long serialVersionUID = 1L;

	@NotEmpty String title;
	Date posted;
	String body;

	// *snip*
}

Model object fields work in the absence of explicit getters and setters - you CAN specify those for additional complexity, but they are not required. You can see here the addition of a validator care of Hibernate Validator. The @NotEmpty annotation works similarly to "not empty" in EL: it requires that the String not be null and also that it not be blank. We also added a date/time field named "Posted" (date- and time-only fields can be added via the java.sql.Date and java.sql.Time classes, respectively) - let's add a form row back on our note.xsp page:

<xe:formRow binding="#{controller.components[note].Title}"/>
<xe:formRow binding="#{controller.components[note].Posted}"/>
<xe:formRow binding="#{controller.components[note].Body}">
	<xe:djTextarea/>
</xe:formRow>

I didn't add any specific information to flag "Title" as required or "Posted" as a date/time field on the XPage, but the controller's component binding picks both up:

The controller will add any validation annotations to the control, but it has special support for @NotEmpty and @NotNull to also add a required validator and property explicitly. Additionally, since the "Posted" field stores both a date and time, the controller builds a text box with a date/time converter and helper, in the same fashion as setting the data type in the GUI.

Now, though the "Posted" field name matches Domino's idiom from the mail template, it's sort of a weird name to present to a user, so we'll use this as an introduction to translation and creating human-friendly names. The way labeling works in Framework component binding, it looks for a resource bundle named in the form "model_translation.properties". Because it builds on the XSP stack's standard resource-bundle code, it also allows localized versions in the format "model_translation_fr.properties", picking the most appropriate one for the browser. The format of the keys to match the fields is the full class name, then a dot, then the name of the field. So here's a basic "model_translation.properties" File Resource:

model.Note.Posted=Created
model.Note.Body=Note Body

By creating this file, the controller automatically picks up on the new human-readable names:

And this goes for other locales as well. Here's a poorly-translated "model_translation_fr.properties" file:

model.Note.Title=Titre
model.Note.Posted=Créé
model.Note.Body=Corps

And if I switch the browser to use French as its top preferred language (the "Save" button is still the purview of normal XPages translation):

By moving these concerns - data type, validation, and translation - into the data model, it not only significantly lowers the amount of code on the XPage itself, but also makes it much easier to find the canonical rules. Types and validation are always in the model class - not strewn across half a dozen XPages and custom controls - and translations are always in the model translation file and tied directly to the fields they're translating - not hidden in XPage-specific properties files keyed by ID paths.

In the next part, we'll add a bit of UI friendliness to the note creation/editing process using a common messages control and a flashScope object.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

On the scale of "bog-standard" to "weird", the next phase of writing a frostillic.us Framework app sits in the middle: the basic steps of creating a model-editing page are pretty standard, but you can use unique aspects of the Framework to save a bunch of coding and add some nice dynamic features.

Create the editing page

The first step to adding the ability to create and edit notes is to adjust the main note-list page to add a button to actually create one. In my standard layout custom control, I have a facet area named "ActionBar" to add buttons to the top of the page:

<xc:layout navigationPath="/Home">
	<xp:this.facets>
		<xc:linksbar xp:key="LeftColumn" />
		<xp:div xp:key="ActionBar">
			<xp:link themeId="Button.Command" value="/note.xsp" text="New Note"/>
		</xp:div>
	</xp:this.facets>
	
	...
</xc:layout>

I like to use normal links for going to the editing page rather than an "openPage" simple action to avoid the extra round trip. By giving it the "themeId" of Button.Command, it ends up with styling that looks more like a button than a link.

Now that that's over with, we move on to the actual editing page. So we create a page named "note.xsp" and add in our basic surrounding XSP and the data source:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" xmlns:xe="http://www.ibm.com/xsp/coreex" xmlns:xc="http://www.ibm.com/xsp/custom" xmlns:ff="http://frostillic.us/framework"
	beforePageLoad="#{controller.beforePageLoad}" afterPageLoad="#{controller.afterPageLoad}" pageTitle="Note">

	<xp:this.data>
		<ff:modelObjectData var="note" managerName="Notes" key="${empty param.id ? 'new' : param.id}"/>
	</xp:this.data>
	
</xp:view>

You can see a couple Framework-isms here: the "ff" namespace for the model data sources, the "beforePageLoad" and "afterPageLoad" events bound to the implied controller class, and the model data source itself. That data source takes the place of the traditional xp:dominoDocument element when using Framework model objects. It has two important properties: "managerName", which is the managed bean name of the manager class to fetch the object from, and "key", which tells the manager to either fetch an existing object by ID (for Domino-backed model objects, the document UNID) or create a new one. For this, I use the ternary operator in EL to check to see if there was an ID passed in via URL - if so, use that ID; otherwise, tell the manager to create a new model. Think of this as the difference between visiting a xp:dominoDocument-using page with action=editDocument&documentId=XXXX and with action=newDocument.

Now, on to the actual "form" portion. For traditional label/field sets, I'm a fan of using the ExtLib's xe:formTable and associated controls, since it's very focused on declaratively describing the form, and then I can use custom renderers to modify the look as I see fit. So let's toss one of those on the page:

<xe:formTable formTitle="Note">
	<xe:this.facets>
		<xp:div xp:key="footer">
			<xp:button id="save" value="Save">
				<xp:eventHandler event="onclick" submit="true" refreshMode="complete" immediate="false" save="true"/>
			</xp:button>
		</xp:div>
	</xe:this.facets>

	<xe:formRow binding="#{controller.components[note].Title}"/>
	<xe:formRow binding="#{controller.components[note].Body}">
		<xe:djTextarea/>
	</xe:formRow>
</xe:formTable>

So far, so good, right? That is, except for that "binding" business. That's the fruit of my recent dabbling in the potential uses of that property. Think of that as telling the back-end framework code that the two rows represent, respectively, the "Title" and "Body" fields of the given note object specifically, and asking it to fill in the messy details. As a result, the framework looks up an appropriate label for the field (by default, just the name of the field in the model, but it also looks for a human-friendly and translated version, which I'll get to in the next entry), creates an input component if one hasn't been provided, and attaches any appropriate validators, converters, and date/time helpers.

There are two types of use here: for the "Title" field, just a plain text box will do, which is what the controller does in the absence of other direction; for the "Body" field, however, I want to use Dojo's auto-expanding textarea control, and so placing one inside the form row will cause the controller to pick up on it and use the existing control rather than adding a new one. We haven't added any extra validation or type information to the model yet, so the result is basic:

There's one last bit of housekeeping to add: sending the user back to the main home page after saving a note. I use navigation rules for this purpose, and this syntax matches what you get if you set the "Next page (success or cancel)" option in the XPage's properties pane:

<xp:this.navigationRules>
	<xp:navigationRule outcome="xsp-success" viewId="/home.xsp"/>
</xp:this.navigationRules>

And that's it! Other than a few Framework-isms in the areas of the data source and the binding code, the result is pretty similar to a page you'd get using traditional methods. In the next entry, I'll spruce things up a bit: adding in some validation, a different field type, and a dash of translation.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

The first stage of building a frostillic.us-Framework-based app was a bit unusual - focusing on creating a model object in Java - but the next phase is about as bog-standard as it gets:

Create the view and add it to an XPage

The view we'll need is a basic one showing all Notes. We'll name it "Notes\All" to match the expected prefix in the model and add our two expected columns: Title and Body. The model classes work with sortable columns, so we'll make Title sortable:

Once we have the view, we can refer to it in EL in an XPage as #{Notes.all}. For demo purposes, we'll drop it on the page using a standard xp:viewPanel - not the fanciest presentation, but it'll get the job done:

<xp:viewPanel value="#{Notes.all}" pageName="/note.xsp">
	<xp:this.facets>
		<xp:pager xp:key="headerPager" id="pager1" partialRefresh="true" layout="Previous Group Next" />
	</xp:this.facets>
	
	<xp:viewColumn columnName="Title" displayAs="link">
		<xp:viewColumnHeader value="Title" sortable="true"/>
	</xp:viewColumn>
	<xp:viewColumn columnName="Body">
		<xp:viewColumnHeader value="Body"/>
	</xp:viewColumn>
</xp:viewPanel>

Here, I'm using value="..." to specify the data source. There is also a normal data source for model collections, but it's rarely needed - the EL syntax is much more concise and clear, and it works well with relations (e.g. #{note.responses}). I also specify a pageName="..." for the generated links, since $$OpenDominoDocument URLs won't help here. I may add something to assist with that down the line, but for now specifying the page will do.

Other than those, though: pretty standard stuff. That's one of the goals of the framework: when it makes sense to do things the standard way, I want to do that - no need to reinvent every wheel. So the lists extend TabularDataModel in addition to implementing List and the model objects themselves implement ViewRowData in addition to DataObject.

The next topic - creating an editing page - continues this trend of working with the standard components while cutting out as much hassle as possible.

Now that the framework I've been building is settling into a real project, I figured one of the best ways for me to explain how it works and to double-check my own assumptions is to make a tutorial on how to actually construct an app using it. This will be a very simple one to start with: a note-taking app with just a couple fields and only one business object to speak of.

The overall conceit of the framework is that it uses standard/ExtLib components and idioms where appropriate but declares its own new idioms when the standard ones are deficient. This is done in a way, though, that is opt-in at each step. Though this app is positively stewing in the "Jesse" way of doing things, each component - controllers, models, servlets, etc. - can operate independently and alongside other methods of XPage development.

Probably unsurprisingly, writing a fully framework-based app is heavy on Java and uses some aspects that may be strange at first (annotations, namely), but I hope that the tutorial will explain why I go this route, and that it will demonstrate how using Java in this way can save a tremendous amount of programming and eliminate swaths of potential bugs.

I have a tentative plan for the series, and I'll adjust this list and add links as appropriate:

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Without further ado, on to part 1:

Define the Data Model

Technically, I should start with part 0: set up the framework. As of this writing, I haven't packaged it up on OpenNTF yet, but I've created a ZIP file containing a current build, the NTF I use to create these apps, as well as the OpenNTF API from my branch - the framework relies on some features not yet released in an API milestone release:

frostillicus.framework-20140709.zip

Requirements-wise, you'll probably need 9.0.1 (the ExtLib that ships with it should suffice) and I haven't tested it on a server without AllPermission granted to all Java.

Now that that's out of the way, on to the actual code. To start with, we'll create the class that represents our core data model - the Note (in the note-taking sense) - and its Manager class. Domino-based model classes in the framework extend the frostillicus.xsp.model.domino.AbstractDominoModel class, while the Managers extend frostillicus.xsp.model.domino.AbstractDominoManager and are kept in the model package:

package model;

import javax.persistence.Table;

import frostillicus.xsp.bean.ApplicationScoped;
import frostillicus.xsp.bean.ManagedBean;
import frostillicus.xsp.model.domino.AbstractDominoModel;
import frostillicus.xsp.model.domino.AbstractDominoManager;

@Table(name="Note")
public class Note extends AbstractDominoModel {



	@ManagedBean(name="Notes")
	@ApplicationScoped
	public static class Manager extends AbstractDominoManager<Note> {

		@Override
		protected String getViewPrefix() {
			return "Notes\\";
		}
	}
}

So! What's going on here? Quite a few things, and just as important are the things that are not going on, such as code to declare all of the fields and each pertinent view's capabilities. Nonetheless, it's still a whole pile of Java concepts - this is the part that's the most unusual from a standard-XPages-dev perspective.

• Right at the start of the class, you see @Table(name="Note"). This is part of my overall attempt to borrow aspects of the Java Persistence API and Hibernate when appropriate. Normally, @Table refers to a SQL table, but I'm using it here to declare the form name to use when creating documents. This may change in future versions.
• There are no getters/setters (or other methods) in the Note class! These objects implement DataObject and so the standard way of accessing data is via getValue(...) and setValue(...). There are ways to add special behavior - complex getters, validation, and types - but the default behavior is to mimic the Domino way of just storing anything with any name.
• The Manager class is stored as what's called an "inner class" in Java. It's a way of storing multiple related classes inside a single .java file. The full name of the class is a combination of the two names - so if you import the Task class in another file, the manager is named Task.Manager.
• I'm using @ManagedBean annotations. These are something I tossed together a while ago based on how managed beans can be done in newer versions of JSF. In essence, these annotations let you specify the name and scope of your beans in-line with the code, without adding them to faces-config.xml. The result acts just the same way.
• The only method specified in the Manager class is to provide a standard prefix for accessing views. I'll demonstrate using that later, but this is specified here to encourage the idiom of naming views like "Notes\All" or "Notes\By Day". It's not required, but following it saves some code elsewhere.
• The Manager is also the place where you can specify an external database to house the data, but I'm not doing that here.

It's a strange bit of code, but my hope is that each line is clear once you understand the concept. My aim is to, as much as possible with Java, make the code in the model classes focus on specifying what the object does and where it's found, not on the messy business of actually retrieving and storing it (while also avoiding the configuration headaches of full JPA/Hibernate).

In the next post in the series, I'll cover adding a view for these documents to the DB and to an XPage.

The core trouble in my eyes with developing on Domino is that it is unloved for what it is. Not so much by customers (I have no interest in whether companies use Exchange or SharePoint or whatever), but more by IBM. The situation reminds me of Apple with products like WebObjects, Xserve, and Aperture: there's clearly a dedicated and talented team behind it, but the organization's heart isn't in it. The analogy is inexact: first, Apple's overarching motivations and strategies are much easier to grok than IBM's; second, enterprise software never really dies - it just goes all COBOL and lingers in maintenance somewhere forever (did you know OS/2 is still a thing?).

So Domino is still around, and still exists, but is generally positioned as something for large companies to use for mail, or continue using if they happened to start doing so decades ago, or as an adjunct to Connections. But it's like with Lotusphere's rebranding: when (dev conference) + (biz conference) = (biz conference), the algebra to figure out the perceived value of (dev conference) isn't difficult. To a certain extent, this is just how IBM does business: they talk to large organizations' higher-ups and make their sales that way, not by being outwardly compelling. However, on this front, Bluemix's entry has provided a telling counter-example: though there's still the stock over-puffed business page, the bluemix.net site talks directly to developers using a layout and writing style that an actual human being might enjoy. They even have honest-to-goodness Developer Evangelists!

It leaves us, as Domino developers, in an awkward position. We have a full-fledged NoSQL server with an integrated Java dev stack, albeit one without a guiding soul. We have an easy-to-install, flawlessly-clustering, cross-platform server that is perpetually 20% away from being perfect for the modern world. Large portions can be charitably characterized as having been in maintenance mode for a long time: the core API, SSL in all protocols, calendar/contacts connectivity, indexing, the admin client, and so forth. The bad news is easy to perceive: less vendor interest and customers driven primarily by inertia make for a poor career path. But the point of this post isn't doom and gloom! The way I figure it, there are a number of bright sides to explain why I and others continue to develop on this platform:

• There's always a chance that Domino will be less "Xserve" and more "Mac Pro", a product seemingly at death's door for years until it was given a fresh breath of life.
• As Paul Graham pointed out years ago, when you're writing server-run software, you can use whichever platform you'd like. Though, all else being equal, it's nicer to have a popular platform, if you can use the tool to do greater work with your time than you would elsewhere, it's worthwhile.
• Due to Domino's nature, almost all of its (technical) problems are solvable even if IBM never touches it again. Some of them, like the API, are things where the community has already built a workaround. Some, like SSL for HTTP, have seen IBM package their own workaround with the server. And the rest, like NIF and CalDAV/CardDAV support, linger as perfect "if I had time" projects.

The last one is crucial for our position: though a totally unsupported problem would eventually fall prey to incompatibility with newer operating systems and machines, Domino has reached a point in the last couple years (thanks to the extensibility API in 8.5.2 and the vital bug-fixes in 8.5.3 and 9) where it's an agglomeration of replaceable parts. There are enough hooks and APIs to - with varying degrees of difficulty - take advantage of the core strengths of the platform while dramatically improving the components on top. That's not enough to last forever, but it doesn't have to be. Apps built in the mean time will still run and improve, and any time spent programming on the platform is valuable experience that can be applied elsewhere, whether directly or more generally.

Just don't ask about licensing.

In a similar vein to my post from the other day, I'd like to discuss one of the other bugaboos that crops up from time to time in the edges of XPages development, usually spoken of in hushed tones: reflection.

If you're not familiar with the term, reflection is an aspect of metaprogramming; specifically, it's the ability to inspect and operate on objects and classes at runtime. Reflection in Java isn't as smooth as it is in other languages, but it makes sense as you work with it.

The most basic form is the getClass() that every non-null object has: you can use it to get an object representing the class of the object in question, which is your main starting point. This code will print out "java.lang.String":

Class<?> stringClass = "".getClass();
System.out.println(stringClass.getName());

From here, you can start to get fancier. For example, this code...

Class<?> stringClass = "".getClass();

for(java.lang.reflect.Method method : stringClass.getMethods()) {
	System.out.println(method);
}

... will print out the signatures for all public instance and static methods, with output like:

public boolean java.lang.String.equals(java.lang.Object)
public int java.lang.String.hashCode()
public java.lang.String java.lang.String.toString()
public char java.lang.String.charAt(int)
public int java.lang.String.compareTo(java.lang.String)
...
public int java.lang.String.compareTo(java.lang.Object)
public final native java.lang.Class java.lang.Object.getClass()
public final native void java.lang.Object.notify()
public final native void java.lang.Object.notifyAll()

There is also a variant to show just the methods from that specific class, including private/protected ones. But the java.lang.reflect.Method objects you get there are useful for more than just printing out signatures: they're proxies for actually executing the method against an appropriate object. To demonstrate, say you have this class:

public class ExampleClass {
	private String name;

	public ExampleClass(final String name) { this.name = name; }

	public void printName() { System.out.println(name); }
}

You can then use reflection to call the method (this also demonstrates an alternate way of getting a class object if you know the class you want):

ExampleClass someObject = new ExampleClass("my name is someObject");
Method printNameMethod = ExampleClass.class.getMethod("printName");
printNameMethod.invoke(someObject);

So, okay, I found a way to call a method that involves way more typing for no gain - so what? Well, in a situation where reflection is actually useful, you won't actually know what methods are available. And this happens constantly in XPages, but you don't have to worry about it: reflection is how EL and SSJS do their work. If you've ever looked through one of the monstrous stack traces emitted by an XPage with an exception, you've probably seen this very call:

java.lang.Exception
    controller.home.getOutput(home.java:18)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
    java.lang.reflect.Method.invoke(Method.java:611)
    com.sun.faces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:96)
	...

That's reflection in action. The EL resolver found the object and then used reflection to determine how it should get the "output" property (most likely the instanceof operator) and then, determining that it was talking to a plain bean, constructed the name "getOutput", asked the object's class for that method, and invoked it.

Like most things in Java, reflection gets more complicated and powerful from here; there are facilities for dealing with generic types, creating new instances, reading annotations, making private properties/methods accessible, and so forth. But the basics are simple and worth knowing: it's all about asking an object what it is and what it is capable of. Once you're comfortable with that, the rest of the pieces will fall into place as you encounter them.

One of the overarching goals for my model framework is to get the job of specifying business logic out of the XPage. So far, most of my work in the area revolved around building in the ability to override getters/setters, establish relationships, and so forth. However, a big problem remains: the XPage still needs to be told things about the data that are really the model's job, not the XSP markup's. Namely, validation.

A little while ago, I "solved" this by adopting Hibernate Validator and building it into the framework. This allows very declarative specification of requirements and also let me build the validation into the model's save event. If a field is marked as required in the model, no matter what the XPage says, the object cannot be saved if the value is empty. So: mission accomplished, technically!

But it's not a great user experience: though I made it so that you get proper Faces messages about what the problem was, the error wasn't tied to the control that caused it, nor could the page do any client-side validation to avoid server round-trips. The solution I found for this and several other problems is, as happens very frequenty, a technical path Tim Tripcony started down a few months ago. Specifically, it's the esoteric "binding" property of each control on the XPage, which is a way of telling the control to set itself as the value of a given binding - say, on the page controller - to provide easy access to it without having to crawl the component tree.

I realized that I could use this with a specially-crafted set of objects to tell the controller and model framework exactly what is going on:

<xp:inputText binding="#{controller.components[task].Summary}"/>

That looks weird at first, but the meaning becomes clear: tell the controller that that text box represents the "Summary" property of the "task" model object. Once that binding happens, the controller can take care of a lot of work that would otherwise have to be done manually:

• Set the value="#{...}" binding
• If the model says the value is required, set required="true", allowing for client-side validation and an Aria attribute to be set
• If the model property specifies another known validation type (say, that it's an email address), add in an appropriate server- and client-side validator. For unknown types, add in a generic validator that will respond to any requirement
• If the property is a Collection type, specify a multipleSeparator property
• If the property is an enumeration and the control is a multi-value control (say, a xp:comboBox or xp:radioGroup), automatically add xp:selectItems for each entry
• Add appropriate converters and other assistant components, such as a date/time picker for Date fields

This could be taken much further, to the extent that there could potentially be a single generic xp:input-type control (or a placeholder CC) and then the controller would construct a control based on the model and client needs - this is a drum NTF has banged before. It could also apply to objects other than my model framework's, but that would take an appropriate adapter for each.

Not only does the XSP code get much trimmer, but the validation and type-specification code in Java is clear and to the point:

public class Task extends AbstractDominoModel {

	@NotEmpty String summary;
	@NotNull TimeFrame timeFrame;
	TaskType type;
	
	public static enum TimeFrame { Normal, Rush, Urgent }
	public static enum TaskType { Normal, Question }

	// ...
}

This feels like a nice step forward in the direction of putting concerns in their right places while also reducing the total amount of code. I'd call that the right kind of victory for app development.

The code for this lives in the framework project inside the XPages Scaffolding repository and the work of the component manipulation is in this class:

https://github.com/jesse-gallagher/XPages-Scaffolding/blob/6d7a9dc59ca070e74a5bd157eb49500b61717e57/frostillicus.framework.plugin/src/frostillicus/xsp/controller/ComponentMap.java

As this post's title implies, I'm considering making this a series on XPages anti-patterns, but no promises there.

In any event, this is a topic that's been stewing in my brain for a little while: my antipathy towards the session scope in XPages. Now, don't get me wrong: other than the "not reset on logout" thing that may be fixed by now, I have no technical qualms with sessionScope; it does what it says on the tin. However, I've often found that many people use it very frequently, whereas I have found fewer and fewer uses over time where it is appropriate.

To set the stage, I use the various scopes in roughly this order by descending frequency:

1. viewScope
2. applicationScope
3. flashScope
4. requestScope
5. sessionScope

The main things that I use sessionScope for are things that truly make sense only for the current browser session, such as the current date range to view in a log-viewing app. Other than that, I generally don't use it for:

Caches

Though it's not wrong, per se, to use the session for this, I've found it better overall to use the applicationScope, either directly (applicationScope.put("someCachedValue", whatever)) or by putting a Map keyed by username in there. The latter gets the same user-specific cache benefits of sessionScope (and more reliable, too, due to the potential for switched authentication) while also having the benefit of keeping the cache if the user logs in from another device. This is particularly potent with Anonymous. This is not a hard-and-fast rule, though - you may decide otherwise for cache-size or other reasons.

Primary Navigation or Context

Unlike the previous one, this is a hard-and-fast rule: do not use sessionScope for important page context. The worst would be something like having an "open document" button that puts the desired document UNID (or, worse, note ID) in sessionScope and then navigates to the page. Never do this! Though XPage URLs are a continuing problem, they're still the correct place for target-document information. The rule of thumb is that you should be able to copy the URL any time, paste it into another browser, and be in basically the same place.

Secondary Context

By this I mean things like the active linksbar category for the current page. I've seen things like having a navigation bar link that sends the user to a certain page while also setting a sessionScope variable to indicate the active menu bar. This is a huge problem for a number of reasons: it's a maintenance nightmare (having to code every link to do this), it's just asking for bugs (links setting the wrong or no value), and it breaks completely when the user bookmarks the page or comes back after session expiration. It's technically better than the previous crime, but only barely. The correct place for this information is handled somewhere in the page structure, though the specifics get murky. I generally take a page from the Extension Library example DB and use a "navigationPath" properly on my layout control to define a slash-delimited hierarchy of navigation context.

Page-to-Page Context

I'm thinking of things like a task-tracking system where you're looking at a Client document and want to add a Task to them, providing the Client document's ID for context. This is another area where the URL is the correct choice: ending up with a URL like "/task.xsp?clientId=whatever" makes the intent ("create a new task for client with ID 'whatever'") clear and stable across visits.

---

Overall, I think of sessionScope as the Petyr Baelish of XPage features: there are some cases where you have to deal with it, but you should generally consider it extremely unreliable and untrustworthy.

One of the most common idioms I run into when writing an app is the desire to cache some expensive-to-compute data in the view or application scope on first fetch, and then use the cached version from then on. It usually takes a form like this:

public Date getCachedTime() {
	Map<String, Object> applicationScope = ExtLibUtil.getApplicationScope();
	if(!applicationScope.containsKey("cachedTime")) {
		// Some actual expensive operation goes here
		applicationScope.put("cachedTime", new Date());
	}
	return (Date)applicationScope.get("cachedTime");
}

That works well, but it bothered me that I had to write the same boilerplate code every time, and I wondered if I could come up with a better way. The short answer is "no, not really". But the longer answer is "sort of!". And though the solution I came up with is kinda pathological and is generally unsuitable for normal humans and not worth the thin advantages, it does provide a good example of a couple of more-advanced Java concepts that you're likely to run across the more you get into the language. So here's the cache function in question, plus a "shorthand" version and an example:

@SuppressWarnings("unchecked")
public static <T> T cache(final String cacheKey, final String scope, final Callable<T> callable) throws Exception {
	Map<String, Object> cacheScope = (Map<String, Object>) ExtLibUtil.resolveVariable(FacesContext.getCurrentInstance(), scope + "Scope");
	if (!cacheScope.containsKey("cacheMap")) {
		cacheScope.put("cacheMap", new HashMap());
	}
	Map<String, Object> cacheMap = (Map<String, Object>) cacheScope.get("cacheMap");
	if (!cacheMap.containsKey(cacheKey)) {
		cacheMap.put(cacheKey, callable.call());
	}
	return (T) cacheMap.get(cacheKey);
}

public static <T> T cache(final String cacheKey, final Callable<T> callable) throws Exception {
	return cache(cacheKey, "application", callable);
}

public Date getFetchedTime() throws Exception {
	return cache("fetchedTime", new Callable<Date>() {
		public Date call() throws Exception {
			return new Date();
		}
	});
}

This code begs a question: what in the name of all that is good and holy is going on? There are a number of Java concepts at work here, some of which you've likely already seen:

• Generics. Generics are the class names in angle brackets, like Map<String, Object> and Callable<Date>. They're Java's way of taking a container or producer class and making it usable with any class types without having to cast everything from Object.
• Overriding methods. This is a small one: you can declare the same method name with different parameter types, which is often useful for "shorthand" versions of methods that allow for default values after a fashion. In this case, the short version of "cache" defaults to using the application scope by name.
• The Callable interface. This is an interface in the java.util.concurrent package and is meant as a utility for multithreading purposes, but it also serves our needs here. Basically, it's an interface that means "this contains code that can be executed by using the call method with no arguments".
• Anonymous classes. This is that business with "new Callable...". Java calls those anonymous classes and what they are are classes declared and instantiated inline with the code, without having a "proper" class declaration somewhere else. They're called "anonymous" because they have no name of their own - just the interface or class they implement/extend. In this case, I'm saying "make a new object that implements Callable and is used for this purpose only". I could make a standalone class, but there's no need. If you're familiar with closures from good languages, anonymous classes can be used like a really crappy version of those.
• Generic return value declarations. The "<T> T" and "Callable<T>" bits build on normal generic use for when declaring a method. The first one, is brackets, basically says "okay Java, rather than returning a known object type, I'm going to let the user use anything, and for this purpose we're going to call it T". So from then on, an occurrence of T in that method body refers to whatever the programmer using the method intends. You can see in the getFetchedTime that I'm using a Callable<Date>; Java picks up on that "Date" name and conceptually substitutes it for all the Ts in the method. I'm essentially saying "I'm going to use the cache method, and I want it to accept and return Dates". Another call to the method could be to cache a List<String> or a SomeUserClass, while the method itself would remain the same.

So is all this worth it for the task at hand? Probably not. There are SOME advantages, in that it's easy to change the default caching scope, and in practice I also used a map that auto-expires its entries over time, but for normal use the first idiom is fine. But hey, it sure provided a whole torrent of Java concepts, so that's worth something.