As happens periodically, I've been thinking about Domino views lately. When I get into one of these moods, I find it helps to take a step back to look at what an NSF is.

An NSF is, in its heart of hearts, a key/value store. Each entry has several keys of which the useful ones are Note ID and the UNID, which are 32-bit and 128-bit integers, respectively, and where the Note ID is fixed and the UNID is mutable. Each entry's value is a multimap with string keys and values that are either effectively blobs or multi-value strings, numbers, or date/times+ranges, plus metadata.

What it does not intrinsically have (conceptually) is an ability to collect, index, and query documents other than "all" or by key (db.search can be thought of as a specialized instance of indexing). Layered on top (again: conceptually) of the NSF are two IBM-supported indexing schemes: the view indexer (NIF) and the full-text index. Though these services are baked into Domino at an API level (including data transit over the same wire), they are in many ways no different from LDAP, the RnR manager, or third-party Domino addins: they are independent services that provide additional capabilities to the server.

This is a long-winded way of saying: there's nothing stopping anyone from doing their own indexer, particularly now that the primary use would be within an XPage running on the server directly, not a legacy client. So what would such an indexer need? The way I see it, there are four conceptual parts: document selection, index entry creation, index updating, and querying. Both of the two built-in methods handle these tasks in their own way. The full-text index's answers are:

• Document selection: all of them.
• Index entry creation: all processable string data in the document, with variations configured at setup time. There is no user-defined index-stored metadata and the resultant index is effectively a black box.
• Index updating: "immediately", by which it means "eventually". Specifically, it's handled by an updater task that I'll charitably assume batches changes for group processing. Can also be done on schedule or manually.
• Querying: a custom string-based DSL that allows for selection of documents and sorting by "relevance" or creation date. It also attempts to provide location/highlight information for matched data in the document, but it's best not to think about that.

NIF's answers are:

• Document selection: formula language, with the limitation that it can select only based on summary data.
• Index entry creation: a combination of formula language (also with the summary limitation) and column and view configuration, resulting in a combined array+tree structure of individual entries, each of which is a combined array+map structure. This also involves specifying categories and keys for later querying.
• Index updating: this is a bit more reliably quick than the FT update, and operates on similar lines: by default, it's triggered by DB change events, but can be updated manually and set to update less frequently.
• Querying: querying is done via a series of operations to read the index structure. These operations focus on getting a single column's values, selecting entries/documents by key, and traversing entries sequentially with some hierarchical operations. The additional information included during entry creation can be used to eliminate the need to access the actual documents later in some situations.

Click-to-sort columns in views are effectively separate indexes that share much of their configuration information. NIF can also be combined with the full-text search index to insersect the pre-selected contents of the view with the FT query result.

When I dabbled with Fancy Views years ago, I focused primarily on the first two components. For selection, I allowed either the standard formula-based selection or an FT-search query, and for the entry creation I took the framework established by NIF and expanded it to allow any JSR-223 language to return a value, to work transparently with MIMEBean values, and to allow storing any Serializable value. The updating was skeletal - basically whenever I ran the agent - and the querying was half-assed, being limited to a single sort value and then just iteration. Still, this concept has promise: because the relative expense of generating the index is dwarfed on modern systems by the value of having a better resultant index, allowing complex operations like non-summary/MIMEBean access and alternative languages is very worthwhile.

The last couple days, I've been taking a look at CQEngine. CQEngine focuses on the final step - querying. It operates on Java Collections, which could range from an ArrayList of HashMaps to an arbitrarily-complex database index that implements Collection and for which the user provides key/value adapters. Where CQEngine shines is being able to build complex queries across multiple attributes and ordering the results, much like you would do in SQL.

I'm not 100% sold on the notion of CQEngine being a building block of a new view indexer, but it has some promise - and any new indexer doesn't have to be a full NIF replacement or even the only new indexer. The lack of a string-based query syntax makes it a bit awkward (would it be represented as a tree of XSP components?) and the fact that the built indexes aren't meant to be serialized means that they'd have to be rebuilt once per session (though the backing index itself wouldn't be). Combined with an initial indexer that takes the Couch* approach of a JavaScript/JSR-223 function to select documents and emit entry values and an index-update task, it could provide some interesting capabilities while being potentially much faster and more flexible than NIF for many operations.

Though this is currently all speculation, it's satisfying to know that, like with the OpenNTF Domino API, there's nothing standing in between speculation and a real system other than doing a bit of programming. It's also just one of many potential non-exclusive paths. One of the coolest aspects of the Cambrian explosion of NoSQL technology in recent years is that each system comes with its own take on indexing/querying and associated support systems have arisen that can be used side-by-side with a document store like Domino. The latter systems also have the side effect of further opening the window to the outside world.

So will I actually try to fully build out one of these index-replacement ideas? Eh, maybe. I get the itch every once in a while, either for performance concerns or my desire to index on MIMEBean data, and having a working index replacement could open up a world of new possibilities. So we'll see. For now, I put my CQEngine tinkering up on GitHub and I expect I'll keep the concept floating around in the back of my brain for the next couple days at least.

A post on Christian Güdemann's blog and a followup on Nathan Freeman's made me figure it could be useful to discuss how I deal with the job of working with document collections in the frostillic.us Framework.

First of all: I am not solving the same original problem Christian had. His post discussed strategies for actually opening and processing every document and secondarily building a collection using inadvisable selection formulas for views (namely, @Today). I don't generally do that stuff, so our paths diverge.

The collection code in the Framework is very focused on dancing on top of existing view indexes: using them for collecting documents, for sorting/categorization, and accessing summary data. In many ways, the main Framework collections can be thought of as a re-implementation of the Domino view data source in XPages, except without the same cheating they do and with some side benefits.

Core

The core essence of a Framework collection - a DominoModelList - is that it stores information about how to access the underlying view, which category filter to use, and which model class to use to create objects. This can be seen in its constructor; it grabs important metadata information about the view and that's about it. It's not until it's required that the list does the dirty work of actually fetching a ViewNavigator (or re-using one in the current request). As much as possible, I aim to use ViewNavigators - which is to say, whenever there's not an FT search involved. Navigators are (for Domino) highly efficient, particularly compared to the shockingly-bad performance characteristics of ViewEntryCollection.

The primary consumer of the navigator is the get(int) method (among other things, collections implement List). If you'll kindly ignore some workaround code for a bug I haven't reliably been able to pin on either my code or the OpenNTF API yet, you can see the navigator use in action: unless there's a search in place (in which case it falls back to VEC), the method fetches an active navigator, skips to the appropriate requested entry, and uses getCurrent() to retrieve it. Though the skip/retrieve mix is odd when the average case will be iterating over successive entries, the performance is speedy enough that I haven't felt the need to try to cover both random and sequential access differently.

Sorting

Since 8.5.0, we have the ability to use click-to-sort columns in the back-end API, and I make use of this to expose sortable columns though TabularDataModel's sorting methods. Once a sorted column is chosen, I pass that along to the underlying view when fetching it. If there's an FT seach query specified, I use the surfaced-in-8.5.3 FTSearchSorted method to retain the sorting.

Collapsible Categories

In 9.0.1, IBM added the ability to collapse categories in navigators via the setAutoExpandGuidance method paired with the view's setEnableNoteIDsForCategories. Using TabularDataModel's expand/collapse methods, I maintain a Set of the faux category note IDs generated when setEnableNoteIDsForCategories is enabled and pass them to the navigator when appropriate. This allows my code to deal with arbitrarily-collapsed categories without containing any other special code - considering how uselessly buggy my implementation of this was before 9.0.1, I'm quite happy those methods are there.

The combination of the sorting and categorization capabilities means that my collections are able to support the same xp:viewPanel UI features that standard xp:dominoView data sources are.

Deferred Data Access

The final key concept in the framework is the deferral of actually accessing a document until it's necessary. Each model object can be constructed in one of three ways: as a new document in a database, as a wrapper around an existing document, and as a wrapper around a view entry. In the view entry case, the model object doesn't touch the underlying document. Instead, it makes a note of the database path and the UNID (if a non-category) for if it DOES need to access it later and then stores the entry's column values in a map. While the model objects don't make a user-side distinction between view entries and documents (you can request any item value whether or not it's in the view), it DOES use these cached column values first. So if your code only requests values that are present in the view, the underlying document is never accessed at all. This leads to exceedingly-efficient (comparatively) data access without making the user of the objects worry about manually accessing the document if the value isn't in the view.

---

The result of all this is that the Framework collections share all the advantages and pitfalls of the underlying views. Some things are easy and fast (categorization, sorting, multi-entry documents, summary data) while some are still impractical or slow (Rich Text, MIMEBeans, arbitrary queries). But you go to production with the database indexer you have, and so far this method has been serving me well.

If you were a Mac user in the 90s and early 2000s, life could be tough. Though you loved your platform of choice and could see its advantages plain as day, you were in a distinct minority. The world was full of people ready to line up to explain why you're an idiot on a sinking ship. And the part that stung was that they weren't always wrong: for every person talking out of their ass about how great having a menu bar in every window was or the joys of running ancient line-of-business DOS apps, someone else pointed out the very real stability and performance issues that plagued classic Mac OS and OS X respectively, the fifth-class-citizen status among game developers, or the very-real possibility that Apple would join Netscape and Be under the treads of Microsoft's then-indomitable war machine.

The recourse for users was reflexive defensiveness. Mac market share is down to 2%? Well, that's the good 2%. Windows 2000 is actually stable and fast now? Well, the UI is still second-rate. Schools are switching en masse to PCs? Boy, they'll regret that when their cheap Dells give out! Much like the slings and arrows thrown at the Mac community, these defenses contained just enough truth to soothe the wounds, but it didn't matter. Maybe the community circling the wagons staunched the bleeding a bit, but the thing that really mattered was Apple breaking its own malaise and making great products again.

Unsurprisingly, what has me thinking about this is Domino. Specifically, the popularity of a post on Chris Miller's blog about Denny's apparently switching away from Notes. Now, I don't have a gripe with the post or the specifics of the situation - for all I know, Denny's is making the worst decision of its corporate-IT life. What bothered me is the paint-by-numbers way the story echoes through the Domino community. We've seen these things go around before, and the components are familiar when Company X decides to ditch Notes/Domino for Competitor Y:

• "Just wait until they see the REAL cost of Y! Then they'll be sorry!"
• "They must not be thinking about all their crucial apps! Y doesn't do that!"
• "Their stated reasons are wrong! They said Domino doesn't do Feature Z, but Domino is actually the BEST at Feature Z!" (nowadays, replace with "Use some other IBM product to get Feature Z!")
• "This is the work of some clueless IT manager who's just following the trends!"
• "They must be using an ancient version of Notes! Modern Notes clients are exemplars of memory efficiency and clean design!" (alternative for server admins: "They must not have configured their mail environment properly!")
• "I betcha they'll still be using Notes for their crucial apps in ten years, 'migration' or no!"

Much like the defenses of classic Mac OS, these all contain kernels of truth, and maybe Company X would indeed be better off sticking with Domino. But are these really claims you want to hang your hat on? The last one in particular is telling. Does it really fill your heart with pride that companies are going to be saddled with ancient, un-migratable code until the sun goes nova? There's a reason why "modernization" is such a big topic for Domino developers: the unfathomable mass of legacy Notes client apps is a severe issue. Whether you attempt to deal with it via clever en-masse approaches, by using remote-desktop workarounds, or by dragging apps one-by-one into the present, there's no getting around the fact that depending on Notes client apps is now an undesirable condition for a company.

So what's to be done about it? Well, for the most part, that's IBM's job. But for an individual developer, it's better to focus on why you should build a Domino app today, not decades ago. It builds on the past, sure - an XPages app I've been building for a client brings together data from dozens of their existing apps built over years in ways that would be much more difficult on other platforms. But even better is to acknowledge reality with clear eyes. Google Apps are really good, and they work on everything! Exchange provides a better mail/contacts/calendar experience for varied clients than Domino does. Non-XPages web-dev environments are brimming with surprising features and deployment has gotten good in recent years. The advantages of other platforms are not necessarily enough to be worth a switch, but they still exist.

Personally, yes, I care whether Domino ends up flourishing, but I profit more from addressing reality as it is rather than immediately dismissing bad news as illegitimate.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Add validation and translation to the model

Now that we have a page to edit the model objects, it's time to spruce them up a bit by adding some type information, an additional field, and some basic translation. When we first set up the Note model, it didn't contain any information about the fields it expects, instead leaving everything up to the pass-through defaults. Now, we'll codify the two fields we added last time, add validation to the title, and add a date/time field:

package model;

import java.util.Date;
import javax.persistence.Table;
import org.hibernate.validator.constraints.NotEmpty;
import frostillicus.xsp.model.domino.AbstractDominoModel;

@Table(name="Note")
public class Note extends AbstractDominoModel {
	private static final long serialVersionUID = 1L;

	@NotEmpty String title;
	Date posted;
	String body;

	// *snip*
}

Model object fields work in the absence of explicit getters and setters - you CAN specify those for additional complexity, but they are not required. You can see here the addition of a validator care of Hibernate Validator. The @NotEmpty annotation works similarly to "not empty" in EL: it requires that the String not be null and also that it not be blank. We also added a date/time field named "Posted" (date- and time-only fields can be added via the java.sql.Date and java.sql.Time classes, respectively) - let's add a form row back on our note.xsp page:

<xe:formRow binding="#{controller.components[note].Title}"/>
<xe:formRow binding="#{controller.components[note].Posted}"/>
<xe:formRow binding="#{controller.components[note].Body}">
	<xe:djTextarea/>
</xe:formRow>

I didn't add any specific information to flag "Title" as required or "Posted" as a date/time field on the XPage, but the controller's component binding picks both up:

The controller will add any validation annotations to the control, but it has special support for @NotEmpty and @NotNull to also add a required validator and property explicitly. Additionally, since the "Posted" field stores both a date and time, the controller builds a text box with a date/time converter and helper, in the same fashion as setting the data type in the GUI.

Now, though the "Posted" field name matches Domino's idiom from the mail template, it's sort of a weird name to present to a user, so we'll use this as an introduction to translation and creating human-friendly names. The way labeling works in Framework component binding, it looks for a resource bundle named in the form "model_translation.properties". Because it builds on the XSP stack's standard resource-bundle code, it also allows localized versions in the format "model_translation_fr.properties", picking the most appropriate one for the browser. The format of the keys to match the fields is the full class name, then a dot, then the name of the field. So here's a basic "model_translation.properties" File Resource:

model.Note.Posted=Created
model.Note.Body=Note Body

By creating this file, the controller automatically picks up on the new human-readable names:

And this goes for other locales as well. Here's a poorly-translated "model_translation_fr.properties" file:

model.Note.Title=Titre
model.Note.Posted=Créé
model.Note.Body=Corps

And if I switch the browser to use French as its top preferred language (the "Save" button is still the purview of normal XPages translation):

By moving these concerns - data type, validation, and translation - into the data model, it not only significantly lowers the amount of code on the XPage itself, but also makes it much easier to find the canonical rules. Types and validation are always in the model class - not strewn across half a dozen XPages and custom controls - and translations are always in the model translation file and tied directly to the fields they're translating - not hidden in XPage-specific properties files keyed by ID paths.

In the next part, we'll add a bit of UI friendliness to the note creation/editing process using a common messages control and a flashScope object.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

On the scale of "bog-standard" to "weird", the next phase of writing a frostillic.us Framework app sits in the middle: the basic steps of creating a model-editing page are pretty standard, but you can use unique aspects of the Framework to save a bunch of coding and add some nice dynamic features.

Create the editing page

The first step to adding the ability to create and edit notes is to adjust the main note-list page to add a button to actually create one. In my standard layout custom control, I have a facet area named "ActionBar" to add buttons to the top of the page:

<xc:layout navigationPath="/Home">
	<xp:this.facets>
		<xc:linksbar xp:key="LeftColumn" />
		<xp:div xp:key="ActionBar">
			<xp:link themeId="Button.Command" value="/note.xsp" text="New Note"/>
		</xp:div>
	</xp:this.facets>
	
	...
</xc:layout>

I like to use normal links for going to the editing page rather than an "openPage" simple action to avoid the extra round trip. By giving it the "themeId" of Button.Command, it ends up with styling that looks more like a button than a link.

Now that that's over with, we move on to the actual editing page. So we create a page named "note.xsp" and add in our basic surrounding XSP and the data source:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" xmlns:xe="http://www.ibm.com/xsp/coreex" xmlns:xc="http://www.ibm.com/xsp/custom" xmlns:ff="http://frostillic.us/framework"
	beforePageLoad="#{controller.beforePageLoad}" afterPageLoad="#{controller.afterPageLoad}" pageTitle="Note">

	<xp:this.data>
		<ff:modelObjectData var="note" managerName="Notes" key="${empty param.id ? 'new' : param.id}"/>
	</xp:this.data>
	
</xp:view>

You can see a couple Framework-isms here: the "ff" namespace for the model data sources, the "beforePageLoad" and "afterPageLoad" events bound to the implied controller class, and the model data source itself. That data source takes the place of the traditional xp:dominoDocument element when using Framework model objects. It has two important properties: "managerName", which is the managed bean name of the manager class to fetch the object from, and "key", which tells the manager to either fetch an existing object by ID (for Domino-backed model objects, the document UNID) or create a new one. For this, I use the ternary operator in EL to check to see if there was an ID passed in via URL - if so, use that ID; otherwise, tell the manager to create a new model. Think of this as the difference between visiting a xp:dominoDocument-using page with action=editDocument&documentId=XXXX and with action=newDocument.

Now, on to the actual "form" portion. For traditional label/field sets, I'm a fan of using the ExtLib's xe:formTable and associated controls, since it's very focused on declaratively describing the form, and then I can use custom renderers to modify the look as I see fit. So let's toss one of those on the page:

<xe:formTable formTitle="Note">
	<xe:this.facets>
		<xp:div xp:key="footer">
			<xp:button id="save" value="Save">
				<xp:eventHandler event="onclick" submit="true" refreshMode="complete" immediate="false" save="true"/>
			</xp:button>
		</xp:div>
	</xe:this.facets>

	<xe:formRow binding="#{controller.components[note].Title}"/>
	<xe:formRow binding="#{controller.components[note].Body}">
		<xe:djTextarea/>
	</xe:formRow>
</xe:formTable>

So far, so good, right? That is, except for that "binding" business. That's the fruit of my recent dabbling in the potential uses of that property. Think of that as telling the back-end framework code that the two rows represent, respectively, the "Title" and "Body" fields of the given note object specifically, and asking it to fill in the messy details. As a result, the framework looks up an appropriate label for the field (by default, just the name of the field in the model, but it also looks for a human-friendly and translated version, which I'll get to in the next entry), creates an input component if one hasn't been provided, and attaches any appropriate validators, converters, and date/time helpers.

There are two types of use here: for the "Title" field, just a plain text box will do, which is what the controller does in the absence of other direction; for the "Body" field, however, I want to use Dojo's auto-expanding textarea control, and so placing one inside the form row will cause the controller to pick up on it and use the existing control rather than adding a new one. We haven't added any extra validation or type information to the model yet, so the result is basic:

There's one last bit of housekeeping to add: sending the user back to the main home page after saving a note. I use navigation rules for this purpose, and this syntax matches what you get if you set the "Next page (success or cancel)" option in the XPage's properties pane:

<xp:this.navigationRules>
	<xp:navigationRule outcome="xsp-success" viewId="/home.xsp"/>
</xp:this.navigationRules>

And that's it! Other than a few Framework-isms in the areas of the data source and the binding code, the result is pretty similar to a page you'd get using traditional methods. In the next entry, I'll spruce things up a bit: adding in some validation, a different field type, and a dash of translation.

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

The first stage of building a frostillic.us-Framework-based app was a bit unusual - focusing on creating a model object in Java - but the next phase is about as bog-standard as it gets:

Create the view and add it to an XPage

The view we'll need is a basic one showing all Notes. We'll name it "Notes\All" to match the expected prefix in the model and add our two expected columns: Title and Body. The model classes work with sortable columns, so we'll make Title sortable:

Once we have the view, we can refer to it in EL in an XPage as #{Notes.all}. For demo purposes, we'll drop it on the page using a standard xp:viewPanel - not the fanciest presentation, but it'll get the job done:

<xp:viewPanel value="#{Notes.all}" pageName="/note.xsp">
	<xp:this.facets>
		<xp:pager xp:key="headerPager" id="pager1" partialRefresh="true" layout="Previous Group Next" />
	</xp:this.facets>
	
	<xp:viewColumn columnName="Title" displayAs="link">
		<xp:viewColumnHeader value="Title" sortable="true"/>
	</xp:viewColumn>
	<xp:viewColumn columnName="Body">
		<xp:viewColumnHeader value="Body"/>
	</xp:viewColumn>
</xp:viewPanel>

Here, I'm using value="..." to specify the data source. There is also a normal data source for model collections, but it's rarely needed - the EL syntax is much more concise and clear, and it works well with relations (e.g. #{note.responses}). I also specify a pageName="..." for the generated links, since $$OpenDominoDocument URLs won't help here. I may add something to assist with that down the line, but for now specifying the page will do.

Other than those, though: pretty standard stuff. That's one of the goals of the framework: when it makes sense to do things the standard way, I want to do that - no need to reinvent every wheel. So the lists extend TabularDataModel in addition to implementing List and the model objects themselves implement ViewRowData in addition to DataObject.

The next topic - creating an editing page - continues this trend of working with the standard components while cutting out as much hassle as possible.

Now that the framework I've been building is settling into a real project, I figured one of the best ways for me to explain how it works and to double-check my own assumptions is to make a tutorial on how to actually construct an app using it. This will be a very simple one to start with: a note-taking app with just a couple fields and only one business object to speak of.

The overall conceit of the framework is that it uses standard/ExtLib components and idioms where appropriate but declares its own new idioms when the standard ones are deficient. This is done in a way, though, that is opt-in at each step. Though this app is positively stewing in the "Jesse" way of doing things, each component - controllers, models, servlets, etc. - can operate independently and alongside other methods of XPage development.

Probably unsurprisingly, writing a fully framework-based app is heavy on Java and uses some aspects that may be strange at first (annotations, namely), but I hope that the tutorial will explain why I go this route, and that it will demonstrate how using Java in this way can save a tremendous amount of programming and eliminate swaths of potential bugs.

I have a tentative plan for the series, and I'll adjust this list and add links as appropriate:

1. Define the data model
2. Create the view and add it to an XPage
3. Create the editing page
4. Add validation and translation to the model
5. Add notification to the model
6. Add sorting to the view
7. Basic servlet
8. REST with Angular.js

Without further ado, on to part 1:

Define the Data Model

Technically, I should start with part 0: set up the framework. As of this writing, I haven't packaged it up on OpenNTF yet, but I've created a ZIP file containing a current build, the NTF I use to create these apps, as well as the OpenNTF API from my branch - the framework relies on some features not yet released in an API milestone release:

frostillicus.framework-20140709.zip

Requirements-wise, you'll probably need 9.0.1 (the ExtLib that ships with it should suffice) and I haven't tested it on a server without AllPermission granted to all Java.

Now that that's out of the way, on to the actual code. To start with, we'll create the class that represents our core data model - the Note (in the note-taking sense) - and its Manager class. Domino-based model classes in the framework extend the frostillicus.xsp.model.domino.AbstractDominoModel class, while the Managers extend frostillicus.xsp.model.domino.AbstractDominoManager and are kept in the model package:

package model;

import javax.persistence.Table;

import frostillicus.xsp.bean.ApplicationScoped;
import frostillicus.xsp.bean.ManagedBean;
import frostillicus.xsp.model.domino.AbstractDominoModel;
import frostillicus.xsp.model.domino.AbstractDominoManager;

@Table(name="Note")
public class Note extends AbstractDominoModel {



	@ManagedBean(name="Notes")
	@ApplicationScoped
	public static class Manager extends AbstractDominoManager<Note> {

		@Override
		protected String getViewPrefix() {
			return "Notes\\";
		}
	}
}

So! What's going on here? Quite a few things, and just as important are the things that are not going on, such as code to declare all of the fields and each pertinent view's capabilities. Nonetheless, it's still a whole pile of Java concepts - this is the part that's the most unusual from a standard-XPages-dev perspective.

• Right at the start of the class, you see @Table(name="Note"). This is part of my overall attempt to borrow aspects of the Java Persistence API and Hibernate when appropriate. Normally, @Table refers to a SQL table, but I'm using it here to declare the form name to use when creating documents. This may change in future versions.
• There are no getters/setters (or other methods) in the Note class! These objects implement DataObject and so the standard way of accessing data is via getValue(...) and setValue(...). There are ways to add special behavior - complex getters, validation, and types - but the default behavior is to mimic the Domino way of just storing anything with any name.
• The Manager class is stored as what's called an "inner class" in Java. It's a way of storing multiple related classes inside a single .java file. The full name of the class is a combination of the two names - so if you import the Task class in another file, the manager is named Task.Manager.
• I'm using @ManagedBean annotations. These are something I tossed together a while ago based on how managed beans can be done in newer versions of JSF. In essence, these annotations let you specify the name and scope of your beans in-line with the code, without adding them to faces-config.xml. The result acts just the same way.
• The only method specified in the Manager class is to provide a standard prefix for accessing views. I'll demonstrate using that later, but this is specified here to encourage the idiom of naming views like "Notes\All" or "Notes\By Day". It's not required, but following it saves some code elsewhere.
• The Manager is also the place where you can specify an external database to house the data, but I'm not doing that here.

It's a strange bit of code, but my hope is that each line is clear once you understand the concept. My aim is to, as much as possible with Java, make the code in the model classes focus on specifying what the object does and where it's found, not on the messy business of actually retrieving and storing it (while also avoiding the configuration headaches of full JPA/Hibernate).

In the next post in the series, I'll cover adding a view for these documents to the DB and to an XPage.

The core trouble in my eyes with developing on Domino is that it is unloved for what it is. Not so much by customers (I have no interest in whether companies use Exchange or SharePoint or whatever), but more by IBM. The situation reminds me of Apple with products like WebObjects, Xserve, and Aperture: there's clearly a dedicated and talented team behind it, but the organization's heart isn't in it. The analogy is inexact: first, Apple's overarching motivations and strategies are much easier to grok than IBM's; second, enterprise software never really dies - it just goes all COBOL and lingers in maintenance somewhere forever (did you know OS/2 is still a thing?).

So Domino is still around, and still exists, but is generally positioned as something for large companies to use for mail, or continue using if they happened to start doing so decades ago, or as an adjunct to Connections. But it's like with Lotusphere's rebranding: when (dev conference) + (biz conference) = (biz conference), the algebra to figure out the perceived value of (dev conference) isn't difficult. To a certain extent, this is just how IBM does business: they talk to large organizations' higher-ups and make their sales that way, not by being outwardly compelling. However, on this front, Bluemix's entry has provided a telling counter-example: though there's still the stock over-puffed business page, the bluemix.net site talks directly to developers using a layout and writing style that an actual human being might enjoy. They even have honest-to-goodness Developer Evangelists!

It leaves us, as Domino developers, in an awkward position. We have a full-fledged NoSQL server with an integrated Java dev stack, albeit one without a guiding soul. We have an easy-to-install, flawlessly-clustering, cross-platform server that is perpetually 20% away from being perfect for the modern world. Large portions can be charitably characterized as having been in maintenance mode for a long time: the core API, SSL in all protocols, calendar/contacts connectivity, indexing, the admin client, and so forth. The bad news is easy to perceive: less vendor interest and customers driven primarily by inertia make for a poor career path. But the point of this post isn't doom and gloom! The way I figure it, there are a number of bright sides to explain why I and others continue to develop on this platform:

• There's always a chance that Domino will be less "Xserve" and more "Mac Pro", a product seemingly at death's door for years until it was given a fresh breath of life.
• As Paul Graham pointed out years ago, when you're writing server-run software, you can use whichever platform you'd like. Though, all else being equal, it's nicer to have a popular platform, if you can use the tool to do greater work with your time than you would elsewhere, it's worthwhile.
• Due to Domino's nature, almost all of its (technical) problems are solvable even if IBM never touches it again. Some of them, like the API, are things where the community has already built a workaround. Some, like SSL for HTTP, have seen IBM package their own workaround with the server. And the rest, like NIF and CalDAV/CardDAV support, linger as perfect "if I had time" projects.

The last one is crucial for our position: though a totally unsupported problem would eventually fall prey to incompatibility with newer operating systems and machines, Domino has reached a point in the last couple years (thanks to the extensibility API in 8.5.2 and the vital bug-fixes in 8.5.3 and 9) where it's an agglomeration of replaceable parts. There are enough hooks and APIs to - with varying degrees of difficulty - take advantage of the core strengths of the platform while dramatically improving the components on top. That's not enough to last forever, but it doesn't have to be. Apps built in the mean time will still run and improve, and any time spent programming on the platform is valuable experience that can be applied elsewhere, whether directly or more generally.

Just don't ask about licensing.

In a similar vein to my post from the other day, I'd like to discuss one of the other bugaboos that crops up from time to time in the edges of XPages development, usually spoken of in hushed tones: reflection.

If you're not familiar with the term, reflection is an aspect of metaprogramming; specifically, it's the ability to inspect and operate on objects and classes at runtime. Reflection in Java isn't as smooth as it is in other languages, but it makes sense as you work with it.

The most basic form is the getClass() that every non-null object has: you can use it to get an object representing the class of the object in question, which is your main starting point. This code will print out "java.lang.String":

Class<?> stringClass = "".getClass();
System.out.println(stringClass.getName());

From here, you can start to get fancier. For example, this code...

Class<?> stringClass = "".getClass();

for(java.lang.reflect.Method method : stringClass.getMethods()) {
	System.out.println(method);
}

... will print out the signatures for all public instance and static methods, with output like:

public boolean java.lang.String.equals(java.lang.Object)
public int java.lang.String.hashCode()
public java.lang.String java.lang.String.toString()
public char java.lang.String.charAt(int)
public int java.lang.String.compareTo(java.lang.String)
...
public int java.lang.String.compareTo(java.lang.Object)
public final native java.lang.Class java.lang.Object.getClass()
public final native void java.lang.Object.notify()
public final native void java.lang.Object.notifyAll()

There is also a variant to show just the methods from that specific class, including private/protected ones. But the java.lang.reflect.Method objects you get there are useful for more than just printing out signatures: they're proxies for actually executing the method against an appropriate object. To demonstrate, say you have this class:

public class ExampleClass {
	private String name;

	public ExampleClass(final String name) { this.name = name; }

	public void printName() { System.out.println(name); }
}

You can then use reflection to call the method (this also demonstrates an alternate way of getting a class object if you know the class you want):

ExampleClass someObject = new ExampleClass("my name is someObject");
Method printNameMethod = ExampleClass.class.getMethod("printName");
printNameMethod.invoke(someObject);

So, okay, I found a way to call a method that involves way more typing for no gain - so what? Well, in a situation where reflection is actually useful, you won't actually know what methods are available. And this happens constantly in XPages, but you don't have to worry about it: reflection is how EL and SSJS do their work. If you've ever looked through one of the monstrous stack traces emitted by an XPage with an exception, you've probably seen this very call:

java.lang.Exception
    controller.home.getOutput(home.java:18)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
    java.lang.reflect.Method.invoke(Method.java:611)
    com.sun.faces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:96)
	...

That's reflection in action. The EL resolver found the object and then used reflection to determine how it should get the "output" property (most likely the instanceof operator) and then, determining that it was talking to a plain bean, constructed the name "getOutput", asked the object's class for that method, and invoked it.

Like most things in Java, reflection gets more complicated and powerful from here; there are facilities for dealing with generic types, creating new instances, reading annotations, making private properties/methods accessible, and so forth. But the basics are simple and worth knowing: it's all about asking an object what it is and what it is capable of. Once you're comfortable with that, the rest of the pieces will fall into place as you encounter them.

One of the overarching goals for my model framework is to get the job of specifying business logic out of the XPage. So far, most of my work in the area revolved around building in the ability to override getters/setters, establish relationships, and so forth. However, a big problem remains: the XPage still needs to be told things about the data that are really the model's job, not the XSP markup's. Namely, validation.

A little while ago, I "solved" this by adopting Hibernate Validator and building it into the framework. This allows very declarative specification of requirements and also let me build the validation into the model's save event. If a field is marked as required in the model, no matter what the XPage says, the object cannot be saved if the value is empty. So: mission accomplished, technically!

But it's not a great user experience: though I made it so that you get proper Faces messages about what the problem was, the error wasn't tied to the control that caused it, nor could the page do any client-side validation to avoid server round-trips. The solution I found for this and several other problems is, as happens very frequenty, a technical path Tim Tripcony started down a few months ago. Specifically, it's the esoteric "binding" property of each control on the XPage, which is a way of telling the control to set itself as the value of a given binding - say, on the page controller - to provide easy access to it without having to crawl the component tree.

I realized that I could use this with a specially-crafted set of objects to tell the controller and model framework exactly what is going on:

<xp:inputText binding="#{controller.components[task].Summary}"/>

That looks weird at first, but the meaning becomes clear: tell the controller that that text box represents the "Summary" property of the "task" model object. Once that binding happens, the controller can take care of a lot of work that would otherwise have to be done manually:

• Set the value="#{...}" binding
• If the model says the value is required, set required="true", allowing for client-side validation and an Aria attribute to be set
• If the model property specifies another known validation type (say, that it's an email address), add in an appropriate server- and client-side validator. For unknown types, add in a generic validator that will respond to any requirement
• If the property is a Collection type, specify a multipleSeparator property
• If the property is an enumeration and the control is a multi-value control (say, a xp:comboBox or xp:radioGroup), automatically add xp:selectItems for each entry
• Add appropriate converters and other assistant components, such as a date/time picker for Date fields

This could be taken much further, to the extent that there could potentially be a single generic xp:input-type control (or a placeholder CC) and then the controller would construct a control based on the model and client needs - this is a drum NTF has banged before. It could also apply to objects other than my model framework's, but that would take an appropriate adapter for each.

Not only does the XSP code get much trimmer, but the validation and type-specification code in Java is clear and to the point:

public class Task extends AbstractDominoModel {

	@NotEmpty String summary;
	@NotNull TimeFrame timeFrame;
	TaskType type;
	
	public static enum TimeFrame { Normal, Rush, Urgent }
	public static enum TaskType { Normal, Question }

	// ...
}

This feels like a nice step forward in the direction of putting concerns in their right places while also reducing the total amount of code. I'd call that the right kind of victory for app development.

The code for this lives in the framework project inside the XPages Scaffolding repository and the work of the component manipulation is in this class:

https://github.com/jesse-gallagher/XPages-Scaffolding/blob/6d7a9dc59ca070e74a5bd157eb49500b61717e57/frostillicus.framework.plugin/src/frostillicus/xsp/controller/ComponentMap.java