This View Indexer Thing Might Be Worth Some Time

Sun Aug 05 16:51:00 EDT 2012

Tags: java views crazy
  1. Aug 04 2012 - I Went Crazy and Made a Small View Indexer
  2. Aug 05 2012 - This View Indexer Thing Might Be Worth Some Time

I've put a little more time into my small view indexer from the other day, and it seems even more promising now. The editor I made for it might give you some idea of the potential:

Fancy View Builder

Since the view building is all being done in Java, I decided to toss in the list of available JSR-223-compliant scripting languages currently available. The scripting context is given a variable "doc" that's a DocumentWrapper class I wrote that implements Map to make its use a bit easier. For added fun, I baked in knowledge of serialized Java objects stored in the document's items:

Fancy View Objects

Normally, the wrapper just returns the value of getItemValue(...), but when it encounters a MIME entity of type "application/x-java-serialized-object", it deserializes it and returns that instead, allowing for real structured data access.

I also realized that I don't have any reason to stick to only objects available in the stock JDK for index storage. Since using this will require extra Java code anyway, I may as well make my life easier and write my own "view entry" class. Once I get that sorted out, I can work on keeping the indexes updated with a server task, dealing with reader fields, and physical storage.

Hmm, maybe I should make myself an editor for normal views that looks like this one. It'd probably be a lot less hassle to deal with than the legacy one.

I Went Crazy and Made a Small View Indexer

Sat Aug 04 11:45:00 EDT 2012

Tags: java views crazy
  1. Aug 04 2012 - I Went Crazy and Made a Small View Indexer
  2. Aug 05 2012 - This View Indexer Thing Might Be Worth Some Time

Warning: this post is almost entirely pie-in-the-sky nonsense, untethered from reality. Probably.

So yesterday, for some reason, I got to thinking about Domino's view indexing and whether or not it can be readily done better. The current view indexer does its intended task with aplomb (most of the time), but it has its problems. For one, iterating over view indexes in Java (read: everything you should do in Domino today) is dog slow. More theoretically, since it deals only with summary data for performance reasons, its ability to deal with structured data is limited - you can't deal with rich text or, for example, store any sort of Map in a Domino document and deal with it in a view (realistically).

I had the notion that it may be faster, at least in some cases, to do your own indexing: make a Java collection of your "view" data and serialize the result into a note. I decided to try a little test: take a view that consists only of a sorted column of 100k documents' UNIDs and compare it to a TreeSet of the same. My initial results were promising: iterating over the collection of documents (via getNextDocument() and a forall with getDocumentByUNID(), respectively) was about twice as fast with the Java version and was stored in about 17% the space, presumably thanks to document compression.

To be fair, that was really stacking the deck in favor of Java: I didn't care about reader fields or any entry metadata like position, and fetching every document in a large view is possibly the most expensive way to use it. Presumably, fetching just the first entry would make the view version much faster. Still, it shows that it might be useful in some cases, so I decided to try something a bit more complicated.

I made another view with 15k documents (task requests) and two columns: the date of the request, sorted, and the summary line. I matched that up with a TreeSet containing Lists of Maps and iterated over both to fetch and print the entry data. Again, the Java version ended up way faster, taking about 25% of the time.

Now it has me wondering about whether it'd be worth investigating further. It's fair to assume that Java can be treated as the new "native" language of Domino, so serialization is potentially a legitimate mechanism. Furthermore, a real implementation of an alternative view index wouldn't have to be as hopelessly naive as the one I put together: the basic java.util.* collections are almost definitely not the best storage mechanisms for a database index, and, better still, this is well-trodden territory. Since normal views would still exist in this magic future world, these fancy views could eschew a lot of the UI and build-performance requirements of the normal ones in favor of fancy tricks like native knowledge of serialized data structures. Imagine storing structured data like a table of line items in a MIME entity but then being able to query that in a "view". Reader fields could be handled by storing the applicable names in the "entry" and then comparing that two the user's names list at runtime, presumably like standard views do. A server task could handle monitoring document changes and making incremental changes (like happens now, really).

So it's theoretically possible. Would it be worth it? I'm not sure - maybe. It's fun to think about, in any event.

Mixing Dojo's BorderContainer with OneUI

Tue Jul 31 21:18:00 EDT 2012

Tags: xpages dojo

Chris Toohey's post earlier today reminded me of a technique I've been using in some of my apps lately. Since most of what I've been doing has been for internal and admin/reporting-type apps, I've been using OneUI extensively - it's straightforward with the Extension Library, looks pretty good, and the consistency of UI works for that type of application.

However, I ran into a problem with large views: putting a very horizontally-large table into the content area of OneUI breaks it horribly, and there's no good fix. With a lot of tweaking, I made it so it at least scrolls and the header extends the full width, but it was still problematic, particularly when I wanted to add a right sidebar.

The solution I ended up going with was to graft the OneUI look onto a Dojo BorderContainer skeleton, since a BorderContainer takes care of overflowing content nicely. The end result works reasonably well:

/tests/oneui-border.nsf

The layout's code is not pretty, but it doesn't have to be. It provides to its children a couple content facets: the primary content area, LeftColumn, RightColumn, and ActionBar, which is pretty much like a Notes client action bar (my project that uses this takes a lot of cues from an existing Notes app). Additionally, it only shows those extra facets when there's something present, so the sidebars and action bar are entirely hidden when not needed. There are a couple caveats, though:

  • It doesn't provide bindings for every property, as Chris Toohey's does. It certainly could - I just haven't had a need for it.
  • Similarly, I don't have a need for the footer area, so that isn't included. However, it easily could be.
  • It needs some stylesheet patching, and the current stylesheet I use has some flaws in non-Safari browsers (I'm still working on it).
  • Since most of the OneUI structure is written out as HTML, it's not as convenient to tweak as the ExtLib's layout control.

Still, it's been working well for my purposes, and perhaps it'll be useful on its own or as a starting point for other projects.

Faking Sortable View Data

Sun Jul 22 15:28:00 EDT 2012

Tags: xpages data

There's a point in most decent-sized XPages apps I write where I switch from using standard data sources like xp:dominoView to writing Java classes that implement List, usually due to some multi-source data merging I have to do that would be extremely cumbersome or slow to do otherwise. This works well, but one problem is that view data sources have special powers that plain-Jane Lists lack, such as easy sortable column headers.

Having encountered TabularDataModel a bit ago when making an activity stream out of several databases' views, I decided to see how easy it would be to take a fairly standard set of Java data objects - say, a List of Maps, which is conceptually similar to a non-categorized view - and put it into a xp:viewPanel and have all sortable columns. In short: weird, but overall easy.

The TabularDataModel abstract class is very easy to implement: you only need to implement methods for the total number of rows and one to get the current row's data (it uses an internal index accessible via .getRowIndex()). On its own, that's not interesting, since it doesn't get you anything a normal List wouldn't. They key is when you override a couple more methods:

  • .isColumnSortable(String columnName): the xp:viewPanel calls this with a column programmatic name to determine whether it's sortable at all - I made mine return true in all cases.
  • .getResortType(String columnName): once the panel knows a column is sortable, it checks to see what sorting directions it supports. In my case, I return 3, indicating it's sortable both ascending and descending.
  • .getResortState(String columnName): returns the current sorted state of the specified column. Presumably, the column name would be useful if you did custom cascading sorting, which I didn't for now - I just have it return the current sort direction of whatever column is sorted.
  • .getResortColumn(): returns the name of the currently-sorted column.
  • .setResortOrder(String columnName, String sortOrder): this is the interesting one, where the view panel tells the model that it wants to sort a specified column in a specified direction (or "toggle").

When the resort method is called, my class sorts its internal List object by the given column/key name in the given direction via a pretty simple comparator. Once it's all put together, you have the data from the view in a standard format and sortable by all columns.

As usual, there are caveats. First and foremost, there's a reason why you can't normally sort by just any column in a Domino view: Notes was made in 1989 and has barely been improved since then sorting large amounts of data arbitrarily is an expensive operation. It's only practical in this sort of situation because the amount of data is pretty small (only 200 entries) and then initial loading is relatively quick. If you had hundreds of thousands of entries in the view, this would be painfully slow to load and rather greedy with server memory. Additionally, I chose to punt on multi-value columns: I just turn anything that isn't directly comparable (like Vectors) into Strings. Nonetheless, this kind of thing may be practical on its own and is definitely a good exercise in extending your data-manipulation capabilities.

For development, I created a quick database with a couple documents with somewhat randomized data: a date/time value offset by a random amount of hours or days, a random number from -512 to 512, and the document's UNID and Note ID, listed in a view with no sortable columns. The class I wrote just scrapes through the view data with a ViewNavigator and puts each of the entries into a Map. The demo page shows two standard xp:viewPanels (one showing the view directly, the other showing my SortableMapView object) and, on the second tab, the Java code for the three objects I used.

A Prototype In-App Messaging System for XPages

Mon Jul 09 14:35:00 EDT 2012

Tags: xpages java

Sven Hasselbach's post about ApplicationListeners the other day and the notion of post-MVC methods of app architecture got me thinking about the notion of generic messaging/events inside an XPages app. The first example that comes to mind is a project-tracking database where there may be events like "new task created" or "delivery ready for review" - business logic stuff where the fact that it's a document being modified is an implementation detail. In my actual project-tracking database at work, I implemented this by creating "Notification Stub" documents with information about the event and having a scheduled agent process them. That's a fine Notes-y way to do it (and has its own advantages, like having another cluster member send the emails), but I want a way for the running XPages app to notice and react to these things without the immediate code knowing about every single concern.

I did a quick search and didn't find any such capability in basic JSF or XPages, but I realized it wouldn't be terribly difficult to implement it myself: have an application-scoped bean (maybe one per scope, if it's useful) with a methods to dispatch events and declare listeners. Then, other beans could attach themselves as listeners and the normal code would only be responsible for sending out its messages, to be received by zero or more unknown objects:

public interface XPagesEventDispatcher {
	public void dispatchEvent(XPagesEvent event);
	public void addListener(XPagesEventListener listener);
}

The actual event objects would be similarly simple: just a name and an arbitrary array of objects:

public interface XPagesEvent {
	public String getEventName();
	public Object[] getEventPayload();
}

Finally, listeners are the simplest of all, being responsible only for providing a method to handle an XPagesEvent (EventListener is the standard "tagging" interface for this kind of thing):

public interface XPagesEventListener extends EventListener {
	public void receiveEvent(XPagesEvent event);
}

However, after creating my gloriously clean set of interfaces, I ran into the ugly marsh of reality: application-scoped beans only exist after first use. That works fine for the dispatcher itself, but not for listener beans, the whole point of which is to not be referenced in code elsewhere. After a bit of searching, I came up with a solution that may work. When defining a managed bean, you can set properties, and those properties can be Lists. So, rather than creating two beans, I define just the dispatcher bean, but provide it a list of class names of listener beans to create at startup:

<managed-bean>
	<managed-bean-name>applicationDispatcher</managed-bean-name>
	<managed-bean-class>frostillicus.event.SimpleEventDispatcher</managed-bean-class>
	<managed-bean-scope>application</managed-bean-scope>
	<managed-property>
		<property-name>listenerClasses</property-name>
		<list-entries>
			<value-class>java.lang.String</value-class>
			<value>frostillicus.Messenger</value>
		</list-entries>
	</managed-property>
</managed-bean>

I gave my dispatcher implementation class a convenience methods to take a string and 0 or more objects and dispatch the event, so it could be used in code like:

applicationDispatcher.dispatch("home page loaded")
applicationDispatcher.dispatch("delivery submitted", [delivery.getUniversalID()])

So far, this setup is working pretty well. I plan to toy with it a bit and then try it out in proper use. Provided it works well (and it doesn't turn out that this capability already exists and I just missed it), I'll turn it into a project on GitHub/OpenNTF. As long as it keeps working, I'm excited about the prospects, especially when combined with XPages Threads and Jobs for non-blocking event handling.

Using Lombok to Write Cleaner Java in Designer

Fri Jul 06 18:48:00 EDT 2012

Tags: java

Java, as a language, has a number of admirable qualities, but succinctness is not among them. Even a simple "Person" class with just "firstName" and "lastName" properties can boil over with at least a dozen lines of boilerplate to add constructors, getters, setters, and equivalency testing. Fortunately, Project Lombok can help. Their main page contains a good video of the basics, while the Feature Overview page covers the rest. If you've written a lot of Java classes, I suspect that the video will have you hooked.

And there's good news: Lombok can work in Designer! It's not as simple as it is for vanilla Eclipse and it took me a good amount of trial-and-error to get it functional, but I figured out what you have to do:

  1. Copy lombok.jar into your Notes program directory (e.g. C:\Program Files\IBM\Lotus\Notes). It has to go there so that the initial program launch sees it.
  2. Additionally, copy the same lombok.jar to your jvm\lib\ext directory inside your Notes program directory. It has to go there so that it's on your project build path. Yes, normally you "shouldn't" put your Java libraries there, but I figure it's alright since this takes local, non-update-site fiddling anyway.
  3. Now for the weird part. Normally, the Lombok installer modifies eclipse.ini to launch some support stuff. That won't work for us, however. What we need is the file "jvm.properties" in the framework\rcp\deploy directory. The syntax isn't the same as for eclipse.ini and I'm not 100% sure my method doesn't have any horrible side effects, but adding these two lines (the two under "Lombok stuff", with "vmarg.javaagent" and "vmarg.Xbootclasspath/a") worked for me:
    vmarg.javaagent=-javaagent:lombok.jar
vmarg.Xbootclasspath/a=-Xbootclasspath/a:lombok.jar

Now, I've only had this installed for a couple hours and have only done a couple basic tests, so I can't rule out the notion that this will interfere with some other Designer thing (the "Xbootclasspath/a" line may override some Designer default), so use with care. On the plus side, Lombok's magic works during compilation, not runtime, so you shouldn't need lombok.jar on the destination server. You'd still need it on the Designer client of any other programmers working on the code, though.

Update: I think you can avoid doubling up your copies of lombok.jar (and avoid a problem where launching the normal Notes client doesn't work) by just putting lombok.jar in jvm/lib/ext and using these lines in jvm.properties instead:

vmarg.javaagent=-javaagent:${rcp.home}/../jvm/lib/ext/lombok.jar
vmarg.Xbootclasspath/a=-Xbootclasspath/a:${rcp.home}/../jvm/lib/ext/lombok.jar

Basic Org Charts With xe:navigator and xe:beanTreeNode

Fri Jul 06 09:30:00 EDT 2012

The topic of generating org charts on an XPage came up recently, and I decided to try my hand at making a pretty basic one using a technique similar to my re-use of Outline design elements from the other week. Not only are the generated hierarchies potentially useful, but it ended up providing a much cleaner example of how to recursively generate an Extension-Library tree.

To get started, I mocked up some basic data: I created a standard Domino directory database and created users with unique ShortName values and the ShortName of their immediate boss in the Manager field, with anyone at the top level having a blank field. Then, because I'm too lazy to look through the existing views to see if one does it, I created a "Managers" view that simply shows Person documents categorized by their Manager field (with a bit of computation to convert self-referential documents to top-level). You can see the live result in the test database, along with applicable code excerpts:

http://frostillic.us/namestest.nsf

The algorithm itself provides a nice exercise in recursion:

  1. Get a list of all top-level managers
  2. For each of those names:
    1. Find their child count to determine whether or not they're a leaf
    2. Create an outline entry with their name
    3. For each child entry, repeat this process

This could potentially be modified to query an LDAP server instead of reading a view directly, which would make it more flexible (and complicated), and the output could be modified or re-used for other visual representations, as needed.

The absolute most important thing when doing recursion, though, is to make sure you listen to music with an appropriately-nerdy title, such as:

Using dojo.behavior

Tue Jul 03 07:56:00 EDT 2012

Tags: dojo

JavaScript in a browser is a messy business. While you can generally avoid writing a lot of client JavaScript when doing XPages development, it will eventually be necessary to get your hands dirty. While a bit of XSP.openDialog() here and there, things can get hairy when you want to start including the same or similar code everywhere. You can move the code blocks themselves off to a script library, but then you're still stuck including function calls inline.

Enter dojo.behavior. I ran across dojo.behavior when I was looking for a replacement for event:Selectors, which I'd been using for years in my non-XPages web sites to patch up Domino's crummy HTML after the fact. Like event:Selectors, dojo.behavior allows you to write JavaScript blocks that are associated at runtime with elements on the page using CSS-style selectors. Possibly the best way to get an idea for how you can use it is to look at a simple (and, as usual, contrived) example:

dojo.behavior.add({
	"#linksbar a": {
		found: function(a) {
			a.target = "_blank"
		},
		onclick: function(thisEvent) {
			alert("I have been clicked!")
		}
	}
})
dojo.behavior.apply()

As you might suspect from reading it, that code looks for links inside an element with the HTML ID "linksbar" and sets them to open in a new window and, when clicked, display an alert. While you could have easily put both of these directly in the HTML/XPage source, just think about all the other things you'd potentially want to do with client JavaScript: fancy hover actions, custom expand/collapse effects, and any kind of post-render patching. It all adds up, and separating the layout from the JavaScript behavior like this can save a lot of debugging and backtracking headaches.

To actually use dojo.behavior in an XPages app, make sure to include the "dojo.behavior" Dojo module somewhere on your page, either via the XPage or CC's resources or via a theme. Then, create a JavaScript Script Library to house your behavior. I generally start with a skeleton similar to this (with the "body" function just to make sure it's set up correctly):

var DojoBehavior = {
	"body": {
		found: function() { alert("Behavior working") }
	}
}

dojo.ready(function() {
	dojo.behavior.add(DojoBehavior)
	dojo.behavior.apply()
})

Then put your code inside that first "DojoBehavior" object. Finally, include the Script Library in your page or theme and you should be all set.

Book Review: XPages Extension Library

Mon Jul 02 19:10:00 EDT 2012

Tags: xpages review

Unlike many development environments, there are only a handful of books about XPages. Combined with the historical lack of documentation, that makes all three - Mastering XPages, XPages Portable Command Guide, and XPages Extension Library - essential. In that sense, the quality of the book is a less important purchasing-decision factor than its mere existence.

Fortunately, XPages Extension Library doesn't skimp on quality or breadth of coverage. The sections of the book track the natural progression of someone new to the Extension Library: how to acquire and install it, how to use its upgraded application-development features like the application layout and slicker Dojo controls, how to do the entirely-new things in the ExtLib like REST and JDBC, and finally a much-needed guide to starting "proper" Java development in XPages.

Since reading through the book in sequence, I have cracked it back open both as an indexed reference and for several of its step-by-step guides.

Part 2, which covers the various application UI components, has proven to be very useful when I have a specific UI task in mind, such as using a particular Dojo layout element or implementing some of the fancy, OneUI-friendly elements like the dataView. The organization of the chapters and code examples provide a much-needed sense of order to the bevy of new controls that you find in Designer's sidebar after installing the Library. And even just reading through the chapters in order is bound to give you a wealth of new UI ideas for your day-to-day use.

Several chapters provide invaluable guides for accomplishing complex and potentially daunting tasks, such as the ideal OSGi-based installation of the library to your servers, setting up a OneUI layout for your app (replete with an explanation of facets and callbacks, which are useful in XPages development generally), and packaging JDBC drivers for OSGi deployment. The last task in particular is almost comically opaque process for anyone thoroughly versed in the peculiar Eclipse way of doing things, and the book provides a clear step-by-step guide on negotiating that thicket.

As anyone who follows this blog can probably surmise, I think that pretty much everyone who programs with XPages would be well-served by familiarizing themselves with Java, and so I'm pleased that the book contains a smooth introduction to how to and why you would write Java classes in your app. This chapter does a fine job explaining how to use already-existing classes and packages, how to set up your Designer environment to make Java development easier, what a "bean" is (it's simpler than you'd think), and how to create and make use of managed beans. It's a great XPages-focused guide to starting down what should be a very fruitful road learning Java and the XPages Java environment.

The last word on this book is simple: if you do XPages development, you should purchase and read this book. XPages development with the Extension Library is much more productive than without, and XPages Extension Library provides an easy-to-read explanation to get your head around its myriad capabilities.

Use Interfaces All The Time

Wed Jun 27 09:10:00 EDT 2012

Tags: java

In addition to being useful concepts generally, Java interfaces can be used literally in code as a way of keeping your code as clean and generic as possible. While you can't ever create a new object based on an interface, you CAN use interface names as object types for variables and parameters. For example, this is a legal way to create a Vector of Objects:

List<Object> someList = new Vector<Object>();

"That's great and all," you say, "but why bother doing that?" And indeed, in a simple case like a mostly-procedural Java agent, you won't get much benefit from doing that. But imagine a method like this:

public List<String> retrieveValuesFromDatabase();

Since all that method guarantees is that it's going to return some sort of List, it has a lot of discretion as to the form that list will take. Maybe the programmer starts out with Vector but then decides that ArrayList is better for the purpose - they're free to change it without troubling the user of the method one bit. Maybe they want to get a little crazier and make their own custom class that implements List and provides a live view of a database - again, that can happen with no change to the user's code.

While the primary benefit comes when large libraries use interfaces in their APIs, even moderately-sized structured programs written by one programmer can benefit. Take a (contrived and unsafe) class like this:

public class ExampleClass {
	private List<String> cache;

	public ExampleClass() {
		this.cache = new ArrayList<String>();
	}
	public List<String> getCache() {
		return this.cache;
	}
	public void setCache(List<String> cache) {
		this.cache = cache;
	}
}

The same benefits mentioned above apply here: because you're referring to your instance variable just as a List, you're free to change the actual implementing class by changing only one line. The benefits are only compounded when you add more and more-complicated methods to your class.

Additionally, using interfaces when they're not strictly necessary can help avoid bad habits and traps. Our go-to example, Vector class, pre-dates the Java Collections Framework and was only retrofitted to the List interface when the framework came out in Java 1.2. It shows its age in a couple ways, not the least of which is the inclusion of a couple methods it uses that are not part of List, such as elementAt(...) and addElement(...). These are best replaced entirely with get(...) and add(...), which are common to all Lists... and are easier on the eyes, to boot.

It's easy to run into problems with APIs that don't use interfaces extensively, like, say, the Domino API*. Presumably due to the fact that the "new" lotus.domino classes were implemented in a pre-1.2 JRE, they use Vector extensively. Most of the time, this is fine - anything expecting a List will happily accept a Vector, but the reverse doesn't hold. You don't have to go far to create a problem. Multi-value controls in XPages, when bound to a Java object, prefer the use of ArrayList (but work great when pointed to an existing List of any stripe). As a result, this will result in an exception:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core">
	<xp:this.dataContexts>
		<xp:dataContext var="newRegData" value="${javascript: new java.util.HashMap()}"/>
	</xp:this.dataContexts>
	
	<xp:checkBoxGroup value="#{newRegData.multi}">
		<xp:selectItems value="#{javascript:['one', 'two']}"/>
	</xp:checkBoxGroup>
	
	<xp:button id="createDoc" value="Create Doc">
		<xp:eventHandler event="onclick" submit="true"><xp:this.action><![CDATA[#{javascript:
			var doc = database.createDocument()
			doc.replaceItemValue("MultiValue", newRegData["multi"])
			doc.save()
		}]]></xp:this.action></xp:this.eventHandler>
	</xp:button>
</xp:view>

The problem is that, while ArrayList is still a List, replaceItemValue(...) doesn't give two hoots about interfaces, forcing you to do something like this:

var vec = new java.util.Vector()
vec.addAll(newRegData["multi"])
doc.replaceItemValue("MultiValue", vec)

Not pretty, and that's one that could be fixed without changing the outward API.

The upshot of all this is that you can derive a lot of benefit from using interfaces extensively, even when they're not strictly necessary. I even have a habit of writing lines like "List<Object> values = (List<Object>)doc.getItemValue("SomeItem");", because I am a madman. You may not go as far as I do, but it's still usually a good idea to follow the policy of using interfaces whenever the methods you want are available that way.

* Technically, the the lotus.domino.* "classes" are all interfaces and the implementing classes vary based on the type of connection you're using (e.g. local vs. CORBA). As usual, Domino provides both good and bad examples simultaneously.