Putting java.util to Use

Fri Jun 22 11:46:00 EDT 2012

Tags: java

I'm in the process of figuring out a good way to combine data from several sources into a single activity stream, which means that they should be categorized by date and then sorted by time. While that's a piece of cake with a single view, it gets hairy when you have several views, or perhaps several different types of sources entirely. Fortunately, abstract data types are here to help.

You're already using Lists and Maps, right? For this, I decided to use Maps and one of my personal favorites, the Set. If you're not familiar with them, Sets are like Lists, but only contain one of each element and don't (normally) guarantee any specific order - DocumentCollections are a type of Set (albeit not actually implementing the interface).

I created the categories by using a Map with the Date as the key and Sets of entries as the value. That would work well enough using HashMaps and HashSets, but they would require manual sorting in the XPage to display them in the right order. Fortunately, Java includes some more-specific types for this purpose: SortedMap and SortedSet. These are used the same way as normal Maps and Sets, but automatically maintain an order (based on either your own Comparator or the "natural" ordering based on the objects' compareTo(...) methods). Better still, the specific TreeMap and TreeSet implementation classes have methods to get at the keys and values, respectively, in descending order.

Once I had my collection objects picked out, all I had to do was start filling them in. I used stock Date objects for the Map's keys and wrote a compareTo(...) method for the entries I'm keeping in the Set. Then, on the containing activity stream class, I just had to write a "serializer" method to write out the current state of the objects into a List for access.

While I may change around the way I do this (I may end up putting the category headers inline so I just use one SortedSet for performance), it provides a pretty good example of when you can use some of Java's built-in classes to do some of the grunt work for you.

Re-using Classic Domino Outlines, Rough Draft

Tue Jun 19 17:34:00 EDT 2012

Tags: xpages

My current big project at work involves, among many other things, viewing individual project databases through a centralized "portal". These all use the same base template, but can be individually customized, usually with new or changed views. Additionally, I'm going to be granting web users varying roles that correspond to existing or planned access roles in the target database. The result is that I'm spending a good amount of time trying to dynamically adapt some classic Notes elements to show up on XPages (hence my DynamicViewCustomizer, which I've since updated and should post).

In addition to views, I wanted to use the existing Outline design elements in the databases, since they do a fine job and there's no reason to re-invent the wheel if it's not necessary. I looked around a bit and didn't see a way to readily re-use them (though I can't rule out the possibility that such a capability exists somewhere), so I decided I'd roll up my sleeves and write my own adapter.

The route I took was to use the xe:beanTreeNode ExtLib control inside the xe:navigator used in most OneUI apps. Though what I conceptually wanted to do was to feed it a tree structure made up of JavaScript arrays and nodes, that's not how it works: you pass it the name of either an already-created managed bean or the full class name of a bean-compatible (read: zero-argument constructor) implementing ITreeNode. Presumably, the right way to do this would be to create a managed bean and pass in configuration parameters via managed-property values. I didn't do that for the time being, though, instead baking a bit of environmental awareness into the Java code - so sue me. I also print stack traces to the server console, which you shouldn't do.

In any event, what my code does is to open the consistently-named outline design element (in this case, "TempOut" for "template outline"), reads through its entries, and recursively builds the node tree. There are a couple environmental assumptions in there, primarily in the constructor, in order to work with the surrounding app, but the core of it should be pretty transportable. One big note is that I haven't gotten around to adding in the "special" outline entries to show the remaining views and folders, nor do I handle "action" entries. Still, it may be a useful reference for using bean tree nodes.

package mcl;

import com.ibm.xsp.extlib.tree.ITreeNode;
import com.ibm.xsp.extlib.tree.impl.*;
import lotus.domino.*;

import javax.faces.context.FacesContext;
import java.net.URLEncoder;

public class ProjectOutline extends BasicNodeList {
	private static final long serialVersionUID = 2035066391725854740L;
	
	private String contextQueryString;
	private String dbURLPrefix;
	private String viewName;

	public ProjectOutline() throws NotesException {
		try {
			ProjectInfo projectDatabase = (ProjectInfo)JSFUtil.getVariableValue("projectDatabase");
			if(projectDatabase.isValidDB()) {
				Outline tempOut = projectDatabase.getOutline("TempOut");
			
				dbURLPrefix = "/__" + projectDatabase.getReplicaID() + ".nsf/";
				contextQueryString = String.valueOf(JSFUtil.getVariableValue("contextQueryString"));
				viewName = (String)FacesContext.getCurrentInstance().getExternalContext().getRequestParameterMap().get("viewName");
				
				OutlineEntry entry = tempOut.getFirst();
				while(entry != null) {
					entry = processNode(projectDatabase, tempOut, entry, null);
				}
			} else {
				addChild(new BasicLeafTreeNode());
			}
		} catch(NullPointerException npe) {
			addChild(new BasicLeafTreeNode());
		} catch(Exception e) {
			e.printStackTrace();
		}
	}
	
	
	private OutlineEntry processNode(ProjectInfo projectDatabase, Outline outline, OutlineEntry entry, BasicContainerTreeNode root) throws NotesException {
		int level = entry.getLevel();
		
		switch(entry.getType()) {
		case OutlineEntry.OUTLINE_OTHER_UNKNOWN_TYPE:
			// Must be a container
			
			BasicContainerTreeNode containerNode = createSectionNode(entry);
			if(root == null) {
				addChild(containerNode);
			} else {
				root.addChild(containerNode);
			}
			// Look for its children
			OutlineEntry nextEntry = outline.getNext(entry);
			while(nextEntry.getLevel() > level) {
				nextEntry = processNode(projectDatabase, outline, nextEntry, containerNode);
			}
			
			return nextEntry;
		case OutlineEntry.OUTLINE_TYPE_NAMEDELEMENT:
			View view = projectDatabase.getView(entry.getNamedElement());
			if(view != null) {
				ITreeNode leafNode = createViewNode(entry);
				if(root == null) {
					addChild(leafNode);
				} else {
					root.addChild(leafNode);
				}
				view.recycle();
			}
			
			return outline.getNext(entry);
		}
		
		return outline.getNext(entry);
	}
	
	private BasicContainerTreeNode createSectionNode(OutlineEntry entry) throws NotesException {
		BasicContainerTreeNode node = new BasicContainerTreeNode();
		node.setLabel(entry.getLabel());
		node.setImage(dbURLPrefix + urlEncode(entry.getImagesText()) + "?Open&ImgIndex=1");
		
		return node;
	}
	private ITreeNode createViewNode(OutlineEntry entry) throws NotesException {
		BasicLeafTreeNode node = new BasicLeafTreeNode();
		node.setLabel(entry.getLabel());
		node.setImage(dbURLPrefix + urlEncode(entry.getImagesText()) + "?Open&ImgIndex=1");
		node.setHref("/Project_View.xsp?" + contextQueryString + "&viewName=" + urlEncode(entry.getNamedElement()));
		
		node.setSelected(entry.getNamedElement().equals(viewName));
		node.setRendered(resolveHideFormula(entry));
		
		return node;
	}

	private String urlEncode(String value) {
		try {
			return URLEncoder.encode(value, "UTF-8");
		} catch(Exception e) { return value; }
	}
	private boolean resolveHideFormula(OutlineEntry entry) throws NotesException {
		String hideFormula = entry.getHideFormula();
		if(hideFormula != null && hideFormula.length() > 0) {
			Session session = JSFUtil.getSession();
			Database database = entry.getParent().getParentDatabase();
			Document contextDoc = database.createDocument();
			// @UserAccess gave me trouble, so I just did a simple string replacement, since I know that's the only way I used it
			hideFormula = hideFormula.replace("@UserAccess(@DbName; [AccessLevel])", "\"" + String.valueOf(database.getCurrentAccessLevel()) + "\"");
			double result = (Double)session.evaluate(hideFormula, contextDoc).get(0);
			contextDoc.recycle();
			return result != 1;
		}
		return true;
	}
}

Reverse-Engineering File Formats for Fun

Mon Jun 11 09:00:00 EDT 2012

Tags: java

Well, okay, it wasn't for fun; it was for work. About a year and a half ago, I had occasion to parse the contents of shared object files from a Flash Media Server ("*.fso" files), and I figured it might be interesting to go back over the kind of things I had to do to accomplish that.

The first thing I did was to search around to see if anyone else had solved the same problem. However, while there are plenty of parsers for "Flash shared objects", not the least of which is in Flash itself, it seemed that, unless I was doing it wrong, that format is different from the one used by servers, so I couldn't use any of the ones I found. So I was left with a folder full of binary files and a task to accomplish. Sounds like a job for programming!

Fortunately, I knew the shape of the data inside the files: they were essentially arrays of maps. I could see the keys and some of the values in there, so the files were binary but not compressed, which gave me hope. However, since the data was variable-length, I couldn't just find the record delimiter and use offsets - I had to go through and find the various byte-value delimiters for each aspect and write a proper parser.

I opened up a couple of the files in vi sessions to compare them and, using the handy ga command, checked the byte values of the non-ASCII characters. I found that the file always starts with 0 then 3, and there appeared to be a "general" delimiter of 0, 0, 0 to break up header sections of the file and each record. Then I saw a number that was different between each file - ah-hah, the record count!

After another delimiter, I found a number followed by the name of the file (for some reason). It turned out that the number corresponded to the length of the file name, indicating that the format uses Pascal-style length-prefixed Strings. Good to know! After the name came a second instance of the record count for some reason, leaving the rest of the file to the actual records.

Conceptually, the records are easy: there should be about $record_count instances of some sort of record delimiter with the data packed in there in, essentially, key-value pairs. That's ALMOST how it is, but it got a bit weird: all records were ended with the same "general" delimiter used in the header, but they ALSO had their own two-byte record delimiter. That is, except when they used a three-byte variant for some reason. And there are a couple unknown values in there - they were different in each record and I'm sure they serve some use, but I couldn't figure out what it was. Once I figured out the hairy bits, though, the data was pretty much as I expected: there was a record index (for some reason), then the expected key-value pairs. Most of them were strings, but not all - there was an "urgent" key in some entries that had no corresponding value (though now that I back look at the code, I suspect that it was a boolean value of \1 to indicate "true").

The end result of all this was by far the most C-like Java I've ever written:

FSOParser.java

I felt pretty good after finishing that. High-level object tress are fun and all, but it's good to, from time to time, get your hands dirty with lower-level stuff like reading files as integer arrays.

PS: I don't know why my FSORecord class didn't just extend HashMap. I blame the brain haze from staring at binary files for too long. Conversely, I think I did have a reason to use arrays of int instead of byte - I think it had to do with Java byte being always signed but the data in the file being unsigned.

Ruby Builder First Draft: Intriguing Failure

Sat Jun 09 09:56:00 EDT 2012

Tags: ruby

For a while now, I've been fiddling with trying to make an Eclipse builder that smoothly translates Ruby files into Java classes and adds them to the project to be compiled, the idea being that, rather than only using Ruby inline in XPages or via "script libraries", you'd be able to write all of your supporting Java classes in it as well.

I'd been giving it about an hour or so of frustration every couple of weeks, but yesterday I decided to hunker down and make it work. After patching the standard "jrubyc" code to work without a real filesystem, wrestling with Ant builder class paths and runtime environments, and with a great deal of assistance from the handy XPages Log File Reader template, I got it working! I now have it so that you can write Ruby files in a ruby-src folder and build the project and the builder reads those files and plants "compiled" Java versions into a folder on the build path, which then causes Eclipse to build them into proper classes, usable on an XPage. Whoo!

However, they're not really usable yet. The way the "compiler" works is that it generates a class that extends RubyObject and, in a static block, loads up the global Ruby runtime and pre-compiles a giant string of Ruby. Then, each method you exposed to Java in the Ruby code calls the equivalent method in the Ruby runtime. It makes sense, but leads to some specific problems. For one, the global Ruby runtime's classloader doesn't know about the classes available in the XPages environment, such as the javax.faces.* classes and any of your own NSF-hosted ones. Moreover, because the object extends RubyObject and Java doesn't do multiple inheritance, it can't extend any OTHER Java classes. The internal Ruby class can extend whatever it wants and works well, but that doesn't help when you want to use the generated classes in Java code. It can implement Interfaces, but then you have to actually have Java-facing versions of every method, which can get hairy.

I have some ideas, though. The "jrubyc" compiler is handy, but it doesn't work any special magic - it just reads through the Ruby source for key elements like java_implements and java_signature and uses those to build a wrapper class that just executes a big string of Ruby code. There's nothing there I couldn't do myself, hooking into the existing parser where necessary and otherwise writing the rest myself. That way, I could generate a standard Java class on the outside but make it create a proxy object internally that's the actual RubyObject that handles all of the method calls. It could be a bit more difficult than I'm thinking, but it'd probably be possible, and it'd let me use the runtime classloader from FacesContext and extend classes properly at will.

XPages MVC: Experiment II, Part 4

Mon Jun 04 19:57:00 EDT 2012

Tags: xpages mvc
  1. May 22 2012 - XPages MVC: Experiment I
  2. May 23 2012 - XPages MVC: Experiment II, Part 1
  3. May 24 2012 - XPages MVC: Experiment II, Part 2
  4. May 28 2012 - XPages MVC: Experiment II, Part 3
  5. Jun 04 2012 - XPages MVC: Experiment II, Part 4

To finish up my series on the infrastructure of my guild forums app, I'd like to mention a couple of the down sides I see with its current implementation, which I'd generally want to fix or avoid if re-implementing it today.

Roll-Your-Own

One of the strengths of this kind of MVC setup is that it works to separate the front-end code from the data source. It would be (relatively) easy for me to replace the model and collection classes with versions that use a SQL database or non-Notes document storage should I so choose. That's a double-edged sword, though: because I'm not using any of the built-in data sources and controls with Domino knowledge, I had to do everything myself. This means a loss of both some nice UI features - like the rich text editor's ability to upload images inline - and the XPage data sources' persistence and caching features.

The collection code deals with View and ViewEntryCollection classes directly, but they can't be serialized, so I had to write my own methods to detect when the object is no longer valid (say, when doing a partial refresh) and re-fetch the collection. This was good in the sense that I learned more about what is and is not efficient in Domino. For example, getNthEntry(...) on a ViewEntryCollection grabbed via getAllEntriesByKey(...) is fast. Conversely, while retrieving data in a view is usually significantly faster than getting the same data from a document, there's a point where the view index size is large enough that, provided you're fetching only a few documents at a time, it's better to use the document. With a lot of work (and a LOT of collection caching), I ended up with something that's quite fast... but since I wrote all the code myself as part of a side project, it was also pretty bug-prone for the first couple weeks after deployment.

Maybe I'd be able to find ways to piggyback more on the built-in functionality if I re-did it, but, as it stands, it's kind of hairy.

Inconsistent Separation

This isn't a TERRIBLE problem, but I'm kind of annoyed with some of the inconsistent choices I made about where DB-specific code goes. For example, collection managers have very little Domino-specific code... except when they need to know about sorting and searching. Similarly, almost all of the code in the model objects deals with pure Java objects and the clean collections API... except the save() methods, which are big blobs of Domino API work. That makes some sense, but the model classes don't handle creation from the database - that's in the collection classes. Like I said, it's not the end of the world, and it's consistent within its own madness, but there are some weird aspects and internal leaks I wouldn't mind cleaning up.

Lack of Data Sources

This is sort of the flip side to my first problem. All of my collection managers are just objects and the collections are just Lists. This is good in the sense that these things work well in Server JavaScript and with all of the built-in UI elements, but it'd be really nice to write my own custom data sources so I can explicitly declare what data I want on the page - using a xp:dataContext or bit of JavaScript in a value property just feels "dirty", like I'm not embracing it completely. However, this problem isn't as much an architectural one as it is a lack of education - I haven't bothered to learn to write my own data sources yet (even though I expect it's more or less straightforward, for Java), so I could remedy that easily enough.

No Proper Controller

This is just another manifestation of the root cause that has me looking into all this MVC stuff to begin with. Even though my collections and models are better than dealing with raw Domino objects, there's still too much of a tie between the UI and the back-end representation, as well as the requisite dependence on the Domino HTTP stack to handle routing requests. Of course, I'm still working on the correct solution to this.

 

Overall, my structure as written has been serving me well. New data elements are pretty easy to set up - I wouldn't mind not having to write three classes per, but hey, it's Java - and working with them is a breeze. Though it took a while to get everything working, now that it is, I can make tons of UI changes without worrying much about the actual data representation. I can change the way data is stored or add on-load or on-save computation beyond the capabilities of Formula language without changing anything in the XPages themselves. So in those senses, my forum back-end code is a huge step up from doing it directly xp:dominoDocuments, but it still doesn't feel completely "right".

Ruby for XPages Programmers, Part 1

Wed May 30 22:54:00 EDT 2012

Tags: ruby xpages
  1. May 30 2012 - Ruby for XPages Programmers, Part 1

Since I'm always gabbing on about Ruby, I figured it'd be useful to give an overview of the language and, in future posts, why it's worth using for XPages scripting. I don't plan to write an exhaustive tutorial for the language (they're aplenty on the web), but I'll go over the basic concepts and some of how it compares to EL, Server JavaScript, and Java.

Ruby is a dynamic, object-oriented language with functional-programming features and syntax designed for programmer-friendliness. In terms of other languages, it's sort of like a combination of Perl's general syntax, Smalltalk's object model, and (some of) Lisp's functional and metaprogramming abilities. Additionally, Ruby has its own unique features and conventions, such as optional parentheses, property-style getters and setters, and yield statements.

Maybe the best way to provide a gentle introduction to Ruby is to show a relatively simple but contrived code example. Here is a "Point" class meant to represent a point in two-dimensional space where the X coordinate must always be positive:

class Point < Object                                           # 1
    attr_accessor :y                                           # 2
    attr_reader :x
	
    def initialize(x=0, y=0)                                   # 3
        @y = y                                                 # 4
        self.x = x                                             # 5
    end
	
    def x=(x)                                                  # 6
        raise Exception.new("x must be >= 0") if x < 0         # 7
        @x = x
    end
	
    def to_s; "[#{@x}, #{@y}]"; end                            # 8
end

There are a lot of Ruby-isms going on, but hopefully the bulk of the code will be readable without knowing the language. Starting from the top, with a focus on the differences from Java:

  1. Classes are created with just class, with no public modifier, and must begin with a capital letter, which is the language-enforced convention for class names and constants. By unenforced convention, classes and constants are CamelCase. The "<" is equivalent to Java's extends. As in Java, extending Object is not needed, but is done here for demonstration purposes. Finally, # is Ruby's single-line comment delimiter, like // in Java or ' in LotusScript.
  2. attr_accessor and attr_reader are built-in Ruby class methods that generate getters+setters and getters only, respectively. The ":"s in :x and :y indicate that those are Symbols. Symbols are interesting beasts, but in this case think of them as meaning "the things named 'x' and 'y'".
  3. The initialize method is equivalent to Java's constructors and LotusScript's New. This one takes two parameters, but provides default values in case one or both are left out. While Ruby objects are strongly typed in that they all have a class, Ruby variables are typeless and not statically checked.
  4. The "@" symbol is used to denote instance variables, which, like all Ruby variables, don't need to be declared before they're used the first time. In this case, the line is equivalent to "this.y = y;" in Java.
  5. "self.x = x" is used here to indicate that the first "x" is calling the current object's x= method as opposed to setting the local variable "x" to itself. This line is equivalent to "this.setX(x);" in Java.
  6. Ruby, like C++, allows operator overloading, including this special type to define setters. This allows setting of the object's "x" parameter with code like "some_point.x = 3".
  7. This line is jam-packed!
    • Ruby's exceptions are essentially like Java's, except they use raise and begin/rescue instead of throw and try/catch.
    • Objects in Ruby are constructed by calling the new method on the class itself, rather than having new be a language keyword.
    • Ruby allows for conditional and looping statements at the end of single lines as syntactic sugar.
    • Ruby allows parentheses to be dropped when the resultant code is unambiguous.
  8. This line is pretty packed as well:
    • to_s is Ruby's equivalent to Java's toString().
    • Ruby, like JavaScript, uses optional semicolons. Here, they're used to pack the method definition into one line.
    • When a method takes no parameters, the () after the name is optional, both in definition and in use.
    • Ruby strings allow for code interpolation via #{...}. It's like value bindings in XPages or the equivalent feature in PHP or Perl - it allows Ruby code to be embedded in a string instead of +-based concatenation.
    • Like Server JavaScript and Formula Language, the last line in a Ruby method (except in initialize and assignment methods) is an implied return.
    • By convention, Ruby uses all-lowercase underscore_separated_words for variable and method names.

That ended up being a bit more complicated than I had expected, but hopefully it's a reasonable start (if not, there are definitely better Ruby tutorials on the web). In the next post in this thread, I'll describe some other unique or unusual Ruby features and concepts, and later I'll go into some more applicable examples to explain why Ruby is worth using in XPages generally.

XPages MVC: Experiment II, Part 3

Mon May 28 20:00:00 EDT 2012

Tags: xpages mvc
  1. May 22 2012 - XPages MVC: Experiment I
  2. May 23 2012 - XPages MVC: Experiment II, Part 1
  3. May 24 2012 - XPages MVC: Experiment II, Part 2
  4. May 28 2012 - XPages MVC: Experiment II, Part 3
  5. Jun 04 2012 - XPages MVC: Experiment II, Part 4

Continuing on from my last post, I'd like to go over a couple specifics about how I handle fetching appropriate collections of objects from a view and a couple areas where I saved myself some programming hassle.

As I mentioned before, both the "manager" and "collection" classes inherit from abstract classes that handle a lot of the dirty work. The AbstractCollectionManager class is by far the smaller of the two, containing mostly convenience methods and a couple overloaded methods for generating the collection objects. The central one is pretty straightforward:

protected AbstractDominoList<E> createCollection( String sortBy, Object restrictTo, String searchQuery, int preCache, boolean singleEntry) { AbstractDominoList<E> collection = this.createCollection(); collection.setSortBy(sortBy); collection.setRestrictTo(restrictTo); collection.setSearchQuery(searchQuery); collection.setSingleEntry(singleEntry); collection.preCache(preCache); return collection; }

The first three properties actually determine the nature of the collection, while the last two set some optimization bits when the code knows ahead of time about how many elements it expects (such as doing a by-id lookup or a "recent news" list that will only ever display 5 entries). The createCollection call at the start is overridden method in each class that returns a basic collection object to avoid Java visibility issues.

The AbstractDominoList class is much longer; I won't go into too much detail, but a couple parts are pertinent. As I've mentioned, it implements the List interface (via extending AbstractList), which means it just needs to provide get and size methods. The way these work is by checking to see if it houses a valid cached ViewEntryCollection and, if it does, translating the method call to get the indexed entry as an object or the size. If the collection hasn't been fetched yet or if it's no longer valid (say, if it was recycled), it uses the view-location information from the implementing class and any parameters on the object to construct another one.

First, it fetches and configures the view properly:

Database database = this.getDatabase(); View view = database.getView(this.getViewName()); view.refresh(); String sortColumn = this.sortBy == null ? null : this.sortBy.toLowerCase().endsWith("-desc") ? this.sortBy.substring(0, this.sortBy.length()-5) : this.sortBy; boolean sortAscending = this.sortBy == null || !this.sortBy.toLowerCase().endsWith("-desc"); if(this.searchQuery != null) { view.FTSearchSorted(this.searchQuery, 0, sortColumn, sortAscending, false, false, false); } else if(this.sortBy != null && !this.sortBy.equals(this.getDefaultSort())) { view.resortView(sortColumn, sortAscending); } else { view.resortView(); }

Then, it uses code similar to this to store the collection in its cache (I say "similar to" because there are additional branches and methods, but this is the idea):

if(this.restrictTo == null) { ViewEntryCollection collection = view.getAllEntries(); this.cachedEntryCollection = new DominoEntryCollectionWrapper(collection, view); } else { ViewEntryCollection collection = view.getAllEntriesByKey(this.restrictTo, true); this.cachedEntryCollection = new DominoEntryCollectionWrapper(collection, view); }

The other part of the get method is translating the ViewEntry into an object, which is done via a method similar to this monster:

protected E createObjectFromViewEntry(ViewEntry entry) throws Exception { Class currentClass = this.getClass(); Class modelClass = Class.forName(JSFUtil.strLeftBack(currentClass.getName(), ".") + "." + JSFUtil.strLeftBack(currentClass.getSimpleName(), "List")); // Create an instance of our model object E object = (E)modelClass.newInstance(); object.setUniversalId(entry.getUniversalID()); object.setDocExists(true); // Loop through the declared columns and extract the values from the Entry Method[] methods = modelClass.getMethods(); String[] columns = this.getColumnFields(); Vector values = entry.getColumnValues(); for(Method method : methods) { for(int i = 0; i < columns.length; i++) { if(values != null && i < values.size() && columns[i].length() > 0 && method.getName().equals("set" + columns[i].substring(0, 1).toUpperCase() + columns[i].substring(1))) { String fieldType = method.getGenericParameterTypes()[0].toString(); if(fieldType.equals("int")) { if(values.get(i) instanceof Double) { method.invoke(object, ((Double)values.get(i)).intValue()); } } else if(fieldType.equals("class java.lang.String")) { method.invoke(object, values.get(i).toString()); } else if(fieldType.equals("java.util.List<java.lang.String>")) { method.invoke(object, JSFUtil.toStringList(values.get(i))); } else if(fieldType.equals("class java.util.Date")) { if(values.get(i) instanceof DateTime) { method.invoke(object, ((DateTime)values.get(i)).toJavaDate()); } } else if(fieldType.equals("boolean")) { if(values.get(i) instanceof Double) { method.invoke(object, ((Double)values.get(i)).intValue() == 1); } else { method.invoke(object, values.get(i).toString().equals("Yes")); } } else if(fieldType.equals("double")) { if(values.get(i) instanceof Double) { method.invoke(object, (Double)values.get(i)); } } else if(fieldType.equals("java.util.List<java.lang.Integer>")) { method.invoke(object, JSFUtil.toIntegerList(values.get(i))); } else { System.out.println("!! unknown type " + fieldType); } } } } return object; }

Yes, I am aware that it's O(m*n). Yes, I am aware I could move values != null outside the loop. Yes, I am aware I do the same string manipulation on each column name many times. Shut up.

Anyway, that code finds the model class and loops through its methods, looking for set* methods for each column (the column list being specified in the implementation class). When it finds one that matches, it checks the data type of the property, and invokes the method with an appropriately-cast or -converted column value. The result is that the implementation class only needs to provide the code needed for fetching any data that isn't in the view, such as rich text. At one point, I think I had it so that it read the view column titles and used those as field names, but I stopped doing that for some reason... maybe it was too slow, even for this method.

In any event, next time, I'll go over some of the disadvantages that I've found with this method that don't involve big-O notation.

XPages MVC: Experiment II, Part 2

Thu May 24 20:43:00 EDT 2012

Tags: xpages mvc
  1. May 22 2012 - XPages MVC: Experiment I
  2. May 23 2012 - XPages MVC: Experiment II, Part 1
  3. May 24 2012 - XPages MVC: Experiment II, Part 2
  4. May 28 2012 - XPages MVC: Experiment II, Part 3
  5. Jun 04 2012 - XPages MVC: Experiment II, Part 4

Continuing on from my last post, I'd like to go over the collection and model structures I used in my guild-forums app.

I set up the data in the Notes DB in a very SQL-ish way (in part because I considered not using Domino initially). Each Form has a single equivalent view (for the most part), which is uncategorized and sorted by default by an ID column. There is one column for each column that I want to sort by and, in the case of simple documents, one for each applicable field, containing only the field name; additionally, I created $-named "utility" columns for when I wanted to sort by a given bit of computed data or have a second secondary sort column for a given column. For the most part, the data is normalized, but I had to give in to reality in a few situations, for things like sorting Topics by the their latest Post dates.

For collection classes in Java, each collection class file contains one public class for the "manager" and one list implementation class, both of which extend from Abstract versions, so the actual implementations may have very little code. For example, here is the "Groups" Java file (minus some Serialization details):

public class Groups extends AbstractCollectionManager<Group> { protected AbstractDominoList<Group> createCollection() { return new GroupList(); } } class GroupList extends AbstractDominoList<Group> { public static final String BY_ID = "$GroupID"; public static final String BY_NAME = "$ListName"; public static final String BY_CATEGORY = "ListCategory"; public static final String DEFAULT_SORT = GroupList.BY_ID; protected String[] getColumnFields() { return new String[] { "id", "name", "category", "description" }; } protected String getViewName() { return "Player Groups"; } protected String getDatabasePath() { return "wownames.nsf"; } protected Group createObjectFromViewEntry(DominoEntryWrapper entry) throws Exception { Group group = super.createObjectFromViewEntry(entry); Document groupDoc = entry.getDocument(); group.setMemberNames((List>String<)groupDoc.getItemValue("Members")); return group; } }

The actual heavy lifting is done by the parent class, letting the specific classes be composed almost entirely of descriptions of where to find the view, what columns map to model fields, and any extra code that is involved with that object mapping. Other classes are larger, but follow the same pattern. For example, here's a snippet from the manager for Posts:

public class Posts extends AbstractCollectionManager<Post> { // [-snip-] public List<Post> getPostsForTag(String tag) throws Exception { return this.search("[Tags]=" + tag); } public List<Post> getPostsForUser(String user) throws Exception { return this.getCollection(PostList.BY_CREATED_BY, user); } public List<Post> getScreenshotPosts() throws Exception { return this.search("[ScreenshotURL] is present", PostList.BY_DATETIME_DESC); } }

The parent abstract class provides utility methods like getCollection and search that take appropriate arguments for sorting, searching (I love you, FTSearchSorted), and keys.

As for the individual model objects, they start out as your typical bean: a bunch of fields with simple data types and straightforward getters and setters. Additionally, they contain "relational" methods to fetch associated other objects, such as the Forum that contains the Topic, or a list of the Posts within it. Finally, the model class is responsible for actually saving back to the Domino database, as well as taking care of any business rules, like making sure that the Topic document is updated with a new Post's date/time.

The result is that a new object type is easy (-ish) to set up, with all the code being written contributing specifically towards the task at hand. And, since the nuts and bolts of XPages are Lists and Maps, working with these objects is smooth, taking advantage of Expression Language's cleanliness and Server JavaScript's bean-notation support in the UI and Java's clunky-but-still-useful collections features.

Next time, I'll go over some of the specifics of how I deal with fetching collections and individual records, use reflection to save me some coding hassle, and other tidbits from the giant messes of code that make up my abstract classes. After that, I think I'll make a fourth post to detail some down sides to my approach as implemented.

XPages MVC: Experiment II, Part 1

Wed May 23 15:38:00 EDT 2012

Tags: xpages mvc
  1. May 22 2012 - XPages MVC: Experiment I
  2. May 23 2012 - XPages MVC: Experiment II, Part 1
  3. May 24 2012 - XPages MVC: Experiment II, Part 2
  4. May 28 2012 - XPages MVC: Experiment II, Part 3
  5. Jun 04 2012 - XPages MVC: Experiment II, Part 4

As I've mentioned a couple times, my largest XPages app to date is the site I did for my guild's forums. The forum part itself isn't particularly amazing, considering it's almost harder to NOT write a forum in Domino than it is to write one, but it gave me a chance to try my hand at abstracting data access away from the XPage.

I put "Part 1" in the title because I figure it will be best to break the topic up into at least three parts: the overall idea and the result, the structure of the model and collection classes, and some nitty-gritty bits about how I get the data out of the back-end classes.

My first draft of this site was straightforward, from an XPages perspective: I mapped XPages to the corresponding Forms and used a bit of Server JavaScript here and there to pull in bits of support information. This worked pretty well at first, but quickly ran into code and speed scalability issues. Just looking at a "Topic" page, the following data needs to be looked up from outside the Topic document itself:

  • The name and ID of the parent Forum document
  • The name and ID of the parent Forum's parent Category document
  • Whether or not the topic is marked as a "favorite" for the current user, which is stored in a prefs doc for the user
  • The collection of Post documents for the Topic
  • For each Post document:
    • Summary and rich text data from the document
    • The display name, title, and signature of the posting user, which are stored in their Person document
    • The number of posts from the posting user that are visible to the current user, updated live (heh)
    • Whether or not the current user has rights to edit the Post document
  • Any surrounding page elements, such as the count of unread topics from the front page, the "who's online" list, etc.

As you can imagine, writing each one of those as inline SSJS turned into a hairy, slow mess. And that's just for the Topic page! That's leaving out the other pages, such as the forum index, which has to show a tree hierarchy of Categories and Forums with up-to-date Topic and Post counts.

I tried wrapping these things up into SSJS functions in a script library, which helped a LITTLE, but it still sucked. The clear solution was to wrap these all in proper model objects and, after a number of revisions, I came up with a pretty good solution.

Conceptually, the back-end structure consists of:

  • Collection Managers. These are analogous to Views, in that there's one per document type and they communicate with a similarly-named uncategorized View with many sortable columns. I write methods in these managers to allow for returning an appropriate collection for the circumstance (e.g. getPostsForForumId(...) and getPostsForTag(...)), as well as generic methods for getting arbitrary collections for a given key, sort column, and FT search query.
  • Collections. These are analogous (and often wrap) ViewEntryCollections. They implement the List interface to allow for use in xp:repeats and handle efficiently getting the needed model objects and re-fetching the View and Collection if they're recycled away.
  • Model Objects. These are what you'd think: Post, Topic, etc.. These provide getters/setters for the fields of the document, getters for associated collections (e.g. topic.getPosts()), and handle saving the document back to the DB.

The net result is that pages went from having giant blocks of lookup code (or SSJS function calls that pointed to giant block functions) to nice little EL expressions like #{category.forums} and #{forum.latestPost.createdBy}. As importantly, this allowed for a lot of caching to make these relational operations speedy when dealing with live data in a Domino DB.

Next time, I'll go into some specifics about how I structured the collection and model classes.

Optimizing for the Wrong Thing

Wed May 23 08:37:00 EDT 2012

Tags: off-topic

My apartment has a core logistical issue: it has two floors and the connection is a spiral staircase. This is generally fine for normal use, but it's a giant hassle when moving in or out: the building has to hire some guys to essentially dismantle the stairs to turn each step to one side so they can hoist large objects like bookshelves and beds up and down.

In order to make this mildly more practical, the building decided to replace one of the upstairs windows in each apartment with a model that makes it easier to move things in an out through it. Nothing to complain about there - a window's a window, and now was a pretty good time to make the switch, since our section of the building has been lightly populated but is picking up now.

The actual installation was problematic in the way you'd expect that kind of thing to be: longer than the estimated hour, several trips, and the powerful aroma of new drywall and paint for several days. I had to call the front desk a couple times to get them to "remind" the maintenance guy that he was supposed to stop by several times but kept forgetting. Still, that's within the expected range.

The problematic part of this window is that, while it is indeed better at the specific task of moving furniture in and out, it's at the significant expense of daily window use. I had assumed that the new window would be basically the same as the old one, but made to be easy to remove when needed. That, however, is not the case. Replacing the two-pane window we used to have, the new one is a single giant pane with bars glued to it, I guess to make it look like the others. It can't be opened from the bottom like the school-style window we used to have or slid up like normal human windows, but instead there's a large handle on the left: turn it 90° and you can swing the entire window inwards, which is what you'd do when moving; give it another quarter turn and you can kind of lean it inwards from the top about three inches to let some fresh air in. Note that this means that it's mildly easier to move the entire window than it is to open it just a bit.

Additionally, they added a "lock" to the window, presumably for policy or regulatory reasons, to make it so that we can't easily swing open the whole thing. I say "lock" because that's how the building guy described it to me. A more accurate description would be "a metal stick screwed into the window frame". It's literally a bar of metal placed a couple inches from the window, so the window will smack into it like it's a doorstop. It's also colored a dark green, which would have matched the color of the old window frame; the new frame, however, is white. Fortunately, it appears easy to remove, so I may do that.

To accommodate the inward-swinging top and this "lock", the maintenance guy had to move the windowblinds several inches, so that they're now placed right in the middle of the previously-deep frame. We used to have a couple plants and a bed for the cats up there; I guess we still could, but then we'd have to choose between never quite lowering the blinds all the way, bonking the cat on the head when lowering them, or manually angling them back behind the obstacles every time.

And the kicker: there's no screen, so we can't actually open the window unless we want a bunch of bugs in here anyway.

So yes, this new window is indeed good at the task of moving furniture in and out through it. Fantastically good, in fact! Unfortunately, that's something that happens only twice per tenant, and in theory tenants should not be coming and going particularly frequently. Certainly, less frequently than they might do things like, say, let some fresh air in, raise or lower blinds, and place things on the windowsill... tasks that are all significantly worse now.

Overall, I feel like my computer room's window was mugged.