One of my (many) for-fun projects lately has been a design-element editor named "Forms 'n' Views". Though it's not really release-quality, it's been coming together enough to toss it up on GitHub and make a post about it. Basically, it's meant to serve a couple purposes:

• Help mitigate the "Windows problem" - when all you want to do is make a quick change to a legacy design element (say, a stylesheet), but doing so would mean firing up a Windows VM, launching Designer, and opening the DB. For quick edits, the amount of time that takes wildly eclipses (heh) the time spent actually coding.
• Experiment with a data-focused way to edit forms and views. The legacy editors are geared towards a Notes client target, dedicating most of their space to aspects that have no meaning when the design element is intended just for backing an XPage.
• Eventually, provide a way to grant "sudo"-style access to other users to modify certain design elements without granting full designer/manager access, setting up WebDAV, or exposing a poor web designer to Designer.
• Let me experiment with a different setup for an XPages app - Forms 'n' Views consists of a Dojo BorderContainer- and TabContainer-based UI on a single XPage, which loads up tabs containing custom controls on the fly. None of it is unexplored territory, but it's good to write different kinds of apps sometimes.
• Let me try out some more backing-class architectures. Each editor is backed by a DOM representation of the exported DXL, which is manipulated through wrapper classes and then re-imported back directly. Things like getting lists of columns are done via XPath queries without storing the data redundantly.

In its current incarnation, it's taken on the pale complexion of Dojo's "claro" theme - I've been going back and forth between that and the Notes-like "soria" theme:

It should do for now, but I'll likely fiddle with it more to make it look better.

In any event, this has been a fun project so far, and hopefully it'll shape up into a useful tool.

I have a confession to make: I'm not crazy about XAgents. Don't get me wrong - they do everything they're supposed to and do it well. However, it's always kind of bothered me that you take a visual design element like an XPage and turn off all the higher levels to get back down to the core servlet. Plus, it muddies the list of XPages in the DB - some are actual XPages, some are just wrappers for scripts. So my objection is essentially pedantry.

However, the fact that my objection is wildly exaggerated has never stopped me from sinking a lot of time into finding the "right" answer before, and it hasn't stopped me now. When what I want to build is, say, an Excel exporter for a particular type of document in a reporting DB, what I really want is a basic servlet that exists only in the database and has access to the surrounding XSP environment and custom classes. Fortunately, between a post by Sven Hasselbach, the referenced Chinese-language developerWorks article, the XSP Starter Kit, and a bit about facesContext.release() on a JSF-focused article and reinforced by the FacesContextServlet class from the ExtLib, I made it work.

The developerWorks article covers the bulk of the work - creating the Factory, setting up the file in META-INF, etc.. After that, you can set up your servlets by extending (using the Starter Kit method) DesignerFacesServlet. It turns out that the important thing to remember is to close out your context when you're done - I ran into a lot of trouble from not doing this, which would break XPages visited after the servlet. As an example, here's the test class I ended up building while figuring out how to make it not blow up my application:

import java.io.*;
import java.util.*;
import com.ibm.commons.util.StringUtil;
import com.ibm.xsp.webapp.DesignerFacesServlet;
import javax.faces.context.FacesContext;
import javax.servlet.*;
import javax.servlet.http.*;

public class TestServlet extends DesignerFacesServlet implements Serializable {
	private static final long serialVersionUID = -1152176824225969420L;

	@SuppressWarnings("unchecked")
	@Override
	public void service(ServletRequest servletRequest, ServletResponse servletResponse) throws ServletException, IOException {
		// Set up handy environment variables
		HttpServletRequest req = (HttpServletRequest)servletRequest;
		HttpServletResponse res = (HttpServletResponse)servletResponse;
		ServletOutputStream out = res.getOutputStream();
		FacesContext facesContext = this.getFacesContext(req, res);

		try {
			res.setContentType("text/plain");

			out.println("start");

			// The sessionScope is available via the ExternalContext. Resolving the variable
			//	would work as well
			Map<Object, Object> sessionScope = facesContext.getExternalContext().getSessionMap();
			sessionScope.put("counter", sessionScope.containsKey("counter") ? (Integer)sessionScope.get("counter") + 1 : 1);
			out.println("Counter: " + sessionScope.get("counter"));

			// A query string map is available via the request. This method, as opposed to
			// 	getting the "param" variable, returns arrays of strings, allowing things like
			//	"?foo=bar&foo=baz" properly
			Map<String, String[]> param = req.getParameterMap();
			for(String key : param.keySet()) {
				out.println(key + " => " + StringUtil.concatStrings(param.get(key), ';', false));
			}

			out.println("done");

		} catch(Exception e) {
			e.printStackTrace(new PrintStream(out));
		} finally {
			out.close();

			// It shouldn't be null if things are going well, but a check never hurt
			if(facesContext != null) {
				facesContext.responseComplete();
				facesContext.release();
			}
		}
	}
}

I set up my ServletFactory to use this one for "/test", and so it's available via a URL like "/database.nsf/xsp/test?foo=bar". As with an XPage, you can also chain more path bits on after the servlet name, like "/database.nsf/xsp/test/some/other/stuff" and get to that via req.getPathInfo() - though with the caveat that, unlike with an XPage, the servlet path is included in the path info, so it would return "/xsp/test/some/other/stuff".

So long as I don't run into any other app-exploding problems, I plan to go this route for non-UI requests like exports and actions. When I have a use for it, I'll also go down the related path of writing custom services, for which the ExtLib provides an extensive foundation.

Update: Looks like Chris Toohey beat me to this by a couple months: http://www.dominoguru.com/pages/04092012113950.html

Marky Roden did a presentation at MWLUG that included, among other things, a demonstration of Pines Notify, a jQuery plugin that provides Growl-style notifications - something that could come in tremendously handy in a lot of situations.

I wanted to use something like this, but I decided to check to see if there's an equivalent in Dojo already, so I don't have to start including jQuery as well. Fortunately, there is... ish. dojox.widget.Toaster serves a similar purpose, though it takes its design cues from the little "toaster" notifications common on Windows. However, their default appearance is the worst thing in the entire world:

Fortunately, that can be cleaned up with a little CSS to be a crummy version of Growl:

With a bit more work and attention to detail, it could end up pretty classy. I think it's still not as featureful as Pines Notify (no "clickthrough" options, its handling of multiple messages with different timeouts is not nuanced, etc.), but it's serviceable.

The API to use this widget is actually pretty straightforward: you run a bit of code at page load to create a toaster object to listen to a specific channel name (e.g. "/save/success") and then publish messages whenever you want something to appear. The code for sending a message is pretty simple, and there are two forms, depending on whether or not you want to include additional parameters:

dojo.publish("/test/channel", ["This is message one, lasting five seconds"])
dojo.publish("/test/channel", [{ message: "This is message two, lasting 10 seconds", duration: 10000 }])

I decided to wrap the instantiation and required resources up into a custom control and make another demo DB:

http://frostillic.us/tests/djToaster.nsf

The "toaster.css" page is the CSS I wrote to override the standard behavior. The margins are intended for the top-right positioning, but could be changed readily to equivalents for other corners.

The control exposes five properties: messageTopic (the channel name, required), defaultType ("message", "error", etc.), duration (in milliseconds), positionDirection (the position and slide direction, defaulting to up from the bottom right), and separator (the text that appears between two simultaneous messages).

Update: Looks like I was late to the game on this as well: http://www.intec.co.uk/xpages-8-5-2-rich-text-extending-the-ckeditor/

A while back, I got annoyed by the lack of a "Source" button in the default rich text editor in XPages. After a bit of digging, I found that there is indeed one (not to mention a whole world of CKEditor toolbar plugins and enhancements) - just not by default. I think there's a way you can pick from a couple named "stock" toolbar layouts, but I ended up going the route of defining my own, naming each button in a group.

In CKEditor, at least pre-3.6 (and I guess it works later too), toolbars can be defined inline via a JSON array-of-arrays in the "toolbar" property of the editor. A hopelessly simple toolbar definition would look something like this:

[
	["Bold", "Italic", "Underline", "Strike", "-", "TextColor", "BGColor" ],
	["Indent", "Outdent"]
]

That would define two button groups with the named controls present. The "-" is a way to manually place a separator - think of them like non-breaking spaces: they look the same as the divider between groups, but will not cause the group to be split onto a second line if the toolbar is too wide for a single row.

For my purposes, I went a bit nuts:

[
	["Format", "Font", "FontSize"],
	["Bold", "Italic", "Underline", "Strike", "-", "TextColor", "BGColor", "-", "JustifyLeft", "JustifyCenter", "JustifyRight", "JustifyBlock", "NumberedList", "-", "BulletedList"],
	["Indent", "Outdent"],
	["Subscript", "Superscript"],
	["RemoveFormat", "-", "MenuPaste", "-", "Undo", "Redo", "Find", "LotusSpellChecker", "-", "Image", "Table", "Link", "Flash", "-", "PageBreak", "HorizontalRule", "SpecialChar", "Blockquote", "Smiley", "ShowBlocks"],
	["Maximize", "Source"]
]

Inside an XPage, you can set this using the "attrs" property in 8.5.3 - though "dojoAttributes" would probably work fine in 8.5.3 and earlier as well:

<xp:inputRichText value="#{doc.Body}">
	<xp:this.attrs>
		<xp:attr name="toolbar"><xp:this.value><![CDATA[
			[
				["Format", "Font", "FontSize"],
				["Bold", "Italic", "Underline", "Strike", "-", "TextColor", "BGColor", "-", "JustifyLeft", "JustifyCenter", "JustifyRight", "JustifyBlock", "NumberedList", "-", "BulletedList"],
				["Indent", "Outdent"],
				["Subscript", "Superscript"],
				["RemoveFormat", "-", "MenuPaste", "-", "Undo", "Redo", "Find", "LotusSpellChecker", "-", "Image", "Table", "Link", "Flash", "-", "PageBreak", "HorizontalRule", "SpecialChar", "Blockquote", "Smiley", "ShowBlocks"],
				["Maximize", "Source"]
			]
		]]></xp:this.value></xp:attr>
	</xp:this.attrs>
</xp:inputRichText>

It looks like the syntax changed for 3.6, but not too much.

If you want to look into more specifics, you can take a look at the examples and un-minified code in (data)/domino/html/ckeditor/_samples and _source.

I realized earlier that I let a couple of the classes in Domino-One-Offs stagnate relative to the versions I currently use. Since originally posting my convenience XML library and, more usefully, the DynamicViewCustomizer, I've tweaked both in my various projects to add more features I ended up needing.

• com.raidomatic.xml
• DynamicViewCustomizer
• SortableMapView

The DynamicViewCustomizer has gone further down the path of Notes-client fidelity at the expense of a bit more overhead, doing things like setting column widths (except where the column is set to extend to the window width, either specifically or by virtue of being the last column, as appropriate). It's still not perfect, as I ran into a bit of trouble with, I think, HTML in category columns (and you can see commented-out remnants of various other changes), but it's potentially useful.

For my "man, I hate dealing with all these stupid orthogonal Java XML libraries" XML library, I've added a number of functions to support my Forms 'n' Views app (coming Soon™), namely better attribute handling, a simple NodeList wrapper (that actually acts like a List), and a basic getXml() method for when you just want a string version.

Finally, I tossed in the current version of my SortableMapView class, meant to gobble up Views or ViewEntryCollections and make them sortable by all columns. That one uses Lombok, but I think you could just remove the bits about making it List-compatible and still be fine.

Ever since I personally came up with the idea of storing serialized Java objects in Notes documents via MIME, I've been trying to use the technique all over the place, since it's basically the best thing since sliced bread.

My latest notion was that the pattern would be all the cooler if the serialized Java objects could be referenced directly in EL, so you could do something like #{doc.LineItems[3].cost}, pulling directly from a more-or-less normal Domino document. I decided to give implementing this a shot, since it would also have the side benefit of introducing me to writing JSF components/complex types.

The path I took sure feels like the long route (lots of manual doubling up of property definitions and proxying methods), but it seems to work: I have a "mimeDominoDocument" data source that acts just like a normal document source except it transparently serializes and deserializes Java objects. It's not exactly a perfect implementation (the setValue() method has a rather naive method of guessing which objects should be serialized and which shouldn't, and I didn't handle the case of setting a value to an object and then using, say, getItemValueString()), but it should be fine for demo and basic purposes.

I set up an example database with pages for the appropriate code here: /tests/mime.nsf.

Every time I make a new XPages app, I start by using the normal xp:dominoView and xp:dominoDocument data sources, on the theory that building with the standard set of components will keep things clean while I build on top of that. However, with each successive app, the time before I get annoyed at how messy the code has to be and start writing Java wrapper and management classes gets shorter and shorter.

For example, in my Forms 'n' Views mini-Designer app, I have the sidebar list of databases, which is just pointing to a view and showing those entries. That works pretty well, since the values I need are in the view itself. However, now I want to show the database icons, but also want to check to make sure the DBs are on the current server, allow the current user to access them, and are accessible via HTTP (I'm just pointing the image to /whatever.nsf/$Icon). Doing that with just the view entries will get messy, requiring a bunch of Server JavaScript to be crammed into my xp:image object. It'd work, but it'll only be a matter of time before I want to add more smarts, such as if I make it so that you can grant enhanced access to certain design elements to a certain user. The best way to handle this will be to make a "DatabaseEntry" class and give it methods to determine the icon and design-element visibility.

I don't think I really saved myself any time by delaying this - it's pretty rare that I DON'T have this sort of calculation eventually, so I'm just adding a bit of time doing things the "dirty" way first before cleaning them up. I'm not really familiar with "vanilla" JSF, but I get the impression that it doesn't really let you put as much business logic into the view portion as XPages do (I think JSF tends to stick to EL normally). Certainly, when you write a Rails app, you start with the models and controllers cleanly before putting the data on the page. The XPage architecture doesn't make that as easy, with its lack of reasonably-modifiable controllers, but adding Java classes and using collection classes in xp:dataContexts or as managed beans work great.

I've toyed with the idea of expanding the classes I use in my forums app to work smoothly with any view/document, picking up the columns by name and storing them in Map representing the entry/document. The problem with that is that the data source and needs tend to be just different enough per app that it's not generic enough (for example, in this one, I'm getting databases for the "add DB" picker via DbDirectory, not a view). In the mean time, I think I'll just start with the access classes first and see what I do most commonly.

While working on the view indexer thing (which seems to be taking the form of "yet another SQL exporter for Domino", but with some advantages), I started thinking about how I often want a nice, quick way to edit some views, forms, or stylesheets without having to launch Windows and Designer. The Mac client is there for views, but the experience still leaves something to be desired. In addition, I do very little lately that's intended to be seen directly in the Notes client - most views are set up as data structures to be used in XPages, yet the default editor dedicates about half of its real estate to actions and a completely unreliable visual preview area and only lets you work with one column's settings at a time (other than position and width). Same goes for forms - many of them lately are just the equivalent of schemas, with the client appearance being almost irrelevant.

So, to scratch my itch, I've started making myself another XPages app:

This has the side benefit of giving me an opportunity to make a UI entirely composed of Dojo layout widgets - I've used them frequently, but not exclusively in a site. The dynamic tabbed panel stuff is pretty cool.

I'm trying to be careful about how the editing happens. Rather than building an object structure from the DXL and then writing out a new DXL document to import - risking deleting any additional attributes that I don't care about - each design element wrapper class works directly on a DOM version of the DXL document, so that anything that it doesn't touch remains when it's imported again. That should allow me to edit existing forms without worrying about destroying the layout - any new fields can just get tacked on to the end, or maybe in a generic table structure, while the rest of the layout remains the same.

I don't know how crazy I want to get with it (I could, for example, make it so that you can allow non-Designers access to modify specific design elements per-DB), but it's a fun little project nonetheless.

I've put a little more time into my small view indexer from the other day, and it seems even more promising now. The editor I made for it might give you some idea of the potential:

Since the view building is all being done in Java, I decided to toss in the list of available JSR-223-compliant scripting languages currently available. The scripting context is given a variable "doc" that's a DocumentWrapper class I wrote that implements Map to make its use a bit easier. For added fun, I baked in knowledge of serialized Java objects stored in the document's items:

Normally, the wrapper just returns the value of getItemValue(...), but when it encounters a MIME entity of type "application/x-java-serialized-object", it deserializes it and returns that instead, allowing for real structured data access.

I also realized that I don't have any reason to stick to only objects available in the stock JDK for index storage. Since using this will require extra Java code anyway, I may as well make my life easier and write my own "view entry" class. Once I get that sorted out, I can work on keeping the indexes updated with a server task, dealing with reader fields, and physical storage.

Hmm, maybe I should make myself an editor for normal views that looks like this one. It'd probably be a lot less hassle to deal with than the legacy one.

Warning: this post is almost entirely pie-in-the-sky nonsense, untethered from reality. Probably.

So yesterday, for some reason, I got to thinking about Domino's view indexing and whether or not it can be readily done better. The current view indexer does its intended task with aplomb (most of the time), but it has its problems. For one, iterating over view indexes in Java (read: everything you should do in Domino today) is dog slow. More theoretically, since it deals only with summary data for performance reasons, its ability to deal with structured data is limited - you can't deal with rich text or, for example, store any sort of Map in a Domino document and deal with it in a view (realistically).

I had the notion that it may be faster, at least in some cases, to do your own indexing: make a Java collection of your "view" data and serialize the result into a note. I decided to try a little test: take a view that consists only of a sorted column of 100k documents' UNIDs and compare it to a TreeSet of the same. My initial results were promising: iterating over the collection of documents (via getNextDocument() and a forall with getDocumentByUNID(), respectively) was about twice as fast with the Java version and was stored in about 17% the space, presumably thanks to document compression.

To be fair, that was really stacking the deck in favor of Java: I didn't care about reader fields or any entry metadata like position, and fetching every document in a large view is possibly the most expensive way to use it. Presumably, fetching just the first entry would make the view version much faster. Still, it shows that it might be useful in some cases, so I decided to try something a bit more complicated.

I made another view with 15k documents (task requests) and two columns: the date of the request, sorted, and the summary line. I matched that up with a TreeSet containing Lists of Maps and iterated over both to fetch and print the entry data. Again, the Java version ended up way faster, taking about 25% of the time.

Now it has me wondering about whether it'd be worth investigating further. It's fair to assume that Java can be treated as the new "native" language of Domino, so serialization is potentially a legitimate mechanism. Furthermore, a real implementation of an alternative view index wouldn't have to be as hopelessly naive as the one I put together: the basic java.util.* collections are almost definitely not the best storage mechanisms for a database index, and, better still, this is well-trodden territory. Since normal views would still exist in this magic future world, these fancy views could eschew a lot of the UI and build-performance requirements of the normal ones in favor of fancy tricks like native knowledge of serialized data structures. Imagine storing structured data like a table of line items in a MIME entity but then being able to query that in a "view". Reader fields could be handled by storing the applicable names in the "entry" and then comparing that two the user's names list at runtime, presumably like standard views do. A server task could handle monitoring document changes and making incremental changes (like happens now, really).

So it's theoretically possible. Would it be worth it? I'm not sure - maybe. It's fun to think about, in any event.