The Curse of TMTOWTDI

Tue Nov 26 15:41:24 EST 2013

Tags: xpages

TMTOWTDI is a Perl-ism, standing for "there's more than one way to do it". It is often a blessing, particularly in the context of Perl, in that it results in multiple synonymous ways to accomplish the same task, each of which fits best in a different context.

However, as you might expect, this is a double-edged sword that can lead to messy code and a difficult learning curve. It's this negative context that bites the XPages platform in a number of ways.

One way is relatively benign but still a pernicious issue: the Java base of XPages contains a plethora of may-or-may-not-be-important choices to make, such as Vector vs. ArrayList vs. LinkedList. Though it doesn't really matter in small cases which you pick, you still have to make that choice, and some of them (Vector) end up hurting you as the scale increases. That's not something you have to pay too much attention to in most other high-level languages, where there's a standard Array type that does its job well.

The larger issue is the massive divergence of ways that even normal XPages apps can be structured. Do you use one main XPage and then put everything either in other XPages brought in with "include page" or in custom controls (with or without Dynamic Content), or do you do a lot of top-level XPages the user visits directly? If the latter, do you still break out chunks of individual-page functionality to custom controls for readability? How about data - do you store your data in the same NSF as the code, in separate NSFs, or in a non-Domino database? Do you use stock controls and make a OneUI app, toss all that OneUI/Dojo stuff aside and build on Bootstrap/Foundation/etc. and jQuery, or take a hybrid approach? For back-end code, do you use inline SSJS, primarily SSJS libraries, or Java? If you base it on Java, how? Controller classes, object data sources, phase listeners, variable resolvers, core business-logic classes called by SSJS? If you put your data access in Java classes, do you build them to work alongside stock XSP Domino data sources for performance, or build on the Domino API?

The list goes on.

There are likely as many divergent approaches to XPages development as there are XPages developers. While this is good in the sense of feeling out the contours of the platform's capabilities, it makes it difficult to get started and to collaborate with other projects.

As you might expect, I have my own opinions on each question, but some of them are still fluid. This is a natural by-product of a still-relatively-young platform with a strange history, no clear single voice, and a primary target market of ancient systems with massive inertia. In time, I hope that this settles down to a good general-purpose set of techniques (and I hope to influence that). Each application I make feels more coherent than the last, and the differences between them are gradually diminishing. The more consistent the methods (assuming they're good), the better for the platform and the developers on it.

Building My App On Custom Renderers

Sun Nov 24 20:41:15 EST 2013

Tags: xpages java
  1. I Am Terribly Excited About Custom Renderers
  2. Building My App On Custom Renderers

A little while ago, I mentioned how I have become smitten with custom renderers - Java classes that take XPage components like application layouts and widgets and render them in custom HTML. Though I've had a lot of other things to work on in between then and today, my ardor for the subject has not dimmed. So today, I mostly completed the process for my task-tracking app.

At a high level, the app is a fairly typical web app built using the popular-for-good-reason Ace - Responsive Admin Template theme from WrapBootstrap. I'm a huge fan of using these pre-built themes - in addition to handling the visual design, they tend to come packaged with a set of specialized widgets and jQuery plugins that are made to work well as a cohesive whole. However, the down side is that you have to write your code specifically targeting the framework, even moreso than stock Bootstrap (and for this reason, I couldn't directly use Bootstrap4XPages). This makes it a perfect candidate for renderer-ification, as most of the components I'm using - the layout, widget boxes, view panels/data tables, form layouts - have direct analogs in either the stock XPage runtime or in the OneUI-based components in the Extension Library.

So what I have done is to take the basic structure of the Bootstrap4XPages project (and more than a smidge of its code) and used it to build custom renderers for the components I need, packaged into an OSGi plugin:

Project Explorer

It's something of a mess of code, and parts of it have been a pain to deal with, but for the most part it's been a process of looking at the original renderer source in the ExtLib and Bootstrap4XPages and then either copying and tweaking, subclassing and overriding, or implementing from scratch. I've run into some bothersome limitations in the stock objects as well, particularly when it comes to areas where I need to apply CSS classes to elements not represented in the components, like the FontAwesome-based icon classes for the nav bar or specific classes for widget title bars and the body area. Since these components also lack convenient "attr" properties, I've resorted to ugly hacks: I've co-opted the "imageAlt" property of the tree nodes to trigger creation of a Bootstrap-style "i" tag with a class for the navigator, and for the widgets I just made the styleClass apply only to the header and added an ugly exception to look for "no-padding" classes and apply them to the body instead. It's not a pretty process sometimes.

But the results are pretty interesting. By using standard components, I have an app that can swap readily between very different themes (with some rough edges for areas where code is still Ace-specific):

Ace

OneUI v2.1

OneUI v3.0.2

All of this does, of course, raise an important question. Namely: why bother? The original way I implemented it - by writing the Bootstrap layout HTML by hand, tweaking individual classes here and there, and forgoing the OneUI-based components - worked just fine, and it took less work. And really, I'm not terribly likely to suddenly decide that I want to switch this app from Ace to OneUI or stock Bootstrap. There are a couple important benefits to this approach that apply both to me personally and to others who may consider doing the same thing:

  • Practice. This seems odd to put first, but it's vitally important. Writing these renderers has improved my knowledge of the platform a great deal. It's given me a reason to delve further into the Extension Library source, to improve my OSGi-plugin-writing knowledge, and to familiarize myself with an easily-overlooked part of the JSF stack. Like many things with the plumbing of XPages, it's more of a PITA than it absolutely needs to be, but, if you do it, you'll come out a much better developer.
  • Focus. Now that I have almost all of the code that's specific to Bootstrap or Ace out of my XPage app, the remaining code has a sharpened focus on the task at hand. Now I use standard widget, pager, and view controls just like in any run-of-the-mill app and hook them up the same way, without having to worry about how it's going to look. Time spent on the problem at hand is time spent well.
  • Maintainability. During my transition from hardcoded HTML in the XSP source to custom renderers, I also jumped from Ace 1.0 to 1.2, which included an upgrade to Bootstrap 3. For this task, that meant I had to learn about a number of the differences between Bootstrap 2 and 3, as well as tweaking lots of little CSS classes and HTML structures across every page of the app. Next time, though, when Ace version 2 running with Bootstrap 4 comes out, I only need to change the theme, entirely outside of the app, and everything else will come along for the ride (mostly).
  • Less Code in the NSF. This is a corollary to the last bullet point, but is important as its own concept. By moving the theme and supporting files (like the CSS and JS assets themselves) out to a plugin, there's less cruft to carry around in the NSF itself and, more importantly, less stuff to keep up-to-date when used in common across multiple apps. This is not unique to this technique (you can move the resource files out without doing renderers, and you can bundle common custom controls that way too), but it's a valuable side effect.
  • Interoperability. This doesn't matter so much to me currently, since I'm the only one working on this app, but it will matter later, and it would matter immediately to any larger company. If each of your apps, regardless of the final appearance, is built using the standard XPage and ExtLib controls, that means you don't need to familiarize every developer working on it with the CSS framework of your choice, whether it be Bootstrap, your company's home-grown HTML, or something else entirely. You can have one person writing the themes and have the rest of the team go about building their apps by the book (or, rather, books).
  • Flexibility. As demonstrated by my screenshots above, this technique brings tremendous flexibility. In this specific case, I knew what theme I wanted to use when I started writing the app, but in the future I won't need to. Whenever I'm struck by inspiration for a new app, I can start writing code immediately (dressing it up in Ace or OneUI) and then decide on and buy another theme later without the hassle of going back into my functional XPages to replace and tweak a bunch of CSS classes.

It's a pity that there's such overhead in the initial work of writing custom renderers, but it will get gradually easier over time, now that Bootstrap4XPages is available to reference. And still, hassle and all, I believe this to be the best way forward for organized XPage development, and represents one of the platform's distinct advantages over others (well, the others that aren't JSF). I strongly encourage everyone to look at least a little into the idea. And I also, as always, encourage you to contact me at will either to pick my brain or commission my company to help renderer-ify or otherwise improve your apps.

Internalize This Deep Wisdom

Tue Nov 19 18:43:28 EST 2013

Tags: programming

Toby Samples tweeted earlier an old blog post by Jeff Atwood about the virtue of minimal code, and I think everyone would be well-served by reading it and the articles it links to (the Wil Shipley post has since moved (incidentally, if you don't currently pay attention to Wil Shipley, you really, really should)).

I've found maintaining the goal of reducing the conceptual weight of my code to be the most valuable tool in my belt. One way of doing that is to separate out unrelated concerns, letting you focus only on the task at hand, but the best way is to eliminate concerns from your code entirely. It's not entirely a matter of "don't reinvent the wheel" (if your whole project is the wheel, feel free to reinvent it), but rather more of a matter of not solving problems you don't have to solve. For example, have you seen the class ExtLibUtil? It solves tons of basic little problems that most XPages run across, such as getting the viewScope. Back when I learned about that, I dropped a bunch of my existing utility functions. I'm presently doing the same with StringUtil. Those are small problems that I have no business re-solving.

I could go on, but I think the links in the first paragraph cover it best anyway. Go read them!

My Current Model Framework, Part 1

Sun Nov 17 12:19:18 EST 2013

Tags: xpages mvc
  1. My Current Model Framework, Part 1
  2. My Current Model Framework, Part 2: An Example

As I do from time to time, I've recently been taking another stab at a standard model framework for my XPage apps. My latest one has been proving its worth in a couple apps I've been writing lately, and I'd like to go over the general goals and advantages of the way it works.

The framework is focused on a couple main ideas:

  • Low Overhead. The Java language requires a certain amount of overhead to get anything done, but I want to minimize that. The task of defining a new model object consists of two classes: one for the object itself and one for the collection manager (e.g. connecting to Domino views), and each is geared towards only writing the code required to describe the necessities.
  • Embracing XSP. The framework is intended for writing XPages apps, and so I want to make sure it works smoothly with EL and standard controls like inputs, repeats, and view panels.
  • Embracing document databases. Since I use almost entirely a document database for storage, I want to make sure to take advantage of that, and that primarily means flexibility in data modeling. Accordingly, data objects allow arbitrary field access, with any strictures from your model class layered on top of that.
  • Embracing Domino. Since the document database I use is Domino, I want to make sure its peculiar features are retained as well: reader/author fields, arbitrary object storage via MIME, (ideally) RT and attachments, full-text search, categorized views, and so forth.
  • Conceptually simple. Though there's some ugly code involved in parts like the Domino view wrapper, the conceptual layout of the framework is kept very simple with few moving parts, making it easy to recognize the function of each aspect at a glance, whether when building it, when looking at another's code, or when returning to your own code months down the line.

For this post, I'll give an example of a model class, while later I'll go into the collection managers and some real-use examples. This is what a basic model class looks like:

package model;

import java.util.Arrays;
import java.util.Collection;
import java.util.List;

import org.openntf.domino.*;

import frostillicus.model.AbstractDominoModel;
import frostillicus.model.DominoColumnInfo;

public class Request extends AbstractDominoModel {
	private static final long serialVersionUID = 589766180414699322L;

	public Request(final Database database) {
		super(database);
		setValue("Form", "Request");
	}

	public Request(final ViewEntry entry, final List<DominoColumnInfo> columnInfo) {
		super(entry, columnInfo);
	}

	public Request(final Document doc) {
		super(doc);
	}

	@Override
	protected Collection<String> nonSummaryFields() {
		return Arrays.asList(new String[] { "Body" });
	}

}

There's more Java overhead than I'd like, but that's the name of the game. Fortunately, each method has a role and can be used as hooks for differing behavior.

The first constructor is the one used to create a new document of the appropriate type in the provided database, so it sets the form field appropriately - it would also set any other appropriate defaults. The second constructor is used when traversing a view - model objects pull values from view entries when available, but also transparently pass along requests to the document when the requested field isn't in the view. The final constructor wraps an existing document.

The "nonSummaryFields" method is one of the hooks available to define the Domino data model, in this case providing a list of fields that should be flagged as non-summary when written, to help work around Domino limits. I also have hooks, not needed here, for authors/readers/names fields and form-style query/post events.

Conspicuously absent are any field definitions. By default, model objects act much like XPage DominoDocuments, passing setValue and getValue calls on to the underlying document more or less directly. However, the framework allows for hooks here by writing getters and setters in the standard format, and their presence changes the behavior of getValue and setValue. When just a getter is present, the field becomes read-only; when a setter is present, the field can be written, but now does data-type validation. The getters and setters allow for arbitrary validation and additional behavior for specific fields (say, changing related fields when one changes) without having to write out giant blocks of "getFoo() { return foo; }" and "setFoo(String foo) { this.foo = foo; }", and this has been a huge win for me. Even though Eclipse helps with the initial creation, the code still has to exist, and it takes a cognitive toll.

I also use these overriding methods to create relations between models through their collection managers. For example:

public Client getClient() {
	String clientId = (String)getValue("ClientID");
	if(clientId == null || clientId.isEmpty()) {
		return null;
	}
	return (Client)JSFUtil.getClientManager().getValue(clientId);
}

This allows for XPage El bindings like #{task.client.name} - no extra panels with data sources, no inline lookup code.

Next time, I will go into the collection-manager side, which provides fairly high-performance access to views while remaining simple and flexible.

I Am Terribly Excited About Custom Renderers

Tue Nov 12 19:56:45 EST 2013

Tags: xpages java
  1. I Am Terribly Excited About Custom Renderers
  2. Building My App On Custom Renderers

It's much to my chagrin that it took me until this past weekend to properly dive into custom renderers. I had seen them before, primarily when working with mypic, but always avoided building my own, primarily because of what a hassle they are to write. However, I'm now convinced that the hassle is worth it, at least for some primary uses.

If you're not familiar with custom renderers, they're sort of an odd beast, but they're the kind of thing where the concept eventually "clicks" in your mind at some point. In XPages, a component (such as a link, a panel, or an application layout) consists of two main aspects: the structural code that defines the properties of the component and the renderer that handles actually outputting the HTML for the browser. What adding your own custom renderers allows is for you to continue to use the same XSP markup to define the control, yet have very different results.

The best example of this (and the source of most of my education) is the recent Bootstrap4XPages project. It takes a standard OneUI applicationLayout-based app and styles it using Bootstrap markup instead of OneUI, even though the XSP markup is the same. In the normal case, this just means that you can reuse some of the basics when building a Bootstrap-targeted app. And that's good enough on its own: it saves you tons of work.

But the more important aspect is that it lets you focus much more on solving the task at hand and much less on writing to the layout framework.

This is a huge advantage for XPages (and any framework with this separation). As low-key as Bootstrap is (and other modern CSS frameworks are), you still have to structure your markup and sprinkle class names around to match its expectations. With a really fleshed-out set of custom renderers, though, you could potentially not care at all about the resultant UI framework when writing your app and instead focus on using stock or ExtLib controls to solve the problem directly, letting the theme+renderers do the work.

In reality, there'll be a bit of leaking (for example, using Bootstrap-friendly FontAwesome classes instead of images doesn't really fit with standard IBM controls), but I aim to minimize that as much as possible in my future projects. My goal is to use applicationLayout and standard controls for as much as I can, and instead writing a set of custom renderers for each primary theme I want to apply.

I'll have more to write about this later, but for now, the TL;DR version is: custom renderers have the potential to dramatically improve the way XPages apps are written, as long as you're willing to do the initial Java legwork.

How I Want To Use Domino, Take 2

Wed Oct 30 16:10:50 EDT 2013

  1. How I Want To Use Domino
  2. How I Want To Use Domino, Take 2

A while back, I wrote a post about how I wanted to use Domino. The gist of that was that I was enamored with the idea of using Domino as a back-end database, but not necessarily as a app-dev platform on its own - basically, how you would use a SQL or NoSQL database. Since then, I've doubled down on my use of XPages as an app-dev platform with many advantages, but I still find it very useful to imagine Domino not as "Notes apps on the web, now with a modern coat of paint", but as a collection of related but not mutually-required components competing with other web-dev stacks.

I started working on an updated take on that post, inspired by some recent posts and chat conversations I've had (and I may return to it), but then I ran across Grand Decentral Station, which is a vision of an OS/app-dev platform taking the best of the lessons of the last decade and turning them into a coherent platform. It's a compelling vision, and reading it made me realize something: most of those goals are describing Domino, or, more accurately, the Domino that could be. Take a look down the list and see how many points Domino goes about 80% towards:

  • App Installer & Updater: though Domino doesn't really handle app versions, the deployment strategy is nonetheless quite good, with all app code contained in a distinct NSF, not a bunch of files strewn in a couple directories.
  • Sandboxed apps: again, Domino doesn't quite sandbox apps, but appropriate use of ACLs can bring you close.
  • Mail server: I hear tell that Domino is capable of acting as a mail server.
  • Calendar server: with proper CalDAV support, Domino could act as a real calendar server for non-Notes clients like OS X and iOS.
  • Addressbook server: like with the calendar server, this is just a matter of adding CardDAV support.
  • Asset handling: CSS and JS optimization in XPages is a huge step in this direction.
  • Avatar server: the Directory can already act as a profile-picture server for Sametime, and something like mypic could standardize this use.
  • vCard server: well, it already serves LDAP.
  • Unified sessions: done.

And to cap this off, the prescribed per-app database is CouchDB, which is already modeled on Domino. And, of course, it already has a standard API for email, which conveniently doubles as a method of cross-app messaging in some cases, and its replication and clustering are top-notch. It's not, itself, an OS, but its fairly-cross-platform nature means that that problem is already "solved": install Domino on the server platform you're most comfortable with and it acts basically the same.

Of course, that final "20%" is, as always, the crux of the problem. Domino is only really a fully-integrated mail/contacts/calendar server when you use Notes or iNotes, XPages and legacy Domino dev are really the only games in town if you want to maintain the benefits of the NSF package, agents aren't integrated with the XSP environment and DOTS isn't a real replacement yet, there are still a number of items on the list not at all touched on by the existing stack, and licensing basically removes Domino from consideration for app development for anyone not already mentally invested in it. But hey, one can dream, no?

Thanks, Bruce

Wed Oct 02 10:55:21 EDT 2013

Tags: thanksbruce

Though I'm not as eloquent on the matter as the many others who have shared their appreciation for what Bruce Elgort has done for the community, I wanted to be sure to pitch in as well. I have been active in this community for a relatively short amount of time, but even before that Bruce's was one of the names I knew looking in from outside, from his personal contributions to the Taking Notes podcast and to his leadership of OpenNTF. And since I started being actively involved in the community, all of my interactions with him have been not only pleasant, but encouraging and inspirational.

So: thanks, Bruce!

URLs in XPages

Tue Oct 01 20:28:59 EDT 2013

Tags: urls xpages

The topic of URLs came up today in a chat and in the most recent NotesIn9, and I think this is an area that deserves some particular attention. When designing any web app, it's easy to fall into some traps with URLs, and writing XPages code juggles things around a bit - some things are much easier than usual, some are insidiously harder.

At a base level, there are four main types of URLs you'll use when writing a web app, distinguished by how they begin (don't quote me on the names, or even whether or not they're technically called "URLs"):

  • Full-protocol URLs beginning with a protocol followed by a colon, such as https://frostillic.us/f.nsf/Home.xsp
  • Protocol-relative URLs beginning with a double-forward-slash, such as //frostillic.us/f.nsf/home.xsp (these are fairly rare)
  • Server-relative URLs beginning with a forward-slash, such as /f.nsf/home.xsp
  • Page-relative URLs beginning with basically anything else, such as home.xsp

As a general rule, page-relative URLs are only a good idea if you are absolutely certain of the structure of your app relative to the URL the browser is using. The tricky part here is that it's very easy to write such a URL that works when you test it (say, pointing from one XPage to another with a URL like <xp:link value="otherXPage.xsp" text="Click me"/>), but fails in other situations. For example, see how such a URL would work when visiting each of these URLs, which all point to the same XPage:

Yes, the last one is legal - you can get to the stuff after the ".xsp" via facesContext.getExternalContext().getRequest().getPathInfo().

Fortunately, the XPage environment provides a nice solution to this problem: when using XSP elements, it converts the third type (server-relative URLs) from being relative to the server to be relative to the app. This is almost always1 what you want. So if I write a link as <xp:link value="/otherXPage.xsp" text="Click me"/>, it will work correctly in all of those situations.

Note the opening caveat: you have to be using XSP elements. Fortunately, this covers all WYSIWYG use and pretty much everything you're going to do elsewhere. It's rare that you actually want a stock-HTML <a/> instead of an <xp:link/>, and this includes resource URLs in an XPage, custom control, or theme. So, as a rule, you should almost always preface your in-app links with a forward slash to piggyback on this feature.

There are two cases that need special treatment: when you want to link out of your app from an XPage element and when you want to link into your app from a non-XPage element. Fortunately, there are two bits of non-obvious code that will do these jobs easily and reliably:

  • To break out of the app-relative "jail" in an XPage element, prefix your URL with "/.ibmxspres/domino/". For an explanation as to what this stands for and a list of similar prefix URLs, read Mastering XPages. For now, just know that that prefix means "get me out of here". So to link to, say, the directory, you could write a link like <xp:link value="/.ibmxspres/domino/names.nsf" text="Domain's Directory"/>.
  • To reliably reference the current app from a non-XPage link, use this bizarre EL formulation: ${facesContext.externalContext.requestContextPath}. What that gets you is basically like @WebDbName prefixed with a slash. So, for example, if you wanted to reference a video attached as a file resource, you could do something like <video src="${facesContext.externalContext.requestContextPath}/someVideo.mp4" />.

Keep in mind that the special XPage behavior applies only to when you're writing XSP markup (or otherwise pulling URLs into XPage elements); it doesn't apply to, for example, rich text or other HTML brought into the page.

And on a final, self-aggrandizing note, the "mentoring" offer in that NotesIn9 applies to me and my company. Feel free to contact us or get in touch with me directly if you're ever in need.

1. Except when writing CSS files. In CSS, URLs are relative to the location of the CSS file. This is usually just fine, since you, not the browser, control the URLs you use to refrence the CSS file. It gets tricky when you use resource aggregation, though.

Domino Wishlist, September 2013

Fri Sep 20 13:06:48 EDT 2013

I joined in a tweet from Paul Hannan earlier about desired Domino features (that's a very worthwhile thread to read - lots of great ideas from others) and it's had me thinking about more on my list. Fortunately, I have a blog for this kind of thing! I'm leaving off a couple big-ticket items like Eclipse-4.x-based Designer on the Mac because IBM may not be required for that.

  1. Make $WSIS not trigger crummy SSL behavior (really, first-class support for proxies in front of Domino, not just IHS/Windows)
  2. Direct access to note item data via Java
  3. Newer JRE
  4. Direct creation/update of NSFs from git server-side
  5. New XSP model-object framework
  6. New views, though that may be our job
  7. Aggressive, small-company-friendly licensing to compete with OSS stacks
  8. NSF-based SSL keychains
  9. NSF-based jvm/lib/ext distribution
  10. "Sensible defaults" mode for clean-install servers to get appropriate memory/cache/performance settings on modern setups without learning dozens of secret notes.ini settings
  11. apt-get-based distribution and updates for Domino on Debian/Ubuntu
  12. Improved XPages file-upload and -download controls to handle modern file uploading and provide better hooks for non-dominoDocument bindings
  13. Bootstrap theme for XPages. Done - thanks!
  14. Built-in default support for authentication fields in LDAP
  15. init.d startup script installed with Domino, because seriously, who ships server software without one?
  16. OS X build of Domino 64-bit
  17. Built-in Firefox sync server, because that would be a nice overture to general open-source use
  18. Support for HTTP authentication filters in Java
  19. Built-in beer database to compete with Couchbase's substantial lead in the area
  20. Easier configuration of multiple TCP/IP adapters on a server and assignment of services to bind to them
  21. Full control over URL routing inside a database (e.g. "no legacy" mode where a Java class can map requests to resources)
  22. Keep up the ODS improvements - 8.x was a great set of releases on this front
  23. CalDAV and CardDAV support - Domino should be a close-to-drop-in replacement for Gmail on OS X and iOS
  24. Along the lines of #7, a general push to establish Domino as a top-flight NoSQL database and app-dev platform, not just "your old company's mail system"
  25. A real focus on standard-API performance, with less XSP-specific cheating

I'm sure this list is not exhaustive.

Progress on the org.openntf.domino Design API

Fri Sep 06 18:57:39 EDT 2013

Tags: java openntf

The primary focus of my recent work with the OpenNTF Domino API has been the implementation of a fleshed-out design API. I've done a lot of work in my Domino-programming career manipulating design elements, usually via DXL (such as with my Forms 'n' Views app), and my aim is to codify all of that into the API itself, eventually growing it to cover almost every design element that Designer does (we'll see about Navigators, Composite Apps, and the like).

One of the big hurdles in recent years when manipulating design elements has been dealing with XPages-related elements - XPages themselves, Java design elements, Jars, faces-config.xml, etc.. Though XPages by their nature have a very nice XML representation, that doesn't carry over to DXL - all of those elements are dumped out in the "raw" data format that DXL uses when it doesn't have a better idea of how to handle what you're trying to export. And more specifically, though the XSP markup and the compiled Java code are present in the result, attempts to actually manipulate that were stymied by a "checksum" block of bits at the start of the data - you couldn't just drop the Java bytecode into the doc and expect it to work.

Fortunately, it turns out that those bits aren't a weird "checksum" at all: the data is just stored as Composite Data, the internal format of rich text in Domino. Specifically, all of those elements share the same internal representation as File Resources and Style Sheets, being composed of a CDFILEHEADER and list of CDFILESEGMENTs. When DXL uses the "raw" format, it exports that data as an array of Base64-encoded bytes directly. After discovering this, I wrote the start of a CD implementation in the API and got it to work: the API is now able to read/write the attached data of any one of those file types. Now, there are a few steps that would have to be implemented between this and "write a new XPage with a few lines of code", but this is a step in that direction (and you could theoretically modify your faces-config.xml, xsp.properties, etc. on the fly now, though I haven't tested much).

More immediately, it's also a huge step in the direction of interesting read-focused uses. One of these uses in particular has caught my fancy: loading and using XSP-related Java classes in other contexts. To go along with my work, I wrote a DatabaseClassLoader that gives you access to all of those class types from anywhere using the API (including Jar design elements). The possibilities for this have made me giddy: re-using data-model objects in external scheduled tasks, sharing classes with Java agents (...with the caveat that you have to use reflection or a scripting language), "loading" other XPage apps from inside another, treating NSFs as replicating, access-controlled Jars, and so on. I have my own notions for some practical uses I may want to explore, and I'm very interested in seeing what kind of things this lets others do as well.