Now that I've had a bit of time, I've started trying out some ideas for new ways to do XPage development. Specifically, I'm trying out the "XPages as the Controller" setup I pondered last time. The general goal is this:

• There would be one XPage for each zone of concern (I don't know the right terminology): Posts.xsp, Users.xsp, etc.
• The XPage itself would have almost nothing on it - it would exist as a trigger for the server to load an appropriate Controller class, which would in turn handle the output, usually by picking a custom control by name.
• The custom control will handle all the appearance logic... and ONLY the appearance logic. The Controller will feed in any variables and data sources that are needed, and then the custom control will just reference those values entirely via EL.
• The Controller will marshal will either create Domino View data sources directly or will use some wrapper Model class, in order to allow the XSP environment to handle efficiency and serialization.

So far, I've made some interesting progress. I originally set out to use a ViewHandler class to find the action and feed in the variables, but I ran into trouble getting to the viewScope or an equivalent in time for on-page-load value bindings to access them - either I was too early (before createView) or too late (after createView). What I really wanted was the beforePageLoad event, so I did just that: I created a view-scoped managed bean called "controller" that has some code to check what XPage is being loaded and to find a Controller class of the equivalent name (e.g. "Posts.xsp" → "xsp.controller.Posts"). Then, I do beforePageLoad="#{controller.control}" (I apologize for the names - I may change it to "router"), and it works nicely. The controller manager thing loads the class, asks it to figure out the current action ("list", "tag", etc.), and then tries to call a method by that name via reflection.

It sounds a bit weird, and the implementation is sort of half-baked, but the result is that you end up with a class with a structure like this (with actual implementation stuff removed):

Each of those methods corresponds to an action that can either be implied (like "list" being the default for Posts.xsp, established via other code) or derived from the URL, like "/Posts.xsp/show/1" to show the post with ID "1". The code in the method would fetch the appropriate object or collection and attach it to the XPage (as a dynamically-added data source, for example). Then, the XPage would load up an appropriate custom control for the action (for example, "pages_list" or "pages_show"), which would know what values to look for in the view scope.

I'm going to keep working on this, but I can think of a couple potential muddy areas:

• I'm not sure how real data-backed custom controls (e.g. a list of posts by month in the sidebar) work into this. You can add beforePageLoad code in a custom control, but the view root is still the main page you're loading, so it's not as clean.
• I'm not sure if it's worth trying to shuttle data source actions through the controller (like "save" for a document) or best to reference the methods in the model from the XPage directly. Since the environment is already "impure", I don't know how much bending-over-backwards is warranted.
• The whole thing may be an exercise in cramming Rails-isms where they don't fit, like when someone familiar with SQL first starts programming for Domino. Just because something works doesn't mean it's correct for the platform.

I'm sure there will be more sticking points, too. Overall, though, this feels mostly good, or at least conceptually better than the other development paradigms I've tried with XPages. As the post title indicates, though, I expect this to be experiment number 1 of many.

I've been thinking more about this MVC thing, thanks to re-learning Rails. I'm fairly convinced that moving as much code as possible out of the XPage and into wrapper objects is a big improvement (convinced enough that it feels silly that wasn't doing it already), but it still feels not quite right.

Two parts of the MVC trinity are straightforward in modern Domino development: Forms and wrapper objects are the Model and XPages are the View. The Controller is where things get muddy. Off the top of my head, routing and controlling are handled by:

• Domino's built-in URL handling
• "Display XPage Instead" on Forms and Views
• Web rules in the Directory
• navigationRules on the XPage itself
• The XPage's names themselves, as used in URLs
• Anything fancy you do with document mode switching and multi-page XPages

It's kind of a mess, and very few of these are really programmable. You can do a lot with web rules, but the application itself doesn't know about them, so you end up with code that's coupled with a specific server configuration - very unsafe. Domino's built-in URLs work well enough for basic operations - show a view, open a document, edit a document - but don't understand more complex concepts, like a shopping cart "Checkout" (yes, I just re-did the depot tutorial from the Rails book).

Honestly, it might be enough to move the action code out of the XPage and into the wrapper classes. That's a big step in readability, maintainability, flexibility, and error prevention, and most sites get along just fine with a lot less. Still, it doesn't feel right.

I don't really have a better option, though. This morning, I've been kicking the idea around that you could treat the XPage itself as the Controller, sacrificing URL and code cleanliness for some power. In this setup, a main XPage would correspond to an aspect of your application (say, "AdminSite") or management tools for a data type (say, "Posts"). You would construct your URLs by chaining path info past the ".xsp" part and then use the main page (or, ideally, a shared library across all pages) to figure out what the real request is. You'd end up with an extra level to your URLs, like "Posts.xsp/search?q=foo". It's horrifyingly ugly, especially right after "whatever.nsf", but it makes a sort of sense: ":application/:controller/:action?:parameters". The alternative now is to have multiple XPages that each deal with very similar things and, if you have a large app, name them in a structured way anyway, like "Posts_List.xsp" and "Posts_Search.xsp?q=foo".

You'd have to put the individual action code in custom controls with loaded parameters to match the action - this would add yet another type of element to the custom controls list ("subforms", actual custom controls, and now the View component of each action), but it may be a necessary evil. The Extension Library has something like this, though not exact, in its "Core_DynamicPage" demo.

The whole idea seems a bit half-baked at the moment but worth investigation, so I'll have to tinker with it a bit when I have some time.

The most recent episodes of Hypercritical and Build and Analyze discuss a new third-party iOS development environment called RubyMotion. Essentially, RubyMotion is an iOS version of MacRuby: an implementation of Ruby on top of the native Objective-C runtime, allowing programmers to write native apps in Ruby. This is similar in nature to JRuby and the like, in that the Ruby objects you deal with are real objects in the underlying platform, and you manipulate them in the same way, at near-native speed, but with a different syntax.

Most of their discussion involves around the point that it's unwise to use an environment like this as a way to avoid learning Objective-C. This is absolutely true: for whatever environment you're using, you should learn the native language, because it should be the best-supported and least quirky way to write programs. In the case of OS X and iOS, this means Objective-C + Cocoa, for classic Domino development, it's Formula and LotusScript, and for newer Domino development, it's Java and JavaScript. Even when the language is really terrible, like LotusScript, you're best served by using it and letting the platform be the platform, particularly when you're learning. In classic Notes/Domino, you can use Java for agents and JavaScript-in-the-client for one or two things, but you can tell very quickly that they're not natural fits - a lot of things aren't supported, you have to do weird stuff like .recycle() every Java object, and things just don't feel "right".

So, in short: don't use a non-native language to avoid learning how program in the "real" one, particularly when that secondary language is unsupported by the vendor. It won't save you much time to begin with, since learning the library is the bigger part by far, and it's likely to cause you hassle down the line.

However, I think there's still a place for this kind of thing — as well I should, considering how much time I spend trying to cram Ruby into Domino to avoid having to write in Java and JavaScript. I think that a wrapper environment like this can be good when:

1. You are well-versed in the native language and the API, such that you wouldn't have any problem accomplishing the task you're setting out to do in it and can readily deal with platform-specific errors.
2. Similarly, you're willing and able to re-write whatever you're doing in the native language should your wrapper become unsupported and incompatible.
3. The wrapper environment/language is significantly better than the native one.

That last one is actually a point against RubyMotion for me (though I'm not a Mac/iOS developer, so I can't speak with authority), because Objective-C, particularly in its latest incarnation, is a very good language with a lot of very powerful features. Ruby is still more succinct and powerful, but I'm not sure it's SO MUCH more powerful that it's worth giving up the benefits of using the native environment and the inherent bonuses of using a strictly-C-compatible language.

Naturally, I find the differences more compelling in the case of Domino. Though Java and Server JavaScript make for a better programming environment than LotusScript did, I still find them distasteful, especially Java's endless verbosity. In that case, expending a significant amount of effort towards the goal of replacing them with a language like Ruby is very worthwhile. Lengthy Ruby programs are pretty much always going to be significantly smaller and easier to read than equivalent Java programs and Ruby supports numerous invaluable programming features that are either not present in Java or are so arcane that they may as well not be (closures & anonymous functions, class extensions & metaprogramming, symbols, easy hash literals, and duck typing, to name a few).

So as long as you go into it with eyes open and can weigh the pros and cons of adding an alien language to your platform, it can be a useful choice, but you should still pay your dues by mastering the "real" way to do it first.

A while back, I wondered about the right way to write XPages. Things have changed a bit since then - Domino has gotten a bit better, the Extension Library exists and is great, and I'm a bit more adept with the environment. The forum app, which I should probably write a post explaining one day, came together kind of like how I mentioned there - Java classes to wrap all of the Domino access, which dramatically reduced the amount of code in the XPages themselves.

Still, I'm not sure I'm doing it right most of the time.

I've never done extensive work in a proper MVC environment, but I've gotten a bit of a taste for it on the occasions that I've tinkered with Rails. I feel like the components are there in XPages, but none of the official documentation encourages using any of them, nor does Designer encourage programming that way. You can do it in bits, though - for event handlers, you can point the action to a Java method via EL (like "#{someObject.doSomething}", if I recall correctly), and the "Next page" things in the XPage's properties panel use this underlying xp:navigationRule architecture, which is probably a mix of View and Controller, but it's an interesting idea.

I may have been on to something with the forum app, but I had to create a very aggressive caching system to make it at all practical. The end result was good - very little code in the XPages themselves - but there was more "framework" necessary than I think I'm comfortable with. Still, that discomfort isn't as bad as seeing a lot of business logic in an XPage (yes, I am aware that my main side project is a way to make it all the easier to do this).

I feel like the "right" way to do it would be to really restrict the XPage itself to be a page-layout engine, concerned only with declaring widgets and connecting them to actions on data. However, you can't get very far with just the components you're given, since you're eventually going to want to, say, set a couple values on a document programmatically. You can sort of stumble along with some of the built-in actions, but that's not necessarily any cleaner... just more XML-y.

Maybe the correct thing to do would be to make it easier to expand on Controllers or Models (my MVC knowledge gets shaky here) in code in Designer so that you could, for example, define a publish action for a blog post document that handles setting the published date, changing the status, and saving it. That way, the XPage would be referencing the actual job to be done - publishing the post - while the back-end code would handle the implementation details of replaceItemValue and so forth. Maybe it'd be best to treat Forms like an object definition - a combination of instance members (fields) and actions to be performed. They're already on their way to being mostly structural support for XPages apps.

I may try to structure my next XPage like this, perhaps writing wrapper objects that take Documents or DominoDocuments in their constructor and providing all the actions that I'd want to perform on them. I could set that up as a dataContext perhaps, and then use its methods instead of adding inline code on the XPage. It's worth some experimenting, I think.

When going about making Ruby in XPages work, I had to figure out which of the three JRuby embed methods to use: Core, JSR 223, and BSF. For my purposes, it seems like the need for BSF was obviated by JSR 223 (there are some differences, but I'm not sure they matter to me), so it's really a choice between the first two. I've tried both, and I keep swapping between them (the next beta, if things stay stable, will likely switch back to Core) and between various configurations inside them, due to various problems I've run into on a couple points:

• Performance. This one is pretty straightforward - I want pages to load as fast as possible, ideally with no discernible difference between a JavaScript-only XPage and a Ruby-laden one.
• Classloading. Ruby should be able to access the same class library as Server JavaScript, such as the javax.* classes and any custom classes in the NSF. If I just go the default route for each embedding method, Ruby can't access those classes, but setting the classloader may or may not work with a given configuration.
• Clean Termination. This one's tough to tell. The "Core" route has a .terminate() method, but I'm not sure if this is strictly necessary or if it's just a good idea to do it when you're going to be spinning up a lot of runtimes, but otherwise it's fine to let the garbage collector do its thing. This should give me an excuse to try out some of that heap dump stuff.
• Independence. By this I mean that, ideally, each page load and its associated script libraries will be completely independent of future calls - for example, a class declared in a script library should only be available when that library is loaded for the page and not otherwise. This conflicts with performance heavily - the fastest way to do it is to just have a single runtime, either for the JVM (via a singleton) or for the application, but this could lead to lingering methods and classes.

On all these metrics, it seems like Core is more flexible, but JSR 223 is the "right" way to do embedded scripting, being as it is the overarching framework for scripting languages in Java now.

Regardless of which of the two I choose, I have four different types of runtimes to choose from: threadsafe, concurrent, single-thread, and singleton. Those are more or less in order of "most independent" towards "fastest". However, it's not as simple as just choosing how fast or safe I want the code to be - I've also run into weird problems with some setups regarding the classloader and OutOfMemoryErrors. When I use the singleton and single-thread routes, everything is speedy, but class loading works unpredictably - it seems to SOMETIMES use the JSF runtime classloader, sometimes not, and sometimes it's different between my dev and production servers. I'm not entirely sure what causes the memory problems, but I think I've seen them on the concurrent and threadsafe routes.

For now, I have it set to use Core with the single-thread context and calling .terminate() at the end of each request. It SEEMS to be stable, but the home page in particular takes painfully long to load. I'll have to fiddle with it some more before I can safely release another beta.

8.5.3 brought with it the very-handy "Display XPage Instead" property for views. It's great! That way, you can keep more of your existing URLs in old apps or just generally use cleaner ones in new apps - XPages are awesome, but ".xsp" in the URL is not.

In a full-blown app, you're probably going to point each view to its own XPage containing the hand-crafted fancified version. Sometimes, though, you just want to toss a xe:dynamicViewPanel on a page and that's good enough. However, unlike with the equivalent property for Forms, you can't just put a xp:dominoView data source on the page and have it pick up the view the XPage is replacing. With a Form replacement, the server translates a URL like "/view/document" to "/page.xsp?documentId=whatever&action=openDocument" for you; for a View, however, all you get is "/page.xsp". This stymied me when I just wanted to make a generic "View.xsp" to use as scaffolding for new views until I make a real page for each one.

Fortunately, though context.url (context.getUrl() in SSJS) shows the XPage path, you can use a longer property facesContext.externalContext.request.requestURI to get the original request's path info (the part after the server name - the other components are also available in that object). You can use that to fetch what's to the right of the database name (you could use ".nsf/", but facesContext.externalContext.requestContextPath + "/" would probably be technically safer in edge cases, assuming you can use ".nsf" in a folder name on a Domino server) to get the view name (extra line breaks for display purposes):

<xp:dominoView var="dbView">
	<xp:this.viewName><![CDATA[${javascript:
		java.net.URLDecoder.decode(@Right(
			facesContext.externalContext.request.requestURI,
			facesContext.externalContext.requestContextPath
			+ "/"), "UTF-8")
	}]]></xp:this.viewName>
</xp:dominoView>

Not beautiful, but it works. With that and a xe:dynamicViewPanel, you can get a sort of $$ViewTemplateDefault replacement, though you still have to specify it for each view.

My company is moving out of the offices it has inhabited for... longer than I've been alive, basically. During this move, we've had plenty of opportunities to come across relics of its past form as an instructor-led training location: old courseware, ancient versions of Windows, NetWare, and Notes/Domino (from before it was Domino), and some priceless marketing materials. Some of the posters we've found are worth a good chuckle or two (a NotesMail poster boasting "Finally, a mail system you will never have to exchange"), but this one is my absolute favorite thing related to Domino:

It's amazing, like a Domino ad out of Yellow Submarine. What I wouldn't give to have IBM's marketing to be more "Then, suddenly, bursting from a magic lotus blossom" and less "The Social server software for Socially Socializing with Social people in a Social way - just ask this guy in a tie."

Fun fact: the hooded "executioner" guy in the top right being punched out by Domino is labeled "SuiteSpot". SuiteSpot, it appears, was some sort of Groupware/web/mail/directory offering from Netscape. I guess Domino won that battle.

So apparently, this site LotusLearns has been re-posting quite a few Lotus blogs without permission, and my recent Ruby action plan post made the cut. It's bad enough to see my post aggregated without permission and plastered with ads that don't make me any money, but it's even worse that the replica of the post is thoroughly corrupt.

For one, it leaves of the "S" in the opening "So". Additionally, the XSP markup I posted is passed through to the browser unencoded, leaving a bunch of blank lines. Finally, the precious, precious Ruby code itself is entirely mangled, with the meaningful line breaks stripped (I used "white-space: pre" in the original).

So, in summary:

1. Don't re-post work without permission.
2. If you're going to illegally re-post work, at least don't mangle it, particularly when it's a post about programming.

I think I've settled on JSR 223, the generic "Scripting in Java" specification, as the likely best way to embed Ruby. It seems like the "correct" way to do it and generally the cleanest. I don't like the notion that the way to customize the runtime is by setting system properties, so I'm still a little wary, but it'll do for now, in any event.

The side benefit of JSR 223 (and this would be true of BSF as well) is that it supports a crapload of languages, and it does so in a very unified and generic way. Accordingly, I modified my ViewHandler and created a GenericBindingFactory to browse through the list of non-Ruby available languages and create EL bindings for each.

The upshot is that, when this code is active, any JSR-223-compliant language in your server's classpath will become available for "#{whatever: ... }" bindings just by virtue of its presence. Note, though, that that doesn't necessarily mean it will be a good experience. For one, the page's variables (like param, facesContext, view, and so forth) aren't just available - you'd have to make something like the method_missing method I wrote for Ruby to automatically resolve them via FacesContext.getCurrentInstance(). Furthermore, some languages are problematic: for some reason, Jython throws NullPointerException errors for even the most basic formulas on almost every page load, while PHP still requires <?php ?> tags and spits anything else out to the server console.

I know what question is on the tip of your tongue, though, so I won't keep you waiting: yes, you can use your home-grown string-concatenation operators in Scheme! You can breathe a sigh of relief now:

<xp:text><xp:this.value><![CDATA[#{scheme: (set! + (let ((original+ +)) (lambda args (if (and (not (null? args)) (string? (car args))) (apply string-append args) (apply original+ args))))) (+ "Hello " "from " "Scheme") }]]></xp:this.value></xp:text>

Yes, that works, and no, there's no reason to do it. If you really like Lisp, though, Scheme and Clojure can now be in your XPages toolbox.

Once I'm able to post to OpenNTF, I'll include this in my first Ruby-in-XPages release, though I may leave the applicable code commented out by default. Who knows what horrors this could unleash?

So far, my naive implementation of Ruby bindings seems to be working out well and I've been coming up with a list of the main things I want to tackle next:

1. Figure out if I'm embedding it correctly. I suspect that working this out will go a long way to solving some of the later items in this list, but it's also the best starting place anyway.

   For now, I'm using the "Core" method of embedding JRuby, which is the most straightforward way to do it: create a runtime, execute your scripts, and terminate it. It's also not so simple that it's not flexible: it has a couple different configurations for dealing with threads and variable context, so it's entirely possible that it may, in fact, be the correct way for me to do it.

   However, there are also a couple other methods: JSR 233, BSF, and directly via the JRuby API. I don't know enough yet about the XPages Java stack to know if either of the former two are at all applicable, so they may or may not be worth investigating. Direct use of the API is clearly the most flexible way to go, but I'm not certain that I NEED to get that arcane. ScriptingContainer may end up working fine. There's also this: http://jruby.org/apidocs/org/jruby/embed/osgi/OSGiScriptingContainer.html.

2. Serialization. The ScriptingContainer itself isn't Serializable, but that doesn't necessarily mean that other parts aren't or that general state can't be retained. Ideally, each XPage's Ruby environment would survive Serialization/restoreState intact, closures and all.

3. Variable lifetime. I want to figure out how best to handle variable scopes. Conceptually, I think it'd make the most sense for basic local variables ("foo = 'bar'") to be available for the duration of the page. But is that really the right way? As it stands right now, variables are local to the value binding where they're declared (as if they're variables in a function), but defined methods are available elsewhere. Here's what I mean:

   <!-- This produces an error because the second xp:text can't find foo -->
<xp:text value="#{ruby: foo = 'bar'; foo}"/>
<xp:text value="#{ruby: foo}"/>

   <!-- This produces "barbar" because the "foo" method persists to the second binding -->
<xp:text value="#{ruby: def foo; 'bar'; end; foo}"/>
<xp:text value="#{ruby: foo}"/>

   That might actually be the right way to do it. However, there are other aspects to it. For example, setting a constant (in Ruby, a constant is a variable with an initial capital letter) currently persists between pages, presumably lasting for the duration of the application. THAT certainly doesn't sound right to me. Furthermore, what about other languages? If I make it so that local variables are available between Ruby bindings, should I try to export those variables to the overall page context so that pure-EL and JavaScript can get to them? I wonder if that'd be possible with methods, too.

4. Spruce up the Domino API a bit. Ruby lets you "re-open" classes at runtime, adding whatever additional methods you want. In JRuby, this power works for Java classes, too. Allow me to demonstrate ("post" is a xp:dominoDocument):

   module Java module LotusDominoLocal class Document def [](field) self.get_item_value(field) end end end end post.document["Form"]

   I think you can imagine how this could improve your programming life.

5. Pre-compiled JRuby classes. This isn't strictly part of "Ruby-in-XPages", since it'd be done via an Eclipse builder, but I think it'd be worth looking into. JRuby can be used to generate Java classes that can be compiled ahead of time with jrubyc. The generated classes would still require the JRuby runtime and would thus be presumably slower than the equivalent "pure" Java version, but they'd be faster than interpreted or JIT-compiled Ruby that's re-generated on every page load. Ruby programmers write everything in Ruby, and I want to write my back-end logic (and agents) in Ruby too.

   I found an old Builder class I made to accomplish this task, but it's very filesystem-bound, which I'd rather avoid. Still, I'm glad I dug it up to serve as a jumping-off point.

Best of all, I think this is all eminently practical. I'm excited. Are you excited?