I think I've settled on JSR 223, the generic "Scripting in Java" specification, as the likely best way to embed Ruby. It seems like the "correct" way to do it and generally the cleanest. I don't like the notion that the way to customize the runtime is by setting system properties, so I'm still a little wary, but it'll do for now, in any event.

The side benefit of JSR 223 (and this would be true of BSF as well) is that it supports a crapload of languages, and it does so in a very unified and generic way. Accordingly, I modified my ViewHandler and created a GenericBindingFactory to browse through the list of non-Ruby available languages and create EL bindings for each.

The upshot is that, when this code is active, any JSR-223-compliant language in your server's classpath will become available for "#{whatever: ... }" bindings just by virtue of its presence. Note, though, that that doesn't necessarily mean it will be a good experience. For one, the page's variables (like param, facesContext, view, and so forth) aren't just available - you'd have to make something like the method_missing method I wrote for Ruby to automatically resolve them via FacesContext.getCurrentInstance(). Furthermore, some languages are problematic: for some reason, Jython throws NullPointerException errors for even the most basic formulas on almost every page load, while PHP still requires <?php ?> tags and spits anything else out to the server console.

I know what question is on the tip of your tongue, though, so I won't keep you waiting: yes, you can use your home-grown string-concatenation operators in Scheme! You can breathe a sigh of relief now:

<xp:text><xp:this.value><![CDATA[#{scheme: (set! + (let ((original+ +)) (lambda args (if (and (not (null? args)) (string? (car args))) (apply string-append args) (apply original+ args))))) (+ "Hello " "from " "Scheme") }]]></xp:this.value></xp:text>

Yes, that works, and no, there's no reason to do it. If you really like Lisp, though, Scheme and Clojure can now be in your XPages toolbox.

Once I'm able to post to OpenNTF, I'll include this in my first Ruby-in-XPages release, though I may leave the applicable code commented out by default. Who knows what horrors this could unleash?

So far, my naive implementation of Ruby bindings seems to be working out well and I've been coming up with a list of the main things I want to tackle next:

1. Figure out if I'm embedding it correctly. I suspect that working this out will go a long way to solving some of the later items in this list, but it's also the best starting place anyway.

   For now, I'm using the "Core" method of embedding JRuby, which is the most straightforward way to do it: create a runtime, execute your scripts, and terminate it. It's also not so simple that it's not flexible: it has a couple different configurations for dealing with threads and variable context, so it's entirely possible that it may, in fact, be the correct way for me to do it.

   However, there are also a couple other methods: JSR 233, BSF, and directly via the JRuby API. I don't know enough yet about the XPages Java stack to know if either of the former two are at all applicable, so they may or may not be worth investigating. Direct use of the API is clearly the most flexible way to go, but I'm not certain that I NEED to get that arcane. ScriptingContainer may end up working fine. There's also this: http://jruby.org/apidocs/org/jruby/embed/osgi/OSGiScriptingContainer.html.

2. Serialization. The ScriptingContainer itself isn't Serializable, but that doesn't necessarily mean that other parts aren't or that general state can't be retained. Ideally, each XPage's Ruby environment would survive Serialization/restoreState intact, closures and all.

3. Variable lifetime. I want to figure out how best to handle variable scopes. Conceptually, I think it'd make the most sense for basic local variables ("foo = 'bar'") to be available for the duration of the page. But is that really the right way? As it stands right now, variables are local to the value binding where they're declared (as if they're variables in a function), but defined methods are available elsewhere. Here's what I mean:

   <!-- This produces an error because the second xp:text can't find foo -->
<xp:text value="#{ruby: foo = 'bar'; foo}"/>
<xp:text value="#{ruby: foo}"/>

   <!-- This produces "barbar" because the "foo" method persists to the second binding -->
<xp:text value="#{ruby: def foo; 'bar'; end; foo}"/>
<xp:text value="#{ruby: foo}"/>

   That might actually be the right way to do it. However, there are other aspects to it. For example, setting a constant (in Ruby, a constant is a variable with an initial capital letter) currently persists between pages, presumably lasting for the duration of the application. THAT certainly doesn't sound right to me. Furthermore, what about other languages? If I make it so that local variables are available between Ruby bindings, should I try to export those variables to the overall page context so that pure-EL and JavaScript can get to them? I wonder if that'd be possible with methods, too.

4. Spruce up the Domino API a bit. Ruby lets you "re-open" classes at runtime, adding whatever additional methods you want. In JRuby, this power works for Java classes, too. Allow me to demonstrate ("post" is a xp:dominoDocument):

   module Java module LotusDominoLocal class Document def [](field) self.get_item_value(field) end end end end post.document["Form"]

   I think you can imagine how this could improve your programming life.

5. Pre-compiled JRuby classes. This isn't strictly part of "Ruby-in-XPages", since it'd be done via an Eclipse builder, but I think it'd be worth looking into. JRuby can be used to generate Java classes that can be compiled ahead of time with jrubyc. The generated classes would still require the JRuby runtime and would thus be presumably slower than the equivalent "pure" Java version, but they'd be faster than interpreted or JIT-compiled Ruby that's re-generated on every page load. Ruby programmers write everything in Ruby, and I want to write my back-end logic (and agents) in Ruby too.

   I found an old Builder class I made to accomplish this task, but it's very filesystem-bound, which I'd rather avoid. Still, I'm glad I dug it up to serve as a jumping-off point.

Best of all, I think this is all eminently practical. I'm excited. Are you excited?

A bit of conversation earlier about Rails set my brain onto a disastrous course. Since I've been doing so much with Ruby lately, it's only natural to have the question of whether anything can be done related to Rails hovering nearby at all times - Rails brought Ruby to prominence and is itself an amazing piece of software with a lot to teach. It's not, however, a natural fit for Domino - Rails, to my knowledge, wears its SQL proudly, and any attempt to use Domino as a backing data source would likely be more headaches than it's worth - you'd likely be better off either using Rails+SQL or XPages+Domino.

However, at least as far as my brain is concerned, that may not have to be the end of the story. Just because the data model doesn't fit doesn't mean that Domino-the-web-server can't be useful, right? Running Rails apps packaged as Websphere WARs is an actual thing you can do for some reason and, while Websphere seems to share more characteristics with a beast from Revelations than with a piece of software I'd want to install on my server, the concept is enticing.

Imagine: a Rails app stored in an NSF in the same kind of way you'd store Java classes or other Eclipse-y resources. It'd still use a SQL database as the back end, but it would "live" inside Domino. And, since it'd be inside an NSF and not some glorified ZIP file, you'd get replication, packaging, developer ACL control, and so forth (the sky's the limit if you mess with the views). Oh, not to mention NTF templates and access to the surrounding Domino environment as needed. It might actually be possible.

It might also not be possible. Off the top of my head, I can imagine that performance concerns, getting Rails to load its files from the NSF VFS, and handing off request routing to the Rails app all have the potential to be showstoppers. It's a nice thought, though, for down the line, after the more plausible things I'm working on are settled.

Now that I have the blog in Domino and acting as an accelerating incubator for Ruby-in-XPages (man, I need a better name for that), I decided I may as well put 8.5.3's filesystem synching to good use and put the whole thing up on GitHub with everything else:

https://github.com/jesse-gallagher/frostillic.us-Blog

I've also started keeping a todo list in the README file there with my notes and future plans for the blog, the Ruby runtime, and a tentative "Misc" section.

As for the interpreter itself, it's been coming along nicely. I've got it so it only needs to create a ScriptingContainer once per request. It's not perfect, and ideally it'd only be one per Application or server - the LocalScriptingContext.CONCURRENT mode seems tailor made for this. As it is, I'm pleased that Domino can serve the HTML for the blog's home page in under 300ms, considering most scripting computation is done in an "alien" language. It's a simple app and it's not Basecamp 2 speed, but it's a start.

I have script libraries (of a sort) working. If you drop Ruby files into /WebContent/WEB-INF/ruby, you can reference them with familiar syntax:

<xp:script clientSide="false" src="/testlib.rb" type="text/x-ruby" />

Unfortunately, it doesn't yet pull in script libraries from themes (I think) and I don't know if it's available on beforePageLoad, but, again, it's a step in the right direction. Once I can get a hold of the AD103 slides from LS2012 or find similar information elsewhere, I'll look at surfacing them as a design element.

Well, I think the new blog has gone relatively smoothly, other than my accidental re-posting of my SQL-migration post (which oddly seems more popular than its first run, just one day earlier). That means it's time to get started on the next phases.

Other than the mundane setting-up-a-blog stuff like implementing search and proper draft posts, I have a lot of work to do surrounding my Ruby bindings. Since I'm going to eventually want Ruby "script libraries" and other handy non-inline uses, I'm going to need a proper Ruby editor. TextMate-via-WebDAV is OKAY, but not exactly elegant, and it involves too much overhead and awkwardness to be a great solution. Better would be to get DLTK - which has Ruby syntax highlighting - working in Designer.

On my first attempts, I ran into problems where using DLTK's update site demands an updated version of Eclipse, which could mean disaster for Designer. However, it turns out that GLTK has been part of mainline Eclipse for a while, so I figured I'd check to see if I could get to it via another route. Evidently, Designer 8.5.3 is based on Eclipse 3.4 (Ganymede), so I added Ganymede's update site to Designer's plugin installation list and, lo and behold, Ruby was there in the "Programming Languages" section. It still required the 3.4 version of org.eclipse.core, but, having backed up my Windows VM for this purpose, I felt daring. I clicked "Select Required" to select all applicable dependencies and hit Yes as many times as it took to finish the installation. After rebooting Designer, I found that Ruby scripts are now all fancily colored:

If you'd like to try this yourself, I strongly suggest you do as I did and make a backup copy of your Windows VM first (or do whatever it is people who use Windows as their host OS do when they're about to ruin their system). Designer can be very picky about what plugins you install.

I've finally done it: I've switched my blog over to Domino. I did it for a couple reasons:

• To silence the voice in the back of my head constantly saying "why are you using WordPress? You're a freaking web programmer! Write your own!"
• To put my Ruby-in-XPages code through its paces in the way only a live site can.

I've already had to fix a couple holes in my Ruby adapter, mostly revolving around the fact that I haven't bothered to properly handle serialization and JSF's StateHolder interfact. For now, I've patched the worst problems, but this gives me a great reason to come up with a proper solution. To test it out, I'm making sure to avoid "#{javascript: ...}" code blocks in favor of using plain-EL and "#{ruby: ... }" exclusively. So far, the only really awkward parts are the giant swaths of yellow-squiggly "I don't understand this" underlining in Designer and having to put xp:repeat values on xp:dataContexts rather than writing the computation inline. Not too shabby.

The whole thing's a bit shoot-from-the-hip at the moment, since it's only existed for a day. There's no search (though that'll be easy), the site design is from my old college-era blog, and I have to write the posts' HTML by hand in the Notes client (you know, like on all modern blogging platforms). But hey, I have the old posts in there, plus an archives list and, as long as it doesn't mysteriously die again like it did a minute ago, Akismet-backed commenting.

So let's see how this thing works! Don't be surprised to see the WordPress version again if things go catastrophically wrong.

I decided I should probably finally get around to moving this blog from WordPress to Domino, if for no other reason than to have a perfect testbed for the weird stuff I've been doing lately. The first task is to write an importer, so I decided to just do a straight SQL rows -> Domino documents import. This couldn't be easier if you follow this simple guide:

1. Write a Java agent that uses the appropriate JDBC connector for your database. In my case, it's MySQL, so I had it do a "show tables" to get a list of tables, then loop over those to do "select * from whatever" statements to get all documents.
2. Since it's Domino, you can just do doc.replaceItemValue(column, rs.getObject(column))!
3. Oh wait, you can't all the time. Make sure to handle a couple cases, like converting BigIntegers to longs and Timestamps to DateTimes.
4. Wait, Domino doesn't even handle Boolean? For frack's sake, FINE: row.replaceItemValue(column, cell.equals(Boolean.TRUE) ? 1 : 0);
5. Oh hey, that import went really smoothly! Now I'll just make some views based on the table name and... crap. Forgot to include the table name! Maybe I shouldn't have written this importer at four in the morning.
6. Better delete all the documents and start over.
7. What's this, an error when I went to delete all the documents in the All view? "Field is too large (32k) or..." oh no.
8. Oh crap.
9. Crap!
10. Ah, I know - I'll just write an agent to do session.CurrentDatabase.AllDocuments.RemoveAll(True) and that'll fix that.
11. Hmm, nope, that didn't work. Alright, based on the documents created around the same time, I can guess that it's the "option_value" field that's too big. Why did it even cram that data into summary if it completely breaks the document? Well, no point in dealing with that now. It's time for StampAll to take care of that!
12. Nope? Okay, how about session.CurrentDatabase.GetView("All").AllEntries.StampAll?
13. Not all categorized? What kind of error message is THAT?
14. Time to delete the database and start over!
15. Alright, THIS TIME, check to see if the String value is over, let's say, 16000 bytes and, if so, store it in a RichTextItem instead.
16. Oh nice, it worked.
17. Oh crap, I forgot to include the table name again.

And that's all there is to it!

I've made some more encouraging progress today in my quest to be able to actually use Ruby when doing XPages development. I implemented method binding (as opposed to just value binding) and found a way to shim in the interpreter so that Ruby can be used for early events like beforePageLoad. The revised code is up on my GitHub profile:

https://github.com/jesse-gallagher/Domino-One-Offs

That last part was kind of tough - I found structures for custom Application objects in the XSP Starter Kit, but I couldn't for the life of me get the faces-config.xml directives to work, like the runtime was ignoring the /faces-config/factory/application-factory node entirely. I set that aside for now and instead picked up on ViewHandlers, which can fire at the start of each page request, which works for this purpose. I wrote a custom handler that adds the RubyBindingFactory if it's not present and then otherwise hands control off to the standard Domino variant.

Another big hurdle I encountered was variable resolution. When making my proof-of-concept servlet, I inserted a couple standard variables, but that wouldn't cut it here - beyond the usual suspects like session and view, I had to make sure that Ruby could find local xp:repeat variables and other non-globals. After trying to find a way to either get a list of available variables in the current Faces context or write a proper variable resolver to attach to the Ruby runtime, I decided to just go the Ruby route for now: method_missing. When instantiating a Ruby runtime, I add facesContext as a global variable and run a bit of hard-coded Ruby to create a method_missing method to act as a variable resolver. Good enough!

So what's next? A couple things I can think of:

1. Efficiency and doing it the right way. The whole thing is still a bit too hacky for my tastes. It may end up being the best way to do it, but I haven't given up on refining it. Additionally, I currently create a new Ruby runtime for each value and method binding, which is overkill - I think there should be one per page.
2. Designer. Though Designer rarely outright complains about Ruby code (it doesn't like Ruby's #{} string-interpolation syntax for understandable reasons), I'd love to get it to stop throwing warnings around, but that's not a high priority.
3. Edge cases. xp:repeat controls appear to do their own syntax-check pass that doesn't like Ruby, so I'd like to figure out how to work around that.
4. Ruby "Script Libraries". If nothing else, I'd like to add the ability to bring in .rb file resources in a similar way to you reference Server JavaScript. Presumably, I could do this by checking, at initial page load, for controls with certain properties and then fetching the script content like I did for servlets.

I also remembered another potential related use for JRuby: writing Java classes in JRuby and having Designer translate them to Java design elements. I did a little bit with this a while ago, adding a custom builder that would fire for .rb files, use JRuby's API to translate it to Java code, and then store it. If I recall correctly, Designer's built-in builders even automatically took the Java and compiled it. If that would actually work smoothly, it should result in code performance somewhere in between pure Java and Server JavaScript.

I'll turn Domino into a developer-friendly environment if it kills me!

Oh man, I think this might actually work. Feeling adventurous this evening, I decided to look into the XPage runtime's expression language handler. After poring through tons of methods, interfaces, implementation classes, EXTENDED implementation classes, and disparate JARs, I narrowed the prefix handler down to the "FactoryLookup" property of the IBM-specific variant of facesContext's Application. With that, which is basically a hash, you map a prefix to a handler factory (it's always factories with Java, isn't it?). Once there's a handler registered, you can then bind expressions with that prefix.

This actually means what it implies:

<xp:text value="#{ruby: 'hi from ruby'}" />

And it works! Better still, the show-stopper in Designer seems to have been resolved in one of the recent versions: rather than error'ing out when it sees an unrecognized EL prefix, it nags you with a warning but otherwise proceeds without problem, saving the script as-is in the resultant Java.

Now, this is VERY much a first pass at the idea - all I did was write extremely skeletal implementations of the Factory and Binding classes and then I register them in the beforePageLoad event of a page:

var app = facesContext.getApplication()
var facts = app.getFactoryLookup()

var rfac = new mtc.ruby.RubyBindingFactory()
facts.setFactory(rfac.getPrefix(), rfac)

I can't even begin to stress how not the right way this is. It doesn't work in all controls (repeats, for example, seem to do an extra syntax check), it doesn't yet have any context from the surrounding environment, and, most importantly, who knows what horrible things it's doing to the HTTP stack?

Still, the crucial point is that it really, really looks like it can work.

The two classes I wrote, which no one should, under any circumstances, use, are here: https://github.com/jesse-gallagher/Domino-One-Offs/tree/master/mcl/ruby

Since I kind of backed my way into Domino development and administration, there are a number of areas of the server's functionality that I'm either unfamiliar with or casually brushed off as unreliable or not overly useful.

The Domain Catalog is one such area: I've been vaguely familiar with it, but have never bothered to tend to it or use it to solve problems. Fortunately, a problem it's perfectly suited to fell into my lap. In an overarching administration database, I want to get from a document with a database doclink to the database itself as quickly as possible. The easiest way, programmatically, is to use the .ServerHint and .DBReplicaID properties in a call to an unconnected NotesDatabase object's .openWithReplicaID(...) method.

This works well, but doesn't take into account the existence of local replicas - the admin database is running on one server, while the databases are created and linked to on the primary production server. Given that these servers aren't even in the same physical state, it's significantly faster to use a local replica when available, and only fall back when it hasn't been created yet. My first attempt at getting around this was direct: I first call .openByReplicaID(...) with a blank first parameter and, if it's not open, try the real server. Again, this worked, but is not ideal. It's programmatically ugly and, besides, there are more than just the two servers. .openWithFailover(...) looked promising, but seems to only accept real file paths, not replica IDs, which ruled it out for this purpose.

This is where the Domain Catalog comes in. It already contains a (hopefully-up-to-date) list of all of the replicas of every database in the domain along with everything I could need to connect to them. Furthermore, I realized I could use this information with a dash of extra intelligence: since I know which servers are physically closest, I could use that to find the "best" replica in each situation. So I made a view for this admin server ("Ganymede") of all of the database stubs, sorted first by replica ID and then by a column with this formula:

name := @Name([CN]; Server);
@If(
     name="Ganymede"; 1;
     name="Demeter"; 2;
     name="Invidia"; 3;
     name="Dionysus"; 4;
     1000
)

With replica ID in hand, this means that I can just do a @DbLookup() in the view and the first result will always be the best, regardless of which server it is. I have another column with the full "Server!!FilePath" path, so I can just pull that value and do it in one function. Nice, easy, and it lets Domino take care of the nitty-gritty details for me.