One of the very-long-term side projects I have going on is a rewrite of OpenNTF's site. While the one we have has served us well, we have a lot of ideas about how we want to present projects differently and similar changes to make, so this is as good a time as any to go whole-hog.

The specific hog in question involves an opportunity to use modern Jakarta EE technologies by way of my Domino Open Liberty Runtime project, as I do with my blog here. And that means I can, also like this blog, use the delightful Jakarta MVC Spec.

However, when moving to JEE 9.1, I ran into some trouble with the current Open Liberty beta and its handling of JSP as the view template engine. At some point, I plan to investigate to see if the bug is on my side or in Liberty's (it is beta, in this case), but in the mean time it gave my brain an opportunity to wander: in theory, I could use ERB (a Ruby-based templating engine) for this purpose. I started to look around, and it turns out the basics of such a thing aren't difficult at all, and I figure it's worth documenting this revelation.

MVC ViewEngines

The way the MVC spec works, you have a JAX-RS endpoint that returns a string or is annotated with a view-template name, and that's how the framework determines what page to use to render the request. For example:

1
2
3
4
5
6
7
8
9

@Path("home")
@GET
@Produces(MediaType.TEXT_HTML)
public String get() {
  models.put("recentReleases", projectReleases.getRecentReleases(30)); //$NON-NLS-1$
  models.put("blogEntries", blogEntries.getEntries(5)); //$NON-NLS-1$

  return "home.jsp"; //$NON-NLS-1$
}

Here, the controller method does a little work to load required model data for the page and then hands it off to the view engine, identified here by returning "home.jsp", which is in turn loaded from WEB-INF/views/home.jsp in the app.

In the framework, it looks through instances of ViewEngine to find one that can handle the named page. The default spec implementation ships with a few of these, and JspViewEngine is the one that handles view names ending with .jsp or .jspx. The contract for ViewEngine is pretty simple:

1
2
3
4

public interface ViewEngine {
  boolean supports(String view);
  void processView(ViewEngineContext context) throws ViewEngineException;
}

So basically, one method to check whether the engine can handle a given view name and another one to actually handle it if it returned true earlier.

Writing A Custom Engine

With this in mind, I set about writing a basic ErbViewEngine to see how practical it'd be. I added JRuby to my dependencies and then made my basic class:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

@ApplicationScoped
@Priority(ViewEngine.PRIORITY_APPLICATION)
public class ErbViewEngine extends ViewEngineBase {

  @Inject
  private ServletContext servletContext;

  @Override
  public boolean supports(String view) {
    return String.valueOf(view).endsWith(".erb"); //$NON-NLS-1$
  }

  @Override
  public void processView(ViewEngineContext context) throws ViewEngineException {
    // ...
  }
}

At the top, you see how a custom ViewEngine is registered: it's done by way of making your class a CDI bean in the application scope, and then it's good form to mark it with a @Priority of the application level stored in the interface. Extending ViewEngineBase gets you a handful of utility classes, so you don't have to, for example, hard-code WEB-INF/views into your lookup code. The bit with ServletContext is there because it becomes useful in implementation below - it's not part of the contractual requirement.

And that's basically the gist of hooking up your custom engine. The devil is in the implementation details, for sure, but that processView is an empty canvas for your work, and you're not responsible for the other fiddly details that may be involved.

First-Pass ERB Implementation

Though the above covers the main concept of this post, I figure it won't hurt to discuss the provisional implementation I have a bit more. There are a couple ways to use JRuby in a Java app, but the way I'm most familiar with is using JSR 223, which is a generic way to access scripting languages in Java. With it, you can populate contextual objects and settings and then execute a script in the target language. The Krazo MVC implementation actually comes with a generic Jsr223ViewEngine that lets you use any such language by extension.

In my case, the task at hand is to read in the ERB template, load up the Ruby interpreter, and then pass it a small script that uses the in-Ruby ERB class to render the final page. This basic implementation looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

@Override
public void processView(ViewEngineContext context) throws ViewEngineException {
  Charset charset = resolveCharsetAndSetContentType(context);

  String res = resolveView(context);
  String template;
  try {
    // From Apache Commons IO
    template = IOUtils.resourceToString(res, StandardCharsets.UTF_8);
  } catch (IOException e) {
    throw new ViewEngineException("Unable to read view", e);
  }

  ScriptEngineManager scriptEngineManager = new ScriptEngineManager();
  ScriptEngine scriptEngine = scriptEngineManager.getEngineByExtension("rb"); //$NON-NLS-1$
  Object responseObject;
  try {
    Bindings bindings = scriptEngine.createBindings();
    bindings.put("models", context.getModels().asMap());
    bindings.put("__template", template);
    responseObject = scriptEngine.eval("require 'erb'\nERB.new(__template).result(binding)", bindings);
  } catch (ScriptException e) {
    throw new ViewEngineException("Unable to execute script", e);
  }

  try (Writer writer = new OutputStreamWriter(context.getOutputStream(), charset)) {
    writer.write(String.valueOf(responseObject));
  } catch (IOException e) {
    throw new ViewEngineException("Unable to write response", e);
  }
}

The resolveCharsetAndSetContentType and resolveView methods come from ViewEngineBase and do basically what their names imply. The rest of the code here reads in the ERB template file and passes it to the script engine. This is extremely similar to the generic JSR 223 implementation, but diverges in that the actual Ruby code is always the same, since it exists just to evaluate the template.

If I continue down this path, I'll put in some time to make this more streamable and to provide access to CDI beans, but it did the job to prove that it's quite doable.

All in all, I found this exactly as pleasant and straightforward as it should be.

Since I've been neck-deep in LotusScript and Java for the past couple weeks, I decided to take a bit of a sanity break today and play around with Ruby. Specifically, I wrote a skeletal wrapper for the Domino Data Service in the ExtLib and the first steps of a Rails app using it a bit. I don't expect this to actually be useful down the line, or even necessarily to get any more work put into it, but it was a fun diversion.

The API takes the same general shape as the normal Domino API, except you start with a database, which can take connection and credential parameters (though the credentials don't actually work for some reason). So to access the testing database I created, I do this:

db = DHTTP::Database.new(
  :server => "api.frostillic.us",
  :path => "tests/http.nsf"
)

In the Database class, I implemented a way to get all views or to get a view by name, while views let you get entries, which in turn contain their values and a way to get documents, which provide the usual item access. For example:

db.views.each do |view|
    puts "Found view: #{view.title}"
end

view = db.get_view("TestView")
view.entries.each do |entry|
	puts "Column values: #{entry.column_values}"
	puts "Doc body: #{entry.document.Body}"
end

It's not particularly amazing, nor is it particularly efficient, but it does the job: it supports the service's paging for view entries and converts Date/Times to Ruby Times for document fields, so it could be used for light read-only use.

Then I set up an extremely bare-bones Rails app that uses the objects in the most direct and ugly way possible, but, again, it works:

class TesterController < ApplicationController
  def index
    db = DHTTP::Database.new(
      :server => "api.frostillic.us",
      :path => "tests/http.nsf"
    )
    @views = db.views
  end
  
  def view
    @title = params[:title]
    db = DHTTP::Database.new(:server => "api.frostillic.us", :path => "tests/http.nsf")
    @view = db.get_view(@title)
  end
end

The two pages just spit out lists of either the views or the view entries:

<ul>
<% @views.each do |view| %>
    <li><%= link_to(view.title, :action => 'view', :title => view.title) %></li>
<% end %>
</ul>

...and...

<ul>
<% @view.entries.each do |entry| %>
    <li><%= entry.column_values.inspect %></li>
<% end %>
</ul>

I think the Domino REST API could be really useful. When I looked into it a while ago, I ran into trouble wherein the view entry counts didn't reflect entries hidden via reader fields, which is probably still the case, but other than that I imagine it could be put to real use. It brings it more in line with the other modern NoSQL databases, which generally use HTTP/JSON-based APIs as well. Combine that with Apache for load balancing and failover and suddenly you have a really compelling modern database back-end for other platforms.

If you're interested in looking at the code, I tossed the API up on GitHub: https://github.com/jesse-gallagher/Domino-HTTP-API-for-Ruby

I've tweeted about this a couple times, but in some of my spare time lately I've put together a new, cleaner implementation of the generic-scripting-language support I first created back when I did my original work with Ruby. I named the new project "Polyglot" and it has two facets:

1. The aforementioned generic-scripting-language support, done more cleanly and with better capabilities.
2. A method for storing standalone page-generating scripts as documents in the database that are executed in the context of an XPage.

Generic Scripting Languages

With the usual when-I-get-around-to-it caveat, I plan to use Polyglot to entirely supplant Ruby-in-XPages, since its JSR-223-based features cover JRuby as well. The main thing I need to check on is proper memory usage - the embedding mechanism I originally used provides clear control over the lifetime of the scripting runtime, while JSR 223 does not. That may be fine, since it may properly manage itself, but it makes me a bit nervous.

I'll also have to work out proper context variable access (e.g. "database", "currentDocument", etc.). Some languages (Ruby, namely) seem to have gained a magic power since last I checked to access those variables without having to add special support, but I need to figure out if others work the same way or if I need to try an idea for an adapter I had.

In addition to the JSR 223 languages, I also just today added support for formula language via #{formula: ... } bindings, since I realized that that was exceedingly easy to do. For that one, I didn't bother binding it to the contextual variables, but instead just have it look for "currentDocument" - if it's present, it passes the lotus.domino.Document from that to the session.evaluate(...) call; if it's not, it forgoes the second parameter. I don't expect I'll use that binding much, but it was fun to add.

Standalone Scripts

The standalone script support is a little different - the idea is that, rather than working another language into an XPage, you write a script entirely in the other language, much like you might do with, say, PHP (don't use PHP). You can write a script using a basic editor, it's stored in a document in the database, and then you can execute it in the context of a ScriptRunner XPage, meaning you have the same Java environment as usual (with the same caveat as above that I need to sort out variable access).

Besides the usual suite of JSR 223 languages, I've also started working on integrating the XSP parser from the XPages Bazaar, both as a way to familiarize myself with the Bazaar and to see if there's any use in a setup where your XSP markup is stored in documents and evaluated at runtime rather than design elements compiled in Designer. Maybe, maybe not, but it's a fun test.

For now, it's still in an "experimental" state, but eventually I hope to cobble it into a proper releasable state and have it supersede Ruby-in-XPages entirely.

After I got Ruby in XPages to the point where it's generally working enough to power this blog, I set my sights on an even-more-important goal: being able to write backing Java classes in Ruby. While replacing SSJS is quite handy, my general use of inline scripting like that has declined significantly in favor of Java classes.

Fortunately, JRuby has a language cross-"compiler" and some hooks to write Java-compatible Ruby classes. Unfortunately, I was repeatedly stymied by a couple things:

• Having the conversion happen automatically
• Having the resultant "compiled" classes use the right class loader, allowing them to access other classes in the app
• Implementing Java Interfaces

The first one was the most work but also the first to be solved: I wrote an Eclipse builder. The latter two were tough until I realized yesterday that the answers were sitting under my nose the whole time. For the classloader, I was able to switch the compiled output from using the global JRuby runtime to a runtime stored in the application scope and set to use the loader from facesContext.getContextClassLoader(). For the interfaces, it turned out that JRuby already had another annotation for declaring implemented interfaces, unsurprisingly called "java_implements".

So the upshot of this is: now I can write classes in Ruby and have Designer automatically convert them to Java and then compile those classes, ready to be used like any "normal" Java class in an XPage app. For demonstration purposes, I whipped up a useless-in-reality class to implement DataObject:

require "java"

java_package "frostillicus"

class TestDataObject
	java_implements "com.ibm.xsp.model.DataObject"
	
	
	def initialize
		@values = {}
	end
	
	java_signature "Object getValue(Object key)"
	def [](key)
		@values[key] or "#{key.to_s} not found"
	end
	
	java_signature "void setValue(Object key, Object value)"
	def []=(key, value)
		@values[key] = value
	end
	
	java_signature "Class<?> getType(Object key)"
	def get_type(key)
		java.lang.Class.class
	end
	
	java_signature "boolean isReadOnly(Object key)"
	def read_only?(key)
		false
	end
end

Once you have that file, it gets automatically converted to Java and compiled (once you have your build path set up correctly), and then it's available for use in other Java classes, in SSJS (or Ruby-in-XPages), and as a managed bean.

As you might expect with something like this, particularly for a first draft, there are a number of caveats:

• The implementation is ugly as sin. There's a part that actually contains Java code that writes Ruby code that writes Java. It's very much a product of a series of "I wonder if this would work..." tests.
• I haven't tested it to look for weird conflicts or leaks, though I suspect that my use of application-specific runtimes will help head off the worst of those potential problems.
• You can't write classes that extend other classes. While I think you can do this within JRuby itself, the resultant objects are children of RubyObject, and there's no multiple inheritance. Interfaces will work in many cases, but that still limits the applicability.
• Since the building happens in Designer, you need to have the feature installed client-side, which is not a problem with Ruby-in-XPages.
• All those Java annotations really harsh the buzz of writing Ruby code. In that example above, the Ruby class isn't really any cleaner than a pure-Java equivalent. However, for larger, more complex classes, the expressiveness benefits of Ruby  would start to show.
• There's basically no IDE help to be had. I haven't written anything into the builder to highlight syntax errors yet and non-syntax errors (like implementing an Interface but not all the methods) only show up with build errors on the resultant Java class. Plus, there's no autocomplete, and you only get syntax highlighting if you specifically install Ruby syntax support (I think I got it from the Eclipse Ganymede update site). Some of those may be fixable with minor effort, but the really big stuff would go far beyond the amount of time I have to dedicate to this kind of thing.

For now, I tossed the first draft up on GitHub for the curious. It will take more work to be be considered anywhere near finished, but getting it this far means I can start using it in personal/demo apps and really start finding the rough edges in practical use.

For a while now, I've been fiddling with trying to make an Eclipse builder that smoothly translates Ruby files into Java classes and adds them to the project to be compiled, the idea being that, rather than only using Ruby inline in XPages or via "script libraries", you'd be able to write all of your supporting Java classes in it as well.

I'd been giving it about an hour or so of frustration every couple of weeks, but yesterday I decided to hunker down and make it work. After patching the standard "jrubyc" code to work without a real filesystem, wrestling with Ant builder class paths and runtime environments, and with a great deal of assistance from the handy XPages Log File Reader template, I got it working! I now have it so that you can write Ruby files in a ruby-src folder and build the project and the builder reads those files and plants "compiled" Java versions into a folder on the build path, which then causes Eclipse to build them into proper classes, usable on an XPage. Whoo!

However, they're not really usable yet. The way the "compiler" works is that it generates a class that extends RubyObject and, in a static block, loads up the global Ruby runtime and pre-compiles a giant string of Ruby. Then, each method you exposed to Java in the Ruby code calls the equivalent method in the Ruby runtime. It makes sense, but leads to some specific problems. For one, the global Ruby runtime's classloader doesn't know about the classes available in the XPages environment, such as the javax.faces.* classes and any of your own NSF-hosted ones. Moreover, because the object extends RubyObject and Java doesn't do multiple inheritance, it can't extend any OTHER Java classes. The internal Ruby class can extend whatever it wants and works well, but that doesn't help when you want to use the generated classes in Java code. It can implement Interfaces, but then you have to actually have Java-facing versions of every method, which can get hairy.

I have some ideas, though. The "jrubyc" compiler is handy, but it doesn't work any special magic - it just reads through the Ruby source for key elements like java_implements and java_signature and uses those to build a wrapper class that just executes a big string of Ruby code. There's nothing there I couldn't do myself, hooking into the existing parser where necessary and otherwise writing the rest myself. That way, I could generate a standard Java class on the outside but make it create a proxy object internally that's the actual RubyObject that handles all of the method calls. It could be a bit more difficult than I'm thinking, but it'd probably be possible, and it'd let me use the runtime classloader from FacesContext and extend classes properly at will.

Since I'm always gabbing on about Ruby, I figured it'd be useful to give an overview of the language and, in future posts, why it's worth using for XPages scripting. I don't plan to write an exhaustive tutorial for the language (they're aplenty on the web), but I'll go over the basic concepts and some of how it compares to EL, Server JavaScript, and Java.

Ruby is a dynamic, object-oriented language with functional-programming features and syntax designed for programmer-friendliness. In terms of other languages, it's sort of like a combination of Perl's general syntax, Smalltalk's object model, and (some of) Lisp's functional and metaprogramming abilities. Additionally, Ruby has its own unique features and conventions, such as optional parentheses, property-style getters and setters, and yield statements.

Maybe the best way to provide a gentle introduction to Ruby is to show a relatively simple but contrived code example. Here is a "Point" class meant to represent a point in two-dimensional space where the X coordinate must always be positive:

class Point < Object                                           # 1
    attr_accessor :y                                           # 2
    attr_reader :x
	
    def initialize(x=0, y=0)                                   # 3
        @y = y                                                 # 4
        self.x = x                                             # 5
    end
	
    def x=(x)                                                  # 6
        raise Exception.new("x must be >= 0") if x < 0         # 7
        @x = x
    end
	
    def to_s; "[#{@x}, #{@y}]"; end                            # 8
end

There are a lot of Ruby-isms going on, but hopefully the bulk of the code will be readable without knowing the language. Starting from the top, with a focus on the differences from Java:

1. Classes are created with just class, with no public modifier, and must begin with a capital letter, which is the language-enforced convention for class names and constants. By unenforced convention, classes and constants are CamelCase. The "<" is equivalent to Java's extends. As in Java, extending Object is not needed, but is done here for demonstration purposes. Finally, # is Ruby's single-line comment delimiter, like // in Java or ' in LotusScript.
2. attr_accessor and attr_reader are built-in Ruby class methods that generate getters+setters and getters only, respectively. The ":"s in :x and :y indicate that those are Symbols. Symbols are interesting beasts, but in this case think of them as meaning "the things named 'x' and 'y'".
3. The initialize method is equivalent to Java's constructors and LotusScript's New. This one takes two parameters, but provides default values in case one or both are left out. While Ruby objects are strongly typed in that they all have a class, Ruby variables are typeless and not statically checked.
4. The "@" symbol is used to denote instance variables, which, like all Ruby variables, don't need to be declared before they're used the first time. In this case, the line is equivalent to "this.y = y;" in Java.
5. "self.x = x" is used here to indicate that the first "x" is calling the current object's x= method as opposed to setting the local variable "x" to itself. This line is equivalent to "this.setX(x);" in Java.
6. Ruby, like C++, allows operator overloading, including this special type to define setters. This allows setting of the object's "x" parameter with code like "some_point.x = 3".
7. This line is jam-packed!
   • Ruby's exceptions are essentially like Java's, except they use raise and begin/rescue instead of throw and try/catch.
   • Objects in Ruby are constructed by calling the new method on the class itself, rather than having new be a language keyword.
   • Ruby allows for conditional and looping statements at the end of single lines as syntactic sugar.
   • Ruby allows parentheses to be dropped when the resultant code is unambiguous.
8. This line is pretty packed as well:
   • to_s is Ruby's equivalent to Java's toString().
   • Ruby, like JavaScript, uses optional semicolons. Here, they're used to pack the method definition into one line.
   • When a method takes no parameters, the () after the name is optional, both in definition and in use.
   • Ruby strings allow for code interpolation via #{...}. It's like value bindings in XPages or the equivalent feature in PHP or Perl - it allows Ruby code to be embedded in a string instead of +-based concatenation.
   • Like Server JavaScript and Formula Language, the last line in a Ruby method (except in initialize and assignment methods) is an implied return.
   • By convention, Ruby uses all-lowercase underscore_separated_words for variable and method names.

That ended up being a bit more complicated than I had expected, but hopefully it's a reasonable start (if not, there are definitely better Ruby tutorials on the web). In the next post in this thread, I'll describe some other unique or unusual Ruby features and concepts, and later I'll go into some more applicable examples to explain why Ruby is worth using in XPages generally.

I think I've settled on JSR 223, the generic "Scripting in Java" specification, as the likely best way to embed Ruby. It seems like the "correct" way to do it and generally the cleanest. I don't like the notion that the way to customize the runtime is by setting system properties, so I'm still a little wary, but it'll do for now, in any event.

The side benefit of JSR 223 (and this would be true of BSF as well) is that it supports a crapload of languages, and it does so in a very unified and generic way. Accordingly, I modified my ViewHandler and created a GenericBindingFactory to browse through the list of non-Ruby available languages and create EL bindings for each.

The upshot is that, when this code is active, any JSR-223-compliant language in your server's classpath will become available for "#{whatever: ... }" bindings just by virtue of its presence. Note, though, that that doesn't necessarily mean it will be a good experience. For one, the page's variables (like param, facesContext, view, and so forth) aren't just available - you'd have to make something like the method_missing method I wrote for Ruby to automatically resolve them via FacesContext.getCurrentInstance(). Furthermore, some languages are problematic: for some reason, Jython throws NullPointerException errors for even the most basic formulas on almost every page load, while PHP still requires <?php ?> tags and spits anything else out to the server console.

I know what question is on the tip of your tongue, though, so I won't keep you waiting: yes, you can use your home-grown string-concatenation operators in Scheme! You can breathe a sigh of relief now:

<xp:text><xp:this.value><![CDATA[#{scheme: (set! + (let ((original+ +)) (lambda args (if (and (not (null? args)) (string? (car args))) (apply string-append args) (apply original+ args))))) (+ "Hello " "from " "Scheme") }]]></xp:this.value></xp:text>

Yes, that works, and no, there's no reason to do it. If you really like Lisp, though, Scheme and Clojure can now be in your XPages toolbox.

Once I'm able to post to OpenNTF, I'll include this in my first Ruby-in-XPages release, though I may leave the applicable code commented out by default. Who knows what horrors this could unleash?

So far, my naive implementation of Ruby bindings seems to be working out well and I've been coming up with a list of the main things I want to tackle next:

1. Figure out if I'm embedding it correctly. I suspect that working this out will go a long way to solving some of the later items in this list, but it's also the best starting place anyway.

   For now, I'm using the "Core" method of embedding JRuby, which is the most straightforward way to do it: create a runtime, execute your scripts, and terminate it. It's also not so simple that it's not flexible: it has a couple different configurations for dealing with threads and variable context, so it's entirely possible that it may, in fact, be the correct way for me to do it.

   However, there are also a couple other methods: JSR 233, BSF, and directly via the JRuby API. I don't know enough yet about the XPages Java stack to know if either of the former two are at all applicable, so they may or may not be worth investigating. Direct use of the API is clearly the most flexible way to go, but I'm not certain that I NEED to get that arcane. ScriptingContainer may end up working fine. There's also this: http://jruby.org/apidocs/org/jruby/embed/osgi/OSGiScriptingContainer.html.

2. Serialization. The ScriptingContainer itself isn't Serializable, but that doesn't necessarily mean that other parts aren't or that general state can't be retained. Ideally, each XPage's Ruby environment would survive Serialization/restoreState intact, closures and all.

3. Variable lifetime. I want to figure out how best to handle variable scopes. Conceptually, I think it'd make the most sense for basic local variables ("foo = 'bar'") to be available for the duration of the page. But is that really the right way? As it stands right now, variables are local to the value binding where they're declared (as if they're variables in a function), but defined methods are available elsewhere. Here's what I mean:

   <!-- This produces an error because the second xp:text can't find foo -->
<xp:text value="#{ruby: foo = 'bar'; foo}"/>
<xp:text value="#{ruby: foo}"/>

   <!-- This produces "barbar" because the "foo" method persists to the second binding -->
<xp:text value="#{ruby: def foo; 'bar'; end; foo}"/>
<xp:text value="#{ruby: foo}"/>

   That might actually be the right way to do it. However, there are other aspects to it. For example, setting a constant (in Ruby, a constant is a variable with an initial capital letter) currently persists between pages, presumably lasting for the duration of the application. THAT certainly doesn't sound right to me. Furthermore, what about other languages? If I make it so that local variables are available between Ruby bindings, should I try to export those variables to the overall page context so that pure-EL and JavaScript can get to them? I wonder if that'd be possible with methods, too.

4. Spruce up the Domino API a bit. Ruby lets you "re-open" classes at runtime, adding whatever additional methods you want. In JRuby, this power works for Java classes, too. Allow me to demonstrate ("post" is a xp:dominoDocument):

   module Java module LotusDominoLocal class Document def [](field) self.get_item_value(field) end end end end post.document["Form"]

   I think you can imagine how this could improve your programming life.

5. Pre-compiled JRuby classes. This isn't strictly part of "Ruby-in-XPages", since it'd be done via an Eclipse builder, but I think it'd be worth looking into. JRuby can be used to generate Java classes that can be compiled ahead of time with jrubyc. The generated classes would still require the JRuby runtime and would thus be presumably slower than the equivalent "pure" Java version, but they'd be faster than interpreted or JIT-compiled Ruby that's re-generated on every page load. Ruby programmers write everything in Ruby, and I want to write my back-end logic (and agents) in Ruby too.

   I found an old Builder class I made to accomplish this task, but it's very filesystem-bound, which I'd rather avoid. Still, I'm glad I dug it up to serve as a jumping-off point.

Best of all, I think this is all eminently practical. I'm excited. Are you excited?

I've made some more encouraging progress today in my quest to be able to actually use Ruby when doing XPages development. I implemented method binding (as opposed to just value binding) and found a way to shim in the interpreter so that Ruby can be used for early events like beforePageLoad. The revised code is up on my GitHub profile:

https://github.com/jesse-gallagher/Domino-One-Offs

That last part was kind of tough - I found structures for custom Application objects in the XSP Starter Kit, but I couldn't for the life of me get the faces-config.xml directives to work, like the runtime was ignoring the /faces-config/factory/application-factory node entirely. I set that aside for now and instead picked up on ViewHandlers, which can fire at the start of each page request, which works for this purpose. I wrote a custom handler that adds the RubyBindingFactory if it's not present and then otherwise hands control off to the standard Domino variant.

Another big hurdle I encountered was variable resolution. When making my proof-of-concept servlet, I inserted a couple standard variables, but that wouldn't cut it here - beyond the usual suspects like session and view, I had to make sure that Ruby could find local xp:repeat variables and other non-globals. After trying to find a way to either get a list of available variables in the current Faces context or write a proper variable resolver to attach to the Ruby runtime, I decided to just go the Ruby route for now: method_missing. When instantiating a Ruby runtime, I add facesContext as a global variable and run a bit of hard-coded Ruby to create a method_missing method to act as a variable resolver. Good enough!

So what's next? A couple things I can think of:

1. Efficiency and doing it the right way. The whole thing is still a bit too hacky for my tastes. It may end up being the best way to do it, but I haven't given up on refining it. Additionally, I currently create a new Ruby runtime for each value and method binding, which is overkill - I think there should be one per page.
2. Designer. Though Designer rarely outright complains about Ruby code (it doesn't like Ruby's #{} string-interpolation syntax for understandable reasons), I'd love to get it to stop throwing warnings around, but that's not a high priority.
3. Edge cases. xp:repeat controls appear to do their own syntax-check pass that doesn't like Ruby, so I'd like to figure out how to work around that.
4. Ruby "Script Libraries". If nothing else, I'd like to add the ability to bring in .rb file resources in a similar way to you reference Server JavaScript. Presumably, I could do this by checking, at initial page load, for controls with certain properties and then fetching the script content like I did for servlets.

I also remembered another potential related use for JRuby: writing Java classes in JRuby and having Designer translate them to Java design elements. I did a little bit with this a while ago, adding a custom builder that would fire for .rb files, use JRuby's API to translate it to Java code, and then store it. If I recall correctly, Designer's built-in builders even automatically took the Java and compiled it. If that would actually work smoothly, it should result in code performance somewhere in between pure Java and Server JavaScript.

I'll turn Domino into a developer-friendly environment if it kills me!

Oh man, I think this might actually work. Feeling adventurous this evening, I decided to look into the XPage runtime's expression language handler. After poring through tons of methods, interfaces, implementation classes, EXTENDED implementation classes, and disparate JARs, I narrowed the prefix handler down to the "FactoryLookup" property of the IBM-specific variant of facesContext's Application. With that, which is basically a hash, you map a prefix to a handler factory (it's always factories with Java, isn't it?). Once there's a handler registered, you can then bind expressions with that prefix.

This actually means what it implies:

<xp:text value="#{ruby: 'hi from ruby'}" />

And it works! Better still, the show-stopper in Designer seems to have been resolved in one of the recent versions: rather than error'ing out when it sees an unrecognized EL prefix, it nags you with a warning but otherwise proceeds without problem, saving the script as-is in the resultant Java.

Now, this is VERY much a first pass at the idea - all I did was write extremely skeletal implementations of the Factory and Binding classes and then I register them in the beforePageLoad event of a page:

var app = facesContext.getApplication()
var facts = app.getFactoryLookup()

var rfac = new mtc.ruby.RubyBindingFactory()
facts.setFactory(rfac.getPrefix(), rfac)

I can't even begin to stress how not the right way this is. It doesn't work in all controls (repeats, for example, seem to do an extra syntax check), it doesn't yet have any context from the surrounding environment, and, most importantly, who knows what horrible things it's doing to the HTTP stack?

Still, the crucial point is that it really, really looks like it can work.

The two classes I wrote, which no one should, under any circumstances, use, are here: https://github.com/jesse-gallagher/Domino-One-Offs/tree/master/mcl/ruby

I've been toying with my Ruby servlet a bit this evening and it didn't take long to start having some fun. For example, here's a snippet from a page I'm building with Markaby, which is an aging little library that makes building HTML pages declaratively a cinch:

$database.views.sort { |a, b| a.name <=> b.name }.each do |view|
li { a view.name } unless view.name =~ /^\(.*\)$/
end

That prints out the names of all the non-hidden views in the database, sorted alphabetically, inside an HTML list. That's barely scratching the surface and not often useful, of course, but it's a proof of concept.. Server JavaScript can do some cool things, but the required syntax makes it a classless language by comparison.

I also realized earlier today that the same idea here can be translated to writing agents in Ruby by creating them as Java agents and putting the Ruby code in attached Resource files. It's a BIT of a hassle, in that I can't use the WebDAV trick to edit the scripts with TextMate, and it seems like I have to manually rebuild the agent whenever I change the script file, but that's not too bad. I'll probably try it out on some non-critical agents on my dev server to see if there are any critical memory issues.

Next step: figuring out this newfangled OSGi doohickey.

As is no doubt clear by now, one of my obsessions when it comes to Domino is trying to make my programming life better, and one of the best ways I can think of accomplishing that is if I can make it so I can program in Ruby instead of one of the godforsaken languages natively supported.

In general, my attempts towards this goal have fallen into three categories:

1. Accessing Domino from Ruby as one might a normal database. The very-much-in-progress fruit of this is my Domino API for Ruby project.
2. Using Ruby as an alternative scripting language for XPages, like "#{ruby: blah blah blah}". This would be nice, but I haven't made any progress towards it.
3. Running Ruby scripts out of an NSF via the web, as one might a non-scheduled agent

The last one has been the one at which I've taken the most swings, and I actually made some progress along those lines today. I decided to give a shot to using Domino's ancient servlet support combined with JRuby to accomplish this, and I think I've done it.

Since this is Domino, there were a couple hurdles:

1. Servlets aren't automatically authenticated as the current Domino user. When I was looking into this, I read a lot about using NotesFactory.createSession(null, token), where "token" is the value of the LtpaToken cookie in an SSO configuration, which I use. However, this didn't work for me, possibly due to using site documents. Fortunately, there appears to be a better way: NotesFactory.createSession(null, req), where "req" is the HttpServletRequest object passed in to the servlet's method. This works splendidly, giving me a session using the current user without having to worry about figuring out their password or dealing with tokens manually.
2. Finding the context database and script. The servlet is triggered properly by ".rb" extensions in file resources, but that still left the problem of actually opening that resource. Fortunately, the Session class has a resolve(String) method that can be used to get a Domino product object from a Notes URL. For file resources, this uselessly gives back a Form object, but you can get its UNID from the getURL() method.
3. Reading the contents of the script. File resources appear to store their data in a $FileData rich text field, but that doesn't make it easy to deal with. For now, at least, I've gone the DXL route: I use the Document's generateXML() method, find the BASE64-encoded value in the <filedata> element, and decode that with sun.misc.BASE64Decoder.
4. Actually executing the script. I banged my head against this for a long time: I kept getting NoClassDefFound exceptions for org.jruby.util.JRubyClassLoader when trying to run the script. I assumed at first that this was a problem with the class loader not finding this secondary class, so I tried tons of stuff with moving the JAR around, repackaging my servlet, and even trying manual class loading.  However, the exception was a red herring: it was really a permissions thing, so, not wanting to take any guff from a JVM, I granted myself all permissions and it worked great.
5. Editing the Ruby script. This part wasn't really a crucial flaw, but the fact that Designer doesn't know about Ruby meant that editing the file resource involved no syntax highlighting or other fancy features. DLTK doesn't seem to play nicely with the Eclipse version in Designer and I've ruined my installation by messing with that stuff too much in the past, so I decided to try WebDAV. Unfortunately, Domino runs the servlet on WebDAV requests too, so, rather than trying to figure out how to handle that nicely, I just switched to accessing the database via a clustered server that doesn't run the servlet.

It was a lot of hassles, but it seems to work! The cleaned-up version of the method I used is:

1. Put the "complete" JRuby JAR into Domino's jvm/lib/ext folder.
2. Put the compiled class file for my servlet into Domino's data/domino/servlets folder.
3. Grant all permissions to the JVM.
4. Enable servlets for the server in question and specify "rb" as one of the file extensions.
5. Enable the servlet in servlets.properties, as in my example.
6. Restart HTTP/the server.
7. Add .rb Ruby scripts as file resources to a database. I made one that makes a list of some documents from a view in my DB.
8. Open the resource in a web browser and see it work! Hopefully!

One caveat: though I set the error and output streams of the Ruby environment to be the HTTP output, this seems to not always work, so I've found it best so far to use "$response.writer.println" instead of "puts" for writing text to the stream. I'll see if I can fix that. Turns out writing to the HTTP output should be done by calling setWriter(Writer) rather than setOutput(Writer).

Additionally, that "$response" there isn't a built-in feature of JRuby - it's a variable I put into the runtime. I do this along with "$request", "$session", and "$database". Think of them like the equivalent variables in XPages.

Some questions you may have are "is it stable?" and "is it fast?" and to those I have a simple answer: beats me! I just got it to work a bit ago and I don't know if I'll ever even use it. It's quite exciting, though, and that's what really matters.

https://github.com/jesse-gallagher/Domino-One-Offs

There are a couple things about writing my Ruby wrapper for the C API that make it particularly fun, mostly related to getting to add abilities that I desperately wish were there in the normal APIs.

1. Ruby-style (forall) looping. Anyone who has iterated over a NotesDocumentCollection knows the drill: set a variable to the first element, start a while loop, and make sure to set the variable to the next one at the end. Writing it one time isn't so bad. Writing it hundreds of times, though? It gets to be a drag. Getting to write docs.each { |doc| ... } is a breath of fresh air.
2. Easier design-elements-as-notes access. The normal API lets you get a NotesDocument version of a NotesView via its UniversalID property, but for everything else you need a NotesNoteCollection, which is a hassle. Since all design elements are Documents anyway, I've just made their wrapper objects subclasses of Document (though I may change that to just a #document method if it gets hairy) and I've put a #get_design_note method on Database that lets you find a design note by name and flag class (NIFFindDesignNoteExt).
3. HTML and DXL everywhere. I use DXL fairly constantly (mostly for design elements), and for the most part it's a hassle. Not only do you have to create a NotesDXLExporter, but you also have to get the Document version of the design note you're dealing with and run through its process. Not impossible, but sort of a hassle. Now, though, I just put a #to_dxl method on everything - this is like the .generateXML() method in Java, but consistently applied. Similarly, I can't count the number of times when it would have been handy to get an HTML representation of some element, even if it was just the dated stuff that the legacy renderer puts out. Since that's all there in the C API, I just put a #to_html method on everything and a #get_item_html method on Document for very easy access to web-friendly versions of MIME and Rich Text items.

It's just a shame that I probably won't be able to use this: what I'd really want to do would be to use it like a database driver on a Ruby-driven web site, but the fact that it's so tied to ID files makes that tough. Still, it's great exercise to write it, and maybe I'll cave and set up some sort of multi-process hydra beast to suit my needs.

As I had mentioned before, I've been tinkering about with the Domino C API, specifically with Ruby. Although I'm not sure I'll actually have a use for it (from what I can tell, the C API is very tied to ID files and threads, which would make a multi-user web server thing cumbersome), I've decided to go for it and write a wrapper for the API generally based on the Java/LotusScript API. This is serving a number of purposes:

1. It gets me off my (metaphorical) duff and in front of a text editor during my off hours.
2. It lets me work with Ruby in a real capacity. And moreover, it'll expose me to how to write a Ruby library.
3. It's introducing me to proper version control, something I absolutely need to do better.
4. It will eventually force me to write structured documentation, which I haven't had to do in the past.
5. It should fill a hole currently only partially filled with OLE-based libraries, though the ID-file thing will make it awkward.
6. It brings me back to the world of C structures and pointers, albeit cushioned by FFI.
7. By the time I'm done, I'll know Domino inside and out.

So far, I have some of the basics working: creating a "Session" by passing in the program directory and the path to an INI file, fetching databases, traversing and full-text-searching views, reading view entries, and reading basic and simple MIME data from documents. Obviously, there's plenty more work to do, and I don't know if I'll bother implementing the more esoteric classes like NotesRegistration, but it's off to a fun start so far:

https://github.com/jesse-gallagher/Domino-API-for-Ruby

Point 7 on that list has proven surprisingly interesting. It's fun to see the bits that apparently never fully made it onto the product (like number ranges), the little optimized paths where the documentation goes out of its way to point out that it's particularly efficient, and all the data structure sizes that create the various limitations that make Domino programming such an... experience.