Because I'm me, one of the paths I'm investigating for my long-term blog-reformation project is seeing if I can get Ruby on Rails in there. I've been carrying a torch for the language and framework for forever, and so it'd be good to actually write a real thing in it for once.

This has been proving to be a very interesting thing to try to do well. Fortunately, the basics of "run Rails in a Java server" have been well worked out: the JRuby variant of the language is top-notch and the adorably-named Warbler project will take a Rails app and turn it into a JEE-style WAR file or self-hosting JAR. That still leaves, though, a few big tasks, in order of ascending difficulty:

1. Cramming a Warbled Rails app into a Maven build
2. Getting the Rails app to see the Java resources from the other parts of the tree
3. Initializing Darwino tooling alongside Rails
4. Making this pleasant to work with

So far, I've managed to get at least a "first draft" answer to the first three tasks.

Cramming a Warbled Rails app into a Maven build

When you get to the task of trying to do something unusual in Maven, the ideal case is that there will be a nice Maven plugin that will just do the job for you. Along those lines, I found a few things, ranging from a tool that will assist with making sure your Gems (Ruby dependencies) are handled nicely to one that outright proxies Gems into Maven dependencies. However, none that I found quite did the job, and so I fell back to the ol'-reliable second option: just shell out to the command line. That's not ideal (for reasons I'll get to below), but it works.

I ended up putting the Rails app into src/main/rails/blog and then using the exec-maven-plugin to do the Warbling for me:

<plugin>
	<groupId>org.codehaus.mojo</groupId>
	<artifactId>exec-maven-plugin</artifactId>
	<executions>
		<execution>
			<id>create-final-war</id>
			<phase>package</phase>
			<goals>
				<goal>exec</goal>
			</goals>
			<configuration>
				<executable>/bin/sh</executable>
				<workingDirectory>.</workingDirectory>
				<arguments>
					<argument>-c</argument>
					<argument>
						rm -f src/main/ruby/blog/*.war
						cd src/main/ruby/blog &amp;&amp; \
						jruby -S bundle install &amp;&amp; \
						jruby -S warble executable war &amp;&amp; \
						cd ../../../.. &amp;&amp;
						mv src/main/ruby/blog/*.war target/${project.build.finalName}.war
					</argument>
				</arguments>
			</configuration>
		</execution>
	</executions>
</plugin>

This amounts to a shell script that clears out any previous build, makes sure the dependencies are up to date (jruby -S bundle install), creates a self-hosting WAR file (jruby -S warble executable war), and then copies that result to the name expected by normal Maven WAR packaging. This basically works.

Getting the Rails app to see the Java resources from the other parts of the tree

Now that I had a properly-building WAR, my next task was to bring in any dependency JARs and in-project Java classes for use at runtime. Fortunately, this is a job that Warbler can handle, by way of its config/warble.rb file. In the root of the blog project, I ran warble config to generate this stub file. Like almost everything else in Rails, the configuration is done in Ruby, and this file is a large block of Ruby code examples, mostly commented out. I adjusted the lines to copy in the dependency JARs (which Maven, in a WAR package, will have previously copied for me) and to copy in any "loose" Java classes I may have alongside Rails in the current project:

config.java_libs += FileList["../../../../target/frostillicus-blog/WEB-INF/lib/*.jar"]
config.java_classes = FileList["../../../../target/frostillicus-blog/WEB-INF/classes/**/*"]
config.pathmaps.java_classes << "%{../../../../target/frostillicus-blog/WEB-INF/classes/,}p"

These lines use a helper class named FileList to glob the appropriate files from the project's target directory and copy them in. In the case of the loose classes, I also had to figure out how to clean up the path names - otherwise, it created a bizarre directory structure within the WAR.

With those lines in place, Warbler set up everything nicely - I could reference code from any of the dependencies, the other modules, or anything from the src/main/java folder within the same module.

Initializing Darwino tooling alongside Rails

The last step I got working is related to the previous one, but has a couple wrinkles. In addition to just having the Darwino classes available on the classpath, a Darwino application has an initialization lifecycle, done in a JEE app via filters defined in web.xml. It may also have some support files for defining beans and properties, which aren't covered by the same process as above. To start with the latter, I needed to figure out how I was going to get the files included in the "normal" JEE project's WEB-INF folder copied into the Rails WAR without destroying anything else. Fortunately, the same config file had a hook for that:

config.webinf_files += FileList["../../webapp/WEB-INF/**/*"] - ["../../webapp/WEB-INF/web.xml"]
config.pathmaps.webinf_files = ["%{../../webapp/WEB-INF/,}p"]

This one is basically the same as above, but with an important subtraction: I want to make sure to not copy the normal app's web.xml file in. If that's copied in, then Warbler will respectfully leave it alone, which would mean that the Rails portion of the app won't be launched. I'm handling that specially, so I made sure to use Ruby's "array subtraction" shorthand to make sure it's not included.

So that left modifying the web.xml itself, in order to serve two masters. Both Darwino and Rails expect certain filters to happen, and so I copied Warbler's web.xml.erb template into the config directory for modification. .erb is the designation for "embedded Ruby", and it's a technique Ruby tools frequently use for sprinkling a bit of Ruby logic into non-Ruby files, with a result that's similar to PHP and other full-powers templating languages. The resultant file is essentially a mix of the stock file created by Darwino Studio and the Warbler one, with some of the Darwino additions commented out in favor of the Rails stack.

Making this pleasant to work with

This final part is going to be the crux of it. Right now, the development process is a little cumbersome: the Rails app is essentially its own little universe, only fused with the surrounding Java code by the packaging process. That means that, even if I got a great Rails IDE, it wouldn't necessarily know anything about the surrounding Java code (unless they're smarter than I'd think). More importantly, the change/view-live loop is lengthy, since I have to make a change in the app and then re-run the Maven build and re-launch the embedded server. I lose the advantages both of Eclipse's built-in run-on-Tomcat capabilities as well as the normal Rails self-hosting hot-code-replace capabilities.

Fortunately, at least for now, the awkwardness of this toolchain may be primarily related to my lack of knowledge. If I can find a way to automate the Warbling inside Eclipse, that would go a tremendous way to making the whole thing a mostly-smooth experience. One potential route to this would be to create a Maven plugin to handle the conversion, and then include an m2e adapter to get it to conform to Eclipse's expectations. That would be a tremendous boon: not only would it be smoother to launch, but it would potentially gain the benefit of referencing workspace projects directly, lessening the need to worry about Maven installation. That would be a good chunk of work, but it's an area I'd like to dive into more eventually anyway.

In the mean time, the latest state of the conversion is up on GitHub for anyone curious:

https://github.com/jesse-gallagher/frostillic.us-Blog

This post follows in my tradition of extremely-narrow-use-case guides, but perhaps this will come in handy in some situations nonetheless.

Specifically, a while back, I wrote a script that "Maven-izes" the XPages artifacts, as provided by IBM's Update Site for Build Management. This may seem a bit counter-intuitive at first, since the entire point of that download is to be able to compile using Maven, but there's a catch to it: the repository is still in Eclipse ("P2") format, which requires that you use Tycho in your project. That's fine enough in most cases - since Domino-targetted projects are generally purely OSGi, it makes sense to have the full OSGi stack that Tycho provides. However, in a case where Domino is only one of many supported platforms, the restrictions that Tycho imposes on your project can be burdensome.

So, for those uses, I write a JRuby script that reads through the P2 site as downloaded and extracted from OpenNTF and generates best-it-can Maven artifacts out of each plugin. It tries to maintain the plugin names, some metadata (vendor, version, etc.), and dependency hierarchy, and the results seem pretty reliable, at least for the purpose of getting a non-Tycho bundle with XSP references to compile. This isn't necessarily a route you'd want to take in all cases (since you don't get the benefits of normal OSGi resolution and services in your compilation), but may make sense sometimes. In any event, if it's helpful, here you go:

https://github.com/jesse-gallagher/Miscellany/blob/master/UpdateSiteConversion/convert.rb

This continues to be a very interesting time for Domino developers, with the consternation of MWLUG giving way to IBM's recent announcement about their plans for Domino. Like everyone, I have my feelings about the matter, but the upshot is that moving-forward tone still stands. With that in mind, let's get down to business, shall we?

I'm going to kick off my long-term blog series of moving my blog itself over to a Darwino+JEE application. I say "long-term" because it's a pure side project, and I still have a lot of decisions yet to make with it - for one, I haven't even decided on which toolkit I'll be using for the UI. However, since this will have the additional effect of being a demo for Darwino, I want to lay the groundwork early. And hey, that's one of the benefits - since the data will keep replicating, I can take whatever time I need on the new form while the old one chugs along.*

As I have time to work on it, I'll go over the steps I've taken and put the current state up on GitHub. These descriptions are meant to be somewhere in between an overview and a tutorial: I'll cover the specific steps I took, but I'm leaving out a lot of background info for now (like installing a PostgreSQL database, for example).

To start out with, I'll create the basic projects. A prototypical Darwino application consists of a two-tiered structure of Maven modules: a root container module and then several related projects. In my case, the tree looks like this:

Since my plans are simple, this only has a few of the potential modules that could be created, but retains the separation-of-concerns structure. The purposes of these projects are, in order of importance:

frostillicus-blog-shared

This project holds the database definition and any business logic that should be shared across each UI for the application. Most of my Java code will go here (unless I pick a Java-heavy UI toolkit, I guess). Any model objects, servlet definitions, scheduled tasks, and so forth will go here.

frostillicus-blog-webui

This project holds the assets used by the web UI of the application, which could be shared by the JEE project and any "hybrid" web-based mobile UIs I make (which I won't for this). By default, this contains a skeletal Ionic-based UI.

frostillicus-blog-j2ee

This project holds the scaffolding for the JEE servlet app ("J2EE" still has a better ring to it than "JEE", though). Depending on where I go with the UI, this will either be a small shim just to get HTML served up or a larger server-side toolkit project.

In a different situation, there could be up to five additional modules: native and hybrid UIs for Android, the same pair for iOS, and an OSGi plugin shim for running on Domino. I may end up wanting to run this on my existing Domino server, but I don't have a need for offline mobile access just for my blog, so I'll definitely be skipping those. (Edit: I forgot one potential UI project: a SWT front-end for desktops)

For the most part, the default created classes do what I want them to do to get it started. There's really only one tweak to make: since I'll want to be able to search, I know up front that I want to enable FT searching in the Darwino DB. This sort of thing is done in the created AppDatabaseDef class. There's quite a bit that you can do there, but I'll mostly just uncomment the lines that enable full-text search in the loadDatabase method:

@Override
public _Database loadDatabase(String databaseName) throws JsonException {
	if(!StringUtil.equalsIgnoreCase(databaseName, DATABASE_NAME)) {
		return null;
	}
	_Database db = new _Database(DATABASE_NAME, "frostillic.us Blog", DATABASE_VERSION);

	db.setReplicationEnabled(true);
	
	db.setInstanceEnabled(false);
	
	{
		_Store _def = db.getStore(Database.STORE_DEFAULT);
		_def.setFtSearchEnabled(true);
		_FtSearch ft = (_FtSearch) _def.setFTSearch(new _FtSearch());
		ft.setFields("$"); //$NON-NLS-1$
	}

	return db;
}

That will ensure that, when the database is deployed (or, if I make changes later, upgraded), FT search is on. The line ft.setFields("$") uses a bit of JSONPath, basically saying "start at the root, and cover all fields". I'll probably come back to this class later to add some more optimizations, but that can wait until I'm sure how the structure of the app will take form.

The last step for now is to set up replication between Domino and this fledgling Darwino app. To do that, I'll set up an adapter definition in the Sync Admin database (the Domino-side application that manages Darwino replication):

The code on the page is the DSL used to define the mapping between Domino documents and Darwino's JSON docs. In this case, it's the code that is automatically generated by the "Generate From Database" tool, and everything after the first line isn't strictly necessary: without guidance, the replicator will try to translate the doc contents as best it can, and the data in this DB is pretty clean. It doesn't hurt to clamp it down a bit, though. I have it pointed to a non-replica copy of the DB for now, since I plan to do some destructive tinkering with the data when I actually make a UI that I don't want replicating back to "production" just yet, but I'll clear the Darwino side and replicate in the live data when I'm ready.

To enable replication in the Darwino app, I commented out the labeled block in the JEE project's web.xml and set a few properties in my Tomcat server's darwino.properties (to externalize sensitive information):

frostillicus_blog.sync-enabled=true
frostillicus_blog.sync-url=http://pelias-l.frostillic.us/darwino.sync

Once that was set, I launched the app using a Tomcat instance in Eclipse, and I could see it doing its thing:

Start deploying database frostillicus_blog, POSTGRESQL, 0/0
Finished deploying database frostillicus_blog, 0, 0/0
Start replication with server http://pelias-l.frostillic.us/darwino.sync
Started replication Pull frostillicus_blog, estimated entries: 5013 [September 15, 2016 2:03:37 PM EDT]
1 processed, 0% (total 5013, remaining time 1m55s, avg rate 1/s, instant rate 0/s)
664 processed, 13% (total 5013, remaining time 13s, avg rate 332/s, instant rate 332/s)
1313 processed, 26% (total 5013, remaining time 11s, avg rate 328/s, instant rate 324/s)
1941 processed, 38% (total 5013, remaining time 9s, avg rate 323/s, instant rate 314/s)
2518 processed, 50% (total 5013, remaining time 7s, avg rate 314/s, instant rate 288/s)
3086 processed, 61% (total 5013, remaining time 5s, avg rate 308/s, instant rate 284/s)
3597 processed, 71% (total 5013, remaining time 4s, avg rate 299/s, instant rate 255/s)
4069 processed, 81% (total 5013, remaining time 2s, avg rate 290/s, instant rate 236/s)
4489 processed, 89% (total 5013, remaining time 1s, avg rate 280/s, instant rate 210/s)
4929 processed, 98% (total 5013, remaining time 0, avg rate 273/s, instant rate 220/s)
5013 processed, 100% (total 5013, remaining time 0, avg rate 278/s, instant rate 84/s)
+++ Finished, 5013 processed (estimated 5013, time 18s, avg rate 278/s)

Once that was done, I went over to the default utilitarian web UI to make sure everything looked good, and it did:

(the advice about reverse proxies still stands, by the way)

Nothing app-specific in there yet (and I haven't hooked up authentication to Domino yet, so I don't have my Gravatar icon), but it shows that the data made the trip none the worse for wear, authors field and all.

That will do it for the initial phase. I plan to revisit this down the line, once I've made a decision on a UI framework and have the time to actually start implementing that. That will be an interesting one - there are strong reasons to make single-page applications in JavaScript, but my heart is still in server-side toolkits. That will be a choice for another day, though.

---

* If you'll forgive the blatant sales pitch.

I had the pleasure of taking part in the long-awaited demise episode of This Week In Lotus!

http://thisweekinlotus.com/115-doing-a-three-way/

We covered a lot of ground, with a reflection on this year's MWLUG, the Sturm und Drang about Domino, Toscana/IBM design, and a few other topics. Most important, though, is the special announcement at the end of the episode, which you'll just have to listen for. Or skip to the end, I guess; there's a tracking bar right on the page.