The main non-work project I've been working on for the last... forever, it seems, has been a forum app for my guild. It started out as a raid-composition utility, then grew to have loot handing, and then added a forum. Most of it is pretty WoW-specific, but the upshot of the whole thing is that it's a moderately-complex XPages application, which has done wonders for my knowledge of the environment.

For programmatic convenient and, in some cases, performance, I've written a ton of wrapper classes, such that I almost never use the standard <xp:dominoDocument/> and <xp:dominoView/> data elements directly, instead using Java beans and collections that implement List. This allows me to do things like #{post.topic.forum.title} if I want to without having to worry about the XPage knowing about how to fetch each parent document in a view.

However, I've run into an annoying problem. Namely, this problem, wherein the Java classloader gets confused about different versions of some classes and stubbornly declares that a class is incompatible with itself, even when I didn't change anything with the class. From what I gather, this is something that you're sort of bound to run into with XPages once you start mucking about with scoped beans enough. It's gotten to the point in my app where I can trigger this behavior 100% of the time by simply re-saving any XPage document. Fortunately, I can clear it up by re-saving a Java class file, but that means that every change I make to an XPage or a Custom Control has to be followed by a re-save of a Java class, which is a step I'd rather not have to take. Worse, it seems to also happen when Domino document data is modified via a mechanism other than the XPages themselves (say, via replication), which is kind of a serious problem.

For the most part, I've learned to work around this. When developing, I re-save a Java class file after each XPage, and I (horrifyingly) wrote a cron script to check the page I know exhibits this error when it happens every 15 minutes and restarts Domino as necessary (which is why I wrote the special login DSAPI filter). These are terrible things to have to do, though, and so I'm really trying to figure out how to actually fix the problem.

Clearly, something I'm doing in my code has caused this, since it doesn't happen in my other apps, but it's not clear what it is and if it's really something I'm doing wrong. I've tried switching all of my managed beans to request scope, implementing Serializable all over the place, changing the way XPages are cached, fiddling with the compiler target version in the Eclipse project, eliminating all Domino classes from object instance variables, installing server fix packs as they come out, and, heck, switching from Windows to Linux, just in case.

At this point, my workarounds pretty much work, but it still drives me nuts, especially the server-bounce script. Since I suspect the culprit is related to all the managed beans, I'm going to try cutting down the data-type collections (Posts, Topics, etc.) and replacing them with <xp:dataContext/>s on the pages that use them, but I don't have terribly high hopes for that. Maybe whatever is causing it on the server side will be fixed in 8.5.3, but I'd still love to fix it before then. I may have to resign myself to it, though, and hope the problem does indeed go away once I'm no longer actively working on it.

For my main project, which contains a forum, one of the problems I ran into was Domino's session handling. Namely, it's designed such that HTTP user sessions last for the duration of the browser session and time out after 30 minutes. That's fine for, say, a corporate app, where you don't want a client logged in indefinitely. However, having to re-log-in to a forum every time you visit it would be a hassle.

The session timeout is easy to fix - you can just up the timeout period in the web site config. The cookie took a little more trickery, but wasn't too bad either. I set up some code in the beforePageLoad event to look for a DomAuthSessId cookie and, if present, create a persistent cookie. The server can't tell the difference between the session and persistent cookies, so this works.

However, it's sort of ugly. While I absolutely don't want to have people re-log-in every time they restart their browser, I also don't really want tons of session-scoped managed beans floating around for months, and this kind of trick defeats any use of the Internet Users list in Domino Administrator (a minor quibble, but a quibble nonetheless). Ideally, I'd be able to automatically log people in on subsequent visits, but not have to actually maintain server sessions for them. From all I've read, this calls for a DSAPI filter. Since that involves C/C++, there's bound to be a lot of setup and tribulation.

First off, I had to grab the Notes/Domino C API toolkit. Fortunately, Google led me directly to the IBM download page and, after re-confirming that I don't want them to send me email, I was given a download link for a quaintly compress'ed file and some installation instructions. I dropped the libraries into /opt/ibm/lotus/notesapi and set up a symlink at /opt/lotus/notesapi just in case. The samples in the toolkit do indeed include a DSAPI filter, and I found another one doing something very much like what I want here:

http://www-10.lotus.com/ldd/46dom.nsf/55c38d716d632d9b8525689b005ba1c0/dd8cc7c9ad12887a85256bca003476bf?OpenDocument

To keep things simple, I started with that code and commented/deleted everything that actually does something - I just wanted the simplest possible filter to see if it works at all. All it does is announce that it's alive and then proceed to not handle any requests:

dsapilogin.c

Now came the issues. First off, the DSAPI example in the toolkit only came with a handful of Makefiles, and none for Linux. The Solaris one was close, so I borrowed and modified that one with some tips from http://www-10.lotus.com/ldd/nd6forum.nsf/55c38d716d632d9b8525689b005ba1c0/fbd4bb196ce4146685256df10038a0b9?OpenDocument . The most vital thing was that, since I'm running in a 64-bit environment but using a 32-bit Domino server, I had to go out of my way to compile a 32-bit library - this is done by adding "-m32" to the CCOPTS and (maybe required) LIBS lines. The Makefile I ended up using is:

Makefile

Along the lines of the "-m32" flag, I had to install some additional development libraries to support 32-bit compiling. In my travails, I ended up installing all sorts of apt packages before it worked, so it's possible that only the last (or another, non-C++ library) is necessary:

g++
lib32stdc++6
libc6-dev-i386
g++-multilib

Unfortunately, even when you get the library building properly, it's tough to get it to load into Domino, and it's not very helpful as to why - all it says is "Failed to load DSAPI filter: " and then the file name... and it gives the same error if you point it to a non-existent file, so there's no way of knowing WHY it failed to load it. It's all trial and error (unless you actually know what you're doing, presumably).

Once all that was in line, though, I could build the library with make, copy it to /opt/ibm/lotus/notes/latest/linux, add "libdsapilogin.so" to the Configuration tab of my Web Site document in the Directory, and restart HTTP. When HTTP is loading, it will print out "HTTP Server: DSAPI DSAPIDLL Loaded successfully" (or whatever message you put in there), so you know it's working.

Now comes the task of actually writing the filter. Considering that I haven't written any C or C++ since college, I expect lots of fun with buffer overflows and mysterious crashes. Exciting!

Because I am insane, I decided to install Domino on my newly-minted 768 slice from Slicehost. That's just above the 512MB RAM minimum that Domino demands and half the recommended 1.5GB, so we'll see how it goes.

The Linux installation I went with for my slice was Ubuntu 10.10 64-bit, which firmly puts it into the realm of unsupported, as far as Domino is concerned. Fortunately, three pages I found made the process relatively painless:

1. http://www.collaborationmatters.com/blog/cmblog.nsf/dx/installing-lotus-domino-8.5.2-on-ubuntu-server?opendocument&comments
2. http://www.danilodellaquila.com/blog/how-to-install-lotus-domino-8.5-on-ubuntu-part-ii
3. http://stackoverflow.com/questions/3747789/how-to-install-sun-jdk-on-ubuntu-10-10-maverick-meerkat

The first wrinkle you run across is that Domino adores Java, but stock Linux installs do not. Fortunately, link #3 there explains how to cleanly install Java on your server. Namely, add this line to /etc/apt/sources.list:

deb http://archive.canonical.com/ubuntu maverick partner

Once you do that, you can run apt-get install sun-java6-jre, wince at the huge number of dependencies it requires, and let it do its thing.

After that, you're almost ready to start installing. If you're running on a 64-bit installation, you'll need the standard 32-bit libraries, since I don't think there's yet a 64-bit Domino server for any-old-Linux. That one's easy, though: apt-get install ia32-libs and you're all set.

The Domino installation proper was relatively straightforward. I created a "notes" user and associated group beforehand, picked and created some directories for the program files and data owned by that user, and went through its command-line wizard via ./install -console . For the final step, I told it to expect a remote installation and fired up the Remote Server Setup tool on Windows, which worked flawlessly.

The next step was the FP2 installer. As is the case with the Windows version, the installer for the fix packs is totally different than the normal installer for some reason. It initially complained about the size of my Terminal window and, rather than trying to convince it that my GUI terminal was plenty good enough for its installer, I took its advice and did export LOTUS_NOROWCOLCHECK=1 . After that, it complained about needing to know where the data directory was, but provided a similar instruction for setting that variable. Once that was in order, the installation went smoothly.

The next tough part was how to get Domino to run at startup. I've been familiar from time to time with various *nixes and their service setups, but it seems to change every couple of years, so I wasn't sure where to go with this. Fortunately, link #2 above did all the work for me. Since I found it long after installation, I didn't do the "customized distribution" bit, but I DID grab the Domino init script. Once I put that in place, modified the variables to point to my directories and user, and followed the other instructions, I was all set. Now I can control Domino via "service domino (start|stop|restart)" like you'd expect.

Due to the nature of Linux, the question of where to keep the program and data directories is a weird one. The defaults are something like "/opt/ibm/lotus" and "/local/notesdata", respectively, which would certainly WORK, but don't fit in with much else. I've fiddled with my placement a bit and settled on the same thing for the former, but "/var/lib/domino/data" for the latter. That allowed me to set up "/var/lib/domino/daos" and "/var/lib/domino/logdir" (for transaction logging) and keep things relatively clean. One thing to note: if you switch around the directories after the fact, make sure to change both the init script and the notes.ini in your program directory to reflect the new locations.

All told, it seems to be working pretty well. When you're using it like a normal Notes server, there's not much distinction other than the lack of OLE. It has all the same bugs (like the aggravating XPages Java classloader "X is not compatible with X" errors I run into constantly) and same features, which is really the idea.

Way back when I wrote my own blog back-end, I went out of my way to make decent-looking URLs that didn't betray the use of PHP for the code. It's not that using PHP was inherently bad, but the ".php" in the URLs was ugly and would have made it tough to move to any other back-ends. It's just one of those good-idea cleanliness things. Back then, I did it with Apache MultiViews and just had names like "archive.php", so I could make URLs like "/archive/2002/12" for that month's posts.

Since I've started using Domino, however, it's gotten a bit tougher. The parts of the URL within a Notes database can actually be pretty great - something like "/People/Foo+Fooson" is about as good as you'd want. However, it's the database path that kills me. You can set up URL substitution rules, but that's a weird combination of setting up a Directory Web Rule plus making sure that every place you make a URL in your code uses that name. For something like a view column, that'd mean either hard-coding your server-specific setting or sucking it up and using @WebDbName. For the most part, I've given up and gone with the latter, for the sake of portability.

XPages bring good news and bad news.

The good news: your application almost never needs to care about the database file path, ".nsf" extensions, or anything of the like. There may be edge cases where you do, but for the most part you can just start your URLs with "/" and the database path is filled in for you (at the cost of starting non-DB paths with "/.ibmxspres/domino").

The bad news: even though the database path reference is now handled by the runtime and not the programmer's code, I don't know of a way to override the behavior. I'm sure it's there SOMEWHERE, buried in the hordes of Java classes and method calls spawned for each page request, but I don't see it. What I'd really want is to tell the XPages runtime that, while "/wow/forums.nsf" is the "right" way to reference the database, I'd rather it just start with something like "/forums". It seems like something that must be handled in the xsp.properties file, but I haven't found anything about it yet. Admittedly, it's tough to search for - most pages with the keywords I want are talking about @WebDbName replacements in SSJS and how to references external Domino resources. If it's not present already, maybe it's something they'll add in a future point release. One can hope.

So I finally wrote a proper Cocoa app to accomplish a specific task! Here it is:

Ok, so its lack of actual UI makes it somewhat unimpressive. The "specific task" part is true, though. After running across a bunch of old AIM chat logs from various sources in my recent hard-drive-trawling, I decided to put them all in a DB for no discernable benefit, because that's the sort of thing I do in my spare time.

The AIM+ and two styles of gaim chats were easy, and I expect the Adium ones to be not too bad, but iChat gave me a little trouble. It's apparently used two formats over time, both of white are Apple-proprietary formats. I decided to focus on the older format, since most of the chat logs were circa-2002/2003.

After doing a little digging, I found that the files are in the format used by NSArchiver and NSUnarchiver, which appear to be the Cocoa method of object serialization. I pointed an NSUnarchiver at one of the chats and... it complained about not having the InstantMessage class. Fair enough.  After finding that Apple's InstantMessage framework is just meant for looking at your buddy list and doing AV stuff and not parsing logs, I grabbed the source of the horrifically-named chat browser Logorrhea. After I plunked its iChat data type classes into my project, all was well.

I found that the files consist of an NSArray (well, NSCFArray) with a mix of children representing chat attributes and the chat lines itself. I looped through that, checked the class of each one, and dealt with each appropriately. "Appropriately" in this case meant "converting to XML", so the agents in my database could read it back in properly. I haven't yet taken a whack at converting the body text from an NSAttributedString to HTML, so for now it's just the plain-text version.

It's rickety and I'm sure I did tons of stuff wrong, but it works! I ended up with a folder full of XML versions of the chats, ready to be imported next to the other logs. There's not much better to get your feet wet with a new environment than a clearly-defined basic task, I say.

For the morbidly curious, the pertinent code is here:

UnarchiverAppDelegate.h

UnarchiverAppDelegate.m

A couple months ago, I set out to cut down as many expenses as I could, and my Comcast bill stuck out like a sore thumb. Thanks to a couple weeks of daily five-minute outages, I had already been switching over to (similarly-priced) FiOS service, but my TV service remained. Not only is cable service with a DVR, reasonable channels, and good HD coverage an expensive proposition regardless, what really gnawed at me was how ludicrously out of proportion the cost was to the actual use I got from it. TV service was about twice as much as my Internet service, but, whereas I use the Internet just about 100% of the time, I only watch a couple hours of TV a week... most of which is available from those channels for free online.

One option was to switch from Comcast TV over to Verizon. It'd be cheaper for a while, but I have no illusions that Verizon wouldn't creep up the price over time as new-customer deals start to wear off, and even if they didn't, the price/usage mismatch would remain.

So I set out to tackle the nigh-insurmountable problem of successfully replacing cable service with a computer with a good net connection. I had a circa-2007 Mac mini laying around that fit the bill nicely, so I cleaned off the old Leopard Server installation and got to setting it up. There are a couple things that make replicating a good TV setup such a sticky wicket:

1. Many shows have their own, usually-crappy delivery mechanism. Some of them, conveniently, use Hulu, but some aren't even readily available at all.
2. Even when shows ARE available, the image quality is pretty much universally far inferior to real HD TV shows.
3. Much like replacing radio with an iPod, it's tougher to discover new shows without seeking them out. Fortunately, this isn't a huge problem.
4. Media center apps, while impressive, tend to have at least one crucial flaw that undermines their utility.

The first two problems have necessitated a step into questionable morality and technical complexity. Namely, I set up an RSS reader to pull torrent files for new TV shows in pretty-much-HD quality, then a torrent client to download them, categorize them by name, seed to 200%, and then stop the torrent. I still don't have EVERY show we used to watch, but I have the big ones, and they're in pretty solid quality.

The last one is an ongoing project. I started out using Plex, which is pretty great and edges out the competition with some Mac-specific support. However, its YouTube plugin is pretty lacking and I'd rather not resort (yet) to learning Python and writing my own. I've switched over to its progenitor XBMC for now, which has a pretty solid YouTube plugin, but took a step back on other video sources. I'm considering setting up a Windows partition on the thing to see how the options are on the other side (like MediaPortal), but I'd rather avoid having to do that.

Overall, my TV experiment has worked out pretty well, but it's primarily due to my specific situation. If normal TV was a bigger deal in the apartment (say, if we had kids), this would be a woefully inadequate replacement. Similarly, if I didn't enjoy doing crap like setting up RSS feeds for torrent processing or manually managing TV-show libraries and video-output settings, it'd be too much hassle to be worth it. And I certainly don't miss the Comcast bills.

Thanks to The Brads, I ran across Dribbble the other day, and it's the new coolest thing ever. The two main things that struck me were "man, modern web designs use a LOT of soft gradients" and "hmm, I really want to make some more web sites now." The latter is pretty much the point.

I've never been a web designer. With enough effort, I can sort of emulate one, but the results are expectedly mediocre. That's mainly because that's a natural extension of being a web developer - the stuff I program has to look like something, and CSS is essentially a programmer's design tool anyway. However, visual design isn't my gig, so I don't feel the least bit squeamish about cobbling together something reasonably clean and aesthetically pleasing from various sources of "inspiration" and calling it a day.

Fortunately, looking at nice, clean designs (like this WordPress admin site, actually, which I kind of like) is good inspiration nonetheless, since it makes me want to write something cool enough to rise to the occasion. A minimalist design with exaggerated tabs, labels, and such implies that the stuff you're looking at is organized logically enough to match, and that's the kind of thing I can do.

Now that GCD is a reasonable 1.0, Dribbble has me itching to move on to what's next. Far and away the best candidates for fancy new UIs are my various projects for work, but that's only ever so exciting. Maybe I'll take another pass at my Minecraft server to see if I can take advantage of its database-attached-ness. Probably better would be really diving into Cocoa programming - that'd be fun AND potentially directly useful for future job prospects. Heck, it'd even be good for my CURRENT job, since having a proper iPhone UI is on our list of big to-dos. Hmm, yep, that's what I'll do.

I'm not sure if "mixed-type" is the right term for what I mean, but it should do.

A couple months ago, I ran into a problem where I wanted to display the posts in a forum topic on an XPage, which isn't a terribly complicated thing to do. However, I ran into a bit of trouble in how, specifically, to go from the <xp:dominoView/>  data source to proper rendering of the rich text for each post, in part because some posts were MIME data and some were the native Notes rich text format. The rich text naturally wasn't in a column value, so I couldn't just refer to the column by name using the var name int he <xp:repeat/>.

I wrangled with this for a little while, writing JavaScript code to check if it's in MIME or native format, but I kept running into problems with how to convert the rich text data on the fly. I didn't want to just lose all the rich text formatting by using NotesItem.getUnformatetdText() or an equivalent, but I couldn't just return a NotesItem object to the <xp:text/> control. I knew that XPages know how to convert it, since they do it just fine if you're working with a NotesXspDocument, so I wanted some way to access that functionality.

<xp:panel/> came to my rescue. Within the <xp:repeat/>, I created an <xp:panel/> with its own data section containing an <xp:dominoDocument/> with its documentId property set to the UNID of the view entry I was currently working with. Then, I could just create an <xp:text value="#{doc.Body}"/> and let the renderer take care of all the details.

For all I know, that was an extraordinarily inefficient way to do it, server-wise, but it may be the cleanest way to do it from the programmer's perspective.

Getting Crap Done checklist:

• iCalendar feed? Check.
• Email reminders? Check.
• Basic mobile support? Check.

I have a couple more little things I'd like to add and some UI changes I'll no doubt make, but I'd call the quick development of GCD a success. Which is good, since the idea is to get me to do stuff, and quickly.

So what's next?

The big thing is to get Raidomatic, my new guild forums and raid composition tool, released and into use. It's mostly there, but there are little problems to take care of and a couple small features I should add before release. The vital thing will be to get it out there - I can add any other big features (like sorting loot rollers by how proper the item is for their spec) in later revisions.

After that, I want to make a Mac client for GCD. It's not like the site is screaming out for native clients, but I want to learn Cocoa programming and nothing helps learning a new environment more than having a very explicit and attainable goal to reach.

In addition, I've got a family-company web site to work on. Between that and the other couple things I have to do, I should be able to keep myself pretty busy for a while.

If you spend enough time working with XPages, you're eventually going to want to access some SQL data, either because of an integration project or because you just want to. It's pretty well-trodden ground, but, in one of my recent projects, I came up with an approach I rather like.

Now, before I get into it, fair warning: this is not scalable, efficient, or good programming practice. This is for when you just want to do a quick query and get back data in a usable fashion. It doesn't handle paging properly and it can easily stomp all over separation of concerns. But if your needs are simple, it may do the job perfectly.

Basically, the problem I had was this: I want to do a couple quick SQL "select" queries on an XPage, but I got tired of writing out similar "stmt = conn.prepareStatement(...); stmt.setString(...)" stuff over and over. Ideally, the solution would be to abstract it all away, but it was a small page deserving of a small fix, so I wrote myself a function:

SQLTools = {}
SQLTools.fetchQuery = function(query, args) {
args = args == null ? [] : args


var stmt = SQL.getConnection().prepareStatement(query)
for(var i = 0; i < args.length; i++) {
    stmt.setObject(i+1, args[i])
}
var rs = stmt.executeQuery()

var cols = []
var meta = rs.getMetaData()
for(var i = 0; i < meta.getColumnCount(); i++) {
    cols.push({
        name: meta.getColumnName(i+1),
        type: meta.getColumnType(i+1)
    })
}

var result = []
while(rs.next()) {
    var row = {}
    for(var i = 0; i < cols.length; i++) {
        row[cols[i].name] = rs.getObject(cols[i].name)

    }
    result.push(row)
}
return result

}

The actual job of connecting to the SQL DB with the right driver is handled by a Java bean called SQL - the aforementioned link covers that part pretty well. What makes this code different is how easy it makes a quick query with some interleaved parameters and how useful the resultant value is for standard XPage controls. For example, you could do a search like this...

SQLTools.fetchQuery("select firstname,lastname,degree from people where age > ? and lastname like ?", [30, "%son"])

...and you'd get back an array containing column-keyed hashes. Your XPage doesn't have to care about ResultSets or ".getString()"s or anything. You could end up with relatively clean code like:

<xp:repeat var="person">
<xp:this.value><![CDATA[#{javascript:
SQLTools.fetchQuery("select firstname,lastname,degree from people where age > ? and lastname like ?", [30, "%son"])
}]]></xp:this.value>

<xp:div><xp:text value="#{person.firstname}"/> <xp:text value="#{person.lastname}"/>, <xp:text value="#{person.degree}"/></xp:div>
</xp:repeat>

That's probably not a final product for displaying the information, but it gets the point across - it's a pretty clean traversal from "I want to query the database" to having XPages-friendly Server JavaScript objects.