Data separation with an asterisk
Thu Jan 09 09:50:59 EST 2014
A little while ago, I converted over to the practice of separating your XPage app from the NSF that contains the data, even when it's going to be a one-to-one mapping. This confers a number of advantages in the areas of easier testing, easier development, programming discipline, and security. One of my favorite aspects is the ability to do this on your data DB:
This alone adds tons of security: no more worrying about ?ReadViewEntries, hiding your views from the web, errant forms allowing too much access, and so forth. Once you have that checked and you funnel your access through a separate app, the surface of vulnerability is much, much smaller.
But there's a hitch.
I discovered this the other day, and it's only a problem in mixed web/Notes environments: when you have that option enabled, you can't see native rich text content from a document in your XPage. I say "native" to refer to the traditional Composite Data format of rich text; MIME content is fine. The reason for this appears to go down to the C-API level, where setting that flag seems to entirely bar the HTML conversion functions from working with that database.
My guess is that that was deemed the most expedient way to implement the feature at the time, but it's problematic now. I can think of a couple ways to deal with it:
- Don't use native rich text. If you're writing whole-cloth XPages apps, you don't have to worry about this at all, since MIME/HTML content is still fine. If you have a mixed app and you're fine with some ugly-looking text in the Notes client, you can turn on the field option to store the data as MIME even in the client.
- Store the app on separate servers. To maintain the security benefits, you could store the data database on a different server, one not publicly accessible via HTTP.
- Disable that option and use web rules instead. You should probably be able to achieve a lot of the security benefits by substituting all HTTP requests for the data database with blank pages, though that requires coordination of the server config with every DB like this, which is a drag. Also don't forget __$replicaid.nsf URLs.
Fortunately, the fact that it only affects conversion from native rich text to MIME means the impact is limited, but it's still a caveat to data separation.
Ben Langhinrichs - Thu Jan 09 11:10:25 EST 2014
I believe that this is a use case a Midas customer of ours has, where they extract MIME (and JSON) from the rich text ccontent from the other database using the Midas LSX, thus both getting better rendering and security. I don't know many details, but if you'd like to play with the idea just in case it is ever useful, I'd be happy to help you try it out.
Tim Tripcony - Fri Jan 10 00:46:45 EST 2014
Here's the root cause: for CD-based rich text rendering, IBM punted to the traditional Domino rendering engine which, admittedly, was impressive back in 1996 even though by now it's generally regarded as (for lack of a better word) "crappy". To leverage the traditional engine, they issue a call to the ?OpenField URL that targets just that specific rich text field. Convenient, no doubt, as long as that field is accessible via that URL. Since the setting you mentioned disables URL access to the entire application, it also blocks any field-specific URL.
Jesse Gallagher - Fri Jan 10 06:47:01 EST 2014
That URL would indeed fail, but I think it fails even lower than that: the C functions HTMLConvertItem and HTMLConvertNote functions return an error code for "You are not authorized to access that database" when pointed at such a DB. I hadn't noticed it until after I ran my own test of that, but the error log bears it out: there are errors on the console for each field and error-log-0.xml has a stack trace about an NAPI call for HTMLConvertItem. I guess they figured that was the best way to really enforce that "no URL open" for the original "Domino" implementation in the server, and it made sense at the time.
I sort of like the notion of a clean converter for RT content in the OpenNTF API. It would be a big barrel of hassle, but not forever impossible. DXL with a stylesheet could do the job with some extra processing overhead, but the really good long-term way to do it would be to read and convert the CD content directly. Probably one day down the line...
Txemanu - Sun Jan 12 15:24:22 EST 2014
How much would be affected the performance having the data separated in another DB or even in another server? Enough to be worried about?
Jesse Gallagher - Mon Jan 13 11:05:57 EST 2014
I haven't don't real performance testing, but I suspect that the impact of having data on another DB on the server would be minimal, and certainly worth the trade for the development and security advantages.
Having the data on another server would usually be a hit for performance, but could theoretically be a huge performance boost in high-scale apps. With one server handling the app and another on data, it'd split the load in heavy use, and you could distribute the data access from one front-end server to a cluster of back-end servers (say, chosen round-robin per user session). That's a fairly common kind of setup with non-Domino apps.
Ben Langhinrichs - Mon Jan 13 14:28:15 EST 2014
Have you tested to see whether the DXL export from the rich text works? I'd be curious to see which API functions they block.