Implementing a Basic JNoSQL Driver for Domino
Jan 25, 2022, 1:36 PM
- Updating The XPages JEE Support Project To Jakarta EE 9, A Travelogue
- JSP and MVC Support in the XPages JEE Project
- Migrating a Large XPages App to Jakarta EE 9
- XPages Jakarta EE Support 2.2.0
- DQL, QueryResultsProcessor, and JNoSQL
- Implementing a Basic JNoSQL Driver for Domino
- Video Series On The XPages Jakarta EE Project
- JSF in the XPages Jakarta EE Support Project
- So Why Jakarta?
- Adding Concurrency to the XPages Jakarta EE Support Project
- Adding Transactions to the XPages Jakarta EE Support Project
- XPages Jakarta EE 2.9.0 and Next Steps
A few weeks back, I talked about my use of DQL and QRP in writing a JNoSQL driver for Domino. In that, I left the specifics of the JNoSQL side out and focused on the Domino side, but that former part certainly warrants some expansion as well.
As a quick overview, Jakarta NoSQL is an approaching-finalization spec for working with NoSQL databases of various stripes in a Jakarta EE app. This is as opposed to the venerable JPA, which is a long-standing API for working with RDBMSes in JEE.
JNoSQL is the implementation of the Jakarta NoSQL spec, and is also an Eclipse project. As a historical note, the individual components of the implementation used to have Greek-mythological names, which is why older drivers like my Darwino driver or original Domino driver are sprinkled with references to "Diana" and "Artemis". The "JNoSQL" name also pre-dates its reification into a Jakarta spec - normally, spec names and implementations aren't quite so similarly named.
The specification is broken up into two main categories. The README for the implementation describes this well, but the summary is:
- "Communication" handles interpreting JNoSQL CRUD operations and actually applying them to the database.
- "Mapping" handles what the app developer interacts with: annotating classes to relate them to the back-end database and querying object repositories.
An individual driver may include code for both sides of this, but only the Communication side is obligatory to implement. A driver would contribute to the Mapping side as well if they want to provide database-specific higher-level concepts. For example, the Darwino driver does this to provide explicit annotations for its full-text search, stored-cursor, and JSQL capabilities. I may do similarly in the Domino driver to expose FT search, view operations, or DQL queries directly.
Jakarta NoSQL handles Key-Value, Column, Graph, and Document data stores, but we only care about the last category for now.
Now, on to the actual implementation in question. The handful of classes in the implementation fall into a few categories:
SessionSupplierare mechanisms for the environment to inject the
Session(as signer) objects for the driver to use. This indirection is to allow the driver to be used outside of the XPages JEE project specifically - in this project, the XPages library bundle provides implementations using the current
DominoDocumentCollectionManagerdefines the interface for the Domino-specific manager type. This interface is currently empty beyond what it inherits, but it's where I may provide hooks for DB-specific capabilities.
DefaultDominoDocumentCollectionManager, in turn, is the implementation of this and actually defines CRUD operations. More on this down the page for sure.
DominoDocumentCollectionManagerFactoryis the class that, based on configuration, emits the above for the environment.
DominoDocumentConfigurationis the class where you can configure the above, and serves as the entrypoint for practical use. It uses Jakarta NoSQL's config and settings concepts, either taking those suppliers from the passed-in configuration or trying to find them from the current CDI environment.
EntityConvertercontains support code for CRUD operations, assisting in mapping between JNoSQL's internal entity representation and Domino documents.
QueryConverterdoes similarly for queries, taking JNoSQL's abstract query tree concept and translating it to DQL.
- Speaking of which,
wholesale ripoffproperly-licensed adaptation of Karsten Lehmann's class of the same name, modified slightly to remove Domino-JNA-specific elements and improve
java.timehandling a bit.
The core entrypoint for data operations is
DefaultDominoDocumentCollectionManager, and JNoSQL specifies a few main operations to implement, basically CRUD plus total count:
insertand its overrides handle taking an abstract
DocumentEntityfrom JNoSQL and turning it into a new
lotus.domino.Documentin the target database.
updatedoes similarly, but with the assumption that the incoming entity represents a modification to an existing document.
deletetakes an incoming abstract query and deletes all documents matching it.
selecttakes an incoming abstract query, finds matching documents, converts them to a neutral format, and returns them to JNoSQL. This is what my earlier post was all about.
countretrieves a count for all documents in a "collection". "Collection" here is a MongoDB-ism and the most-practical Domino equivalent is "documents with a specific Form name".
select methods have as part of their jobs the task of translating between Domino's storage and JNoSQL's intermediate representation, and this happens in
Now, this point of the code has some... nomenclature-based issues. There's
lotus.domino.Document, our legacy representation of a Domino document handle. Then, there's
jakarta.nosql.document.Document: this oddly-named interface actually represents a single key-value pair within the conceptual document - roughly, this corresponds to
lotus.domino.Item. Finally, there's
jakarta.nosql.document.DocumentEntity, which is the higher-level representation of a conceptual document on the JNoSQL side, and this contains many
jakarta.nosql.document.Documents. This all works out in practice, but it's important to know about when you look into the implementation code.
The first couple methods in this utility class handle converting query results of different types: QRP result JSON, QRP result views, and generic
DocumentCollections. Strictly speaking, I could remove the first one now that it's unused, but there's a non-zero chance that I'll return to it if it ends up being efficient down the line.
Each of those methods will eventually call to
toDocuments, which converts a
lotus.domino.Document object to an equivalent
List of JNoSQL
Documents (i.e. the individual key-value pairs). Due to the way JNoSQL works, this method has no way to know what the actual desired fields the higher level will want are, so it attempts to convert all items in the document to more-common Java types. There's much more work to do here, some of it based on just needing to add other types (like improving rich text handling) and some of it based on needing a better Notes API (like proper conversion from Notes times to
In the other direction, there's the method that converts from a JNoSQL
DocumentEntity to a Domino document, which is used by the
update methods. This converts some known common incoming types and converts them to Domino item values. Like the earlier methods, this could use some work in translating types, but that's also something that a better Notes API could handle for me.
QueryConverter class has a slightly-simpler job: taking JNoSQL's concept of a query and translating it to a document selection.
jakarta.nosql.document.DocumentQuery type does a bit of double duty: it's used both for arbitrary queries (
Foo='Bar'-type stuff) as well as for selecting documents by UNID. The
select method covers that, producing a
QueryConverterResult object to ferry the important information back to
The core work of
QueryConverter is in
getCondition, where it performs an AST-to-DQL conversion. JNoSQL has a couple mechanisms for querying entities: explicit Java-based queries, implicit queries based on repository definition, or a SQL-like query language. Regardless of what the higher level does to query, though, it comes to the Communication driver as this tree of objects (technically, the driver can handle the last specially, but by default it arrives parsed).
Fortunately, this sort of work is a common sort of idiom. You start with the top node of the tree, handle it based on its type and, as needed, recurse down into the next node. So if, for example, the top node is an
EQUALS, all this converter needs to do is return the DQL representation of "this field equals that value", and so forth for other comparison operators. If it encounters
NOT, then the job changes to making a composite query of that operator plus the results of converting whatever the operator is applied to - which is where the recursion back into the same method comes in.
The main immediate work to do here is enhancing the data conversion: handling more outgoing Domino item types and incoming Java object types. A good deal of this can be done as-is, but doing some other parts reliably will be best done by changing out the specific Notes API in use. I used
lotus.domino because it's present already, but it's a placeholder for sure.
There are also a bunch of efficiency tweaks I can make: more lazy loading in conversion, optimizing data fetching for specific queries, and logging DQL explain results for developers.
Beyond that, I'll have to consider if it's worth adding extensions to the mapping side. As I mentioned, the Darwino driver has some extensions for its JSQL language and similar concepts, and it's possible that it'd be worth adding similar things for Domino, in particular direct FT searching. That said, DQL does a pretty good job being the all-consuming target, and so translating JNoSQL queries to DQL may suffice to extract what performance Domino can provide.
So we'll see. A lot of this will be based on what I need when I actually put this into real use, since right now it's partly hypothetical. In any event, I'm looking forward to finding places where I can use this instead of explicitly coding to Notes API objects for sure.