What Makes XPages "Not Modern"?

Aug 23, 2019 1:53 PM

Tags: xpages
  1. What Makes XPages "Not Modern"?
  2. Medium-Term Ways to Improve XPages

A big part of figuring out how to move past XPages is discussing what makes it no longer modern in the first place. Some of these reasons will be gimmes - out-of-date or outright-missing features - while some will be less about what XPages is strictly capable of and instead more about how XPages development and deployment works. I'm going to leave out some of the ones that are just from recent lack of development attention, such as Bootstrap 4 support remaining in alpha state.

Admittedly, I suspect that this post will come across as an enumeration of complaints, but I don't really mean it that way. XPages served us well for a while, but it's one thing to say "ah, XPages, it's a bit old and creaky" and another to think about just how pervasive that is.

Technical Limitations

The most straightforward reasons are the Java technologies that are included in the XPages stack, but are significantly out of date. This includes big-ticket core standards like Servlet 2.4 (faux 2.5), the forked JSF 1.x, JavaMail 1.3, pre-Unified EL, and JAX-RS 1.1; lesser-known standards like Activation 1.1; and even third-party libraries that show up in the fixed XPages classpath like Apache BeanUtils 1.6, Apache Collections 2.1, Apache Logging 1.0, and Guava 19 (a recent addition).

Some of these can be worked around relatively simply (including a newer JAR in your app/plugin or adding an OSGi bundle), but others are either more pernicious (like Logging and Guava, causing runtime ClassLoader trouble) or are outright showstoppers. I've done a lot of work to bring newer and missing Jakarta EE standards to XPages, but the ancient Servlet version is a hard limit on a lot of things. In addition to its own limitations like the lack of Web Sockets, third-party libraries and implementations often expect Servlet 3 as a bare minimum, and nowadays make use of Servlet 4.

Self-Imposed Hurdles

Beyond the technical limitations that largely come from outdated implementations, XPages is afflicted by what I'll deem "self-imposed hurdles": difficulties that spring from choices that Lotus/IBM/HCL actively made or chose to not make that resulted in things being a bit more difficult than in other environments. These decisions were often even good decisions in the aggregate, but they have tradeoffs.

The first big one is the use of the NSF container. Overall, this is a positive: not only did it create a more unified development experience with classic Notes elements, but the NSF itself brings attributes like external-to-the-webapp ACLs and seamless replication, which are extra addons at best with other frameworks. However, because the NSF is essentially a proprietary binary blob, it means that it can't readily participate with other tooling. You can't crack one open with a ZIP file tool, inside-JVM reflection/classpath tools can stumble over it, and it can't inherently participate in source control or automated builds. As above, some of that can be worked around with immense effort, but even then not nearly as smoothly as elsewhere.

Along the lines of the NSF, due to Designer's lineage and (presumably) the state of technology at the time that XPages was integrated, the normal XPages development process has no participation in the larger unified Java development world that Maven largely ushered in. Even besides not fitting into the build system, this introduces just a bit more friction in normal development. Yes, an XPages app can use many third-party Java libraries by just adding them to the Code/Jars section, but they don't benefit at all from version management or automated fetching of source and Javadoc. It may sound like a small thing, but it makes a world of difference once you're used to it.

That plays in to a general Domino/Designer issue that stretches back to the introduction of Java to the platform: the libraries include no parameter info or Javadoc. Designer has its help sidebar for Notes.jar classes and the Javadoc exists for a subset of the XPages stack as of 9.0.1, but they're not integrated in the IDE, and large swaths of the platform don't have any Javadoc at all. This leads to stumbling blocks where you'll seek out documentation for a class that nominally exists in XPages, but you have to be wary of the "Since" note for every method (and even then you may be thrown for a loop, like UIViewRoot#getViewMap() that's "since" 2.0 but does exist in XPages).

Then there's a big sticking point that's created a weird cargo cult within the community: the default Java policy. While the file itself does indeed come from stock Java (including the adorable "properies" typo), the way it interacts with the SecurityManager and ClassLoader setup in XPages has an outsized negative impact. XPages apps butt heads with this policy constantly, in particular when using reflection - which is something that third-party libraries do often as a matter of course. It's had the effect of making many people shy away from using third-party libraries or code that uses normal Java features like this out of a partially-justified desire to not tweak the settings.

The last one in this category is the use of OSGi generally and Equinox specifically. I can't knock the original choice too hard: OSGi is a logical pick for making a large, pluggable system and IBM institutionally had tremendous experience with it by way of Eclipse and WebSphere. However, while it's a solid choice for system architecture, it's often very awkward for app architecture. And, hypothetically, XPages is designed to avoid this. Like the various Java EE servers that are implemented with OSGi but run plain-old WAR-based apps, a basic XPages app isn't meant to care about the OSGi layer at all. However, a combination of the above security-policy troubles, Designer's limitations as an IDE, explicit blocks on what classes an NSF can access, and the severely-limited EE services provided by the XPages runtime means that OSGi plug-in development came immediately to the fore. And developing plug-ins for XPages is hard. It involves all sorts of ceremony and incantations just to get started, and maintaining even a well-structured plug-in project is much more difficult than a normal Maven or Gradle project. Some of this is OSGi, some of it is Equinox specifically, and some of it is unique to XPages, but it all combines to make it all a daunting process even for people who aren't put off by Java-the-language. So, much like with the security policy, you have a large subset of the XPages community that for either developer or admin reasons don't develop XPages libraries and often don't bring in any outside of that HCL ships in the box.

Development Model

And, finally, I'd like to cover the development model created by XPages even when everything is firing on all cylinders.

The first one is the lack of MVC (or similar) structure. XPages was introduced as bringing MVC to Domino development, in large part thanks to the underlying JSF nature, but this was something of a dirty trick. While it's theoretically possible to make a cleanly-structured XPages app, the framework provides no useful model layer, and Designer is almost actively hostile to this kind of separation. And, because the framework doesn't give any guidance and few of us in the community came in as seasoned web developers, the discussion about XPages and MVC is largely about each person's individual version of "MVC-style" development, and Lord knows I'm as guilty as anyone. This, like the lack of dependency management, is something that you don't fully appreciate until you work with a system with real structure and clear guidance on how to do it.

The second point in this section is one that I actually debated putting on this list, and that's XPages's use of server-side state. Depending on who you talk to, this alone is something that makes XPages dead in the water, an ancient relic of a forgotten era. Personally, I'm not so universally down on the notion: I feel that the worries about scalability are usually based on hypotheticals, and being able to make complicated dynamic apps maintained by the server cuts down on the work of writing and securing REST APIs for absolutely everything. Still, it's pretty clear that the server-state model has fallen out of favor, and that's worth at least considering when determining "not modern".

So What About Workarounds?

A running theme through all of this has been "X is behind the times or weird, but can be overcome with a bunch of work". And this shows up a ton in practice: a lot of XPages developers have gone to Herculean lengths to bring in other technologies or carve out a way to do structured programming, and it sometimes works. Others eschew XPages UI components entirely and build JavaScript apps backed by REST services written with the ExtLib components, JAX-RS, or other mechanisms. Especially since the Java 8 switch, you can do a lot to drag XPages forward.

But that doesn't make it "modern" really. This is all work fighting against the platform, work that doesn't need to be done with other tooling. In other environments, if you want to use the latest version of a spec, you make sure your container is up-to-date and then you just use it. If you want to bring in a dependency, you declare it in your "pom.xml" and it shows up, source and all. And then there are the things that other environments provide for you, like OpenAPI documentation, and concepts that don't even exist in XPages, like fault tolerance. Maybe some of those things could be brought to XPages too, but, again, it'd be a struggle every step of the way. There'd always be some weird thing to do with the Servlet version, or Tycho, or ClassLoaders, or loggers, or some of the many other little asterisks that accompany the platform.

The Work To Move Forward

My plan for future entries in this series is to discuss some of the specific steps that I've been taking with my largest active client project to prepare it for the future. That future is as-yet-undefined - it may stay within the Domino web container, it may not - but the good part is that a lot of the work is in common regardless of where we take it, and I think that it will prove useful to others facing similar situations.

XPages to Java EE, Part 11: Mixing MVC and an API

Feb 16, 2019 12:49 PM

Tags: javaee
  1. XPages to Java EE, Part 1: Overview
  2. XPages to Java EE, Part 2: Terminology
  3. XPages to Java EE, Part 3: Hello, World
  4. XPages to Java EE, Part 4: Application Servers
  5. XPages to Java EE, Part 5: Web Pages
  6. XPages to Java EE, Part 6: Dependencies
  7. XPages to Java EE, Part 7: MVC
  8. XPages to Java EE, Part 8: IDE Server Integration
  9. XPages to Java EE, Part 9: IDE Features Grab Bag
  10. XPages to Java EE, Part 10: Data Storage
  11. XPages to Java EE, Part 11: Mixing MVC and an API
  12. XPages to Java EE, Part 12: Container Authentication
  13. XPages to Java EE, Part 13: Why Do This, Anyway?

When we set up our MVC controller classes, we put the @Controller annotation at the class level, which tells the environment that the entire class is dedicated to running the UI. However, we don't necessarily always want to do that - JAX-RS is the way to build REST APIs, after all, and so we should also add JSON versions of our Person methods.

Person Model Modification

Before we get to the meat of the code, go back to the Person class and modify it to remove the parameter to the @Id annotation and switch it to a String type:

package model;

import javax.validation.constraints.Email;
import javax.validation.constraints.NotBlank;
import javax.ws.rs.FormParam;

import org.jnosql.artemis.Column;
import org.jnosql.artemis.Entity;
import org.jnosql.artemis.Id;

@Entity
public class Person {
	@Id
	private String id;
	@Column @FormParam("name") @NotBlank
	private String name;
	@Column @FormParam("emailAddress") @NotBlank @Email
	private String emailAddress;
	
	public String getId() { return id; }
	public void setId(String id) { this.id = id; }
	public String getName() { return name; }
	public void setName(String name) { this.name = name; }
	public String getEmailAddress() { return emailAddress; }
	public void setEmailAddress(String emailAddress) { this.emailAddress = emailAddress; }
}

The reason for the change is that my original example was based on code from an older version of JNoSQL, and long IDs end up causing trouble when updating existing documents.

Also go to PersonRepository and modify it to use a String for the key:

package model;

import java.util.List;

import org.jnosql.artemis.Repository;

public interface PersonRepository extends Repository<Person, String> {
	List<Person> findAll();
}

 

Tweaking the Controller

The first step to adding in API methods doing this is to move the @Controller annotation down to just the methods that emit JSP responses (and adjust for the changed ID field while we're here):

package com.example;

import java.util.Random;

import javax.enterprise.context.RequestScoped;
import javax.inject.Inject;
import javax.mvc.Controller;
import javax.mvc.Models;
import javax.validation.Valid;
import javax.ws.rs.BeanParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;

import org.bson.types.ObjectId;
import org.jnosql.artemis.Database;
import org.jnosql.artemis.DatabaseType;

import model.Person;
import model.PersonRepository;

@Path("/people")
@RequestScoped
public class PersonController {
	@Inject
	Models models;
	
	@Inject
	@Database(DatabaseType.DOCUMENT)
	PersonRepository personRepository;
	
	@GET
	@Controller
	public String home() {
		models.put("people", personRepository.findAll()); //$NON-NLS-1$
		return "person-new.jsp"; //$NON-NLS-1$
	}
	
	@POST
	@Controller
	public String createPerson(@BeanParam @Valid Person person) {
		if(person.getId() == null || person.getId().isEmpty()) {
			person.setId(new ObjectId().toHexString());
		}
		
		models.put("person", personRepository.save(person)); //$NON-NLS-1$
		models.put("people", personRepository.findAll()); //$NON-NLS-1$
		return "person-created.jsp"; //$NON-NLS-1$
	}
}

Doing this shouldn't change the behavior of the app, and that's what we want.

Add Some API methods

Now, to be a proper REST API, we'll want a suite of Create-Read-Update-Delete methods using standard HTTP verbs. Add these methods to the class:

@GET
@Produces(MediaType.APPLICATION_JSON)
public List<Person> list() {
  return personRepository.findAll();
}

@GET
@Path("{id}")
@Produces(MediaType.APPLICATION_JSON)
public Person getPerson(@PathParam("id") String id) {
  return personRepository.findById(id).orElseThrow(() -> new javax.ws.rs.NotFoundException("Could not find person for ID " + id)); //$NON-NLS-1$
}

@POST
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Person createPersonApi(@Valid Person person) {
  if(person.getId() == null) {
    person.setId(new ObjectId().toHexString());
  }
  return personRepository.save(person);
}

@DELETE
@Path("{id}")
@Produces(MediaType.APPLICATION_JSON)
public JsonObject deletePersonApi(@PathParam("id") String id) {
  personRepository.findById(id).orElseThrow(() -> new javax.ws.rs.NotFoundException("Could not find person for ID " + id)); //$NON-NLS-1$
  personRepository.deleteById(id);
  return Json.createObjectBuilder()
      .add("success", true) //$NON-NLS-1$
      .build();
}

@PUT
@Path("{id}")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Person updatePersonApi(@PathParam("id") String id, @Valid Person person) {
  person.setId(id);
  return personRepository.save(person);
}

Here, we're taking advantage of JAX-RS's MIME-type-based routing: because our @Controller methods deal with HTML but these new methods declare that they're working with JSON, JAX-RS will route incoming browser visits to the controller and incoming API requests to the others.

Testing It Out

We can see this in action by trying out the API from the command line (or with a REST client app like Postman):

$ curl -i http://localhost:9091/javaeetutorial/resources/people
HTTP/1.1 200 OK
X-Powered-By: Servlet/4.0
Content-Type: application/json
Date: Sat, 16 Feb 2019 17:05:16 GMT
Content-Language: en-US
Content-Length: 246

[{"emailAddress":"api@test.com","id":"5c683fd40048ce11b5f6aee8","name":"API Test"},{"emailAddress":"foo@foo.com","id":"5c6841520048cea7c9f6c2d5","name":"Foo Fooson"},{"emailAddress":"foo@foo.com","id":"5c6841690048cea7c9f6c2d7","name":"API mod"}]

$ curl -i -X POST -H"Content-Type: application/json" http://localhost:9091/javaeetutorial/resources/people -d "{\"emailAddress\":\"foo@foo.com\",\"name\":\"Created with cURL\"}"
HTTP/1.1 200 OK
X-Powered-By: Servlet/4.0
Content-Type: application/json
Date: Sat, 16 Feb 2019 17:11:55 GMT
Content-Language: en-US
Content-Length: 89

{"emailAddress":"foo@foo.com","id":"5c68445b0048cea7c9402b85","name":"Created with cURL"}

$ curl -i -X PUT -H"Content-Type: application/json" http://localhost:9091/javaeetutorial/resources/people/5c68445b0048cea7c9402b85 -d "{\"emailAddress\":\"foo_mod@foo.com\",\"name\":\"Modified with cURL\"}"
HTTP/1.1 200 OK
X-Powered-By: Servlet/4.0
Content-Type: application/json
Date: Sat, 16 Feb 2019 17:12:30 GMT
Content-Language: en-US
Content-Length: 94

{"emailAddress":"foo_mod@foo.com","id":"5c68445b0048cea7c9402b85","name":"Modified with cURL"}

$ curl -i -X DELETE http://localhost:9091/javaeetutorial/resources/people/5c68445b0048cea7c9402b85
HTTP/1.1 200 OK
X-Powered-By: Servlet/4.0
Content-Type: application/json
Date: Sat, 16 Feb 2019 17:12:54 GMT
Content-Language: en-US
Content-Length: 16

{"success":true}

OpenAPI Documentation

OpenAPI is the boring-ified name of the standardized spec of Swagger, a mechanism for documenting REST APIs - kind of like what WSDL is for SOAP web services. This spec has become an important part of MicroProfile, the set of Java server technologies geared towards writing microservices that shares a lot of core technologies with Java EE. One of the niceties that MicroProfile includes is an automatic OpenAPI generator for JAX-RS services. There are a few things to add to our environment to enable this, but not too much.

To begin with, we'll have to enable the OpenAPI generator feature in Open Liberty (TomEE may have something like this; I don't know). To do that, open up the "server.xml" file (labeled "Server Configuration" in Eclipse's Servers view) and add "mpOpenAPI-1.0" to the feature list:

<featureManager>
    <feature>javaee-8.0</feature>
    <feature>localConnector-1.0</feature>
    <feature>mpOpenAPI-1.0</feature>
</featureManager>

Doing that alone will enable the API documentation, available at http://localhost:9091/openapi. However, if you look closely at the output, you'll see it's not exactly what we'd want: the GET operation for /resources/people points to our MVC home method, which it considers an unstructured string. It also lists the "helloworld" and "markdown" resources, and you can feel free to delete those classes outright - we won't be returning to them.

To fix this, first go to the project's "pom.xml" and add a dependency on the MicroProfile OpenAPI spec:

<dependency>
    <groupId>org.eclipse.microprofile.openapi</groupId>
    <artifactId>microprofile-openapi-api</artifactId>
    <version>1.1</version>
    <scope>provided</scope>
</dependency>

This is another one we can mark as "provided" since the implementation is included with the server.

Now, go back to the PersonController class, add an import line for org.eclipse.microprofile.openapi.annotations.Operation, and annotate the two MVC methods to mark them hidden from OpenAPI:

@GET
@Controller
@Operation(hidden=true)
public String home() {
  models.put("people", personRepository.findAll()); //$NON-NLS-1$
  return "person-new.jsp"; //$NON-NLS-1$
}

@POST
@Controller
@Operation(hidden=true)
public String createPerson(@BeanParam @Valid Person person) {
  if(person.getId() == null) {
    person.setId(new ObjectId().toHexString());
  }
  personRepository.save(person);

  models.put("person", person); //$NON-NLS-1$
  models.put("people", personRepository.findAll()); //$NON-NLS-1$
  return "person-created.jsp"; //$NON-NLS-1$
}

Now, if you refresh the /openapi output, you can see that it switched to the list method, and it knows that it outputs JSON, and includes a reference to the Person object structure at the bottom of the file.

There's a good deal more you can do with these annotations to customize the output, but it's nice to know that you can get an immediately-useful file that could be used to generate structured client libraries "for free".

Next Steps

Next, I think we'll dive into the world of Java EE authentication, which will be a very-different experience from what we're used to with Domino, for better and for worse.

XPages to Java EE, Part 10: Data Storage

Feb 15, 2019 12:48 PM

Tags: javaee
  1. XPages to Java EE, Part 1: Overview
  2. XPages to Java EE, Part 2: Terminology
  3. XPages to Java EE, Part 3: Hello, World
  4. XPages to Java EE, Part 4: Application Servers
  5. XPages to Java EE, Part 5: Web Pages
  6. XPages to Java EE, Part 6: Dependencies
  7. XPages to Java EE, Part 7: MVC
  8. XPages to Java EE, Part 8: IDE Server Integration
  9. XPages to Java EE, Part 9: IDE Features Grab Bag
  10. XPages to Java EE, Part 10: Data Storage
  11. XPages to Java EE, Part 11: Mixing MVC and an API
  12. XPages to Java EE, Part 12: Container Authentication
  13. XPages to Java EE, Part 13: Why Do This, Anyway?

How you store your data in an application is a potentially-huge topic, which is one of the reasons I've pushed it off for a while.

Designer's Curse

This is particularly the case because of the habits we learned over the years as Domino developers. One of the most grievous wounds Domino inflicted on us was an encouragement to always write directly to the data-storage implementation objects - forms and views for Notes client design or the lsxbe/lotus.domino classes for LotusScript and Java. They work, sure - fetching a document, setting fields, and storing it will get the job done - but it's an extremely-bad habit to work without a model framework and some level of indirection. Various people, including me, have made valiant efforts to add a model/DAO layer into XPages development in particular, but they've met with little uptake outside the individual developers who wrote them.

Fortunately, Java EE does not suffer from this specific brain poison, and it has a long history of abstracted data access, primarily via the Java Persistence API, traditionally backed by a JDBC driver for a SQL database. The point of an API like that is to let you write your model objects with just some annotations to explain to JPA bits about how it should be stored, and JPA will take care of the dirty work of actually mapping data types, writing queries, fetching data, and so forth.

JNoSQL

We won't be using JPA for this example, though. Instead, we'll be adding our second incubating spec: JNoSQL. JNoSQL is intended to be essentially "JPA for NoSQL", a largely-rethought API that won't crash into the hackiness of Hibernate's valiant attempt of re-using JPA directly. JNoSQL is currently slated for standardization as part of Jakarta EE and is under active development, but reached a point a while ago where it's good for use.

However, while there's technically a Domino JNoSQL driver that I put together last year, it's more of a POC than a real thing, and we'll skip it for this. For my uses, I've been using Darwino, which does have a splendid JNoSQL driver, but this series isn't the place to go through getting set up with that. For simplicity's sake, we'll be using MongoDB, which is quick to set up and is probably the furthest-developed driver in core JNoSQL.

MongoDB

So, to start out with, install MongoDB somewhere locally. This differs system-by-system - on Linux and macOS, I think it's available with package managers, or for any OS you can download an installer from their site.

Once it's installed, create a database named "exampledb" and a collection within it named "Person", as seen here with Compass, the standard admin app.

MongoDB collection configuration

Add the Driver

In your project's "pom.xml", add the JNoSQL document DB packages and MongoDB driver to your dependencies block:

		<!-- JNoSQL -->
		<dependency>
			<groupId>org.jnosql.artemis</groupId>
			<artifactId>artemis-core</artifactId>
			<version>0.0.7</version>
		</dependency>
		<dependency>
			<groupId>org.jnosql.artemis</groupId>
			<artifactId>artemis-document</artifactId>
			<version>0.0.7</version>
		</dependency>
		<dependency>
			<groupId>org.jnosql.artemis</groupId>
			<artifactId>artemis-validation</artifactId>
			<version>0.0.7</version>
		</dependency>
		<dependency>
			<groupId>org.jnosql.diana</groupId>
			<artifactId>mongodb-driver</artifactId>
			<version>0.0.7</version>
		</dependency>

For reference, "artemis" in JNoSQL terms refers to the mapping API - the annotations we're going to use - while "diana" refers to the driver portion.

Create the Configuration Class

Create a new class in the model package called DocumentCollectionManagerProducer:

package model;

import java.util.Collections;
import java.util.Map;

import javax.annotation.PostConstruct;
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;

import org.jnosql.diana.api.Settings;
import org.jnosql.diana.api.document.DocumentCollectionManager;
import org.jnosql.diana.api.document.DocumentCollectionManagerFactory;
import org.jnosql.diana.api.document.DocumentConfiguration;
import org.jnosql.diana.mongodb.document.MongoDBDocumentCollectionManager;
import org.jnosql.diana.mongodb.document.MongoDBDocumentCollectionManagerFactory;
import org.jnosql.diana.mongodb.document.MongoDBDocumentConfiguration;

@ApplicationScoped
public class DocumentCollectionManagerProducer {
	private DocumentConfiguration<MongoDBDocumentCollectionManagerFactory> configuration;
	private DocumentCollectionManagerFactory<MongoDBDocumentCollectionManager> managerFactory;
	
	@PostConstruct
	public void init() {
		configuration = new MongoDBDocumentConfiguration();
    // Modify this if MongoDB is not on localhost
		Map<String, Object> settings = Collections.singletonMap("mongodb-server-host-1", "localhost:27017"); //$NON-NLS-1$ //$NON-NLS-2$
		managerFactory = configuration.get(Settings.of(settings));
	}
	
	@Produces
	public DocumentCollectionManager getManager() {
		return managerFactory.get("exampledb"); //$NON-NLS-1$
	}
}

There's a lot there, but fortunately some of it builds on the CDI producer/scope functionality we encountered earlier. What we're doing here is setting up an application-wide bean that produces a configuration object for JNoSQL to use - specifically, using MongoDB. In a real situation, you'd want to externalize the settings in some way, but putting it into the code will do for now. The getManager() method will be used behind the scenes when JNoSQL asks the environment for a document-database manager.

Create the Model

Create another new class in the model package, this time named Person:

package model;

import javax.validation.constraints.Email;
import javax.validation.constraints.NotBlank;
import javax.ws.rs.FormParam;

import org.jnosql.artemis.Column;
import org.jnosql.artemis.Entity;
import org.jnosql.artemis.Id;

@Entity
public class Person {
	@Id("id")
	private long id;
	@Column @FormParam("name") @NotBlank
	private String name;
	@Column @FormParam("emailAddress") @NotBlank @Email
	private String emailAddress;
	
	public long getId() { return id; }
	public void setId(long id) { this.id = id; }
	public String getName() { return name; }
	public void setName(String name) { this.name = name; }
	public String getEmailAddress() { return emailAddress; }
	public void setEmailAddress(String emailAddress) { this.emailAddress = emailAddress; }
}

This class uses JNoSQL's annotations to define an object that can be stored (@Entity), its unique ID field (@Id), and the fields contained in it (@Column, a name that matches JPA's SQL-based view of the world). You can also see that it includes the JAX-RS and validation annotations from the class we set up when learning about MVC. With the artemis-validation dependency we included, JNoSQL will, like JAX-RS, automatically enforce bean property constraints like this when saving, meaning we don't have to spent so much time dealing with validation logic ourselves.

Whether or not it's a good idea mix the JAX-RS and persistence annotations like this is something I'm not entirely sure about, but it'll work for our purposes.

Create the Repository

Create a new interface (not a class) in the model package named PersonRepository:

package model;

import java.util.List;

import org.jnosql.artemis.Repository;

public interface PersonRepository extends Repository<Person, Long> {
	List<Person> findAll();
}

You may be thinking at this point, as I originally did, that the next step will be to create an implementation class to do the work for this. Nope! This is where some real CDI voodoo comes into play: inside JNoSQL is a bean that produces "proxy" classes on the fly for Repository interfaces and figures out implementations of the methods based on their names, return types, and parameters. It's not magic - there are limits - but in cases like this it'll do what we'd otherwise expect to have to do ourselves.

Back to the PersonController

Return to the PersonController class we created before and rework it to use our newly-minted JNoSQL objects:

package com.example;

import javax.enterprise.context.RequestScoped;
import javax.inject.Inject;
import javax.mvc.Controller;
import javax.mvc.Models;
import javax.validation.Valid;
import javax.ws.rs.BeanParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;

import org.jnosql.artemis.Database;
import org.jnosql.artemis.DatabaseType;

import model.Person;
import model.PersonRepository;

@Path("/people")
@Controller
@RequestScoped
public class PersonController {
	@Inject
	Models models;
	
	@Inject
	@Database(DatabaseType.DOCUMENT)
	PersonRepository personRepository;
	
	@GET
	public String home() {
		models.put("people", personRepository.findAll()); //$NON-NLS-1$
		return "person-new.jsp"; //$NON-NLS-1$
	}
	
	@POST
	public String createPerson(@BeanParam @Valid Person person) {
		if(person.getId() == 0) {
			person.setId(new Random().nextLong());
		}
		personRepository.save(person);
		
		models.put("person", person); //$NON-NLS-1$
		models.put("people", personRepository.findAll()); //$NON-NLS-1$
		return "person-created.jsp"; //$NON-NLS-1$
	}
}

Add a Person List Tag

Back in the "Deployed Resources" section of the project, create a new directory beneath "WEB-INF" called "tags", and within that create a new file named "personList.tag":

WEB-INF/tags directory

Set the file contents to:

<%@tag description="Displays List&kt;model.Person&gt;" pageEncoding="UTF-8" trimDirectiveWhitespaces="true" %>
<%@attribute name="value" required="true" type="java.util.List" %>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>

<h1>People</h1>
<table>
	<thead>
		<tr>
			<th>ID</th>
			<th>Name</th>
			<th>Email Address</th>
		</tr>
	</thead>
	<tbody>
		<c:forEach items="${pageScope.value}" var="person">
		    <tr>
		    	<td><c:out value="${person.id}"/></td>
		    	<td><c:out value="${person.name}"/></td>
		    	<td><c:out value="${person.emailAddress}"/></td>
		    </tr>
		</c:forEach>
	</tbody>
</table>

This is our JSP equivalent of an XPages custom control, though all of the configuration is done inline instead of via XPages's auto-maintained .xsp-config side file.

Update the Person Views

Modify "person-new.jsp":

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" trimDirectiveWhitespaces="true"%>
<%@taglib prefix="t" tagdir="/WEB-INF/tags" %>
<!DOCTYPE html>
<html lang="${translation._lang}">
	<head>
		<title>${translation.appTitle}</title>
	</head>
	<body>
		<h1>Create Person</h1>
		<form method="post" action="people">
			<dl>
				<dt>Name</dt>
				<dd><input name="name" required/></dd>
			</dl>
			<dl>
				<dt>Email Address</dt>
				<dd><input type="email" name="emailAddress" required/></dd>
			</dl>
			<input type="submit" value="Submit"/>
		</form>
		
		<t:personList value="${people}"/>
	</body>
</html>

Do similarly to "person-created.jsp":

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" trimDirectiveWhitespaces="true"%>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<%@taglib prefix="t" tagdir="/WEB-INF/tags" %>
<!DOCTYPE html>
<html lang="${translation._lang}">
	<head>
		<title>${translation.appTitle}</title>
	</head>
	<body>
		<h1>Created Person</h1>
		<dl>
			<dt>Name</dt>
			<dd><c:out value="${person.name}"/></dd>
		</dl>
		<dl>
			<dt>Email Address</dt>
			<dd><c:out value="${person.emailAddress}"/></dd>
		</dl>
		
		<t:personList value="${people}"/>
	</body>
</html>

Take It For a Spin

Launch the Liberty server and visit http://localhost:9091/javaeetutorial/resources/people. You should be able to add new entries, with the browser taking care of the client-side validation for you and JNoSQL and JAX-RS handling it on the server side. Best of all, the data should persist!

Updated UI with person listing

If you look at the database in Compass, you'll see entries there as well. JNoSQL mapped the Person class name to the "Person" collection in the database:

Data stored in MongoDB

Next Steps

In the next post, I plan to touch a bit on mixing MVC controller methods with JSON-based REST APIs, to bring these parts together into something that starts to approach a real application.

Update: Troubleshooting Note

One thing I encountered in my fiddling was an intermittent case where the server wouldn't load the app, instead complaining about an unsatisfied provider for the PersonRepository class. If you run into this, make sure you have a "beans.xml" file inside your "webapp/WEB-INF" directory in "Deployed Resources", and set its contents to:

<?xml version="1.0" encoding="UTF-8"?>
<beans bean-discovery-mode="all"
  xmlns="http://xmlns.jcp.org/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/beans_2_0.xsd"/>

This is the CDI configuration file. Though it's mostly empty, the critical part is bean-discovery-mode="all", which causes it to check all available providers in the classpath.

XPages to Java EE, Part 9: IDE Features Grab Bag

Feb 13, 2019 11:50 AM

Tags: javaee
  1. XPages to Java EE, Part 1: Overview
  2. XPages to Java EE, Part 2: Terminology
  3. XPages to Java EE, Part 3: Hello, World
  4. XPages to Java EE, Part 4: Application Servers
  5. XPages to Java EE, Part 5: Web Pages
  6. XPages to Java EE, Part 6: Dependencies
  7. XPages to Java EE, Part 7: MVC
  8. XPages to Java EE, Part 8: IDE Server Integration
  9. XPages to Java EE, Part 9: IDE Features Grab Bag
  10. XPages to Java EE, Part 10: Data Storage
  11. XPages to Java EE, Part 11: Mixing MVC and an API
  12. XPages to Java EE, Part 12: Container Authentication
  13. XPages to Java EE, Part 13: Why Do This, Anyway?

In today's post, I'm going to go over a handful of features that IDEs, particularly Eclipse, bring to the table to improve the development experience. Some of these aren't unique to EE development, but our use of Maven and standardized technology makes them better than their equivalents in XPages development.

Open Declaration

In Eclipse, if you right click on most anything in a Java file, you can choose "Open Declaration" (or press F3 with the text cursor inside it), you can go to the source of whatever it is, if available:

Open Declaration menu item

This also works in Designer, but it's often much more useful here: because Designer ships without source for its JVM or really any of the classes that make up the XPage stack, you tend to be greeted by the white "here's a bunch of bytecode" screen, which is rarely particularly useful.

Since we're working with open-source components and we told Eclipse to download source and Javadoc for Maven artifacts, though, almost everything will have available source, letting you explore what's happening much more easily.

This will also work for your own code, making F3 an extremely-useful method of navigation. While you're there, try out the "Open Type Hierarchy" and "Open Call Hierarchy" options too.

Open Implementation

CDI's technique of using @Inject to inject implementations of interfaces automatically is a great way to abstract away the business of doing a new SomethingImpl() or finding the managed bean. However, sometimes it's good to find out where the objects you're getting are actually coming from, and Eclipse lets you do this by holding Command (probably Control on Windows, if I had to guess) and hovering over an interface name:

Open Implementation menu option

In this menu, "Open Declaration" will take you to the Models interface class file, while "Open Implementation" will take you to the ModelsImpl implementation class.

Note, though, that Eclipse isn't really following CDI logic here - instead, it's just trying to find implementing classes and either opening a single implementation if there's only one or showing a menu to select from multiple. It happens that that's usually the same thing effectively, but it's an important distinction.

IntelliJ is a bit smarter on this point: it will try to figure out all the CDI logic and show a little bean icon on a line with an injection to take you to the object or method providing it:

IntelliJ bean detection

JBoss Tools

To be fair to Eclipse, there is a project to provide a great deal of improved Java EE behavior: JBoss Tools. It comes with a bunch of addins, editors, and configurators meant to make Eclipse much more aware of things like CDI's real logic. I personally have found it pretty janky in practice, but it's been a while since I gave it a proper shot. Your mileage may vary.

JAX-RS Resources

Since JAX-RS is such a critical part of modern Java development, it comes in handy that Eclipse has some specialized knowledge of it. If you expand the "Services" folder in your project, you'll have a "REST" folder that contains all of your declared JAX-RS endpoint classes and their methods:

Eclipse JAX-RS resource listing

You can use this as a general overview of your app's external API (and, when using MVC, its full URL layout) and you can also double-click any of the entries to go to the declaration.

If you get into a situation where you're writing a paired set of a server module and a client module, you can also let Eclipse generate a resource client class for you.

Deployment Descriptor

Within the "Deployment Descriptor" node in your project, Eclipse lists a bunch of EE/Servlet stuff:

Deployment Descriptor node

Historically, this referred specifically to the contents of the web.xml, but it also shows annotated classes as applicable - such as the Listener coming from the Ozark MVC implementation. Depending on how you're constructing your app, this may be more or less useful, but it's definitely good to know it's there.

Up Next

Next time, maybe I'll finally get to talking about users and data. Or maybe I won't! We'll see.

 

XPages to Java EE, Part 8: IDE Server Integration

Feb 12, 2019 10:52 AM

Tags: javaee
  1. XPages to Java EE, Part 1: Overview
  2. XPages to Java EE, Part 2: Terminology
  3. XPages to Java EE, Part 3: Hello, World
  4. XPages to Java EE, Part 4: Application Servers
  5. XPages to Java EE, Part 5: Web Pages
  6. XPages to Java EE, Part 6: Dependencies
  7. XPages to Java EE, Part 7: MVC
  8. XPages to Java EE, Part 8: IDE Server Integration
  9. XPages to Java EE, Part 9: IDE Features Grab Bag
  10. XPages to Java EE, Part 10: Data Storage
  11. XPages to Java EE, Part 11: Mixing MVC and an API
  12. XPages to Java EE, Part 12: Container Authentication
  13. XPages to Java EE, Part 13: Why Do This, Anyway?

I said that the next post was going to be about authentication or databases, but I'm turning that into a filthy lie. Instead, we're going to take a bit of a detour to talk about a couple ways to let the IDE help you out in development. I'll be talking about Eclipse specifically, but I know that at least the paid version of IntelliJ IDEA has similar features. The first feature on the docket is having the IDE manage an app server for you.

Servers

Up until this point, we've been using the TomEE Maven plugin to create and launch a server for us, which is convenient and portable across whatever environment you're working with. However, once you have a couple applications or want to make persistent server configurations outside of the individual app's pom.xml, it makes sense to run a server and deploy the app to it.

This is where Eclipse's "Servers" view comes into play. If the panel isn't currently in your workspace, go to Window -> Show View -> Other... and choose "Servers" under the "Server" category. By default, it'll be stacked with some other tabs, but I like to position this pane in the bottom-right of my IDE:

Eclipse Servers view

The way this view tends to work is that it points either at a local server installation or to a running remote server. We'll want the former, but, sadly, this is where we part ways with TomEE. Though the server is more than capable for our needs, the Eclipse integration (based on base Tomcat) is not, and it hinders its use here. Instead, we'll be switching to my current favorite: Open Liberty. Download the latest build from that page (19.0.0.1 as of this writing) and extract the archive to some location on your system (I like to keep a directory of various app servers).

Next, we'll need to install the Liberty support plugin in Eclipse, which is a fortunately-simple matter. Visit the Liberty in Eclipse download page and drag the "Install" button onto your Eclipse toolbar. Once it loads, click Next and Finish a couple times until it's done, and then let it restart Eclipse.

Once Eclipse restarts, click that "Click this link to create a new server..." link. In the resultant dialog, choose "Liberty Server" under the "IBM" category and give it a descriptive name:

Eclipse's New Server dialog

Click Next > and enter the path where you extracted Open Liberty in the "Path" field:

Server path

Click Next >, leave everything at its default, and click Finish.

Click the twistie next to the newly-created server and double-click the "Server Configuration [server.xml] new server" entry to open the server config editor. If it opens on the "Design" tab, click the "Source" tab to see the XML configuration. Set it to:

<server description="new server">
    <!-- Enable features -->
    <featureManager>
        <feature>javaee-8.0</feature>
        <feature>localConnector-1.0</feature>
    </featureManager>

    <!-- To access this server from a remote client add a host attribute to the following element, e.g. host="*" -->
    <httpEndpoint httpPort="9091" httpsPort="9443" id="defaultHttpEndpoint"/>

    <!-- Automatically expand WAR files and EAR files -->
    <applicationManager autoExpand="true"/>
  
    <keyStore id="defaultKeyStore" password="testKeystore"/>
</server>

 

The first difference here from the default is to swap out the jsp-2.3 feature for javaee-8.0. Liberty organizes its capabilities by "features" in order to let you really trim down the runtime if you don't need specific capabilities. For our sake, we want the whole EE shebang available, though. localConnector-1.0 remains, and is what allows Eclipse to control the server.

We also changed the httpPort of the httpEndpoint element to 9091 to match what we've been using via the Maven config. We also added a basic not-exactly-secure keyStore configuration. This would be filled in more if you enable SSL.

Deploying the App

There are several ways to actually get our app onto this server, but the one I've built up a habit of using is to right-click the project, and then select Run As -> Run On Server:

Run on Server

The newly-created server should be selected by default in the resultant dialog, so just click Finish immediately.

The server will churn for a bit and eventually output [AUDIT ] CWWKT0016I: Web application available (default_host): http://localhost:9091/javaeetutorial/. It will also, I note with chagrin, output this:

[ERROR   ] SRVE0283E: Exception caught while initializing context: java.lang.IllegalArgumentException: The controller com.example.HomeController is not a managed CDI bean. Maybe the controller class is missing a scope annotation (e.g. @RequestScoped).
	at org.mvcspec.ozark.servlet.OzarkServletContextListener.failIfNoCdiBean(OzarkServletContextListener.java:76)
	at org.mvcspec.ozark.servlet.OzarkServletContextListener.contextInitialized(OzarkServletContextListener.java:60)
	at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:2377)
	at [internal classes]

That's because I forgot a step in the setup of MVC controllers yesterday, which is that you're supposed to add one of those annotations to the class too. Fortunately, the examples still work, and we'll fix that omission next time we edit the code.

Most likely, Eclipse will automatically open the app's default page in whatever browser it's configured to use - by default, a browser embedded in Eclipse itself.

Updating the App

In future posts, rather than specifically saying to run the clean install tomee:run Maven goal, I'll just say to run the app - either way should still work. By default, Eclipse will re-deploy the application to Liberty after changes, so you probably won't need to do anything other than refresh the page to see future changes.

Next Up

Next, I think I'm going to cover a grab bag of other features Eclipse has for working with Java EE apps before returning to the dirty business of writing code.

 

XPages to Java EE, Part 7: MVC

Feb 11, 2019 3:04 PM

Tags: javaee
  1. XPages to Java EE, Part 1: Overview
  2. XPages to Java EE, Part 2: Terminology
  3. XPages to Java EE, Part 3: Hello, World
  4. XPages to Java EE, Part 4: Application Servers
  5. XPages to Java EE, Part 5: Web Pages
  6. XPages to Java EE, Part 6: Dependencies
  7. XPages to Java EE, Part 7: MVC
  8. XPages to Java EE, Part 8: IDE Server Integration
  9. XPages to Java EE, Part 9: IDE Features Grab Bag
  10. XPages to Java EE, Part 10: Data Storage
  11. XPages to Java EE, Part 11: Mixing MVC and an API
  12. XPages to Java EE, Part 12: Container Authentication
  13. XPages to Java EE, Part 13: Why Do This, Anyway?

I mentioned in the last post that the rest of this tutorial is going to involve making some technology choices that are less universal than the ones up until this point. While pretty much every new EE app is going to involve CDI and JAX-RS, things will start to diverge once you decide on how you're going to handle authentication, your UI, and your data storage. We're going to dip into the the second one of those today.

Specifically, we'll be choosing the characteristically-dryly-named MVC Spec as the foundation for our interface.

MVC

If you're not familiar with the term "MVC" in a general sense, it stands for Model-View-Controller, and it represents one of the common ways to structure your application to keep the code clean and growable without it turning into a nightmare. There are other ways, and there are some flaws in the design that require band-aid solutions, but in general it remains a very-useful way to write your app. The general idea is that you have three components: the "model" (your data), the "view" (what your user sees), and the "controller" (what connects the two). On the web, this often takes the shape of having controllers pointed to by the "routing" within your application (e.g. /posts/new), which then do the work of fetching the data and binding it to the view.

XPages is ostensibly MVC-based, but that doesn't really play out in reality. The stack and IDE encourage direct mingling of the view and data (the <xp:dominoDocument/> data source and SSJS are the primary culprits here), and the lack of a model framework and the stultifying strictures of the built-in NHTTP routing make it very difficult to do it right even if you try (and lord knows we tried).

The MVC Spec

I've talked a bit about the spec before, and the idea of it is to build a simple-to-write-and-read MVC standard for Java EE, using existing EE technologies for the "view" part. I believe that the spec is heavily based on Spring MVC, though I haven't written any Spring apps.

The main reason I'm such a big fan of this spec is that it builds cleanly on top of JAX-RS, which already provides an excellent skeleton for cleanly-organized apps. JAX-RS already encourages clean, RESTful design for accessing data objects, and MVC builds on that to provide a natural way to display a web UI for non-API clients (which is to say, humans).

One thing to bear in mind with MVC 1.0 is that it's not quite a true Java EE component. It was slated for inclusion in Java EE 8, but Oracle cut it from the list before release. However, the spec has support within the Jakarta EE community and remains a likely candidate for future inclusion. Moreover, because the spec is so small and strongly encourages clean design, I feel that it's worth building upon - even if it disappeared tomorrow, almost all of your code would still work.

Adding to the Project

Since the spec isn't included in the default Java EE 8 spec or with EE servers, we'll need to add explicit dependencies to the pom.xml for the project, for both the spec itself and the reference implementation (currently named "Ozark", but it's going through a rename to "Krazo" for trademark purposes):

<!-- MVC 1.0 -->
<dependency>
  <groupId>javax.mvc</groupId>
  <artifactId>javax.mvc-api</artifactId>
  <version>1.0-pfd</version>
</dependency>
<dependency>
  <groupId>org.mvc-spec.ozark</groupId>
  <artifactId>ozark-core</artifactId>
  <version>1.0.0-m04</version>
</dependency>

Creating the Controller

MVC Controllers are implemented as normal JAX-RS resources with the extra @Controller annotation, and whose methods return an indicator of what view to use. Create a class in the com.example package called HomeController:

package com.example;

import java.time.LocalDateTime;

import javax.inject.Inject;
import javax.mvc.Controller;
import javax.mvc.Models;
import javax.ws.rs.GET;
import javax.ws.rs.Path;

@Path("/")
@Controller
public class HomeController {
	@Inject
	Models models;
	
	@GET
	public String hello() {
		models.put("hello", "Hello, MVC!"); //$NON-NLS-1$
		models.put("now", LocalDateTime.now()); //$NON-NLS-1$
		
		return "hello.jsp"; //$NON-NLS-1$
	}
}

The "Models" object is a MVC-provided object that acts sort of like a viewScope: you toss whatever objects you'd like into it and they're available as variables on your page. Despite having an important-sounding name, it's really just a simplified Map.

Creating the View

MVC supports several "templating" technologies, among them JSP and JSF Facelets. However, though it uses JSF technology, it's not meant to be used for a full JSF app with server-side state. From what I can tell, the JSF support is more for those who already have JSF apps to use. Despite XPages's heritage, that doesn't really apply to us. Create a new folder named "views" within the "webapp/WEB-INF" directory in "Deployed Resources" (which is "src/main/webapp" in the filesystem). Then, create a new file and call it "hello.jsp":

hello.xhtml location

Set its contents to:

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" trimDirectiveWhitespaces="true"%>
<!DOCTYPE html>
<html lang="${translation._lang}">
	<head>
		<title>${translation.appTitle}</title>
	</head>
	<body>
		<h1>${hello}</h1>
		<p>I was loaded at ${now}.</p>
	</body>
</html>

Since this still uses CDI, we're still able to use the translation bean we set up earlier, but now we have access to the extra variables the controller set up.

If you do another Maven build with goals "clean install tomee:run" and visit http://localhost:8081/javaeetutorial/resources now, you should be greeted with the expected basic page:

Basic MVC output

Working With a Model

So that's two out of three down; time to simulate some data access. Using a real backing database will be its own large topic, so for now we'll create a simple in-memory object.

Create a new class in the "com.example" package named "PersonController":

package com.example;

import javax.inject.Inject;
import javax.mvc.Controller;
import javax.mvc.Models;
import javax.validation.Valid;
import javax.validation.constraints.NotEmpty;
import javax.ws.rs.BeanParam;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;

@Path("/people")
@Controller
public class PersonController {
	public static class FormPerson {
		@FormParam("firstName") @NotEmpty
		private String firstName;
		@FormParam("lastName") @NotEmpty
		private String lastName;
		
		public String getFirstName() { return firstName; }
		public String getLastName() { return lastName; }
	}

	
	@Inject
	Models models;
	
	@GET
	public String home() {
		return "person-new.jsp"; //$NON-NLS-1$
	}
	
	@POST
	public String createPerson(@BeanParam @Valid FormPerson person) {
		models.put("person", person); //$NON-NLS-1$
		return "person-created.jsp"; //$NON-NLS-1$
	}
}

Next, create a new file in the "WEB-INF/views" directory named "person-new.jsp":

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" trimDirectiveWhitespaces="true"%>
<!DOCTYPE html>
<html lang="${translation._lang}">
	<head>
		<title>${translation.appTitle}</title>
	</head>
	<body>
		<h1>Create Person</h1>
		<form method="post" action="people">
			<dl>
				<dt>First Name</dt>
				<dd><input name="firstName" required/></dd>
			</dl>
			<dl>
				<dt>Last Name</dt>
				<dd><input name="lastName"/></dd>
			</dl>
			<input type="submit" value="Submit"/>
		</form>
	</body>
</html>

Finally, create a second JSP file in the same directory named "person-created.jsp":

<%@page contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" trimDirectiveWhitespaces="true"%>
<%@taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<!DOCTYPE html>
<html lang="${translation._lang}">
	<head>
		<title>${translation.appTitle}</title>
	</head>
	<body>
		<h1>Created Person</h1>
		<dl>
			<dt>First Name</dt>
			<dd><c:out value="${person.firstName}"/></dd>
		</dl>
		<dl>
			<dt>Last Name</dt>
			<dd><c:out value="${person.lastName}"/></dd>
		</dl>
	</body>
</html>

Now, do another Maven build and visit http://localhost:9091/javaeetutorial/resources/people:

Create Person page

In this screenshot, you can see that the "First Name" field got a red outline because it's marked as required and I entered and then deleted a first name value. Neat!

Anyway, fill in something for "First Name" but not "Last Name", and hit "Submit". You'll be greeted with... nothing! Well, visibly nothing, anyway. You'll have a 400 response in your browser with no content, and you'll see a line like this in your server console:

11-Feb-2019 14:52:55.942 WARNING [http-nio-9091-exec-2] org.apache.cxf.jaxrs.validation.ValidationExceptionMapper.toResponse Value '' of PersonController.createPerson.arg0.lastName: {javax.validation.constraints.NotEmpty.message}

Hey, the server-side validation worked! Down the line, we'll probably add an exception handler to display this type of thing to the user, but, for now, the important part is that invalid data was blocked before it even got to our code.

Go back and enter something for each of the name fields, and then hit "Submit" again. While doing so, bask in the pleasant fact that, because there's no server-side state, you don't need to worry about mangled JSF view state or anything. Once you submit, you should be greeted with the "Created Person" page with your data:

Created Person page

As a nice bonus, because this output page uses the JSTL <c:out/> tag, the values are nicely HTML-escaped, making it a bit more secure than our original Hello World page.

Next Steps

In the next couple posts, we'll cover the ominous topics of authentication and data storage.

Letting Madness Take Hold: XPages Outside Domino

Jan 7, 2019 10:59 AM

Tags: xpages
  1. Letting Madness Take Hold: XPages Outside Domino
  2. XPages on Android
  3. Concept Proven: A Complex Production XPages App Running Outside Domino

(Opening caveat: unlike some of my other recent dalliances, I don't plan to actually do anything with this one, and it's more of a meandering exploration of the XPages platform)

Since I've been on a real Open Liberty kick lately, over the weekend I decided to go another step further and test something I'd been wondering for a while: whether it'd be possible to run the current form XPages outside of the Domino HTTP stack.

I say "the current form" because XPages's history is long and winding, and led a fruitful life for a long time before being glommed onto Domino at all. If you poke around the core, you can see it bears all the scars of its life: references to WebSphere Portal abound, half of the plugins that make up the runtime are just thin OSGi wrappers around plain old Jars, and all of the "Domino" bits are clearly labeled as "adapters".

Still, it's been over a decade since the stack was intended to run anywhere outside Domino, and that's a lot of time for ingrained assumptions about nHTTP specifically to creep in. Still, I was curious if it was possible to load it up outside of Domino and without OSGi.

Short Answer

Yep!

Long Answer

There are a couple things that contribute to making this setup practical, and they each bear some expansion.

Platforms and Execution Contexts

At a couple levels, the runtime breaks things up into generic concepts of "Platforms" and "ExecutionContexts" to handle some specifics about context directories, class loaders, and other bits. For example, if you get a type hierarchy on com.ibm.commons.Platform in Designer, you'll get a pretty immediate idea of what's going on:

OSGi/Services Bridge

Anyone who has written an XPages Library plugin is familiar with the concept of an OSGi extension: you declare your extension (for our purposes, and usefully, com.ibm.commons.Extension) in plugin.xml and then the environment picks up on it by the code looking for such extensions. The core Java runtime has a similar mechanism - ServiceLoader - that looks for files with the name of the extension type in the META-INF/services directory in your classpath. The result of both is the same: individual Jars/plugins can declare services and some other part of the app can pick up on them without knowing the specifics.

XPages uses IBM Commons's generic "Extension" type to paper over the differences between these, and the runtime will look for both or either depending on where it's working. And here's another part that conveniently still retains the vestiges of its youth: if you look inside the com.ibm.xsp.core plugin (since it's just a ZIP file), you can see these extensions declared both in the top-level plugin.xml and as individual files inside the embedded Jar:

So, if you load in these inner Jars as normal Maven dependencies in a .war file, the services will still tie together in much the same way, at least for the core runtime. Things get less convenient the newer the code is, though: the Extension Library, for example, primarily uses plugin.xml for its services, and so either an adapter runtime would have to look for this or you'd have to re-declare them in the "normal" way.

Light OSGi Use

Speaking of OSGi, that's one of the big potential stumbling blocks. XPages nowadays expects to run inside an Equinox container, and so a lot of code (say, the Dojo plugins) make assumptions about the loading of Activator classes and other things. These need some patching. Fortunately, the actual use of OSGi in most of these cases is extremely light: mostly, it's about instantiating these activators and then getting bundle class loaders. For basic needs, these can just be shimmed in: find the (blessedly public) static instance property in the applicable classes and put in small BundleContext Bundle adapters that just return the context class loader. I'm sure there are bits that run deeper than that, and long-term it'd probably be more practical to just fire up Equinox, but this works for now.

FacesServlet

The core work of rendering an XPage runs through the class FacesServlet and more specifically DesignerFacesServlet (as a side note, I've gathered that seeing "Designer" in these classes refers most likely to "Lotus Component Designer", since those parts of the stack enter in before the Domino dependencies). In a modern JEE context, this'll take a little bit of wrapping, since it implements Servlet but doesn't extend HttpServlet, but not too much. For the most part, once you have your platform set up above, you can make a standard @WebServlet-annoted class and delegate the HttpServletRequest and HttpServletResponse objects to one of these, and it'll pick up on any compiled xsp.PageName classes in your .war:

@Override
public void init(ServletConfig config) throws ServletException {
  this.delegate = new DesignerFacesServlet();
  delegate.init(config);
}

@Override
protected void service(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {	
    delegate.service(new LibertyServletRequestWrapper(req), resp);
}

 

DesignerGlobalResourceServlet

Alongside the core Faces servlet, the DesignerGlobalResourceServlet does the work of, well, serving up global resources. This one's simpler than the Faces servlet, since it is indeed a fully-fledged HttpServlet. You could just declare this in your web.xml, but I like extending these classes in case I want to fiddle with them later:

@WebServlet(urlPatterns="/xsp/.ibmxspres/*")
public class LibertyGlobalFacesResourceServlet extends DesignerGlobalResourceServlet {
	private static final long serialVersionUID = 1L;
}

The NSF Part

Up until now, what I was able to create was a way to run XPages inside a normal web app without any real connection to Domino (other than pulling in the binary plugins). Actually running an existing XPage out of an NSF requires a little more bootstrapping, and unfortunately confines the page a bit.

Specifically, the most expedient route I found to accomplishing this was to fire up an LCDEnvironment object and ferry requests for NSF-hosted apps to this. With the presence of an active Notes runtime (via NotesThread.sinitThread() and bringing in Notes.jar and the NAPI plugin), LCDEnvironment#initialize will do a lot of legwork in assembling its own little world inside your application. It will look for com.ibm.designer.runtime.domino.adapter.HttpService declarations and bring them in, including the vitally-important NSFService.

The nice part of this is that it does a ton of work, handling not just XPages requests, but also in-NSF resource requests. The down side is that the NSFService does its work by heavily wrapping the environment, down to providing a servlet context that declares itself as 2.4 even in a 4.0 runtime. Still, a bit of code in the service method gets it working nicely:

String contextPath = StringUtil.toString(req.getContextPath());
String path = req.getRequestURI().substring(contextPath.length());
RequestContext requestContext = new RequestContext(contextPath, path);
HttpSessionAdapter sessionAdapter = new ServletHttpSessionAdapter(req.getSession());
HttpServletRequestAdapter requestAdapter = new LibertyServletRequestWrapper(req);
HttpServletResponseAdapter responseAdapter = new ServletHttpServletResponseAdapter(resp);
lcdEnvironment.service(requestContext, sessionAdapter, requestAdapter, responseAdapter);

Those adapter/wrapper classes really just delegate the calls, but they're needed because that's what LCDEnvironment expects. I imagine those interfaces exist to create a consistent environment in lots of situations without even tying to the stock HttpServlet* classes. In general, the XPages stack loves adapters.

So, Is This A Good Way To Run XPages?

Nnnnnnnnnope! I mean, not really. Particularly in the first route, where you load up an XPage without any knowledge of the NSF part, you have some intriguing paths to interact with the surrounding Servlet 4.0 environment directly from the page. However, I don't know why you would do this. You could hypothetically create some components or hooks to allow use of, say, Web Sockets, but you'd be better off just using current JSF for that, since the work is already done. The in-NSF XPages runtime adds some extra barriers to that, too... I'm sure it'd be possible to provide a path to it, but, again, you'd be better off using existing tech.

Additionally, this isn't a way to bring XPages "home" to JSF. The JSF API and implementation that XPages uses isn't merely old, but hacked to pieces: if you look at javax.faces.component.UIComponent, you can see it's riddled with "_xsp" methods, indicating a thoroughly-unclean layering. You wouldn't be able to shim in the current JSF without forking it into a distinct project.

Still, it's nice to know it's possible, and it sure was a fun project to tinker with. I do admit that the notion of building an XPages app using MVC 1.0 with XPages as the view technology is a little tantalizing, but it's certainly not worth traveling down that road on the back of a chopped-up hacky rework of the platform. And the only reason it's tantalizing is that I'm still more comfortable with XPages than any of the other front-end stacks, and that doesn't itself make it a good fit. Fun to think about, though.

How Do You Solve a Problem Like XPages?

Nov 2, 2018 12:08 PM

Tags: xpages gloom

(Fair warning: this is a meandering one and I'm basically a wet blanket the whole way through)

Last week, HCL held the third of their Twitter-based developer Q&As, with this one focusing on XPages and Designer. The majority of the questions (including, admittedly, all of mine) were along the lines of either "can we get some improvements in the Java/XPages stack?" or "is XPages still supported?". The answer to the latter from HCL, as it would have to be, is that XPages is still alive and "fully supported".

I don't doubt at all that XPages is supported in the sense that it has been for the last couple years: if you as a customer encounter a bug in the platform, support will take your call and will most likely either have a workaround or will get a fix in. This will no doubt happen naturally as time marches on, primarily when a new version of a browser breaks something in the version of Dojo that XPages uses (currently, I believe, a couple notches down the list as 1.9.7). So, that's good, and is better than the worst-case scenario.

It's not great, though. Version 10 had essentially no changes for XPages - that makes sense with its "stanch the bleeding" market goal, but it continues the history of very little progress. Outside of the forced-for-security-reasons bump to Java 8 in 9.0.1FP8 (which is admittedly nice), the last major addition to XPages was the Bluemix tooling via the Extension Library, which, as far as I can tell, only exists because of the way funding politics works inside IBM. Before that, I'd mark it as the promotion of Bootstrap renderers from Bootstrap4XPages to the main ExtLib just shy of four years ago. Before that, it was... I guess adding the ExtLib to the main product in 9.0, which sort of counts. Before that, it was pretty much the introduction of the extension points and ExtLib in the 8.5.2 era. And sessionAsSigner, I suppose.

Not to belabor the point too much further, XPages developers are in an uncomfortable spot. For a decade or so now, XPages was the clear "this is what Domino developers should be doing" choice, especially in the face of many Domino shops wanting to at the very least get rid of the Notes client. However, though it was modern enough when it was introduced, the stack has missed the boat on a lot of evolution, both in simple terms of its Java EE common ancestor improving on its own and in larger terms of changes in the web development world.

Had XPages continued under real active development, it could have gradually improved to fit more comfortably in a world of transpiled JavaScript and CSS pre-processors, strict focus on REST APIs, and reactive and streaming APIs.

But...

But even in that "active development" alternate universe, the path would have been awkward. Though XPages has the capacity to be well-structured, Designer and IBM provided no help on this: no model framework, no internal routing, not even an indication that writing Java code was possible in an XPages app until several versions in. The "MVC" aspects of its JSF components were beaten down to match the expectations of an NSF container and of LotusScript developers. There's very good reason for that - very few Domino shops were likely to send developers to computer-science boot camps to learn about proper Java EE structure, and the only way XPages was going to work at all was if it started out as "forms but with partial refresh".

And, after that first unstructured version, it largely fell prey to the usual problems of enterprise software: IBM isn't in the business of doing things their customers aren't asking for, and the community members asking for, say, a faces-config.xml editor weren't backing those requests up with big licensing checks. I get that, too, I really do. If you spend too much time hypothesizing about and implementing what customers might want, you run the risk of throwing your money down a bottomless hole while your actual customers suffer and leave. So IBM generally swings hard in the other direction, and it's understandable if unfortunate in cases like this, especially when what your customers are strictly asking for is stasis.

So What Now?

Our current situation now is that HCL plans to have a roadmap in Q1 2019, so I suppose we'll wait and see what they say again. I've been mulling over what I think should be the way forward for XPages and its users, and I don't see any clear good solution.

The option that immediately springs to mind is "add more features to it": HTTP/2 and WebSockets, newer renderers, an IDE that encourages good development practices, better way to bring in third-party libraries, cleaner JavaScript, and so forth. But what makes this questionable for me is the sheer amount of work, especially since they'd have to start by digging out of an immense amount of technical debt. And this would be very specialized work indeed - the XPages stack is complicated. I'd wager that just the part of the stack that handles ferrying attachments between the browser and a document is more complex than most entire Domino applications by a good margin. While some of the improvements would be handled via the core Domino server team (part of the HTTP upgrades, namely), HCL would likely have to acquire a team of Java developers and have them learn a giant stack of OSGi, JSF, Domino APIs, a couple decades of legacy decisions before even getting going. Possible? Certainly. It's just kind of a hard sell.

Another possibility would be open-sourcing the stack and either maintaining it as a project themselves, giving it to OpenNTF, or handing it to an organization like Eclipse, which now holds the reins of Java/Jakarta EE generally. I think that open-sourcing it would have immediate benefits to XPages developers regardless of what else they do with it, but I'm skeptical of how much of a life it would have if it was converted to a community-run project. As it stands, I can only think of a handful of people who a) are aware of XPages and b) would be capable of contributing to its core code. That's not a problem if you are employing people to work on it full-time, but I don't think it has a large enough base to exist on a "side project" basis. Attracting new blood would be an uphill battle: even projects like Andmore that have a clear purpose for existing and a contingent of people desperate to keep their workflow can wither on the vine immediately. Outside of Domino developers, XPages would be viewed as "JSF, except old and restricted to a platform you thought died in the 90s".

The other main thing to do with the stack that doesn't involve killing it, I think, would be pushing it to a state that focuses on REST APIs instead of handling the UI itself, which is something that some XPages devs have been doing already, either by switching to plugins serving up JAX-RS services or via in-NSF controls. This is something that IBM kind-of-sort-of said they were aiming for a couple years ago when they put forward SmartNSF as a good option, but the effective demise of the Extension Library cut off its path into the core, at least for now. Overall, I think that this would make sense. The experience of writing REST services inside an NSF (or in an OSGi plugin) is significantly worse than JAX-RS in a normal Java EE or Spring app, but it provides a clear path for existing NSFs and code to continue being used - more or less - without having to set up a second app server. It may not be the preferred solution for Java in Domino, but it would have the advantage of improving the platform without having to worry anymore about how, say, all the core renderers use tables for layout.

For Developers

For XPages developers, my long-proffered advice remains the same: learn things that aren't XPages. Whether that means diving head-first into Java EE or Spring, focusing on client-JS development, learning Node, or any other option, you'll likely be well-served by it.

It's possible that you could continue to do XPages work or go back to the Notes client indefinitely - XPages will get patches if nothing else, and Nomad breathes undeserved life into LotusScript - but there's no guarantee there. There's no guarantee anywhere else, I suppose, but staying too tied to Domino-specific technology keeps you at immediate risk of a from-above directive to switch away.

In short: do not expect a cavalry to ride to your aid.

Reforming the Blog in Darwino, Part 4

Jul 20, 2018 6:59 PM

Tags: java darwino
  1. Reforming the Blog in Darwino, Part 1
  2. Cramming Rails Into A Maven Tree
  3. Reforming the Blog in Darwino, Part 2
  4. Reforming the Blog in Darwino, Part 3
  5. Reforming the Blog in Darwino, Part 4

Last time, I went over my switch in tack for how I'm making the new version of my blog, and my overall focus on picking an interesting stack of JEE technologies. In this post, I'm going to start diving into the implementation of the UI, though I think that it will make sense to dedicate two posts to it.

The biggest decision I made with the UI side of this app is that I didn't want to make a client-side JS app. There's a reason they're so ascendant, and I find development with React or Stencil pretty enjoyable, but I wanted to go a different route here for a few reasons:

  • For a blog, a CSJS app is wildly overkill, and, in fact, would require extra work to fulfull one of the basic requirements of a blog, which is being web-crawler friendly.
  • I want to see how svelte I can make the client payload.
  • Skipping a JS framework (and a CSS one) is a great way to brush up on what plain HTML and CSS are capable of nowadays.
  • Unlike a typical Darwino app, my only target is a full-on Java web server, so I'm not held back on the Java side by the capabilities, say, of Dalvik on Android 4.
  • Part of me misses the simplicity of my early PHP days, albeit not the language.

The Java Side

I decided to go with the MVC 1.0 draft spec because it lets me write extremely focused code. Here is the controller for the home page:

package controller;

import javax.inject.Inject;
import javax.mvc.Models;
import javax.mvc.annotation.Controller;
import javax.ws.rs.GET;
import javax.ws.rs.Path;

import model.PostRepository;

@Path("/")
@Controller
public class HomeController {
	@Inject
	Models models;
	
	@Inject
	PostRepository posts;
	
	@GET
	public String get() {
		models.put("posts", posts.homeList());
		
		return "home.jsp";
	}
}

Naturally, there's a lot of magic going on behind the scenes - there's tons of heavy lifting going on here by JAX-RS, MVC, CDI, JNoSQL, and Darwino - but that's the point. All the other components are off doing their jobs in their areas, while the code that provides the UI doesn't have to care about the specifics.

Things can get more complicated on the pages that actually have some functionality to them, but the code remains pleasantly focused. Take the handler for deleting posts:

@DELETE
@Path("{postId}")
@RolesAllowed("admin")
public String delete(@PathParam("postId") String postId) {
	Post post = posts.findByPostId(postId).orElseThrow(() -> new IllegalArgumentException("Unable to find post matching ID " + postId));
	posts.deleteById(post.getId());
	return "redirect:posts";
}

This adds another level of magic in the form of javax.security.annotation.RolesAllowed, but it's more of the good kind: even with no knowledge of the underlying frameworks, it's pretty clear what every bit of code is doing here. It's a refreshing bit of that Rails simplicity, just more compile-type-safe and much uglier.

Even beyond the minimal code is the cleanliness that this brings to the structure of the application: other than the img, css, and js paths, all of the routing within the application is done care of JAX-RS and MVC. It's not beholden to the folder structure in the project or to a Domino-style implicit app router.

JSP

JSP has been the prototypical Java HTML language for about 20 years, and it's had a rough upbringing. The early versions committed the PHP/XPages sin of encouraging you to put business logic right on the page, and it even still has the typical Java problem that it's tricky to find advice about using it that uses technologies added since 2005.

Still, when used properly, it can be a nice, clean templating language. Again from the main home page:

<%@page contentType="text/html" pageEncoding="UTF-8"%>
<%@taglib prefix="t" tagdir="/WEB-INF/tags" %>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<t:layout>
	<c:forEach items="${posts}" var="post">
		<t:post value="${post}"/>
	</c:forEach>
</t:layout>

For an XPages developer, this is extremely comfortable. It's also very refreshingly elemental: there's no server-side persistence of the page, so everything is "load-time bound" and, with just HTML tags and core JSTL tags, nothing ends up on the page that you don't explicitly put there.

Ozark, the MVC implementation, also supports using JSF "Facelets" for the view portion, but JSP suits the task just fine.

HTML + CSS

It'd been far too long since I last really sat down and looked at what baseline HTML and CSS are like - in particular, I'd watched the rise of CSS Flexbox and Grid from afar, and never gave them a shot. Using components that generate their own HTML and pre-existing CSS frameworks to target with class names is all well and good, but it does leave you a bit disconnected from the fundamentals.

So, for this iteration, I tossed aside the very-nice Bootstrap framework I've been using, dusted off one of my old hand-built ones, and got to translating it into CSS Grid. This cut down on the page size enormously: I had already echewed Dojo by not using XPages, but this now also meant that I could ditch the core bootstrap.css, jQuery, and any jQuery plugins.

Beyond CSS Grid, have you seen how nice HTML forms are nowadays? Just looking at this post reveals how much is built in in the way of validation and different input types, even before you write a line of JavaScript.

Turbolinks

Having such a trimmed-down UI means that pages already load extremely quickly, but I figured this was also a perfect chance to try out a bit of clever tech from the team at Basecamp: Turbolinks. Turbolinks is a JS file that you bring into your app which then transparently takes over your in-app links to minimize the amount of rendering you have to do. Since the surrounding boilerplate of the app usually doesn't change between requests, it can figure out the "diff" between old and new and just replace the body. It's essentially like partial refreshes without the server knowing anything about it.

It's still technically inefficient to have the server render and transfer surrounding page elements that are just going to be discarded anyway. But, on the other hand, skipping that means that I don't have to write JavaScript handlers myself, use a full CSJS app framework, or keep state on the server side. The server just keeps doing what it does with a fully context-less request and the browser sorts it out. Basecamp's programmers are masters at the targeted deployment of kludges for maximum benefit.


In the next (final?) post in the series, I'll finish up with my "low-JS" experience and other lessons learned from this project.

Reforming the Blog in Darwino, Part 3

Jul 18, 2018 10:30 AM

Tags: java
  1. Reforming the Blog in Darwino, Part 1
  2. Cramming Rails Into A Maven Tree
  3. Reforming the Blog in Darwino, Part 2
  4. Reforming the Blog in Darwino, Part 3
  5. Reforming the Blog in Darwino, Part 4

A good while back, I created a project structure for reforming my blog here in Darwino, but, as happens with low-priority side projects, it withered on the vine, untouched since then. Beyond just the "cobbler's children" aspect to it, I also lost steam due to a couple technology paths I initially headed down.

The first was basing the UI on Angular, which I've never really enjoyed working with. I'm sure I could have ended up with a decent result with it, but Angular always rubbed me the wrong way. And not just Angular: for a dead-simple UI like this, a full JS UI is just weird overkill.

The second was off in the other direction: I initially tried cramming a Rails app in the tree, which could be made to work, but it introduced so many weird edge cases outside of the problem at hand. That alone isn't the end of the world, but not much of what I'd have to solve to make that work would be transferrable elsewhere any time soon, so it'd end up a real time sink.

So, taking what I've learned since and the projects that I've been working on, I've decided to take another swing at it. Before I get into the implementation side, it will be useful to go over the technologies I did choose for the new form.

Java/Jakarta EE

I've recently become kind of enamored with the modern form of the Jakarta EE stack, and so I decided to use this as an opportunity to really dive in to what a blue-ocean small by-the-books Java app looks like nowadays.

JEE got a well-deserved bad rap over the years for its configuration complexity and general impenetrable-ness, but I've been very pleased to find that those tides have largely receded. It's all still there if you want it, but a fresh new app primarily consists of decorating a handful of Java classes with declarative annotations.

JEE consists of a series of individual specs, and building an app involves choosing which ones you want to use, plus (depending on which you choose) picking your app server target.

Tomcat

I originally gave a shot to adding enough OSGi metadata and bundles to target Domino, but decided quickly that it was just not worth it. The HTTP/servlet stack in Domino is just so old that, even if I got everything bound together, I'd still be fighting the platform every step of the way.

The better route was to put it aside and just run a modern Java app server. I went down the list of GlassFish, Payara, WebSphere Liberty (the nearest miss), TomEE, and WildFly, but each one ended up having some problem with either the dependencies I wanted or with their Eclipse integration. I ended up settling on good ol' reliable Tomcat. Tomcat itself isn't actually a JEE server, but it's kind of like a Raspberry Pi: it gives you the baseline for a Java servlet engine, and then you can cobble together your own EE stack on top of it by explicitly bringing in implementations. Though the final .war file is far less svelte this way, I found that this build-your-own method results in the lowest chance of being held back by the platform currently.

As an aside, Sven Hasselbach has been writing a very interesting series on running Jetty on top of the Domino JVM to achieve a similar end, albeit with Spring.

Darwino

For all the same reasons as when I set out on this journey originally, I'm using Darwino for the baseline. This lets me replicate in my existing blog data smoothly while getting the advantages of a superior backing database. I'm not making use of mobile clients or most Darwino services with this, but the baseline is nonetheless a step up, and fits in with a JEE app like a glove.

JNoSQL

I brought in the JNoSQL Darwino driver I wrote a little while ago to handle the model layer. JNoSQL is essentially JPA but reformed for NoSQL access - no cruft, no relational/NoSQL impedence mismatch, and designed to fit with current JEE technologies.

CDI

CDI is one such technology, and it's a very interesting one to work with. The whole "dependency injection" realm is a little fraught and, if my Eclipse UI error reporter is any indication, prone to bizarre errors, but the core concept is good and very useful. I've gotten it into the swing of using it both as the "managed bean" provider for the front end as well as the general service provider glue for the app. It still takes some getting used to, and the learning curve falls prey to a similar problem as when I was learning Maven: something about learning how it works makes you forget what it was like to not know, and so a lot of the answers online assume way more knowledge than a neophyte has.

Bean Validation

I've long been a fan of the Java bean validation API, and it's a clean fit here too: JNoSQL picks up on the presence of Hibernate Validator without configuration beyond the dependency and it just works. No muss, no fuss.

JAX-RS + MVC Spec

JAX-RS is at this point familiar territory for a lot of Domino developers, but I decided to use it as the underpinnings of the whole UI, in tandem with a draft framework called MVC 1.0. The latter's generic name doesn't really give much detail, but it's essentially a spec that enhances JAX-RS entities with knowledge of HTML templating frameworks, allowing you to write a very clear app structure. It's not a server-state-based framework like JSF, but rather a bit "closer to the metal", where you deal directly with the HTTP method cycle.

As I'll go more into in the "UI" post, it's been surprisingly refreshing to get back to basics in this way - JSF/XPages is often a bit conceptually easier to work with (at first) and client-side JS frameworks have some REST+JSON purity to them, but just "this server-rendered HTML page with no server state is everything you need" feels really good sometimes.

Admittedly, the MVC spec itself is in a weird place. It was originally a candidate for inclusion in Java EE 8, but was dropped in the final runup. It's possible that this will prove to be a kiss of death, but the spec is so small but functional that I don't feel bad about taking the risk of building an app on it.


That about covers the technology stack. When I get around to writing the next post, I'll go into some of the specifics about how I decided to set up the UI, which has been a fun experiment of its own. In the mean time, the active repository is up at:

https://github.com/jesse-gallagher/frostillic.us-Blog/tree/develop/frostillicus-blog

Provisions for the Journey

Jun 19, 2016 11:27 AM

Tags: development

In my last post, I ended up recommending that Domino developers and administrators take some time, if at all possible, to dive into new tools. In some cases, that may be in order to stay indefinitely, but, even otherwise, learning a new environment would have tremendous benefit to your existing Domino work.

Since I'm not a full-time administrator myself, my advice is admittedly a bit thin. It mostly involves my oft-repeated suggestion to try out a reverse-proxy setup for HTTP. That could parlay into use with mail as well, but even just becoming more familiar with a multi-tiered setup and the essentially-standard OpenSSL tools for certificate manipulation would pay dividends.

For developers, there are a great many potential paths to walk, and I think it's extremely useful to at least take one step down each. The use of that reminds me of the "Programming Languages" course I took in college. It involved rapid-fire coverage of C, C++, Scheme, and Prolog - four distinct languages with pretty distinct programming styles (the latter two moreso than the former two), meant to introduce students to the different notions of programming. I don't use Lisp or (lord knows) Prolog in my day-to-day programming life, but having learned a little bit about functional and logic programming at the time has paid dividends over the years.

There are a couple broad topics I can think of that are worth investigating, including ones that we as a community have already been working on.

Source Control

For the most part, the importance of source control has basically sunk in as a "we should do this" sort of thing in the Domino community. However, the inconvenience associated with maintaining an on-disk project means we're not all the best at doing this religiously. And, indeed, we're often hampered by both old habits and Domino's licensing terms that prevent us from doing this properly: there's a strong strain of "multiple people working on the same NSF" in Domino's history, and that doesn't mix well with source control (or XPages, for that matter).

So I recommend not just knowing about source control, but also trying (as much as licensing allows) to use this as an opportunity to switch to a more "traditional" model for app development. I like to think of this type of development as equivalent to working on a desktop app: it would be madness to have multiple people working in the same Xcode/VS/etc. workspace on the same binary build, and then particularly trying to have an end user also use that resultant binary. Instead, it's better to have each developer work separately on distinct non-replica NSFs, ideally on local dev servers, and then deploy the changes as an NTF.

Java EE WARs

Java EE in general is a very worthwhile target for interest, especially because it encompasses so many other possibilities. But I'd like to call attention in particular to the WAR file model, which is roughly equivalent to an NSF for app deployment.

When I first learned web programming, with PHP, it followed the "just toss a bunch of files on disk and visit their paths in a web browser" model, which basically worked, but taught some very bad habits. Domino improves on this a bit, with everything in its place in an NSF, but still suffers from a bit of the same malady: the "filesystem" of design elements is what is exposed to the browser directly, URL-wise. In a more fully-app-based system, though, there's a lot more flexibility about how the incoming URLs map to the underlying code and resources, and Java EE's encapsulated nature plays on this well.

With a WAR file, you have the "app container" advantage of an NSF but also gain the "everything after the app name is mine" benefit of a controlled URL router, and that shift helps conceptualize apps as something of a "mount point" onto an app server. So you may have a blog app that you load up as "/blog" on your server, and then that app can control the traffic beneath it, but is similarly a bit walled off from everything above it. It's the sort of "feel" that settle in with time, and then it feels weird to lack that flexibility.

This model also fits particularly well into app platforms like BlueMix or Heroku.

Testing

This one's difficult for Domino developers! No matter the tooling used for an NSF, being able to dive into automated testing is difficult. You can coax Maven into running some tests, though that will lack an XPages environment. You can test on the XPages side with JUnit4Xpages, though that won't be part of an automated build without more hoops. There's no real equivalent to the type of fully-fleshed-out testing tools available for other platforms now.

So that's lead to unit/integration/whatever testing not even being on the horizon for most Domino developers. That's particularly a shame because it's already something of a chore to get into the testing habit even when the tools do everything they can to help you. It's essentially like brushing your teeth when you're a kid: you do it because you're told to, but it's not something you'd ever do of your own accord.

It's worth it, though! It will probably require working on a fully non-Domino project, but going through the steps of building a fleshed-out system with automated tests along the way is a valuable experience. The goal is to get to the point where it feels weird to not have tests, much in the same way as lacking source control - an untested change should feel like it doesn't "count". It's a big world, too: beyond just small unit tests, there's tooling to run a web app in a test environment and run user interactions, or do the same on mobile-device simulators.

Build Servers

This dovetails with testing, in that one of the primary benefits of having a build server like Jenkins or Bamboo is that they can act as a neutral, clean environment to build the whole app and run the tests, informing you of any errors or failures. This is another thing that's made much more difficult by Domino, though it's possible.

Beyond testing, having a build server reaps benefits with project organization and delivery pipelines. Jenkins's recent updates have focused on that sort of thing, where the tool will help you manage pulling down the latest code from SCM, building it, and then deploying the results to dev or live servers, as desired. Build servers won't necessarily change the way you develop, but they will prove to be a faithful companion, with more uses the more you get comfortable with them.

Unix

Get comfortable with the command line. Other than Microsoft's tooling, which has a very schizophrenic relationship with the CLI, basically all modern development environments swim in the Unix world. Not everything has to be done via CLI - certainly, it's much more comfortable to use SourceTree than to do all git commands manually - but it's hard to find a tutorial that doesn't have at least a few tossed-off terminal commands. Mac- and Linux-based developers are at a distinct advantage with this, but Windows has Cygwin and the ported Linux userland in the next Windows 10 release.

Along this line, it's tremendously useful to have a bit of Linux server admin knowledge. I recommend either spinning up a Linux VM locally (Ubuntu has served me well) or using Linode. Linode in particular is so cheap and convenient that it's very much worth tossing them a couple bucks to run a server for a little while with full confidence that you can melt it into the ground without consequence. The delightful iOS-focused podcast Under the Radar had an episode on this topic a while back that's worth a listen.

You should probably also learn a little about vi.

MVC

Back in the realm of pure programming knowledge, it's extremely valuable to learn about MVC and related concepts. XPages sort of brought this to the Domino world, but it suffered from lack of wholehearted adoption and its legacy baggage made it very difficult to really go full MVC even if you wanted to.

The good news is that essentially every other active framework encourages or enforces this approach. When diving into one, it can be a bit jarring at first - there's a lot of "ugh, can't I just put the code here?" going on - but it falls into place very quickly.

So where to start?

Picking a place to start is a daunting prospect, given the ever-shifting sands of the modern development world.

One of my favorite tutorials for a web framework is Michael Hartl's Rails tutorial, which is kindly available for free online (with options to purchase other forms). It not only provides an up-to-date introduction to Rails development, but also covers a lot of the other ground mentioned above: the command line, Heroku, Git, and test-driven development.

In another direction, I recommend taking a look at Meteor, which is something of an all-in approach to JavaScript-based development. It does an interesting job blurring the lines between client and server and makes a very compelling case for "JavaScript everywhere".

Beyond that, really just learning any development toolchain will help. There's not too much need to sweat picking "the right one" at each step - any knowledge gained in one will help in others, either directly or by giving you some additional perspective. The most important thing, especially now, is learning something.

My Current Model Framework, Part 2: An Example

Feb 21, 2014 3:27 PM

Tags: xpages mvc
  1. My Current Model Framework, Part 1
  2. My Current Model Framework, Part 2: An Example

To go along with the updated release of my Scaffolding project yesterday, I've created an example database created with that template and containing a basic use of my model framework:

model-framework-example.zip

This demonstrates a few things about the framework:

  • Setting up the two classes that go with each object type - the object itself (e.g. model.Department) and the "manager" class that handles fetching objects and collections (e.g. model.DepartmentManager) and adding the collections to faces-config.xml.
  • Tying the manager class to the views in a database using views with a common prefix. In this case, they point to the current database, though I recommend storing the data in another DB.
  • Adding relations between objects. A Person object is tied to a given Department by way of the getDepartment method, while a Department lists its people via the getPeople method.
  • Using the collections on an XPage with xp:viewPanel controls, including showing "columns" that are not in the view - the objects automatically fetch from the document when the column isn't present (which is inefficient but useful sometimes).
  • Attaching a model object to an XPage using a xp:dataContext and using a controller class to save it.

I have a few ideas in mind to improve day-to-day use of the model framework (for example, I may set up proper data sources for individual objects so they can tie into the normal save events), so they may change over time, but this is how I develop most apps today.

My Current Model Framework, Part 1

Nov 17, 2013 12:19 PM

Tags: xpages mvc
  1. My Current Model Framework, Part 1
  2. My Current Model Framework, Part 2: An Example

As I do from time to time, I've recently been taking another stab at a standard model framework for my XPage apps. My latest one has been proving its worth in a couple apps I've been writing lately, and I'd like to go over the general goals and advantages of the way it works.

The framework is focused on a couple main ideas:

  • Low Overhead. The Java language requires a certain amount of overhead to get anything done, but I want to minimize that. The task of defining a new model object consists of two classes: one for the object itself and one for the collection manager (e.g. connecting to Domino views), and each is geared towards only writing the code required to describe the necessities.
  • Embracing XSP. The framework is intended for writing XPages apps, and so I want to make sure it works smoothly with EL and standard controls like inputs, repeats, and view panels.
  • Embracing document databases. Since I use almost entirely a document database for storage, I want to make sure to take advantage of that, and that primarily means flexibility in data modeling. Accordingly, data objects allow arbitrary field access, with any strictures from your model class layered on top of that.
  • Embracing Domino. Since the document database I use is Domino, I want to make sure its peculiar features are retained as well: reader/author fields, arbitrary object storage via MIME, (ideally) RT and attachments, full-text search, categorized views, and so forth.
  • Conceptually simple. Though there's some ugly code involved in parts like the Domino view wrapper, the conceptual layout of the framework is kept very simple with few moving parts, making it easy to recognize the function of each aspect at a glance, whether when building it, when looking at another's code, or when returning to your own code months down the line.

For this post, I'll give an example of a model class, while later I'll go into the collection managers and some real-use examples. This is what a basic model class looks like:

package model;

import java.util.Arrays;
import java.util.Collection;
import java.util.List;

import org.openntf.domino.*;

import frostillicus.model.AbstractDominoModel;
import frostillicus.model.DominoColumnInfo;

public class Request extends AbstractDominoModel {
	private static final long serialVersionUID = 589766180414699322L;

	public Request(final Database database) {
		super(database);
		setValue("Form", "Request");
	}

	public Request(final ViewEntry entry, final List<DominoColumnInfo> columnInfo) {
		super(entry, columnInfo);
	}

	public Request(final Document doc) {
		super(doc);
	}

	@Override
	protected Collection<String> nonSummaryFields() {
		return Arrays.asList(new String[] { "Body" });
	}

}

There's more Java overhead than I'd like, but that's the name of the game. Fortunately, each method has a role and can be used as hooks for differing behavior.

The first constructor is the one used to create a new document of the appropriate type in the provided database, so it sets the form field appropriately - it would also set any other appropriate defaults. The second constructor is used when traversing a view - model objects pull values from view entries when available, but also transparently pass along requests to the document when the requested field isn't in the view. The final constructor wraps an existing document.

The "nonSummaryFields" method is one of the hooks available to define the Domino data model, in this case providing a list of fields that should be flagged as non-summary when written, to help work around Domino limits. I also have hooks, not needed here, for authors/readers/names fields and form-style query/post events.

Conspicuously absent are any field definitions. By default, model objects act much like XPage DominoDocuments, passing setValue and getValue calls on to the underlying document more or less directly. However, the framework allows for hooks here by writing getters and setters in the standard format, and their presence changes the behavior of getValue and setValue. When just a getter is present, the field becomes read-only; when a setter is present, the field can be written, but now does data-type validation. The getters and setters allow for arbitrary validation and additional behavior for specific fields (say, changing related fields when one changes) without having to write out giant blocks of "getFoo() { return foo; }" and "setFoo(String foo) { this.foo = foo; }", and this has been a huge win for me. Even though Eclipse helps with the initial creation, the code still has to exist, and it takes a cognitive toll.

I also use these overriding methods to create relations between models through their collection managers. For example:

public Client getClient() {
	String clientId = (String)getValue("ClientID");
	if(clientId == null || clientId.isEmpty()) {
		return null;
	}
	return (Client)JSFUtil.getClientManager().getValue(clientId);
}

This allows for XPage El bindings like #{task.client.name} - no extra panels with data sources, no inline lookup code.

Next time, I will go into the collection-manager side, which provides fairly high-performance access to views while remaining simple and flexible.

More On "Controller" Classes

Jan 7, 2013 7:39 PM

Tags: xpages mvc java
  1. "Controller" Classes Have Been Helping Me Greatly
  2. More On "Controller" Classes

Since my last post on the matter , I've been using this "controller" class organization method in a couple other projects (including a refresh of the back-end of this blog), and it's proven to be a pretty great way to go about XPages development.

As I mentioned before, the "controller" term comes from the rough equivalent in Rails. Rails is thoroughly MVC based, so a lot of your programming involves creating the UI of a page in HTML with a small sprinkling of Ruby (the "view"), backed by a class that is conceptually tied to it and stores all of the real business logic associated with that specific page (the "controller"). XPages don't have quite this same assumption built in, but the notion of pairing an XPage with a single backing Java class is a solid one. Alternatively, you can think of the "controller" class as being like a stylesheet or client JavaScript file: the HTML page handles the design, and only contains references to functionality defined elsewhere.

What this means in practice is that I've been pushing to eliminate all traces of non-EL bindings in my XSP markup in favor of writing the code in the associated Java class - this includes not only standard page events like beforeRenderResponse , but also anywhere else that Server JavaScript (or Ruby) would normally appear, like value and method bindings. Here's a simple example, from an XPage named Test and its backing class:

Test.xsp
<p><xp:button id="clickMe" value="Click Me">
	<xp:eventHandler event="onclick" submit="true" refreshMode="partial" refreshId="output"
		action="#{pageController.refreshOutput}"/>
</xp:button></p>
	
<p><xp:text id="output" value="#{pageController.output}"/></p>
controller/Test.java
package controller;

import frostillicus.controller.BasicXPageController;
import java.util.Date;

public class Test extends BasicXPageController {
	private static final long serialVersionUID = 1L;

	private String output = "default";
	public String getOutput() { return this.output; }

	public void refreshOutput() {
		this.output = new Date().toString();
	}
}

In a simple case like this, it doesn't buy you much, but imagine a more complicated case, say code to handle post-save document processing and notifications, or complicated code to generate the value for a container control. The separation between the two pays significant dividends when you stop having to worry scrolling through reams of code in the XSP markup when you're working on the page layout, while having a known location for the Java code associated with a page makes it easier to track down functionality. Plus, Server JavaScript is only mildly more expressive than Java, so even the business logic itself stays just about as clean (moving from Ruby to Java, on the other hand, imposes a grotesque LOC penalty).

It sounds strange, but the most important benefit I've derived from this hasn't been from performance or cleanliness (though those are great), but instead the discipline it provides. Now, there's no question where business logic associated with a page should go: in a class that implements XPageController with the same name as the page and put in the "controller" package. If I want pages to share functionality, that's handled either via a common method stored in another class or, as appropriate, via class inheritance. No inline code, no Server JavaScript script libraries. If something is going to be calculated, it's done in Java.

There is one area that's given me a bit of trouble: custom controls. For example, the linksbar on this blog requires computation to generate, but it's not associated with any specific page. For now, I put that in the top-level controller class, but that feels a bit wrong. The same solution also doesn't apply to the code that handles rendering each individual post in the list, which may be on the Home page, the Month page, or individually on the Post page, in which case it also has edit/save/delete actions associated with it. For now, I created a "helper" class that I instantiate (yes, with JavaScript, which is depressing) for each instance. I think the "right" way to do it with the architecture is to create my controls entirely in Java, with renderers and all. That would allow me to handle the properties passed in cleanly, but the code involved with creating those things is ugly as sin. Still, I'll give that a shot later to see if it's worth the tradeoff.

"Controller" Classes Have Been Helping Me Greatly

Dec 26, 2012 4:54 PM

Tags: xpages mvc java
  1. "Controller" Classes Have Been Helping Me Greatly
  2. More On "Controller" Classes

I mentioned a while ago that I've been using "controller"-type classes paired with specific XPages to make my code cleaner. They're not really controllers since they don't actually handle any server direction or page loading, but they do still hook into page events in a way somewhat similar to Rails controller classes. The basic idea is that each XPage gets an object to "back" it - I tie page events like beforePageLoad and afterRenderResponse to methods on the class that implements a standard interface:

package frostillicus.controller;

import java.io.Serializable;
import javax.faces.event.PhaseEvent;

public interface XPageController extends Serializable {
	public void beforePageLoad() throws Exception;
	public void afterPageLoad() throws Exception;

	public void afterRestoreView(PhaseEvent event) throws Exception;

	public void beforeRenderResponse(PhaseEvent event) throws Exception;
	public void afterRenderResponse(PhaseEvent event) throws Exception;
}

I have a basic stub class to implement that as well as an abstract class for "document-based" pages:

package frostillicus.controller;

import iksg.JSFUtil;

import javax.faces.context.FacesContext;

import com.ibm.xsp.extlib.util.ExtLibUtil;
import com.ibm.xsp.model.domino.wrapped.DominoDocument;

public class BasicDocumentController extends BasicXPageController implements DocumentController {
	private static final long serialVersionUID = 1L;

	public void queryNewDocument() throws Exception { }
	public void postNewDocument() throws Exception { }
	public void queryOpenDocument() throws Exception { }
	public void postOpenDocument() throws Exception { }
	public void querySaveDocument() throws Exception { }
	public void postSaveDocument() throws Exception { }

	public String save() throws Exception {
		DominoDocument doc = this.getDoc();
		boolean isNewNote = doc.isNewNote();
		if(doc.save()) {
			JSFUtil.addMessage("confirmation", doc.getValue("Form") + " " + (isNewNote ? "created" : "updated") + " successfully.");
			return "xsp-success";
		} else {
			JSFUtil.addMessage("error", "Save failed");
			return "xsp-failure";
		}
	}
	public String cancel() throws Exception {
		return "xsp-cancel";
	}
	public String delete() throws Exception {
		DominoDocument doc = this.getDoc();
		String formName = (String)doc.getValue("Form");
		doc.getDocument(true).remove(true);
		JSFUtil.addMessage("confirmation", formName + " deleted.");
		return "xsp-success";
	}

	public String getDocumentId() {
		try {
			return this.getDoc().getDocument().getUniversalID();
		} catch(Exception e) { return ""; }
	}

	public boolean isEditable() { return this.getDoc().isEditable(); }

	protected DominoDocument getDoc() {
		return (DominoDocument)ExtLibUtil.resolveVariable(FacesContext.getCurrentInstance(), "doc");
	}
}

I took it all one step further in the direction "convention over configuration" as well: I created a ViewHandler that looks for a class in the "controller" package with the same name as the current page's Java class (e.g. "/Some_Page.xsp" → "controller.Some_Page") - if it finds one, it instantiates it; otherwise, it uses the basic stub implementation. Once it has the class created, it plunks it into the viewScope and creates some MethodBindings to tie beforeRenderResponse, afterRenderResponse, and afterRestoreView to the object without having to have that code in the XPage (it proved necessary to still include code in the XPage for the before/after page-load and document-related events).

So on its own, the setup I have above doesn't necessarily buy you much. You save a bit of repetitive code for standard CRUD pages when using the document-controller class, but that's about it. The real value for me so far has been a clarification of what goes where. Previously, if I wanted to run some code on page load, or attached to a button, or to set a viewScope value, or so forth, it could go anywhere: in a page event, in a button action, in a dataContext, in a this.value property for a xp:repeat, or any number of other places. Now, if I want to evaluate something, it's going to happen in one place: the controller class. So if I have a bit of information that needs recalculating (say, the total cost of a shopping cart), I make a getTotalCost() method on the controller and set the value in the XPage to pageController.totalCost. Similarly, if I need to set some special values on a document on load or save, I have a clear, standard way to do it:

package controller;

import com.ibm.xsp.model.domino.wrapped.DominoDocument;
import frostillicus.controller.BasicDocumentController;

public class Projects_Contact extends BasicDocumentController {
	private static final long serialVersionUID = 1L;

	@Override
	public String save() throws Exception {
		DominoDocument doc = this.getDoc();
		doc.setValue("FullName", ("CN=" + doc.getValue("FirstName") + " " + doc.getValue("LastName")).trim() + "/O=IKSGClients");

		return super.save();
	}

	@Override
	public void postNewDocument() throws Exception {
		super.postNewDocument();

		DominoDocument doc = this.getDoc();
		doc.setValue("Type", "Person");
		doc.setValue("MailSystem", "5");
	}
}

It sounds like a small thing - after all, who cares if you have some SSJS in the postNewDocument event on an XPage? And, besides, isn't that where it's supposed to go? Well, sure, when you only have a small amount of code, doing it inline works fine. However, as I've been running into for a while, the flexibility of XPages makes them particularly vulnerable to becoming tangled blobs of repeated and messy code. By creating a clean, strict system from the start, I've made it so that the separation is never muddied: the XPage is about how things appear and the controller class is about how those things get to the XPage in the first place.

We'll see how it holds up as I use it more, but so far this "controller"-class method strikes a good balance between code cleanliness without getting too crazy on the backing framework (as opposed so some of the "model" systems I've tried making).

My Current Data-Source Musings

Nov 26, 2012 6:41 PM

Tags: mvc

My quest to find the ideal way to access data in XPages continues. Each time I make a new project, I come up with a slight variation on my previous theme, improving in one or more of the various conflicting factors involved. The problem, as always, is that the standard xp:dominoView and xp:dominoDocument data sources alone are insufficient to properly separate business logic from presentation. On the other hand, accessing view data in a way that is both efficient and flexible via just the standard Java API is a non-trivial prospect. The advantage to using xp:dominoView in particular is that IBM already did an incredible amount of work making performance passable when dealing with cases of resorting, key-lookups, FT searches, response hierarchies, and the like.

The ideal method would build upon the efficient-access and serialization work already done while retaining that Rails-like collection-fetching and relation code from my forum attempts. Accordingly, I'm trying something a bit different with my latest project: I extended xp:dominoView to be able to take another parameter, modelClass, that is the class name of the objects it should return, rather than just DominoViewEntry. In turn, the objects I'm using wrap the real DominoViewEntry and allow for transparent access to the document (as needed, e.g. when a requested field is not a column) and the usual benefits of writing Java classes: namely, I can write code to handle field-change side effects without having to care about the logic behind that in the XPage itself.

This is sort of a half-formed idea so far, and I'm not 100% comfortable with it, but I think it's promising. Ideally, there wouldn't be any data sources written into the XPage itself at all (since it should be just a View), but that's sort of a necessity given the current state of things. This should be a reasonable next-best thing, and maybe a stepping stone to a new, faster, and cleaner data-access method.

The end goal of all this is to make it so that creating a new data "type" is very straightforward but still more declarative and structured than just making a new form name in the xp:dominoDocument data source. There should be classes to go with each type, but those classes should be very simple and easy to write. The quest continues!

A Prototype In-App Messaging System for XPages

Jul 9, 2012 2:35 PM

Tags: xpages java

Sven Hasselbach's post about ApplicationListeners the other day and the notion of post-MVC methods of app architecture got me thinking about the notion of generic messaging/events inside an XPages app. The first example that comes to mind is a project-tracking database where there may be events like "new task created" or "delivery ready for review" - business logic stuff where the fact that it's a document being modified is an implementation detail. In my actual project-tracking database at work, I implemented this by creating "Notification Stub" documents with information about the event and having a scheduled agent process them. That's a fine Notes-y way to do it (and has its own advantages, like having another cluster member send the emails), but I want a way for the running XPages app to notice and react to these things without the immediate code knowing about every single concern.

I did a quick search and didn't find any such capability in basic JSF or XPages, but I realized it wouldn't be terribly difficult to implement it myself: have an application-scoped bean (maybe one per scope, if it's useful) with a methods to dispatch events and declare listeners. Then, other beans could attach themselves as listeners and the normal code would only be responsible for sending out its messages, to be received by zero or more unknown objects:

public interface XPagesEventDispatcher {
	public void dispatchEvent(XPagesEvent event);
	public void addListener(XPagesEventListener listener);
}

The actual event objects would be similarly simple: just a name and an arbitrary array of objects:

public interface XPagesEvent {
	public String getEventName();
	public Object[] getEventPayload();
}

Finally, listeners are the simplest of all, being responsible only for providing a method to handle an XPagesEvent (EventListener is the standard "tagging" interface for this kind of thing):

public interface XPagesEventListener extends EventListener {
	public void receiveEvent(XPagesEvent event);
}

However, after creating my gloriously clean set of interfaces, I ran into the ugly marsh of reality: application-scoped beans only exist after first use. That works fine for the dispatcher itself, but not for listener beans, the whole point of which is to not be referenced in code elsewhere. After a bit of searching, I came up with a solution that may work. When defining a managed bean, you can set properties, and those properties can be Lists. So, rather than creating two beans, I define just the dispatcher bean, but provide it a list of class names of listener beans to create at startup:

<managed-bean>
	<managed-bean-name>applicationDispatcher</managed-bean-name>
	<managed-bean-class>frostillicus.event.SimpleEventDispatcher</managed-bean-class>
	<managed-bean-scope>application</managed-bean-scope>
	<managed-property>
		<property-name>listenerClasses</property-name>
		<list-entries>
			<value-class>java.lang.String</value-class>
			<value>frostillicus.Messenger</value>
		</list-entries>
	</managed-property>
</managed-bean>

I gave my dispatcher implementation class a convenience methods to take a string and 0 or more objects and dispatch the event, so it could be used in code like:

applicationDispatcher.dispatch("home page loaded")
applicationDispatcher.dispatch("delivery submitted", [delivery.getUniversalID()])

So far, this setup is working pretty well. I plan to toy with it a bit and then try it out in proper use. Provided it works well (and it doesn't turn out that this capability already exists and I just missed it), I'll turn it into a project on GitHub/OpenNTF. As long as it keeps working, I'm excited about the prospects, especially when combined with XPages Threads and Jobs for non-blocking event handling.

XPages MVC: Experiment II, Part 4

Jun 4, 2012 7:57 PM

Tags: xpages mvc
  1. XPages MVC: Experiment I
  2. XPages MVC: Experiment II, Part 1
  3. XPages MVC: Experiment II, Part 2
  4. XPages MVC: Experiment II, Part 3
  5. XPages MVC: Experiment II, Part 4

To finish up my series on the infrastructure of my guild forums app, I'd like to mention a couple of the down sides I see with its current implementation, which I'd generally want to fix or avoid if re-implementing it today.

Roll-Your-Own

One of the strengths of this kind of MVC setup is that it works to separate the front-end code from the data source. It would be (relatively) easy for me to replace the model and collection classes with versions that use a SQL database or non-Notes document storage should I so choose. That's a double-edged sword, though: because I'm not using any of the built-in data sources and controls with Domino knowledge, I had to do everything myself. This means a loss of both some nice UI features - like the rich text editor's ability to upload images inline - and the XPage data sources' persistence and caching features.

The collection code deals with View and ViewEntryCollection classes directly, but they can't be serialized, so I had to write my own methods to detect when the object is no longer valid (say, when doing a partial refresh) and re-fetch the collection. This was good in the sense that I learned more about what is and is not efficient in Domino. For example, getNthEntry(...) on a ViewEntryCollection grabbed via getAllEntriesByKey(...) is fast. Conversely, while retrieving data in a view is usually significantly faster than getting the same data from a document, there's a point where the view index size is large enough that, provided you're fetching only a few documents at a time, it's better to use the document. With a lot of work (and a LOT of collection caching), I ended up with something that's quite fast... but since I wrote all the code myself as part of a side project, it was also pretty bug-prone for the first couple weeks after deployment.

Maybe I'd be able to find ways to piggyback more on the built-in functionality if I re-did it, but, as it stands, it's kind of hairy.

Inconsistent Separation

This isn't a TERRIBLE problem, but I'm kind of annoyed with some of the inconsistent choices I made about where DB-specific code goes. For example, collection managers have very little Domino-specific code... except when they need to know about sorting and searching. Similarly, almost all of the code in the model objects deals with pure Java objects and the clean collections API... except the save() methods, which are big blobs of Domino API work. That makes some sense, but the model classes don't handle creation from the database - that's in the collection classes. Like I said, it's not the end of the world, and it's consistent within its own madness, but there are some weird aspects and internal leaks I wouldn't mind cleaning up.

Lack of Data Sources

This is sort of the flip side to my first problem. All of my collection managers are just objects and the collections are just Lists. This is good in the sense that these things work well in Server JavaScript and with all of the built-in UI elements, but it'd be really nice to write my own custom data sources so I can explicitly declare what data I want on the page - using a xp:dataContext or bit of JavaScript in a value property just feels "dirty", like I'm not embracing it completely. However, this problem isn't as much an architectural one as it is a lack of education - I haven't bothered to learn to write my own data sources yet (even though I expect it's more or less straightforward, for Java), so I could remedy that easily enough.

No Proper Controller

This is just another manifestation of the root cause that has me looking into all this MVC stuff to begin with. Even though my collections and models are better than dealing with raw Domino objects, there's still too much of a tie between the UI and the back-end representation, as well as the requisite dependence on the Domino HTTP stack to handle routing requests. Of course, I'm still working on the correct solution to this.

 

Overall, my structure as written has been serving me well. New data elements are pretty easy to set up - I wouldn't mind not having to write three classes per, but hey, it's Java - and working with them is a breeze. Though it took a while to get everything working, now that it is, I can make tons of UI changes without worrying much about the actual data representation. I can change the way data is stored or add on-load or on-save computation beyond the capabilities of Formula language without changing anything in the XPages themselves. So in those senses, my forum back-end code is a huge step up from doing it directly xp:dominoDocuments, but it still doesn't feel completely "right".

XPages MVC: Experiment II, Part 3

May 28, 2012 8:00 PM

Tags: xpages mvc
  1. XPages MVC: Experiment I
  2. XPages MVC: Experiment II, Part 1
  3. XPages MVC: Experiment II, Part 2
  4. XPages MVC: Experiment II, Part 3
  5. XPages MVC: Experiment II, Part 4

Continuing on from my last post, I'd like to go over a couple specifics about how I handle fetching appropriate collections of objects from a view and a couple areas where I saved myself some programming hassle.

As I mentioned before, both the "manager" and "collection" classes inherit from abstract classes that handle a lot of the dirty work. The AbstractCollectionManager class is by far the smaller of the two, containing mostly convenience methods and a couple overloaded methods for generating the collection objects. The central one is pretty straightforward:

protected AbstractDominoList<E> createCollection( String sortBy, Object restrictTo, String searchQuery, int preCache, boolean singleEntry) { AbstractDominoList<E> collection = this.createCollection(); collection.setSortBy(sortBy); collection.setRestrictTo(restrictTo); collection.setSearchQuery(searchQuery); collection.setSingleEntry(singleEntry); collection.preCache(preCache); return collection; }

The first three properties actually determine the nature of the collection, while the last two set some optimization bits when the code knows ahead of time about how many elements it expects (such as doing a by-id lookup or a "recent news" list that will only ever display 5 entries). The createCollection call at the start is overridden method in each class that returns a basic collection object to avoid Java visibility issues.

The AbstractDominoList class is much longer; I won't go into too much detail, but a couple parts are pertinent. As I've mentioned, it implements the List interface (via extending AbstractList), which means it just needs to provide get and size methods. The way these work is by checking to see if it houses a valid cached ViewEntryCollection and, if it does, translating the method call to get the indexed entry as an object or the size. If the collection hasn't been fetched yet or if it's no longer valid (say, if it was recycled), it uses the view-location information from the implementing class and any parameters on the object to construct another one.

First, it fetches and configures the view properly:

Database database = this.getDatabase(); View view = database.getView(this.getViewName()); view.refresh(); String sortColumn = this.sortBy == null ? null : this.sortBy.toLowerCase().endsWith("-desc") ? this.sortBy.substring(0, this.sortBy.length()-5) : this.sortBy; boolean sortAscending = this.sortBy == null || !this.sortBy.toLowerCase().endsWith("-desc"); if(this.searchQuery != null) { view.FTSearchSorted(this.searchQuery, 0, sortColumn, sortAscending, false, false, false); } else if(this.sortBy != null && !this.sortBy.equals(this.getDefaultSort())) { view.resortView(sortColumn, sortAscending); } else { view.resortView(); }

Then, it uses code similar to this to store the collection in its cache (I say "similar to" because there are additional branches and methods, but this is the idea):

if(this.restrictTo == null) { ViewEntryCollection collection = view.getAllEntries(); this.cachedEntryCollection = new DominoEntryCollectionWrapper(collection, view); } else { ViewEntryCollection collection = view.getAllEntriesByKey(this.restrictTo, true); this.cachedEntryCollection = new DominoEntryCollectionWrapper(collection, view); }

The other part of the get method is translating the ViewEntry into an object, which is done via a method similar to this monster:

protected E createObjectFromViewEntry(ViewEntry entry) throws Exception { Class currentClass = this.getClass(); Class modelClass = Class.forName(JSFUtil.strLeftBack(currentClass.getName(), ".") + "." + JSFUtil.strLeftBack(currentClass.getSimpleName(), "List")); // Create an instance of our model object E object = (E)modelClass.newInstance(); object.setUniversalId(entry.getUniversalID()); object.setDocExists(true); // Loop through the declared columns and extract the values from the Entry Method[] methods = modelClass.getMethods(); String[] columns = this.getColumnFields(); Vector values = entry.getColumnValues(); for(Method method : methods) { for(int i = 0; i < columns.length; i++) { if(values != null && i < values.size() && columns[i].length() > 0 && method.getName().equals("set" + columns[i].substring(0, 1).toUpperCase() + columns[i].substring(1))) { String fieldType = method.getGenericParameterTypes()[0].toString(); if(fieldType.equals("int")) { if(values.get(i) instanceof Double) { method.invoke(object, ((Double)values.get(i)).intValue()); } } else if(fieldType.equals("class java.lang.String")) { method.invoke(object, values.get(i).toString()); } else if(fieldType.equals("java.util.List<java.lang.String>")) { method.invoke(object, JSFUtil.toStringList(values.get(i))); } else if(fieldType.equals("class java.util.Date")) { if(values.get(i) instanceof DateTime) { method.invoke(object, ((DateTime)values.get(i)).toJavaDate()); } } else if(fieldType.equals("boolean")) { if(values.get(i) instanceof Double) { method.invoke(object, ((Double)values.get(i)).intValue() == 1); } else { method.invoke(object, values.get(i).toString().equals("Yes")); } } else if(fieldType.equals("double")) { if(values.get(i) instanceof Double) { method.invoke(object, (Double)values.get(i)); } } else if(fieldType.equals("java.util.List<java.lang.Integer>")) { method.invoke(object, JSFUtil.toIntegerList(values.get(i))); } else { System.out.println("!! unknown type " + fieldType); } } } } return object; }

Yes, I am aware that it's O(m*n). Yes, I am aware I could move values != null outside the loop. Yes, I am aware I do the same string manipulation on each column name many times. Shut up.

Anyway, that code finds the model class and loops through its methods, looking for set* methods for each column (the column list being specified in the implementation class). When it finds one that matches, it checks the data type of the property, and invokes the method with an appropriately-cast or -converted column value. The result is that the implementation class only needs to provide the code needed for fetching any data that isn't in the view, such as rich text. At one point, I think I had it so that it read the view column titles and used those as field names, but I stopped doing that for some reason... maybe it was too slow, even for this method.

In any event, next time, I'll go over some of the disadvantages that I've found with this method that don't involve big-O notation.

XPages MVC: Experiment II, Part 2

May 24, 2012 8:43 PM

Tags: xpages mvc
  1. XPages MVC: Experiment I
  2. XPages MVC: Experiment II, Part 1
  3. XPages MVC: Experiment II, Part 2
  4. XPages MVC: Experiment II, Part 3
  5. XPages MVC: Experiment II, Part 4

Continuing on from my last post, I'd like to go over the collection and model structures I used in my guild-forums app.

I set up the data in the Notes DB in a very SQL-ish way (in part because I considered not using Domino initially). Each Form has a single equivalent view (for the most part), which is uncategorized and sorted by default by an ID column. There is one column for each column that I want to sort by and, in the case of simple documents, one for each applicable field, containing only the field name; additionally, I created $-named "utility" columns for when I wanted to sort by a given bit of computed data or have a second secondary sort column for a given column. For the most part, the data is normalized, but I had to give in to reality in a few situations, for things like sorting Topics by the their latest Post dates.

For collection classes in Java, each collection class file contains one public class for the "manager" and one list implementation class, both of which extend from Abstract versions, so the actual implementations may have very little code. For example, here is the "Groups" Java file (minus some Serialization details):

public class Groups extends AbstractCollectionManager<Group> { protected AbstractDominoList<Group> createCollection() { return new GroupList(); } } class GroupList extends AbstractDominoList<Group> { public static final String BY_ID = "$GroupID"; public static final String BY_NAME = "$ListName"; public static final String BY_CATEGORY = "ListCategory"; public static final String DEFAULT_SORT = GroupList.BY_ID; protected String[] getColumnFields() { return new String[] { "id", "name", "category", "description" }; } protected String getViewName() { return "Player Groups"; } protected String getDatabasePath() { return "wownames.nsf"; } protected Group createObjectFromViewEntry(DominoEntryWrapper entry) throws Exception { Group group = super.createObjectFromViewEntry(entry); Document groupDoc = entry.getDocument(); group.setMemberNames((List>String<)groupDoc.getItemValue("Members")); return group; } }

The actual heavy lifting is done by the parent class, letting the specific classes be composed almost entirely of descriptions of where to find the view, what columns map to model fields, and any extra code that is involved with that object mapping. Other classes are larger, but follow the same pattern. For example, here's a snippet from the manager for Posts:

public class Posts extends AbstractCollectionManager<Post> { // [-snip-] public List<Post> getPostsForTag(String tag) throws Exception { return this.search("[Tags]=" + tag); } public List<Post> getPostsForUser(String user) throws Exception { return this.getCollection(PostList.BY_CREATED_BY, user); } public List<Post> getScreenshotPosts() throws Exception { return this.search("[ScreenshotURL] is present", PostList.BY_DATETIME_DESC); } }

The parent abstract class provides utility methods like getCollection and search that take appropriate arguments for sorting, searching (I love you, FTSearchSorted), and keys.

As for the individual model objects, they start out as your typical bean: a bunch of fields with simple data types and straightforward getters and setters. Additionally, they contain "relational" methods to fetch associated other objects, such as the Forum that contains the Topic, or a list of the Posts within it. Finally, the model class is responsible for actually saving back to the Domino database, as well as taking care of any business rules, like making sure that the Topic document is updated with a new Post's date/time.

The result is that a new object type is easy (-ish) to set up, with all the code being written contributing specifically towards the task at hand. And, since the nuts and bolts of XPages are Lists and Maps, working with these objects is smooth, taking advantage of Expression Language's cleanliness and Server JavaScript's bean-notation support in the UI and Java's clunky-but-still-useful collections features.

Next time, I'll go over some of the specifics of how I deal with fetching collections and individual records, use reflection to save me some coding hassle, and other tidbits from the giant messes of code that make up my abstract classes. After that, I think I'll make a fourth post to detail some down sides to my approach as implemented.

XPages MVC: Experiment II, Part 1

May 23, 2012 3:38 PM

Tags: xpages mvc
  1. XPages MVC: Experiment I
  2. XPages MVC: Experiment II, Part 1
  3. XPages MVC: Experiment II, Part 2
  4. XPages MVC: Experiment II, Part 3
  5. XPages MVC: Experiment II, Part 4

As I've mentioned a couple times, my largest XPages app to date is the site I did for my guild's forums. The forum part itself isn't particularly amazing, considering it's almost harder to NOT write a forum in Domino than it is to write one, but it gave me a chance to try my hand at abstracting data access away from the XPage.

I put "Part 1" in the title because I figure it will be best to break the topic up into at least three parts: the overall idea and the result, the structure of the model and collection classes, and some nitty-gritty bits about how I get the data out of the back-end classes.

My first draft of this site was straightforward, from an XPages perspective: I mapped XPages to the corresponding Forms and used a bit of Server JavaScript here and there to pull in bits of support information. This worked pretty well at first, but quickly ran into code and speed scalability issues. Just looking at a "Topic" page, the following data needs to be looked up from outside the Topic document itself:

  • The name and ID of the parent Forum document
  • The name and ID of the parent Forum's parent Category document
  • Whether or not the topic is marked as a "favorite" for the current user, which is stored in a prefs doc for the user
  • The collection of Post documents for the Topic
  • For each Post document:
    • Summary and rich text data from the document
    • The display name, title, and signature of the posting user, which are stored in their Person document
    • The number of posts from the posting user that are visible to the current user, updated live (heh)
    • Whether or not the current user has rights to edit the Post document
  • Any surrounding page elements, such as the count of unread topics from the front page, the "who's online" list, etc.

As you can imagine, writing each one of those as inline SSJS turned into a hairy, slow mess. And that's just for the Topic page! That's leaving out the other pages, such as the forum index, which has to show a tree hierarchy of Categories and Forums with up-to-date Topic and Post counts.

I tried wrapping these things up into SSJS functions in a script library, which helped a LITTLE, but it still sucked. The clear solution was to wrap these all in proper model objects and, after a number of revisions, I came up with a pretty good solution.

Conceptually, the back-end structure consists of:

  • Collection Managers. These are analogous to Views, in that there's one per document type and they communicate with a similarly-named uncategorized View with many sortable columns. I write methods in these managers to allow for returning an appropriate collection for the circumstance (e.g. getPostsForForumId(...) and getPostsForTag(...)), as well as generic methods for getting arbitrary collections for a given key, sort column, and FT search query.
  • Collections. These are analogous (and often wrap) ViewEntryCollections. They implement the List interface to allow for use in xp:repeats and handle efficiently getting the needed model objects and re-fetching the View and Collection if they're recycled away.
  • Model Objects. These are what you'd think: Post, Topic, etc.. These provide getters/setters for the fields of the document, getters for associated collections (e.g. topic.getPosts()), and handle saving the document back to the DB.

The net result is that pages went from having giant blocks of lookup code (or SSJS function calls that pointed to giant block functions) to nice little EL expressions like #{category.forums} and #{forum.latestPost.createdBy}. As importantly, this allowed for a lot of caching to make these relational operations speedy when dealing with live data in a Domino DB.

Next time, I'll go into some specifics about how I structured the collection and model classes.

XPages MVC: Experiment I

May 22, 2012 5:28 PM

Tags: xpages mvc
  1. XPages MVC: Experiment I
  2. XPages MVC: Experiment II, Part 1
  3. XPages MVC: Experiment II, Part 2
  4. XPages MVC: Experiment II, Part 3
  5. XPages MVC: Experiment II, Part 4

Now that I've had a bit of time, I've started trying out some ideas for new ways to do XPage development. Specifically, I'm trying out the "XPages as the Controller" setup I pondered last time. The general goal is this:

  • There would be one XPage for each zone of concern (I don't know the right terminology): Posts.xsp, Users.xsp, etc.
  • The XPage itself would have almost nothing on it - it would exist as a trigger for the server to load an appropriate Controller class, which would in turn handle the output, usually by picking a custom control by name.
  • The custom control will handle all the appearance logic... and ONLY the appearance logic. The Controller will feed in any variables and data sources that are needed, and then the custom control will just reference those values entirely via EL.
  • The Controller will marshal will either create Domino View data sources directly or will use some wrapper Model class, in order to allow the XSP environment to handle efficiency and serialization.

So far, I've made some interesting progress. I originally set out to use a ViewHandler class to find the action and feed in the variables, but I ran into trouble getting to the viewScope or an equivalent in time for on-page-load value bindings to access them - either I was too early (before createView) or too late (after createView). What I really wanted was the beforePageLoad event, so I did just that: I created a view-scoped managed bean called "controller" that has some code to check what XPage is being loaded and to find a Controller class of the equivalent name (e.g. "Posts.xsp" → "xsp.controller.Posts"). Then, I do beforePageLoad="#{controller.control}" (I apologize for the names - I may change it to "router"), and it works nicely. The controller manager thing loads the class, asks it to figure out the current action ("list", "tag", etc.), and then tries to call a method by that name via reflection.

It sounds a bit weird, and the implementation is sort of half-baked, but the result is that you end up with a class with a structure like this (with actual implementation stuff removed):

public class Posts extends AbstractController { public void list() { ... } public void show() { ... } public void tag() { ... } }

Each of those methods corresponds to an action that can either be implied (like "list" being the default for Posts.xsp, established via other code) or derived from the URL, like "/Posts.xsp/show/1" to show the post with ID "1". The code in the method would fetch the appropriate object or collection and attach it to the XPage (as a dynamically-added data source, for example). Then, the XPage would load up an appropriate custom control for the action (for example, "pages_list" or "pages_show"), which would know what values to look for in the view scope.

I'm going to keep working on this, but I can think of a couple potential muddy areas:

  • I'm not sure how real data-backed custom controls (e.g. a list of posts by month in the sidebar) work into this. You can add beforePageLoad code in a custom control, but the view root is still the main page you're loading, so it's not as clean.
  • I'm not sure if it's worth trying to shuttle data source actions through the controller (like "save" for a document) or best to reference the methods in the model from the XPage directly. Since the environment is already "impure", I don't know how much bending-over-backwards is warranted.
  • The whole thing may be an exercise in cramming Rails-isms where they don't fit, like when someone familiar with SQL first starts programming for Domino. Just because something works doesn't mean it's correct for the platform.

I'm sure there will be more sticking points, too. Overall, though, this feels mostly good, or at least conceptually better than the other development paradigms I've tried with XPages. As the post title indicates, though, I expect this to be experiment number 1 of many.

More Musing About Controllers

May 13, 2012 8:45 AM

Tags: xpages mvc

I've been thinking more about this MVC thing, thanks to re-learning Rails. I'm fairly convinced that moving as much code as possible out of the XPage and into wrapper objects is a big improvement (convinced enough that it feels silly that wasn't doing it already), but it still feels not quite right.

Two parts of the MVC trinity are straightforward in modern Domino development: Forms and wrapper objects are the Model and XPages are the View. The Controller is where things get muddy. Off the top of my head, routing and controlling are handled by:

  • Domino's built-in URL handling
  • "Display XPage Instead" on Forms and Views
  • Web rules in the Directory
  • navigationRules on the XPage itself
  • The XPage's names themselves, as used in URLs
  • Anything fancy you do with document mode switching and multi-page XPages

It's kind of a mess, and very few of these are really programmable. You can do a lot with web rules, but the application itself doesn't know about them, so you end up with code that's coupled with a specific server configuration - very unsafe. Domino's built-in URLs work well enough for basic operations - show a view, open a document, edit a document - but don't understand more complex concepts, like a shopping cart "Checkout" (yes, I just re-did the depot tutorial from the Rails book).

Honestly, it might be enough to move the action code out of the XPage and into the wrapper classes. That's a big step in readability, maintainability, flexibility, and error prevention, and most sites get along just fine with a lot less. Still, it doesn't feel right.

I don't really have a better option, though. This morning, I've been kicking the idea around that you could treat the XPage itself as the Controller, sacrificing URL and code cleanliness for some power. In this setup, a main XPage would correspond to an aspect of your application (say, "AdminSite") or management tools for a data type (say, "Posts"). You would construct your URLs by chaining path info past the ".xsp" part and then use the main page (or, ideally, a shared library across all pages) to figure out what the real request is. You'd end up with an extra level to your URLs, like "Posts.xsp/search?q=foo". It's horrifyingly ugly, especially right after "whatever.nsf", but it makes a sort of sense: ":application/:controller/:action?:parameters". The alternative now is to have multiple XPages that each deal with very similar things and, if you have a large app, name them in a structured way anyway, like "Posts_List.xsp" and "Posts_Search.xsp?q=foo".

You'd have to put the individual action code in custom controls with loaded parameters to match the action - this would add yet another type of element to the custom controls list ("subforms", actual custom controls, and now the View component of each action), but it may be a necessary evil. The Extension Library has something like this, though not exact, in its "Core_DynamicPage" demo.

The whole idea seems a bit half-baked at the moment but worth investigation, so I'll have to tinker with it a bit when I have some time.

Separation of Concerns

May 9, 2012 7:53 PM

Tags: xpages

A while back, I wondered about the right way to write XPages. Things have changed a bit since then - Domino has gotten a bit better, the Extension Library exists and is great, and I'm a bit more adept with the environment. The forum app, which I should probably write a post explaining one day, came together kind of like how I mentioned there - Java classes to wrap all of the Domino access, which dramatically reduced the amount of code in the XPages themselves.

Still, I'm not sure I'm doing it right most of the time.

I've never done extensive work in a proper MVC environment, but I've gotten a bit of a taste for it on the occasions that I've tinkered with Rails. I feel like the components are there in XPages, but none of the official documentation encourages using any of them, nor does Designer encourage programming that way. You can do it in bits, though - for event handlers, you can point the action to a Java method via EL (like "#{someObject.doSomething}", if I recall correctly), and the "Next page" things in the XPage's properties panel use this underlying xp:navigationRule architecture, which is probably a mix of View and Controller, but it's an interesting idea.

I may have been on to something with the forum app, but I had to create a very aggressive caching system to make it at all practical. The end result was good - very little code in the XPages themselves - but there was more "framework" necessary than I think I'm comfortable with. Still, that discomfort isn't as bad as seeing a lot of business logic in an XPage (yes, I am aware that my main side project is a way to make it all the easier to do this).

I feel like the "right" way to do it would be to really restrict the XPage itself to be a page-layout engine, concerned only with declaring widgets and connecting them to actions on data. However, you can't get very far with just the components you're given, since you're eventually going to want to, say, set a couple values on a document programmatically. You can sort of stumble along with some of the built-in actions, but that's not necessarily any cleaner... just more XML-y.

Maybe the correct thing to do would be to make it easier to expand on Controllers or Models (my MVC knowledge gets shaky here) in code in Designer so that you could, for example, define a publish action for a blog post document that handles setting the published date, changing the status, and saving it. That way, the XPage would be referencing the actual job to be done - publishing the post - while the back-end code would handle the implementation details of replaceItemValue and so forth. Maybe it'd be best to treat Forms like an object definition - a combination of instance members (fields) and actions to be performed. They're already on their way to being mostly structural support for XPages apps.

I may try to structure my next XPage like this, perhaps writing wrapper objects that take Documents or DominoDocuments in their constructor and providing all the actions that I'd want to perform on them. I could set that up as a dataContext perhaps, and then use its methods instead of adding inline code on the XPage. It's worth some experimenting, I think.

Learning the "Right" Way to Write XPages

Nov 28, 2010 1:52 PM

Tags: xpages

XPages are a tough nut to crack. Thanks to IBM's apparent disinterest in people using their software (which is a big topic in itself), it's an uphill battle to figure out what to do with them. For simple applications, they're overkill - loading a Java framework stack that seems to go on for miles has some noticeable performance penalties. But for larger applications, how are you supposed to write them? The main tutorials I've run across usually involve replicating basic form/view functionality or porting an example Notes application over. That's great, but I already know how to make apps with forms and views.

A couple sources, such as the excellent Mindoo blog and the well-produced Notes in 9 give hints at the "other" way to do XPages development, the way involving pure Java classes and scoped beans. From what I've gathered in my development, this is the "Right" way to do it, but, unfortunately, those blog posts and videos are about the extent of what I've found. I imagine that I could learn more if I hired consultants or attended LUGs in various parts of the world, but those seem like pretty ridiculous solutions to the problem.

Since XPages are an application of JSF, I could run off in that direction, but there are severe problems with that plan, not the least of which is that the XPage tag library is very distinct from other JSF applications and so reading any of those books would be analogous to learning MFC by reading a book about Qt. Same language and same general concepts, sure, but it's not the same environment.

This ambiguity manifests in practical problems. I'm writing a forum at the moment, and I'm wondering what the best way to do it is. On a topic page, for example, I have a topic ID and I want to display a paginated view of the posts in that topic. The straightforward and functional way to do it is to use a <xp:dominoView/> data source with a categoryFilter and feed that to a <xp:repeat/> control. That works and initially is pretty clean - showing data from a view is easy. But then I want to show the author's preferred alias instead of their username, so I put in a custom control with code to look that up. Oh, and some code to determine and show the author's current avatar image. And some code to determine if the current user can edit or delete the post. And some code to look for previous versions of the post to show an edit history. And some code to add or remove the topic from the list of the user's favorite topics. And so on and so on.

None of that code is particularly crazy, and I've moved a lot of it into custom controls to keep it nice and clean, but, at this point, I've got all kinds of business logic performing actions and converting <xp:dominoView>s and <xp:dominoDocument/>s into topic lists and posts. The actual XPage itself is supposed to be the View part of a MVC setup, right? Instead, now I have this View/Controller hybrid with a dash of Model when I have to make a secondary data source to get the rich text of the post. That can't be right. Instead, I should probably write some Java classes called Topic, Post, and Author and get all that crap out of the XPage.

And therein lies the problem. I started clean with a fancy new framework, but I've been largely left to my own devices when it comes to figuring out how to use it. Plus, I've not even totally convinced that Java-ifying everything is actually the right way. Once I step away from the pre-provided Data Sources, I'm suddenly on the hook for handling efficient pagination and cacheing myself, plus "smaller" things like getting the rich text data out of the documents and to the web browser. Should I try to read the MIME entity in Java? Should I punt and provide a .getDocument() method on the Post to let the <xp:dominoDocument/> source do the work? It's not clear. Should I read in all of the pertinent data from the document as soon as I instantiate a Post, or should I leave the Document object sitting around and only pull it in on-demand? I guess I'll have to try both and find out if one is much slower.

Eventually, I expect to find a smooth way of accomplishing everything I want to do and all this will be like second nature. But that "eventually" is quite a while, and a far cry from more opinionated software like Rails (which is written in a better language, to boot). Maybe Mastering XPages will help, but it seems I'll have to content myself with muddling through until at least the end of January.

Turns out Partial Execute is handy

Aug 22, 2010 10:53 PM

My main "spare time" project involves working on a lineup editor for raids in WoW. Basically, each player can have one or more characters, each of which can perform one or two roles (out of "Tank", "Healer", or "DPS"), and the UI should allow the person setting it up to pick which character that player will bring and which role they will perform. The setup is a table, with each row looking like:

The checkbox indicates whether or not the player is going on the run, then there's the player name, the currently-selected character (the icon denotes the character's current specialization), then how they originally signed up and their desired role, then three checkboxes for the role, and then their skill/gear rank.

Originally, I had the lineup set up as a Server-Side JavaScript array with ad-hoc objects with fields for each column. This worked out alright at first, but not only stepped all over MVC separation, since the controls had to "know" about the nature of the data they were working with, but made the necessary event-handler code really tangled.

What clinched the deal was the side effects of switching a character or role - not every character can fill every role, so switching a character or role could easily switch the other. Case in point - if I switch my lineup line there to my hunter Elegabal, the role has to change as well:

With a JavaScript array and event handlers, every change meant code attached to the control itself that looked up the current character in the database, the roles they can fill, if they're eligible for the selected role, then either what roles they CAN fill (if a character was picked) or what character is available to fill the selected role. Not impossible, but VERY hairy.

So I switched over to Java objects on the back-end. While the Java code itself is less expressive than JavaScript, the organization is significantly cleaner and more logical. The Lineup is a class that descends from Vector to provide the overall table, and each element is a LineupLine object with getters and setters for each important element. That way, rather than each role radio button having to have code to sanity-check the character, they're just bound to "#{line.role}" and the LineupLine.setRole(String) method handles all the particulars.

Early on, though, I started running into some oddities - I'd pick a different role, the table would refresh, and then my role selection would go back to its previous state. I added in some System.out.print() lines in the Java objects to make sure setRole() was being called and, sure enough, it was - I could see that it was called with the right new value, yet it kept switching back to the old one. Upon further investigation, I found the problem: while the new role was being set, the server was also getting a setCharacter() call with the existing character value. So the role would be switched, then the character would be set, and, since that character couldn't do the new role, it'd naturally switch the character back.

The solution I found was to enable Partial Execution on the input controls. I can only assume that this means that, rather than sending the server the entire contents of the row as the new data to process, it sends only the control's (i.e. the drop-down's or the radio button's) data, which limits the method calls to only the one that actually changed. Presumably, this has the side effect of being slightly faster, since less data is sent to the server to be processed.