The Big Apple Silicon Switch, Followup
Thu Feb 16 13:17:49 EST 2023
Yesterday, I wrote a post detailing my recent switch to Apple Silicon, and in it I noted that the main remaining problem I was butting heads with was running Domino in Docker containers, specifically in the context of working with Java.
While my general solution of connecting to Docker running remotely is good, it's all the better to have this stuff working locally. I gave it another shot today and found that the solution is in the post: disable the JIT.
Java uses the JIT - its just-in-time compiler - to speed up execution dynamically when running a program, and it's part of what makes Java surprisingly fast in practice. It's a whole rabbit hole of performance implications and comparisons, but it will suffice to say "JIT = good". However, as I found when running test cases that load libnotes.dylib directly, J9's JIT doesn't play well with the x64-to-ARM JIT that these Macs use, either Rosetta's or qemu's.
So, long story short, the solution was to modify my testing container to disable the JIT when in such an environment (the referenced commit has since been followed up a few times).
First off, I added a bit in my Domino One-Touch Config file to point to a Java options file in the notes.ini:
1 2 3 4 5 6 7 8 | { /* ... */ "notesINI": { /* ... */ "JavaUserOptionsFile": "/local/JavaOptionsFile.txt" } /* ... */ } |
I modified the Dockerfile to bring this in from the build environment:
1 2 | # ... COPY --chown=notes:notes staging/JavaOptionsFile.txt /local/ |
Finally, I modified the class I use to build the container to optionally populate this file with the flag to disable the JIT:
1 2 3 4 5 6 | String arch = DockerClientFactory.instance().getInfo().getArchitecture(); if(!"x86_64".equals(arch)) { withFileFromTransferable("staging/JavaOptionsFile.txt", Transferable.of("-Djava.compiler=NONE")); } else { withFileFromTransferable("staging/JavaOptionsFile.txt", Transferable.of("")); } |
(Transferable
is a Testcontainers-ism for any available source of a file to put into the build environment. Often, this will be something from the project classpath, but it can also be arbitrary data like this.)
With that in place, the test suite works! Eventually!
The thing to know about this is that it's slow. It's going to be naturally slow due to emulation, and I imagine the lack of Java's JIT doesn't help anything. Running the XPages JEE test suite takes about 520 seconds in this setup, as compared to 180 seconds when run via my remote native VM. Such a dramatic drop in speed is enough that I'm likely to continue using the remote VM for most things, and only use local emulation for things that significantly benefit from filesystem binds. For what it's worth, it seems like multithreading is what really kills it: most tests are slower by roughly 50%, while a big multithread test balloons from 20s to 160s. That's consistent with my experience with local unit tests, where the multithread ones were the ones that crashed the process.
For the record, I found that this currently only works at all with Docker Desktop's default (qemu-based) emulation. When I switch the emulator to Rosetta, I hit a StackOverflowError from the HTTP JVM during init without any useful other information. That's fair enough, since that emulation type is still flagged as Experimental in Docker Desktop anyway.
Anyway, it's good to know that there's a way to make it work. It's still no replacement for a native container, but at least it works at all. Similar techniques should work with other tasks, like setting the above option in the JAVA_OPTS
environment variable.