Seriously, Though: Reverse Proxies
Sep 16, 2015, 5:28 PM
So, Domino administrators: what are your feelings about SSL lately? Do they include, perhaps, stress? It's "oh crap, my servers are broken" season again, and this time the culprit is a change in Apple's operating systems. Fortunately, in this case, the problem isn't as severe as an outright security vulnerability like POODLE and, better still, there is a definitive statement from IBM indicating that they are going to bring their security stack up to snuff almost in time.
But this isn't the first time we've been in this position, nor will it be the last. The focus on cracking and hardening TLS, particularly in the context of HTTPS, is not going to let up any time soon, nor will the basic movement towards encryption everywhere. So I would like to reiterate my stance: Domino is not suitable for direct external exposure via HTTP. The other protocols are problematic as well, but HTTP is the big one and, fortunately, the easiest to solve.
Whenever I've made this exhortation, part of the response I get is that administrators "should not" have to take this step. That Domino should be fully modern in its security stack or, at least, that IBM should handle this problem for them in one way or another. Or that one of Domino's traditional strengths is its all-on-one nature, with a single easy installation that takes care of everything, and that installing a separate web server is a complicated step that administrators shouldn't have to take.
Well... tough.
The promise of an integrated server system that took care of everything is a great promise, but it's always been extremely difficult to achieve, even for a platform firing on all cylinders. No matter the ideal, Domino does not perform at this level, and I still maintain that it should not need to. Outside of Domino and PHP, the application server is not generally expected to also be a full-fledged front-end web server, for exactly this sort of reason. Domino's job with respect to the web is to generate and serve up HTML, JSON, and other content; it's something else's job to make sure that that leaves your company's network securely.
If you still maintain that this should be Domino's job due to how much you pay for licensing, then that's a conversation between you and your IBM sales rep. I, though, am entirely fine with a paid-for app server not covering this ground, and that's in large part because the products that do perform this task are superb and often open-source.
These other products – nginx, Apache, HAProxy, and so forth – are made for this job. This flurry of SSL/TLS features and bugs you've been hearing about? These are all implemented or fixed in dedicated products, sometimes years before they come to your attention. And when new problems crop up, they're fixed and talked about immediately across the web, with guides for what to do appearing as soon as the problem arises.
Is it easier to continue using Domino HTTP directly than to set up a reverse proxy? Sure! Well, sort of, when there's not an active disaster to mitigate. And, much like how keeping an XPages (or other web) app up to spec and working on all target devices is more complicated than a legacy Notes app, sometimes that's just how the world goes. Deciding that it's complexity you don't want, or that your company's policy doesn't allow for an additional server, is not a tenable stance. Unless you're Apple, your company's policy will not bend the arc of the industry.
So, I implore you, at least give this kind of setup a real look and a trial run. I think you'll find that the basic setup is not dramatically more complicated than just Domino alone and will also open the door to new non-security features like improving page load speeds on the fly. If you want, with eyes open, to maintain an externally-facing Domino HTTP stack, that's fine, but I'll see you when the next security apocalypse comes around.
Stephen Bailey - Sep 17, 2015, 5:50 AM
I'm sure there are many Domino administrators out there who agree with you, and then provide real-world examples where putting a proxy in front of Domino causes complications, or even stops the product working as expected.
If you raise a PMR with IBM Support about a Domino web server issue, one of the first questions you'll get asked is "Does the issue still occur if you by-pass the proxy?"
For me, right now I have a PMR open where we are unable to activate a Windows phone / Windows 10 machine through Traveler. It seems that the cause is that the Traveler server is running port 80, but is being secured by a reverse proxy sat in front of it. For some reason, Windows phone is being more fussy than Android or iOS, preferring the SSL certificate etc to be hosted in Domino, rather than at the proxy level.
Oh, and don't get me started about Quickr in front of proxies either... (I know, I know... but we're still using it).
Broadly I fully agree with you, but in the real world we have to get stuff to work! :)
Sean Cull - Sep 17, 2015, 7:18 AM
I can see both sides of this. I spent a great deal of time during the Domino SSL fiasco putting proxy servers in front of our web facing servers and blogged about it a lot. While having the proxy servers helped there were lots of edge cases and it was a very iterative process getting all of the settings right.
When IBM released the SSL fixes we removed most of the proxies and to be honest found it a real relief that we were only administering a single stack. There are often times when something is not working and the fewer variables and interfaces involved the better.
I had an example last week where a dev server that still had an Apache proxy did not work correctly with Font Awesome resources - I think the url was getting mashed by Apache so I removed Apache and all was well. Yes I could probably have gotten Apache to work but it was easier to just use the native IBM stack. I know that we also had issues around the faces.context thinking everything was HTTP whereas the end users were obviously using HTTPS - we had to add a configuration document to the applications so manually force the correct response for some functionality.
So I can see both sides. I loved the way Apache allowed multiple SSL sites on the same IP address and I loved being able to run PHP and Domino services on the same box. But I just want a simple life where I can provide repeatable appliances or SAAS servers to customers.
Mind you using a single proxy server in front of many domino SAAS servers has attractions.
Richard Moy - Oct 31, 2015, 12:08 AM
Jesse,
After attending your reverse proxy presentation at MWLUG 2014, we have been using nginx with Domino and have not looked back. We find it easier to manage using nginx. With nginx in front of Domino we can do many others things that we can not do with Domino alone. An infrastructure should utilize the best of the best and nginx is far better than Domino when it comes to the HTTP stack. Thanks.
Richard