"Build Automatically", "Refresh automatically", and "Team Development"

Thu Jan 23 18:06:26 EST 2014

TL;DR: "Build Automatically" and "Refresh automatically" are unrelated, but work together when both enabled when using SourceTree for source control.

I just had a conversation on Twitter where I went a little crazy talking about my hopefully-correct view of how the various sync processes in Designer work when you're dealing with source control.

The original impetus was a post by Mark Leusink about the fact that you have to build your NSF project in order to sync it with an on-disk-project (ODP) set up via Team Development. From there, that got into the topic of the "Refresh automatically" Eclipse setting that is helpful when using source control tools outside of Designer (e.g. SourceTree instead of eGit). The problem is that this stack of builders and settings is hard to get a hold of conceptually.

There are three "entities" to know about when it comes to how these things work together, along with a couple processes that can get your source code from one to the other. I tossed together this terrible image to visualize them:

I'll work from right to left, starting with the NSF. That part is nothing new, but the thing to know is how the NSF "knows" how to sync to an ODP. It's done via something called an Eclipse Builder. A Builder is a task you can attach to a project in Eclipse to do essentially arbitrary things when the build process happens for a project (or an individual part of a project). NSFs have a ton of them:

The one in question is the unnamed one at the top (don't ask me why it's nameless): that's the Builder that exports your changes from the NSF to the ODP. There's a companion Builder, also unnamed, in the ODP, that does the reverse when you make a change to the files there (normally you don't, but source control does). Because this sync is implemented with a Builder, it doesn't run automatically when you have "Build Automatically" disabled. Instead, it will run when you manually do a "Build All" or "Clean", or when you choose "Team Development" → "Sync with On-Disk Project" from the application list.

So the Builder handles all of the syncing between the ODP and the NSF. Where does "Refresh automatically" (which is very different from "Build Automatically") come in, and why has it only become a thing recently with the cultural shift away from eGit and towards SourceTree?

The culprit is the way Eclipse/DDE deals with the filesystem. Because OSes have traditionally been bad at notifying programs of when files change on disk, DDE doesn't actually know when a file is changed on disk. This is expected for something like an NSF (say, if someone else adds a view to an NSF you have open, it's no surprise that you have to refresh to see it in Designer (also, don't edit the same NSF from two places like that)), but counterintuitive with local files. The view of the ODP you see in Designer's Package Explorer is analogous to running an "ls" or "dir" command in the terminal and then adding a new file: the output from your command won't change until you run it again. What "Refresh automatically" does is attempt to keep its file listing up-to-date. Later versions of Eclipse modified this option to be more explicit and technical:

The upshot is that turning this on will (hopefully) keep the ODP in step with the filesystem. The chart I made is misleading here: the ODP and filesystem are technically the same storage area, but the ODP you see in DDE is really a listing of what Designer saw the last time it looked.

This is useful when using a tool like SourceTree to handle your source-control operations. Because DDE doesn't normally "know" about filesystem changes, you could update your local source repository from the server with tons of changes, but Designer wouldn't know until you kicked it. With "Refresh automatically", it should notice those changes on its own, update the ODP in Package Explorer, and, if "Build Automatically" is on, sync those changes into your NSF.

The reason this matters now and didn't/doesn't when using eGit is that, because eGit is an Eclipse plugin and does its filesystem access through Eclipse, Designer already knows when a source-control-related operation changes files on the filesystem: since it made the change, it doesn't need to refresh. So if you're using an Eclipse plugin for source control and not editing files on the filesystem manually, you can remain in blissful ignorance about "Refresh automatically."

Commenter Photo

Nathan T. Freeman - Fri Jan 24 00:25:47 EST 2014

Sure, you don't have to tell eGit to refresh the ODP. But there are 3 rules in working with git from Eclipse.

1) don't use eGit.

2) seriously, don't use eGit.

3) I'm not kidding. Don't use eGit.

eGit is far superior to command-line git in terms of ease of use. Unfortunately, it doesn't hold a candle to SourceTree. And as it happens, it really doesn't matter than it doesn't hold a candle. There really isn't any reason for your SCM to be inside your IDE. It *feels* like it should be, but the two have very little to do with each other beyond the occasional need to refresh a folder. Beyond that, SourceTree makes your life 150% better than it would be with any other git UI.

Commenter Photo

Martin Rolph - Fri Jan 24 03:59:30 EST 2014

Great post and thanks for the explanation!  It's starting to make sense to me now.

I'm probably remembering incorrectly but I'm sure I read somewhere that you should turn off "Refresh automatically" when working against server copies of databases. I thought that was a general rule for server dbs but by the sound of it that must have just been related to ODPs. Your explanation suggests that the Refresh option is irrelevant when you don't have an ODP, even when working against a server (yes I know that isn't a good idea but sometimes needs must)

The issue I still have is will dam jar files. At the moment I'm needing to leave build automatically OFF even when working locally as otherwise DDE goes into an infinite build/refresh cycle.  Only happens when the nsf has a jar file. There's an explanation here 

http://stackoverflow.com/questions/17669777/importing-a-jar-file-into-an-nsf-with-build-automatically-switch-on-causes-a-rec

but the solution doesn't seem to fully work. keeps coming back.  Does anyone else have this issue?  Because I had build automatically switched OFF I wasn't getting the changes reflected in DDE via SourceTree unless I manually did a Team>Sync with ODP, which was the prompt for my original question. I understand why now.

Thanks again!

Commenter Photo

John Dalsgaard - Fri Jan 24 10:41:39 EST 2014

@Jesse, thanks for the post. I have experimented quite a lot with these settings and found that "build automaticlly" works fine for me - but "refresh automatically" can get confused and create local conflicts that I needed to solve manually. So now I just refresh manually when I "know" there is a need (I have a blog article on its way about the setup that works for me...).

@Martin - your life will become much easier deploying jar-files as OSGi plug-ins, I found ;-)

I tried to give a recipe as to how to do that in this article: http://www.dalsgaard-data.eu/blog/wrap-an-existing-jar-file-into-a-plug-in/ (and the one following it)

New Comment