Issues with git submodule URLs

[dra27]

Quick brain-dumping issue of the state of play (roughly) for submodules.

There are still a couple of issues which remain:

1. When cloning for the first time, our handling of errors in the submodule URLs isn't right (we should clean-up and fail the clone, but it leaves the working directory in an inconsistent state which can affect future jobs). In particular, note that the cache may be cloned from a PR
2. When git submodule update fails, we end up leaving working directories in a slightly weird state.

The fundamental problem is that we move working files and then copy the .git directory when setting up build contexts. This erases the .git cookie files in submodule directories which removes Git's knowledge that .git/modules/ contains repos which may need updating. When doing the move, we should ignore any file/directory called .git and leave these behind. git submodule sync will then work as intended, updating both .git/config and also .git/modules/<directory>/config. However, this may still be brittle in repos where the submodules may be changing (e.g. recent addition of incr, etc.).

In addition to handling the caching correctly, it may be worth having a fallback at git submodule sync that if it fails then we erase .git/modules and obviously if the whole process fails we should clean the working directory.