Skip to content

Commit 5064d06

Browse files
support home being ldmx sw (#138)
* remove entrypoint script now that denv has been more fully adopted * put GENIE ld libs and PATH into image build at temp location * don't define LDMX_BASE, update LDMX_SW_INSTALL deduction - support ldmx-sw being HOME or inside HOME - remove GENIE LD_LIBRARY_PATH and PATH now in image - cleanup CMAKE_PREFIX_PATH deduction * update comments in init script * move genie ld libs and PATH defs to where GENIE is built * remove custom Geant4 scripting * first draft of updating documentation of custom geant4 * move base definition of CMAKE_PREFIX_PATH into Dockerfile * reread and fixup custom geant4 instructions
1 parent fe6f55b commit 5064d06

5 files changed

Lines changed: 137 additions & 112 deletions

File tree

Dockerfile

Lines changed: 9 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -316,7 +316,9 @@ RUN mkdir -p ${GENIE} &&\
316316
--with-hepmc3-inc=/usr/local/include \
317317
&& \
318318
make -j$NPROC && \
319-
make -j$NPROC install
319+
make -j$NPROC install && \
320+
echo "${GENIE}/lib" > /etc/ld.so.conf.d/genie.conf
321+
ENV PATH="${PATH}:${GENIE}/bin"
320322

321323
ENV GENIE_REWEIGHT_VERSION=1_04_00
322324
ENV GENIE_REWEIGHT=/usr/local/src/GENIE/Reweight
@@ -326,7 +328,10 @@ RUN mkdir -p ${GENIE_REWEIGHT} &&\
326328
${__untar_to} ${GENIE_REWEIGHT} &&\
327329
cd ${GENIE_REWEIGHT} &&\
328330
make -j$NPROC && \
329-
make -j$NPROC install
331+
make -j$NPROC install && \
332+
echo "${GENIE_REWEIGHT}/lib" > /etc/ld.so.conf.d/genie-reweight.conf
333+
ENV PATH="${PATH}:${GENIE_REWEIGHT}/bin"
334+
330335

331336
###############################################################################
332337
# Catch2
@@ -444,18 +449,6 @@ RUN update-ca-certificates
444449

445450
# copy environment initialization script into container
446451
# and make sure the default profile will call it as well
452+
ENV CMAKE_PREFIX_PATH=/usr/local
447453
COPY ./ldmx-env-init.sh /etc/
448-
RUN printf "%s\n" \
449-
"# make sure LDMX_BASE is defined for ldmx-env-init.sh" \
450-
"if [ -z \"\${LDMX_BASE+x}\" ]; then" \
451-
" export LDMX_BASE=\"\${HOME}\"" \
452-
"fi" \
453-
". /etc/ldmx-env-init.sh" \
454-
>> /etc/skel/.profile
455-
456-
#run environment setup when docker container is launched and decide what to do from there
457-
# will require the environment variable LDMX_BASE defined
458-
COPY ./entry.sh /etc/
459-
RUN chmod 755 /etc/entry.sh
460-
ENTRYPOINT ["/etc/entry.sh"]
461-
454+
RUN printf "\n%s\n" ". /etc/ldmx-env-init.sh" >> /etc/skel/.profile

docs/src/SUMMARY.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,10 @@
33
[Welcome](README.md)
44

55
# Using
6+
- [Determing an Image's Version](image-version.md)
67
- [Custom Geant4](custom-geant4.md)
78
- [Custom Acts](custom-acts.md)
89
- [Parallel Containers](parallel_containers.md)
9-
- [Determing an Image's Version](image-version.md)
1010

1111
# Developing
1212
- [Contributing](contributing.md)

docs/src/custom-geant4.md

Lines changed: 105 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,44 +3,41 @@
33
Geant4 is our main simulation engine and it has a large effect on the products of our simulation samples.
44
As such, it is very common to compare multiple different versions, patches, and tweaks to Geant4 with our simulation.
55

6-
With release 4.2.0 of the ldmx/dev image, the entrypoint script now checks the environment variable `LDMX_CUSTOM_GEANT4` for a path to a local installation of Geant4.
7-
This allows the user to override the Geant4 that is within the image with one that available locally. In this way, you can choose whichever version of Geant4 you want,
8-
with whatever code modifications applied, with whatever build instructions you choose.
9-
106
~~~admonish warning title="Confirm Image Version"
117
Make sure you have an image that is at least v4.2.0.
128
You can check your version of the image by [inspecting the image labels](image-version.md).
139
~~~
1410

15-
### Building Your Geant4
11+
## Building Your Geant4
1612
You can build your Geant4 in a similar manner as ldmx-sw. It does take much longer to compile than ldmx-sw since it is larger, so be sure to leave enough time for it.
1713

1814
```admonish warning title="Remember"
1915
You can only run this custom build of Geant4 with whatever image you are building it with, so make sure you are happy with the image version you are using.
2016
```
2117

2218
``` shell
23-
cd path/to/ldmx # directory that contains ldmx-sw
19+
cd path/to/ldmx/ldmx-sw # ldmx-sw you want to build with custom geant4
2420
git clone git@github.com:LDMX-Software/geant4.git # or could be mainline Geant4 or an unpacked tar-ball
2521
denv cmake -B geant4/build -S geant4 <cmake-options>
2622
denv cmake --build geant4/build --target install
2723
```
2824
Now building Geant4 from source has a lot of configuration options that can be used to customize how it is built.
2925
Below are a few that are highlighted for how we use containers and their interaction with the Geant4 build.
3026

31-
- `CMAKE_INSTALL_PREFIX`: This should be set to a path accessible from the container so that the programs within the container can read from and write to this directory. If the geant4 build directory is within `LDMX_BASE` (like it is above), then you could do something like `-DCMAKE_INSTALL_PREFIX=../install` when you run `ldmx cmake` within the build directory.
32-
- `GEANT4_INSTALL_DATADIR`: If you are building a version of Geant4 that has the same data files as the Geant4 version built into the container image, then you can tell the Geant4 build to use those data files with this option, saving build time and disk space. This is helpful if (for example) you are just re-building the same version of Geant4 but in Debug mode. You can see where the Geant4 data is within the container with `ldmx 'echo ${G4DATADIR}'` and then use this value `-DGEANT4_INSTALL_DATADIR=/usr/local/share/geant4/data`.
27+
- `CMAKE_INSTALL_PREFIX`: This should be set to a path accessible from the container so that the programs within the container can read from and write to this directory. If the geant4 build directory is within ldmx-sw (like it is above), then you could do something like `-DCMAKE_INSTALL_PREFIX=geant4/install` when you run `denv cmake ...`
28+
- If you are keeping Geant4 outside of ldmx-sw, then you may need to mount it into the ldmx-sw container image with `denv config mounts` if it is not already within a mounted directory.
29+
- `GEANT4_INSTALL_DATADIR`: If you are building a version of Geant4 that has the same data files as the Geant4 version built into the container image, then you can tell the Geant4 build to use those data files with this option, saving build time and disk space. This is helpful if (for example) you are just re-building the same version of Geant4 but in Debug mode. You can see where the Geant4 data is within the container with `denv printenv G4DATADIR` and then use this value `-DGEANT4_INSTALL_DATADIR=/usr/local/share/geant4/data`.
3330

34-
The following are the build options used when setting up the container and are likely what you want to get started
31+
The following are the build options used when setting up the image and are likely what you want to get started
3532
- `-DGEANT4_USE_GDML=ON` Enable reading geometries with the GDML markup language which is used in LDMX-sw for all our geometries
3633
- `-DGEANT4_INSTALL_EXAMPLES=OFF` Don't install the Geant4 example applications (just to save space and compilation time)
3734
- `-DGEANT4_USE_OPENGL_X11=ON` enable an X11-based GUI for inspecting geometries
3835
- `-DGEANT4_MULTITHREADED=OFF` If you are building a version of Geant4 that is multithreaded by default, you will want to disable it with. The dynamic loading used in LDMX-sw will often not work with a multithreaded version of Geant4
3936

40-
#### Concerns when building different versions of Geant4 than 10.2.3
37+
### Concerns when building different versions of Geant4 than 10.2.3
4138

42-
For most use cases you will be building a modified version of the same release of Geant4 that is used in the container (10.2.3). It is also possible to build and use later versions of Geant4 although this should be done with care. In particular
43-
- Different Geant4 release versions will require that you rebuild LDMX-sw for use with that version, it will not be sufficient to set the `LDMX_CUSTOM_GEANT4` environment variable and pick up the shared libraries therein
39+
For most use cases you will be building a modified version of the same release of Geant4 that is used in the image (10.2.3). It is also possible to build and use later versions of Geant4 although this should be done with care. In particular
40+
- Different Geant4 release versions will require that you rebuild LDMX-sw for use with that version, it will not be sufficient to just source the custom Geant4's environment and pick up the shared libraries therein
4441
- Recent versions of Geant4 group the electromagnetic processes for each particle into a so-called general process for performance reasons. This means that many features in LDMX-sw that rely on the exact names of processes in Geant4 will not work. You can disable this by inserting something like the following in [RunManager::setupPhysics()](https://github.com/LDMX-Software/SimCore/blob/20d9bcb6d2bad2b99255cf32c1b3f099b26752b0/src/SimCore/RunManager.cxx#L60)
4542
```C++
4643
// Make sure to include G4EmParameters if needed
@@ -49,9 +46,8 @@ auto electromagneticParameters {G4EmParameters::Instance()};
4946
// i.e. G4GammaGeneralProcess and G4ElectronGeneralProcess
5047
electromagneticParameters->SetGeneralProcessActive(false);
5148
```
52-
- Geant4 relies on being able to locate a set of datasets when running. For builds of 10.2.3, the ones that are present in the container will suffice but other versions may need different versions of these datasets. If you run into issues with this, use `ldmx env` and check that the following environment variables are pointing to the right location
49+
- Geant4 relies on being able to locate a set of datasets when running. For builds of 10.2.3, the ones that are present in the container will suffice but other versions may need different versions of these datasets. If you run into issues with this, use `denv printenv` and check that the following environment variables are pointing to the right location
5350
- `GEANT4_DATA_DIR` should point to `$LDMX_CUSTOM_GEANT4/share/Geant4/data`
54-
- You can define the `LDMX_CUSTOM_GEANT4_DATA_DIR` environment variable in the container environment to manually point it to a custom location
5551
- The following environment variables should either be unset or point to the correct location in `GEANT4_DATA_DIR`
5652
- `G4NEUTRONHPDATA`
5753
- `G4LEDATA`
@@ -64,8 +60,8 @@ electromagneticParameters->SetGeneralProcessActive(false);
6460
- `G4ABLADATA`
6561
- `G4INCLDATA`
6662
- `G4ENSDFSTATEDATA`
67-
- When using CMake, ensure that the right version of Geant4 is picked up at configuration time (i.e. when you run `ldmx cmake`)
68-
- You can always check the version that is used in a build directory by running `ldmx ccmake .` in the build directory and searching for the Geant4 version variable
63+
- When using CMake, ensure that the right version of Geant4 is picked up at configuration time (i.e. when you run `denv cmake`)
64+
- You can always check the version that is used in a build directory by running `denv ccmake .` in the build directory and searching for the Geant4 version variable
6965
- If the version is incorrect, you will need to re-configure your build directory. If `cmake` isn't picking up the right Geant4 version by default, ensure that the `CMAKE_PREFIX_PATH` is pointing to your version of Geant4
7066
- Make sure that your version of Geant4 was built with multithreading disabled
7167
@@ -216,8 +212,100 @@ mostly the same datasets, it is easier just to have each Geant4 version have its
216212
own downloaded copies of the datasets.
217213
~~~
218214
215+
## Running with your Geant4
216+
The way we use different versions of Geant4 has changed over the years, so it depends on which version of the image you are using.
217+
218+
### >=5.1.1
219+
Since we are using `denv` to interact with the development image, you now have access to a local file that
220+
can customize your development environment within the container image.
221+
This file is `.profile` located within the container's home directory.
222+
To find the location of this file, run
223+
```sh
224+
denv printenv HOME
225+
```
226+
from the location where you want to use the custom Geant4.
227+
The path output by this command is where the `.profile` is that you will edit.
228+
229+
~~~admonish warning title="System `.profile`" collapsible=true
230+
The `.profile` file is a file that exists in many normal Linux (and MacOS) systems.
231+
I am just pointing this out because if you edit your system one (located at `~/.profile`)
232+
instead of the one that is located within the denv workspace, you will not get the changes
233+
to the container environment you want _and_ you could break your system.
234+
~~~
235+
236+
All this stuff should go at the _end_ of the `.profile` so that you are "updating" the default
237+
environment.
238+
239+
First, make sure to unset the image-specific versions of the Geant4 environment variables defining
240+
the location of the data directories.
241+
This list may not be complete depending on the version of Geant4 installed in the image, you can use
242+
`denv printenv` to see the full list of environment variables within the container environment.
243+
```
244+
unset G4NEUTRONHPDATA
245+
unset G4LEDATA
246+
unset G4LEVELGAMMADATA
247+
unset G4RADIOACTIVEDATA
248+
unset G4PARTICLEXSDATA
249+
unset G4PIIDATA
250+
unset G4REALSURFACEDATA
251+
unset G4SAIDXSDATA
252+
unset G4ABLADATA
253+
unset G4INCLDATA
254+
unset G4ENSDFSTATEDATA
255+
unset G4NEUTRONXSDATA
256+
```
257+
If you changed the location of the data directory when building Geant4, make sure to also use that
258+
location here by defining `GEANT4_DATA_DIR` _before_ sourcing the Geant4 environment script.
259+
```
260+
# only needed if changed data location when building geant4
261+
export GEANT4_DATA_DIR=/full/path/to/custom/data/location
262+
```
263+
Then source the custom Geant4's environment script.
264+
```
265+
. /full/path/to/custom/geant4/bin/geant4.sh
266+
# this stuff below is helpful to make sure a data directory is found
267+
# and allows folks to have a debug build of Geant4 without re-downloading the data
268+
# it goes _after_ the script because the script will define GEANT4_DATA_DIR if the
269+
# build is configured with a specific data location
270+
if [ -z "${GEANT4_DATA_DIR+x}" ]; then
271+
export GEANT4_DATA_DIR="${G4DATADIR}"
272+
fi
273+
```
274+
And finally, update `CMAKE_PREFIX_PATH` so that ldmx-sw will prefer this custom Geant4 instead
275+
of the one installed within the image.
276+
```
277+
export CMAKE_PREFIX_PATH="/full/path/to/custom/geant4/lib/cmake:${CMAKE_PREFIX_PATH}"
278+
```
279+
280+
After these changes, you should be able to compile and run ldmx-sw from this environment using
281+
your custom build of Geant4 with the normal development commands.
282+
```
283+
just compile
284+
just fire config.py
285+
```
286+
287+
You can make sure your Geant4 was found and is being used by going into the build and inspecting
288+
the configuration.
289+
```
290+
cd build && denv ccmake .
291+
```
292+
You should see `Geant4_DIR` set to the path of your custom Geant4 instead of some path in `/usr/local/...`.
293+
294+
~~~admonish note title="Returning to Normal" collapsible=true
295+
If you want to return to the normal environment, you can comment-out or remove your `.profile` changes
296+
at the bottom of that file.
297+
298+
A nuclear option is to have the image re-copy of new `.profile` by removing the `.profile` file and a `denv`
299+
internal file signalling that the profile has already been copied.
300+
```
301+
rm .profile .denv/skel-init
302+
```
303+
~~~
219304

220-
### Running with your Geant4
305+
### <5.1.1,>=4.2.0
306+
With release 4.2.0 of the ldmx/dev image, the entrypoint script now checks the environment variable `LDMX_CUSTOM_GEANT4` for a path to a local installation of Geant4.
307+
This allows the user to override the Geant4 that is within the image with one that available locally. In this way, you can choose whichever version of Geant4 you want,
308+
with whatever code modifications applied, with whatever build instructions you choose.
221309
Just like with ldmx-sw, you can only run a specific build of Geant4 in the same image that you used to build it.
222310
``` shell
223311
just setenv LDMX_CUSTOM_GEANT4=/path/to/geant4/install

entry.sh

Lines changed: 0 additions & 16 deletions
This file was deleted.

ldmx-env-init.sh

Lines changed: 22 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -2,75 +2,35 @@
22
# Environment initialization for LDMX SW container images
33
# Assumptions:
44
# - The installation location of ldmx-sw is defined in LDMX_SW_INSTALL
5-
# or it is located at LDMX_BASE/ldmx-sw/install.
5+
# or it is located at ${HOME}/ldmx-sw/install or ${HOME}/install.
66
###############################################################################
77

8-
# add ldmx-sw and ldmx-analysis installs to the various paths
98
# LDMX_SW_INSTALL is defined when building the production image or users
109
# can use it to specify a non-normal install location
1110
if [ -z "${LDMX_SW_INSTALL+x}" ]; then
1211
if [ -z "${LDMX_BASE+x}" ]; then
13-
printf "[ldmx-env-init.sh] WARNING: %s\n" \
14-
"Neither LDMX_BASE nor LDMX_SW_INSTALL is defined." \
15-
"At least one needs to be defined to ensure a working installation."
12+
# LDMX_BASE not defined
13+
if [ -f "${HOME}/CMakeLists.txt" ]; then
14+
# HOME is ldmx-sw
15+
export LDMX_SW_INSTALL="${HOME}/install"
16+
elif [ -d "${HOME}/ldmx-sw" ]; then
17+
# HOME is ldmx-sw's parent directory
18+
export LDMX_SW_INSTALL="${HOME}/ldmx-sw/install"
19+
else
20+
# unable to auto-detect
21+
printf "[ldmx-env-init.sh] WARNING: %s\n" \
22+
"LDMX_SW_INSTALL is not defined and I wasn't able to deduce the location relative to ${HOME}." \
23+
"You may not be able to run ldmx-sw programs in this environment."
24+
fi
25+
else
26+
# LDMX_BASE defined
27+
export LDMX_SW_INSTALL="${LDMX_BASE}/ldmx-sw/install"
1628
fi
17-
export LDMX_SW_INSTALL="${LDMX_BASE}/ldmx-sw/install"
1829
fi
19-
export LD_LIBRARY_PATH="${LDMX_SW_INSTALL}/lib:${LD_LIBRARY_PATH}"
20-
export PYTHONPATH="${LDMX_SW_INSTALL}/python:${LDMX_SW_INSTALL}/lib:${PYTHONPATH}"
21-
export PATH="${LDMX_SW_INSTALL}/bin:${PATH}"
22-
23-
#add what we need for GENIE
24-
export LD_LIBRARY_PATH="${GENIE}/lib:${GENIE_REWEIGHT}/lib:/usr/local/pythia6:${LD_LIBRARY_PATH}"
25-
export PATH="${GENIE}/bin:${GENIE_REWEIGHT}/bin:${PATH}"
2630

27-
# Developer option: If a custom geant4 install is to be used, source the
28-
# environment script from that install
29-
#
30-
# Note: Use with care!
31-
#
32-
# The custom Geant4 install still needs to have been built with the same
33-
# container environment
34-
if [ -n "${LDMX_CUSTOM_GEANT4+x}" ]; then
35-
# Overly obnoxious warning to make sure this feature isn't used accidentally
36-
# Also detail how to set custom Geant4 data directories
37-
if [ -z "${LDMX_CUSTOM_GEANT4_CONFIRM_DEV+x}" ]; then
38-
echo "Warning: You are relying on a non-container version of Geant4. This mode of operation can come with some reproducibility concerns if you aren't careful. "
39-
echo "Define the environment variable LDMX_CUSTOM_GEANT4_CONFIRM_DEV in the container environment to suppress this message"
40-
echo "If using the standard ldmx-env.sh shell script, use 'ldmx setenv' to set environment variables within the container environment"
41-
echo "You may also want to define LDMX_CUSTOM_GEANT4_DATA_DIR if you are using a version of Geant4 different from 10.2.3 and the Geant4 build you intend to use has the data directory in an non-standard location (i.e. one that isn't picked up by the geant4.sh script) "
42-
fi
43-
# First: Unset the container-specific versions of the Geant4 data directories
44-
unset G4NEUTRONHPDATA
45-
unset G4LEDATA
46-
unset G4LEVELGAMMADATA
47-
unset G4RADIOACTIVEDATA
48-
unset G4PARTICLEXSDATA
49-
unset G4PIIDATA
50-
unset G4REALSURFACEDATA
51-
unset G4SAIDXSDATA
52-
unset G4ABLADATA
53-
unset G4INCLDATA
54-
unset G4ENSDFSTATEDATA
55-
unset G4NEUTRONXSDATA
56-
# If explicitly requested, use a custom location for Geant4's data directories
57-
if [ -n "${LDMX_CUSTOM_GEANT4_DATA_DIR+x}" ]; then
58-
export GEANT4_DATA_DIR="${LDMX_CUSTOM_GEANT4_DATA_DIR}"
59-
fi
60-
# Source the custom geant's environment script
61-
# shellcheck disable=SC1091
62-
. "${LDMX_CUSTOM_GEANT4}/bin/geant4.sh"
63-
# Prioritize the cmake config in the Geant4 installation over the container location (/usr/local)
64-
export CMAKE_PREFIX_PATH="${LDMX_CUSTOM_GEANT4}/lib/cmake:/usr/local/:${CMAKE_PREFIX_PATH}"
65-
66-
# If no directory was found by the geant4.sh script and the user didn't
67-
# explicitly ask for a location (e.g. for a debug build):
68-
#
69-
# Assume we are using 10.2.3 (container provided) data
70-
if [ -z "${GEANT4_DATA_DIR+x}" ]; then
71-
export GEANT4_DATA_DIR="${G4DATADIR}"
72-
fi
73-
else
74-
# Tell CMake to look for configuration files in the container location by default
75-
export CMAKE_PREFIX_PATH="/usr/local/:${LDMX_SW_INSTALL}"
31+
if [ -n "${LDMX_SW_INSTALL+x}" ]; then
32+
export LD_LIBRARY_PATH="${LDMX_SW_INSTALL}/lib:${LD_LIBRARY_PATH}"
33+
export PYTHONPATH="${LDMX_SW_INSTALL}/python:${LDMX_SW_INSTALL}/lib:${PYTHONPATH}"
34+
export PATH="${LDMX_SW_INSTALL}/bin:${PATH}"
35+
export CMAKE_PREFIX_PATH="${CMAKE_PREFIX_PATH}:${LDMX_SW_INSTALL}"
7636
fi

0 commit comments

Comments
 (0)