Regression: make install does not work within build tree

[PetrilloAtWork]

In a development MRB environment, running make install from a subdirectory of the build tree of a package installs the files into $MRB_INSTALL instead of ${MRB_INSTALL}/<package>/<version>.
This is a regression in that this used to work correctly in the past (not sure when... guessing: old MRB 4).

Tested with larsoft v09_38_00 and its dependencies.

Steps to reproduce

Setup

This is a plain MRB developing area based on LArSoft v09_38_00 with larcore checked out.

mkdir -p makeInstallTest
cd makeInstallTest
source /cvmfs/larsoft.opensciencegrid.org/products/setup
setup mrb
export MRB_PROJECT="larsoft"
mrb newDev -v v09_38_00 -q e20:prof
source localProducts*/setup
cd "$MRB_SOURCE"
mrb gitCheckout larcore
cd "$MRB_TOP"
mrbsetenv
mrb build -j 24
cd "$MRB_BUILDDIR"
make -j24  # second build, to address build system weirdness

Test

From the build directory (from $MRB_TOP is the same):

### OK: installs in larcore package ###
cd "$MRB_BUILDDIR"
make install

shows e.g. /icarus/app/users/petrillo/LArSoft/tests/makeInstallTest/localProducts_larsoft_v09_38_00_e20_prof/larcore/v09_04_00/source/larcore/CoreUtils/ServiceUtil.h.

From the main build directory of the package it also works:

### OK: installs in larcore package ###
cd "${MRB_BUILDDIR}/larcore"
make install

yields the same as above.

From a subdirectory therein:

### WRONG: installs in $MRB_INSTALL directory ###
cd "${MRB_BUILDDIR}/larcore/larcore"
make install

shows e.g. /icarus/app/users/petrillo/LArSoft/tests/makeInstallTest/localProducts_larsoft_v09_38_00_e20_prof/source/larcore/CoreUtils/ServiceUtil.h.
Note the absence in the installation path of the package-specific part: larcore/v09_04_00.

Priority

This is a low priority bug.
That feature was quite handy to me (I spare the reader a description of the frustration that led to the discovery).

[knoepfel]

We will take a look, @PetrilloAtWork. This may be the expected behavior with MRB 5.

[lgarren]

@PetrilloAtWork I am curious. Why "cd $MRB_TOP" to run mrbsetenv? That command is designed to be run from $MRB_BUILDDIR.

[lgarren]

I do not reproduce this behavior (working from a fresh login). Also, I do not see the "build system weirdness" referenced above. Is it possible that you have something set unexpectedly in your environment?

[lgarren]

mkdir makeInstallTest
cd makeInstallTest
mrb newDev -v v09_38_00 -q e20:prof
source localProducts_larsoft_v09_38_00_e20_prof/setup 
cd srcs
mrb g larcore
cd $MRB_BUILDDIR
mrbsetenv
mrb t -j20 (build and test)
cd larcore
make
make install

[lgarren]

Apologies for the misunderstanding, you are trying to install from larcore/larcore. That most definitely does not work. I am somewhat surprised that it ever worked.

[PetrilloAtWork]

@PetrilloAtWork I am curious. Why "cd $MRB_TOP" to run mrbsetenv? That command is designed to be run from $MRB_BUILDDIR.

I was not aware of that. In fact, I have never, ever bothered to enter $MRB_BUILDDIR for that setup: it has always worked from the top directory and I haven't questioned further.

I do not see the "build system weirdness" referenced above

What I perceived as build "weirdness" is that the second make seems to still perform some action ("consolidating dependencies"). It did not use to, and I tend to assume that after make is done, it's done until code changes, so it feels weird to me. Maybe it's the new "normal".

Apologies for the misunderstanding, you are trying to install from larcore/larcore. That most definitely does not work. I am somewhat surprised that it ever worked.

Even now it goes quite close to work... just not quite enough any more.

[knoepfel]

@chissg, can you comment on this issue when you return from vacation?

[greenc-FNAL]

Running partial installs with the native build tool is not recommended practice under CMake: can I ask what need this satisfies in your workflow?

In order to have correct behavior for installation and packaging both for UPS and non-UPS installations, we append ${product}/${version} to CMAKE_INSTALL_PREFIX for UPS installations only at the beginning of the installation procedure for each project; and remove it at the end. We also have to relocate files installed in the wrong place due to legacy UPS-only CMake code. Unfortunately this means that packages can only be installed as a unit.

As for the, "build system weirdness:" there is now some protection against hysteresis caused by GLOBbed file lists including files in build areas, causing an extra CMake configuration step where necessary. By reducing use of cet_make() and the non-LIST variants of install__XXX() functions, we should be able eventually to eliminate this extra configuration step (and the checks for the necessity thereof).

[PetrilloAtWork]

can I ask what need this satisfies in your workflow?

This is in the context of a development workflow, and it does not apply in production workflow.
The usefulness of the feature to me is twofold, both aspects of partial build:

1. in the rapid development cycle, it happens that something from the develop branch of the experiment does not compile; rather than figuring out which directories to CMakeLists.txt away, it's usually easier to just compile the directory with the code I am working on, and happily Let Somebody Else Take Care Of It;
2. in certain situations, I change something in a very upstream code (e.g. LArSoft geometry or data product) and observe the effect in a small experiment-specific test; in that situation, I'd rather avoid the automatic build of the whole suite, but rather let CMake pick up what is actually needed according to the dependency tree.

Sometimes the testing of the changes requires the install stage too, hence the issue.
Needless to say, a developer picking this route is also prepared to a potential for obscure failures.