Relative paths of extra-include-dirs and extra-lib-dirs are resolved from each package's root instead of current working directory

[sergv]

TLDR

Relative paths in command line flags --extra-include-dirs, --extra-lib-dirs and their counterparts in cabal.project resolved against each package's root which causes surprising and user-unfriendly build errors.

Details

I created following repository to reproduce: https://github.com/random-random-stuff/test-cabal-relative-extra-include-and-lib-dirs and guide the following explanation.

I have a package cabal-test-dependency which wraps a foreign library. When building it a cabal_test_foo.h header and a libcabal_test_foo.so file must be available.

Then I have my main package cabal-test-main that uses cabal-test-dependency:

packages:
  cabal-test-main.cabal
  ../dependency/cabal-test-dependency.cabal

The directory with cabal-test-main is where I, as a user, is working in and is where I’m calling cabal from.

I have local copy of .h and .so that are needed to complete the build in root of cabal-test-main package under directory deps (in this though experiment a user puts .so where needed beforehand like here).

$ ls deps/
cabal_test_foo.c  cabal_test_foo.h  libcabal_test_foo.so

Now from cabal-test-main root I’m calling cabal build and pass deps/ directory so that dependency would build but the build fails:

$ cabal build --extra-lib-dirs deps --extra-include-dirs deps
Resolving dependencies...
Build profile: -w ghc-9.14.1 -O1
In order, the following will be built (use -v for more details):
 - cabal-test-dependency-0.1 (lib) (first run)
 - cabal-test-main-0.1 (exe:main) (first run)
Configuring library for cabal-test-dependency-0.1...
Warning: [unknown-directory] 'extra-lib-dirs: deps' specifies a directory
which does not exist.
Warning: [unknown-directory] 'include-dirs: deps' specifies a directory which
does not exist.
Error: [Cabal-4345]
Missing dependency on a foreign library:
* Missing (or bad) header file: cabal_test_foo.h
* Missing (or bad) C library: cabal_test_foo
If the header file does exist, it may contain errors that are caught by the C compiler at the preprocessing stage. In this case you can re-run 'Setup configure' with the verbosity flag -v3 to see the error messages.
Error: [Cabal-7125]
Failed to build cabal-test-dependency-0.1 (which is required by exe:main from cabal-test-main-0.1). The failure occurred during the configure step.

The errors are

Warning: [unknown-directory] 'extra-lib-dirs: deps' specifies a directory
which does not exist.
Warning: [unknown-directory] 'include-dirs: deps' specifies a directory which
does not exist.

apparently because deps directory is being resolved under root directory of cabal-test-dependency package where it does not exist.

If I specify absolute paths, e.g. cabal build --extra-lib-dirs "$(pwd)/deps" --extra-include-dirs "$(pwd)/deps" then build succeeds.

It seems like relative paths should work too and should resolve against where user is currently situated. For command line options that would be the current working directory and for cabal.project options it seems the location of the cabal.project file should serve as a root to resolve relative paths against.

According to #2641 the relative paths for extra-include-dirs and extra-lib-dirs are allowed but neither that PR nor manual provide concrete information as to how relative paths are resolved.

Current behavior seems like an artefact of the implementation where extra-include-dirs and extra-lib-dirs are added to each package’s include-dirs and library-dirs respectively and only then are resolved in each package’s context.

It does not seems like a good idea that that standing in package A and specifying a path relative to A’s root, e.g. foo/ results in dependent package B getting a relative foo/ directory in any part of its package description. While looking at A the user has no visibility under package B and what directories it contains.

System information

• Operating system - Linux x86_64
• cabal - 3.17.0.0, built from master, ghc - 9.14.1

[Mikolaj]

Thank you for the report. I've removed the 'bug' label, because the current behaviour seems intentional. Isn't it? If so, what we'd need to discuss is a change to the UI design with a long history. @geekosaur thinks this was intentional, but not well thought-out, so maybe we would need a design change, e.g., 3 different ways of resolving, depending on a context. I am, OTOH, afraid of complicating things.

Let's discuss. At least, if we keep this unchange, the error messages and the documentation could surely us an improvement.

[sergv]

Current behavior seems more like an implementation artifact for me rather than intentional choice.

With my user hat on when I have dependent package bar and do cabal --extra-include-dirs=foo in my main package I don't expect the command to make cabal look for bar/foo directory in the dependency. Relative paths on command line are usually resolved against where I'm executing a command, that's how e.g. ls works and all the other command-line tools, so as a user I would expected cabal to behave the same.

Similar thinking applies when I specify relative path in cabal.project, like extra-include-dirs: foo, I expect it to be relative to the cabal.project file. Having relative paths resolve in each dependency with different results, i.e. each dependent package gets its own <pkg-N>/foo directory to check, seems like a bad idea. I'd say let all the dependencies get the same absolute paths obtained by resolving against cabal.project.

It's true that relative paths on command line and in cabal.project would resolve according to different rules. Perhaps it would make sense to unify the rules and resolve command-line paths against cabal.project too, seeing it as the cabal's gravitation center for a Haskell project. Still, maybe command-line should remain command-line and work like about all the other command-line tools in existence?

[geekosaur]

That's more or less what I think, but there's one additional consideration: there isn't always a project file (or project), and both of those fields can appear in *.cabal files. Which would imply that in that case they resolve relative to the cabal file they appear in.

There's also the question of what existing uses break if any of this changes, especially the cabal file part. That said, I honestly have trouble imagining relative paths being used anywhere but the command line because there's no guarantee they'd work on other systems, so neither the cabal file nor a git project repo would be portable with relative paths.

[sergv]

Fields coming from cabal files should be ok, I think they should work as before and should not be changed. If there's no cabal.project then only command-line interface remains which still has current working directory so previously outlined considerations apply.

My concern is only about extra-include-dirs and the like (e.g. extra-lib-dirs, static lib dirs and framework dirs) that come from 1. command-line and 2. cabal.project. Currently they work exactly the same as corresponding directories specified in cabal file, namely: each cabal file in a project gets its config and cabal-install just appends received paths there as-is.

I'm arguing that paths originating from command line and cabal.project should resolve relative paths before passing them to each cabal file's config. This will make packages in a project to search for e.g. include files in the same existing directory that user specified, rather than pretend that each cabal file specified the directory independently on its own and will attempt looking for includes in non-existing directories user did not actually mean.