When pch are used for a target meson will make the compiler to include
the pre-compiled header. While this is useful, this needs to happen
before any other header has been included, otherwise:
1) we won't take advantage of pch for anything else previously included
2) gcc will just fail as it won't even try to look for a pre-compiled
header in this case [1]
This case can happen quite a easily when a dependency provides an
included header in its cflags.
As per this, split _generate_single_compile() in two phases, one is
responsible of initializing the compiler data, while the other is
defining commands for the context.
In this way, when pch can be used, we can insert the pch inclusion
earlier than any other provided by the target.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100462
Even if the command is not wrapped by meson, it could have been modified
to add java/mono interpreters. This fix potential inconsistency between
wrapped and unwrapped commands.
Our approach to unity builds with vala is broken, you cannot unify the
generated C files, as they contain duplicate symbols. We would need to
instead combine the files while they are still in their vala form, then
convert that to C and compile the unified C file.
This does not fix the linked issue, as this removed the ability to do
vala unity builds, but it does allow running vala with `--unity=on`.
Related: #5280
Replacements are already done by eval_custom_target_command() and must
be done BEFORE calling as_meson_exe_cmdline() anyway. replace_paths() is
still used by generators.
Make eval_custom_target_command() more readable by handling error in the
final else case instead of in the middle of elif.
Re-implement it in backend using the same code path as for
custom_target(). This for example handle setting PATH on Windows when
command is an executable.
Currently mesonlib does some import tricks to figure out whether it
needs to use windows or posix specific functions. This is a little
hacky, but works fine. However, the way the typing stubs are implemented
for the msvcrt and fnctl modules will cause mypy to fail on the other
platform, since the functions are not implemented.
To aleviate this (and for slightly cleaner design), I've split mesonlib
into a pacakge with three modules. A universal module contains all of
the platform agnositc code, a win32 module contains window specific
code, a posix module contains the posix specific code, and a platform
module contains no-op implementations. Then the package's __init__ file
imports all of the universal functions and all of the functions from the
approriate platform module, or the no-op versions as fallbacks. This
makes mypy happy, and avoids `if`ing all over the code to switch between
the platform specific code.
[why]
If we build and test a library we need to make sure that we find the
currently build library object first, before an older system installed
one.
This can be broken if the library in question is installed in a custom
path, and another library we depend on also is installed there.
[how]
Just move the rpath to the current build artifacts to the front.
Solves #8030.
Signed-off-by: Fini Jastrow <ulf.fini.jastrow@desy.de>
In Fortran it is common to use capital F in the suffix (eg. '.F90') if
the source file makes use of preprocessor statements. Such files should
probably be treated like all other fortran files by meson.
Case insensitivity for suffixes was already implemented several places
in meson before this. So most likely, the few places changed here were
oversights anyway.
There are still caveats here. Rust/cargo handles generated sources by
writing out all targets of a single repo into a single output directory,
setting a path to that via a build-time environment variable, and then
include those files via a set of functions and macros. Meson's build
layout is naturally different, and ninja makes working with environment
variables at compile time difficult.
Fixes#8157
we have two functions to do the exact same thing, and they're basically
implemented the same way. Instead, let's just use the BuildTarget one,
as it's more generally available.
This patches takes the options work to it's logical conclusion: A single
flat dictionary of OptionKey: UserOptions. This allows us to simplify a
large number of cases, as we don't need to check if an option is in this
dict or that one (or any of 5 or 6, actually).
I would have prefered to do these seperatately, but they are combined in
some cases, so it was much easier to convert them together.
this eliminates the builtins_per_machine dict, as it's duplicated with
the OptionKey's machine parameter.
These do not go into the command line, and therefore do not matter
for the purpose of avoiding unnecessary rebuilds after meson is
rerun. However, they complicate the task of finding differences
between build lines across meson reruns.
So take the easy way out and sort everything after | and ||.
With this change, there is absolutely no change in QEMU's 40000-line
build.ninja file after meson is rerun.
There are two bugs here, first is that we open coded the output args,
instead of using the compiler method. The second is that rust args are
not passed down to the backend invocation.
Using the std option, so now `rust_std=..` will work. I've chosen to use
"std" even though rust calls these "editions", as meson refers to
language versions as "standards", which makes meson feel more uniform,
and be less surprising.
Fixes: #5100
I've always found ninja reporting 'a meson_exe.py custom command'
unclear and confusing. Instead say we are invoking a custom command,
wrapped by meson, and why.
Don't stop generating STATIC_LINKER rules at the first MachineChoice
which isn't available (i.e. generate a STATIC_LINKER rule, even if
STATIC_LINKER_FOR_BUILD isn't needed)
Also warn about internal errors which lead to non-existent rule
references, rather than exploding with a KeyError.
This allows the NINJA environment variable to support all the Windows special
cases, especially allowing an absolute path without extension.
Based on a patch by Yonggang Luo.
Fixes: #7659
Suggested-by: Nirbheek Chauhan <nirbheek@centricular.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
extra_paths only matter for the host machine if using an exe_wrapper.
However, because CustomTarget.for_machine is always MachineChoice.HOST,
they were computed unnecessarily in as_meson_exe_cmdline.
Defer computation of extra_paths until after we have found out if the
custom target executable is really for the host or the build machine,
so that we can use exe_for_machine; for_machine then becomes unused
and can be removed.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Exceptions raised during subproject setup were ignored.
- Allow c_stdlib in native file, was already half supported.
- Eliminate usage of subproject variable name by overriding
'<lang>_stdlib' dependency name.
The lambda in NinjaBuildElement.write is quite expensive, totalling 0.3s
just to do a couple function calls. Since it is used just once, simply
inline it.
On a QEMU build, the total time spent in write from this series goes from
5.321s to 3.238s, though part of it can be attributed to previous patches.
Optimize the regular expression so that the variable expansion part
always ends up in group 1, and the trailer after the variable is discarded
in the same match. Do not use re.sub to remove braces, and do not bother
building the expanded command, just adjust the estimated length on the fly.
functools.reduce is extremely slow, so I am keeping ' '.join(chunk).
On a QEMU build the time spend in the function goes from 1.072s to 0.757s.
Use regular expressions to quickly weed out strings that require quoting
On a QEMU build the time spent in ninja_quote goes from 1.978s to 1.281s,
with str.replace being kicked completely out of the profile.
Without the parenthesis, the command evaluates to `[]` if
`use_llvm_cov` is `False`.
Also fix tests to actually check whether or not coverage reports are
generated.
Fixes https://github.com/mesonbuild/meson/issues/7553
otherwise we are getting errors like:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/mesonmain.py", line 131, in run
return options.run_func(options)
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/msetup.py", line 245, in run
app.generate()
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/msetup.py", line 159, in generate
self._generate(env)
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/msetup.py", line 215, in _generate
intr.backend.generate()
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/backend/ninjabackend.py", line 518, in generate
self.generate_coverage_rules()
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/backend/ninjabackend.py", line 991, in generate_coverage_rules
self.generate_coverage_command(e, [])
File "/usr/local/lib/python3.6/dist-packages/mesonbuild/backend/ninjabackend.py", line 975, in generate_coverage_command
for compiler in target.compilers.values():
AttributeError: 'RunTarget' object has no attribute 'compilers'
This extends the 109 generatecode test case to also define a test, so
coverage can really detect something.
Since the CompileArgs class already needs to know about the compiler,
and we really need at least per-lanaguage if not per-compiler
CompilerArgs classes, let's get the CompilerArgs instance from the
compiler using a method.
It's assumed that where we use DEPFILE in command or rspfile_content, it
can be quoted by quoting the ninja variable (e.g. $DEPFILE ->
'$DEPFILE')
This is nearly always true, but not for gcc response files, where
backslash is always an escape, even inside single quotes.
So this fails if the value of DEPFILE contains backslashes (e.g. a
Windows path)
Do some special casing, adding DEPFILE_UNQUOTED, so that the value of
depfile is not shell quoted (so ninja can use it to locate the depfile
to read), but the value of DEPFILE used in command or rspfile_content is
shell/response file quoted)
(It would seem this also exists as a more general problem with built-in
ninja variables: '$out' appearing in command is fine, unless one of the
output filenames contains a single quote. Although forbidding shell
metacharacters in filenames seems a reasonable way to solve that.)
(How does this even work, currently? Backslashes in the value of all
ninja variables, including DEPFILE were escaped, which protected them
against being treated as escapes in the gcc response file. And
fortunately, the empty path elements indicated by a double backslash in
the value of depfile are ignored when ninja opens that file to read it.)
Now that all command-line escaping for ninja is dealt with in the ninja
backend, escape_extra_args() shouldn't need to do anything.
But tests of existing behaviour rely on all backslashes in defines being
C escaped: This means that Windows-style paths including backslashes can
be safely used, but makes it impossible to have a define containing a C
escape.
We avoided having to get this right previously, as we'd always use a
response file if possible.
But this is so insane, I can't imagine it's right.
See also: subprocess.list2cmdline() internal method
In certain exotic configurations, the style of quoting expected in the
response file may not match that expected by the shell.
e.g. under MSYS2, ninja invokes commands via CreateProcess (which
results in cmd-style quoting processed by parse_cmdline or
CommandLineToArgvW), but gcc will use sh-style quoting in any response
file it reads.
Future work: The rspfile quoting style should be a method of the
compiler or linker object, rather than hardcoded in ninjabackend.
(In fact, can_linker_accept_rsp() should be extended to do this, since
if we can accept rsp, we should know the quoting style)
Rather than ad-hoc avoiding quoting where harmful, identify arguments
which contain shell constructs and ninja variables, and don't apply
quoting to those arguments.
This is made more complex by some arguments which might contain ninja
variables anywhere, not just at start, e.g. '/Fo$out'
(This implementation would fall down if there was an argument which
contained both a literal $ or shell metacharacter and a ninja variable,
but there are no instances of such a thing and it seems unlikely)
$DEPFILE needs special treatment. It's used in the special variable
depfile, so it's value can't be shell quoted (as it used as a filename
to read by ninja). So instead that variable needs to be shell quoted
when it appears in a command.
(Test common/129, which uses a depfile with a space in it's name,
exercises that)
If 'targetdep' is not in raw_names, test cases/rust all fail.
We need to count rsp and non-rsp references separately, which we need to
do after build statement variables have been set so we can tell the
difference, which introduces a bit of complexity.
Writing rsp files on Windows is moderately expensive, so only use them
when the command line is long enough to need them.
This also makes the output of 'ninja -v' useful more often (something
like 'cl @exec@exe/main.c.obj.rsp' is not very useful if you don't have
the response file to look at)
For a rule where using a rspfile is possible, write rspfile and
non-rspfile versions of that rule. Choose which one to use for each
build statement, depending on the anticpated length of the command line.
When classifying generated sources, we were treating gir/typelib files
generated by gobject-introspection as headers. This is bad because it
serializes the build by adding order-only dependencies to every target
even though sources will never actually use them for anything.
Treat them as libraries, which is somewhat more accurate.
We do not need to *always* rebuild generated sources when a generated
header changes. We will get that information from the compiler's
dependency file, and ninja will track it for us. This is exactly the
same as static sources.
However, we do need an order-only dependency on all generated headers,
because we cannot know what headers will be needed at compile time
(which is when the compiler's dependency file is generated).
This fixes spurious rebuilds and relinking in many cases.
A current rather untyped storage of options is one of the things that
contributes to the options code being so complex. This takes a small
step in synching down by storing the compiler options in dicts per
language.
Future work might be replacing the langauge strings with an enum, and
defaultdict with a custom struct, just like `PerMachine` and
`MachineChoice`.
This make relative pathes shorter an too give a chance to
de-duplicate -isystem flags just like -I flags.
Fix common test case 203 for OSX build host too
When a source file for a library is changed without adding new extern
symbols, only that library should be rebuilt. Nothing that uses it
should be relinked.
Along the way, also remove trailing `.` in all Ninja rule
descriptions. It's very confusing to see messages like:
```
Linking target mylib.dll.
```
It's confusing that the period at the end of that is not part of the
filename. Instead of removing that period manually in the tests (which
feels wrong!) just don't print it at all.
We actually use this while linking on Windows, and hence we need to
extract symbols from this file, and not the DLL.
However, we cannot pass it instead of the DLL because it's an optional
output of the compiler. It will not be written out at all if there are
no symbols in the DLL, and we cannot know that at configure time. This
means we cannot describe it as an output of any ninja target, or the
input of any ninja target. We must pass it as an argument without
semantic meaning.
This is more correct, and forces the target(s) to be rebuilt if the
PDB files are missing. Increases the minimum required Ninja to 1.7,
which is available in Ubuntu 16.04 under backports.
We can't do the same for import libraries, because it is impossible
for us to know at configure time whether or not an import library will
be generated for a given DLL.
This is similar to what we currently do for scan-build except there is
no environment variable to choose a specific clang-format to run. If an
environment variable is needed for better control, we can add it later.
Detect scan-build the same way when trying to launch it and when
generating the target.
The detection method is:
1. look within SCANBUILD env variable
2. shutil.which('scan-build')
3. *on non-linux platforms only*: go through all the possible
name candidates and test them individually.
The third step is added following this comment
https://github.com/mesonbuild/meson/pull/5857#issuecomment-528305788
However, going through a list of all the possible candidates is neither
easily maintainable nor performant, and is therefore skipped on
platforms that should not require such a step (currently, only Linux
platforms).
This is a follow-up to the issue raised by @lantw44 during PR:
https://github.com/mesonbuild/meson/pull/5857
Jussi Pakkanen