Initially this is just used for getting builtin option default values,
but it could be extended to show the default value of an option in the
summary and in the introspection API.
self.post is only ever appended to on the right hand. However,
it is then reversed twice in flush_pre_post(), by using "for a in
reversed.post()" and appendleft() within the loop. It would be tempting
to use appendleft() in __iadd__ to avoid the call to reversed(), but that
is not a good idea because the loop of flush_pre_post() is part of a slow
path. It's rather more important to use a fast extend-with-list-argument
in the fast path where needs_override_check if False.
For clarity, and to remove the temptation, make "post" a list instead
of a deque.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Unless an argument is marked as Dedup.OVERRIDDEN, pre_flush_set and
post_flush_set will always be empty and the loops in flush_pre_post()
will not be doing anything interesting:
for a in self.pre:
dedup = self._can_dedup(a)
if a not in pre_flush_set:
# This just makes new a copy of self.pre
new.append(a)
if dedup is Dedup.OVERRIDDEN:
# this never happens
pre_flush_set.add(a)
for a in reversed(self.post):
dedup = self._can_dedup(a)
if a not in post_flush_set:
# Here self.post is reversed twice
post_flush.appendleft(a)
if dedup is Dedup.OVERRIDDEN:
# this never happens
post_flush_set.add(a)
new.extend(post_flush)
In this case it's possible to avoid expensive calls and loops, instead
relying as much on Python builtins as possible. Track whether any options
have that flag and if not just concatenate pre, _container and post.
Before:
ncalls tottime cumtime
45127 0.251 4.530 arglist.py:142(__iter__)
81866 3.623 5.013 arglist.py:108(flush_pre_post)
76618 3.793 5.338 arglist.py:273(__iadd__)
After:
35647 0.156 0.627 arglist.py:160(__iter__)
78998 2.627 3.603 arglist.py:116(flush_pre_post)
73774 3.605 5.049 arglist.py:292(__iadd__)
The time in __iadd__ is reduced because it calls __iter__, which flushes
pre and post.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
"Inline" CompilerArgs.__iter__() into CompilerArgs.__init__(), so that
replace list(Iterable) is replaced by the much faster list(List).
Before:
ncalls tottime cumtime
19268 0.163 3.586 arglist.py:97(__init__)
After:
ncalls tottime cumtime
18674 0.211 3.442 arglist.py:97(__init__)
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
We don't actually allow anything except elementry types (`str`, `bool`,
`int`, `array[str]`) here, so we can tighten the typing. This in turn
helps us to simplify the typing in environment.py
OptionKey objects are used extensively. We want them with a simple API,
but they also need to be optimized to not compromise meson performances.
Since this is an immutable object, it is possible to cache the
OptionKey object creation. We need to do it using the __new__
to make the caching mechanism transparent.
Fixes#14245
Trying to chmod a symlink results in trying to chmod the file or
directory it points to, not the symlink itself which has no
permissions. Either a symlink points to within the tree we're making
writable in which case it'll be handled eventually by os.walk() or
it points outside of the tree we're making writable in which case
we don't want to touch it. Let's avoid touching files outside of the
tree by simply skipping symlinks in _make_tree_writable().
Store both a full version with the nightly and beta suffixes, and the
version as just X.Y.Z. This sort of distinction is why full_version
exists.
This fixes an issue with the bindgen module where we pass invalid
version of X.Y.Z-beta and X.Y.Z-nightly.
Which has both changed the error message and relaxed the check. In
theory we wouldn't hit this unless you have a very old bindgen and a
very new rustc.
When running tests with `--interactive` we don't redirect stdin, stdout
or stderr and instead pass them on to the user's console. This redirect
causes us to hang in case the test in question needs parsing, like it is
the case for TAP output, because we cannot read the process's stdout.
Fix this hang by not parsing output when running in interactive mode.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
When running tests in interactive mode then the standard file streams
will remain connected to the executing terminal so that the user can
interact with the tests. This has the consequence that Meson itself does
not have access to those streams anymore, which is problematic for any
of the test types that require parsing, like for example with the TAP
protocol. This means that Meson is essentially flying blind in those
cases because the test result cannot be determined by parsing the exit
code of the test, but can only reliably be derived from the parsed
output.
One obvious solution to this problem would be to splice the test output
so that both Meson and the user's terminal have access to it. But when
running in interactive mode it is quite likely that the test itself will
actually be driven from the command line, and the chance is high that
the resulting data on stdout cannot be parsed as properly anymore. This
is for example the case in the Git project, where interactive mode is
typically used to drop the user into a shell or invoke a debugger.
So continuing to treat the output as properly formatted output that can
be parsed is likely a dead end in many use cases. Instead, we introduce
a new "IGNORED" test result: when executing tests in interactive mode,
and when the test type indicates that it requires parsing, we will not
try to parse the test at all but mark the test result as ignored
instead.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Move the `console_mode` property into the TestRun Class. This will be
required by a subsequent commit where we start to ignore test results
for parsed interactive tests.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
After a test run finishes we print a summary that sums up test counts by
type, e.g. failed or skipped tests. In many cases though it is expected
that most of the counts will be zero, and thus the summary is needlessly
cluttered with irrelevant lines. This list of mostly-irrelevant results
will grow in a subsequent commit where we introduce "Ignored" test
results.
Prepare for this by filtering results. Instead of unconditionally
printing every result, we will now only print those results where we
have at least one counted test. The exception is "Ok:" and "Fail:",
which will always be printed.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Executing tests can take a very long time. As an example, the Git test
suite on Windows takes around 4 hours to execute. The Git project has
been working around the issue by splitting up CI jobs into multiple
slices: one job creates the build artifacts, and then we spawn N test
jobs with those artifacts, where each test job executes 1/Nth of the
tests.
This can be scripted rather easily by using `meson test --list`,
selecting every Nth line, but there may be other projects that have a
similar need. Wire up a new option "--slice i/n" to `meson test` that
does implements this logic.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
In large repositories, transitive link dependency resolution using the current
recursive algorithm can result in enough duplicate calls to cause the full
system memory space to be used up.
This commit simplifies link dep resolution by converting the currently used
recursive algorithm to an iterative one that avoids performing work more than
once. If a target's direct dependencies have already been processed, that
target will not be processed again.
These changes result in multiple orders of magnitude of improvements to dep
resolution time and memory usage in the worst case.
Co-authored-by: Xavier Claessens <xavier.claessens@collabora.com>
The swift compiler does not update the modification time of output
object files when the code has not changed. Without this, Swift
targets may continuously be rebuilt.
When supplying -Wno-vla-larger-than to compiler.get_supported_arguments,
meson will inject -Wvla-larger-than as an argument which considered
invalid by GCC, as the converse argument is -Wvla-larger-than=<value>.
Just like CLikeCompiler._has_multi_arguments special-cases
-Wno-attributes=, do the same for -Wno-vla-larger-than.
Resolves: https://github.com/mesonbuild/meson/issues/14208
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This gives us a simpler way to annotate things returning an option
value, and gives us an easy single place to change that will affect
everywhere if the option value types are changed.
The `mesonlib.is_*` functions are not correct to use here, since they
are for the build machine, not the host machine. This means if the build
machine if Linux but the host is Haiku, then pkg-config files willb e
installed into $libdir/pkgconfig, instead of $prefix/develop/lib/pkgconfig
The fact that UserOption is generic is really an implementation detail,
not something to be used publicly. So by having an `AnyOptionType`
alias, we can get better type checking, as can be seen by the patch as a
whole. One of the big fixes it replace open-coded equivlalents of
`MutableKeydOptionDictType` with that type alias.
Which wants a string, but then passes that string to a function that
wants an OptionKey, which means that we'll always miss the lookup in
BULITIN_DIR_NOPREFIX_OPTIONS, and return the default. The only case this
gets used we cast an OptionKey to str, and then pass that. So instead,
do the cast inside the function when necessary and pass the OptionKey
This reduces code, makes this clearer, and will be a nice step toward
the goal of getting everything typesafe.
For `UserIntegerOption` this makes a fairly nice, but substantial change
in that the old method used a tuple of `(min, value, max)` to pass to the
initializer, while all other types just passed `value`. The new
`UserIntegerOption` does the same, with keyword arguments for the min
and max values.
This saves a *tiny* bit of typing, but at the cost of requiring either
the current solution of throwing up our hands and saying "typing is too
hard, better to have bugs!" or an extensive amount of `TypedDict`s,
`overloads`, and a very new version of mypy. Let's get our type safety
back, even if it means writing a little bit more code.
This is an annoying issue to look at, because shutil.chown has (for our
purposes) three signatures:
```python
chown(path: int | AnyPathLike, uid: int | str, group: None = None) -> None: ...
chown(path: int | AnyPathLike, uid: None, group: int | str) -> None: ...
chown(path: int | AnyPathLike, uid: int | str, group: int | str) -> None: ...
```
This is a really difficult thing to guarantee from our code. We more or
less depend on being able to pass two parameters of `None | int | str`,
and it working. In our only caller we do ensure that at least one of the
variables is not None, but convincing mypy of this is more work than
it's worth.
This will show up in our CI only for python >= 3.13. An updated typshed
will make this show up for earlier versions, however. Pyright (which is
used by the VSCode Python extension) will spot this for earlier
versions. I have changed the code in such a way to make our CI turn
green.
Since we're going to split generate_rust_target() in multiple functions,
eliminate the only variable that spans large parts of it.
The cratetype ninja variable has been unused since Meson started invoking
rustc directly nine years ago (commit d952812b1, "Fix Rust to work with 1.3
release. Closes #277.", 2015-10-11).
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Add functions to RustCompiler() to account for differences
between rustc and "rustdoc --test": rustdoc always generates
a binary, does not support -g, and does not need --emit.
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Since the introduction of dep-info=... it is possible to move the depfile
away from the main build directory without using --out-dir. This is
less surprising, since the rules for mixing --emit, --out-dir and -o
are not really documented.
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Implement RustCompiler.build_rpath_args, so that more code can
be shared between non-Rust and Rust targets. Then, RustCompiler
can override it to convert the arguments to "-C link-arg=" and
add the rustup sysroot.
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
os.path.abspath of the target subdir is not guaranteed to give a sensible
answer; depending on what directory you run meson from, it could build
an absolute path relative the source directory or the build directory
or possibly neither one. In fact, using os.path.abspath is quite rare
in Meson's code and generally only done in code like
subdir = os.path.abspath(os.path.join(self.sourcedir, target['subdir']))
or
ndir1 = os.path.abspath(os.path.realpath(dir1))
While at it, don't use getattr unnecessarily, the cratetype is available.
Reviewed-by: Dylan Baker <dylan@pnwbakers.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
SharedModule (like Python extension modules) are loaded dynamically.
Therefore, they cannot be detected from executable dependencies, and
we must call `determine_windows_extra_paths` on them as well.
Also, those extra paths need to be computed even if the module or the
executable is not installed in the default location.
A Defaultable PerMachine has a type of `None | T`, in other words,
they have a base type of `PerMachine[None | T]`, but the purpose of
`PerMachine.default_missing()` is to get a `PerMachine[T]` from that
`PerMachine[None | T]`, therefore we should ensure that and annotate
that.
This enables generating Python bindings and linking against
`python3-embed` without resorting to later `install_name_tool` changes,
as the pkg-config module provided by Xcode doesn't say that
Python3.framework requires a rpath entry:
$ otool -L /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Python3
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Python3:
@rpath/Python3.framework/Versions/3.9/Python3 (compatibility version 3.9.0, current version 3.9.0)
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1933.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)
Some dependencies can bring include paths pointing to older macOS SDK's.
In this case, it was libffi pointing to SDK from 12.0. When the
Foundation
framework is imported in Swift, swiftc attempts to import the FFI module
from the most recent version of the SDK, which causes a compilation
error
because of conflicting definitions between the two SDK versions.
SwiftPM also had this problem:
https://github.com/swiftlang/swift-package-manager/pull/6772
The solution on our side is a simplified version of what SwiftPM did.
Let's naively look for .sdk paths in the compile args of our
dependencies
and replace them with the most recent one.
I included a test which is confirmed to fail without the workaround
added in this patch. This was not tested on anything else than macOS,
but I don't expect it to make the situation worse in any case.
On Windows, if you accidently add a space at the end of a file name, like
`files('myfile.txt ')`, the file is not reported as missing, because of
the normalization performed by the OS. However, ninja will reference it
with the trailing space, and will fail because the file does not exist.
See https://github.com/python/cpython/issues/115104 for reference.
For whatever reason Meson has always used None == <C Language>. This
doesn't make a lot of sense to me, but it's how things currently work,
and our dependency factories should handle that correctly.
It's not especially explanatory to say:
```
meson.build:357:34: ERROR: Automatic wrap-based subproject downloading is disabled
```
But if we instead say this:
```
ERROR: Subproject libsamplerate is buildable: NO
meson.build:357:34: ERROR: Automatic wrap-based subproject downloading is disabled
```
It becomes a lot clearer to casual inspection, why it failed. And it
matches the way we otherwise report errors for an unbuildable subproject
(configure errors).
Bug: https://github.com/jacktrip/jacktrip/issues/1380
It catches the exception message itself, but for multi-line exceptions
it may be worth print an error() as well as raising, to communicate
multiple bits of information.
When using the VS backend, this means that we get an actual `ERROR: ...`
printed during a successful run, which then breaks msbuild as msbuild
parses stdout of successful commands, regexes them for the word "error:"
and interprets that as... an error. So a meson project tests example
that uses testcase expect_error() and then successfully configures and
builds, fails to successfully `meson --internal regenerate`.
Sneak around this by doing our own pattern replace to evade msbuild.
There is probably a way to tell msbuild to stop doing this, but that
would require me understanding the vs backend well enough to patch the
xml it generates. No thanks...
Since this is optional, we should not accept that GCC is a valid ObjC or
G++ is a valid ObjC++ Compiler unless we've tested that they can
actually do a basic compile.
This requires fixing a number of tests that have broken assumptions. In
some cases I've split tests where issues with one language would hide
the other. It would be great if we had a competent test framework that
allowed subtests to skip, unfortunately we have python's unittest
instead. Because of that we can't avoid extra tests by use of subtests.
This is a very hot function, improve the memoization of the results by removing
an argument (that is almost always empty, in fact).
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
rpaths are calculated in bytes, and that's also how depfixer processes
them. We need to ensure that the ascii padding we use (bytes == unicode)
is the correct offset for the install rpath - build rpath (both
specified in unicode which then gets converted to bytes).
In the event of a unicode install_rpath, we can get into a situation
where the install rpath is longer than the padding, since we assumed
that the install_rpath was shorter than it actually is -- because we
counted the length in characters instead of the length in bytes.
This then broke installation for people who e.g. install into a prefix
inside their home directory, when their home directory contains
multibyte unicode characters.
Bug: https://gitlab.gnome.org/GNOME/gnome-builder/-/issues/2280
This inclusion was a misunderstanding on my part: this warning isn't generally
applicable to standard C (it prevents using double literals whatsoever since C
doesn't have a suffix for them), but exists to support a GNU C extension. It
also has no counterpart in clang. So, remove it, since
warning_level=everything doesn't include such things.
Expand the expression passed into cross_compute_int using the
preprocessor first and then try to evaluate the expanded expression
using the host machine compiler and test if the result is valid.
Co-authored-by: Charles Brunet <charles.brunet@optelgroup.com>
Add an exception to the disabler check to allow objects with a `get_variable`
method to not always pick a disabler if their arguments contain one. This
mimics the behaviour already in place for calls to function, which has a set
of excepted functions.
Closes#13717
Signed-off-by: Andrew McNulty <amcn102@gmail.com>
#13837 broke both_lib transitive deps, because the
`as_static` and `as_shared` functions return libraries
that still contain references to the other lib type.
Currently, the vulkan system dep detects its vulkan version by building
and running:
int main() {
printf("%i.%i.%i", VK_VERSION_MAJOR(VK_HEADER_VERSION_COMPLETE),
VK_VERSION_MINOR(VK_HEADER_VERSION_COMPLETE),
VK_VERSION_PATCH(VK_HEADER_VERSION_COMPLETE));
return 0;
}
this causes cross builds that do not have the possibility of running on
the build machine to evaluate the vulkan dependency with an 'Unknown'
version.
Instead of evaluating beforementioned piece of C code, the new
implementation will instead use cc.compute_int to evaluate the three
preprocessor macros.
This is relativly expensive for cross builds right now but further
optimizations can be made. See #13910 for more details.
In a3679a64e (programs: favor version numbers with dots, 2025-01-03) we
have changed the regex used to extract version numbers to favor dotted
versions. It was reported though that the regex doesn't match major
version numbers that start with multiple digits correctly. Fix this.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Two expensive parts of length_estimate() are executed for each target, but they are really
always the same. Cache them in __init__, there will always be more targets than rules in
cases where speed counts.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
_should_use_rspfile() is expensive due to the call to length_estimate().
Make it a lazy property so that it is only called once.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Version objects are created thousands of times by the check_version method of
decorators. Creation is inefficient, resulting in five calls to re.match()
that do not even use precompiled regex. The use of re.match() however is
fully redundant, as finditer() can provide the same information with a better
use of groupings. Do that and precompile the tokenization regex.
This saves about 3% in the "meson setup" run of QEMU.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Except for set_variable(), the variable name is certainly an identifier because it
comes from the parser; thus, the check is unnecessary. Move the regular expression
match to func_set_variable().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Without a version certain kinds of warnings will be suppressed, which is
bad.
I've picked 1.0 because it's pretty old, except for Rust where I've
maintained the 1.3.0 requirement
When using `find_program('perl')` we misdetect its version number:
Program perl found: YES 40 40 (/usr/bin/perl)
This is caused by Perl outputting the version information in a somewhat
weird format:
$ perl --version
This is perl 5, version 40, subversion 0 (v5.40.0) built for x86_64-linux-thread-multi
...
The problem here is that our version number detection picks the first
match of at one digit followed by at least one more digit and/or dot.
Consequently, as "40" matches that regular expression, we'd return its
value as the version number.
Naturally, the version number detection we perform is best-effort, only,
as there is no standardized format used by all programs. That being
said, we can do better here by first trying to match a dotted version
number so that we'd match the "5.40.0" string, not the "40". And given
that most projects use dotted version numbers this should be a strict
improvement in cases where we have multiple digits in-text. The old
behaviour continues to be used as a fallback though in case we weren't
able to match anything to not regress functionality.
The new regex also fixes another case: when the version information ends
with a trailing dot, like "foo version 1.2.3.", then we'd also have
matched that trailing dot. This can be for example the case when version
numbers end with ".rc1" or something similar. While we'd ideally include
such suffixes into the detected version, that issue is left for another
day.
Add a couple of tests.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Meson builds libraries before running clippy, thus all dependencies must be present
and clippy is run only for the warnings (instead, Cargo treats clippy as a separate
toolchain and builds everything). In Meson's case thus it is pointless to run
clippy whenever --cap-lints allow is included in the command line.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This adds --cap-lints allow, matching how Cargo builds them. In the case of
Cargo, this is only applied to non-path dependencies.
Without this change, clippy will complain about dependencies as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This is a better and more backwards-compatible way to disable all warnings,
compared to "-A warnings". The Rust RFC (https://rust-lang.github.io/rfcs/1193-cap-lints.html)
explains the rationale:
> We would very much like to be able to modify lints, however. For example
> rust-lang/rust#26473 updated the missing_docs lint to also look for missing
> documentation on const items. This ended up breaking some crates in the
> ecosystem due to their usage of #![deny(missing_docs)].
While at it, document that Rust deviates from the other languages in its
interpretation of warning_level=0.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This aims to bring the support of QML modules to meson, the goal is to
provide something similar to CMake `qt_add_qml_module` function provided by Qt
(see https://doc.qt.io/qt-6/qt-add-qml-module.html )
Fixes: #6988, #9683
in `func_install_data`, `rename` parameter is `ContainerTypeInfo(list, str)`
in `module/python.py`, rename is `None`
`rename` is stored in `build.Data` object under the type `T.List[str]`
Inconsistency in the original implementation of commit
79e2c52a15.
If an explicit list of targets is passed on the CLI, then that is passed
to rebuild_deps. If not, we pass every loaded test to rebuild_deps
instead. This means we cannot distinguish between "trying to run all
tests" and "trying to run specific tests". We then load all the deps for
all tests, and try to build them all as explicit arguments to the
underlying ninja.
There are two situations where this falls flat:
- given underspecified deps
- given all (selected?) tests legitimately happen to have no
dependencies
In both cases, we calculate that there are no deps to rebuild, we run
ninja without any targets, and this invokes the default "all" rule and
maybe builds a few thousand targets that this specific test run does not
need.
Additionally, in large projects which define many tests with many
dependencies, we could end up overflowing ARG_MAX when processing *all*
tests.
Instead, pass no tests to rebuild_deps. We then specially handle this by
directly running the relevant ninja target for "all test deps", which is
overall more elegant than specifying many many dependencies by name.
Given a subset of tests to guarantee the freshness of, we instead skip
running ninja at all if there are indeed no test dependencies.
On Windows, if the native file contains a path without the
extension, the executable may be found, but custom target would
fail because that path does not exist.
Internal dependency names were generated from object id. This cause
problem when objects are copied, especially when generating partial
dependency, or when extracting shared or static dependencies from
both_library, because dependency names in target and dependencies
introspection files become unrelated.
This fixes that by generating the dependency name from the internal id,
and by using that base name when generating partial dependencies.
With
CC=ccache meson ...
meson crashes with
[...]
File "/usr/lib/python3.10/site-packages/mesonbuild/compilers/detect.py", line 364, in _detect_c_or_cpp_compiler
compiler_name = os.path.basename(compiler[0])
IndexError: list index out of range
Improve this by throwing an EnvironmentException to fail gracefully when
no compiler is specified.
Fixes#9933Fixes#13589
Regexes can be surprisingly slow. This small change brings
ninja_quote() from 12 to 3 seconds when building QEMU.
Before:
ncalls tottime percall cumtime percall
3734443 4.872 0.000 11.944 0.000
After:
ncalls tottime percall cumtime percall
3595590 3.193 0.000 3.196 0.000
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
Do not reinvent it in NinjaBackend.determine_ext_objs(), so as to use
the recently added caching of the results of File.from_built_relative().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
get_target_generated_sources often calls File.from_built_relative on
the same file, if it is used by many sources. This is a somewhat
expensive call both CPU- and memory-wise, so cache the creation
of build-directory files as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
is_source() is called almost 900000 times in a QEMU setup. Together with
the previously added caching, this basically removes _determine_ext_objs()
from the profile when building QEMU.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
Almost exactly the same as how the dl dependency works. On certain
systems (like BSDs that use clang), stdatomic is provided by compiler-rt
and doesn't need a separate library explictly linked. On a typical
GNU/LINUX system, atomic is a separate library that must be explictly
found and linked against. So just add a builtin and system method for
these two use cases.
When followed by a comma, we can be absolutely sure that these are
argument prefixes, and will not consume the next argument to form
a single argument. Fixes spammy warnings on apple clang:
`ld: warning: duplicate -rpath 'build/dist/darwin_universal/arm64/lib/pkgconfig/../../lib' ignored`
Continuation from https://github.com/mesonbuild/meson/pull/13819
This was already done for dedup2_prefixes, also do it for
dedup1_prefixes, and move export-dynamic to dedup1_args, where it
belongs.
Also modify some comments around this to clearly distinguish
standalone argument matching and argument-prefix matching.
It's not the first time I run into an issue with an intentionally missing
exe_wrapper during cross compilation. In pretty much all the cases the project
I tried to build already had code available to not need one in the first place.
Print out what command was actually the culprit to make debugging this easier.
bindgen by default may output code that an older rustc cannot
successfully consume. To avoid this, we check if bindgen supports the
rustc version we're using, if so, and if the user didn't set the
`--rust-target` option, we will supply it to ensure that bindgen writes
out code our rustc can use. If it does not support that version
explicitly, we leave it at the default, assuming that our compiler
version is newer than bindgen.
the upstream behavior of configure_file in cmake parses both @VAR@ and ${VAR} and only supports disabling the bracket variables through the `@ONLY` argument, meson needs to follow this behavior for compatibility
The following log output:
Checking if "strnstr() available" : links: NO
now becomes:
Checking if "strnstr() available" links: NO
This is more consistent with the compiles() method.
`-fuse-ld=` is a driver option for selection of a linker; it shall not be
passed to a linker with `-Wl,`.
For the Microsoft compiler and linker, the options for the compiler and those
for the linker are separated by `/LINK`, which looks like `cl /cl-options ...
/link /link-options ...`. Formally, they are passed in the same command line.
When Clang is invoking the Microsoft linker or a Microsoft-style linker (that
is, LLD-LINK), every linker option has to prefixed by `-Wl,` or `-Xlink`.
Previously, using Clang-CL and LLD-LINK, given:
cc = meson.get_compiler('c')
assert(cc.has_link_argument('/LTCG'))
This code failed to detect the `/LTCG` option, because `-fuse-ld=lld` was
passed to the linker, as an invalid option:
Command line: `clang E:\lh_mouse\Desktop\t\build\meson-private\tmpg0221fee\testfile.c -o E:\lh_mouse\Desktop\t\build\meson-private\tmpg0221fee\output.exe -D_FILE_OFFSET_BITS=64 -O0 -Werror=implicit-function-declaration -Wl,-WX -Wl,/LTCG -Wl,-fuse-ld=lld` -> 4044
stdout:
LINK : warning LNK4044: unrecognized option '/fuse-ld=lld'; ignored
LINK : error LNK1218: warning treated as error; no output file generated
However, it should be noted that not all LINK options can be passed with
`-Wl,`. The `/subsystem:windows,6.1` option has a comma, which would be
converted to a space. Therefore, this option must be passed as
`-Xlinker -subsystem:windows,6.1`. This issue is not addressed in this commit.
Signed-off-by: LIU Hao <lh_mouse@126.com>
clippy-driver is not meant to be a general-purpose compiler front-end.
Since Meson can now provide natively the ability to invoke clippy,
raise a warning if someone uses it that way.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Similar to the "ninja scan-build" target for C, add a clippy internal
tool that runs clippy-driver on all crates in the project.
The approach used is more efficient than with "ninja scan-build", and
does not require rerunning Meson in a separate build directory; it
uses the introspection data to find the compiler arguments for the
target and invokes clippy-driver with a slightly modified command
line.
This could actually be applied to scan-build as well, reusing the
run_tool_on_targets() function.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This improves the handling of keyboard interrupt, and also makes it easy to
buffer the output and not mix errors from different subprocesses. This
is useful for clang-tidy and will be used by clippy as well. In addition,
the new code supports MESON_NUM_PROCESSES.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Right now, the clang-tidy and clang-format targets use the program default
and do not let b_colorout decide whether to colorize output.
However, the wrappers that run the tool are going to be changed to buffer
output, and that would disable colorization unconditionally. So pass
a --color option to the tools and use it when building the command line.
clang-format's -fcolor-diagnostics option simply does not work, and the
"right" (or at least working) option is --color which is undocumented.
--color is present all the way back to clang 10, but I digged into
clang-format's source code to figure out what's happening. The problem
is that -fcolor-diagnostics is a complete no-operation; in fact it is
a bool that is initialized to true. gdb shows:
(gdb) p ShowColors
$2 = {<llvm:🆑:Option> = {
... <llvm:🆑:opt_storage<bool, false, false>> = {Value = true, ... }, ...}
on entry to clang-format's main, meaning that specifying the option on
the command line does nothing at all.
To see how clang-format determines whether to use colors you need to look
at enters SMDiagnostic::print, which simply does
ColorMode Mode = ShowColors ? ColorMode::Auto : ColorMode::Disable;
showing once more that in fact the option cannot force-on the colors (
-fno-color-diagnostics instead works). Continuing in SMDiagnostic::print,
this RAII constructor would write the escape sequence to the terminal:
WithColor S(OS, raw_ostream::SAVEDCOLOR, true, false, Mode);
It ends up in WithColor::changeColor, which does
if (colorsEnabled())
OS.changeColor(Color, Bold, BG);
Digging further down, colorsEnabled() is where the Mode member is consulted:
bool WithColor::colorsEnabled() {
switch (Mode) {
case ColorMode::Enable:
return true;
case ColorMode::Disable:
return false;
case ColorMode::Auto:
return AutoDetectFunction(OS);
}
llvm_unreachable("All cases handled above.");
}
and the "AutoDetectFunction" is
static bool DefaultAutoDetectFunction(const raw_ostream &OS) {
return *UseColor == cl::BOU_UNSET ? OS.has_colors()
: *UseColor == cl::BOU_TRUE;
}
UseColor is controlled by the "--color" option, so if that option was
unset you go to OS.has_colors() even in the presence of -fcolor-diagnostics.
This has been around for over 5 years in clang-format, and it was present
even earlier, so use it in meson as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Even though the "targets" introspection info already includes the
command line arguments used to invoke the compiler, this is not
enough to correlated with the "compilers" introspection info and
get extra information from there.
Together with the existing "language" key, adding a "machine" key
is enough to identify completely an entry in the compilers info.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
It is useful to apply a limit to the number of processes even outside "meson test",
and specifically for clang tools. In preparation for this, generalize
determine_worker_count() to accept a variable MESON_NUM_PROCESSES instead of
MESON_TESTTHREADS, and use it throughout instead of multiprocessing.cpu_count().
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
import mintro and its attendant module dependency tree just
so we can programmatically get filenames which are documented
as a stable API in https://mesonbuild.com/IDE-integration.html.
Suggested-by: Eli Schwartz <eschwartz93@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
There is no need to create and look up a dictionary when MachineChoice is
an enum, and there is no need to create a PerMachine object on every
__setitem__ of another PerMachine object.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
In old versions of Vulkan SDK, VK_SDK_PATH environment variable was used instead
of VULKAN_SDK.
This patch check both environment variables is fallback mode.
Xcode treats this dict value as a space-separated string, any spaces in
the path will make the path invalid by splitting it into pieces.
Split out header path processing into a helper function.
Recently, it is possible to install Clang with Visual Studio Installer. By
default this Clang has a MSVC target, and invokes the Microsoft Linker; if
`-fuse-ld=lld` is specified, it will invoke LLD-LINK. Both linkers take
MSVC-style arguments, and take DEF files with `/DEF:<path>`.
Previously DEF files were passed in the GNU way, directly on the linker
command line like an object file, which caused errors like
lld-link: error: ..\my.def: unknown file type
While Clang-CL takes Unix-style options, it actually passes MSVC-style
options to LINK or LLD-LINK with `-Wl,`. There is already a check for both
linkers in `linker_to_compiler_args()`, so it's necessary to do the same
in `gen_vs_module_defs_args()`.
This commit closes https://github.com/mesonbuild/meson/issues/13988.
Signed-off-by: LIU Hao <lh_mouse@126.com>
Doctests have a slightly different output compared to what "protocol: rust"
supports:
running 2 tests
test ../doctest1.rs - my_func (line 7) ... ignored
test ../doctest1.rs - (line 3) ... ok
test result: ok. 1 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.12s
Add a little more parsing in order to accept this; a simple minded split()
fails to unpack the tuple. I plan to contribute an extension of the rust
module to invoke doctests, for now this allows running rustdoc --test with
"protocol: 'rust'" and get information about the subtests:
▶ 4/8 ../doctest1.rs:my_func:7 SKIP
▶ 4/8 ../doctest1.rs:3 OK
4/8 rust_unit_tests:doctests / rust doctest OK 0.28s 1 subtests passed
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
It is possible to run a container or chroot with one ABI on a CPU and
kernel that would normally have a different ABI, most commonly by
running a 32-bit container on a 64-bit CPU and kernel. When we do a
native build in such an environment, the build and host architectures
are both equal to the architecture of the container, and it is safe to
assume that we can run executables from that architecture, because if
we could not, we wouldn't be running Python successfully.
Until now, we have been handling this by adding explicit special
cases in `machine_info_can_run()` for each known-good combination of
the detected CPU and the host architecture: every x86_64 can run x86
binaries, and every mips64 is assumed to be able to run 32-bit mips
binaries. However, the equivalent would not be true on ARM systems: *most*
aarch64 CPUs can run arm binaries, but not all (according to Wikipedia,
ARM Cortex-A34 is an example of a purely 64-bit CPU that cannot execute
32-bit instructions).
Instead, assume that if we are doing a native build (not a cross build),
by definition we can run build-architecture executables, and since the
host architecture is equal to the build architecture during a native
build, this implies that we can run host-architecture executables too.
This makes the behaviour of `need_exe_wrapper()` consistent with
`meson.can_run_host_binaries()`, which in turn avoids `Compiler.run()`
failing with error message "Can not run test applications
in this cross environment" during native builds even though
`meson.can_run_host_binaries()` has previously succeeded.
Resolves: https://github.com/mesonbuild/meson/issues/13841
Signed-off-by: Simon McVittie <smcv@debian.org>
Using a rustup-based toolchain fails the "rust/2 sharedlib" test for me:
./prog: error while loading shared libraries: libstd-211931512faabf29.so: cannot open shared object file: No such file or directory
This happens because recent rustup places the standard library under
SYSROOT/lib/rustlib/TARGET/lib. Retrieve the right directory using
"--print target-libdir". This also provides a more accurate version
for rustc installed in /usr.
Before:
$ echo $(/usr/bin/rustc --print sysroot)/lib
/usr/lib
After:
$ /usr/bin/rustc --print target-libdir
/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib
While at it, cache the value to avoid repeated process invocation.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
If the same source is provided by multiple dependencies it was added
multiple times, as `added_sources` was only guarding against duplicates
within the same source list. This was not a problem with ninja, but it
triggers multiple sanity checks within xcode backend while attempting to
create multiple ids for the same file.
Rename `added_sources` to `seen_sources` as per reviewers request
The deps and orderdeps are sorted on output, so there is no need to preserve
their order.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
version_compare can take a few milliseconds. If you have a thousand object files
or a multiple thereof, it adds up.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Eli Schwartz <eschwartz93@gmail.com>
Any argument from the base target is copied to the test target, but some
keyword arguments for libraries are not available in executable.
Remove them.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
On Python 3.12, self.prefix.relative_to(self.prefix.drive) no longer
works and raises error like:
```
ValueError: 'C:/msys64/mingw64' is not in the subpath of 'C:'
```
We don't need an extra process in the process tree (specifically the
`meson devenv` process itself). Aside for the redundancy, it's actively
problematic if you abort a program in the devenv by using CTRL+C, as
meson itself will then emit a traceback (KeyboardInterrupt) which is
quite ugly.
It's not actually useful, we can just add the root meson.build file to
the depfiles when we translate the meson.build into ast. If there's a
provided ast then we don't go down that path anyway.
This was only used in a couple of places, and requires extra tracking to
ensure it is correct, while we already have `current_node.lineno`, which
is always accurate and up to date.
I have also fixed a potential strict-null issue by using a sentinel node
for `current_node`
Cargo sometimes adds new keys and Meson needs to gracefully handle
those. Currently, an unknown key will trigger an uncaught Python
exception, which is pretty much the worse case. With this change Meson
will instead issue a warning much like the one for unknown cpu
architectures.
See #13826 for the motivation for this change
`--html-nested` was added to gcovr in version 6.0 so passing
it to versions before this will result in gcovr complaining that
it doesn't recognise the argument.
Added a version check to ensure that we pass a recognised argument
for versions before 6.0
Fixes#13781
Add a check before using an `appleframeworks` to make sure it was found.
This fixes an exception in meson while encountering an optional dep for
the target.
This reverts commit 8242187eb0.
This same change was previously introduced in #12495 / commit
30ab9747ae and subsequently reverted
in #12672 / commit 2fbc7b5ce3
Reintroduced in #13819 but it's still the same old problem. To repeat
the problem:
This breaks at least:
- frameworks/17 mpi
- frameworks/30 scalapack
The problem is that openmpi's pkg-config emitted link arguments
includes:
```
-Wl,-rpath -Wl,/path/to/libdir
```
The deduplication logic in meson doesn't contain sufficient information
to tell when the compiler is passing an argument that requires values,
and definitely cannot tell when that argument is split across argv. But
for arguments that *can* do this, it is not possible to deduplicate a
single argument as standalone, because it is not standalone.
The argument for deduplicating rpath here was that if you have multiple
dependencies that all add the same rpath, the Apple ld64 emits a
non-fatal warning "duplicate -rpath ignored". Since this is non-fatal,
it's not a major issue. A major issue is when builds fatally error out
with:
```
FAILED: scalapack_c
cc -o scalapack_c scalapack_c.p/main.c.o -Wl,--as-needed -Wl,--no-undefined -Wl,--start-group /usr/lib64/libscalapack.so /usr/lib64/liblapack.so /usr/lib64/libblas.so -Wl,-rpath -Wl,/usr/lib64 -Wl,/usr/lib64 -Wl,--enable-new-dtags /usr/lib64/libmpi.so -Wl,--end-group
/usr/libexec/gcc/x86_64-pc-linux-gnu/ld: error: /usr/lib64: read: Is a directory
```
Today we have CI for this so the change actually caused our own unittest
CI to go red.
Dylan Baker