Dependencies is already a large and complicated package without adding
programs to the list. This also allows us to untangle a bit of spaghetti
that we have.
All changes were created by running
"pyupgrade --py3-only --keep-percent-format"
and committing the results. I have not touched string formatting for
now.
- use set literals
- simplify .format() parameter naming
- remove __future__
- remove default "r" mode for open()
- use OSError rather than compatibility aliases
- remove stray parentheses in function(generator) scopes
Rather than having to manually build the locale aware man paths with
`install_data('foo.fr.1', install_dir: join_paths(get_option('mandir'), 'fr', 'man1'), rename: 'foo.1')`
Support doing
`install_man('foo.fr.1', locale: 'fr')`
By default all subprojects are installed. If --skip-subprojects is given
with no value only the main project is installed. If --skip-subprojects
is given with a value, it should be a coma separated list of subprojects
to skip and all others will be installed.
Fixes: #2550.
run_target() does some variable substitutions since 0.57.0. This is a
new behavior, and undocumented, caused by sharing more code with
custom_target(). More consistency is better, so document it now.
custom_target() was doing variable substitution in the past, because it
shared some code with generator(), but that was undocumented. Some
refactoring in 0.57.0 caused it to not replace @CURRENT_SOURCE_DIR@,
@SOURCE_DIR@, and @BUILD_DIR@ anymore. This patch adds back
@CURRENT_SOURCE_DIR@ and document it. It does not add back @SOURCE_DIR@
because it is duplicate with @SOURCE_ROOT@ that has a better name. Also
do not add back @BUILD_DIR@ which is duplicate of @PRIVATE_DIR@, and
not @BUILD_ROOT@ surprisingly, adding to the confusion.
Even if the command is not wrapped by meson, it could have been modified
to add java/mono interpreters. This fix potential inconsistency between
wrapped and unwrapped commands.
Our approach to unity builds with vala is broken, you cannot unify the
generated C files, as they contain duplicate symbols. We would need to
instead combine the files while they are still in their vala form, then
convert that to C and compile the unified C file.
This does not fix the linked issue, as this removed the ability to do
vala unity builds, but it does allow running vala with `--unity=on`.
Related: #5280
Replacements are already done by eval_custom_target_command() and must
be done BEFORE calling as_meson_exe_cmdline() anyway. replace_paths() is
still used by generators.
Make eval_custom_target_command() more readable by handling error in the
final else case instead of in the middle of elif.
Re-implement it in backend using the same code path as for
custom_target(). This for example handle setting PATH on Windows when
command is an executable.
It is common, at least in GNOME projects, to have scripts that must be
run only in the final destination, to update system icon cache, etc.
Skipping them from Meson ensures we can properly log that they have not
been run instead of relying on such scripts to to it (they don't
always).
On Windows this would fail because of missing DLL:
```
mylib = library(...)
exe = executable(..., link_with: mylib)
meson.add_install_script(exe)
```
The reason is on Windows we cannot rely on rpath to find libraries from
build directory, they are searched in $PATH. We already have all that
mechanism in place for custom_target() using ExecutableSerialisation
class, so reuse it for install/dist/postconf scripts too.
This has bonus side effect to also use exe_wrapper for those scripts.
Fixes: #8187
The /ZI flag adds in "Edit and Continue" debug information, which will
cause massive slowdown. It is not a flag that we should be adding by
default to debug builds.
/Zi will still be added.
This adds enough type annotations for InstallData and friends to make
minstall happy. There is also a small change in that I've replaced the
List[List] with List[Tuple], as tuples are more appropraite data types
for the information (fixed length, position matters, different types at
different indexes)
Rust has it's own built in unit test format, which is invoked by
compiling a rust executable with the `--test` flag to rustc. The tests
are then run by simply invoking that binary. They output a custom test
format, which this patch adds parsing support for. This means that we
can report each subtest in the junit we generate correctly, which should
be helpful for orchestration systems like gitlab and jenkins which can
parse junit XML.
This patches takes the options work to it's logical conclusion: A single
flat dictionary of OptionKey: UserOptions. This allows us to simplify a
large number of cases, as we don't need to check if an option is in this
dict or that one (or any of 5 or 6, actually).
I would have prefered to do these seperatately, but they are combined in
some cases, so it was much easier to convert them together.
this eliminates the builtins_per_machine dict, as it's duplicated with
the OptionKey's machine parameter.
If static_library is used as a convenience library (e.g. for link_whole)
it should in principle not need position independent code.
However, if the executables that the libraries is linked to are PIE,
the non-PIC objects in the static library will cause linker errors.
To avoid this, obey b_pie for static libraries if either b_staticpic=false
or they use "pic: false".
Without this patch, QEMU cannot use b_staticpic, which causes a slowdown
on some QEMU benchmarks up to 20%.
- Fixed using debug and optimization built-in options in MSVC.
- Fixed that VS backend does not create pdb files in release mode.
VS implicitly adds the debug fields if left out.
- Fix that it is possible to add debug info with ninja backend with
optimizations.
I've always found ninja reporting 'a meson_exe.py custom command'
unclear and confusing. Instead say we are invoking a custom command,
wrapped by meson, and why.
This reverts commit 8ee1c9a07a.
No rationale was given for this change prior to merging. After the fact
it was described as desired by distro packagers, however as a distro
packager I believe this commit hurts me.
From a distro packaging perspective, we primarily care about one thing:
previously building code should still build. Hence, -Werror is bad for
our use case. meson handles this via -D werror which is different from
-D warning_level and as long as the former is disabled, warnings in the
build do not cause harm (but may attract review attention for upstream
to fix).
buildtype is a completely unrelated concern, and the intention for
=plain is to disable debug or optimization settings that result in
codegen differences and thus different built artifacts. This must not
happen in distro builds because the *distributed programs* should
conform to policy settings.
Unfortunately, completely disabling warnings happens silently, and
cannot be overridden even if you really, really believe you know what
you're doing. It is thematically broken, since use of
add_project_arguments() to add more -W flags is not likewise ignored.
But if you try to add -Wall in that manner, meson will lecture you to
use warning_level which you cannot do. And if you have custom warning
flags which depend on options enabled by judicious use of -Wall via
default_options: 'warning_level=1', then you end up with generated
warnings complaining about your command line rather than your code, such
as:
cc1: warning: ‘-Wformat-y2k’ ignored without ‘-Wformat’ [-Wformat-y2k]
cc1: warning: ‘-Wformat-extra-args’ ignored without ‘-Wformat’ [-Wformat-extra-args]
cc1: warning: ‘-Wformat-zero-length’ ignored without ‘-Wformat’ [-Wformat-zero-length]
cc1: warning: ‘-Wformat-contains-nul’ ignored without ‘-Wformat’ [-Wformat-contains-nul]
cc1: warning: ‘-Wformat-security’ ignored without ‘-Wformat’ [-Wformat-security]
which then break the build with -Werror.
Throughout all this, a buildtype of "plain" does *not* disable -D
werror=true, which is the part where distro builds actually break down!
Users who both wish to disable debug/optimization codegen, *and* disable
warning commentary, are encouraged to do so by doing both, not by doing
one and having the other be assumed.
Fixes#7399
Add the ids of any target that needs to be rebuilt before running the
tests as computed by the backend, to the introspection data for tests and benchmarks.
This also includes anything that appears on the test's command line.
Without this information, IDEs must update the entire build before running
any test. They can now instead selectively build the test executable
itself and anything that is needed to run it.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
extra_paths only matter for the host machine if using an exe_wrapper.
However, because CustomTarget.for_machine is always MachineChoice.HOST,
they were computed unnecessarily in as_meson_exe_cmdline.
Defer computation of extra_paths until after we have found out if the
custom target executable is really for the host or the build machine,
so that we can use exe_for_machine; for_machine then becomes unused
and can be removed.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
- Exceptions raised during subproject setup were ignored.
- Allow c_stdlib in native file, was already half supported.
- Eliminate usage of subproject variable name by overriding
'<lang>_stdlib' dependency name.
Since -Wl,-rpath= is not the only valid rpath ldflags syntax we
need to try and match all valid rpath ldflags.
In addition we should prevent -Wl,--just-symbols from being used to
set rpath due to inconsistent compiler support.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Fixup for b4b1a2c5a1.
A warning would be printed for any rule with multiple outputs, for
example:
WARNING: custom_target 'coredump.conf.5' has more than one output! Using the first one.
WARNING: custom_target 'dnssec-trust-anchors.d.5' has more than one output! Using the first one.
WARNING: custom_target 'halt.8' has more than one output! Using the first one.
Fixes https://github.com/systemd/systemd/issues/16461.
Since the CompileArgs class already needs to know about the compiler,
and we really need at least per-lanaguage if not per-compiler
CompilerArgs classes, let's get the CompilerArgs instance from the
compiler using a method.
This is needed in the case where a custom_target directly depends on a
shared library, and somehow loads it.
(Specifically this can be the case with gtkdoc, when it invokes
gtkdoc-scangobj, which will build and run it's own code to load a shared
library, to introspect it)
Handle command arguments which contain multiple substitutions correctly
in Backend.eval_custom_target_command()
In particular, gnome.gtkdoc() makes arguments of the form '--cflags
-I@SOURCE_ROOT@ -I@BUILD_ROOT' (where these arguments are then passed
down to a compiler invocation)
Normally, these are subsequently made right by
NinjaBackend.replace_paths(), but if Backend.as_meson_exe_cmdline()
decides that the command needs to be pickled, that doesn't happen.
(Although having two places where this substitution might happen smells
really bad)
Now that all command-line escaping for ninja is dealt with in the ninja
backend, escape_extra_args() shouldn't need to do anything.
But tests of existing behaviour rely on all backslashes in defines being
C escaped: This means that Windows-style paths including backslashes can
be safely used, but makes it impossible to have a define containing a C
escape.
Otherwise a wrapper script which takes an executable as an argument will
mistakenly run when that executable is cross compiled. This does not
wrap said executable in an exe_wrapper, just skip it.
Fixes#5982
If an executable is passed as an argument to a script in the build
directory that it resides in then it will not execute (on *nix) due to a
lack of ./. Ie, `foo` must be called as `./foo`. If it is called from a
different directory it will work. Ie `../foo` or `bar/foo`.
Fixes#5984
Gtest can output junit results with a command line switch. We can parse
this to get more detailed results than the returncode, and put those in
our own Junit output. We basically just throw away the top level
'testsuites' object, then fixup the names of the tests, and shove that
into our junit.
A current rather untyped storage of options is one of the things that
contributes to the options code being so complex. This takes a small
step in synching down by storing the compiler options in dicts per
language.
Future work might be replacing the langauge strings with an enum, and
defaultdict with a custom struct, just like `PerMachine` and
`MachineChoice`.
When a static library link_whole to a bunch of other static libraries,
we have to extract all their objects recursively. But that could
introduce duplicated objects. ar is dumb enough to allow this without
error, but once the resulting static library is linked into an
executable or shared library, the linker will complain about duplicated
symbols.
* Do not strip static archives
Stripping static archives without more fine-grained options (e.g. `-g`)
leads to failures such as
ld: libfoo.a: error adding symbols: archive has no index; run ranlib to add one
because GNU strip removes *every* symbol in a static archive by default.
Given that static archives are not final build artifacts (unlike
executables and shared libraries), stripping them gains little and only
causes more edge case failures.
* Gentoo's portage only strips debug information:
86f211e3a5/bin/estrip (L322)
* Fedora also only strips debug information:
e9c13c6565/scripts/brp-strip-static-archive (L18)
* Debian also only does some very light stripping:
72ed1d3261/dh_strip (L374)Fixes#4138
* Add test case for static archive stripping
* backends/vs: Only set platform_toolset if it isn't already set
* interpreter: set backend up after the compiler
Otherwise we won't be able to check which VS toolchain to use.
* docs/using-visual-studio: wrap lines
* docs: recommend the py launcher instead of python3 for windows
* set backend.environment when building a dummy version
* backends/vs: Add support for clang-cl with vs2017 and vs2019 backends
* backends/vs: Add support for ICL (19.x) with vs2015 and vs2017 backends
"exe.is_cross and exe.needs_exe_wrapper" is the same condition under which
meson chooses whether to include the exe_wrapper. meson_exe has an assertion
for that, but now that meson_exe does not need anymore exe.is_cross,
we can simplify the code if we just "trust" meson to do the right thing.
Remove both fields from ExecutableSerialisation and just test the presence
of the wrapper, and also remove the executable basename which is only
used to "beautify" an assertion failure.
Move the magic to execute jar and .exe files from "meson --internal exe"
to the backend, so that "ninja -v" shows more clearly what is happening.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
If meson_exe is only being used to capture the output of the command,
we can skip going through a pickled ExecutableSerialization object.
This makes "ninja -v" output more useful.
Return the command line from serialize_executable, which is then
renamed to as_meson_exe_cmdline. This avoids repeating code that
is common to custom targets and generators.
There are two problems with this:
- It has false positives when the code that trips it is conditional and
no run on cross.
- It confuses users who never wrote any `native` flags and don't care
about cross.
Fixes#5509
In most cases instead pass `for_machine`, the name of the relevant
machines (what compilers target, what targets run on, etc). This allows
us to use the cross code path in the native case, deduplicating the
code.
As one can see, environment got bigger as more information is kept
structured there, while ninjabackend got a smaller. Overall a few amount
of lines were added, but the hope is what's added is a lot simpler than
what's removed.
the problem here is, that get_custom_target_provided_libraries iterated
over all generated sources of a target. In each output we check if this
is a library or not. In projects like EFL we have added a lot of
generated target to many different targets, so the iterating of the
output is rather consistent, with this commit we drop from 19% of the
time spending in get_custom_target_provided_libraries down to 3.51%.
This avoids the duplication where the option is stored in a dict at its
name, and also contains its own name. In general, the maxim in
programming is things shouldn't know their own name, so removed the name
field just leaving the option's position in the dictionary as its name.
Also, we should at some point decide whether we're going to use American
spelling (serialize/serialization) or British (serialise/serialisation)
because we have both, and both is always worse than one or the other.
Currently C++ inherits C, which can lead to diamond problems. By pulling
the code out into a standalone mixin class that the C, C++, ObjC, and
Objc++ compilers can inherit and override as necessary we remove one
source of diamonding. I've chosen to split this out into it's own file
as the CLikeCompiler class is over 1000 lines by itself. This also
breaks the VisualStudio derived classes inheriting from each other, to
avoid the same C -> CPP inheritance problems. This is all one giant
patch because there just isn't a clean way to separate this.
I've done the same for Fortran since it effectively inherits the
CCompiler (I say effectively because was it actually did was gross
beyond explanation), it's probably not correct, but it seems to work for
now. There really is a lot of layering violation going on in the
Compilers, and a really good scrubbing would do this code a lot of good.
Instad of having special casing of threads in the backends and
everywehre else, do what we did for openmp, create a real
dependency. Then make use of the fact that dependencies can now have
sub dependencies to add threads.
Currently we specialcase OpenMP like we do threads, with a special
`need_openmp` method. This seems like a great idea, but doesn't work
out in practice, as well as it complicates the opemp
implementation. If GCC is built without opemp support for example, we
still add -fopenmp to the the command line, which results in
compilation errors.
This patch discards that and treats it like a normal dependency,
removes the need_openmp() method, and sets the compile_args attributes
from the compiler.
Fixes#5115
Instead use coredata.compiler_options.<machine>. This brings the cross
and native code paths closer together, since both now use that.
Command line options are interpreted just as before, for backwards
compatibility. This does introduce some funny conditionals. In the
future, I'd like to change the interpretation of command line options so
- The logic is cross-agnostic, i.e. there are no conditions affected by
`is_cross_build()`.
- Compiler args for both the build and host machines can always be
controlled by the command line.
- Compiler args for both machines can always be controlled separately.
A custom_target, if install is set to true, will always be built by
default even if build_by_default is explicitly set to false.
Ensure that this does not happen if it's set explicitly. To keep
backward compatibility, if build_by_default is not set explicitly and
install is true, set build_by_default to true.
Fixes#4107
Building a cross compiler (`build == host != target`) is not cross
compiling. As such, it doesn't make sense to handle it under
`is_cross_build`.
(N.B. Building a standard library for a cross compiler would require
cross compiling, but Meson has support to do such a thing as part of a
compiler build currently.)
Handle clang's cl or clang-cl being in PATH, or set in CC/CXX
Future work: checking the name of the executable here seems like a bad idea.
These compilers will fail to be detected if they are renamed.
v2:
Update compiler.get_argument_type() test
Fix comparisons of id inside CCompiler, backends and elsewhere
v3:
ClangClCPPCompiler should be a subclass of ClangClCCompier, as well
Future work: mocking in test_find_library_patterns() is effected, as we
now test for a subclass, rather than self.id in CCompiler.get_library_naming()
This reduces the build time about 2 sec. The result itself is not hard
to calculate. However, persistent join calls with the same 2 strings are
not that usefull. This also caused about 600'000 calls to
get_target_dir, we are now down to 60'000 calls form this function to
get_target_dir.
Remove the code responsible for implicitly compressing manpages as .gz
files. It has been established that manpage compression is a distro
packager's task, with existing distros already having their own
implementations of compression.
Fixes#4330
If builddir and sourcedir have different drive letters, a relative path
doesn't exist, and os.path.relpath fails with a ValueError exception.
This just fixes the places which are hit by test cases in a simple-minded
way. There are several other uses of os.path.relpath(), which might be
suspect.
Meson 0.48.0 some validation for using compiled binaries in custom
targets and generators, which is nice. It didn't take into account
though that as long as the OS is the same, some architectures support
running a related architecture natively (x86_64 can run x86 natively,
for example).
Fortunately we already have a method for covering this case available
through the Environment class.
Fixes#4254
Before, the mappings has been created over all the links, while it
actaully only used the Shared or Static Targets. This structure now is
tree like structured and cached, thus the results can be computed a lot
faster.
The generator step generate_install is now for EFL from 6 sec. down to
0.3s. Which improves the overall build time from ~20 sec. to ~14 sec.
there is a huge amount of isinstance calls, this reduces the amount of
these calls while splitting up a rather big function. It also assosiates
every target type with theire default install directory.
Otherwise if we for some reason get '/usr/lib/../lib' in there
we end up saying it is not a system path.
And for some reason here I got:
```
$ pkg-config --libs libffi 148 ST 117 hotdoc
-L/usr/lib/../lib -lffi
```
The problem with the earlier position of the generation code was, that
the results could not be cached, because the list of all link_deps was
overall different. However, it shared a special kind of subsets with
other build build targets.
Generating the set of subdirs that are required for linking, alongside
with the link dependencies brings the possibility of caching this.
This reduces the buildting from 1 min. in efl down to 20 sec. And
reduces the amount of 30872534 calls down.
this saves ~40 sec.
This means that we will take into account all the flags set in the
cross file when fetching the list of library dirs, which means we
won't incorrectly look for 64-bit libraries when building for 32-bit.
Signed-off-by: Nirbheek Chauhan <nirbheek@centricular.com>
Closes https://github.com/mesonbuild/meson/issues/3881
Shared modules may be resource-only DLLs, or might automatically
self-initialize using C constructors or WinMain at DLL load time.
When an import library is not found for a shared module, just print
a message about it instead of erroring out.
Fixes#3965
We used to immediately try to use whatever exe_wrapper was defined in
the cross file, but some people generate the cross file once and use
it for several projects, most of which do not even need an exe wrapper
to build.
Now we're a bit more resilient. We quietly fall back to using
non-exe-wrapper paths for compiler checks and skip the sanity check.
However, if some code needs the exe wrapper, f.ex., if you run a built
executable using custom_target() or run_target(), we will error out
during setup.
Tests will, of course, continue to error out when you run them if the
exe wrapper was not found. We don't want people's tests to silently
"pass" (aka skip) because of a bad CI setup.
Closes https://github.com/mesonbuild/meson/issues/3562
This commit also adds a test for the behaviour of exe_wrapper in these
cases, and refactors the unit tests a bit for it.
We already have code to fetch and find binaries specified in a cross
file, so use the same code for exe_wrapper. This allows us to handle
the same corner-cases that were fixed for other cross binaries.
D is not a 'c-like' language, but it can link to C libraries. The same
might be true of Rust in the future and Go when we add support for it.
This contains no functionality changes.
Since `build_always` also adds a target to the set of default targets,
this option is marked deprecated in favour of the new option
`build_always_stale`.
`build_always_stale` *only* marks the target to be always considered out
of date, but does *not* add it to the set of default targets.
The old behaviour can still be achieved by combining
`build_always_stale` with `build_by_default`.
fixes#1942
On macOS, we set the install_name for built libraries to
@rpath/libfoo.dylib, and when linking to the library, we set the RPATH
to its path in the build directory. This allows all built binaries to
be run as-is from the build directory (uninstalled).
However, on install, we have to strip all the RPATHs because they
point to the build directory, and we change the install_name of all
built libraries to the absolute path to the library. This causes the
install name in binaries to be out of date.
We now change that install name to point to the absolute path to each
built library after installation.
Fixes https://github.com/mesonbuild/meson/issues/3038
Fixes https://github.com/mesonbuild/meson/issues/3077
With this, the default workflow on macOS matches what everyone seems
to do, including Autotools and CMake. The next step is providing a way
for build files to override the install_name that is used after
installation for use with, f.ex., private libraries when combined with
the install_rpath: kwarg on targets.
When the exe runner is `wine` or `wine32` or `wine64`, etc.
This allows people to run tests with wine.
Note that you also have to set WINEPATH to point to your custom
prefix(es) if your tests use external dependencies.
Closes https://github.com/mesonbuild/meson/issues/3620
To maintain backward compatibility we cannot add recursive objects by
default. Print a warning when there are recursive objects to be pulled
and the argument is not set. After a while we'll do pull recursive
objects by default.
- determine_ext_objs: What matters is if extobj.target is a unity build,
not if the target using those objects is a unity build.
- determine_ext_objs: Return one object file per compiler, taking into
account generated sources.
- object_filename_from_source: No need to special-case unity build, it
does the same thing in both code paths.
- check_unity_compatible: For each compiler we must extract either none
or all its sources, taking into account generated sources.
This option controls the permissions of installed files (except for
those specified explicitly using install_mode option, e.g. in
install_data rules.)
An install-umask of 022 will install all binaries, directories and
executable files with mode rwxr-xr-x, while all data and non-executable
files will be installed with mode rw-r--r--.
Setting install-umask to the string 'preserve' will disable this
feature, keeping the permissions of installed files same as the files in
the build tree (or source tree for install_data and install_subdir.)
Note that, in this case, the umask used when building and that used when
checking out the source tree will leak into the install tree.
Keep the default as 'preserve', to show that no behavior is changed and
all tests keep passing unchanged.
Tested: ./run_tests.py
Jussi Pakkanen