Instad of having special casing of threads in the backends and
everywehre else, do what we did for openmp, create a real
dependency. Then make use of the fact that dependencies can now have
sub dependencies to add threads.
Previously cross, but not native, external args were used. Then in
d451a4bd97 the cross special cases were
removed, so external args are never used.
This commit switches that so they are always used. Sanity checking works
just the same as compiler checks like has header / has library.
I recall that @jpakkane never wanted this, but @nirbheek did, but then
@nirbheek changed his mind.
I am fine either way except for the cross inconsistency that exists
today: There is no `c_preproc_args` or similar one can put in the cross
file, so no way to replicate the effect of CPPFLAGS during cross
compilation.
This patch creates an enum for selecting libtype as static, shared,
prefer-static, or prefer-shared. This also renames 'static-shared'
with 'prefer_static' and 'shared-static' with 'prefer_shared'. This is
just a refactor with no behavioral changes or user facing changes.
Currently we specialcase OpenMP like we do threads, with a special
`need_openmp` method. This seems like a great idea, but doesn't work
out in practice, as well as it complicates the opemp
implementation. If GCC is built without opemp support for example, we
still add -fopenmp to the the command line, which results in
compilation errors.
This patch discards that and treats it like a normal dependency,
removes the need_openmp() method, and sets the compile_args attributes
from the compiler.
Fixes#5115
Instead use coredata.compiler_options.<machine>. This brings the cross
and native code paths closer together, since both now use that.
Command line options are interpreted just as before, for backwards
compatibility. This does introduce some funny conditionals. In the
future, I'd like to change the interpretation of command line options so
- The logic is cross-agnostic, i.e. there are no conditions affected by
`is_cross_build()`.
- Compiler args for both the build and host machines can always be
controlled by the command line.
- Compiler args for both machines can always be controlled separately.
macOS provides the tool `lipo` to check the archs supported by an
object (executable, static library, dylib, etc). This is especially
useful for fat archives, but it also helps with thin archives.
Without this, the linker will fail to link to the library we mistakenly
'found' like so:
ld: warning: ignoring file /path/to/libfoo.a, missing required architecture armv7 in file /path/to/libfoo.a
Instead of only doing a naive filesystem search, also run the linker
so that it can tell us whether the -F path specified actually contains
the framework we're looking for.
Unfortunately, `extraframework` searching is still not 100% correct in
the case when since we want to search in either /Library/Frameworks or
in /System/Library/Frameworks but not in both. The -Z flag disables
searching in those prefixes and would in theory allow this, but then
you cannot force the linker to look in those by manually adding -F
args, so that doesn't work.
It appears that LIB/LINK default to the host architecture if they can't
guess it from the first object. With the MSVC toolchain, resource files
are (usually) compiled to an arch-neutral .res format. Always
explicitly provide a '/MACHINE:' argument to avoid it guessing
incorrectly when cross-compiling.
Store the MSVC compiler target architecture ('x86', 'x64' or 'ARM' (this
is ARM64, I believe)), rather than just if it's x64 or not.
The regex used for target architecture should be ok, based on this list
of [1] version outputs, but we assume x86 if no match, for safety's
sake.
[1] https://stackoverflow.com/a/1233332/1951600
Also detect arch even if cl outputs version to stdout.
Ditto for clang-cl
Future work: is_64 is now only used in get_instruction_set_args()
This allows each implementation (gnu-like) and msvc to be implemented in
their respective classes rather than through an if tree in the CCompiler
class. This is cleaner abstraction and allows us to clean up the Fortran
compiler, which was calling CCompiler bound methods without an instance.
Because we need to inherit them in some cases, and python's
keyword-or-positional arguments make this really painful, especially
with inheritance. They do this in two ways:
1) If you want to intercept the arguments you need to check for both a
keyword and a positional argument, because you could get either. Then
you need to make sure that you only pass one of those down to the
next layer.
2) After you do that, if the layer below you decides to do the same
thing, but uses the other form (you used keyword by the lower level
uses positional or vice versa), then you'll get a TypeError since two
layers down got the argument as both a positional and a keyword.
All of this is bad. Fortunately python 3.x provides a mechanism to solve
this, keyword only arguments. These arguments cannot be based
positionally, the interpreter will give us an error in that case.
I have made a best effort to do this correctly, and I've verified it
with GCC, Clang, ICC, and MSVC, but there are other compilers like Arm
and Elbrus that I don't have access to.
It seems that clang-cl isn't quite compatible with cl in the way it handles
pch, and when the precompiled header is used, the pathname of the header is
needed, not just its filename.
This fixes test\common\13 pch with clang-cl
When invoked as clang-cl to compile, it doesn't emit cl-compatible D9002
warnings about unknown options, but fortunately also supports
-Werror-unknown-argument instead.
When invoked to link, and using LINK, it does emit cl-compatible LNK4044
warnings about unknown options.
Currently, ComplierHolder.determine_args() unconditionally adds the link
arguments to the commmand, even if we aren't linking, because it doesn't
have access to the mode (preprocess, compile, link) that
_get_compiler_check_args() will use.
This leads to command lines like:
'cl testfile.c /nologo /showIncludes /c /Fooutput.obj /Od kernel32.lib
user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib
uuid.lib comdlg32.lib advapi32.lib'
which clang-cl considers invalid; MSVS cl accepts this, ignoring the
unneeded libraries
Change from passing extra_args down to _get_compiler_check_args(), to
passing down a callback to CompilerHolder.determine_args() (with a bound
kwargs argument), so it can consult mode and kwargs to determine the args to
use.
Handle clang's cl or clang-cl being in PATH, or set in CC/CXX
Future work: checking the name of the executable here seems like a bad idea.
These compilers will fail to be detected if they are renamed.
v2:
Update compiler.get_argument_type() test
Fix comparisons of id inside CCompiler, backends and elsewhere
v3:
ClangClCPPCompiler should be a subclass of ClangClCCompier, as well
Future work: mocking in test_find_library_patterns() is effected, as we
now test for a subclass, rather than self.id in CCompiler.get_library_naming()
Some compilers try very had to pretend they're another compiler (ICC
pretends to be GCC and Linux and MacOS, and MSVC on windows), Clang
behaves much like GCC, but now also has clang-cl, which behaves like MSVC.
This method provides an easy way to determine whether testing for MSVC
like arguments `/w1234` or gcc like arguments `-Wfoo` are likely to
succeed, without having to check for dozens of compilers and the host
operating system, (as you would otherwise have to do with ICC).
Replace several checks against GCC_MINGW or (GCC_MINGW, GCC_CYGWIN) with
is_windows_compiler instead, so that clang and other gcc-like compilers
using MinGW work appropriately with vs_module_defs, c_winlibs, and
cpp_winlibs.
Fixes#4434.
For PE/COFF it is not possible to allow undefined symbols, so do not
try to use the option to do so.
While gcc ld silently ignores it, this is not the case for the llvm
linker.
Fix#4415
* This helps with reproducibility on macOS in the same way
`$ORIGIN` improves reproducibility on Linux-like systems.
* This makes the build-tree more resilient to users injecting
rpaths via `LDFLAGS`. Currently Meson on macOS crashes when
a build-tree rpath and a user-provided `-Wl,-rpath` in
LDFLAGS collide, leading to `install_name_tool` failures.
While this still does not solve the root cause, it makes
the occurrence much less likely, as users will generally
pass absolute `-Wl,-rpath` arguments into Meson.
Occasionally Darwin libraries can be .so rather than .dylib e.g. tensorflow_cc.so
tensorflow_cc is a c++ API for Tensorflow (https://github.com/FloopCZ/tensorflow_cc)
which was primarily written for Linux but is also compilable on Darwin. Possibly
through laziness, possibly just to have consistent filenames, the developers did not
opt to change the suffix from the Linux default when this is compiled on Darwin.
Also, the Darwin linker will find libraries with a .so suffix if they are
in its path. find_library() needs to match the linker behaviour.
In get_library_dirs() we are trusting the compiler to return a correct
list of directories to search for libraries, based on whether or not
we are compiling 64-bit or 32-bit. Unfortunately, this is often not the
case, as 64-bit libraries often are returned when compiling with -m32 on
a 64-bit OS.
Since system directories do not contain a mix of libraries, the solution
here is to check each directory, by picking a .so file in the directory
and checking whether its 64-bit or 32-bit. If we can't determine if we
want 32-bit or 64-bit, just skip the checks and assume the directory is
good.
Using the -pthread argument is not needed with clang when compiling for
darwin, and it results in the warning:
warning: argument unused during compilation: '-pthread'
GNU binutils ld silently ignores -rpath flags when targeting windows
(and it is already commented within ninjabackend.py that rpath as
concept doesn't exist on windows), and build_rpath_args in
VisualStudioCCompiler also returns an empty array. Therefore skip
this flag altogether.
This fixes linking with lld in MinGW mode, which doesn't support the
rpath flag.
* Enums are strongly typed and make the whole
`gcc_type`/`clang_type`/`icc_type` distinction
redundant.
* Enums also allow extending via member functions,
which makes the code more generalisable.
It's fairly common on Linux and *BSD platforms to check for these
attributes existence, so it makes sense to me to have this checking
build into meson itself. Autotools also has a builtin for handling
these, and by building them in we can short circuit cases that we know
that these don't exist (MSVC).
Additionally this adds support for two common MSVC __declspec
attributes, dllimport and dllexport. This implements the declspec
version (even though GCC has an __attribute__ version that both it and
clang support), since GCC and Clang support the MSVC version as well.
Thus it seems reasonable to assume that most projects will use the
__declspec version over teh __attribute__ version.
- For optimization=s add /O1: Use Maximum Optimization (Favor Size),
and remove /Os as it's implied by /O1.
- Because we add /O1, this implies /Gy, i.e. Function-Level Linking, so
unused code can be omitted.
- Add /Gw: Optimize Global Data, so unused data can be omitted.
With buildtype=minsize on x86 this reduces the size of a statically
linked Vala compiler binary from 5 MB down to just 1.87 MB.
This means that we will take into account all the flags set in the
cross file when fetching the list of library dirs, which means we
won't incorrectly look for 64-bit libraries when building for 32-bit.
Signed-off-by: Nirbheek Chauhan <nirbheek@centricular.com>
Closes https://github.com/mesonbuild/meson/issues/3881
* get_library_naming: Use templates instead of suffix/prefix pairs
This commit does not change functionality, and merely sets the
groundwork for a more flexibly naming implementation.
* find_library: Fix manual searching on OpenBSD
On OpenBSD, shared libraries are called libfoo.so.X.Y where X is the
major version and Y is the minor version. We were assuming that it's
libfoo.so and not finding shared libraries at all while doing manual
searching, which meant we'd link statically instead.
See: https://www.openbsd.org/faq/ports/specialtopics.html#SharedLibs
Now we use file globbing to do searching, and pick the first one
that's a real file.
Closes https://github.com/mesonbuild/meson/issues/3844
* find_library: Fix priority of library search in OpenBSD
Also add unit tests for the library naming function so that it's
absolutely clear what the priority list of naming is.
Testing is done with mocking on Linux to ensure that local testing
is easy
We used to immediately try to use whatever exe_wrapper was defined in
the cross file, but some people generate the cross file once and use
it for several projects, most of which do not even need an exe wrapper
to build.
Now we're a bit more resilient. We quietly fall back to using
non-exe-wrapper paths for compiler checks and skip the sanity check.
However, if some code needs the exe wrapper, f.ex., if you run a built
executable using custom_target() or run_target(), we will error out
during setup.
Tests will, of course, continue to error out when you run them if the
exe wrapper was not found. We don't want people's tests to silently
"pass" (aka skip) because of a bad CI setup.
Closes https://github.com/mesonbuild/meson/issues/3562
This commit also adds a test for the behaviour of exe_wrapper in these
cases, and refactors the unit tests a bit for it.
We already have code to fetch and find binaries specified in a cross
file, so use the same code for exe_wrapper. This allows us to handle
the same corner-cases that were fixed for other cross binaries.
Paths provided to us by the user or by pkg-config can be (and must be)
assumed to be usable since they might not be usable standalone.
Closes https://github.com/mesonbuild/meson/issues/3832
Rather than storing in the cache of search paths the full list returned
from the compiler and having each call ignore the non-existent ones, remove
from the list all non-existent ones before returning to the caching function.
When find_library is used to find dependencies, meson checks all paths for
libraries with all prefixes that could match. This means that when we are
compiling with -m32 on a 64-bit system, meson will find 64-bit libraries and
assumes that they will work. Naturally that is not the case.
The obvious fix is to do a test link against those libraries, but the extra
wrinkle here is that we need to do a "whole link" so as to test the static
libs. A check with gcc+ld on linux shows that unless there are unresolved
symbols from the main.c file, the static library is never checked so we avoid
the error from an incompatible library.
On macOS, we set the install_name for built libraries to
@rpath/libfoo.dylib, and when linking to the library, we set the RPATH
to its path in the build directory. This allows all built binaries to
be run as-is from the build directory (uninstalled).
However, on install, we have to strip all the RPATHs because they
point to the build directory, and we change the install_name of all
built libraries to the absolute path to the library. This causes the
install name in binaries to be out of date.
We now change that install name to point to the absolute path to each
built library after installation.
Fixes https://github.com/mesonbuild/meson/issues/3038
Fixes https://github.com/mesonbuild/meson/issues/3077
With this, the default workflow on macOS matches what everyone seems
to do, including Autotools and CMake. The next step is providing a way
for build files to override the install_name that is used after
installation for use with, f.ex., private libraries when combined with
the install_rpath: kwarg on targets.
Added method concatenate_string_literals to CCompiler. Will concatenate
string literals.
Added keyword argument 'concatenate_string_literals' to Compiler.get_define.
If used will apply concatenate_string_literals to its return value.
Otherwise we can end up searching for the same library tens of times,
because pkg-config does not de-duplicate -lfoo args before returning
them.
We use -Wl,--start-group/end-group, so we do not need to worry about
ordering issues in static libraries.
This checks not only for existence, but also for usability of the
header, which means it does a full compilation and not just
pre-processing or __has_include.
Fixes https://github.com/mesonbuild/meson/issues/2246
* mesonbuild/compilers/c.py: Make the `find_library` method more generic by allowing the user to supply the `code` for compiling and linking.
* mesonbuild/compilers/fortran.py: Use the methods inherited from `Compiler` base class where appropriate. Also reuse `CComiler` methods where applicable. This should be sufficient to get various compiler/linker arguments as well as to compile and link Fortran programs. This was tested with `gfortran` compiler, and while the other compilers ought to work for simple cases, their methods are primarily inherited from the base `FortranCompiler` class.
* test cases/fortran/10 find library/gzip.f90: Fortran module with some basic Fortran wrapper interfaces to `gzopen`, `gzwrite`, and `gzclose` C `zlib` functions.
* test cases/fortran/10 find library/main.f90: Fortran program using the `gzip` Fortran interface module to write some data to a gzip file.
* test cases/fortran/10 find library/meson.build: Meson build file for this test case. This demonstrates the ability to link the Fortran program against an external library.
GCC does not print a warning or error for unknown options if the options
are to disable warnings. Therefore, when checking for options starting
'-Wno-', also check the opposite enabling option. This fixes the case
where e.g. -Wno-implicit-fallthrough is incorrectly reported as supported
by gcc 5.4. To avoid missed warnings when using combinations of flags, such
as in test case "112 has arg", we limit the checking of for the positive
option to where the negative option is checked alone.
This patch exploits the information residing in ltversion to set the
-compatibility_version and -current_version flags that are passed to the
linker on macOS.
The linkers currently do not support ninja compatible output of
dependencies used while linking. Try to guess which files will be used
while linking in python code and generate conservative dependencies to
ensure changes in linked libraries are detected.
This generates dependencies on the best match for static and shared
linking, but this should not be a problem, except for spurious
rebuilding when only one of them changes, which should not be a problem.
Also makes sure to ignore any libraries generated inside the build, to
keep the optimisation working where changes in a shared library only
cause relink if the symbols have changed as well.
This caching is only for a single run, so it doesn't help reconfigure.
However, it is useful for subproject setups where different subprojects
will run the same compiler checks.
The cache is also per compiler instance and is not used for functions
that want to read or run the outputted object file or binary.
For gst-build, this halves the number of compiler checks that are run
and reduces configuration time by 20%.
Copy the algorithm used by autoconf.
It computes the upper and lower limits by starting at [-1,1] and
multiply by 2 at each iteration. This is even faster for small numbers
(the common case), for example it finds value 0 in just 2 compilations
where old algorithm would check for 1024, 512, ..., 0.
Recent versions of systemd (starting with v238) started to check for the
existence of the statx structure using the cc.sizeof() operation. The cc
compiler implementation fails to detect this structure because it's size
limit is 128, meaning it will fail for any type larger than 128 bytes in
the following way during cross-compilation checks:
meson.build:10:2: ERROR: Cross-compile check overflowed
Increase the size limit for data types to 1024 bytes, which should give
plenty of room for even large data structures. This is obviously not
guaranteed to be an upper bound, but given the binary search algorithm
implemented in the cross-compile check, raising the limit too high may
significantly increase the time required for this check on smaller data
types.
Signed-off-by: Thierry Reding <treding@nvidia.com>
- Fixing flake8 error in compilers.py - [E124] closing bracket does not match visual indentation
- Updating ARMCCompiler constructor in c.py to raise error as per comments
A hard error makes this feature useless in most cases since a static
library usually won't be found for every library, particularly system
libraries like -lm. Instead, warn so the user can provide the static
library if they wish.
This feature will be expanded and made more extensible and more usable
in the future.
Closes https://github.com/mesonbuild/meson/issues/2785
After PR #2662, running test case common/125 shared module/ on Cygwin gets
me:
$ ninja -C _build
ninja: Entering directory `_build'
[7/7] Linking target prog.exe.
/usr/lib/gcc/x86_64-pc-cygwin/6.4.0/../../../../x86_64-pc-cygwin/bin/ld: warning: --export-dynamic is not supported for PE+ targets, did you mean --export-all-symbols?
Also, fix doc for correct version of first apperance.
Future work: Notwithstanding the hint that ld gives, these options are not
equivalent, and it's not clear we should be using it here:
--export-all-symbols is the default behaviour, and if the exports are
restricted by explicit annotations or a .def file, this option might be
overriding that...
According to Python documentation[1] dirname and basename
are defined as follows:
os.path.dirname() = os.path.split()[0]
os.path.basename() = os.path.split()[1]
For the purpose of better readability split() is replaced
by appropriate function if only one part of returned tuple
is used.
[1]: https://docs.python.org/3/library/os.path.html#os.path.split
has_argument and other similar methods of compiler objects only support
checking compiler flags. If they are used to check linker flags, the
results are very likely to be wrong and developers should be warned.
See issue #2762
Adds full_version to class Compiler. If set full_version will be printed
additionally.
Added support for CCompiler and CPPCompiler
Added support for gcc/g++, clang/clang++, icc.
We can't know if the .lib is a static or import library, but that's
a problem in general too. The only way to figure out if a specific
file is an import or a static library is to dump its symbols and check
if it starts with __imp or not.
Even then, some libs are hybrid import and static, i.e., they contain
references to DLLs for some symbols and also provide implementations
for other symbols so this is a difficult problem.
Closes https://github.com/mesonbuild/meson/issues/2659
We can now specify the library type we want to search for, and whether
we want to prefer static libraries over shared ones or the other way
around. This functionality is not exposed to build files yet.
This method accepts a single function that takes no arguments and
returns a single value which can be a value that can be cast to
a 64-bit signed integer, or a string, and returns that value.
Mostly useful for running foolib_version() functions that return the
currently-available version of libraries.
CCache requires this flag when building with precompiled headers.
Without it, the preprocessor fails and CCache fallbacks to running the
real compiler.
Users still need to set 'sloppiness' to 'pch_defines,time_macros' in
their ccache.conf file for CCache to cache builds that use precompiled
headers. See the CCache manual for more info:
https://ccache.samba.org/manual.html#_precompiled_headers
In addition to filtering libs out while generating the command-line, we
must also filter them out in find_library() otherwise these libs will be
detected as "found" on Windows with MSVC.
Closes https://github.com/mesonbuild/meson/issues/1509
Michael Hirsch, Ph.D