Commit Graph

113 Commits

Author SHA1 Message Date
Eli Schwartz 43c60318fd python module: cache dependency() lookup between invocations
Modeled similarly after the installations cache, but using the coredata
dependency cache because it has a nice mechanism for handling the cache
key.
2022-06-19 21:44:17 +03:00
Eli Schwartz 85f3fc2022 flake8: don't use f-strings for strings without formatting 2022-06-13 13:34:39 +03:00
Eli Schwartz aa874ea0d0 python module: default extensions to hidden symbol visibility
python compiled extensions should never need to expose any symbol other
than PyInit_* which is declared with default visibility via
PyMODINIT_FUNC on supported compilers.

Thus, a reasonably sane default is to mark any other symbols as hidden,
while still respecting any manually specified visibility.

Gate this on the version of python itself, as not all versions decorate
PyMODINIT_FUNC properly.
2022-05-09 10:49:04 -04:00
Chun-wei Fan 99ad11bd9c Windows: Improve Python 3.8+ module check on Windows
On Python 3.8.x and later, if the imported module requires non-system DLLs that
are not installed nor bundled with the module package, os.add_dll_directory()
must be called on every path that contains the required DLLs, so that the module
can be imported successfully by Python.

Make things easier for people by calling os.add_dll_directory() on the
valid directories in %PATH%, so that such module checks can be carried out
successfully with much less manual intervention.
2022-05-02 02:45:51 +03:00
Matti Picus 73fa3aad43 python module: windows dll name for pypy needs special casing 2022-04-12 06:58:04 -04:00
Jussi Pakkanen 3c80f8f965 Use a temp file to invoke the introspection command.
This is more reliable as '-c' can, for example, exhaust
the maximum command line length.
2022-04-10 17:13:51 +03:00
Eli Schwartz d27fd555e8 python module: restore logging for broken python
This check was erroneously removed in commit c5c02b72e1
2022-04-10 15:17:22 +03:00
Xavier Claessens 54213683e2 python: Remove warning about invalid install path
It was originally added because proper detection was not working on
Debian, but that has been fixed since. It was causing annoying warning
by default when prefix is /usr/local that can only be avoided by setting
options.
2022-03-16 07:03:43 -04:00
Eli Schwartz 68b8fbcf6d Revert "devenv: Set PYTHONPATH where we install python modules"
This reverts commit 79c6075b56.

# Conflicts:
#	docs/markdown/snippets/devenv.md
#	mesonbuild/modules/python.py
#	test cases/unit/91 devenv/test-devenv.py

PYTHONPATH cannot be reliably determined. The standard use case for
installing python modules with Meson is mixed pure sources (at least
`__init__.py`) and compiled extension_modules or configured files.
Unfortunately that doesn't actually work because python will not load
the same package hierarchy from two different directories, one a source
directory and one a (mandatory) out of tree build directory.

(It kind of can, but you need to do what this test case accidentally
stumbled upon, which is namespace packages. Namespace packages are a
very specific use case and you are NOT SUPPOSED to use them outside that
use case, so people are not going to use them just to circumvent Meson
devenv stuff as that would have negative install-time effects.)

Adding PYTHONPATH anyway will just lead to documentation commitments
which we cannot actually uphold, and confusing issues at time of use
because some imports *will* work... and some will *not*. The end result
will be a half-created tree of modules which just doesn't work together
at all, but because it partially works, users attempting to debug it
will spend time wondering why parts of it do import.

For any case where the automatic devenv would work correctly, it will
also work correctly to use `meson.add_devenv()` a single time, which is
very easy to manually get correct and doesn't provide any significant
value to automate.

In the long run, an uninstalled python package environment will require
"editable installs" support.
2022-03-15 11:22:59 +02:00
Ralf Gommers 44104e820a Remove a spurious debug print in Python module
This prints many lines of unwanted "done /absolute/path",
I noticed this when testing 0.62.0rc1 with SciPy.

[ci skip]
2022-03-10 15:30:31 -05:00
Eli Schwartz 2540ad6e9e
fix correctly detecting whether python is found if link args are empty
There are two cases where we can assume we found the python dependency
with its requisite libraries using sysconfig:

- we found the library with find_library and are prepared to link to it
- the library is not actually part of the dependency, so its presence or
  absence is irrelevant

In the latter case, we should consider it found if link_libpython is
False. Originally we did this, but the logic was inverted in commit
5b422fce87 in an unrelated change and
without explanation, likely by accident.

Normally this doesn't much matter, since a python invariably comes with
a predictably located libpython and the first condition evaluates true.
But that is not true for pypy, and in fact that is the reason the
link_libpython check was originally added in commit
1bd14b52b2.

Restore that original logic.

Fixes #8570
2022-03-07 20:35:43 -05:00
Eli Schwartz 187dc656f4
merge various TYPE_CHECKING blocks into one
A bunch of files have several T.TYPE_CHECKING blocks that each do some
things which could just as well be done once, with a single `if`
statement. Make them do so.
2022-03-07 19:01:04 -05:00
Xavier Claessens 79c6075b56 devenv: Set PYTHONPATH where we install python modules 2022-02-28 09:03:27 -05:00
Xavier Claessens c5c02b72e1 Cache the result of python.find_installation()
This avoids running sanity checks everytime find_installation() is
called.
2022-02-28 09:03:27 -05:00
Eli Schwartz 78945fb983
python module: add option to specify a python environment to install to
The default behavior of installing relative to prefix may be unexpected,
and is definitely wrong in many cases.

Give users control in order to specify that yes, they actually want to
install to a venv.

This is particularly useful for projects that use meson as a build
system for a python module, where *all* files shall be installed into
the python site-packages.
2022-02-22 22:22:16 -05:00
Eli Schwartz 2a99252604
python module: only find a pkg-config dependency from the found python
If the found python returns None from sysconfig.get_config_var('LIBPC')
then we cannot (and don't) set PKG_CONFIG_LIBDIR from it. In fact, we
can virtually guarantee we won't find a PkgConfigDependency either,
because any python that doesn't have a LIBPC is presumably not installed
to the system pkg-config directory (maybe it's an isolated relocatable
install, maybe it just doesn't have pkg-config support for who knows
what reason).

Trying to find one anyway using pkg-config's builtin search paths can
unexpectedly succeed, though, by finding a completely unrelated python
installation installed to a system location, which isn't the one we are
actually building for.

Instead, return early so that we use the system dependency class
fallback.

While we are at it, add back the debug messages from #3989 which got
removed.
2022-01-20 23:50:04 -05:00
Matthew Brett bd5e520672 Fix to find Python files for Windows virtualenvs
Virtualenvs do not have their Python DLLs etc in the `sys.prefix`
directory, but in the `sys.base_prefix` directory.  This directory is
the same as `sys.prefix` if not in a virtualenv, so is safe for either
case:

https://docs.python.org/3/library/sys.html#sys.base_prefix
2021-11-15 23:20:48 +02:00
Xavier Claessens 898bf6e518 python: Better detect when install path is not in sys.path
Using pathlib ensure propre platform specific path handling, such as
case sensitivity.
2021-11-09 18:49:03 +02:00
Xavier Claessens c8d125653d
python.dependency(): Do not stop when first candidate is not found
It has to lookup the dependency with required=False otherwise it raises
an exception when the first candidate (pkg-config) failed.
2021-11-02 14:24:08 -04:00
Eli Schwartz 8947352889 fix various flake8 whitespace errors 2021-10-27 09:51:52 -04:00
Thomas Heijligen ecdf192f46 dep.name(): return dependency name even if dependency is not found
The dep.name() function schould always return the name of the
dependency as documented. No matter if it was found or not.
https://mesonbuild.com/Reference-manual_returned_dep.html#depfound
2021-10-26 09:36:22 -04:00
Christian Clauss a5020857f3 Fix typos discovered by codespell 2021-10-10 16:12:25 -04:00
Xavier Claessens 329d111709 python: Add platlibdir and purelibdir options 2021-10-08 17:47:35 -04:00
Eli Schwartz d06cc042eb
f-strings 2021-10-04 16:29:32 -04:00
Filipe Laíns 05b5a1e56f modules: python: better handling of the Python paths for Debian
Hardcoding the name is fragile, and enabling it based on the existence of
/etc/debian_version (as the is_debianlike helper does) will result in
incorrect paths if the Python binary is not provided by Debian.

Using the deb_system distuils scheme instead makes sure we use the
install path from the current interpreter, which Debian could change
between releases, and gives us the correct value on Python installations
that are not provided by Debian (eg. deadsnakes, Github Action Python,
etc.)

Do notice, though, that there is still no guarantee that these are the
correct paths, as they assume all schemes paths have the install prefix
as a base, see #9284.

Signed-off-by: Filipe Laíns <lains@riseup.net>
2021-09-29 08:54:05 -04:00
Filipe Laíns af8b55d49b mintro: add installed_plan
Signed-off-by: Filipe Laíns <lains@riseup.net>
2021-09-06 18:10:55 +02:00
Eli Schwartz 34ac77d619 python module: fix error message mentioning setuptools
We use distutils, not setuptools, for probing information.
2021-08-31 08:42:27 -07:00
Xavier Claessens 276063a1d2 Add missing "disabler" kwarg to python.dependency()
There is a unit test using it and now fails because the warning about
unknown kwarg became fatal.
2021-08-30 14:00:54 -04:00
Dylan Baker 11fbaf29d8 interpreter: fix cases of `KwargInfo(..., T, default=None)`
The correct way to mark these is `KwargInfo(..., (T, type(None)))`.
There's also a few cases of `(T, None)` which is invalid, as `None`
isn't a type
2021-08-27 14:54:29 -07:00
Eli Schwartz 6170f1175e python module: produce the correct install path on every OS
The sysconfig paths are, by default, correct for every OS -- they are
supposed to follow the scheme that python knows about per default.

For some reason, this overrode the scheme to posix_prefix, which is the
default for posix OSes like linux and macOS, but wrong on Windows.
Simply deleting this entirely makes everything that used to work, still
work, and a couple new things start working.
2021-08-27 07:46:58 -04:00
Xavier Claessens 0063eb251e python: Workaround broken install path 2021-08-22 22:14:59 -04:00
Eli Schwartz b0ffb80ecf
python module: fix extensions without explicit subdir being installed to libdir
They are documented to go in site-packages, and indeed belong there.

Regression from the initial implementation via commit ad296976f0

Fixes #6331
2021-08-18 17:58:30 -04:00
Eli Schwartz d9a9c3b5da
python module: make external program wrapper have access to useful metadata
Basically just reorganize code. Try to make it a little neater, while
we're at it.
2021-08-18 17:54:52 -04:00
Eli Schwartz 445f08869b
Fix incorrect .name attribute for python program
It is always set to python3 even if we asked for python2...
2021-08-18 17:54:52 -04:00
Eli Schwartz 9eac9e0ff2
fix some confusingly indirect code
rsplit(..., 1) always produces exactly one split, by design, there's no
need to then join a 1-element list via a generator comprehension after
extracting the end of it via pop. If this commit message sounds
confusing, then so was I when trying to figure out what this actually
did and if it needed extracting to PythonExternalModule.
2021-08-18 17:54:51 -04:00
Xavier Claessens 8c5aa031b5 Add install tags
Fixes: #7007.
2021-08-17 15:19:18 -04:00
Xavier Claessens 3f71779f8a Fix python module leaving PKG_CONFIG_LIBDIR set in os.environ. 2021-08-03 14:44:03 -04:00
Dylan Baker b4e826bff3 modules/python: fix up a few simply typing warnings/errors
These were spotted by mypy and pyright. One is a string where a Path is
expected, another other is a possibly unbound variable, and the third is
bound but unused variables.
2021-07-13 16:43:14 -07:00
Dylan Baker eac2d5eec5 modules/python: Allow trying a macos framework as well other methods 2021-07-13 16:43:14 -07:00
Dylan Baker d6e606166e modules/python: make some internal helpers protected 2021-07-13 16:43:14 -07:00
Dylan Baker 4d67dd19e5 modules/python: use factory for dependency
This removes the odd 'pkgdep' attribute thing, and makes it behave more
like a proper dependency
2021-07-13 16:43:14 -07:00
Dylan Baker a881e849b5 modules/python: simplify a number of interfaces
Including not calling back into `Interpreter.func_*`, which is not a
good idea both from a type saftey and perforamance point of view.
Instead there's now a shared _impl method
2021-07-13 16:43:14 -07:00
Dylan Baker 9eec2a131b modules/python: use typed_pos_args
And note that the way that find_installation works is completely broken
in regards to machine files
2021-07-13 16:43:14 -07:00
Dylan Baker 1f7ab2f010 modules/python: Add type annotations
There's still a number of things that don't properly type check, that's
expected though as the input is often unvalidated and assumed good.
2021-07-13 16:43:14 -07:00
Dylan Baker 3b3c580817 modules/python: sort imports 2021-07-13 16:41:37 -07:00
Daniel Mensinger 7c757dff71 holders: Fix the remaining code to respect the holder changes 2021-06-18 23:48:33 +02:00
Eli Schwartz bbcc91c1e5
expose SystemDependency and BuiltinDependency as toplevel classes
mesonbuild.dependencies.__init__ exposes configtool, pkgconfig, cmake
and more in __init__.py, so there's no reason we should be tying
SystemDependency to the internal organization implementation of the
subpackage!

In the 2nd previous commit it took quite some effort to figure out that
the python module "does not exist" because of import errors while
refactoring something completely different.
2021-06-17 13:22:25 -04:00
Eli Schwartz 493dc6ed10
move base class for system dependencies into base.py
In accordance with review comments; it's small enough this seems fitting.
2021-06-17 13:22:25 -04:00
Dylan Baker 203a548d60 dependencies: Use the SystemDependency
This fixes these dependencies, which currently return the name of the
dependency as the type.

Fixes #8877
2021-06-14 09:09:32 -07:00
Daniel Mensinger 95b70bcb97 deps: Split dependencies.base
Split the Factory and dependency classes out
of the base.py script to improve maintainability.
2021-06-03 10:23:27 -07:00