Commit Graph

209 Commits

Author SHA1 Message Date
Jussi Pakkanen 9ce607665a Add a simple reproducibility test command. 2024-10-01 16:33:24 +03:00
Eli Schwartz 2b80d4cca1
When configuring fails in Github Actions, print folded logs
A common, and challenging, issue in CI runners is debugging issues when
you know the information you want to check, but it's in the log file
which you don't have because remote CI machines.

There are various edge cases where this is especially hard to solve,
such as inside of `pip install` where the build directory with the log
file is automatically cleaned up. But it's never really *easy* when you
don't expect it, and the best case scenario is your iteration time gets
cut in half as you hurriedly go add some `cat`s to your CI scripts.

Meson can, at least sometimes, detect platforms where text can be
emitted inside of "folds", which are auto-collapsed and don't obscure
the general output, but when clicked will expand the logfile contents.
Hook this up.

We start off with a Github Actions implementation. We had some internal
code used by our own project tests runner, which can be utilized.

Also permit forcing it via an environment variable, in case
autodetection fails and you just want to force *something*, especially
when meson is called a couple layers deep inside some other tool.
2024-09-15 19:25:10 -04:00
Tristan Partin 03ffb5bd6f Suggest mingw Python URL instead of specific package names
Suggested by @lazka in order to be clearer when delineating between
mingw-w64-x86_64-python and mingw-w64-ucrt-x86_64-python.

Fixes: #12772
2024-06-23 22:01:14 +03:00
Charles Brunet 2b37101998 meson format command 2024-04-08 10:43:57 -07:00
Eli Schwartz fc41fd3542
add type annotations to mesonmain
An oddity: sys.stdout is statically defined as type TextIO instead of
TextIOWrapper, and thus doesn't have a .reconfigure method. This is
because they expect people to override sys.stdout with other objects,
which we do not do. Instead, assume it is always correct.

There are two final errors due to metaprogramming:
```
mesonbuild/mesonmain.py:196:13: error: Returning Any from function declared to return "int"  [no-any-return]
mesonbuild/mesonmain.py:225:9: error: Returning Any from function declared to return "int"  [no-any-return]
```
2024-02-12 18:52:43 -05:00
Dylan Baker e991c4d454 Use SPDX-License-Identifier consistently
This replaces all of the Apache blurbs at the start of each file with an
`# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing
uses to be consistent in capitalization, and to be placed above any
copyright notices.

This removes nearly 3000 lines of boilerplate from the project (only
python files), which no developer cares to look at.

SPDX is in common use, particularly in the Linux kernel, and is the
recommended format for Meson's own `project(license: )` field
2023-12-13 15:19:21 -05:00
Eli Schwartz 3070bd49a2 do not resolve symlinks when calculating the meson command
We embed the route to executing meson in various cases, most especially
regen rules in build.ninja. And we take care to ensure that it's a
canonicalized path. Although the code has moved around over time, and
adapted in ways both bad and good, the root of the matter really comes
down to commit 69ca8f5b54 which notes the
importance of being able to run meson from any location, potentially
not on PATH or anything else.

For this reason, we switched from embedding sys.argv[0] to
os.path.realpath, a very heavy stick indeed. It turns out that that's
not actually a good thing though... simply resolving the absolute path
is enough to ensure we can accurately call meson the same way we
originally did, and it avoids cases where the original way to call meson
is via a stable symlink, and we resolved a hidden location.

Homebrew does this, because the version of a package is embedded into
the install directory. Even the bugfix release. e.g.

```
/opt/homebrew/bin/meson
```

is symlinked to

```
/opt/homebrew/Cellar/meson/1.0.0/bin/meson
```

Since we went beyond absolutizing the path and onwards to canonicalizing
symlinks, we ended up writing the latter to build.ninja, and got a
"command not found" when meson was upgraded to 1.0.1. This was supposed
to work flawlessly, because build directories are compatible across
bugfix releases.

We also get a "command not found" when upgrading to new feature
releases, e.g. 0.64.x to 1.0.0, which is a terrible error message. Meson
explicitly "doesn't support" doing this, we throw a MesonVersionMismatchException
or in some cases warn you and then effectively act like --wipe was given.
But the user is supposed to be informed exactly what the problem is, rather
than getting "command not found".

Since there was never a rationale to get the realpath anyways, downgrade
this to abspath.

Fixes #11520
2023-03-28 15:29:52 +03:00
Eli Schwartz 680b5ff819
treewide: add future annotations import 2023-02-01 17:01:30 -05:00
Eli Schwartz 118396f344 runpython: make it work for -c as well
In commit 4e4f97edb3 we added support for
runpython to accept `-c 'code to execute'` in addition to just script
files. However, doing so would mangle the sys.argv in the executed code
-- which assumes, as python itself does, that argv is the stuff after
the code to execute. We correctly handled this for script files, but the
original addition of -c support pushed this handling into a script-file
specific block.
2023-01-30 15:13:06 +02:00
Eli Schwartz 9aff189e31
add a hidden environment variable to make Meson complain hard on deprecations
Useful for running the testsuite with this environment variable and
catching obscure issues. Just like with the encoding warning later down,
we have to do this inside meson itself, not externally injected.
2023-01-04 16:46:49 -05:00
Dylan Baker 2d349eae8c
pylint: enable the set_membership plugin
Which adds the `use-set-for-membership` check. It's generally faster in
python to use a set with the `in` keyword, because it's a hash check
instead of a linear walk, this is especially true with strings, where
it's actually O(n^2), one loop over the container, and an inner loop of
the strings (as string comparison works by checking that `a[n] == b[n]`,
in a loop).

Also, I'm tired of complaining about this in reviews, let the tools do
it for me :)
2022-11-30 16:23:29 -05:00
Eli Schwartz 608a6196a0
python 3.11: suppress an incorrect EncodingWarning
python 3.11 adds a warning that in 3.15, UTF-8 mode will be default.
This is fantastic news, we'd love that. Less fantastic: this warning is
silly, we *want* these checks to be affected. Plus, the recommended
alternative API would (in addition to warning people when UTF-8 mode
removed the problem) also require using a minimum python version of 3.11
(in which the warning was added) or add verbose if/else soup.

The simple, and obvious, approach is to add a warnings filter to hide
it.
2022-11-03 15:44:41 -04:00
Xavier Claessens 2dfd952eb9 Move classes used by scripts to their own module
Those classes are used by wrapper scripts and we should not have to
import the rest of mesonlib, build.py, and all their dependencies for
that.

This renames mesonlib/ directory to utils/ and add a mesonlib.py module
that imports everything from utils/ to not have to change `import
mesonlib` everywhere. It allows to import utils.core without importing
the rest of mesonlib.
2022-09-28 19:36:13 -04:00
Eli Schwartz 29ac7dd088
clean up message for OSError errorhandler
Currently it repeats the text of the exception, which appeared
immediately above. And strerror is already part of args, so that was
then doubled. `str(e)` formats this much better anyway.

While we are at it, point out that it is probably a build environment
issue, rather than just saying "this is probably not a meson bug". The
former comes across a bit more as constructive advice rather than just
"idk but it's not our fault".
2022-09-22 15:24:58 -04:00
Eli Schwartz 389b2f6785
fix regression in logging runpython errors
In commit fa044e011d we caught OSError and
started emitting a special error message disclaiming that it probably
isn't Meson's fault, and skipping the generic "This is a meson bug and
should be reported" handler.

But, we no longer did this after verifying that the command *isn't*
runpython, so arbitrary scripts that raised OSError would start saying
stuff about Meson, which was wrong.

Start checking for runpython first.
2022-09-22 15:24:57 -04:00
Eli Schwartz 28103614ef
refactor errorhandler for deduplication of logic
Mostly just move a comment around back to where it's more relevant, and
handle MESON_FORCE_TRACEBACK once instead of twice.
2022-09-22 15:24:56 -04:00
Eli Schwartz d364985365
fix regression in handling errors during reconfigure
In commit 9ed5cfda15 we refactored
startup to be a bit faster and import less. But this had the side effect
of moving out of our errorhandler. Refactor this so it can be easily
used elsewhere.
2022-09-22 15:23:39 -04:00
Eli Schwartz 9ed5cfda15
avoid importing the entire codebase at first startup
We want to optimize out some internal codepaths used at build time by
avoiding work such as argparse. This doesn't work particularly well when
the argparse arguments are imported before then. Between them, they
indirectly import pretty much all code anywhere, and msetup alone
imports most of it.

Also make sure the regenerate internal script goes directly to msetup.
2022-09-19 15:19:00 -04:00
Eli Schwartz 2fe07adc0e
fixup! deprecate running "meson builddir" without setup subcommand
Also pass the setup command when rewriting --internal regenerate. This
avoids the issue where `ninja` triggers a reconfigure, and this warning
gets printed as a side effect.
2022-08-16 23:39:37 -04:00
Eli Schwartz 3c7ab542c0
deprecate running "meson builddir" without setup subcommand
This is ambiguous, if the build directory has the same name as a
subcommand then we end up running the subcommand. It also means we have
a hard time adding *new* subcommands, because if it is a popular name of
a build directory then suddenly scripts that try to set up a build
directory end up running a subcommand instead.

The fact that we support this at all is a legacy design. Back in the
day, the "meson" program was for setting up a build directory and all
other tools were their own entry points, e.g. `mesontest` or
`mesonconf`. Then in commit fa278f351f we
migrated to the subcommand mechanism. So, for backwards compatibility,
we made those tools print a warning and then invoke `meson <tool>`. We
also made the `meson` tool default to setup.

However, we only warned for the other tools whose entry points were
eventually deleted. We never warned for setup itself, we just continued
to silently default to setup if no tool was provided.

`meson setup` has worked since 0.42, which is 5 years old this week.
It's available essentially everywhere. No one needs to use the old
backwards-compatible invocation method, but it continues to drag down
our ability to innovate. Let's finally do what we should have done a
long time ago, and sunset it.
2022-08-16 17:40:03 -04:00
Omer Pereg fa044e011d handle OSError exception in run function 2022-07-31 14:54:44 +03:00
Prathamesh 7af0078d4e Resolve KeyError while executing help command with unknown param 2022-07-17 00:45:26 -04:00
Xavier Claessens 1a3d29dd0e Move python version check as first thing 2022-06-22 09:13:05 -07:00
Eli Schwartz bb171c2dff
pyupgrade --py37-plus
Some more old style code crept in again.

Additionally, pyupgrade learned to catch more if/elsed code based on the
python version, and delete it.
2022-04-14 18:37:03 -04:00
Christoph Reiter 26903d7b12 main: Make the msys/python check work again, in some cases
msys/python in MSYS2 pretends to be cygwin in all cases for some time
now, so this check was impossible to hit.

The underlying confusion it tried to prevent is still there, namely trying
to build with mingw but wrongly using a msys/cygwin python/meson.

We can use the MSYSTEM env var to detect if we are in a mingw shell, and
in case the Python doesn't match we suggest installing mingw variants of both
python and meson.

Using msys/python + meson in a MSYS environment works fine on the other hand,
so no need to error out in that case.

Fixes #8726

Also addresses the concern raised in
https://github.com/mesonbuild/meson/issues/3653#issuecomment-474122564
2022-04-01 13:15:33 -04:00
Elliott Sales de Andrade 969ae6e0e1 Correctly handle --version argument to runpython
Followup to #10204.
2022-03-29 21:57:46 -04:00
Eli Schwartz 80192aa577
runpython: support --version
argparse is the gift that keeps on giving, hahaha. Suppress the script
argument when --version is specified to avoid "required argument not
provided" errors, and print the python version.

The version argument is required in order to make this baseline
functional as a resolved python for find_program, which may specify a
version and expect this to work with python itself. Our incomplete CLI
wrapper over the python CLI interface was missing this.

Fixes #10162
2022-03-29 01:26:18 -04:00
Jussi Pakkanen 04ef7ec45d Add new env2mfile command. 2022-03-09 18:02:04 -05:00
Eli Schwartz 7cc4ca2cbb
Revert "Add new env2cross command."
This reverts commit e257a870fe.

The PR adding this command had infinitely hanging CI, and now that it is
merged to master we cannot get any CI on any PR to succeed.
2022-03-08 21:34:39 -05:00
Jussi Pakkanen e257a870fe Add new env2cross command. 2022-03-08 23:40:03 +02:00
Eli Schwartz a6e08e8fa7
use a more sane check instead of run_custom_lint
Unfortunately, checking for strings without context is exceedingly prone
to false positives, while missing anything that indirectly opens a file.

Python 3.10 has a feature to warn about this though -- and it uses a
runtime check which runs at the same time that the code fails to open
files in the broken Windows locale. Set this up automatically when
running the testsuite.

Sadly, Python's builtin feature to change the warning level, e.g. by
setting EncodingWarning to error at startup, is utterly broken if you
want to limit it to only certain modules. This is tracked in order to be
more efficiently ignored at https://bugs.python.org/issue34624 and
https://github.com/python/cpython/pull/9358

It is also very trigger happy and passing stuff around via environment
variable either messes with the testsuite, or with thirdparty programs
which are implemented in python *such as lots of gnome*, or perhaps
both.

Instead, add runtime code to meson itself, to add a hidden "feature".
In the application source code, running the 'warnings' module, you can
actually get the expected behavior that $PYTHONWARNINGS doesn't have. So
check for a magic testsuite variable every time meson starts up, and if
it does, then go ahead and initialize a warnings filter that makes
EncodingWarning fatal, but *only* when triggered via Meson and not
arbitrary subprocess scripts.
2022-03-01 21:42:52 -05:00
Eli Schwartz 078175314a
fine-tune the logic for reporting context on tracebacks
Do not report a MesonBugException if the command is `runpython`, because
that is 100% other people's code, not ours. It's only used as an
alternative to sys.executable for reporting a path to python, in the
event of a Windows MSI install.

While we are at it, refactor the logic for PermissionError to be a bit
more unified (and sadly use isinstance instead of except, but it is what
it is).
2022-02-15 23:51:46 -05:00
Eli Schwartz 0db2de2b0a
do not report context on python traceback, for PermissionError
It's not a MesonBug which needs to be reported, and the existing error
already adequately points out the problematic file.

It is impossible to get a PermissionError for files created by meson
itself, once the build directory has been created, anyway.
2022-01-27 18:49:17 -05:00
Eli Schwartz ad525dcce4
bump minimum required version of python to 3.7
Comment out the pending deprecation notice. It cannot be reached
anymore, but is still useful for the next time we do a version bump.
2022-01-10 18:36:56 -05:00
Eli Schwartz 09f03a8424 add pending deprecation notice for python 3.6 2022-01-10 01:36:46 +02:00
Eli Schwartz 3c039f42a0
report the context, if possible, on python tracebacks
The interpreter tries to catch any exception and add the latest node
information to it, but currently we only used that to print better
formatted error messages on MesonException.

Since we should theoretically have that property for most/all
exceptions, let's percolate that upward, and message the user that an
unexpected traceback was encountered, that it should be reported as a
bug, and the helpful information of "how far into parsing this
meson.build did we get before erroring out, anyway?"
2021-11-29 16:36:51 -05:00
Xavier Claessens 928078982c Add --vsenv command line option and active VS only when needed 2021-10-10 23:15:18 +03:00
Dylan Baker 4d7031437c pylint: turn on superflous-parens
We have a lot of these. Some of them are harmless, if unidiomatic, such
as `if (condition)`, others are potentially dangerous `assert(...)`, as
`assert(condtion)` works as expected, but `assert(condition, message)`
will result in an assertion that never triggers, as what you're actually
asserting is `bool(tuple[2])`, which will always be true.
2021-08-31 16:28:54 -04:00
xth a79980da6b fix bat_info UTF-8 string error
" bat_info = json.loads(bat_json) " may produce error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 ...
Because the vswhere.exe's output is not utf-8 by default
Use UTF-8 encoding for vswhere.exe can fixing it .
2021-08-03 00:16:11 +03:00
Daniel Mensinger 3e396b3782
fix: Always explicitly set encoding for text files (fixes #8263) 2021-06-29 11:28:08 +02:00
Daniel Mensinger 28175bbee2
pathlib: Patch pathlib to work around some bugs (fixes #7295) 2021-06-29 11:28:08 +02:00
Daniel Mensinger 7fc755b334 typing: Fully annotate run_project_tests.py 2021-06-09 13:25:36 +02:00
Xavier Claessens 25fa2d4f7b vsenv: Recommend using "meson compile" wrapper
When meson has setup the VS environment, running ninja to build won't
work, user should use meson wrapper to compile.
2021-05-28 17:34:25 -04:00
Naveen M K 495e76d10a Remove `Microsoft.VisualStudio.Workload.WDExpress` 2021-05-28 15:16:37 -04:00
Naveen M K 7ec5267939 Find Visual Studio Build Tools 2019
Got the Idea from setuptools
a5131f0b82/setuptools/msvc.py (L176)
2021-05-28 15:16:37 -04:00
Jussi Pakkanen 799b96f753 Also skip VS activation if gcc is found. 2021-05-23 21:33:24 +01:00
Jacob Nielsen 3e5ec00a40
Better detection of Visual Studio compiler 2021-05-22 01:05:18 +03:00
Jussi Pakkanen 0b2a7300c0 Set up VS environment automatically when run. 2021-05-13 11:03:10 +03:00
Xavier Claessens 598e968993 Add `meson devenv` command and meson.add_devenv() 2021-03-16 09:00:50 -04:00
Eli Schwartz 6a0fabc647
mass rewrite of string formatting to use f-strings everywhere
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04 17:16:11 -05:00