Commit Graph

280 Commits

Author SHA1 Message Date
Dylan Baker e991c4d454 Use SPDX-License-Identifier consistently
This replaces all of the Apache blurbs at the start of each file with an
`# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing
uses to be consistent in capitalization, and to be placed above any
copyright notices.

This removes nearly 3000 lines of boilerplate from the project (only
python files), which no developer cares to look at.

SPDX is in common use, particularly in the Linux kernel, and is the
recommended format for Meson's own `project(license: )` field
2023-12-13 15:19:21 -05:00
Xavier Claessens 11dec13a19 mtest: -C argument does not need type convertor
It is already done by RealPathAction and mypy started complaining about
it.
2023-11-10 14:22:30 -05:00
Luke Elliott ce691f8c98 Add comments suggesting to keep shell completion scripts up-to-date near cmd line argument code 2023-11-01 00:06:19 +02:00
Tristan Partin 46cedeb431 Support -j as a shorthand for --num-processes
We already use -j to support parallelism in meson compile. So let's add
the same for meson test and meson subprojects.
2023-10-20 12:49:38 -04:00
Sam James 7b7d2e060b mtest: set ASAN_OPTIONS and UBSAN_OPTIONS to abort by default
Do as we do for MALLOC_PERTURB and set a sensible value for both ASAN_OPTIONS
and UBSAN_OPTIONS to abort on failure and give more helpful output at the
same time. We do not set these options if the user has exported a value
themselves to allow override.

In the last week alone, I've observed two cases where people were expecting
sanitizers to abort on failure and were surprised when it didn't:
1) 252d693797
2) c47df433f7

Correct this - which is in-line with meson's DWIM/DTRT philosophy.

Signed-off-by: Sam James <sam@gentoo.org>
2023-10-19 14:47:19 -04:00
Tristan Partin 543e9ca0cf Remove XML filter from testlog.{json,txt} and std streams
This was an unintended consequence of the original patch in #11977.

Co-authored-by: Benoit Pierre <benoit.pierre@gmail.com>
2023-08-17 17:31:30 -04:00
Eli Schwartz 90ce084144
treewide: automatic rewriting of all comment-style type annotations
Performed using https://github.com/ilevkivskyi/com2ann

This has no actual effect on the codebase as type checkers (still)
support both and negligible effect on runtime performance since
__future__ annotations ameliorates that. Technically, the bytecode would
be bigger for non function-local annotations, of which we have many
either way.

So if it doesn't really matter, why do a large-scale refactor? Simple:
because people keep wanting to, but it's getting nickle-and-dimed. If
we're going to do this we might as well do it consistently in one shot,
using tooling that guarantees repeatability and correctness.

Repeat with:

```
com2ann mesonbuild/
```
2023-08-11 13:41:03 -04:00
Eli Schwartz de1cc0b02b
rewrite a couple comment-style type annotations for oddly indented dicts
Make them into real type annotations. These are the only ones that if
automatically rewritten, would cause flake8 to error out with the
message: "E128 continuation line under-indented for visual indent".
2023-08-11 13:39:07 -04:00
Eli Schwartz a01418db0a
remove useless type annotations
These annotations all had a default initializer of the correct type, or
a parent class annotation.
2023-08-11 13:37:17 -04:00
Eli Schwartz 7afc69254d
fix implicit_reexport issues and enforce them going forward
This detects cases where module A imports a function from B, and C
imports that same function from A instead of B. It's not part of the API
contract of A, and causes innocent refactoring to break things.
2023-07-19 18:31:37 -04:00
Eli Schwartz 6894bb5a30 mtest: avoid meddling with stdout by default
The original point of specifying Optional was to default to None...
oops. The practical effect of this change is that the testsuite no
longer repeatedly logs "No tests defined." in between more meaningful
output.
2023-07-18 21:58:58 -04:00
Nazir Bilal Yavuz bd3d2cf918 mtest: fix unencodable XML chars
Replace unencodable XML chars with their printable representation, so
that, xmllint can parse test outputs without error.

Closes #9894

Co-authored-by: Tristan Partin <tristan@partin.io>
2023-07-13 09:38:55 -07:00
Eli Schwartz 1a72f00252
mtest: avoid stdout when printing warnings about the lack of things to print
Since people may parse the output of `--list` as a list of tests,
putting logging info in stderr is nicer.
2023-07-10 20:10:04 -04:00
Eli Schwartz 2fd9055810
mtest: redirect automatic reconfiguring to stderr when listing tests
It is not the primary purpose of mtest, and it ends up mingled with a
list of actual tests, which isn't nice if you're trying to capture and
parse this.
2023-07-10 20:10:02 -04:00
Eli Schwartz 9d446d80db
mtest: try a bit harder to avoid weird non-parseable output at startup
In commit 628effb369 we started verifying
the build.ninja file was up to date before even `--list`ing tests,
because we could end up with incorrect information. This meant that
ninja always runs at startup, and typically returns "no work to do",
which ended up listed as "one of" the tests.

Instead of unconditionally running ninja attached to the console, first
check it in dry-run mode with stdout intercepted, to see if ninja
considers itself up to date. If it is, continue. Only if an actual
refresh is needed, do we run it while attached to the console.

In the garden path, this avoids useless information. In cases where we'd
already print a full meson reconfigure log, we continue to do so.
2023-07-10 20:08:01 -04:00
Charles Brunet e7b9dfac98 mtest: wildcard selection
Allow the use of wildcards (e.g. *) to match test names in `meson test`.

Raise an error is given test name does not match any test.

Optimize the search by looping through the list of tests only once.
2023-05-25 13:44:13 -04:00
Charles Brunet 70e2da0773 mtest: prevent parse error with gtest protocol
Replace illegal characters when reading gtest generated xml file,
to prevent a ParseError and a stacktrace.

catch et.ParseError, just in case, to prevent stopping other tests
if the xml file was malformed.
2023-04-24 15:48:31 -04:00
Josh Soref cf9fd56bc9 fix various spelling issues
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-04-11 19:21:05 -04:00
HiPhish 439a61affa Change "can not" to "cannot" throughout project
The word "cannot" expresses inability to do something whereas "can not"
expresses the ability to refrain from doing something.
2023-04-11 17:10:01 +03:00
Xavier Claessens b30cd5d2d5 Make --vsenv a readonly builtin option
We need to remember its value when reconfiguring, but the Build object
is not reused, only coredata is.

This also makes CLI more consistent by allowing `-Dvsenv=true` syntax.

Fixes: #11309
2023-03-29 09:33:41 -04:00
Eli Schwartz a024d75e15
backends: add a new "none" backend
It can only be used for projects that don't have any rules at all, i.e.
they are purely using Meson to:

- configure files
- run (script?) tests
- install files that exist by the end of the setup stage

This can be useful e.g. for Meson itself, a pure python project.
2023-03-20 17:22:50 -04:00
Eli Schwartz fccada6703
mtest: move the detection of rebuild compatibility out into the entry point
What we are mainly doing here is checking that the options make sense,
or fixing that up if they don't. And in the next commit we will want to
do that by also checking the build object.
2023-03-20 16:58:15 -04:00
Dylan Baker b2f7f2da1c mtest: fix annotaion of stdo_task and stde_task
Which are `Task`s, not `Future`s, and they return `None`, not `str`.
Spotted by newer versions of mypy
2023-02-08 15:31:58 +05:30
Eli Schwartz d6b81307f6
pylint 2.16: remove pointless parens around equality assignments
Given the construct `foo = (bar == baz)` some people like parentheses
and some do not. They're pointless and don't mean anything, though. I
don't feel this is particularly helpful to code clarity, tbh, and pylint
now notices this and warns about it in our current pylint config.

I think this is reasonable, so let's remove the odd parens.
2023-02-01 17:01:30 -05:00
Eli Schwartz a21af43200
micro-optimize: define typing-only objects in TYPE_CHECKING
Union types that exist solely for use as annotations don't need to be
created in normal runs.
2023-02-01 17:01:30 -05:00
Eli Schwartz 680b5ff819
treewide: add future annotations import 2023-02-01 17:01:30 -05:00
Eli Schwartz 8d39c9273b mtest: do not warn when the TAP stream version is 12
Partial rollback of commit b7a5c384a1. The
rationale was based on a confusing wording of the TAP14 spec, which is
under discussion for clarification / amendment.

TAP14 doesn't (shouldn't) really say that missing a version line is
potentially an error. Rather, this is the correct way to denote TAP12,
which a TAP14 harness may not understand or try to parse. The intention
was never to suggest that harnesses "should" take exception to the
missing version line on the grounds that one should really add a version
line.

So, stop emitting an annoying warning for something that's valid usage.
Meson understands TAP12 and that's okay.

However, we do need to keep the part of that commit which set the
version to 12 if it was otherwise unspecified. But instead of
distinguishing between None and a version, just default to 12.
2023-01-22 18:45:35 -08:00
Eli Schwartz 40315e6ebb
mtest: simplify deprecated access to current loop
These functions constantly want the current asyncio loop, and we run a
function call each time to get it. And the function call is a deprecated
one. Python 3.7 brings the more explicit get_running_loop for use when
we know we're inside one, with the aim of getting rid of get_event_loop
once support for python <3.7 disappears. Meson no longer supports python
<3.7 either.

Switch to the new API, and save the reference for reuse instead of
constantly re-calculating it.
2023-01-04 16:46:52 -05:00
Eli Schwartz ab8081fab6
mtest: clean up asyncio event loop instantiation
Fix a TODO comment about moving to asyncio.run, now that we use
sufficiently new python to do it.

Note that we create an event loop for Windows using the new python
defaults, but in a completely different part of the code from where we
need to use it. Since asyncio.run creates the loop on its own, we need
to set the default policy instead -- which we probably should have done
all along.
2023-01-04 16:46:51 -05:00
Eli Schwartz 56312c0579
mtest: delay creation of asyncio lock until event loop exists
In https://bugs.python.org/issue42392 this stopped implicitly creating
an event loop if none exists. We created it before running _run_tests(),
so it would auto-create an event loop and set the default, which means
we cannot create one explicitly on our own schedule or we end up with
two of them.

Delay this until we actually start the logger. This happens inside the
actual testsuite loop, so it finds the running loop and doesn't create a
new one, even on python <3.10.
2023-01-04 16:46:51 -05:00
Eli Schwartz b7a5c384a1 mtest: handle TAP tests with unknown version.
TAP 14 states:
> Harnesses may treat any TAP stream lacking a version as a failed test.

TAP 13 states:
> In the absence of any version line version 12 is assumed. It is an
> error to explicitly specify any version lower than 13.

So, modern TAP is saying that we should treat a missing version as a
test definition bug, it's no longer okay to use a missing version as
saying "let's use TAP 12". But, we can choose whether to treat it that
way or error out.

Let's do a diagnostic, as we do elsewhere. But allow TAP streams that
aren't well defined, if they used to be well defined (back in TAP 12).
2022-12-17 00:26:38 +02:00
Eli Schwartz d0054f2c3c
mtest: warn on invalid TAP output
In commit a7e458effa we stopped erroring
out on invalid TAP stream contents, with the rationale that "prove" has
become more lenient.

A close reading of the TAP spec indicates why, though:

> A TAP parser is required to not consider an unknown line as an error but
> may optionally choose to capture said line and hand it to the test
> harness, which may have custom behavior attached. This is to allow for
> forward compatability. Test::Harness silently ignores incorrect lines,
> but will become more stringent in the future. TAP::Harness reports TAP
> syntax errors at the end of a test run.

The goal of treating unknown lines as an error in the TAP parser is not
because unknown lines are fine and dandy. The goal is to allow
implementing future versions of TAP, and handling it via existing
parsers. Since Meson has both a parser and a harness, let's do exactly
that -- pass these lines as a distinctive status to the test harness,
then have the test harness complain.
2022-12-05 15:46:46 -05:00
Eli Schwartz 7c9705b801
mtest: early return when parsing blank lines in TAP
Just like comment lines, blank lines do nothing. Before commit
a7e458effa we ended off the parser by
returning if the line was blank, because we needed to in order to catch
non-blank lines as errors. But really, we should have always returned
much earlier and not wasted time attempting to process anything.
2022-12-05 15:46:46 -05:00
Jussi Pakkanen 3ae89a7150 Handle freezing tests. Fixes #10752. 2022-11-21 16:44:24 +02:00
Eli Schwartz 462759dd33
mtest: implement a maxfail option
This allows early exit of the project tests once a certain number of
failures are detected. For example `meson test --maxfail=1` will abort
as soon as a single test fails.

Currently running tests are marked as failed via INTERRUPT.

Resolves #9352
2022-09-23 16:10:32 -04:00
Dylan Baker b8e53ed5ea pylint: enable use-dict-literal 2022-09-19 20:57:52 -04:00
Dylan Baker 798b6c5624 pylint: enable use-list-literal 2022-09-19 20:57:52 -04:00
Andres Freund 628effb369 mtest: Run ninja build.ninja before loading tests
When the build definition has changed since the last ninja invocation meson
test operated on an outdated list of tests and their dependencies. That could
lead to some tests not being run / not all dependencies being built. This was
particularly confusing because the user would see the output of
reconfiguration and rebuilding, and the next mtest invocation would have the
updated configuration.

One issue with this is that that we will now output more useless ninja output
when nothing needs to be done (the "Entering directory" part is not repeated,
as we happen to be in the build directory already). It likely is worth
removing that output, perhaps by testing if anything needs to be done with
ninja -n, but that seems better addressed separately.

Fixes: #9852
2022-09-04 19:05:14 -04:00
Andres Freund f501e3b187 mtest: pull detection of ninja into TestHarness
A later commit will add a second invocation of ninja, no point in having the
detection code twice.

This changes the exit code in case ninja isn't found from 125 to 127, which
seems more appropriate given the justification for returning 125 (to make git
bisect run skip that commit). If we can't find ninja we'll not succeed in
other commits either.
2022-09-04 19:05:14 -04:00
Andres Freund 2096e394fc mtest: Move loading of test data into its own function
A subsequent commit will do a bit more during test data loading, making a
dedicated function seem advisable.

The diff looks a bit odd, using git show --diff-algorithm=patience
will make it clearer.
2022-09-04 19:05:14 -04:00
Dimitri John Ledkov c3cc66a1cc mesonbuild/mtest.py: filter more invalid googletest JUnit4 attributes
googletest 1.12.1 generates new JUnit4 invalid attributes file and
line.

Maybe all gtest "invalid" attributes are actually valid JUnit5
attributes, and maybe schema should be upgraded to JUni5.

Signed-off-by: Dimitri John Ledkov <dimitri.ledkov@canonical.com>
2022-08-12 21:16:07 +03:00
Hemmo Nieminen 952dd7773d mtest: unify parsed and non-parsed output handling
Use the same routines to handle output both when parsing the output and
when not. Also fixes broken stderr handling for parsed tests.
2022-08-07 20:48:42 +03:00
Elliott Sales de Andrade 27626124d0 Inline try_wait_one using asyncio.wait_for
Although the former accepts multiple awaitables, it is only ever called
with a single one, so just use `wait_for` instead.

Additionally, the `try_wait_one` fails in Python 3.11, as
`Process.wait()` returns a coroutine, and `asyncio.wait` only accepts
tasks, so it errors out.
2022-08-01 15:51:16 -04:00
Xavier Claessens 57909b53d6 Improve WINEPATH reduction
- Remove duplicated code in mdevenv.py
- Change the limit to 1024 instead of 2048 which is what has been
  tested.
- Skip shortening if it is already short enough.
- Skip shortening with wine >= 6.4 which does not seems to have that
  limitation any more.
- Downgrade exception to warning in the case WINEPATH cannot be
  shortened under 1024 chars, it is possible that it will still work.
2022-06-17 11:53:38 -04:00
Hemmo Nieminen e8a3f4d38c mtest: do not process zero byte reads in read_decode()
AsyncIO.StreamReader.readuntil() occasionally raises IncompleteRead
exception before a byte of data has been read. Do not process the "read"
data in those cases.
2022-06-09 18:58:33 -04:00
Hemmo Nieminen 657a6eeb81 mtest: enable access to the console logger instance
Store a reference to the console logger instance in a test harness'
member variable to allow accessing it (and its logging utilities) from
any other functions in test harness.

This added functionality will be used in future commits.
2022-06-09 18:58:33 -04:00
Hemmo Nieminen 4304df30c2 mtest: store all test results directly to TestRun
Store return code, test result and additional error directly to the
relevant TestRun instance. This reduces the number of individual
arguments to other relevant functions that need to be passed around and
thus simplifies the code. The test output (and error) were earlier
similarly moved to be stored directly to the TestRun instance for the
same reason.
2022-06-09 18:58:33 -04:00
Hemmo Nieminen 702f3fe15f mtest: store test output directly to a TestRun object
By storing test output directly to the TestRun instance we avoid the
need to pass the outputs around in individual function arguments thus
simplifying the code.

The amount of individual arguments will be further reduced in a
future commit.
2022-06-09 18:58:33 -04:00
Hemmo Nieminen a0eb04788d mtest: print only one empty line before test summary 2022-06-09 18:58:33 -04:00
Hemmo Nieminen 565b12183f mtest: improve text test log formatting
Write additional metadata such as exit details and try to visually
differentiate the tests better from each other in the (textual) test
logs.
2022-06-09 18:58:33 -04:00