Commit Graph

83 Commits

Author SHA1 Message Date
Xavier Claessens 4c5e976a75 msubprojects: Fix --sourcedir argument not working
This is a regression caused by
https://github.com/mesonbuild/meson/pull/11951.

Fixes: #12466
2023-11-06 12:52:09 -05:00
Luke Elliott ce691f8c98 Add comments suggesting to keep shell completion scripts up-to-date near cmd line argument code 2023-11-01 00:06:19 +02:00
Tristan Partin 46cedeb431 Support -j as a shorthand for --num-processes
We already use -j to support parallelism in meson compile. So let's add
the same for meson test and meson subprojects.
2023-10-20 12:49:38 -04:00
Nirbheek Chauhan bd0ab92bba msubprojects: Fix typo 'any way' -> 'anyway' 2023-10-17 13:02:23 +05:30
Xavier Claessens 746c4eff48 msubprojects: Fix crash if wrapdb_version is in bad format
Fixes: #12378
2023-10-17 13:02:23 +05:30
Nirbheek Chauhan 3c6f04b294 msubprojects: Checkout if the branch is tracking upstream
A rebase of the current branch is the wrong thing to do if the
revision (branch) specified in the wrap changed, which is the case in
the majority of cases, such as when switching from one release branch
to another.
2023-10-11 23:48:13 +05:30
Nirbheek Chauhan b78aa8f9e9 msubprojects: Abort the rebase if there's conflicts
Don't leave the subproject tree in a broken / conflicting state. The
user is most likely not a git expert who will know what magic command
to run to fix their source tree.
2023-10-11 23:48:13 +05:30
Xavier Claessens 49e7e3b9cc Allow to fallback to cmake subproject
The method can be overridden by setting the `method` key in the wrap
file and always defaults to 'meson'. cmake.subproject() is still needed
in case specific cmake options need to be passed.

This also makes it easier to extend to other methods in the future e.g.
cargo.
2023-09-22 15:50:26 -04:00
Xavier Claessens 1b6c9ad02a msubprojects: Speedup subproject_dir extraction
The interpreter takes significant amount of time to initialize
everything in project() function. We only need to extract a string from
AST, just like we do in handle_meson_version_from_ast().
2023-09-12 17:32:17 +05:30
Eli Schwartz 0bb1647fd1
move various bits of type-checking only code to TYPE_CHECKING blocks
Mostly detected with flake8-type-checking. Also quote T.cast() first
arguments, since those are not affected by future annotations.
2023-07-19 18:31:37 -04:00
Rafael Silva 339b248abc parse meson.build for subproject_dir in msubprojects
we parse project from source meson.build with IntrospectionInterpreter
2023-07-10 17:52:42 +03:00
Rafael Silva 6cb7ef1f3b rename msubprojects src_dir to source_dir
source_dir seems to be the most common name across the code base with 400+ hits
2023-07-10 17:52:42 +03:00
Xavier Claessens 11521c6db7 ProgressBar: Fix some rendering issues
- Do not hardcode terminal width of 100 chars, that breaks rendering on
  smaller terminal. It already uses current console width by default.
- Disable progress bar when downloading from msubprojects because it
  fetches multiple wraps in parallel.
- Scale unit when downloading e.g. MB/s.
- Do not display rate when it's not a download.
- Do not display time elapsed to simplify the rendering.
2023-05-25 11:06:09 -04:00
Eli Schwartz f6d13c69e5
fix regression that broken git-based wraps with dirty contents
In commit 8da060706c we fixed an issue
where stashing would ignore untracked files, potentially conflicting
with what we reset to.

Unfortunately in the process it broke the functionality entirely, by
producing an invalid git command... according to some versions of git.
This is actually a git bug, fixed upstream in git 2.37.0 via commit
b02fdbc80a41f73ceb6c99e8e27b22285243a335. It causes git itself to
re-synthesize the command line for a negation pathspec with a "."
rather than "", as that's what is needed to make it match all files
other than the negation.

For backwards compatibility to older versions of git, we manually pass
the dot pathspec ourselves.
2023-03-30 22:42:41 -04:00
Xavier Claessens 5c5d261845 Fix "meson subprojects checkout -b" regression
The argument position is wrong since
1c631ec8ab
2023-03-17 15:17:56 -04:00
Benoit Pierre 8da060706c msubprojects: fix potential error when resetting a git checkout
Untracked files need to be stashed too, or resetting may fail when
trying to (re-)apply a patch that adds one of those untracked files.
2023-03-04 20:35:30 -05:00
Eli Schwartz c5e766af75
be more explicit about asyncio usage
`get_event_loop()` would always implicitly create one on demand, but
Python upstream has decided it's a bad/confusing API. Explicitly
starting our own, then setting it, is exactly equivalent other than not
being scheduled for deprecation.
2023-01-04 16:46:52 -05:00
Xavier Claessens 7737398cd3 msubprojects: Fix issues when updating all wraps
When updating all wraps, it is not an error if some have not been
downloaded or some does not have a wrap file.
2022-10-21 17:35:26 +03:00
Xavier Claessens aaabd6224f Make `meson wrap update` command update all wraps in parallel
This moves the implementation into msubprojects because it has all the
infrastructure to update wraps in parallel while keeping "meson wrap"
UX.
2022-10-11 15:54:59 -04:00
Tristan Partin 838367ca60 Fix mypy lint
Unused "type:ignore"
2022-09-27 11:14:19 -04:00
Daniel Carson 004575874f Warn if wrap file changes
Save off the hash of the wrap file when first configuring a subproject.
When reconfiguring a subproject, check the hash of the wrap file
against the stored hash. If they don't match then warn the user.
2022-09-18 22:48:50 -04:00
Eli Schwartz a49cd00d64 treewide: various cleanups to move imports for mypy into typechecking blocks
Along the way, add __future__ annotations where lacking.
2022-06-10 09:15:48 -04:00
Paweł Marczewski 9061c3a52d wrap: Add support for applying a list of patch files
Co-authored-by: Xavier Claessens <xavier.claessens@collabora.com>
2022-06-07 06:43:39 -04:00
Eli Schwartz bb171c2dff
pyupgrade --py37-plus
Some more old style code crept in again.

Additionally, pyupgrade learned to catch more if/elsed code based on the
python version, and delete it.
2022-04-14 18:37:03 -04:00
Eli Schwartz 479a84455f wrap: implement allow-insecure for 'meson wrap' 2022-03-27 18:57:07 -04:00
Eli Schwartz b28e6aead4 wrap: add functionality to specify whether insecure downloads should be used
We have a fallback route in `meson subprojects download` and friends,
which tries to retrieve wrapdb urls via http, if Python was not built
with SSL support.

Stop doing this. Replace it with a command line option to specify that
insecure downloads are wanted, and reference it in the error message if
downloading fails due to SSL issues.
2022-03-27 18:57:07 -04:00
Eli Schwartz 60c893f1c2
flake8: remove some redundant separators 2022-02-16 18:19:13 -05:00
Eli Schwartz 4b351aef26
first pass at migrating to dataclasses
In some cases, init variables that accept None as a sentinel and
immediately overwrite with [], are migrated to dataclass field
factories. \o/

Note: dataclasses by default cannot provide eq methods, as they then
become unhashable. In the future we may wish to opt into declaring them
frozen, instead/additionally.
2022-01-10 18:36:57 -05:00
Tristan Partin b361fc52dd Add typing to msubprojects.py 2021-11-28 07:10:12 -05:00
Eli Schwartz e72da5ff5a msubprojects: implement "packagefiles --save"
Copies packagefiles from the subproject back to the patch_directory for
safe storage and to check into git etc.
2021-10-10 10:17:55 -04:00
Eli Schwartz 8e50ba732e msubprojects: add new subcommand "packagefiles"
This will re-apply the meson.build patch overlay, ensuring it is up to
date. Also take the opportunity offered by this infrastructure to
repatch when performing `update --reset` since internally this will run
`git stash` and thus cause the (possibly locally modified) meson.build
files to disappear.
2021-10-10 10:17:55 -04:00
Eli Schwartz 352380f659 Revert "msubprojects: if fetching the remote fails, gracefully fallback to local copy"
This reverts commit 4568482316.

As it turns out, the rationale for this was completely bogus. This
command doesn't re-apply the patch_directory etc. and in fact there is
no command whatsoever that does this. So, this command does not have two
purposes, and we are not making one of them more robust -- instead it
has one purpose, and we are making it lie about whether it failed.

Instead of trying to freeload off of this command, we will just add
another command to properly apply patch overlays.
2021-10-10 10:17:55 -04:00
Eli Schwartz 99112d6c79
msubprojects: when revision is an available commit, do not fetch updates
It's not even worth bothering to contact the network because it can
never be anything other than what we have already.
2021-09-19 23:34:33 -04:00
Eli Schwartz 4568482316
msubprojects: if fetching the remote fails, gracefully fallback to local copy
This command is useful to e.g. update a cloned subproject which does not
have its packagefiles merged, or which has updated packagefiles. This
does not strictly require internet, so if we can satisfy the ref, simply
log a warning and continue.

This enables a workflow where for network-isolated builds, the
subproject is cloned and moved into place and later on the build
orchestrator runs `meson subprojects update` without a network interface
to initialize the subproject.
2021-09-19 23:34:33 -04:00
Eli Schwartz 82e68e98ad
indent code in preparation for refactoring
Because figuring out what changed on a single line in the middle of a
reindent is pain.
2021-09-19 23:29:26 -04:00
Eli Schwartz a387e8d743
fix traceback while trying to print error message 2021-09-14 15:50:29 -04:00
Andrei Alexeyev 254969b128 msubprojects: make purge print full paths to comply with the tests 2021-07-27 13:11:46 +03:00
Andrei Alexeyev 5d88c45861 msubprojects: do not follow symlinks when purging
Fixes #8891
2021-07-27 13:11:46 +03:00
Xavier Claessens 2379fc72bc msubprojects: catch exceptions and continue with others
If the command fails for one subproject we should still continue with
others. Failed subprojects are reported at the end.

This issue became more problematic when doing parallel tasks because the
reason the command fails was completely hidden by other parallel tasks.
2021-07-21 19:45:37 +02:00
Tristan Partin f21685a833 Delete redirected wrap files in subprojects purge
We need to store the original filename as well as whether the wrap was
redirected in order to properly purge the redirected wrap.
2021-07-03 13:14:20 +03:00
Xavier Claessens 4aaccdb6ba msubprojects: Display progress while tasks are running
Logs are printed only when the task is done to not interleave logs while
running multiple tasks in parallel. That means that nothing is printed
until the task is done and that could take time.
2021-06-25 15:16:55 -04:00
Xavier Claessens 27970b1d85 msubprojects: Add message at the end of purge 2021-06-25 15:16:55 -04:00
Xavier Claessens 2b6fe00e3d msubprojects: Run action on all subprojects in parallel
This moves all the code into a class and call its run() method in a
thread. The class queues all logs to print them at the end to avoid
mixing output of multiple actions.
2021-06-25 15:16:55 -04:00
Xavier Claessens 62c53b834d wrap: Port to v2 protocol
Fixes: #8754.
2021-06-04 15:35:29 -04:00
Tristan Partin bb12587e0b Add subprojects purge wrap-git coverage 2021-04-26 12:06:12 -04:00
Tristan Partin b6d277c140 Add 'subprojects purge' command
This will help facilitate cache busting in certain situations, and
replaces hand-rolled solutions of writing a length command to remove
various files/folders within the subprojects directory.
2021-04-26 16:42:29 +03:00
Xavier Claessens 1ad469342b msubprojects: wrap-file should be re-extracted with --reset
When using --reset we should guarantee that next reconfigure will pick
the latest code. For wrap-file we have no way to know if the revision
changed, so we have to delete the source tree and extract again.

It is unlikely that user has local changes in non-git subprojects, and
--reset is known to be dangerous.
2021-03-28 23:24:01 +03:00
Eli Schwartz 6a0fabc647
mass rewrite of string formatting to use f-strings everywhere
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04 17:16:11 -05:00
Nirbheek Chauhan 1c631ec8ab msubprojects: Ignore worktrees when checking out
Otherwise it's not possible to share git subprojects via worktrees
when creating a worktree of a git repository that uses meson
subprojects.

The downside is that the user needs to be careful while adding commits
to each tree's index.
2020-12-07 09:02:44 -05:00
Dylan Baker f6672c7a19 use real pathlib module
We added the _pathlib module to work around defeciencies in python 3.5's
implementation, since we now rely on 3.6 lets drop this
2020-11-20 15:08:40 -08:00