Commit Graph

141 Commits

Author SHA1 Message Date
Eli Schwartz 4b9ec4f6d5 wraps: implement exponential backoff for downloading files
Implementation modeled after mesonlib.windows_proof_rmtree().

Implements the other half of #9688
2022-01-18 16:47:05 +02:00
Eli Schwartz 7deeb293b6 wraps: on failure to apply a patch, delete the subproject directory
This code path is only called on initial download of a subproject, but
if we raise an error and abort while downloading/applying a patch
overlay we still have the upstream sources extracted... which appears to
be a subproject, then turns out to be broken due to missing meson.build

Partially implements #9688
2022-01-18 16:47:05 +02:00
Eli Schwartz 4b351aef26
first pass at migrating to dataclasses
In some cases, init variables that accept None as a sentinel and
immediately overwrite with [], are migrated to dataclass field
factories. \o/

Note: dataclasses by default cannot provide eq methods, as they then
become unhashable. In the future we may wish to opt into declaring them
frozen, instead/additionally.
2022-01-10 18:36:57 -05:00
Randy 79c3e9ca4d Fix typo in wrap.py 2021-11-30 20:12:04 +01:00
Xavier Claessens d23ae8b58e wrap: Fix concurrent os.mkdir()
Since 0.59.0 Meson downloads multiple wraps in parallel, so the
packagecache directory could be created by one then the 2nd would hit
error when calling os.mkdir() because it already exists.
2021-11-02 17:27:00 -04:00
Eli Schwartz 6ed1332603 wrap: fix support for git < 2.28
both of these do the same thing:

init -b <branchname>
-c init.defaultBranch=<branchname> init

The latter contributes to unreadably long lines of code, but has the
advantage of working on older versions of git.

Fixes #9449
2021-10-27 10:44:11 -04:00
Eli Schwartz 8947352889 fix various flake8 whitespace errors 2021-10-27 09:51:52 -04:00
Eli Schwartz f52a5a7cd3
wrap: move FeatureNew checks to a more natural place
Now, warnings are unconditionally raised when parsing the wrap file,
whether they are used or not. That being said, these warnings literally
just check for a couple of keys used in the .wrap ini file.

Moving these checks from the time of use to the time of loading, means
that we no longer report warnings only when originally downloading or
extracting the file or VCS repo.

It also means we no longer report warnings in one subproject, when a
wrap file is picked up from a different subproject because the first
subproject actually does a dependency lookup. This caused issues for the
WrapDB tooling, which uses patch_directory everywhere and the
superproject requires a suitable minimum version of meson for this...
but individual wraps might use a much lower version, and would then
raise a warning (in strict mode, converted to an error) when it resolved
a dependency from another WrapDB project.

Fixes #9118
2021-10-15 14:38:44 -04:00
Eli Schwartz b57b1050a6
wrap clone: be less noisy when doing automated code checkouts
There are two possible issues, both of which emit very long messages
from git:
- when shallow cloning via depth + a pinned commit we locally init
  rather than cloning; use an almost-certainly not conflicting dummy
  branch name
- any time a checkout of a revision is performed, it might not be a
  branch name, in which case it will be a detached HEAD. By default git
  is very noisy about this -- it wants you to know what happened and how
  not to mess up. But wraps aren't per default intended for user editing
  and development, it's just part of the internal transport which meson
  uses. So, temporarily squelch this advice. Do not permanently
  configure the repo like this though, because the user *might* cd in
  and start developing on the subproject; in this case, any additional
  git advice they trigger is their responsibility (and they probably do
  want it).

Fixes #9318
2021-10-04 22:48:09 -04:00
Eli Schwartz 800c3462f0
condense lines 2021-10-04 17:01:03 -04:00
Eli Schwartz d06cc042eb
f-strings 2021-10-04 16:29:32 -04:00
Eli Schwartz 4745037f4b log a more informative error when wrap-git subprojects cannot be downloaded
"ERROR: Git program not found" is both highly true, and somewhat
inscrutable. Sure, looking at the line number you can basically figure
out that subproject('something') must somehow need git to operate, but
that may not be immediately obvious.

Make mention of the fact that it is needed to "download foo.wrap".

Fixes #7764
2021-09-03 08:02:02 -04:00
Tristan Partin f21685a833 Delete redirected wrap files in subprojects purge
We need to store the original filename as well as whether the wrap was
redirected in order to properly purge the redirected wrap.
2021-07-03 13:14:20 +03:00
Daniel Mensinger 3e396b3782
fix: Always explicitly set encoding for text files (fixes #8263) 2021-06-29 11:28:08 +02:00
Xavier Claessens b6d754a40c interpreter: Extract dependency() logic into its own helper class
The dependency lookup is a lot of complex code. This refactor it all
into a single file/class outside of interpreter main class. This new
design allows adding more fallbacks candidates in the future (e.g. using
cc.find_library()) but does not yet add any extra API.
2021-06-16 19:04:03 -04:00
Xavier Claessens 62c53b834d wrap: Port to v2 protocol
Fixes: #8754.
2021-06-04 15:35:29 -04:00
Dan fde305e359 Add trailing -- parameter to git checkouts. 2021-04-22 16:23:48 -04:00
Seungha Yang 448b11cb7f wrap: Tell which wrap-redirect file is missing
Otherwise user cannot understand which wrap file is missing or wrong
2021-03-29 10:57:17 -04:00
Eli Schwartz 6a0fabc647
mass rewrite of string formatting to use f-strings everywhere
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04 17:16:11 -05:00
Eli Schwartz 4340bf34fa
various python neatness cleanups
All changes were created by running

"pyupgrade --py3-only --keep-percent-format"

and committing the results. I have not touched string formatting for
now.

- use set literals
- simplify .format() parameter naming
- remove __future__
- remove default "r" mode for open()
- use OSError rather than compatibility aliases
- remove stray parentheses in function(generator) scopes
2021-03-04 17:11:26 -05:00
Xavier Claessens b7bcdb7839 wrap: sha256 digest is not case sensitive
Fixes: #8450.
2021-03-02 12:38:46 -05:00
Eli Schwartz 9d602e6531
do not require git installed in order to use wraps
It doesn't make sense to check for the presence of git every time we use
it, but short-circuit any attempt to use a wrap right from the get-go
because we are trying to be fancy with submodules.

If git is not installed, simply do not try to figure out whether the
wrap is a submodule that can potentially be checked out/updated for the
user. Just take it on faith that it isn't one.

Fixes #2623
2021-01-26 08:09:53 -05:00
Jussi Pakkanen efe7270429 Do not store config parser object in Wrap object. Closes: #7920. 2021-01-17 21:04:21 +00:00
Jussi Pakkanen ca30550e06 Handle uppercase dependency names in wraps. 2020-12-30 20:54:41 +00:00
Dylan Baker f6672c7a19 use real pathlib module
We added the _pathlib module to work around defeciencies in python 3.5's
implementation, since we now rely on 3.6 lets drop this
2020-11-20 15:08:40 -08:00
Dylan Baker 2bd309543f wrap: fix type annotations 2020-11-03 21:42:18 -05:00
Xavier Claessens 42cc9294e9 wrap: Fix git exception not catched by interpreter
This is a regression introduced in Meson 0.56.0, it was fatal
error when optional dependencies fails to download their fallback
subproject.
2020-11-03 21:28:32 +02:00
Jussi Pakkanen 64f8479097 In text, betterer grammer usings. 2020-10-19 00:04:14 +03:00
Xavier Claessens 3a01823786 wrap: Add 'redirect' type and use it when auto promote 2020-10-13 17:55:16 -04:00
Xavier Claessens a20d7ad67d wrap: Use sub-subproject packagefiles 2020-10-13 17:55:16 -04:00
Xavier Claessens 6333ee88c1 Merge wraps from subprojects into wraps from main project
wraps from subprojects are now merged into the list of wraps from main
project, so they can be used to download dependencies of dependencies
instead of having to promote wraps manually. If multiple projects
provides the same wrap file, the first one to be configured wins.

This also fix usage of sub-subproject that don't have wrap files. We can
now configure B when its source tree is at
`subprojects/A/subprojects/B/`. This has the implication that we cannot
assume that subproject "foo" is at `self.subproject_dir / 'foo'` any
more.
2020-10-13 17:55:16 -04:00
Daniel Mensinger 1dfaccfd91 pathlib: Fix resolve() by overriding it in Python 3.5 2020-10-04 10:45:48 +02:00
Xavier Claessens e0cd54a322 msubprojects: Stop trying to guess subproject type
It was done to include them in `meson subprojects foreach` without
--types argument, but it's better to special case missing --types and
include wraps that have type=None too. It was a bad idea because that
was messing them in `meson subprojects update`, now they are ignored by
that command.
2020-09-18 03:01:15 +00:00
Xavier Claessens 14c1a6983d msubprojects: Allow comma separated list of types 2020-09-10 11:39:30 -04:00
Xavier Claessens 276c3fcb5a Move verbose_git() and quiet_git() to mesonlib 2020-09-10 11:39:30 -04:00
Xavier Claessens 98d445a070 msubprojects: Support git subprojects with no wrap file
User could have cloned manually a subproject.
2020-09-10 11:39:30 -04:00
Daniel Mensinger e681235e5f
typing: fix code review 2020-09-08 20:15:58 +02:00
Daniel Mensinger 449dd8e72a
typing: fully annotate wrap 2020-09-08 20:15:56 +02:00
Xavier Claessens 2353d67c25 wrap: Raise MesonException when git command fails
This avoid printing long backtrace by default, the user already has the
output of the git command printed for debugging purpose since we don't
redirect stdout/stderr.
2020-07-14 20:49:47 +03:00
Xavier Claessens e1094df77e wrap: Add back filename member in PackageDefinition
It is still used by msubprojects.py and cause issues when updating
wrapdb.
2020-07-04 13:58:36 +03:00
Xavier Claessens 41aefd2145 wrap: Abort if multiple wraps provide the same dep or program 2020-07-01 09:51:57 -04:00
Xavier Claessens 13316f99fe wrap: Refactor to split wraps dictionary into 3 separate dicts
It makes the code cleaner to have 3 separate dictionaries for
packagename, dependency and programs.
2020-07-01 09:51:57 -04:00
Xavier Claessens fba796cf13 Fix typo: coma->comma 2020-07-01 09:51:57 -04:00
Xavier Claessens f08eed37cb find_program: Fallback if a wrap file provide the program name
We don't need the legacy variable name system as for dependency()
fallbacks because meson.override_find_program() is largely used already,
so we can just rely on it.
2020-07-01 09:51:57 -04:00
Xavier Claessens 71804e56eb wrap: Add special 'dependency_names' key in [provide] section
The value for that key must be a coma separated list of dependecy names
provided by that subproject, when no variable name is needed because the
subproject uses override_dependency().
2020-07-01 09:51:57 -04:00
Xavier Claessens 2a7f72885f wrap: Add 'provide' section 2020-07-01 09:51:44 -04:00
Xavier Claessens 4a371c97f4 wrap: Apply patch even in VCS cases 2020-06-30 19:42:43 +03:00
Eli Schwartz 33167d40a2 wrap: add user agent when downloading files
This lets servers know when they're being used by meson. It also avoids
issues where the Independent JPEG Group decided to ban the
"Python-urllib" default user agent.

Fixes https://github.com/mesonbuild/libjpeg/issues/9
2020-06-26 16:07:29 -04:00
Xavier Claessens e353b2e8d4 wrap: Add patch_directory support
Copy a tree instead of extracting an archive.

Closes: #7216
2020-06-21 23:39:12 +03:00
Daniel Mensinger ccdf7f6d34
wrap: Add support for local files via only `*_filename` 2020-04-25 11:43:42 +02:00