It doesn't make sense to check for the presence of git every time we use
it, but short-circuit any attempt to use a wrap right from the get-go
because we are trying to be fancy with submodules.
If git is not installed, simply do not try to figure out whether the
wrap is a submodule that can potentially be checked out/updated for the
user. Just take it on faith that it isn't one.
Fixes#2623
wraps from subprojects are now merged into the list of wraps from main
project, so they can be used to download dependencies of dependencies
instead of having to promote wraps manually. If multiple projects
provides the same wrap file, the first one to be configured wins.
This also fix usage of sub-subproject that don't have wrap files. We can
now configure B when its source tree is at
`subprojects/A/subprojects/B/`. This has the implication that we cannot
assume that subproject "foo" is at `self.subproject_dir / 'foo'` any
more.
It was done to include them in `meson subprojects foreach` without
--types argument, but it's better to special case missing --types and
include wraps that have type=None too. It was a bad idea because that
was messing them in `meson subprojects update`, now they are ignored by
that command.
This avoid printing long backtrace by default, the user already has the
output of the git command printed for debugging purpose since we don't
redirect stdout/stderr.
We don't need the legacy variable name system as for dependency()
fallbacks because meson.override_find_program() is largely used already,
so we can just rely on it.
The value for that key must be a coma separated list of dependecy names
provided by that subproject, when no variable name is needed because the
subproject uses override_dependency().
This lets servers know when they're being used by meson. It also avoids
issues where the Independent JPEG Group decided to ban the
"Python-urllib" default user agent.
Fixes https://github.com/mesonbuild/libjpeg/issues/9
It can happen that a server is temporaly down, tarballs often have
many mirrors available so we should be able to add at least one fallback
mirror in wrap files.
Reuse the git helper for `meson wrap` and `meson subprojects` so we
don't need to maintain the same git-colors-on-windows workarounds in
multiple places.
`git submodule update --recursive` calls git clone recursively, and on
Windows it will undo the console mode we set in mlog and cause ANSI
colors to stop working. We could set it again only when we call that,
but we will definitely miss other instances where this could happen
in the future and regress.
wrap: add imposter URL test
this test shows that meson wrap subsystem historically allows
imposter URLs like https://wrapdb.mesonwrap.com.evil/v1/foo.zip
while the new code does no.
In my opinion, we should not fall back to http:// from the SSL HSTS WrapDB URL,
**for systems that have Python SSL** as that is controverting the point
of HSTS + SSL.
For systems that do not have Python SSL, they continue to work with a
colored mlog.warning instead of only a stderr console print.
attempt to stop masquerade URLS containing wrapdb.mesonbuild.com.evil.stuff.com
507cf47507cf47 broke the combination of clone-recursive without depth.
Because it passed depth (as None) to git.
To fix this the depth option is now generated once as a list when depth is set
and else set to an empty list and unpacked into the argument this when
needed. This reduces duplication and allows code to work the same for
both cases.
Don't manually emulate a partial `git clone` when `depth` option is not
used. This keeps `git describe` working as before and generally supports
workflows that depend on tags and branches to exist in a wrap downloaded
subproject.
This also fixes downloading via git not working at all on CentOS 7
(git version 1.8.3.1).
For the `depth` case use `git clone --branch=... --depth=...` when
possible and only fall back to manual emulation wraps that specify a
full commit id, because for some reason that does not work with the
clone based workflow.
Fixes: #5991 (Regression in wrap support with git)
* If clone-recursive is specified, populate submodules only after
the specified revision/branch has been checked out. Fixes#4996
* Update submodules recursively on `meson subprojects update`
If the directory exists we early return or raise exception in resolve()
method. It was already like that even before the recent refactoring of
the code.
It is safer like that anyway, we don't want a project reconfigure to
silently pull new code. Updating subprojects should be an explicit
action of the user. For example gst-build has a 'git-update' script that
does that. In the future we could add a 'meson subprojects update'
command.
It is sometimes important to be able to build projects offline, in that
case subproject tarballs and patches could be shipped directly within
the project's repository.
- Use CongifParser instead of parsing ourself, this will give more
flexibility in the future.
- Always read the .wrap file first, because if it contains the
'directory' key we should use that value instead of packagename for
the path where do download/lookup for the subproject.
- If we download the subproject (git/submodule/tarball/etc) we should
still check it contains a meson.build file.
Otherwise, git fiddles with the cmd.exe console's color behaviour and
disables it, which causes it to stop interpreting ANSI color codes
outputted by meson for colored output.
Effectively, as soon as any subproject is initialized, cmd.exe goes
nice coloured output to ugly raw ANSI codes.
The '+' and 'U' cases should not happen normally because we don't run
any git commands if the subproject directory exists and contains
a meson build file. However, if the user accidentally messed up the
subproject checkout to a version that had no build files, we would
error out with an assertion.
Using wrap mechanism in enterprise environment,
Some package is very large, example, sdk package from
BSP vendor.
so:
- open file in the output directory with a temporary name
- download a chunk, update hash calculation, write chunk to file
- when finished close file and check the hash
- if hash is incorrect, delete temp file and raise error
- if hash is correct, atomically rename temp file to final file
fix issue: #2358
fix shutil.unpack_archive() failure when the
destination files already exists and is read-only.
Example: all files in opensource package live555
is `-r--r--r--`, if some file in patch archive try to
patch original files of live555, shutil.unpack_archive
will raise `[Errno 13] Permission denied`
This also adds a "# noqa: F401" comment on an unused "import lzma",
which we are using it in a try/except block that is being used to
check if the lzma module is importable; of course it is unused.
v2: This turned out to be a little tricky.
mesonbuild/modules/__init__.py had the "unused" import:
from ..interpreterbase import permittedKwargs, noKwargs
However, that meant that the various modules could do things like:
from . import noKwargs # "." is "mesonbuild.modules"
Which breaks when you remove __init__.py's "unused" import. I
could have tagged that import with "# noqa: F401", but instead I
chose to have each of the module import directly from
"..interpreterbase" instead of ".".
Using Meson from outside a git repo results in an error when trying to
resolve submodule subprojects. Running git from inside subproject root
should be enough to fix it.
Partially fixes#1679
Signed-off-by: Ernestas Kulik <ernestas.kulik@gmail.com>
After an initial checkout, submodules aren’t initialized and thus trying
to update them fails.
Partially fixes#1679
Signed-off-by: Ernestas Kulik <ernestas.kulik@gmail.com>
Special wrap modes:
nofallback: Don't download wraps for dependency() fallbacks
nodownload: Don't download wraps for all subproject() calls
Subprojects are used for two purposes:
1. To download and build dependencies by using .wrap files if they
are not provided by the system. This is usually expressed via
dependency(..., fallback: ...).
2. To download and build 'copylibs' which are meant to be used by
copying into your project. This is always done with an explicit
subproject() call.
--wrap-mode=nofallback will never do (1)
--wrap-mode=nodownload will do neither (1) nor (2)
If you are building from a release tarball, you should be able to
safely use 'nodownload' since upstream is expected to ship all
required sources with the tarball.
If you are building from a git repository, you will want to use
'nofallback' so that any 'copylib' wraps will be download as
subprojects.
Note that these options do not affect subprojects that are git
submodules since those are only usable in git repositories, and you
almost always want to download them.
This will benefit projects such as GNOME Recipes that prefer using
submodules over wraps because it's easier to maintain since git is
aware of it, and because it integrates with their existing
workflow. Without this, these projects have to manually initialize
the submodules which is completely unnecessary.
Closes https://github.com/mesonbuild/meson/issues/1449
Also use a pathlib.Path object for the checks since it does I/O only
once and is much more efficient anyway. Path objects are available
since Python 3.4, so this is fine.
Xavier Claessens