Commit Graph

61 Commits

Author SHA1 Message Date
Eli Schwartz 5e6122b2a5
mdist: correctly detect dirty hg repos with non-English locale, redux
https://www.gnu.org/software/gettext/manual/html_node/The-LANGUAGE-variable.html

GNU Gettext defines a feature, whereby for translation purposes, if
LC_ALL / LANG are *not* set to C, but rather define an active
translation, the LANGUAGE variable can be used to specify fallback
languages in a colon-separated list wherein the first option is the
primary language and the rest are fallbacks.

CPython, instead, checks the LANGUAGE variable first, and the first
variable that has a non-null value is treated as the canonical language
specification, splitted, and iterated over. LC_ALL=C is therefore
totally ignored, which is a major problem, and the variables aren't
checked for consistency, which is a less major problem.

GNU libc documents the same behavior CPython does -- which is broken as
it makes LC_ALL=C useless.

POSIX issue 8 standardizes on option 3: do like GNU Gettext, except do
not require the primary language in $LANGUAGE to be consistent with LANG
/ LC_ALL.

Thus, we sanitize the environment even harder. What an absolute
disaster. Even if this was fixed tomorrow we would need to maintain this
hack until 2030.

Bug: https://bugs.gentoo.org/936670
2024-09-20 00:49:00 -04:00
Eli Schwartz 81c5088568
mdist: detect pathological git repo cases and show a slightly better error
When `meson dist` fails with the error:

```
Dist currently only works with Git or Mercurial repos
```

It is sometimes inaccurate, since a git repo may exist but be
nonfunctional. Offer some better guidance in that case.

Fixes: #10866
2024-09-17 17:18:25 -04:00
Dylan Baker 49cc4862e7 mdist: don't mutate meson_command, which is supposed to be immutable 2024-08-23 22:46:16 +03:00
Eli Schwartz a05b790d66 mdist: correctly detect dirty hg repos with non-English locale
The command we use to heuristically parse whether it is dirty by
interpreting prose descriptions of the repository state, is vulnerable
to changes in locale resulting in failing to match the English word that
means it is clean.

Unfortunately, I am no mercurial expert so I am unaware if mercurial
supports scripting, like git does. Perhaps the technology simply does
not exist. A quick attempt at searching for the answer turned nothing
up. It appears that #4278 had good cause indeed for using this prose
parsing command.

So, we simply sanitize the environment due to lack of any better idea.

Bug: https://bugs.gentoo.org/936670
2024-07-30 12:28:43 -07:00
Eli Schwartz e9037e7b9f
mdist: don't fail on readonly source trees
In commit c9aa4aff66 we added a refresh
call to git to catch cases where checking for uncommitted changes would
misfire. Unfortunately, that refresh performs a write operation, which
in turn misfires on readonly media. We don't actually care about the
return value of the refresh, since its purpose is solely to make the
next command more accurate -- so ignore it.

Fixes: c9aa4aff66
Fixes: #13461
2024-07-22 13:30:57 -04:00
Jussi Pakkanen 0d7bb776e2 Move OptionKey in the option source file. 2024-07-11 11:53:39 +03:00
Benjamin Gilbert c9aa4aff66 mdist: gracefully handle stale Git index
Running `touch` on a tracked file in Git, to update its timestamp, and
then running `meson dist` would cause dist to fail:

    ERROR: Repository has uncommitted changes that will not be included in the dist tarball
    Use --allow-dirty to ignore the warning and proceed anyway

Unlike `git status` and `git diff`, `git diff-index` doesn't refresh the
index before comparing, so stat changes are assumed to imply content
changes.  Run `git update-index -q --refresh` first to refresh the index.

Fixes: #12985
2024-04-27 21:58:41 -04:00
Tristan Partin 2d3954efac Add bztar support to meson dist
Some projects, like Postgres, distribute code in this format.
2024-04-15 16:17:44 -04:00
Jussi Pakkanen 54996132af
Merge pull request #12804 from joukewitteveen/dist-rewriter
Support `meson dist` when getting project versions from VCS
2024-03-02 00:59:32 +02:00
Dylan Baker 95b3b9f710 coredata: use a Protocol for shared options between setup configure and dist
These are all passed around interchangably inside Meson, so use a shared
protocol for them.
2024-02-23 08:47:01 -08:00
Jouke Witteveen 524a66116b Add MESONREWRITE to `meson dist` scripts
Fixes #688
2024-02-12 22:19:59 +01:00
Jouke Witteveen db51dcfa47 mdist: Hoist leftover late import
From 23d3b98, the meson command is exposed through a function.
Therefore, the caveat no longer applies.
2024-02-03 16:40:00 +01:00
Dylan Baker e991c4d454 Use SPDX-License-Identifier consistently
This replaces all of the Apache blurbs at the start of each file with an
`# SPDX-License-Identifier: Apache-2.0` string. It also fixes existing
uses to be consistent in capitalization, and to be placed above any
copyright notices.

This removes nearly 3000 lines of boilerplate from the project (only
python files), which no developer cares to look at.

SPDX is in common use, particularly in the Linux kernel, and is the
recommended format for Meson's own `project(license: )` field
2023-12-13 15:19:21 -05:00
Luke Elliott ce691f8c98 Add comments suggesting to keep shell completion scripts up-to-date near cmd line argument code 2023-11-01 00:06:19 +02:00
Eli Schwartz b411be5edc
fully type mdist.py 2023-06-25 10:08:15 -04:00
Eli Schwartz b05764b73e
mdist: consolidate facts about the current dist using a dataclass
And avoid passing variables around several functions just to finally get
where they need to be. These function signatures were kind of ugly...

Also the use of dataclasses makes a big chunk of this file now typed
properly.
2023-06-25 10:08:15 -04:00
Eli Schwartz a146ee6946
mdist: refactor lots of code into VCS-specific classes
Most of the dist handling is either git-specific or hg-specific.
Tangling it all together makes it much harder to analyze what is going
on.
2023-06-25 10:08:14 -04:00
Xavier Claessens b30cd5d2d5 Make --vsenv a readonly builtin option
We need to remember its value when reconfiguring, but the Build object
is not reused, only coredata is.

This also makes CLI more consistent by allowing `-Dvsenv=true` syntax.

Fixes: #11309
2023-03-29 09:33:41 -04:00
Eli Schwartz 3e7c08f358
fix one more missing explicit "setup" subcommand 2023-03-20 17:22:51 -04:00
Eli Schwartz 680b5ff819
treewide: add future annotations import 2023-02-01 17:01:30 -05:00
Dylan Baker a5d547e0d9
pylint: enable useless-return 2022-11-29 23:26:05 -05:00
Eli Schwartz 1eaf49c880 flake8: remove no longer used imports 2022-06-10 09:15:48 -04:00
Eli Schwartz 1420d0dace
mdist: use better approach to finding original configured options
Instead of reading intro-buildoptions.json, a giant json file containing
every option ever + its current value, use the private file that is
internally used by msetup for e.g. --wipe to restore settings.

This accurately tracks exactly the options specified on the command
line, and avoids lengthy summary messages containing all the overridden
defaults.

It also avoids passing potentially incompatible options, such as
explictly specifying -Dpython.install_env while also having a non-empty
-Dpython.{x}libdir

Fixes #10181
2022-04-11 14:10:59 -04:00
andy5995 b1e6cc5553 implement 'dist --allow-dirty' flag
closes #9824
2022-02-10 21:23:20 -05:00
Xavier Claessens 928078982c Add --vsenv command line option and active VS only when needed 2021-10-10 23:15:18 +03:00
Eli Schwartz 604088b541 mdist: use git archive with git repositories for correctness
Fixes various inconsistencies:
- gitattributes is respected
  - export-subst
  - export-ignore
- submodules with relative paths are not checked out relative to the
  local clone (which does not work anyway)
- no need to manually remove gitfiles with inaccurate heuristics

Fixes #2287
Fixes #3081
Fixes #8144
2021-09-03 14:14:15 -04:00
Dylan Baker 06fdb29daa pylint: turn on superfluous parens warning
Which is really useful for catching parens used with keywords like
assert. Don't use parens with assert, it's bad.
2021-08-31 16:28:54 -04:00
Paolo Bonzini 5215f4fa98 cleanup self.options.wd
It is never None and always an absolute path
2021-07-07 18:00:37 +02:00
Paolo Bonzini b44a51d0fd resolve symlinks passed to -C
"meson setup" is resolving symlinks for the build directory in
validate_core_dirs.  For consistency with it, do the same when
the build directory is passed via -C to devenv, dist, init, install
and test.

This ensures for example that the path to test dependencies is
computed correctly in "meson test".

Fixes: #8765
2021-07-07 18:00:37 +02:00
Andrea Pappacoda d0101cbfb7 dist: only exclude actual git files
Instead of excluding all files starting with .git, meson dist now only excludes files really used by git
2021-07-04 01:25:23 +03:00
Daniel Mensinger 3e396b3782
fix: Always explicitly set encoding for text files (fixes #8263) 2021-06-29 11:28:08 +02:00
Naveen M K 7a6ad2953a Don't use `os.path.relpath` in dist command
This is problematic when we meson is installed in the different 
root(say C:) while building from another root(say D:).
This is how it is done in mesonpep517 and causes problems
because of that.
2021-05-30 00:49:43 +03:00
Tristan Partin cc54284885
Fix issue where multiple dist files were not being produced for comma separated formats value 2021-04-26 17:13:13 +01:00
Xavier Claessens 3990754bf5 Allow add_dist_script() in subprojects
Fixes: #8440.
2021-03-18 08:41:21 -04:00
Eli Schwartz 6a0fabc647
mass rewrite of string formatting to use f-strings everywhere
performed by running "pyupgrade --py36-plus" and committing the results
2021-03-04 17:16:11 -05:00
Xavier Claessens 0626465ea8 Fix executable as script on Windows
On Windows this would fail because of missing DLL:
```
mylib = library(...)
exe = executable(..., link_with: mylib)
meson.add_install_script(exe)
```

The reason is on Windows we cannot rely on rpath to find libraries from
build directory, they are searched in $PATH. We already have all that
mechanism in place for custom_target() using ExecutableSerialisation
class, so reuse it for install/dist/postconf scripts too.

This has bonus side effect to also use exe_wrapper for those scripts.

Fixes: #8187
2021-01-30 09:51:06 +00:00
Xavier Claessens e6ab364a69 dist: Use windows_proof_rmtree() instead of shutil.rmtree() 2021-01-26 21:32:03 -05:00
Xavier Claessens baa9eeebe4 dist: Allow packaging subproject in same git repo as main project 2021-01-26 21:32:03 -05:00
Dylan Baker 23d3b98fc1 split mesonlib into a package
Currently mesonlib does some import tricks to figure out whether it
needs to use windows or posix specific functions. This is a little
hacky, but works fine. However, the way the typing stubs are implemented
for the msvcrt and fnctl modules will cause mypy to fail on the other
platform, since the functions are not implemented.

To aleviate this (and for slightly cleaner design), I've split mesonlib
into a pacakge with three modules. A universal module contains all of
the platform agnositc code, a win32 module contains window specific
code, a posix module contains the posix specific code, and a platform
module contains no-op implementations. Then the package's __init__ file
imports all of the universal functions and all of the functions from the
approriate platform module, or the no-op versions as fallbacks. This
makes mypy happy, and avoids `if`ing all over the code to switch between
the platform specific code.
2021-01-23 12:48:29 +01:00
Aman Verma cb10ba75d4 mdist: Print file path at the end of run() instead of in create_hash(). 2020-12-22 21:38:28 -05:00
Aman Verma 1f7c8ec7e2 mdist: Ensure correct format for sha256sum.
We conform to the format described at
<https://www.gnu.org/software/coreutils/manual/html_node/md5sum-invocation.html>.
and compatible with busybox and Perl's shasum utility.
2020-12-22 21:38:28 -05:00
Dylan Baker f6672c7a19 use real pathlib module
We added the _pathlib module to work around defeciencies in python 3.5's
implementation, since we now rely on 3.6 lets drop this
2020-11-20 15:08:40 -08:00
Daniel Mensinger 1dfaccfd91 pathlib: Fix resolve() by overriding it in Python 3.5 2020-10-04 10:45:48 +02:00
Dylan Baker 87248d9020 mdist: actually constrain the choices of the --formats option
Since we already have that list available.
2020-09-22 16:58:08 -07:00
Paolo Bonzini 492afe50a4 environment: use ExternalProgram to find ninja
This allows the NINJA environment variable to support all the Windows special
cases, especially allowing an absolute path without extension.

Based on a patch by Yonggang Luo.

Fixes: #7659
Suggested-by: Nirbheek Chauhan <nirbheek@centricular.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-09-04 15:38:12 +02:00
Xavier Claessens 13316f99fe wrap: Refactor to split wraps dictionary into 3 separate dicts
It makes the code cleaner to have 3 separate dictionaries for
packagename, dependency and programs.
2020-07-01 09:51:57 -04:00
Nirbheek Chauhan ec1bd22b15 mdist: Filter out buildtype to avoid warning
Since we parse buildoptions.json to pass options, we end up passing
-Dbuildtype and also -Doptimization and -Ddebug which triggers the
warning:

WARNING: Recommend using either -Dbuildtype or -Doptimization + -Ddebug [...]

Filter out buildtype. It is redundant.
2020-06-13 11:02:19 +00:00
Xavier Claessens f4d98a3e72 dist: Print location of generated tarballs
Otherwise the location is not discoverable without reading online
documentation.
2020-03-29 21:12:34 +03:00
Xavier Claessens 8603c3b445 dist: Add --no-tests option
It is not always needed to test generated tarballs, for example when
they are generated by CI that already does its own testing.
2020-03-29 21:12:34 +03:00
Michael Brockus 98ddd52ced
Cherry-picking - (rm python2 % add python3 .format) (#6725) 2020-03-03 21:45:43 +02:00