Merge pull request #7740 from bonzini/fallback-false

Allow blocking/forcing automatic subproject search

docs/markdown/Reference-manual.md

@@ -440,33 +440,61 @@ system) with the given name with `pkg-config` and [with
 CMake](Dependencies.md#cmake) if `pkg-config` fails. Additionally,
 frameworks (OSX only) and [library-specific fallback detection
 logic](Dependencies.md#dependencies-with-custom-lookup-functionality)
-are also supported. This function supports the following keyword
+are also supported.
-arguments:
+
+Dependencies can also be resolved in two other ways:
+
+* if the same name was used in a `meson.override_dependency` prior to
+  the call to `dependency`, the overriding dependency will be returned
+  unconditionally; that is, the overriding dependency will be used
+  independent of whether an external dependency is installed in the system.
+  Typically, `meson.override_dependency` will have been used by a
+  subproject.
+
+* by a fallback subproject which, if needed, will be brought into the current
+  build specification as if [`subproject()`](#subproject) had been called.
+  The subproject can be specified with the `fallback` argument.  Alternatively,
+  if the `fallback` argument is absent, *since 0.55.0* Meson can
+  automatically identify a subproject as a fallback if a wrap file
+  [provides](Wrap-dependency-system-manual.md#provide-section) the
+  dependency, or if a subproject has the same name as the dependency.
+  In the latter case, the subproject must use `meson.override_dependency` to
+  specify the replacement, or Meson will report a hard error.  See the
+  [Wrap documentation](Wrap-dependency-system-manual.md#provide-section)
+  for more details.  This automatic search can be controlled using the
+  `allow_fallback` keyword argument.
+
+This function supports the following keyword arguments:
 
 - `default_options` *(since 0.37.0)*: an array of default option values
   that override those set in the subproject's `meson_options.txt`
   (like `default_options` in [`project()`](#project), they only have
   effect when Meson is run for the first time, and command line
   arguments override any default options in build files)
-- `fallback`: specifies a subproject fallback to use in case the
+- `allow_fallback` (boolean argument, *since 0.56.0*): specifies whether Meson
-  dependency is not found in the system. The value is an array
+  should automatically pick a fallback subproject in case the dependency
-  `['subproj_name', 'subproj_dep']` where the first value is the name
+  is not found in the system.  If `true` and the dependency is not found
+  on the system, Meson will fallback to a subproject that provides this
+  dependency. If `false`, Meson will not fallback even if a subproject
+  provides this dependency.  By default, Meson will do so if `required`
+  is `true` or  [`enabled`](Build-options.md#features); see the [Wrap
+  documentation](Wrap-dependency-system-manual.md#provide-section)
+  for more details.
+- `fallback` (string or array argument): manually specifies a subproject
+  fallback to use in case the dependency is not found in the system.
+  This is useful if the automatic search is not applicable or if you
+  want to support versions of Meson older than 0.55.0.  If the value is an
+  array `['subproj_name', 'subproj_dep']`, the first value is the name
   of the subproject and the second is the variable name in that
   subproject that contains a dependency object such as the return
   value of [`declare_dependency`](#declare_dependency) or
   [`dependency()`](#dependency), etc. Note that this means the
   fallback dependency may be a not-found dependency, in which
   case the value of the `required:` kwarg will be obeyed.
-  *(since 0.54.0)* `'subproj_dep'` argument can be omitted in the case the
+  *Since 0.54.0* the value can be a single string, the subproject name;
-  subproject used `meson.override_dependency('dependency_name', subproj_dep)`.
+  in this case the subproject must use
-  In that case, the `fallback` keyword argument can be a single string instead
+  `meson.override_dependency('dependency_name', subproj_dep)`
-  of a list of 2 strings. *Since 0.55.0* the `fallback` keyword argument can be
+  to specify the dependency object used in the superproject.
-  omitted when there is a wrap file or a directory with the same `dependency_name`,
-  and subproject registered the dependency using
-  `meson.override_dependency('dependency_name', subproj_dep)`, or when the wrap
-  file has `dependency_name` in its `[provide]` section.
-  See [Wrap documentation](Wrap-dependency-system-manual.md#provide-section)
-  for more details.
 - `language` *(since 0.42.0)*: defines what language-specific
   dependency to find if it's available for multiple languages.
 - `method`: defines the way the dependency is detected, the default is

docs/markdown/Wrap-dependency-system-manual.md

@@ -182,10 +182,12 @@ endif
   `dependency('foo-1.0', required: get_option('foo_opt'))` will only fallback
   when the user sets `foo_opt` to `enabled` instead of `auto`.
 
-If it is desired to fallback for an optional dependency, the `fallback` keyword
+If it is desired to fallback for an optional dependency, the `fallback`
-argument must be passed explicitly. For example
+or `allow_fallback` keyword arguments must be passed explicitly. *Since
-`dependency('foo-1.0', required: get_option('foo_opt'), fallback: 'foo')` will
+0.56.0*, `dependency('foo-1.0', required: get_option('foo_opt'),
-use the fallback even when `foo_opt` is set to `auto`.
+allow_fallback: true)` will use the fallback even when `foo_opt` is set
+to `auto`.  On version *0.55.0* the same effect could be achieved with
+`dependency('foo-1.0', required: get_option('foo_opt'), fallback: 'foo')`.
 
 This mechanism assumes the subproject calls `meson.override_dependency('foo-1.0', foo_dep)`
 so Meson knows which dependency object should be used as fallback. Since that

docs/markdown/snippets/fallback_bool.md

@@ -0,0 +1,8 @@
+## Controlling subproject dependencies with  `dependency(allow_fallback: ...)`
+
+As an alternative to the `fallback` keyword argument to `dependency`,
+you may use `allow_fallback`, which accepts a boolean value. If `true`
+and the dependency is not found on the system, Meson will fallback
+to a subproject that provides this dependency, even if the dependency
+is optional. If `false`, Meson will not fallback even if a subproject
+provides this dependency.

mesonbuild/dependencies/base.py

@@ -2320,9 +2320,11 @@ def get_dep_identifier(name, kwargs) -> T.Tuple:
         # 'version' is irrelevant for caching; the caller must check version matches
         # 'native' is handled above with `for_machine`
         # 'required' is irrelevant for caching; the caller handles it separately
-        # 'fallback' subprojects cannot be cached -- they must be initialized
+        # 'fallback' and 'allow_fallback' is not part of the cache because,
+        #     once a dependency has been found through a fallback, it should
+        #     be used for the rest of the Meson run.
         # 'default_options' is only used in fallback case
-        if key in ('version', 'native', 'required', 'fallback', 'default_options', 'force_fallback'):
+        if key in ('version', 'native', 'required', 'fallback', 'allow_fallback', 'default_options'):
             continue
         # All keyword arguments are strings, ints, or lists (or lists of lists)
         if isinstance(value, list):

mesonbuild/interpreter.py

@@ -2642,10 +2642,9 @@ class Interpreter(InterpreterBase):
                     FeatureNew.single_use('stdlib without variable name', '0.56.0', self.subproject)
                 kwargs = {'fallback': di,
                           'native': for_machine is MachineChoice.BUILD,
-                          'force_fallback': True,
                           }
                 name = display_name = l + '_stdlib'
-                dep = self.dependency_impl(name, display_name, kwargs)
+                dep = self.dependency_impl(name, display_name, kwargs, force_fallback=True)
                 self.build.stdlibs[for_machine][l] = dep
 
     def import_module(self, modname):
@@ -3701,31 +3700,41 @@ external dependencies (including libraries) must go to "dependencies".''')
                     build.DependencyOverride(d.held_object, node, explicit=False)
         return d
 
-    def dependency_impl(self, name, display_name, kwargs):
+    def dependency_impl(self, name, display_name, kwargs, force_fallback=False):
         disabled, required, feature = extract_required_kwarg(kwargs, self.subproject)
         if disabled:
             mlog.log('Dependency', mlog.bold(display_name), 'skipped: feature', mlog.bold(feature), 'disabled')
             return self.notfound_dependency()
 
-        has_fallback = 'fallback' in kwargs
+        fallback = kwargs.get('fallback', None)
-        if not has_fallback and name:
+        allow_fallback = kwargs.get('allow_fallback', None)
+        if allow_fallback is not None:
+            FeatureNew.single_use('"allow_fallback" keyword argument for dependency', '0.56.0', self.subproject)
+            if fallback is not None:
+                raise InvalidArguments('"fallback" and "allow_fallback" arguments are mutually exclusive')
+            if not isinstance(allow_fallback, bool):
+                raise InvalidArguments('"allow_fallback" argument must be boolean')
+
+        # If "fallback" is absent, look for an implicit fallback.
+        if name and fallback is None and allow_fallback is not False:
             # Add an implicit fallback if we have a wrap file or a directory with the same name,
             # but only if this dependency is required. It is common to first check for a pkg-config,
             # then fallback to use find_library() and only afterward check again the dependency
             # with a fallback. If the fallback has already been configured then we have to use it
             # even if the dependency is not required.
             provider = self.environment.wrap_resolver.find_dep_provider(name)
+            if not provider and allow_fallback is True:
+                raise InvalidArguments('Fallback wrap or subproject not found for dependency \'%s\'' % name)
             dirname = mesonlib.listify(provider)[0]
-            if provider and (required or self.get_subproject(dirname)):
+            if provider and (allow_fallback is True or required or self.get_subproject(dirname)):
-                kwargs['fallback'] = provider
+                fallback = provider
-                has_fallback = True
 
-        if 'default_options' in kwargs and not has_fallback:
+        if 'default_options' in kwargs and not fallback:
-            mlog.warning('The "default_options" keyworg argument does nothing without a "fallback" keyword argument.',
+            mlog.warning('The "default_options" keyword argument does nothing without a fallback subproject.',
                          location=self.current_node)
 
         # writing just "dependency('')" is an error, because it can only fail
-        if name == '' and required and not has_fallback:
+        if name == '' and required and not fallback:
             raise InvalidArguments('Dependency is both required and not-found')
 
         if '<' in name or '>' in name or '=' in name:
@@ -3734,31 +3743,31 @@ external dependencies (including libraries) must go to "dependencies".''')
 
         identifier, cached_dep = self._find_cached_dep(name, display_name, kwargs)
         if cached_dep:
-            if has_fallback:
+            if fallback:
-                dirname, varname = self.get_subproject_infos(kwargs)
+                dirname, varname = self.get_subproject_infos(fallback)
                 self.verify_fallback_consistency(dirname, varname, cached_dep)
             if required and not cached_dep.found():
                 m = 'Dependency {!r} was already checked and was not found'
                 raise DependencyException(m.format(display_name))
             return DependencyHolder(cached_dep, self.subproject)
 
-        # If the dependency has already been configured, possibly by
+        if fallback:
-        # a higher level project, try to use it first.
+            # If the dependency has already been configured, possibly by
-        if has_fallback:
+            # a higher level project, try to use it first.
-            dirname, varname = self.get_subproject_infos(kwargs)
+            dirname, varname = self.get_subproject_infos(fallback)
             if self.get_subproject(dirname):
                 return self.get_subproject_dep(name, display_name, dirname, varname, kwargs)
 
-        wrap_mode = self.coredata.get_builtin_option('wrap_mode')
+            wrap_mode = self.coredata.get_builtin_option('wrap_mode')
-        force_fallback_for = self.coredata.get_builtin_option('force_fallback_for')
+            force_fallback_for = self.coredata.get_builtin_option('force_fallback_for')
-        force_fallback = kwargs.get('force_fallback', False)
+            force_fallback = (force_fallback or
-        forcefallback = has_fallback and (wrap_mode == WrapMode.forcefallback or \
+                              wrap_mode == WrapMode.forcefallback or
-                                          name in force_fallback_for or \
+                              name in force_fallback_for or
-                                          dirname in force_fallback_for or \
+                              dirname in force_fallback_for)
-                                          force_fallback)
+
-        if name != '' and not forcefallback:
+        if name != '' and (not fallback or not force_fallback):
             self._handle_featurenew_dependencies(name)
-            kwargs['required'] = required and not has_fallback
+            kwargs['required'] = required and not fallback
             dep = dependencies.find_external_dependency(name, self.environment, kwargs)
             kwargs['required'] = required
             # Only store found-deps in the cache
@@ -3770,8 +3779,8 @@ external dependencies (including libraries) must go to "dependencies".''')
                 self.coredata.deps[for_machine].put(identifier, dep)
                 return DependencyHolder(dep, self.subproject)
 
-        if has_fallback:
+        if fallback:
-            return self.dependency_fallback(name, display_name, kwargs)
+            return self.dependency_fallback(name, display_name, fallback, kwargs)
 
         return self.notfound_dependency()
 
@@ -3798,8 +3807,8 @@ external dependencies (including libraries) must go to "dependencies".''')
             message.append(mlog.bold(command_templ.format(l[len(self.source_root) + 1:])))
         mlog.warning(*message, location=self.current_node)
 
-    def get_subproject_infos(self, kwargs):
+    def get_subproject_infos(self, fbinfo):
-        fbinfo = mesonlib.stringlistify(kwargs['fallback'])
+        fbinfo = mesonlib.stringlistify(fbinfo)
         if len(fbinfo) == 1:
             FeatureNew.single_use('Fallback without variable name', '0.53.0', self.subproject)
             return fbinfo[0], None
@@ -3807,8 +3816,8 @@ external dependencies (including libraries) must go to "dependencies".''')
             raise InterpreterException('Fallback info must have one or two items.')
         return fbinfo
 
-    def dependency_fallback(self, name, display_name, kwargs):
+    def dependency_fallback(self, name, display_name, fallback, kwargs):
-        dirname, varname = self.get_subproject_infos(kwargs)
+        dirname, varname = self.get_subproject_infos(fallback)
         required = kwargs.get('required', True)
 
         # Explicitly listed fallback preferences for specific subprojects

test cases/common/239 dependency allow_fallback/meson.build

@@ -0,0 +1,12 @@
+project('subproject fallback', 'c')
+
+foob_dep = dependency('foob', allow_fallback: true, required: false)
+assert(foob_dep.found())
+
+# Careful!  Once a submodule has been triggered and it has
+# overridden the dependency, it sticks.
+foob_dep = dependency('foob', allow_fallback: false, required: false)
+assert(foob_dep.found())
+
+foob3_dep = dependency('foob3', allow_fallback: false, required: false)
+assert(not foob3_dep.found())

test cases/common/239 dependency allow_fallback/subprojects/foob/meson.build

@@ -0,0 +1,2 @@
+project('foob', 'c')
+meson.override_dependency('foob', declare_dependency())

test cases/common/239 dependency allow_fallback/subprojects/foob3/meson.build

@@ -0,0 +1,2 @@
+project('foob3', 'c')
+# Note that there is no override_dependency here

test cases/failing/110 no fallback/meson.build

@@ -0,0 +1,2 @@
+project('no fallback', 'c')
+foob_dep = dependency('foob', allow_fallback: false, required: true)

test cases/failing/110 no fallback/subprojects/foob/meson.build

@@ -0,0 +1,2 @@
+project('foob', 'c')
+meson.override_dependency('foob', declare_dependency())

test cases/failing/110 no fallback/test.json

@@ -0,0 +1,8 @@
+{
+  "stdout": [
+    {
+      "match": "re",
+      "line": ".*/meson\\.build:2:0: ERROR: (Pkg-config binary for machine MachineChoice\\.HOST not found\\. Giving up\\.|Dependency \"foob\" not found, tried .*)"
+    }
+  ]
+}