This patch updates the mbarrier.arrive.* family of Ops to include
all features added up-to Blackwell.
* Update the `mbarrier.arrive` Op to include shared_cluster
memory space, cta/cluster scope and an option to lower using
relaxed semantics.
* An `arrive_drop` variant is added for both the `arrive` and
`arrive.nocomplete` operations.
* Updates for expect_tx and complete_tx operations.
* Verifier checks are added wherever appropriate.
* lit tests are added to verify the lowering to the intrinsics.
TODO:
* Updates for the remaining mbarrier family will be done in
subsequent PRs. (mainly, arrive.expect-tx, test_wait and try_waits)
Signed-off-by: Durgadoss R <durgadossr@nvidia.com>
Previously, only literal upper-bounded loops were recognized. This patch
relaxes this matching to accept any compile-time deducible constant
expression.
It would be better to rely on the SVals (values from the symbolic
domain), as those could potentially have more accurate answers, but this
one is much simpler.
Note that at the time we calculate this value, we have not evaluated the
sub-exprs of the condition, consequently, we can't just query the
Environment for the folded SVal.
Because of this, the next best tool in our toolbox is comp-time
evaluating the Expr.
rdar://165363923
The 16-bit immediate operand of s_waitcnt_depctr / s_wait_alu has some
unused bits. Previously codegen would set these bits to 1, but setting
them to 0 matches the SP3 assembler behaviour better, which in turn
means that we can print them using the human readable SP3 syntax:
s_wait_alu 0xfffd ; unused bits set to 1
s_wait_alu 0xff9d ; unused bits set to 0
s_wait_alu depctr_va_vcc(0) ; unused bits set to 0, human readable
Note that the set of unused bits changed between GFX10.1 and GFX10.3.
If we are given the same index in the comparator callback, simply return
false. Otherwise we will end up adding invalid items to
occludedChildren, causing extra items to get removed that should not be,
resulting in failures that manifest in different forms (assertions, asan
failures, ubsan failures, etc.).
This patch introduces SpecificFP matcher for SelectionDAG nodes.
This includes:
Adding SpecificFP_match() in SDPatternMatch.h.
Adding test coverage in SelectionDAGPatternMatchTest.cpp.
Closes#165566
After this commit, DAGCombiner will have more opportunities to perform
vector folding. This patch includes several foldings, as follows:
- VANDN(x,NOT(y)) -> AND(NOT(x),NOT(y)) -> NOT(OR(X,Y))
- VANDN(x, SplatVector(Imm)) -> AND(NOT(x), NOT(SplatVector(~Imm)))
This patch proposes to move the AVX512 CTLZ/CTTZ i256/i512 codegen to
ReplaceNodeResults to allow them to be declared as custom lowering -
this allows expansion of larger int types (e.g. i1024) to fallback to
them during their expansion.
However to declare these i256/i512 ops as custom, we need to add
MVT::i256/i512 simple types - I'm intending to add further large integer
handling in the future, some of which will use vector register
instructions, and its going to be much easier if this can be handled
with i128/i256/i512 types that match the vector register sizes.
This exposed a regression in NVPTX due to their use of EVT::isSimple()
to match their upper integer size bounds.
Move building the .mod files from openmp/flang to openmp/flang-rt using
a shared mechanism. Motivations to do so are:
1. Most modules are target-dependent and need to be re-compiled for each
target separately, which is something the LLVM_ENABLE_RUNTIMES system
already does. Prime example is `iso_c_binding.mod` which encodes the
target's ABI. Most other modules have `#ifdef`-enclosed code as well.
2. CMake has support for Fortran that we should use. Among other things,
it automatically determines module dependencies so there is no need to
hardcode them in the CMakeLists.txt.
3. It allows using Fortran itself to implement Flang-RT. Currently, only
`iso_fortran_env_impl.f90` emits object files that are needed by Fortran
applications (#89403). The workaround of #95388 could be reverted.
Some new dependencies come into play:
* openmp depends on flang-rt for building `lib_omp.mod` and
`lib_omp_kinds.mod`. Currently, if flang-rt is not found then the
modules are not built.
* check-flang depends on flang-rt: If not found, the majority of tests
are disabled. If not building in a bootstrpping build, the location of
the module files can be pointed to using
`-DFLANG_INTRINSIC_MODULES_DIR=<path>`, e.g. in a flang-standalone
build. Alternatively, the test needing any of the intrinsic modules
could be marked with `REQUIRES: flangrt-modules`.
* check-flang depends on openmp: Not a change; tests requiring
`lib_omp.mod` and `lib_omp_kinds.mod` those are already marked with
`openmp_runtime`.
As intrinsic are now specific to the target, their location is moved
from `include/flang` to `<resource-dir>/finclude/flang/<triple>`. The
mechnism to compute the location have been moved from flang-rt
(previously used to compute the location of `libflang_rt.*.a`) to common
locations in `cmake/GetToolchainDirs.cmake` and
`runtimes/CMakeLists.txt` so they can be used by both, openmp and
flang-rt. Potentially the mechnism could also be shared by other
libraries such as compiler-rt.
`finclude` was chosen because `gfortran` uses it as well and avoids
misuse such as `#include <flang/iso_c_binding.mod>`. The search location
is now determined by `ToolChain` in the driver, instead of by the
frontend. Now the driver adds `-fintrinsic-module-path` for that
location to the frontend call (Just like gfortran does).
`-fintrinsic-module-path` had to be fixed for this because ironically it
was only added to `searchDirectories`, but not
`intrinsicModuleDirectories_`. Since the driver determines the location,
tests invoking `flang -fc1` and `bbc` must also be passed the location
by llvm-lit. This works like llvm-lit does for finding the include dirs
for Clang using `-print-file-name=...`.
The xcnt wait is actually required before any memory access that can
only be done once, so atomic stores and volatile accesses are affected.
This patch also ensures buffer instructions are handled.
In the MachineSMEABIPass, if we have a function with ZT0 state, then
there are some additional cases where we need to zero ZA and ZT0.
If the function has a private ZA interface, i.e., new ZT0 (and new ZA if
present). Then ZT0/ZA must be zeroed when committing the incoming ZA
save.
If the function has a shared ZA interface, e.g. new ZA and shared ZT0.
Then ZA must be zeroed on function entry (without a ZA save commit).
The logic in the ABI pass has been reworked to use an "ENTRY" state to
handle this (rather than the more specific "CALLER_DORMANT" state).
- creates a BTI j|c landing pad MCInst.
- create getBTIHintNum utility in AArch64/Utils, to make sure BOLT
generates BTI immediates the same way as LLVM.
- add MCPlusBuilder unittests to cover new function.
The expansion of move immediate in `expandMOVImm` follows the priority
of the `MOV` alias. In addition, the selection there properly prefers
expansion based on perf optimality order. This change adds a simple
assert that `expandMOVImmSimple` expands a single optimal MOVZ/MOVK.
This patch moves the print functions from `NVVMIntrinsicUtils.h` to
`NVVMIntrinsicUtils.cpp`, a file created in the `llvm/lib/IR` directory.
Signed-off-by: Dharuni R Acharya <dharunira@nvidia.com>
This commit enhances MLIR's TBAA export with support for anonymous TBAA roots. The import for this was around for a bit but the export was missing.
Fixes: #160721
As far as I understand, the MVE fp vadd/vsub/vmul instructions will set
exception flags in the same ways as scalar fadd/fsub/fmul, but will not
honor flush-to-zero (for f32 they always flush, for f16 they follows the
fpsrc flags) and will always use the default rounding mode.
This means that we cannot convert the vadd_f23/vsub_f32/vmul_f32
intrinsics to llvm.constrained.fadd/fsub/fmul and then vadd/vsub/vmul
without changing the expected behaviour under strict-fp. This patch
introduces a set in intrinsics that we can use instead, going from
vadd_f32 -> llvm.arm.mve.vadd -> MVE_VADD.
The current implementations assumes that the standard variant of a
strictfp alternative will be a IRBuilder, this can be changed to take a
IRBuilder or IRInt.
This can be lowered to a 64bit add where we only use the bottom 32bits
of the result. It is conceptually the same as
https://alive2.llvm.org/ce/z/Xfz3Rf, but with the sext replaced by an
anyext.
This PR contains the following updates:
| Package | Type | Update | Change | Pending |
|---|---|---|---|---|
| ghcr.io/llvm/ci-ubuntu-24.04-abi-tests | container | digest |
`f80125c` -> `9138b6a` | |
|
[github/codeql-action](https://redirect.github.com/github/codeql-action)
| action | patch | `v4.31.3` -> `v4.31.4` | `v4.31.5` |
---
> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.
---
### Release Notes
<details>
<summary>github/codeql-action (github/codeql-action)</summary>
###
[`v4.31.4`](https://redirect.github.com/github/codeql-action/releases/tag/v4.31.4)
[Compare
Source](https://redirect.github.com/github/codeql-action/compare/v4.31.3...v4.31.4)
##### CodeQL Action Changelog
See the [releases
page](https://redirect.github.com/github/codeql-action/releases) for the
relevant changes to the CodeQL CLI and language packs.
##### 4.31.4 - 18 Nov 2025
No user facing changes.
See the full
[CHANGELOG.md](https://redirect.github.com/github/codeql-action/blob/v4.31.4/CHANGELOG.md)
for more information.
</details>
---
### Configuration
📅 **Schedule**: Branch creation - Between 12:00 AM and 12:59 AM, only on
Monday ( * 0 * * 1 ) (UTC), Automerge - At any time (no schedule
defined).
🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.
♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.
👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/llvm/llvm-project).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi4xNi4xIiwidXBkYXRlZEluVmVyIjoiNDIuMTkuNSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
The restrictions here aren't nearly as much about the OS as the compiler
and architecture, but the Bazel restriction was OS-based. Everything
seems to work well on even Arm64 macOS, and I would expect most BSDs and
other OSes to work well with Clang's support on x86-64.
The source code here already handles detecting when there is compiler
support for the type. And the users of this don't `select` or do
anything else to conditionally include the header, so it seems better to
not restrict access to the header from the build system, and instead
continue making the source code compatible or a no-op on relevant
configurations.
This consists of marking the various strict opcodes as legal, and
adjusting instruction selection patterns so that 'op' is 'any_op'. The
changes are similar to those in D114946 for AArch64 and #160696 for ARM.
Only Mips32/64 FPU instructions are affected.
Added lowering for for STRICT_FP_TO_UINT and STRICT_FP_TO_SINT ops.
I hit this when using a RegisterClass with a ValueTypeByHwMode that
was missing the RegInfos field. Add a test for this error.
Reviewed By: arsenm
Pull Request: https://github.com/llvm/llvm-project/pull/169439
