Commit Graph

14694 Commits

Author SHA1 Message Date
Erik Desjardins
87d02e0dfd Recommit "[Support] change StringMap hash function from djbHash to xxHash"
This reverts commit 37eb9d13f8.

Test failures have been fixed:

- ubsan failure fixed by 72eac42f21
- warn-unsafe-buffer-usage-fixits-local-var-span.cpp fixed by
  03cc52dfd1 (wasn't related)
- test-output-format.ll failure was spurious, build failed at
  https://lab.llvm.org/buildbot/#/builders/54/builds/3545 (b4431b2d94)
  but passed at
  https://lab.llvm.org/buildbot/#/builders/54/builds/3546 (5ae99be037)
  which is before my revert
  b4431b2d94...5ae99be037

Original commit message:

    Depends on https://reviews.llvm.org/D142861.

    Alternative to https://reviews.llvm.org/D137601.

    xxHash is much faster than djbHash. This makes a simple Rust test case with a large constant string 10% faster to compile.

    Previous attempts at changing this hash function (e.g. https://reviews.llvm.org/D97396) had to be reverted due to breaking tests that depended on iteration order.
    No additional tests fail with this patch compared to `main` when running `check-all` with `-DLLVM_ENABLE_PROJECTS="all"` (on a Linux host), so I hope I found everything that needs to be changed.

    Differential Revision: https://reviews.llvm.org/D142862
2023-02-19 16:52:26 -05:00
Kazu Hirata
cbde2124f1 Use APInt::popcount instead of APInt::countPopulation (NFC)
This is for consistency with the C++20-style bit manipulation
functions in <bit>.
2023-02-19 11:29:12 -08:00
Fabian
09fd9ef4f4 [mlir] Execute all requested translations in MlirTranslateMain
Currently, MlirTranslateMain only executes one of the requested translations, and does not error if multiple are specified. This commit enables translations to be chained in the specified order.

This makes round-trip tests easier, since existing import/export passes can be reused and no combined round-trip passes have to be registered (example: mlir-translate  -serialize-spirv -deserialize-spirv).

Additionally, by leveraging TranslateRegistration with file-to-file TranslateFunctions, generic pre- and post-processing can be added before/after conversion to/from MLIR.

Reviewed By: lattner, Mogball

Differential Revision: https://reviews.llvm.org/D143719
2023-02-18 21:31:59 +01:00
Hanhan Wang
5fa9933c13 [mlir][tensor] Do not use affine ops on non-perfect-tiling unpack cases.
Do not create an Affine ops for expanded size because the affine op is
too complicated which would hit an assertion in affine ops
simplification.

Reviewed By: mravishankar

Differential Revision: https://reviews.llvm.org/D144151
2023-02-17 11:08:03 -08:00
Jeff Niu
f25cfd339a [mlir] Reintroduce API for creating operations with a DictionaryAttr
This patch reintroduces an API to create operations with a pre-existing
DictionaryAttr. This API does not populate the attributes with any
default attributes the operation may have, like the API that takes a
NamedAttrList does. NamedAttrList is effective at not re-hashing the
attributes if no default attributes were added, but this new API speeds
up clone-heavy workloads slightly (~5%).

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D144204
2023-02-17 10:07:53 -08:00
Thomas Raoux
ddeb55ab33 [mlir] add option to multi-buffering
Allow user to apply multi-buffering transformation for cases where proving
that there is no loop carried dependency is not trivial. In this case user needs
to ensure that the data are written and read in the same iteration otherwise the
result is incorrect.

Differential Revision: https://reviews.llvm.org/D144227
2023-02-17 15:34:51 +00:00
Christian Ulmann
87a0479538 [mlir][llvm] Fuse access_group & loop export (NFC)
This commit moves the access group translation into the
LoopAnnotationTranslation class as these two metadata kinds only appear
together.

Drops the access group cleanup from `ModuleTranslation::forgetMapping`
as this is only used on function regions. Access groups only appear in the
region of a global metadata operation and will thus not be cleaned here.

Analogous to https://reviews.llvm.org/D143577

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144253
2023-02-17 15:31:21 +01:00
Marius Brehler
83e9ef7e51 [mlir][standalone] Extend sample with custom type
This extends the standalone example to illustrate how to structure the
files needed to create own types.

Reviewed By: jpienaar

Differential Revision: https://reviews.llvm.org/D143715
2023-02-17 15:14:56 +01:00
Christian Ulmann
e133cb9c6d [mlir][llvm] Add DINamespace attribute
This commit introduces the DINamespaceAttr to model LLVM's DINamespace metadata.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144243
2023-02-17 13:09:29 +01:00
Christian Ulmann
276358fd3a [mlir][llvm] Make DI param optional to match LLVM
This commit makes DIDerivedTypeAttr's baseType parameter optional to ensure imported IR
doesn't break the printer.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144247
2023-02-17 13:05:09 +01:00
Alexander Belyaev
cdf8f06469 [mlir] Add a folder for lbs, ubs, steps of scf.forall.
Differential Revision: https://reviews.llvm.org/D144245
2023-02-17 11:01:13 +01:00
Christian Ulmann
bf91cd6ea9 [mlir][llvm] Verify LLVM module before import
This commit ensures that the importing of LLVM modules first verifies
that the module is even valid. As many tests did not work with valid
LLVM IR, they were fixed as part of this commit.

Some error messages were only reachable with invalid input IR, thus they
were replaced with a failures.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144186
2023-02-17 10:25:23 +01:00
Alexander Belyaev
eb2f946e78 [mlir][scf] Rename ForeachThreadOp->ForallOp, PerformConcurrentlyOp->InParallelOp.
Differential Revision: https://reviews.llvm.org/D144242
2023-02-17 09:59:39 +01:00
Tobias Gysi
6f4af64b74 [mlir][llvm] Add atomic support to the LoadOp.
This revision adds atomic support to the LoadOp. It chooses
to print the atomic keywords together with the syncscope and
ordering arguments, which simplifies parsing and printing compared
to the LLVM IR printer that puts the atomic keyword at the beginning.
It uses the ordering attribute to check if the load is atomic.

The revision also implements verifiers to ensure the constraints
that apply to atomic load operations are checked.

Reviewed By: Dinistro

Differential Revision: https://reviews.llvm.org/D144112
2023-02-17 09:29:40 +01:00
Johannes de Fine Licht
a048d7394b [MLIR][LLVM] Only disallow inlining for selected function attributes.
This loosens the requirement of no passthrough function attribute being
present to checking for specific attributes that prevent inlining. Since
these attributes are no longer strictly passthrough, they should
eventually be upgraded to some form of addressable attributes.

Drops the expensive StringSwitches over call and function attributes in
favor of selectively disallowing attributes that prevent inlining
(similiar to the LLVM inliner), thereby moving to a less conservative
approach.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144104
2023-02-17 09:18:17 +01:00
Alexander Belyaev
310deca248 [mlir] Add loop bounds to scf.foreach_thread.
https://discourse.llvm.org/t/rfc-parallel-loops-on-tensors-in-mlir/68332

Differential Revision: https://reviews.llvm.org/D144072
2023-02-17 08:57:52 +01:00
Thomas Raoux
0eabb884ab [mlir][gpu] NFC let user pick the threadID values when distributing foreach_thread
Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D144219
2023-02-17 03:25:15 +00:00
Thomas Raoux
e3a88a41af Revert "[mlir][vector] Prevent duplicating operations during vector distribute"
This reverts commit 2fc3c5c34c.
2023-02-17 03:07:16 +00:00
Peiming Liu
e6758c81f8 [mlir][sparse] comment out test cases in sparse_conversion with similiar behavior.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144236
2023-02-17 01:15:35 +00:00
Peiming Liu
0af5f84a13 [mlir][sparse] split reshape.mlir into expand/collapse_shape.mlir.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144231
2023-02-17 00:07:41 +00:00
Kiran Chandramohan
22cdeb54a1 [MLIR][OpenMP] Add Conversion for Atomic Update Op
Reviewed By: TIFitis

Differential Revision: https://reviews.llvm.org/D143964
2023-02-16 23:57:35 +00:00
Peiming Liu
9e8d9316ce [mlir][sparse] allow foreach operation to generate out-of-order loop on non-annotated tensor.
No need for a temp COO and sort even when converting dense -> CSC, we can instead rotate the loop to yield a ordered coordinates at beginning.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144213
2023-02-16 23:23:20 +00:00
bixia1
c2e248c6ae [mlir][sparse] Remove the expansion of symmetric MTX in the sparse tensor storage.
We will support symmetric MTX without expanding the data in the sparse tensor
storage.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144059
2023-02-16 13:02:17 -08:00
Hanhan Wang
c6c41c31f1 [mlir][linalg][tensor] Delete duplicate tests and fix typo in filename
These two files contain the identical tests. The commit deletes one of
the files, and fix a typo in filename for the other one.

Reviewed By: chelini

Differential Revision: https://reviews.llvm.org/D144152
2023-02-16 11:05:45 -08:00
Nicolas Vasilache
ec640382fc [mlir][MemRef] NFC - Add debug information to MultiBuffer.cpp 2023-02-16 07:19:58 -08:00
Matthias Springer
e44f405bb4 [mlir][bufferization] Fix bug in findValueInReverseUseDefChain
`alwaysIncludeLeaves` was not respected by all code paths.

Differential Revision: https://reviews.llvm.org/D144187
2023-02-16 16:12:31 +01:00
Maya Amrami
ace6072bca [mlir] PromoteBuffersToStackPass - Copy attributes of original AllocOp
Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D143185
2023-02-16 17:06:45 +02:00
Christian Ulmann
28542e99bb [mlir][llvm] Use before def debug intrinsic import
This commit adds special handling for the debug intrinsic value
handling. LLVM allows to relax the def before use property for debug
intrinsics, so this property cannot be assumed for metadata values.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144177
2023-02-16 15:49:23 +01:00
Nicolas Vasilache
96179dff46 [mlir][Linalg] Add a transform dialect op to rewrite ops to destination passing style.
A new transform dialect op is introduced to perform the rewrite.
The test pass option is now obsolete and is removed in favor of the transform.

In the process I realized the tensor.pad nofold attribute was not taken into account
and added support to emit a bufferization.alloc_tensor + linalg.copy.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D143943
2023-02-16 05:26:33 -08:00
Johannes Reifferscheid
b58daf91d6 Add a lowering for memref.dealloc with unranked memrefs.
This is permitted by the op, but the current lowering generates invalid IR.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D144090
2023-02-16 14:19:16 +01:00
Christian Ulmann
873ea45115 [mlir][llvm] Rename void debug type to null
This commit renames the "di_void_result_type" to "di_null_type" as LLVM
does use null not exclusively for void types. An added test demonstrates
this for variadic function declarations, whose DISubroutine indicates the
start of variadic types with `null`.

Reviewed By: gysit

Differential Revision: https://reviews.llvm.org/D144109
2023-02-16 12:13:24 +01:00
Quentin Colombet
4bc2357c3d [mlir][MemRef|Tensor] Fix the handling of DimOp
Although specifying an index that is out of bounds for both `memref.dim`
and `tensor.dim` produces an undefined behavior, this is still valid IR.
In particular, we could expose an out of bound index because of some
optimizations, for instance as demonstrated with
https://github.com/llvm/llvm-project/issues/60295, and this shouldn't
cause the compiler to abort.

This patch removes the overzealous verifier checks and properly handles
out of bound indices (as in it doesn't crash the compiler, but still
produces UB).

This fixes https://github.com/llvm/llvm-project/issues/60295.

Note: That `shape.dim` has a similar problem but we're not supposed to
produce UB in this case. Instead we're supposed to propagate an error in
the resulting value and I don't know how to do that at the moment. Hence I
left this part out of the patch.

Differential Revision: https://reviews.llvm.org/D143999
2023-02-16 11:38:19 +01:00
Tobias Gysi
4469ec1d19 [mlir][llvm] Import alias scope metadata from LLVM IR.
The revision adds support for importing alias.scope and noalias metadata
from LLVM IR into LLVM dialect. It also adds a verifier to the
AliasScopeMetadataOp to check that the associated domain exists
and is of type AliasScopeDomainMetadataOp.

Reviewed By: Dinistro

Differential Revision: https://reviews.llvm.org/D143923
2023-02-16 09:52:59 +01:00
jacquesguan
86c8ea9ac9 [mlir] Add check for value that might be null.
Because we are generating uninitialized value for no integer type and use `isUninitialized()` to judge if it is valid after https://reviews.llvm.org/rG93f081c896536112e1ca8133991d23cb1134793a, we should check the value before use `getValue` to get it.
Fixes https://github.com/llvm/llvm-project/issues/59984.

Reviewed By: Mogball

Differential Revision: https://reviews.llvm.org/D141661
2023-02-16 16:02:35 +08:00
Lei Zhang
a1aad28d29 [mlir][vector] NFC: Improve vector type accessor methods
Plain `getVectorType()` can be quite confusing and error-prone
given that, well, vector ops always work on vector types, and
it can commonly involve both source and result vectors. So this
commit makes various such accessor methods to be explicit w.r.t.
source or result vectors.

Reviewed By: ThomasRaoux

Differential Revision: https://reviews.llvm.org/D144159
2023-02-16 04:08:33 +00:00
Kazu Hirata
5382d28815 [mlir] Use std::optional instead of llvm::Optional (NFC)
This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-02-15 19:40:10 -08:00
Peiming Liu
f30ca850ef [mlir][sparse] reduce the input size in stress_test.py to make it finish in time.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144150
2023-02-16 01:18:03 +00:00
Peiming Liu
c738b430c4 [mlir][sparse] introduce operations to query sparse tensor slice offset/strides at the given dimenion
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D141442
2023-02-16 00:31:11 +00:00
Peiming Liu
e2e83f4c8f [mlir][sparse] support coiteration over sparse tensor slices
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140736
2023-02-15 23:52:22 +00:00
Peiming Liu
7d62bc9cd3 [mlir][sparse] disable sparse convolution test cases.
These test cases will be re-enabled after new algorithm is implemented.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144141
2023-02-15 23:21:35 +00:00
Peiming Liu
9ed587ea9b [mlir][sparse] split sparse concatenate test cases smaller ones.
To allow a better parallelization.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144139
2023-02-15 22:54:18 +00:00
wren romano
ae7942e296 [mlir][sparse] adding SparseTensorType::get{Pointer,Index}Type methods
Depends On D143800

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143946
2023-02-15 14:37:55 -08:00
wren romano
d950bdc73e [mlir][sparse] misc code cleanup
* Flattening/simplifying some nested conditionals
* const-ifying some local variables

Depends On D143800

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143949
2023-02-15 13:29:00 -08:00
wren romano
bb4fc6b6d6 [mlir][sparse] Adding SparseTensorType::{operator==, hasSameDimToLvlMap}
Depends On D143800

Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D144052
2023-02-15 12:05:29 -08:00
Hanhan Wang
be75cf931f [mlir][tensor] Fix a bug in tiling unpack op.
The inner tiling sizes could be dynamic (which are Values). In this
context, they should be added to tiledOperand when cloning the op.

Reviewed By: chelini

Differential Revision: https://reviews.llvm.org/D143978
2023-02-15 10:25:50 -08:00
Markus Böck
9048ea28da Reland "[mlir] Make the vast majority of intgration and runner tests work on Windows"
This reverts commit 5561e17411

The logic was moved from cmake into lit fixing the issue that lead to the revert and potentially others with multi-config cmake generators

Differential Revision: https://reviews.llvm.org/D143925
2023-02-15 19:14:43 +01:00
Lei Zhang
78e172fc92 [mlir][spirv] Support i32 memref.atomic_rmw conversion
These cases can be directly mapped to spirv.AtomicI* ops.

Reviewed By: kuhar

Differential Revision: https://reviews.llvm.org/D143952
2023-02-15 17:53:56 +00:00
Thomas Raoux
3cf7f22498 [mlir][vectorToGPU] Fix type used when folding transpose into read op
Pick the right result type when folding transpose op into a read

Differential Revision: https://reviews.llvm.org/D144113
2023-02-15 17:34:09 +00:00
Rahul Kayaith
82ae83a848 [mlir] Silence a few -Wunused-but-set-parameter warnings
Some older versions of gcc hit false positives here:

/workdir/llvm-project/mlir/include/mlir/IR/Attributes.h:391:49: warning: parameter 'ty' set but not used [-Wunused-but-set-parameter]
  391 |   static inline bool isPossible(mlir::Attribute ty) {

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81676
2023-02-15 11:57:31 -05:00
Krzysztof Drewniak
22f0c7a451 [mlir][AMDGPU] 8-bit float usage in the AMDGPU dialect
Upcoming AMD hardware will include functions that accept 8-bit floats.
Specifically, there are MFMA instructions that accept 8-bit floats,
either using the same or mixed formats. This patch adds MLIR wrappers
for these intrinsics and explicitly adds support for 8-bit floats in
the gpu-to-rocdl conversion by way of amdgpu-to-rocdl.

Since LLVM does not have f8 types, when targeting LLVM for compilation
on an AMD GPU, both f8 types used on AMD hardware (f8E5M2FNUZ and
f8E4M3FNUZ) are rewritten to i8.

This patch also relaxes the restriction that the types of both source
operands to a amdgpu.mfma instructions match exactly, as this is not
necessarily required for the bf8 (f8E5M2FNUZ) and fp8 (f8E4M3FNUZ)
instructions. In addition, since the buffer_{load,store} operations
maintain a whitelist of permitted types, we add the relevant f8 types
to that list.

This patch does not add any implementations of arithmetic operations
for f8 types.

Reviewed By: jakeh-gc

Differential Revision: https://reviews.llvm.org/D143956
2023-02-15 16:46:08 +00:00