Commit Graph

362 Commits

Author SHA1 Message Date
Aart Bik
10db57b7ea [mlir][sparse] replace magic constant with symbol
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137177
2022-11-01 11:13:20 -07:00
bixia1
d45be88736 [mlir][sparse] Implement the rewrite for sparse_tensor.push_back a value n times.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136654
2022-10-31 08:19:12 -07:00
Peiming Liu
e5cb0ee383 [mlir][sparse] run canonicalization pass after DenseOpBufferize.
As pointed out by Matthias: "DenseBufferizationPass should be run right after TensorCopyInsertionPass. (Running it after bufferizing the sparse IR is also OK.) The reason for this is that whether copies are needed for not depends on the structure of the program (SSA use-def chains). In particular, running the canonicalizer in-between is problematic because it could introduce new RaW conflicts"

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136980
2022-10-29 01:00:05 +00:00
Peiming Liu
63baab8b39 [mlir][sparse] Fold invariant op only when it has only one use.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136990
2022-10-28 23:42:28 +00:00
Aart Bik
5661647e85 [mlir][sparse] build proper insertion chain
The alloc->insert/compress->load chain needs to be
properly represented with an SSA chain now in loops
and if statements to properly reflect the modifying
behavior (runtime support lib is forgiving on breaking
this, but the new codegen is not).

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136966
2022-10-28 15:58:51 -07:00
Alexander Belyaev
b4db15a949 [mlir] Rename getInputs->getDpsInputs and getOutputs->getDpsInits in DPS interface.
https://discourse.llvm.org/t/rfc-interface-for-destination-style-ops/64056

Differential Revision: https://reviews.llvm.org/D136943
2022-10-28 15:41:12 +02:00
Peiming Liu
32c512e49f [mlir][sparse] code refactoring, move <tid, loop id> -> dim map to Merger.
To address unresolved comments in D136185

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136780
2022-10-27 21:01:06 +00:00
Aart Bik
5c32742214 [mlir][sparse] fix typo "admissable" -> "admissible"
Reviewed By: wrengr, Peiming

Differential Revision: https://reviews.llvm.org/D136878
2022-10-27 13:52:02 -07:00
Jordan Rupprecht
71ac3ce938 [NFC] Remove unused variables 2022-10-27 12:23:22 -07:00
Aart Bik
80b08b68f2 [mlir][sparse] add a cursor to sparse storage scheme
This prepare a subsequent revision that will generalize
the insertion code generation. Similar to the support lib,
insertions become much easier to perform with some "cursor"
bookkeeping. Note that we, in the long run, could perhaps
avoid storing the "cursor" permanently and use some
retricted-scope solution (alloca?) instead. However,
that puts harder restrictions on insertion-chain operations,
so for now we follow the more straightforward approach.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136800
2022-10-27 11:18:50 -07:00
Peiming Liu
015bc3464d [mlir][sparse] fix crash when sparsifying broadcast operations.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136866
2022-10-27 17:43:22 +00:00
bixia1
14504cae9a [mlir][sparse] Extend sparse_tensor.push_back to allow push_back a value n times.
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D136653
2022-10-26 09:21:03 -07:00
Peiming Liu
b0f8057e4c [mlir][sparse] use loop emitter to generate loop in sparsification
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136185
2022-10-26 00:27:56 +00:00
Michał Górny
12281099dc [mlir] Fix INSTALL_INTERFACE path for MLIRSparseTensorEnums
Fix INSTALL_INTERFACE path for exported MLIRSparseTensorEnums target
to include the include directory.  Otherwise, CMake attempts to find it
relatively to CMAKE_INSTALL_PREFIX rather than the include directory,
resulting in flang failing to configure with the following error:

    CMake Error at /usr/lib/llvm/16/lib64/cmake/llvm/AddLLVM.cmake:581 (add_library):
      Cannot find source file:

        /usr/lib/llvm/16/mlir/Dialect/SparseTensor/IR/Enums.h

      Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm .h
      .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90 .f95 .f03 .hip .ispc
    Call Stack (most recent call first):
      cmake/modules/AddFlang.cmake:64 (llvm_add_library)
      lib/Lower/CMakeLists.txt:3 (add_flang_library)

Differential Revision: https://reviews.llvm.org/D136590
2022-10-24 20:27:40 +02:00
Peiming Liu
bca2b14a59 [mlir][sparse] fix bufferizableOpInterface for InsertOp
Address comments in D136372

Reviewed By: aartbik, springerm

Differential Revision: https://reviews.llvm.org/D136510
2022-10-24 16:45:21 +00:00
wren romano
d8cb5d3c6e [mlir][sparse] Refining the IR/CMakeLists.txt (followup to D136477)
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136506
2022-10-21 15:23:50 -07:00
Murali Vijayaraghavan
e78247112a Fixed CMakeLists.txt to export mlir/include/mlir/Dialenct/SparseTensor/IR/Enums.h correctly for installing MLIRSparseTensorEnums as a library in other projects using MLIR
Differential Revision: https://reviews.llvm.org/D136477
2022-10-21 21:47:02 +00:00
Jeff Niu
dc63ca78f7 [mlir][sparse] Fix build warning (NFC) 2022-10-21 13:13:07 -07:00
bixia1
e445349d2c [mlir][sparse] Add rewrite rule for the sparse_tensor.out operator.
Also fix the rewrite rule for sparse_tensor.new to reflect the recent change of
the runtime C interface and to use utilities for memref.alloca.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135891
2022-10-21 12:20:53 -07:00
Aart Bik
0f3e4d1afa [mlir][sparse] lower number of entries op to actual code
works both along runtime path and pure codegen path

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136389
2022-10-21 10:48:37 -07:00
bixia1
b056d0bb69 [mlir][sparse] Refactor the convert operator conversion to support codegen for the operator.
Outline the code that generates the loop structure to iterate over a dense
tensor or a sparse constant to genDenseTensorOrSparseConstantIterLoop.

Move a few routines to CodegenUtils for sharing.

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D136210
2022-10-21 08:52:47 -07:00
Peiming Liu
35b3a0ce8d [mlir][sparse] support foreach on dense tensor.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136384
2022-10-21 00:12:37 +00:00
bixia1
cf7906cbbd [mlir][sparse] Fix getUnorderedCOOFromType for rank 1 tensor.
Previously, it used DimLevelType::SingletonNo to represent an unorder COO
tensor of rank 1 while it should use DimLevelType::CompressedNuNo.

Reviewed By: Peiming, wrengr

Differential Revision: https://reviews.llvm.org/D136387
2022-10-20 16:33:01 -07:00
Peiming Liu
fd7469404b [mlir][sparse] attach bufferizableOpInterface to InsertOp
To fix D136286

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136372
2022-10-20 20:54:37 +00:00
bixia1
0128f80167 [mlir][sparse] Fix the codegen for the convert operator to handle hidden nop convert.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136291
2022-10-20 12:30:21 -07:00
Peiming Liu
d12d4857c5 [mlir][sparse] add source materizalization callback for sparse tensor codegen type converter.
Required by scf.for to achieve 1:N type conversion (See D136314).

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136367
2022-10-20 18:04:57 +00:00
Aliia Khasanova
fb4cedcc1e [mlir][nfc] Clean-up usage of kDynamicSize.
This patch prepares MLIR code base to change the value of kDynamicSize.
https://discourse.llvm.org/t/rfc-unify-kdynamicsize-and-kdynamicstrideoroffset/64534/4

Differential Revision: https://reviews.llvm.org/D136327
2022-10-20 13:54:57 +00:00
wren romano
5e12c18392 [mlir][sparse] Fix breakage on older versions of cmake
Per https://reviews.llvm.org/D136005#3866692 the introduction of the MLIRSparseTensorEnums target in D136002 caused breakage on some versions of cmake.  This differential aims to fix those errors.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136217
2022-10-19 13:55:54 -07:00
Aart Bik
96cab659a1 [mlir][sparse] end-to-end sparse vector insertion codegen
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136275
2022-10-19 12:32:20 -07:00
Peiming Liu
26eb2c6b42 [mlir][sparse] remove vector support in sparsification
Sparse compiler used to generate vectorized code for sparse tensors computation, but it should really be delegated to other vectorization passes for better progressive lowering.

 https://discourse.llvm.org/t/rfc-structured-codegen-beyond-rectangular-arrays/64707

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136183
2022-10-19 18:11:29 +00:00
bixia1
58b449c3bb [mlir][sparse] Replace the folding of nop convert with a codegen rule.
This is to allow the use of a nop convert to express that the sparse tensor
allocated through bufferization::AllocTensorOp will be expanded to sparse
tensor storage by sparse tensor codegen.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136214
2022-10-19 10:20:47 -07:00
wren romano
e0401d2252 [mlir][sparse] Removing the DimLvlType and DimLevelFormat types
This removes another massive source of redundancy, and instead has the Merger.{h,cpp} reuse the SparseTensorEnums library.

Depends On D136005

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136123
2022-10-18 15:47:40 -07:00
wren romano
062f29c8d0 [mlir][sparse] Moving Enums.h into Dialect/SparseTensor/IR
Move the SparseTensorEnums library out of the ExecutionEngine directory and into Dialect/SparseTensor/IR.

Depends On D136002

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136005
2022-10-18 15:15:18 -07:00
wren romano
181b04d276 [mlir][sparse] Factoring out SparseTensorEnums library
This differential splits the SparseTensorEnums library out from the SparseTensorRuntime library. The actual moving of files will be handled in the next differential.

Depends On D135996

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136002
2022-10-18 14:20:33 -07:00
Aart Bik
d22df0ebba [mlir][sparse] refine insertion code
builds SSA cycle for compress insertion loop
adds casting on index mismatch during push_back

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136186
2022-10-18 14:16:38 -07:00
wren romano
0e77b63bc0 [mlir][sparse] Use the runtime DimLevelType instead of a separate tablegen enum
This differential replaces all uses of SparseTensorEncodingAttr::DimLevelType with DimLevelType.  The next differential will break out a separate library for the DimLevelType enum, so that the Dialect code doesn't need to depend on the rest of the runtime

Depends On D135995

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135996
2022-10-18 13:45:26 -07:00
wren romano
d9affadd00 [mlir][sparse] rename the values of the runtime DimLevelType
This change is to make way for reusing the DimLevelType enum in lieu of the SparseTensorEncodingAttr::DimLevelType enum, but broken out to make it quick and easy to review

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135995
2022-10-18 12:08:19 -07:00
bixia1
c1864ab953 [mlir][sparse] Add options to sparse-tensor-rewrite to disable rewriting rules for operators foreach and convert.
This is to help simplify FileCheck tests for sparse-tensor-rewrite.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136093
2022-10-18 09:27:32 -07:00
bixia1
933e7e2393 [mlir][sparse] Change getUnorderedCOOFromType to propagate the overhead types.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136112
2022-10-18 08:01:16 -07:00
Aart Bik
9f596a7c67 [mlir][sparse] implement simple codegen for insertion (and related ops)
This is a proof of concept insertion implementation that sets up
the basic framework and implements it with push backs for just
sparse vectors. It adds insertion/compression through SSA values,
so that we properly update the memref after after pushback operation.

Note that properly using SSA values in sparsification is still TBD
but I will wait until Peiming's loop emitter is in to avoid conflicts.

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D136008
2022-10-17 18:02:08 -07:00
Alexander Belyaev
a7cccb9cbb [mlir] Simplify DestinationStyleOpInterface.
Differential Revision: https://reviews.llvm.org/D135348
2022-10-17 12:43:41 +02:00
bixia1
5f919cd439 [mlir][sparse] Move routines for generating memref.alloca to CodegenUtils.
This is to allow the use of the routines in the rewrite pass.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135890
2022-10-16 08:22:02 -07:00
Oleg Shyshkov
c38d9cf20e [mlir] Remove iterator_types() method from LinalgStructuredInterface.
`getIteratorTypesArray` should be used instead. It's a better substitute for all the current usages of the interface.

The current `ArrayAttr iterator_types()` has a few problems:
* It creates an assumption operation has iterators types as an attribute, but it's not always the case. Sometime iterator types can be inferred from other attribute, or they're just static.
* ArrayAttr is an obscure contained and required extracting values in the client code.
* Makes it hard to migrate iterator types from strings to enums ([RFC](https://discourse.llvm.org/t/rfc-enumattr-for-iterator-types-in-linalg/64535/9)).

Concrete ops, like `linalg.generic` will still have iterator types as an attribute if needed.

As a side effect, this change helps a bit with migration to prefixed accessors.

Differential Revision: https://reviews.llvm.org/D135765
2022-10-13 07:52:43 +00:00
Mehdi Amini
d7802c4811 Apply clang-tidy fixes for llvm-qualified-auto in SparseTensorCodegen.cpp (NFC) 2022-10-13 02:58:34 +00:00
Mehdi Amini
9b170d126f Apply clang-tidy fixes for readability-identifier-naming in SparseBufferRewriting.cpp (NFC) 2022-10-13 02:58:34 +00:00
Kazu Hirata
b2674de375 [mlir] Fix a warning
This patch fixes:

  mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorRewriting.cpp:587:27:
  error: comparison of integers of different signs: 'int64_t' (aka
  'long') and 'uint64_t' (aka 'unsigned long')
  [-Werror,-Wsign-compare]
2022-10-12 15:25:17 -07:00
Mehdi Amini
f628d6c3e8 Apply clang-tidy fixes for modernize-use-using in SparseBufferRewriting.cpp (NFC) 2022-10-12 16:22:43 +00:00
Mehdi Amini
add7bf150c Apply clang-tidy fixes for performance-unnecessary-copy-initialization in CodegenUtils.cpp (NFC) 2022-10-12 16:22:43 +00:00
bixia1
67f61b0863 [mlir][sparse] Add rewriting rules for the sparse_tensor.new operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135692
2022-10-11 15:23:11 -07:00
bixia1
2d252a0f5c [mlir][sparse] Move a few routines to CodegenUtils.
Move a few supporting routines for generating function calls to CodegenUtils so
that they can be used by the codegen path for sparse tensor file input and
output.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135691
2022-10-11 14:42:18 -07:00