Commit Graph

112 Commits

Author SHA1 Message Date
Peiming Liu
fc126022e8 [mlir][sparse] fuse collapse_shape on sparse tensor with GenericOp.
Instead of always materializing a new sparse tensor after reshape, this patch tries to fuses the reshape (currently only on COO) with GenericOp and coiterates with the reshaped tensors without allocating a new sparse tensor.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D145016
2023-03-01 19:05:48 +00:00
Peiming Liu
85dbb3fc4b [mlir][sparse] support sparse tensor element type conversion in codegen path
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144578
2023-02-23 17:49:50 +00:00
Peiming Liu
44ff23d5e4 [mlir][sparse] unconditionally use IndexType for sparse_tensor.specifier
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144574
2023-02-22 20:21:34 +00:00
Peiming Liu
9e8d9316ce [mlir][sparse] allow foreach operation to generate out-of-order loop on non-annotated tensor.
No need for a temp COO and sort even when converting dense -> CSC, we can instead rotate the loop to yield a ordered coordinates at beginning.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144213
2023-02-16 23:23:20 +00:00
bixia1
c2e248c6ae [mlir][sparse] Remove the expansion of symmetric MTX in the sparse tensor storage.
We will support symmetric MTX without expanding the data in the sparse tensor
storage.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D144059
2023-02-16 13:02:17 -08:00
Peiming Liu
c738b430c4 [mlir][sparse] introduce operations to query sparse tensor slice offset/strides at the given dimenion
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D141442
2023-02-16 00:31:11 +00:00
wren romano
ae7942e296 [mlir][sparse] adding SparseTensorType::get{Pointer,Index}Type methods
Depends On D143800

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143946
2023-02-15 14:37:55 -08:00
wren romano
f708a549b8 [mlir][sparse] Factoring out SparseTensorType class
This change adds a new `SparseTensorType` class for making the "dim" vs "lvl" distinction more overt, and for abstracting over the differences between sparse-tensors and dense-tensors.  In addition, this change also adds new type aliases `Dimension`, `Level`, and `FieldIndex` to make code more self-documenting.

Although the diff is very large, the majority of the changes are mechanical in nature (e.g., changing types to use the new aliases, updating variable names to match, etc).  Along the way I also made many variables `const` when they could be; the majority of which required only adding the keyword.  A few places had conditional definitions of these variables, requiring actual code changes; however, that was only done when the overall change was extremely local and easy to extract.  All these changes are included in the current patch only because it would be too onerous to split them off into a separate patch.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143800
2023-02-14 19:17:19 -08:00
Peiming Liu
6dbca86d83 [mlir][sparse] introduce sparse_tensor::unpack operation
An inverse operation of sparse_tenosr::pack

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143669
2023-02-11 01:01:35 +00:00
Peiming Liu
0352690421 [mlir][sparse] make foreach operation support sparse tensor slices.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140713
2023-02-08 18:58:35 +00:00
Peiming Liu
1f07853f2b [mlir][sparse] introduce sparse_tensor.pack operation
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D143224
2023-02-03 22:30:52 +00:00
wren romano
9916ab03f1 [mlir][sparse] (re)introducing getRankedTensorType/getMemrefType
The bulk of D142074 seems to have gotten overwritten due to some sort of merge conflict (afaict there's no record of it having been reverted intentionally).  So this commit redoes those changes.  In addition to the original changes, this commit also:
* moves the definition of `getRankedTensorType` (from `Transforms/CodegenUtils.h` to `IR/SparseTensor.h`), so that it can be used by `IR/SparseTensorDialect.cpp`.
* adds `getMemRefType` as another abbreviation.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D142503
2023-01-25 11:29:54 -08:00
Kazu Hirata
aef953154a [mlir] Fix a warning
This patch fixes:

  mlir/lib/Dialect/SparseTensor/IR/SparseTensorDialect.cpp:528:22:
  error: comparison of integers of different signs: 'uint64_t' (aka
  'unsigned long') and 'int64_t' (aka 'long') [-Werror,-Wsign-compare]
2023-01-20 14:33:57 -08:00
wren romano
743fbcb79d [mlir][sparse] IR/SparseTensorDialect.cpp: misc code cleanup
Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D142072
2023-01-20 13:31:39 -08:00
Kazu Hirata
91682b2631 Remove redundant initialization of std::optional (NFC) 2023-01-14 14:06:18 -08:00
Peiming Liu
885a1f4316 [mlir][sparse] support parsing slices in sparse tensor encoding attribute
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D140712
2023-01-12 22:35:24 +00:00
Markus Böck
7df761217c [mlir][NFC] Migrate rest of the dialects to the new fold API 2023-01-11 21:47:25 +01:00
bixia1
9bde3d0cc5 [mlir][sparse] Add operator sparse_tensor.indices_buffer.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140762
2023-01-05 09:35:55 -08:00
bixia1
90aa436291 [mlir][sparse] Add layout to the memref for the indices buffers to prepare for the AOS storage optimization for COO regions.
Fix relevant FileCheck tests.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140742
2023-01-04 07:36:11 -08:00
bixia1
840e2ba336 [mlir][sparse] Use DLT in the mangled function names for insertion.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140484
2022-12-28 08:21:22 -08:00
Peiming Liu
988733c600 [mlir][sparse] use sparse_tensor::StorageSpecifier to store dim/memSizes
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140130
2022-12-23 00:47:36 +00:00
Peiming Liu
083ddffe47 [mlir][sparse] introduce sparse_tensor::StorageSpecifierToLLVM pass
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140122
2022-12-22 22:45:15 +00:00
Ramkumar Ramachandra
22426110c5 mlir/tblgen: use std::optional in generation
This is part of an effort to migrate from llvm::Optional to
std::optional. This patch changes the way mlir-tblgen generates .inc
files, and modifies tests and documentation appropriately. It is a "no
compromises" patch, and doesn't leave the user with an unpleasant mix of
llvm::Optional and std::optional.

A non-trivial change has been made to ControlFlowInterfaces to split one
constructor into two, relating to a build failure on Windows.

See also: https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716

Signed-off-by: Ramkumar Ramachandra <r@artagnon.com>

Differential Revision: https://reviews.llvm.org/D138934
2022-12-17 11:13:26 +01:00
Peiming Liu
a3672add76 [mlir][sparse] avoid unnecessary tmp COO buffer and convert when lowering ConcatentateOp.
When concat along dim 0, and all inputs/outputs are ordered with identity dimension ordering,
the concatenated coordinates will be yield in lexOrder, thus no need to sort.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D140228
2022-12-16 18:26:39 +00:00
wren romano
96fef4dc3c [mlir][sparse] Added new SparseTensorEncodingAttr::withoutOrdering factory
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D140171
2022-12-15 18:14:54 -08:00
Aart Bik
9c9b47c976 [mlir][sparse] add dim level type toString convenience method
Reviewed By: wrengr, bixia

Differential Revision: https://reviews.llvm.org/D140165
2022-12-15 16:11:30 -08:00
Peiming Liu
509974af02 [mlir][sparse] add folder to sparse_tensor.storage.get operation.
Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D140172
2022-12-15 23:51:38 +00:00
Peiming Liu
71cc0f1c04 [mlir][sparse] introduce sparse_tensor::StorageSpecifierType and related operations on it
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139961
2022-12-14 22:27:22 +00:00
wren romano
387755a35d [mlir][sparse] Simplifying SparseTensorEncodingAttr function arguments
Since STEA isa Attribute, and that's just (a wrapper around) a pointer, the extra `const` and `&` aren't necessary for function arguments.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D139886
2022-12-12 17:05:56 -08:00
Kazu Hirata
1a36588ec6 [mlir] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-03 18:50:27 -08:00
Peiming Liu
dd33481f48 [mlir][sparse] add getPointerType/getIndexType to SparseTensorEncodingAttr.
add new interfaces to SparseTensorEncodingAttr to construct the pointer/index types based on pointer/index bitwidth.

Reviewed By: aartbik, wrengr

Differential Revision: https://reviews.llvm.org/D139141
2022-12-01 22:01:52 +00:00
Diego Caballero
bf3f7016b1 [mlir][NFC] Generalize getPermutedPosition
Small change to support projected permutations in the
`getPermutedPosition` utility. Renamed to `getResultPosition`.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D138946
2022-12-01 18:58:25 +00:00
bixia1
974b4bf9fd [mlir][sparse] Add expand_symmetry attribute to the new operator.
The attribute tells the operator to handle symmetric structures for 2D tensors.
By default, the operator assumes the input tensor is not symmetric.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D138230
2022-11-23 16:32:15 -08:00
Aliia Khasanova
399638f98c Merge kDynamicSize and kDynamicSentinel into one constant.
resolve conflicts

Differential Revision: https://reviews.llvm.org/D138282
2022-11-21 13:01:26 +00:00
Aart Bik
0e1708ff64 [mlir][sparse] cleanup small vector constant hints
Following advise from

https://llvm.org/docs/ProgrammersManual.html#llvm-adt-smallvector-h

This revision removes the size hints from SmallVector (unless we are
certain of the resulting number of elements). Also, this replaces
SmallVector references with SmallVectorImpl references.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138063
2022-11-15 15:09:06 -08:00
Mehdi Amini
239737eda5 Apply clang-tidy fixes for performance-unnecessary-value-param in SparseTensorDialect.cpp (NFC) 2022-11-10 23:31:16 +00:00
Peiming Liu
4fa00ce15c [mlir][sparse] extend foreach operation to accept reduction arguments; fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
This patch re-commit D137468 and D137463, which were reverted by mistakes.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137579
2022-11-07 21:40:30 +00:00
Kazu Hirata
c4b74658c7 [mlir] Fix a warning (NFC)
This patch fixes:

  mlir/lib/Dialect/SparseTensor/IR/SparseTensorDialect.cpp:717:48:
  error: comparison of integers of different signs: 'int64_t' (aka
  'long') and 'uint64_t' (aka 'unsigned long')
  [-Werror,-Wsign-compare]
2022-11-07 09:54:10 -08:00
Stella Stamenova
99171078bb Revert "[mlir][sparse] extend foreach operation to accept reduction arguments."
This reverts commit 53d5d34011.

This is causing a build failure on the windows mlir bot that was previously hidden by another sparse tensor change that caused failures:

https://lab.llvm.org/buildbot/#/builders/13/builds/28006
2022-11-07 09:34:10 -08:00
Stella Stamenova
ec224e3b68 Revert "[mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly."
This reverts commit 70508b614e.

This change depends on a reverted change that broke the windows mlir buildbot; reverting to bring remaining mlir bots to green
2022-11-07 09:00:08 -08:00
bixia1
cf24d49dc8 [mlir][sparse] Add sparse_tensor.sort_coo operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137442
2022-11-07 08:23:51 -08:00
Peiming Liu
70508b614e [mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137468
2022-11-04 23:43:54 +00:00
Peiming Liu
53d5d34011 [mlir][sparse] extend foreach operation to accept reduction arguments.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137463
2022-11-04 23:34:16 +00:00
bixia1
eb877006a6 [mlir][sparse] Add rewriting rule for the convert operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136301
2022-11-01 15:57:34 -07:00
bixia1
14504cae9a [mlir][sparse] Extend sparse_tensor.push_back to allow push_back a value n times.
Reviewed By: aartbik, Peiming

Differential Revision: https://reviews.llvm.org/D136653
2022-10-26 09:21:03 -07:00
bixia1
0128f80167 [mlir][sparse] Fix the codegen for the convert operator to handle hidden nop convert.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136291
2022-10-20 12:30:21 -07:00
Aliia Khasanova
fb4cedcc1e [mlir][nfc] Clean-up usage of kDynamicSize.
This patch prepares MLIR code base to change the value of kDynamicSize.
https://discourse.llvm.org/t/rfc-unify-kdynamicsize-and-kdynamicstrideoroffset/64534/4

Differential Revision: https://reviews.llvm.org/D136327
2022-10-20 13:54:57 +00:00
bixia1
58b449c3bb [mlir][sparse] Replace the folding of nop convert with a codegen rule.
This is to allow the use of a nop convert to express that the sparse tensor
allocated through bufferization::AllocTensorOp will be expanded to sparse
tensor storage by sparse tensor codegen.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136214
2022-10-19 10:20:47 -07:00
wren romano
e0401d2252 [mlir][sparse] Removing the DimLvlType and DimLevelFormat types
This removes another massive source of redundancy, and instead has the Merger.{h,cpp} reuse the SparseTensorEnums library.

Depends On D136005

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D136123
2022-10-18 15:47:40 -07:00
wren romano
0e77b63bc0 [mlir][sparse] Use the runtime DimLevelType instead of a separate tablegen enum
This differential replaces all uses of SparseTensorEncodingAttr::DimLevelType with DimLevelType.  The next differential will break out a separate library for the DimLevelType enum, so that the Dialect code doesn't need to depend on the rest of the runtime

Depends On D135995

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135996
2022-10-18 13:45:26 -07:00