This patch adds the capability to perform exact integer emptiness checks for FlatAffineConstraints using the General Basis Reduction algorithm (GBR). Previously, only a heuristic was available for emptiness checks, which was not guaranteed to always give a conclusive result.
This patch adds a `Simplex` class, which can be constructed using a `FlatAffineConstraints`, and can find an integer sample point (if one exists) using the GBR algorithm. Additionally, it adds two classes `Matrix` and `Fraction`, which are used by `Simplex`.
The integer emptiness check functionality can be accessed through the new `FlatAffineConstraints::isIntegerEmpty()` function, which runs the existing heuristic first and, if that proves to be inconclusive, runs the GBR algorithm to produce a conclusive result.
Differential Revision: https://reviews.llvm.org/D80860
This enables better support for traits such as SameOperandsAndResultType, and other situations in which a variadic operand may be resolved from a non-variadic.
Differential Revision: https://reviews.llvm.org/D83011
This allow lowering to support scf.for and scf.if with results. As right now
spv region operations don't have return value the results are demoted to
Function memory. We create one allocation per result right before the region
and store the yield values in it. Then we can load back the value from
allocation to be able to use the results.
Differential Revision: https://reviews.llvm.org/D82246
MSVC 2017 doesn't support the case where a trailing variadic template list comes after template types with default parameters. Until we upgrade to VS 2019, we can't use the simplified definitions.
Moving forward dialects should only be registered in a thread safe context. This matches the existing usage we have today, but it allows for removing quite a bit of expensive locking from the context.
This led to ~.5 a second compile time improvement when running one conversion pass on a very large .mlir file(hundreds of thousands of operations).
Differential Revision: https://reviews.llvm.org/D82595
This revision adds support to ODS for generating interfaces for attributes and types, in addition to operations. These interfaces can be specified using `AttrInterface` and `TypeInterface` in place of `OpInterface`. All of the features of `OpInterface` are supported except for the `verify` method, which does not have a matching representation in the Attribute/Type world. Generating these interface can be done using `gen-(attr|type)-interface-(defs|decls|docs)`.
Differential Revision: https://reviews.llvm.org/D81884
This revisions add mechanisms to Attribute/Type for attaching traits and interfaces. The mechanisms are modeled 1-1 after those for operations to keep the system consistent. AttrBase and TypeBase now accepts a trailing list of `Trait` types that will be attached to the object. These traits should inherit from AttributeTrait::TraitBase and TypeTrait::TraitBase respectively as necessary. A followup commit will refactor the interface gen mechanisms in ODS to support Attribute/Type interface generation and add tests for the mechanisms.
Differential Revision: https://reviews.llvm.org/D81883
Summary:
This changes the casing of MLIRGPUtoGPURuntimeTransforms to be consistent
with other transform libraries.
Differential Revision: https://reviews.llvm.org/D82841
Summary:
This changes the casing of MLIRGPUtoROCDLTransforms to be consistent
with other transform libraries.
Differential Revision: https://reviews.llvm.org/D82843
More efficient implementation of the multiply-reduce pair,
no need to add in a zero vector. Microbenchmarking on AVX2
yields the following difference in vector.contract speedup
(over strict-order scalar reduction).
SPEEDUP SIMD-fma SIMD-mul
4x4 1.45 2.00
8x8 1.40 1.90
32x32 5.32 5.80
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D82833
Summary: The current BufferPlacement implementation does not support
nested region control flow. This CL adds support for nested regions via
the RegionBranchOpInterface and the detection of branch-like
(ReturnLike) terminators inside nested regions.
Differential Revision: https://reviews.llvm.org/D81926
This changes the casing of MLIRGPUtoVulkanTransforms to be consistent
with other transform libraries.
Differential Revision: https://reviews.llvm.org/D82840
The declarative conversion patterns caused crashes in the asan configuration.
The non-declarative implementation circumvents this.
Differential Revision: https://reviews.llvm.org/D82797
These map to the similar accessors on ArrayRef and other random access containers.
This fixes a compilation error on MLIR ODS for variadic operands/results, which relied on the availability of front in certain situations.
Added conversion pattern and tests for `spv.Bitcast` op. This one has
a direct mapping in LLVM dialect so `DirectConversionPattern` was used.
Differential Revision: https://reviews.llvm.org/D82748
Also fixed bug in type inferface generator to address bug where operands and
attributes are interleaved.
Differential Revision: https://reviews.llvm.org/D82819
This patch introduces new conversion patterns for bit and logical
negation op: `spv.Not` and `spv.LogicalNot`. They are implemented
by applying xor on the operand and mask with all bits set.
Differential Revision: https://reviews.llvm.org/D82637
Current Affine comparison builders, which use operator overload, default to signed comparison. This creates the possibility of misuse of these builders and potential correctness issues when dealing with unsigned integers. This change makes the distinction between signed and unsigned comparison builders and forces the caller to make a choice between the two.
Differential Revision: https://reviews.llvm.org/D82323
Summary:
The patch makes the index type lowering of the GPU to NVVM/ROCDL conversion configurable. It introduces a pass option that controls the bitwidth used when lowering index computations and uses the LowerToLLVMOptions structure to control the Standard to LLVM lowering.
This commit fixes a use-after-free bug introduced by the reverted commit d10b1a3. It implements the following changes:
- Added a getDefaultOptions method to the LowerToLLVMOptions struct that returns a reference to statically allocated default options.
- Use the getDefaultOptions method to provide default LowerToLLVMOptions (instead of an initializer list).
- Added comments to clarify the required lifetime of the LowerToLLVMOptions
Reviewed By: ftynse
Differential Revision: https://reviews.llvm.org/D82475
`llvm.mlir.constant` was originally introduced as an LLVM dialect counterpart
to `std.constant`. As such, it was supporting "function pointer" constants
derived from the symbol name. This is different from `std.constant` that allows
for creation of a "function" constant since MLIR, unlike LLVM IR, supports
this. Later, `llvm.mlir.addressof` was introduced as an Op that obtains a
constant pointer to a global in the LLVM dialect. It naturally extends to
functions (in LLVM IR, functions are globals) and should be used for defining
"function pointer" values instead.
Fixes PR46344.
Differential Revision: https://reviews.llvm.org/D82667
When the origin of a shape is an extent tensor the operation `get_extent` can be
lowered directly to `extract_element`.
This choice circumvents the necessity to materialize the shape in memory.
Differential Revision: https://reviews.llvm.org/D82645
When the shape is derived from a tensor argument the shape extent can be derived
directly from that tensor with `std.dim`.
This lowering pattern circumvents the necessity to materialize the shape in
memory.
Differential Revision: https://reviews.llvm.org/D82644
The error message in the `std.constant` verifier for function-typed constants
had the name of the undefined function hardcoded to `bar`. Report the actual
name instead.
Differential Revision: https://reviews.llvm.org/D82666
This test largely predates MLIR testing guidelines. Update it to match the
guidelines. In particular, avoid pattern-matching SSA value names, avoid
unnecessary CHECK-NEXT, relax assumptions about the form of SSA names.
Value-returning operations are still matched agaist _any_ name in order to
check that the operation indeed produces values.
Differential Revision: https://reviews.llvm.org/D82656
Rationale:
In general, passing "fastmath" from MLIR to LLVM backend is not supported, and even just providing such a feature for experimentation is under debate. However, passing fine-grained fastmath related attributes on individual operations is generally accepted. This CL introduces an option to instruct the vector-to-llvm lowering phase to annotate floating-point reductions with the "reassociate" fastmath attribute, which allows the LLVM backend to use SIMD implementations for such constructs. Oher lowering passes can start using this mechanism right away in cases where reassociation is allowed.
Benefit:
For some microbenchmarks on x86-avx2, speedups over 20 were observed for longer vector (due to cleaner, spill-free and SIMD exploiting code).
Usage:
mlir-opt --convert-vector-to-llvm="reassociate-fp-reductions"
Reviewed By: ftynse, mehdi_amini
Differential Revision: https://reviews.llvm.org/D82624
To be able to have more meaningful performance out of workloadsi going through
the vulkan-runner we need to use buffers from GPU device memory as access to
system memory is significantly slower for GPU with dedicated memory. This adds
code to do a copy through staging buffer as GPU memory cannot always be mapped
on the host.
Differential Revision: https://reviews.llvm.org/D82504
Implemented conversion for `spv.BitReverse` and `spv.BitCount`. Since ODS
generates builders in a different way for LLVM dialect intrinsics, I
added attributes to build method in `DirectConversionPattern` class. The
tests for these ops are in `bitwise-ops-to-llvm.mlir`.
Differential Revision: https://reviews.llvm.org/D82286
Add a pass to rewrite sequential chains of `spirv::CompositeInsert`
operations into `spirv::CompositeConstruct` operations.
Reviewed By: antiagainst
Differential Revision: https://reviews.llvm.org/D82198
This patch add support for 'spv.CopyMemory'. The following changes are
introduced:
- 'CopyMemory' op is added to SPIRVOps.td.
- Custom parse and print methods are introduced.
- A few Roundtripping tests are added.
Differential Revision: https://reviews.llvm.org/D82384
Summary: The patch fixes an off by one error in the method collapseParallelLoops. It ensures the same normalized bound is used for the computation of the division and the remainder.
Reviewers: herhut
Reviewed By: herhut
Subscribers: mehdi_amini, rriddle, jpienaar, shauheen, antiagainst, nicolasvasilache, arpith-jacob, mgester, lucyrfox, aartbik, liufengdb, stephenneuendorffer, Joonsoo, grosul1, Kayjukh, jurahul, msifontes
Tags: #mlir
Differential Revision: https://reviews.llvm.org/D82634
Conversions of allocation-related operations in Standard-to-LLVM need
declarations of "malloc" and "free" (or equivalents). They use locally created
OpBuilders pointed at the module level to declare these functions if necessary.
This is poorly compatible with the pattern infrastructure that is unaware of
new operations being created. Update the insertion point of the main rewriter
instead.
Differential Revision: https://reviews.llvm.org/D82649
Initially, unranked memref descriptors in the LLVM dialect were designed only
to be passed into functions. An assertion was guarding against returning
unranked memrefs from functions in the standard-to-LLVM conversion. This is
insufficient for functions that wish to return an unranked memref such that the
caller does not know the rank in advance, and hence cannot allocate the
descriptor and pass it in as an argument.
Introduce a calling convention for returning unranked memref descriptors as
follows. An unranked memref descriptor always points to a ranked memref
descriptor stored on stack of the current function. When an unranked memref
descriptor is returned from a function, the ranked memref descriptor it points
to is copied to dynamically allocated memory, the ownership of which is
transferred to the caller. The caller is responsible for deallocating the
dynamically allocated memory and for copying the pointed-to ranked memref
descriptor onto its stack.
Provide default lowerings for std.return, std.call and std.indirect_call that
maintain the conversion defined above.
This convention is additionally exercised by a runtime test to guard against
memory errors.
Differential Revision: https://reviews.llvm.org/D82647
Using fully qualified names wherever possible avoids ambiguous class and function names. This is a follow-up to D82371.
Reviewed By: rriddle
Differential Revision: https://reviews.llvm.org/D82471
When there is a mix of affine load/store and non-affine operations (e.g. std.load, std.store),
affine-loop-fusion ignores the present of non-affine ops, thus changing the program semantics.
E.g. we have a program of three affine loops operating on the same memref in which one of them uses std.load and std.store, as follows.
```
affine.for
affine.store %1
affine.for
std.load %1
std.store %1
affine.for
affine.load %1
affine.store %1
```
affine-loop-fusion will produce the following result which changed the program semantics:
```
affine.for
std.load %1
std.store %1
affine.for
affine.store %1
affine.load %1
affine.store %1
```
This patch is to fix the above problem by checking non-affine users of the memref that are between the source and destination nodes of interest.
Differential Revision: https://reviews.llvm.org/D82158
Lower `shape.rank` to standard dialect.
A shape's size is the same as the extent of the first and only dimension of the
`tensor<?xindex>` it is represented by.
Differential Revision: https://reviews.llvm.org/D82080
Replace any `rank(shape_of(tensor))` that relies on a ranked tensor with the
corresponding constant `const_size`.
Differential Revision: https://reviews.llvm.org/D82077
