Commit Graph

348 Commits

Author SHA1 Message Date
Mateusz Jablonski
3d9b5d441a Add UNRECOVERABLE_IF to avoid nullptr dereference
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-08-09 15:44:49 +02:00
Lukasz Jobczyk
eb8cd33dc6 Do not flush tag update if already flushed
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-08-08 14:11:42 +02:00
Neil R Spruit
ada9b5d4a9 Handle if L0 Event Memory is Shareable
- Properly check for IPC event handle flag to determine if the event
pool memory is sharable between processes.
- Given Host Visible Event Pool, a check is done to determine if the
Host memory can be shared between the processes.
- Enabled handling if Event Host Memory is shareable for DRM
- If Event Pool Memory is Not shareable, then retrieving the IPC Event
Pool Handle returns unsupported.

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2022-08-05 00:11:05 +02:00
Milczarek, Slawomir
601ace6a25 Add unit test for append memory prefetch for regular command list
Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-08-04 10:32:08 +02:00
Dunajski, Bartosz
98d776867f Add initial support for KernelArgsBuffer allocation
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
2022-08-03 20:28:21 +02:00
Milczarek, Slawomir
046f9d95fc zeCommandListAppendMemoryPrefetch with memory prefetch manager
Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-08-02 20:07:31 +02:00
Kamil Kopryk
17f22990e6 Cleanup includes 4/n
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-28 09:44:48 +02:00
Kamil Kopryk
0603819b68 Cleanup includes 3/n
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-27 15:28:13 +02:00
Kamil Kopryk
d4d54f5093 Cleanup includes
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-25 09:58:38 +02:00
Lukasz Jobczyk
4ac6e09117 Do not reuse mem obj's allocation
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-07-19 11:42:17 +02:00
Milczarek, Slawomir
3e22f30ed5 clEnqueueMigrateMemINTEL with memory prefetch for KMD migrated shared allocation
Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-07-14 17:35:22 +02:00
Daniel Chabrowski
bdf0483068 Hide details of AllocationsList from header
Signed-off-by: Daniel Chabrowski <daniel.chabrowski@intel.com>
2022-07-14 12:19:19 +02:00
cameron
60c819bb72 Fix Memory leak in drm_memory_manager of OsHandleLinux
Add virtual deconstructor to OsHandle and deconstructor to OsHandleLinux
Add override keyword to destructor
Add overriding deconstructor to OsHandleWin
Add newline before private members

https://github.com/intel/compute-runtime/pull/550

Signed-off-by: Cameron S Murtagh <cameron.murtagh00@gmail.com>
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-07-13 11:23:48 +02:00
Compute-Runtime-Validation
f53ae0a50e Revert "Add debug flag for EOT WA"
This reverts commit cf3817e058.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-07-09 01:21:56 +02:00
Mateusz Jablonski
cf3817e058 Add debug flag for EOT WA
EOT WA requires allocating last 64KB of kernel heap and putting EOT
signature at the last 16 bytes of kernel heap

Related-To: NEO-7099
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-07-07 16:58:24 +02:00
Milczarek, Slawomir
6a9fcd38b1 Create KMD-migrated unified shared memory with multiple local memory regions
Remove the restriction on USM allocation created in a single local memory region
with latest KMD fix for cross tile migration thrashing b/t lmem (dii-3516)

Related-To: NEO-6909

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-07-04 13:33:23 +02:00
Mateusz Hoppe
31dbc04f23 L0Debug - capture SurfaceState heaps
Related-To: NEO-7103

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2022-06-24 15:05:56 +02:00
Maciej Plewka
6ab6e1abff Fix mutex order for event task and move args to gpu
This commit fixes problem with untransfered shared usm memory to gpu
when there is submit to gpu trigerred by user event. Also there is a fix
for dead lock problem caused by mixed orders of locking mutexes in csr
and in direct submission controller.

Related-To: NEO-6762

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-06-21 11:28:25 +02:00
Maciej Plewka
213dc2fe24 Make CPU copy for read buffer when host ptr is write combined on DG2
With this commit on DG2 32bit driver will check if passed host ptr for
clEnqueueReadBuffer is write combined memory. If check will be true copy
will be make on CPU.

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-06-13 21:23:21 +02:00
Dominik Dabek
cdfe2ce8ad Feature: Flag for device usm allocation reusing
With flag enabled, when app calls freeSVMAlloc on device usm allocation,
don't free it immediately but save it,
and try to use it on subsequent allocations.

This allocation cache will be trimmed if an allocation fails.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-06-13 20:02:52 +02:00
Bartosz Dunajski
39c1c4d530 Remove virtual padding support
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-06-08 12:42:44 +02:00
Krzysztof Gibala
77dde01503 Pass canonized gpuAddress in setCpuPtrAndGpuAddress
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-07 10:05:18 +02:00
Krzysztof Gibala
c51ce2a35c Use GmmHelper as an object in freeGraphicsMemoryImpl
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-07 09:53:41 +02:00
Krzysztof Gibala
81899c4477 Add canonized gpuAddress to GraphicsAllocation constructor
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-06 18:17:36 +02:00
Krzysztof Gibala
dc1fe7d59a Change MemoryPool to enum class
Use enum class for MemoryPool in GraphicsAllocation
This change will ensure that GA is constructed in the proper way

- Rename namespace for isSystemMemoryPool method
- Add method getMemoryPoolString for logging actual pool which is in used
- Remove wrong pattern in GraphicsAllocation constructor

Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-02 12:46:15 +02:00
Jaime Arteaga
325db6a99c Fix P2P support for implicit scaling
when using implicit scaling, 2 dma-buf handles, one per tile, are
needed to support dma access from peer.

Related-To: LOCI-3122

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-06-01 20:10:32 +02:00
Krzysztof Gibala
ae56d50b4f Pass canonized gpuAddress to GraphicsAllocation
Related-To: NEO-6523

Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-30 16:54:38 +02:00
Daniel Chabrowski
7463e1970b Cleanup headers
Make TUs and headers self-contained, remove unused headers

Signed-off-by: Daniel Chabrowski <daniel.chabrowski@intel.com>
2022-05-18 11:42:06 +02:00
Artur Harasimiuk
3f04769f07 style: configure readability-identifier-naming.FunctionCase
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-17 20:55:56 +02:00
Szymon Morek
4266f861ac Make implicit flush for cross-device dependency
Related-To: NEO-6418

If there's a cross-device dependency, flush batched
submissions to avoid deadlock.

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2022-05-16 16:29:09 +02:00
Milczarek, Slawomir
9d31d36491 Disable cross-tile kmd migration for usm allocations
Ensure KMD migrations for USM allocations to occur between smem and lmem only

Related-To: NEO-6969

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-05-16 15:23:34 +02:00
Artur Harasimiuk
819e0f5515 style: configure readability-identifier-naming.LocalVariableCase
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-16 12:39:44 +02:00
Artur Harasimiuk
e9be9b64c6 clang-tidy configuration cleanup
Define single .clang-tidy configuration with all used checks and use
NOLINT to selectively silence tool. That way cleanup should be easier.
third_part/ has its own configuration that disables clang-tidy for this
folder.

Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-11 14:02:04 +02:00
Krzysztof Gibala
2fcda0a528 Refactor: Change decanonize method accessing point
Accessing decanonize method as a member of GmmHelper class object

Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-11 12:57:02 +02:00
Mateusz Jablonski
d935815d74 Move residency data from WddmAllocation to GraphicsAllocation
Related-To: NEO-6848

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 18:16:42 +02:00
Mateusz Jablonski
943ad0e1eb style: skip redundant unique_ptr::get function
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 13:22:40 +02:00
Krzysztof Gibala
4f1e01d279 Create getGmmHelper function in MemoryManager
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 16:51:23 +02:00
Krzysztof Gibala
1c366d1ec0 Refactor: Change canonize method accessing point
Accessing canonize method as a member of GmmHelper class object

Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 14:16:31 +02:00
Jaime Arteaga
3f26f45c10 Add support for IPC handles with implicit scaling
When using implicit scaling, device allocations may have
more than one internal allocation created internally. In that case,
a separate dma-buf handle per internal allocation needs to be
exported.

So introduced two driver experimental extensions to export and
import more than one IPC handle:

- zexMemGetIpcHandles
- zexMemOpenIpcHandles

Related-To: LOCI-2919

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-05-09 00:38:17 +02:00
Maciej Plewka
0a16dc6c47 Fix multi thread usage of external host alloc
This is fixed reupload of this commit after auto revert
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.

Resolves: NEO-6807

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-05-05 13:32:15 +02:00
Compute-Runtime-Validation
00a1a14652 Revert "Fix multi thread usage of external host alloc"
This reverts commit 54eee2a88b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-04-28 17:42:07 +02:00
Krzysztof Gibala
9b778863b4 Store GmmHelper in Gmm class
Store GmmHelper in Gmm class instead of GmmClientContext

Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-04-27 15:45:49 +02:00
Maciej Plewka
54eee2a88b Fix multi thread usage of external host alloc
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.

Resolves: NEO-6807

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-04-26 15:31:24 +02:00
Mateusz Hoppe
c9d61840a2 Allocate SBA buffers per HW context
- different physical storage for every HW context
- adds support for debugging with implicit scaling on
- reorganize tests

Relates-To: NEO-6883

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2022-04-15 10:01:28 +02:00
Dominik Dabek
8d1ad5a4f3 Refactor: use stack vector for root device indices
Stack vector will not cause dynamic allocations in most circumstances
ie. number of root device indices not more than 16

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-14 14:05:42 +02:00
Patryk Wrobel
352583b9d9 Detect GPU hang in evictUnusedAllocations()
This change introduces checking of the return value
of wait function in case of blocking version of
evictUnusedAllocations(). Furthermore, it propagates
the error to the callers. It contains also ULTs.

Related-To: NEO-6681
Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
2022-04-13 15:39:02 +02:00
John Falkowski
789587bacc Return nullptr for gfx alloc mmap error
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2022-04-12 10:08:19 +02:00
Dominik Dabek
29e4518407 Update formatting in dynamic memory tracking
Format as follows:
backtrace start
call_1
...
call_n
backtrace end

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-08 13:17:30 +02:00
Lukasz Jobczyk
fffcf2612e Flush tag only if needed in deferred deleter
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-04-07 15:36:29 +02:00
Compute-Runtime-Validation
7a3976ad64 Revert "Force 64KB page size for cpu alignment in dual storage allocation"
This reverts commit 7ff6a5c1fa.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-04-07 00:45:07 +02:00