Commit Graph

316 Commits

Author SHA1 Message Date
Artur Harasimiuk
e9be9b64c6 clang-tidy configuration cleanup
Define single .clang-tidy configuration with all used checks and use
NOLINT to selectively silence tool. That way cleanup should be easier.
third_part/ has its own configuration that disables clang-tidy for this
folder.

Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-11 14:02:04 +02:00
Krzysztof Gibala
2fcda0a528 Refactor: Change decanonize method accessing point
Accessing decanonize method as a member of GmmHelper class object

Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-11 12:57:02 +02:00
Mateusz Jablonski
d935815d74 Move residency data from WddmAllocation to GraphicsAllocation
Related-To: NEO-6848

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 18:16:42 +02:00
Mateusz Jablonski
943ad0e1eb style: skip redundant unique_ptr::get function
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 13:22:40 +02:00
Krzysztof Gibala
4f1e01d279 Create getGmmHelper function in MemoryManager
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 16:51:23 +02:00
Krzysztof Gibala
1c366d1ec0 Refactor: Change canonize method accessing point
Accessing canonize method as a member of GmmHelper class object

Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 14:16:31 +02:00
Jaime Arteaga
3f26f45c10 Add support for IPC handles with implicit scaling
When using implicit scaling, device allocations may have
more than one internal allocation created internally. In that case,
a separate dma-buf handle per internal allocation needs to be
exported.

So introduced two driver experimental extensions to export and
import more than one IPC handle:

- zexMemGetIpcHandles
- zexMemOpenIpcHandles

Related-To: LOCI-2919

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-05-09 00:38:17 +02:00
Maciej Plewka
0a16dc6c47 Fix multi thread usage of external host alloc
This is fixed reupload of this commit after auto revert
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.

Resolves: NEO-6807

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-05-05 13:32:15 +02:00
Compute-Runtime-Validation
00a1a14652 Revert "Fix multi thread usage of external host alloc"
This reverts commit 54eee2a88b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-04-28 17:42:07 +02:00
Krzysztof Gibala
9b778863b4 Store GmmHelper in Gmm class
Store GmmHelper in Gmm class instead of GmmClientContext

Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-04-27 15:45:49 +02:00
Maciej Plewka
54eee2a88b Fix multi thread usage of external host alloc
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.

Resolves: NEO-6807

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-04-26 15:31:24 +02:00
Mateusz Hoppe
c9d61840a2 Allocate SBA buffers per HW context
- different physical storage for every HW context
- adds support for debugging with implicit scaling on
- reorganize tests

Relates-To: NEO-6883

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2022-04-15 10:01:28 +02:00
Dominik Dabek
8d1ad5a4f3 Refactor: use stack vector for root device indices
Stack vector will not cause dynamic allocations in most circumstances
ie. number of root device indices not more than 16

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-14 14:05:42 +02:00
Patryk Wrobel
352583b9d9 Detect GPU hang in evictUnusedAllocations()
This change introduces checking of the return value
of wait function in case of blocking version of
evictUnusedAllocations(). Furthermore, it propagates
the error to the callers. It contains also ULTs.

Related-To: NEO-6681
Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
2022-04-13 15:39:02 +02:00
John Falkowski
789587bacc Return nullptr for gfx alloc mmap error
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2022-04-12 10:08:19 +02:00
Dominik Dabek
29e4518407 Update formatting in dynamic memory tracking
Format as follows:
backtrace start
call_1
...
call_n
backtrace end

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-08 13:17:30 +02:00
Lukasz Jobczyk
fffcf2612e Flush tag only if needed in deferred deleter
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-04-07 15:36:29 +02:00
Compute-Runtime-Validation
7a3976ad64 Revert "Force 64KB page size for cpu alignment in dual storage allocation"
This reverts commit 7ff6a5c1fa.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-04-07 00:45:07 +02:00
Kamil Diedrich
7ff6a5c1fa Force 64KB page size for cpu alignment in dual storage allocation
There is no need to force 2MB alignment for CPU allocation in dual
storage usage. Additionaly for WSL this will allow to avoid usage of
malloc in driver path.

Relates-To: NEO-6620
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2022-04-06 15:28:17 +02:00
Dominik Dabek
8d5c674110 Dynamic memory tracking, update function printing
Print only demangled name if it succeeded.

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-06 14:34:24 +02:00
Dominik Dabek
f7a767c767 Add demangling to dynamic memory tracking
Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-05 20:38:27 +02:00
Filip Hazubski
88193cc242 Minor fixes
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2022-03-31 20:02:11 +02:00
Michal Mrozek
0c0603966b Add debug functionality to track dynamic allocations.
- available only via manual build with ENABLE_DYNAMIC_MEMORY_TRACKING.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-03-31 13:24:15 +02:00
Compute-Runtime-Validation
91cfd3cd1a Revert "Unify command/ring/semaphore buffers placement"
This reverts commit e035199de4.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-03-30 14:05:47 +02:00
Mateusz Jablonski
e035199de4 Unify command/ring/semaphore buffers placement
put them all to the same memory location

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-29 17:55:48 +02:00
Lukasz Jobczyk
a230f267e1 Poll task count indefinitely on high throttle command queue
Resolves: NEO-6781

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-03-25 10:06:16 +01:00
Mateusz Jablonski
8a8b4866cb XeHPC: force local memory for command/ring/semaphore buffer
require 48bit resource for ring/semaphore buffer
for multi tile allocations select first tile
for single tile allocation select preferred tile

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-24 10:54:26 +01:00
Milczarek, Slawomir
f03f530327 Extend zeCommandListAppendMemoryPrefetch to migrate to associated device
Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-23 14:21:17 +01:00
Mateusz Jablonski
3792481d33 XeHPC Implicit scaling: put command/ring/semaphore buffer to first memory bank
In direct submission scenario command/ring/semaphore buffer allocations
are placed in the same memory bank to ensure that their memory is updated in
correct order

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-21 08:10:50 +01:00
Daniel Chabrowski
754efbdaff Cleanup mocks
Signed-off-by: Daniel Chabrowski daniel.chabrowski@intel.com
Related-To: NEO-6591
2022-03-18 11:54:03 +01:00
Jaime Arteaga
c0e2251ceb Skip adding allocations to remote devices if not allocated there
When making graphics allocations resident in multi-GPU scenarios,
we should make them resident only if there's an allocation for that
device. So return appropriate null pointer and skip it.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-16 23:58:07 +01:00
Maciej Bielski
a71b88fefb PageFaultHandler: speedup UM allocations lookup
Add a per-instance SVMAllocsManager::nonGpuDomainAllocs container for
all allocations to be removed in
moveAllocationsWithinUMAllocsManagerToGpuDomain. This approach replaces
the current iterative search and performs the task faster.

Add 7 new unit-tests to verify the functionality related to
nonGpuDomainAllocs container, both in expected and unexpected/synthetic
scenarios.

For UTs replace a dummy unifiedMemoryManager pointer with a pointer to
an instace of SVMAllocsManager, otherwise a SegFault error is thrown at
the end of tests.

Perform overall cleanup in related tests implementation, includes but
not limited to removal of:

- givenInitialPlacementGpu\
WhenMovingToGpuDomainThenFirstAccessDoesNotInvokeTransfer

As it is fully covered by:

givenAllocationMovedToGpuDomain\
WhenVerifyingPagefaultThenAllocationIsMovedToCpuDomain

- givenInitialPlacementGpu\
WhenVerifyingPagefaultThenFirstAccessDoesNotInvokeTransfer

As it is fully covered by:

givenTbxAndnitialPlacementGpu\
WhenVerifyingPagefaultThenMemoryIsUnprotectedOnly

Finally, reduce code duplication where possible.

Related-To: NEO-6658
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2022-03-16 11:18:24 +01:00
Milczarek, Slawomir
c0b7f05897 Add memory prefetch for kmd migrated shared allocations
This feature is disabled by default, controlled with the knob
AppendMemoryPrefetchForKmdMigratedSharedAllocations

Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-09 16:02:18 +01:00
Jaime Arteaga
a010fb3634 Dont close shared handle on imported allocations
Related-To: LOCI-2272

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-06 04:35:47 +01:00
Lukasz Jobczyk
090bfb9642 Reuse kernel allocation
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-28 19:26:19 +01:00
Aravind Gopalakrishnan
e4a93f9870 Set page table cloning on for event device allocs
Related-To: NEO-5968

Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@intel.com>
2022-02-24 22:35:03 +01:00
Lukasz Jobczyk
0634aa3f1b Create resource with given address
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-24 17:06:19 +01:00
Lukasz Jobczyk
c405fb5c2c Use shared mutex for unified memory manager
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-23 19:00:38 +01:00
Bartosz Dunajski
71f48c89c8 CacheSettingsHelper - timestamp allocations
Related-To: NEO-6664

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 19:32:39 +01:00
Bartosz Dunajski
61ca84e94b Pass memory placement info to StorageInfo
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 16:55:32 +01:00
Bartosz Dunajski
a95198521e Initial implementation of CacheSettingsHelper
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 16:18:06 +01:00
Bartosz Dunajski
c88fce0def Gmm construction cleanup
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 10:20:24 +01:00
Bartosz Dunajski
4b0d986876 Move AllocationType enum out of GraphicsAllocation class
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-04 17:49:09 +01:00
Katarzyna Cencelewska
45d23868de Remove device enqueue part 15
- remove DEVICE_QUEUE_BUFFER

Related-To: NEO-6559
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
2022-02-03 16:19:07 +01:00
Milczarek, Slawomir
2be98a1e62 Create kmd migrated allocation with initial placement
Implements ZE_HOST_MEM_ALLOC_FLAG_BIAS_INITIAL_PLACEMENT
for zeMemAllocShared with KMD migrated allocation.

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-02-01 15:42:10 +01:00
Spruit, Neil R
ae77bd1bd2 Enable Device Memory to be shared in WSL-2 with L0
- Add getMemoryManagerType to check which memory manager has been init
to determine if Linux + WDDM memory manager is in use.
- Add isNTHandle to test and verify if a handle is an NT handle during
L0 Open IPC Handle.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-01-27 23:14:29 +01:00
Dominik Dabek
a5067d6b0b Remove duplicate check in residency container
Remove find in SVMAllocsManager
addInternalAllocationsToResidencyContainer,
not needed, CSR resolves duplicates at makeResident calls

Co-authored-by: Michal Mrozek <michal.mrozek@intel.com>

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-01-26 17:09:31 +01:00
Michal Mrozek
592eb4607e [4/n] Improve indirect allocations handling.
Add memory manager interfaces to obtain the status.
Only support when VmBind is supported.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-26 15:24:32 +01:00
Jaime Arteaga
dbf0f90186 Return pageSize in getMemAllocProperties
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-01-25 18:26:13 +01:00
Michal Mrozek
6df17f5a30 [3/n] Optimize indirect allocations handling.
Add new debug variable to trigger new mode.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-25 16:40:56 +01:00