Commit Graph

302 Commits

Author SHA1 Message Date
John Falkowski 789587bacc Return nullptr for gfx alloc mmap error
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2022-04-12 10:08:19 +02:00
Dominik Dabek 29e4518407 Update formatting in dynamic memory tracking
Format as follows:
backtrace start
call_1
...
call_n
backtrace end

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-08 13:17:30 +02:00
Lukasz Jobczyk fffcf2612e Flush tag only if needed in deferred deleter
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-04-07 15:36:29 +02:00
Compute-Runtime-Validation 7a3976ad64 Revert "Force 64KB page size for cpu alignment in dual storage allocation"
This reverts commit 7ff6a5c1fa.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-04-07 00:45:07 +02:00
Kamil Diedrich 7ff6a5c1fa Force 64KB page size for cpu alignment in dual storage allocation
There is no need to force 2MB alignment for CPU allocation in dual
storage usage. Additionaly for WSL this will allow to avoid usage of
malloc in driver path.

Relates-To: NEO-6620
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2022-04-06 15:28:17 +02:00
Dominik Dabek 8d5c674110 Dynamic memory tracking, update function printing
Print only demangled name if it succeeded.

Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-06 14:34:24 +02:00
Dominik Dabek f7a767c767 Add demangling to dynamic memory tracking
Related-To: NEO-6837

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-05 20:38:27 +02:00
Filip Hazubski 88193cc242 Minor fixes
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2022-03-31 20:02:11 +02:00
Michal Mrozek 0c0603966b Add debug functionality to track dynamic allocations.
- available only via manual build with ENABLE_DYNAMIC_MEMORY_TRACKING.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-03-31 13:24:15 +02:00
Compute-Runtime-Validation 91cfd3cd1a Revert "Unify command/ring/semaphore buffers placement"
This reverts commit e035199de4.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-03-30 14:05:47 +02:00
Mateusz Jablonski e035199de4 Unify command/ring/semaphore buffers placement
put them all to the same memory location

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-29 17:55:48 +02:00
Lukasz Jobczyk a230f267e1 Poll task count indefinitely on high throttle command queue
Resolves: NEO-6781

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-03-25 10:06:16 +01:00
Mateusz Jablonski 8a8b4866cb XeHPC: force local memory for command/ring/semaphore buffer
require 48bit resource for ring/semaphore buffer
for multi tile allocations select first tile
for single tile allocation select preferred tile

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-24 10:54:26 +01:00
Milczarek, Slawomir f03f530327 Extend zeCommandListAppendMemoryPrefetch to migrate to associated device
Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-23 14:21:17 +01:00
Mateusz Jablonski 3792481d33 XeHPC Implicit scaling: put command/ring/semaphore buffer to first memory bank
In direct submission scenario command/ring/semaphore buffer allocations
are placed in the same memory bank to ensure that their memory is updated in
correct order

Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-21 08:10:50 +01:00
Daniel Chabrowski 754efbdaff Cleanup mocks
Signed-off-by: Daniel Chabrowski daniel.chabrowski@intel.com
Related-To: NEO-6591
2022-03-18 11:54:03 +01:00
Jaime Arteaga c0e2251ceb Skip adding allocations to remote devices if not allocated there
When making graphics allocations resident in multi-GPU scenarios,
we should make them resident only if there's an allocation for that
device. So return appropriate null pointer and skip it.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-16 23:58:07 +01:00
Maciej Bielski a71b88fefb PageFaultHandler: speedup UM allocations lookup
Add a per-instance SVMAllocsManager::nonGpuDomainAllocs container for
all allocations to be removed in
moveAllocationsWithinUMAllocsManagerToGpuDomain. This approach replaces
the current iterative search and performs the task faster.

Add 7 new unit-tests to verify the functionality related to
nonGpuDomainAllocs container, both in expected and unexpected/synthetic
scenarios.

For UTs replace a dummy unifiedMemoryManager pointer with a pointer to
an instace of SVMAllocsManager, otherwise a SegFault error is thrown at
the end of tests.

Perform overall cleanup in related tests implementation, includes but
not limited to removal of:

- givenInitialPlacementGpu\
WhenMovingToGpuDomainThenFirstAccessDoesNotInvokeTransfer

As it is fully covered by:

givenAllocationMovedToGpuDomain\
WhenVerifyingPagefaultThenAllocationIsMovedToCpuDomain

- givenInitialPlacementGpu\
WhenVerifyingPagefaultThenFirstAccessDoesNotInvokeTransfer

As it is fully covered by:

givenTbxAndnitialPlacementGpu\
WhenVerifyingPagefaultThenMemoryIsUnprotectedOnly

Finally, reduce code duplication where possible.

Related-To: NEO-6658
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2022-03-16 11:18:24 +01:00
Milczarek, Slawomir c0b7f05897 Add memory prefetch for kmd migrated shared allocations
This feature is disabled by default, controlled with the knob
AppendMemoryPrefetchForKmdMigratedSharedAllocations

Related-To: NEO-6740

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-09 16:02:18 +01:00
Jaime Arteaga a010fb3634 Dont close shared handle on imported allocations
Related-To: LOCI-2272

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-06 04:35:47 +01:00
Lukasz Jobczyk 090bfb9642 Reuse kernel allocation
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-28 19:26:19 +01:00
Aravind Gopalakrishnan e4a93f9870 Set page table cloning on for event device allocs
Related-To: NEO-5968

Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@intel.com>
2022-02-24 22:35:03 +01:00
Lukasz Jobczyk 0634aa3f1b Create resource with given address
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-24 17:06:19 +01:00
Lukasz Jobczyk c405fb5c2c Use shared mutex for unified memory manager
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-23 19:00:38 +01:00
Bartosz Dunajski 71f48c89c8 CacheSettingsHelper - timestamp allocations
Related-To: NEO-6664

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 19:32:39 +01:00
Bartosz Dunajski 61ca84e94b Pass memory placement info to StorageInfo
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 16:55:32 +01:00
Bartosz Dunajski a95198521e Initial implementation of CacheSettingsHelper
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 16:18:06 +01:00
Bartosz Dunajski c88fce0def Gmm construction cleanup
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 10:20:24 +01:00
Bartosz Dunajski 4b0d986876 Move AllocationType enum out of GraphicsAllocation class
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-04 17:49:09 +01:00
Katarzyna Cencelewska 45d23868de Remove device enqueue part 15
- remove DEVICE_QUEUE_BUFFER

Related-To: NEO-6559
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
2022-02-03 16:19:07 +01:00
Milczarek, Slawomir 2be98a1e62 Create kmd migrated allocation with initial placement
Implements ZE_HOST_MEM_ALLOC_FLAG_BIAS_INITIAL_PLACEMENT
for zeMemAllocShared with KMD migrated allocation.

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-02-01 15:42:10 +01:00
Spruit, Neil R ae77bd1bd2 Enable Device Memory to be shared in WSL-2 with L0
- Add getMemoryManagerType to check which memory manager has been init
to determine if Linux + WDDM memory manager is in use.
- Add isNTHandle to test and verify if a handle is an NT handle during
L0 Open IPC Handle.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-01-27 23:14:29 +01:00
Dominik Dabek a5067d6b0b Remove duplicate check in residency container
Remove find in SVMAllocsManager
addInternalAllocationsToResidencyContainer,
not needed, CSR resolves duplicates at makeResident calls

Co-authored-by: Michal Mrozek <michal.mrozek@intel.com>

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-01-26 17:09:31 +01:00
Michal Mrozek 592eb4607e [4/n] Improve indirect allocations handling.
Add memory manager interfaces to obtain the status.
Only support when VmBind is supported.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-26 15:24:32 +01:00
Jaime Arteaga dbf0f90186 Return pageSize in getMemAllocProperties
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-01-25 18:26:13 +01:00
Michal Mrozek 6df17f5a30 [3/n] Optimize indirect allocations handling.
Add new debug variable to trigger new mode.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-25 16:40:56 +01:00
Michal Mrozek 52d636394c [1/n] Improve indirect allocations handling.
Add new functions that would treat all indirect allocations as single pack.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-25 06:27:33 +01:00
Michal Mrozek 151aaf7678 Fix alignment for host allocations.
- it is 4k not 64k.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-24 13:24:03 +01:00
Compute-Runtime-Validation 6082865eb4 Revert "Optimize Level Zero indirect allocations handling."
This reverts commit 3ecbc55ba9.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-01-20 11:41:13 +01:00
Kamil Diedrich 1b7949432f Add shareable allocation on windows dGPUs
Add default initialization for object members

Related-To: LOCI-2665

Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2022-01-19 19:03:18 +01:00
Michal Mrozek 3ecbc55ba9 Optimize Level Zero indirect allocations handling.
Make them resident directly instead of populating residency container
Remove finds, not needed, CSR resolves duplicates at makeResident calls
Observed gain is 32x for 10k indirect allocations.


Co-authored-by: Michal Mrozek <michal.mrozek@intel.com>

Co-authored-by: Dominik Dabek <dominik.dabek@intel.com>

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-01-19 13:08:35 +01:00
Raiyan Latif 394c0e90e1 Return error when failing on submission
Signed-off-by: Raiyan Latif <raiyan.latif@intel.com>
2022-01-12 16:42:30 +01:00
Joshua Santosh Ranjan 7fc9b2c3dc Implement UUID using pcie bdf
This patch implements the fallback method for Device UUID
using the BDF of the device for all product families.


Related-To: LOCI-2827

Signed-off-by: Joshua Santosh Ranjan <joshua.santosh.ranjan@intel.com>
2022-01-05 23:31:00 +01:00
Jaime Arteaga 3b497270c8 Add support for memory free policies
Add support for ZE_DRIVER_MEMORY_FREE_POLICY_EXT_FLAG_BLOCKING_FREE
added in v1.3.

Related-To: LOCI-2672

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-12-28 08:27:50 +01:00
Bartosz Dunajski e040c8c6bb Initial compression support for L0 Device allocations
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-16 15:55:13 +01:00
Bartosz Dunajski 2fd536104d Extend GfxPartition to support 57b addressing
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-08 16:05:05 +01:00
Bartosz Dunajski dfdd3c597a Remove redundant BUFFER_COMPRESSED allocation type
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-07 13:35:49 +01:00
Bartosz Dunajski 0346a5679f Fix returned local memory size
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-06 15:57:55 +01:00
Bartosz Dunajski 68aea5bf62 Rename compression flags and helpers
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-03 18:09:02 +01:00
Bartosz Dunajski 9481d8a88b Compression refactor [2/n]
Dont use allocation type for compression preference

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-03 08:47:46 +01:00