Commit Graph

261 Commits

Author SHA1 Message Date
Milczarek, Slawomir c9fdeb200c [perf] Prefetch of kmd-migrated shared allocation with initial placement on GPU
Related-To: NEO-7646

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-25 17:27:33 +02:00
Mateusz Jablonski 06bd405e88 feature: add debug flag to control usage of heap extended for USM Host
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-25 15:39:49 +02:00
Milczarek, Slawomir bf778be99e [fix] zeCommandListAppendMemAdvise to clear preferred location correctly
The memadvise with preferred location for kmd-migrated shared allocation
is set to device associated with cmd list by default to migrate data
to lmem on non-atomic gpu page fault as well (for performance reasons).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-24 14:51:49 +02:00
Maciej Bielski 09b111cc97 refactor: improve code reuse and readability
Extract distinct steps as dedicated functions, especially when the code
is duplicated. This eases analysis of the logic and highlights
differences between callers of a common code.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-04-24 12:30:01 +02:00
Milczarek, Slawomir 0ff46562db [perf] Improve placement of KMD shared allocation in scenario with subdevice 1
Improves performance for benchmarks with KMD-migrated shared allocation
in scenarios with ZE_AFFINITY_MASK=0.1.

Related-To: NEO-7881

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-13 17:55:50 +02:00
Mateusz Jablonski e4a446df58 feature usm: add debug flag to allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-13 11:30:09 +02:00
Milczarek, Slawomir 8e04a7a83f Access counters mode to not rely on KMD cross-tile migrations (by default)
Add new regkey KMDSupportForCrossTileMigrationPolicy
(disabled by default, in absence of KMD suppport for cross-tile migrations)
to control placement of shared allocation and memory prefetch behavior.

Related-To: NEO-7885

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-11 15:56:14 +02:00
Fabian Zwolinski a1066177a5 Attach UUID to SbaAllocations
Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-11 12:53:03 +02:00
John Falkowski e056082710 refactor graphics allocation structure elements for sub-allocation properties
Resolves:  LOCI-3772

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-07 16:53:23 +02:00
Milczarek, Slawomir 4e6995bc4c Set VM advise with preferred location to device by default
Apply the KMD advise with preferred device location for KMD-migrated
shared allocation to migrate to lmem on every GPU page fault
(default KMD migration policy).

Related-To: NEO-7851

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-30 17:04:23 +02:00
Compute-Runtime-Validation b11a64718a Revert "feature usm: allocate shared USM in heap extended"
This reverts commit 03ed1e1e12.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 11:39:59 +02:00
Mateusz Jablonski 03ed1e1e12 feature usm: allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 16:04:05 +02:00
Mateusz Jablonski 7a03c78b0d feature usm: allocate host USM in HEAP_EXTENDED
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 10:09:00 +02:00
Milczarek, Slawomir 5936734550 Add regkey to set preferred location for kmd-migrated shared allocation
The regkey SetVmAdvisePreferredLocation sets the KMD VM advise
with preferred location for KMD-migrated shared allocation
(default - none, 1 - system, 2 - device memory).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-27 12:48:46 +02:00
Spruit, Neil R 9a056318a4 fix_l0:Unbind immediately during unmap
- Explicitly force unbind of Buffer Objects during unmap to ensure that
Buffer Objects can be reused in the same application.

Related-To: LOCI-4162

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-23 18:47:48 +01:00
Maciej Bielski 3ec0a637ba fix(l0): return API error on ISA allocation OOM
It is possible that a module has so many kernels that the 4GB limit of
GPU VA is depleted when each kernel allocates a 64 KB page for its own
ISA. In such case, propagate the ZE_RESULT_ERROR_OUT_OF_DEVICE_MEMORY to
the API caller to indicate the actual problem.

Currently such scenario is not detected, the execution advances a bit
further and the following crashes do not let the user to easily
understand what happened.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-03-23 17:30:15 +01:00
Compute-Runtime-Validation 7785a27f2f Revert "feature usm: allocate host / shared USM in HEAP_EXTENDED"
This reverts commit 2d0c61aa33.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-22 02:36:23 +01:00
Mateusz Jablonski 2d0c61aa33 feature usm: allocate host / shared USM in HEAP_EXTENDED
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-21 15:31:33 +01:00
Igor Venevtsev 062abef8ef Revert "Deferred deletion of allocations in main thread"
This reverts commit 2ea734491a.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2023-03-20 18:17:15 +01:00
John Falkowski a1e2eca9e8 Add zeMemGetAllocProperties extension for sub-allocations
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-03-17 21:21:44 +01:00
Milczarek, Slawomir eff7439eca [perf] Set local memory pool for kmd-migrated shared allocation
Select ccs engine for usm device and shared allocations
(i.e. for transfers from local to local).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-15 16:02:37 +01:00
Spruit, Neil R 86e739e9dc feature: Relaxed Virtual Memory reservation with pStart
Related-To: LOCI-3871

- Relaxed the Virtual Memory Reservation to allow pStart and not fail if
the pStart value is not obtained.
- Moves checks on pStart to the user to check and determine if they want
to re-reserve or use the address allocated.
- Changed reserveGpuAddress to use unit64_t type to allow internal
address range structure assignment without cast.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-08 08:06:50 +01:00
Milczarek, Slawomir 674691d170 Fix check for page faults enabled in get memory access properties
Related-To: NEO-6465

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-02-28 10:13:25 +01:00
Lukasz Jobczyk 2f5be7a48d Copy command buffer into ring buffer
Resolves: NEO-7422

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-02-22 16:37:34 +01:00
Neil Spruit 2aaebddb37 Revert "Heap allocation allow base address and growable addresses"
This reverts commit 44ec497b1a.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 18:34:40 +01:00
Spruit, Neil R 44ec497b1a feature: Allow for Allocating a base address in the heap and grow an allocation
Related-To: LOCI-3871

- Enabled allocation of specified base address in the targeted heap.
- Enabled virtual memory reservations to grow by allocating at the start
of the heap vs the end of the heap.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 08:13:38 +01:00
Wrobel, Patryk 4c58eda90d Ensure that BO handle is closed only once
When one process had exported and then opened IPC handle
of memory, then close function was called twice for the
same BO handle. It caused debugBreak() and aborted
an application.

This change allows multiple separate BOs to share one
handle. The last shared handle owner calls close() function.

Related-To: NEO-7200
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-13 14:42:49 +01:00
Kamil Kopryk 2484c7ceb2 refactor: rename hw_helper files to gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-01 19:37:51 +01:00
Kamil Kopryk a72cae15fe refactor: don't use global ProductHelper getter 9/n
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-25 11:04:40 +01:00
Warchulski, Jaroslaw bc13db734d Cleanup includes 41
Cleaned up files:
shared/source/command_stream/aub_command_stream_receiver_hw.h
shared/source/helpers/common_types.h
shared/source/os_interface/linux/drm_neo.h
shared/source/os_interface/windows/hw_device_id.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-24 14:37:36 +01:00
Warchulski, Jaroslaw 11764dd9bf Cleanup includes 40
Cleaned up files:
shared/source/os_interface/linux/drm_neo.h
shared/source/os_interface/windows/wddm/um_km_data_translator.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-23 16:19:35 +01:00
Spruit, Neil R f928d695e7 Support for Mapping Physical with Virtual Memory
Related-To: LOCI-3422, LOCI-3421

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-01-20 05:17:10 +01:00
Filip Hazubski 2ed2ac73d1 refactor: correct ioctl and related functions to return signed int
Change IoctlHelper::ioctl function to return int.
Change IoctlHelper::createGemExt function to return int.
Change IoctlHelper::queryDistances function to return int.
Change UuidRegisterResult::retVal type to int32_t.
Rename res to ret in debug_session.cpp for consistency.

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2023-01-19 22:04:27 +01:00
Lukasz Jobczyk 3bf4291558 Align mmap cpu va to 2MB for sizes greater than 2MB
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-01-17 18:26:49 +01:00
Jaime Arteaga 84dec57f0a fix: Add debug message for BO from shared-handle
Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-01-13 17:19:49 +01:00
Lukasz Jobczyk dff2c2d8e3 Align external host ptr allocation gpu va to 2MB
Related-To: NEO-7116

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-01-13 12:37:10 +01:00
HeFan2017 2ea734491a Deferred deletion of allocations in main thread
Add a clearQueueTillFirstFailure interface to DeferredDeleter, which
iterates the queue from the front and delete the allocations in the
queue till a failure. It is called by defer deletion of allocations
occupied by mutliple contexts to unlock the execution in main thread

Related-To: NEO-7532

Signed-off-by: HeFan2017 <fan.f.he@intel.com>
2023-01-13 09:20:35 +01:00
Spruit, Neil R d1ee840ba5 Enforce 64KB alignment when mmaping GPU_TIMESTAMP_DEVICE_BUFFER
Related-To: LOCI-3866

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-01-12 22:52:25 +01:00
Compute-Runtime-Validation ddb7eb07b0 Revert "Align external host ptr allocation gpu va to 2MB"
This reverts commit d24f403cc4.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-01-12 09:33:16 +01:00
Lukasz Jobczyk d24f403cc4 Align external host ptr allocation gpu va to 2MB
Related-To: NEO-7116

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-01-11 13:45:06 +01:00
Lukasz Jobczyk 39df92a16f Align only CPU VA for SVM_CPU
Resolves: NEO-7608

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-01-10 16:52:29 +01:00
Compute-Runtime-Validation af81c79076 Revert "Deferred deletion of allocations in main thread"
This reverts commit c268e30189.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-01-04 18:33:39 +01:00
HeFan2017 c268e30189 Deferred deletion of allocations in main thread
Add a clearQueueTillFirstFailure interface to DeferredDeleter, which
iterates the queue from the front and delete the allocations in the
queue till a failure. It is called by defer deletion of allocations
occupied by mutliple contexts to unlock the execution in main thread

Related-To: NEO-7532

Signed-off-by: HeFan2017 <fan.f.he@intel.com>
2023-01-03 13:51:15 +01:00
Warchulski, Jaroslaw 0eac749fcc Cleanup includes 19
Cleaned up files:
opencl/source/api/cl_types.h
shared/source/compiler_interface/external_functions.h
shared/source/compiler_interface/linker.h
shared/source/device_binary_format/elf/elf.h
shared/source/helpers/preamble.h
shared/source/memory_manager/definitions/storage_info.h
shared/source/memory_manager/memory_manager.h
shared/source/memory_manager/os_agnostic_memory_manager.h
shared/source/program/program_info.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-03 11:02:05 +01:00
Warchulski, Jaroslaw c3a86232d9 Cleanup includes 15
Cleaned up files:
shared/source/command_stream/command_stream_receiver.h
shared/source/command_stream/submissions_aggregator.h
shared/source/helpers/flat_batch_buffer_helper.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2022-12-23 12:36:51 +01:00
Neil R Spruit a829ce8941 Fix ReserveGPUAddress to reserve on specific RootDevice indices
Related-To: LOCI-3790

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2022-12-20 23:08:02 +01:00
Warchulski, Jaroslaw 803bbb89d2 Cleanup includes 13
Cleaned up files:
shared/source/helpers/blit_commands_helper.h
shared/source/helpers/heap_assigner.h
shared/source/memory_manager/alignment_selector.h
shared/source/memory_manager/gfx_partition.h
shared/source/memory_manager/memory_manager.h
shared/source/os_interface/os_memory.h
shared/source/utilities/heap_allocator.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2022-12-16 12:36:27 +01:00
Mateusz Jablonski ecea487cf0 Reduce usage of global gfx core helper getter [2/n]
Related-To: NEO-6853
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-12-12 13:43:21 +01:00
Kamil Kopryk 03b687881f Rename HwHelper -> GfxCoreHelper
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-12-09 10:29:06 +01:00
Jaime Arteaga 675272a32c Return a unique pointer for multiple calls to openIpcHandles
This to make zexMemOpenIpcHandles and zeMemOpenIpcHandle
behave similarly.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-12-08 21:56:21 +01:00