Commit Graph

885 Commits

Author SHA1 Message Date
Diedrich, Kamil
26ca64bb28 Add process safety to cl_cache on Linux
Current flow will be to have one synchronization point
config.file. Read remains unblocking, only write(caching)
operation will be blocking (lock on config.file)

Related-To: NEO-4262

Signed-off-by: Diedrich, Kamil <kamil.diedrich@intel.com>
2023-04-25 17:35:40 +02:00
Milczarek, Slawomir
c9fdeb200c [perf] Prefetch of kmd-migrated shared allocation with initial placement on GPU
Related-To: NEO-7646

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-25 17:27:33 +02:00
Mateusz Jablonski
06bd405e88 feature: add debug flag to control usage of heap extended for USM Host
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-25 15:39:49 +02:00
Fabian Zwolinski
2022592f3d Apply CamelCase for class and struct names 2/2
Additionally change .clang-tidy not to ignore struct names.

Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-25 13:10:23 +02:00
Dunajski, Bartosz
6e9257c623 Debug flag to force early exit
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
2023-04-25 09:44:44 +02:00
Krzysztof Gibala
16db7cc890 fix: Add missing checks in multi gpu scenario
- Check allocation root device index during eviction
- Wait for and marked allocation only from the current root device index

Related-To: NEO-7920
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2023-04-24 23:26:28 +02:00
Milczarek, Slawomir
bf778be99e [fix] zeCommandListAppendMemAdvise to clear preferred location correctly
The memadvise with preferred location for kmd-migrated shared allocation
is set to device associated with cmd list by default to migrate data
to lmem on non-atomic gpu page fault as well (for performance reasons).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-24 14:51:49 +02:00
Maciej Bielski
09b111cc97 refactor: improve code reuse and readability
Extract distinct steps as dedicated functions, especially when the code
is duplicated. This eases analysis of the logic and highlights
differences between callers of a common code.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-04-24 12:30:01 +02:00
Lukasz Jobczyk
853a65aae9 Add PCI barrier implementation
Resolves: NEO-7850

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-04-24 10:33:03 +02:00
Cencelewska, Katarzyna
659109adc6 fix: do not add link engines on integrated gpu
- caps check is not needed when link engines are not available for
product

Related-To: NEO-7886
Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
2023-04-20 13:28:20 +02:00
Maciej Plewka
bab299ee78 Increment fenceValue only after successful bind operation
Related-To: NEO-7835

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2023-04-19 12:26:45 +02:00
Fabian Zwolinski
3fe0272381 Do not force Cooperative Context in Offline Debugging Mode
Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-13 22:52:01 +02:00
Milczarek, Slawomir
0ff46562db [perf] Improve placement of KMD shared allocation in scenario with subdevice 1
Improves performance for benchmarks with KMD-migrated shared allocation
in scenarios with ZE_AFFINITY_MASK=0.1.

Related-To: NEO-7881

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-13 17:55:50 +02:00
Mateusz Jablonski
e4a446df58 feature usm: add debug flag to allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-13 11:30:09 +02:00
Milczarek, Slawomir
8e04a7a83f Access counters mode to not rely on KMD cross-tile migrations (by default)
Add new regkey KMDSupportForCrossTileMigrationPolicy
(disabled by default, in absence of KMD suppport for cross-tile migrations)
to control placement of shared allocation and memory prefetch behavior.

Related-To: NEO-7885

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-11 15:56:14 +02:00
Fabian Zwolinski
a1066177a5 Attach UUID to SbaAllocations
Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-11 12:53:03 +02:00
John Falkowski
007d39cbf7 move two functions to cpp file
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-11 09:50:50 +02:00
John Falkowski
e056082710 refactor graphics allocation structure elements for sub-allocation properties
Resolves:  LOCI-3772

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-07 16:53:23 +02:00
Fabian Zwolinski
fd971f7a61 Generate Offline Dump Context ID
On linux OfflineDumpContextId consists of
32b processId in bits 63:32
32b drmContestId in bits 31:0
Also cache linux implementation of getProcessId since
the value is constant.

Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-04 15:58:56 +02:00
Milczarek, Slawomir
b839f26ed7 Call VM prefetch for KMD-migrated shared allocation on all VMs
Call VM prefetch ioctl on all VMs for the KMD to apply
a synchronoues bind operation of buffer objects on all VMs.

Related-To: NEO-7841

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-04 13:01:44 +02:00
Bellekallu Rajkiran
310380119a feat(sysman): Support for registering events during listen
Related-To: LOCI-3682

Signed-off-by: Bellekallu Rajkiran <bellekallu.rajkiran@intel.com>
2023-04-04 07:07:19 +02:00
Milczarek, Slawomir
4e6995bc4c Set VM advise with preferred location to device by default
Apply the KMD advise with preferred device location for KMD-migrated
shared allocation to migrate to lmem on every GPU page fault
(default KMD migration policy).

Related-To: NEO-7851

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-30 17:04:23 +02:00
Compute-Runtime-Validation
b11a64718a Revert "feature usm: allocate shared USM in heap extended"
This reverts commit 03ed1e1e12.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 11:39:59 +02:00
Mateusz Jablonski
03ed1e1e12 feature usm: allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 16:04:05 +02:00
Mateusz Jablonski
7a03c78b0d feature usm: allocate host USM in HEAP_EXTENDED
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 10:09:00 +02:00
Mateusz Hoppe
d8f99161dd fix: create VMs with correct flags when perContextVms used
Related-To: NEO-7813

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-28 13:09:46 +02:00
Mateusz Jablonski
7e5e27f0b9 feature usm: reserve heap extended in 57 bit address space
use mocked mmap function in unit tests

Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-28 11:06:03 +02:00
Zhenjie Pan
1ce269a9dd fix: wrong return value of zeCommandQueueExecuteCommandLists when OOM
zeCommandQueueExecuteCommandLists return ZE_RESULT_ERROR_UNKNOWN when OOM
in some scenario of direct submission.

Related-To: NEO-7840

Signed-off-by: Pan Zhenjie <zhenjie.pan@intel.com>
2023-03-28 09:52:17 +02:00
Mateusz Hoppe
d344945625 fix: remove print from xe ioctl helper
- driver should not print to stdout

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-27 23:46:18 +02:00
Milczarek, Slawomir
5936734550 Add regkey to set preferred location for kmd-migrated shared allocation
The regkey SetVmAdvisePreferredLocation sets the KMD VM advise
with preferred location for KMD-migrated shared allocation
(default - none, 1 - system, 2 - device memory).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-27 12:48:46 +02:00
Spruit, Neil R
9a056318a4 fix_l0:Unbind immediately during unmap
- Explicitly force unbind of Buffer Objects during unmap to ensure that
Buffer Objects can be reused in the same application.

Related-To: LOCI-4162

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-23 18:47:48 +01:00
Maciej Bielski
3ec0a637ba fix(l0): return API error on ISA allocation OOM
It is possible that a module has so many kernels that the 4GB limit of
GPU VA is depleted when each kernel allocates a 64 KB page for its own
ISA. In such case, propagate the ZE_RESULT_ERROR_OUT_OF_DEVICE_MEMORY to
the API caller to indicate the actual problem.

Currently such scenario is not detected, the execution advances a bit
further and the following crashes do not let the user to easily
understand what happened.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-03-23 17:30:15 +01:00
Francois Dugast
e2a4cd0d48 xe: Add IOCTL support for GemCreate
This is needed to execute some workloads with iGPUs.

Signed-off-by: Francois Dugast <francois.dugast@intel.com>
2023-03-22 12:20:13 +01:00
Compute-Runtime-Validation
7785a27f2f Revert "feature usm: allocate host / shared USM in HEAP_EXTENDED"
This reverts commit 2d0c61aa33.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-22 02:36:23 +01:00
Mateusz Jablonski
415d2cb121 fix xe: add missing wait on fence before unbind
Related-To: NEO-7306
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-21 15:37:44 +01:00
Mateusz Jablonski
2d0c61aa33 feature usm: allocate host / shared USM in HEAP_EXTENDED
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-21 15:31:33 +01:00
Igor Venevtsev
062abef8ef Revert "Deferred deletion of allocations in main thread"
This reverts commit 2ea734491a.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2023-03-20 18:17:15 +01:00
John Falkowski
a1e2eca9e8 Add zeMemGetAllocProperties extension for sub-allocations
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-03-17 21:21:44 +01:00
Fabian Zwolinski
93a30f002b L0 Debugger - check debug_eu entry.
Related-To: NEO-7790
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-03-15 16:14:49 +01:00
Milczarek, Slawomir
eff7439eca [perf] Set local memory pool for kmd-migrated shared allocation
Select ccs engine for usm device and shared allocations
(i.e. for transfers from local to local).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-15 16:02:37 +01:00
Compute-Runtime-Validation
3e1d931296 Revert "L0 Debugger - check debug_eu entry"
This reverts commit 9f935276a0.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-15 12:28:08 +01:00
Fabian Zwolinski
1e4c91fb08 Do not disable scratch pages when dbgr is enabled
Related-To: NEO-7990
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-03-14 15:03:18 +01:00
Fabian Zwolinski
9f935276a0 L0 Debugger - check debug_eu entry
Related-To: NEO-7790
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-03-13 14:46:28 +01:00
Kamil Kopryk
fa8579602f refactor: rename product helper files n/n
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-10 13:24:38 +01:00
Kamil Kopryk
57e68624f3 refactor: rename product helper wddm and drm files
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-09 17:51:01 +01:00
Mateusz Hoppe
37dbec305d feature: add AssertHandler
- initial implementation to support assert() on GPU

Related-To: NEO-5753

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-08 17:55:23 +01:00
Kamil Kopryk
49b01a8ea4 refactor: rename product helper uuid files
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-08 17:25:57 +01:00
Kamil Kopryk
c7311bc140 refactor: rename product helper files for xehpc and later
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-08 16:28:59 +01:00
Mateusz Jablonski
553dd7f21f refactor: return thread per eu from compiler product helper
Related-To: NEO-7442
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-08 16:25:20 +01:00
Spruit, Neil R
86e739e9dc feature: Relaxed Virtual Memory reservation with pStart
Related-To: LOCI-3871

- Relaxed the Virtual Memory Reservation to allow pStart and not fail if
the pStart value is not obtained.
- Moves checks on pStart to the user to check and determine if they want
to re-reserve or use the address allocated.
- Changed reserveGpuAddress to use unit64_t type to allow internal
address range structure assignment without cast.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-08 08:06:50 +01:00