Commit Graph

1339 Commits

Author SHA1 Message Date
Aravind Gopalakrishnan
1883161e1e fix: Add debug key to Force Tlb flush
Related-To: GSD-4457

Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@intel.com>
2023-05-01 17:52:22 +02:00
Diedrich, Kamil
5149d74141 refactor: Remove globaly enabled cl_cache
Current behaviour will be detecd path existence

Related-To: NEO-4262

Signed-off-by: Diedrich, Kamil <kamil.diedrich@intel.com>
2023-04-28 23:28:49 +02:00
Fabian Zwolinski
cbce863dc2 refactor: Rename member variables to camelCase 3/n
Additionally enable clang-tidy check for member variables

Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-28 16:01:14 +02:00
Mateusz Jablonski
fd1ad7c1f0 feature: setup heap extended host size based on system memory size
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-28 15:23:01 +02:00
Mateusz Jablonski
5a5c20f99c fix: create separate heap for host and shared usm in 48-56b VA
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-28 10:09:38 +02:00
Kamil Kopryk
171e97e71a feature: introduce release helper
Related-To: NEO-7786
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-04-27 21:38:08 +02:00
Fabian Zwolinski
e351a90f81 refactor: Rename member variables to camelCase 2/n
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-27 20:39:22 +02:00
Mateusz Jablonski
2f9135a4e6 fix: change type of container with registered engines per root device
use StackVec instead of unordered map
resize container at MemoryManager's creation time

Related-To: NEO-7925
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-27 17:06:42 +02:00
Milczarek, Slawomir
3ead7f6ac9 refactor: Simplify prefetch memory logic for kmd-migrated shared allocation
Related-To: NEO-7641

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-27 12:14:16 +02:00
Mateusz Jablonski
32d8a3bc6d fix: store registered engines per root device
in most cases we need to iterate over engines associated to single root device

Related-To: NEO-7925
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-27 10:54:07 +02:00
Spruit, Neil R
364c2da9fb feature: Add Support for zeMemPutIpcHandle & zeMemGet IPC Handle converters
Related-To: LOCI-4172, LOCI-4305, LOCI-4306

- Create a new IPC Memory handle upon call to getIpcMemHandle if the
previous handle has been freed.
- Release the Ipc Memory Handle when zeMemPutIpcHandle is called.
- Create a new IPC Handle for tracking thru zeMemGetAllocProperties
when ze_external_memory_export_fd_t is used.
- Convert FD to opaque IPC handle and IPC Handle to FD.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-04-27 03:53:52 +02:00
Fabian Zwolinski
c441e9e971 refactor: Rename member variables to camelCase
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-26 16:05:07 +02:00
Compute-Runtime-Validation
79800d235c Revert "feature: enable checking indirect access"
This reverts commit 6ec044563f.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-04-26 04:28:13 +02:00
Diedrich, Kamil
26ca64bb28 Add process safety to cl_cache on Linux
Current flow will be to have one synchronization point
config.file. Read remains unblocking, only write(caching)
operation will be blocking (lock on config.file)

Related-To: NEO-4262

Signed-off-by: Diedrich, Kamil <kamil.diedrich@intel.com>
2023-04-25 17:35:40 +02:00
Milczarek, Slawomir
c9fdeb200c [perf] Prefetch of kmd-migrated shared allocation with initial placement on GPU
Related-To: NEO-7646

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-25 17:27:33 +02:00
Mateusz Jablonski
06bd405e88 feature: add debug flag to control usage of heap extended for USM Host
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-25 15:39:49 +02:00
Fabian Zwolinski
2022592f3d Apply CamelCase for class and struct names 2/2
Additionally change .clang-tidy not to ignore struct names.

Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-25 13:10:23 +02:00
Dunajski, Bartosz
6e9257c623 Debug flag to force early exit
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
2023-04-25 09:44:44 +02:00
Krzysztof Gibala
16db7cc890 fix: Add missing checks in multi gpu scenario
- Check allocation root device index during eviction
- Wait for and marked allocation only from the current root device index

Related-To: NEO-7920
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2023-04-24 23:26:28 +02:00
Dominik Dabek
6ec044563f feature: enable checking indirect access
Enable for zebin format but not CM kernels.
Use heuristic of simdSize == 1 to detect CM kernels.

Related-To: NEO-7712

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-04-24 14:57:50 +02:00
Milczarek, Slawomir
bf778be99e [fix] zeCommandListAppendMemAdvise to clear preferred location correctly
The memadvise with preferred location for kmd-migrated shared allocation
is set to device associated with cmd list by default to migrate data
to lmem on non-atomic gpu page fault as well (for performance reasons).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-24 14:51:49 +02:00
Maciej Bielski
09b111cc97 refactor: improve code reuse and readability
Extract distinct steps as dedicated functions, especially when the code
is duplicated. This eases analysis of the logic and highlights
differences between callers of a common code.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-04-24 12:30:01 +02:00
Lukasz Jobczyk
853a65aae9 Add PCI barrier implementation
Resolves: NEO-7850

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-04-24 10:33:03 +02:00
Kacper Nowak
83e9a148ca Fix: Unify fp16/fp32/fp64 flags across all platforms
Unify fp16/fp32/fp64 across all platforms. The capabilities indicated by
those flags now refer to both emulated and native-supported (HW) ones:

- Global/local atomic load: HW support on all platforms (handled by native
i16 atomic or) for FP16, FP32 and FP64.
- Global/local atomic store: HW support on all platforms (handled by
native i16 atomic exchange) for FP16, FP32 and FP64.
- Global/local atomic compare/exchange: HW support on all platforms
for FP32.
- Global/local atomic min/max: Emulation support on all platforms for
FP64, HW support on all platforms for FP32, HW support on XE+ platforms
and emulation support on all others for FP16.
- Global atomic add: HW support for PVC+ platforms, emulation support on
all other platforms for FP64, HW support on XE+ platforms and emulation
support on all other platforms for FP32.
- Local atomic add: Emulation on all platforms for both FP64 and FP32.

Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
Related-To: NEO-7734
2023-04-20 15:03:39 +02:00
Cencelewska, Katarzyna
659109adc6 fix: do not add link engines on integrated gpu
- caps check is not needed when link engines are not available for
product

Related-To: NEO-7886
Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
2023-04-20 13:28:20 +02:00
Daria Hinz
2647efc23c feat: Add support for ip version value in ocloc
Ocloc supports passing hw ip version value to -device arg in
the form of major.minor.revision.
This change adds support for directly passed value as uint32_t as well.
Support added for single and fat binary.

Signed-off-by: Daria Hinz <daria.hinz@intel.com>
Related-To: NEO-7903
2023-04-20 13:20:11 +02:00
Maciej Plewka
bab299ee78 Increment fenceValue only after successful bind operation
Related-To: NEO-7835

Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2023-04-19 12:26:45 +02:00
Kulkarni, Ashwin Kumar
3213f96109 Disable context creation and HWS when NEO_L0_SYSMAN_NO_CONTEXT_MODE is set
Related-To: LOCI-4295

Signed-off-by: Kulkarni, Ashwin Kumar <ashwin.kumar.kulkarni@intel.com>
2023-04-14 11:51:53 +02:00
Fabian Zwolinski
3fe0272381 Do not force Cooperative Context in Offline Debugging Mode
Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-13 22:52:01 +02:00
Milczarek, Slawomir
0ff46562db [perf] Improve placement of KMD shared allocation in scenario with subdevice 1
Improves performance for benchmarks with KMD-migrated shared allocation
in scenarios with ZE_AFFINITY_MASK=0.1.

Related-To: NEO-7881

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-13 17:55:50 +02:00
Mateusz Jablonski
e4a446df58 feature usm: add debug flag to allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-13 11:30:09 +02:00
Daria Hinz
c3f4074f0a fix: Unification of aot config with hw ip version
In the case of mtl+ platforms, the returned config value
should equal the hardware ip version value.
This change fixes situations where some config has not been
added and in this case we returned an unknown value.

Signed-off-by: Daria Hinz <daria.hinz@intel.com>
Related-To: NEO-7738
2023-04-12 18:34:03 +02:00
Compute-Runtime-Validation
a10c4f7920 Revert "feat: enable checking indirect access"
This reverts commit 392e6ab0a7.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-04-12 06:46:02 +02:00
Dominik Dabek
392e6ab0a7 feat: enable checking indirect access
If indirect accesses in kernel are not detected by IGC, indirect
allocations will not be made resident for this kernel.

Related-To: NEO-7712

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-04-11 17:45:24 +02:00
Milczarek, Slawomir
8e04a7a83f Access counters mode to not rely on KMD cross-tile migrations (by default)
Add new regkey KMDSupportForCrossTileMigrationPolicy
(disabled by default, in absence of KMD suppport for cross-tile migrations)
to control placement of shared allocation and memory prefetch behavior.

Related-To: NEO-7885

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-11 15:56:14 +02:00
Fabian Zwolinski
a1066177a5 Attach UUID to SbaAllocations
Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-11 12:53:03 +02:00
John Falkowski
007d39cbf7 move two functions to cpp file
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-11 09:50:50 +02:00
John Falkowski
e056082710 refactor graphics allocation structure elements for sub-allocation properties
Resolves:  LOCI-3772

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-07 16:53:23 +02:00
Fabian Zwolinski
fd971f7a61 Generate Offline Dump Context ID
On linux OfflineDumpContextId consists of
32b processId in bits 63:32
32b drmContestId in bits 31:0
Also cache linux implementation of getProcessId since
the value is constant.

Related-To: NEO-7630
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-04 15:58:56 +02:00
Milczarek, Slawomir
b839f26ed7 Call VM prefetch for KMD-migrated shared allocation on all VMs
Call VM prefetch ioctl on all VMs for the KMD to apply
a synchronoues bind operation of buffer objects on all VMs.

Related-To: NEO-7841

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-04 13:01:44 +02:00
Bellekallu Rajkiran
310380119a feat(sysman): Support for registering events during listen
Related-To: LOCI-3682

Signed-off-by: Bellekallu Rajkiran <bellekallu.rajkiran@intel.com>
2023-04-04 07:07:19 +02:00
Compute-Runtime-Validation
9608879372 Revert "feature: pvc, cpu copy in program init"
This reverts commit 77d70fd4a7.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-31 13:56:39 +02:00
Dominik Dabek
77d70fd4a7 feature: pvc, cpu copy in program init
Use cpu copy for globals surface when allocated through svm, allocation
not set as lockable but locking allocation succeeds.
Make sure gfx allocations is unlocked after copy.

Related-To: NEO-7796

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-03-30 18:05:24 +02:00
Milczarek, Slawomir
4e6995bc4c Set VM advise with preferred location to device by default
Apply the KMD advise with preferred device location for KMD-migrated
shared allocation to migrate to lmem on every GPU page fault
(default KMD migration policy).

Related-To: NEO-7851

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-03-30 17:04:23 +02:00
Compute-Runtime-Validation
b11a64718a Revert "feature usm: allocate shared USM in heap extended"
This reverts commit 03ed1e1e12.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 11:39:59 +02:00
Compute-Runtime-Validation
6bb48ff809 Revert "feature: pvc, cpu copy in program init"
This reverts commit 4c891e80a5.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 07:51:25 +02:00
Mateusz Jablonski
03ed1e1e12 feature usm: allocate shared USM in heap extended
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 16:04:05 +02:00
Mateusz Jablonski
7a03c78b0d feature usm: allocate host USM in HEAP_EXTENDED
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-29 10:09:00 +02:00
Dominik Dabek
4c891e80a5 feature: pvc, cpu copy in program init
Use cpu copy for globals surface when allocated through svm, allocation
not set as lockable but locking allocation succeeds.

Related-To: NEO-7796

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-03-28 23:23:40 +02:00
Mateusz Hoppe
d8f99161dd fix: create VMs with correct flags when perContextVms used
Related-To: NEO-7813

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-28 13:09:46 +02:00