Commit Graph

174 Commits

Author SHA1 Message Date
Eric Mortensen
69f2cda2fa fix: add error checks & return to unMapVirtualMem
Related-To: NEO-8566

Signed-off-by: Eric Mortensen <eric.mortensen@intel.com>
2025-07-17 23:53:19 +02:00
Mateusz Hoppe
cb152ba821 fix: copy sip binary to allocation based on blitRequired query
- readOnly ISA allocations must be written through CPU pointer
- command buffer allocation in singleAddressSpaceSbaTracking mode cannot
be readonly - it is written by SBA tracking commands
- this change is fixing ZET_ENABLE_PROGRAM_DEBUGGING mode 2

Related-To: GSD-10359

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-06-30 19:41:19 +02:00
Chandio, Bibrak Qamar
c1867ed981 feature: atomic attributes for shared system alloc
Related-To: NEO-13715, NEO-14862

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-06-23 20:14:47 +02:00
Bartosz Dunajski
91940d2a79 fix: initial support for single temporary allocations list
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-21 13:16:15 +02:00
Maciej Bielski
0f8ee57f98 feature: add NEO_LOCAL_MEMORY_ALLOCATION_MODE
Allow the application to force storageInfo.localOnly and get the
out-of-memory returned if not possible.

This is a windows-only feature supported on discrete platforms.

Related-To: NEO-13428
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-05-20 14:15:17 +02:00
Chandio, Bibrak Qamar
bed2ad1ac0 feature: memAdise for System Allocator
Related-To: NEO-13715

Implements basic functionality for zeCommandListAppendMemoryAdvise for
system allocator

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-04-16 07:11:14 +02:00
Dominik Dabek
915d657420 fix: flag to limit usm reuse based on memory usage
Host usm and device usm for igfx checks system memory usage.
Device usm for dgfx checks local memory usage.

If used memory is above limit threshold:
- no new allocations will be saved for reuse
- cleaner will use shorter hold time of 2 seconds
- cleaner will free all eligible allocations, regardless of async
deleter thread having work

Motivation: in case of gfx memory being full, making resident new
allocations will require evictions which leads to massive slowdown on
enqueue calls.
This change aims to minimize cases where extra memory usage from usm
reuse mechanism leads to above situation.

Related-To: NEO-6893, NEO-14160

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-27 10:25:19 +01:00
Dominik Dabek
6e998fc3c1 fix: move host usm reuse max size to mem manager
Intialize value on memory manager creation.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-24 08:53:30 +01:00
John Falkowski
4d281cf51d feature: Implement appendMemoryPrefetch for Shared System USM allocations
Related-To: NEO-12989

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-03-13 06:26:38 +01:00
Compute-Runtime-Validation
fa2e3adad3 Revert "feature: Implement appendMemoryPrefetch for Shared System USM Allocat...
This reverts commit 97799b3faf.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-03-12 05:55:32 +01:00
John Falkowski
97799b3faf feature: Implement appendMemoryPrefetch for Shared System USM Allocations
Related-To: NEO-12989

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-03-11 09:12:48 +01:00
Lukasz Jobczyk
9085f6aeca fix: drain gem close worker on csr destroy
Resolves: HSD-18041493278

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-02-13 20:34:13 +01:00
Maciej Bielski
971b7c27a2 fix: enable usm compression on linux
Related-To: NEO-12056
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-02-04 13:09:04 +01:00
Lukasz Jobczyk
2dd9940f60 Revert "fix: count active modules for enabling per-dispatch private memory"
This reverts commit a483b361f9.

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-01-15 15:03:37 +01:00
Wenbin Lu
a483b361f9 fix: count active modules for enabling per-dispatch private memory
Related-To: NEO-13086

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2025-01-10 15:03:34 +01:00
Jack Myers
7f9fadc314 fix: regression caused by tbx fault mngr
Addresses regressions from the reverted merge
of the tbx fault manager for host memory.

Recursive locking of mutex caused deadlock.

To fix, separate tbx fault data from base
cpu fault data, allowing separate mutexes
for each, eliminating recursive locks on
the same mutex.

By separating, we also help ensure that tbx-related
changes don't affect the original cpu fault manager code
paths.

As an added safe guard preventing critical regressions
and avoiding another auto-revert, the tbx fault manager
is hidden behind a new debug flag which is disabled by default.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2025-01-09 07:48:53 +01:00
Compute-Runtime-Validation
124e755b9d Revert "fix: regression caused by tbx fault mngr"
This reverts commit 9a14fe2478.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-12-19 17:35:03 +01:00
Jack Myers
9a14fe2478 fix: regression caused by tbx fault mngr
Addresses regressions from the reverted merge
of the tbx fault manager for host memory.

This fixes attempts by the tbx fault manager
to protect/unprotect host buffer memory, even
if the host ptr was not driver-allocated.

In the case of the smoke test that triggered
the critical regression, clCreateBuffer was
called with the CL_MEM_USE_HOST_PTR flag.
The subsequent `mprotect` calls on the
provided host ptr then failed.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2024-12-18 23:16:36 +01:00
Filip Hazubski
a0cc124b2e performance: Pass RootDeviceIndicesContainer by reference
Additionally pass std::map by reference in UsmMemAllocPoolsManager c-tor.

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2024-12-17 14:18:30 +01:00
Mateusz Hoppe
0589a70dc7 feature(sysman): reinitialize gfxPartition on reset
Related-To: NEO-13203

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-12-12 16:27:25 +01:00
Compute-Runtime-Validation
6c5d9a6ed7 Revert "feature: extend TBX page fault manager from CPU implementation"
This reverts commit 51c0e80299.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-12-12 12:30:22 +01:00
Jack Myers
51c0e80299 feature: extend TBX page fault manager from CPU implementation
In TBX mode, the host could not write to host buffers after access from device
code due to the lack of a migration mechanism post-initial TBX upload.
Migration is unnecessary with real hardware, but required for TBX.

This patch introduces a new page fault manager type that extends the original
CPU fault manager, enabling automatic migration of host buffers in TBX mode.

Refactoring was necessary to avoid diamond inheritance, achieved by using a
template parameter as the base class for OS-specific fault managers.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2024-12-11 09:09:50 +01:00
Fabian Zwoliński
d2ce3badfc fix: bindlessHeapsHelper handle unavailable external heap
This PR handles the situation in which a component
has reserved a front window space for itself in the external heap,
so that the Compute Runtime cannot access this area.

In such a situation, we perform the following steps:
1. reserve 4GB chunk in heapStandard
2. split our chunk into 2 parts: heapFrontWindow, heapRegular
3. from this point on, map all linearStream allocations in reserved 4GB
chunk

Patch applies to Windows and WSL.
Patch only applies when the bindless global allocator is enabled.

Related-To: HSD-16025889919
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-12-05 14:18:01 +01:00
Dominik Dabek
e55aa958b7 fix: track usm reuse usage in multiple contexts
Add tracking of memory used for usm reuse mechanism when multiple cl
contexts are used.
Tracking for device added to NEO::Device, for host added to
NEO::MemoryManager.

This fixes usm reuse using x% of memory per each context instead of
globally.

Related-To: NEO-13308

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-11-26 16:00:45 +01:00
Wenbin Lu
2ba80ce114 feature: support physical host memory
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-11-20 08:19:52 +01:00
Alicja Lukaszewicz
654fdc1345 feature: add query for additional device properties
Related-To: NEO-12590

Signed-off-by: Alicja Lukaszewicz <alicja.lukaszewicz@intel.com>
2024-10-29 20:40:27 +01:00
Wenbin Lu
a8a40d2afd feature: support SVM heap in reserveVirtualMem
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-10-22 16:47:14 +02:00
Bartosz Dunajski
f430279d62 fix: change doorbell type
Related-To: NEO-12870

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-10-14 14:58:11 +02:00
Mateusz Hoppe
5ae2552b4b fix: track shifted contextIds in bitset in bindlessHeapsHelper
- bitset is 64 bit in size, context ids may go beyond that limit
when multiple devices are available
- this change subtracts contextId of first context for a given root
device - tracked state dirty contexts ids are now zero-based

Resolves: GSD-10025

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-10-09 10:32:29 +02:00
Compute-Runtime-Validation
41df1a6f47 Revert "feature: support SVM heap in reserveVirtualMem"
This reverts commit bfaeeb01d6.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-10-03 14:53:50 +02:00
Michal Mrozek
dd631610b3 refactor: move memory tracking to memory manager
- remove wddm specific code
- improve total size reported to be in decimal

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-09-12 17:32:38 +02:00
Wenbin Lu
bfaeeb01d6 feature: support SVM heap in reserveVirtualMem
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-09-09 23:22:04 +02:00
Compute-Runtime-Validation
99f62ac866 Revert "feature: support SVM heap in reserveVirtualMem"
This reverts commit 93cde3ee12.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-09-03 20:20:25 +02:00
Wenbin Lu
93cde3ee12 feature: support SVM heap in reserveVirtualMem
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-09-03 11:38:51 +02:00
Bartosz Dunajski
bca8478bd4 feature: use iotclhelper to get num media engines
Related-To: NEO-9777

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-08-08 09:36:28 +02:00
Dominik Dabek
26428d5af3 feature: track used memory by allocations
Track memory used by memory allocations. System and local per device.
Will be used for heuristics in memory pooling.

Related-To: NEO-11356

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-08-02 17:54:34 +02:00
Szymon Morek
b03ac6abd1 fix: disable usm compression on linux
Related-To: NEO-12047

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-16 14:59:33 +02:00
Szymon Morek
432ecbc8f4 fix: disable compression for exported allocations
Related-To: NEO-12021

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-16 05:41:14 +02:00
Bartosz Dunajski
bb21d25ed0 feature: initial support for creating media context
Related-To: NEO-11386

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-07-03 12:23:31 +02:00
Jaroslaw Warchulski
daec249430 refactor: remove not needed createGraphicsAllocationFromNTHandle method
Related-To: NEO-11498
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2024-06-18 14:25:00 +02:00
Jaroslaw Warchulski
76a05c1cab refactor: pass arrayIndex to Wddm::openNTHandle function
Related-To: NEO-11498
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2024-06-12 12:11:35 +02:00
Jaroslaw Warchulski
0ac1be7669 refactor: pass arrayIndex to Wddm::openSharedHandle function
Related-To: NEO-11498
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2024-06-07 09:51:26 +02:00
Maciej Bielski
ec21260fda refactor: extract StorageInfo::memoryBanks computation
Move the code determining the memoryBanks parameter into separate
virtual function for further memory-manager-specific customization.

Related-To: NEO-9754
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2024-06-05 17:50:55 +02:00
Bartosz Dunajski
64af8dd956 feature: allocating interrupt support
Related-To: NEO-8179

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-06-03 18:35:24 +02:00
Maciej Bielski
dc32da2507 feature: prepare StorageInfo for Drm-specific customization
Get the local memory regions count from the primary source (MemoryInfo)
and store for further use when using DrmMemoryManager.
Add a point of dispatch (virtual `createStorageInfoFromProperties`) for
further Drm-specific customizations related to StorageInfo. As the
function became virtual, move one of its callers
(`isLocalMemoryUsedForIsa()`) from the constructor of `MemoryManager` to
respective constructors of all derivative classes. This prevents
bypassing the virtual call dispatch.

Related-To: NEO-9754
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2024-05-16 12:24:48 +02:00
Szymon Morek
d708c5f300 performance: Allow indirect allocs as pack on windows
Related-To: NEO-11228

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-05-06 15:03:12 +02:00
Mateusz Hoppe
d6675b14b3 fix: release secondaryContexts prior to primary context
Related-To: NEO-10920

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-04-05 16:34:39 +02:00
Maciej Plewka
564e0f0319 performance: Align host mem to 2MB when range is not limited
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>

Related-To: NEO-10217
2024-01-30 14:43:13 +01:00
Mateusz Jablonski
7b40b01f54 feature: add debug key for toggling bit in 57bit GPU VA for specific allocations
Related-To: NEO-9419
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-01-15 19:37:00 +01:00
Mateusz Hoppe
31e9b5e9fa feature: add support for secondary contexts in group
Related-To: NEO-7824

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-12-28 13:31:08 +01:00