Commit Graph

388 Commits

Author SHA1 Message Date
Maciej Plewka
88fb60a515 fix: do not align to 64kb when requested alignment is bigger
Related-To: NEO-14082
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2025-07-22 19:32:29 +02:00
Eric Mortensen
69f2cda2fa fix: add error checks & return to unMapVirtualMem
Related-To: NEO-8566

Signed-off-by: Eric Mortensen <eric.mortensen@intel.com>
2025-07-17 23:53:19 +02:00
Compute-Runtime-Validation
377b99e741 Revert "fix: do not align to 64kb when requested alignment is bigger"
This reverts commit 33749b8b5a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-07-15 14:58:49 +02:00
Maciej Plewka
33749b8b5a fix: do not align to 64kb when requested alignment is bigger
Related-To: NEO-14082
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2025-07-11 19:59:28 +02:00
Filip Hazubski
f8dc80c7fc test: Adjust direct submission tests
Disable direct submission at the end of the test for cases where
direct submission is enabled in test body.

Adjust mock csr to restore os context in cases where
context has been replaced in test body.

Related-To: NEO-15017

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2025-07-10 11:01:33 +02:00
Dominik Dabek
ff662b3e88 fix: usm reuse aub/tbx indirect access
When allocation is saved for reuse, remove it from allocations map.
On get from reuse, reinsert it with new allocId.

Allocations saved for reuse should not be made resident for indirect
access.

Only applicable on aub/tbx csr types.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-10 08:02:03 +02:00
Dominik Dabek
3dcfef5812 fix: usm reuse aub/tbx update alloc id
On getting from reuse update alloc id and increment allocations counter.
This is to ensure that making indirect allocations resident picks up the
reuse allocation.
Only applicable on aub/tbx csr types.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-07 15:49:16 +02:00
Slawomir Milczarek
76090f041d fix: Fixed out-of-bounds write to usageInfos array with residency task count
Related-To: NEO-12952

Signed-off-by: Slawomir Milczarek <slawomir.milczarek@intel.com>
2025-07-03 18:00:37 +02:00
Dominik Dabek
e52fa32271 fix: disable usm reuse if debugger enabled
Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-01 14:22:15 +02:00
Mateusz Hoppe
cb152ba821 fix: copy sip binary to allocation based on blitRequired query
- readOnly ISA allocations must be written through CPU pointer
- command buffer allocation in singleAddressSpaceSbaTracking mode cannot
be readonly - it is written by SBA tracking commands
- this change is fixing ZET_ENABLE_PROGRAM_DEBUGGING mode 2

Related-To: GSD-10359

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-06-30 19:41:19 +02:00
Dominik Dabek
dde8bf1b9b fix: usm reuse blocking free handling
on blocking free wait for engines completion before putting into reuse
mark allocation as completed and do not check is in use status on
getting

Related-To: NEO-6893, HSD-18042686276

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-30 11:46:45 +02:00
Dominik Dabek
4cbd6c79a0 fix: mark aub/tbx writable on get from usm reuse
Related-To: NEO-6893, NEO-15359, HSD-18042681561

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-25 19:36:16 +02:00
Dominik Dabek
5c7fd72d5f performance: enable l0 usm reuse
Do not reuse internal or imported allocations.
Mark as aub writable on getting from reuse.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-25 15:49:15 +02:00
Chandio, Bibrak Qamar
c1867ed981 feature: atomic attributes for shared system alloc
Related-To: NEO-13715, NEO-14862

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-06-23 20:14:47 +02:00
Dominik Dabek
77e25f90d7 fix: prepare for l0 usm reuse enable
adjust ULTs
disable usm reuse in multi device l0
don't initialize usm caches if max reuse size is 0
check requested alignment when returning from cache
guard against double free

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-23 17:17:18 +02:00
Michal Mrozek
f6cb013896 refactor: remove redundant member
- it is true on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-06-18 16:15:30 +02:00
Fabian Zwoliński
e2228201ce fix: Avoid redundant padding in ISA allocations
- Add isaPaddingIncluded flag to AllocationProperties to control ISA
padding
- Modify MemoryManager to skip automatic padding when flag is set
- Update ISAPoolAllocator to manage padding explicitly and set the
flag
- Use actual buffer size for HeapAllocator instead of requested size
- Add test covering different padding scenarios

This prevents double padding addition which was wasting memory when
ISAPoolAllocator managed its own padding on top of MemoryManager's
automatic padding.

Related-To: NEO-12287, HSD-18042276431
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-05-30 15:40:38 +02:00
Dominik Dabek
ad4d7a2ce8 fix: l0 usm pooling, ipc handling
Move getting usm pool for ptr to its own method.
Move trying to free via pooling to its own method.

Use base ptr of pool for tracking in IPCHandleMap.
Track ipc handle refcount in freeMem.
Return error when trying to use a not allocated ptr within pool.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-29 16:07:19 +02:00
Aleksandra Nizio
1dfc9227c4 fix: Removind address_mapper.h
Related-To: NEO-14718
Signed-off-by: Aleksandra Nizio <aleksandra.nizio@intel.com>
2025-05-27 20:37:56 +02:00
Bartosz Dunajski
9c19e9eb96 fix: enable single temporary allocations list mode 2
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-27 17:35:17 +02:00
Compute-Runtime-Validation
42a8b63725 Revert "fix: enable single temporary allocations list mode"
This reverts commit b496274d1a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-26 13:22:15 +02:00
Bartosz Dunajski
b496274d1a fix: enable single temporary allocations list mode
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-22 08:45:46 +02:00
Bartosz Dunajski
91940d2a79 fix: initial support for single temporary allocations list
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-21 13:16:15 +02:00
Maciej Bielski
0f8ee57f98 feature: add NEO_LOCAL_MEMORY_ALLOCATION_MODE
Allow the application to force storageInfo.localOnly and get the
out-of-memory returned if not possible.

This is a windows-only feature supported on discrete platforms.

Related-To: NEO-13428
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-05-20 14:15:17 +02:00
John Falkowski
468c62086e refactor: Gate shared system mem caps with KMD cap
If KMD not capable then set sharedSystemAllocCapabilities=0

Related-To: NEO-12988

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-05-19 11:08:19 +02:00
Compute-Runtime-Validation
0a096f1d50 Revert "fix: propagate error createInternalHandle wddm"
This reverts commit eeddd779f0.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-16 02:41:16 +02:00
Dominik Dabek
eeddd779f0 fix: propagate error createInternalHandle wddm
Unify with linux implementation, return -1 on error and 0 on success.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-15 15:52:37 +02:00
Michal Mrozek
6f4a397cfc refactor: remove not required parameter
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-08 23:13:06 +02:00
Compute-Runtime-Validation
7167f45199 Revert "refactor: Gate shared system mem caps with KMD cap"
This reverts commit 8e59ac7576.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-08 11:26:52 +02:00
Michal Mrozek
afdc04975f refactor: remove ftrSvm variable
- it is true on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-07 16:16:42 +02:00
Michal Mrozek
c3eec70a97 refactor: remove not needed hw info member
- it is false on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-07 14:17:49 +02:00
John Falkowski
8e59ac7576 refactor: Gate shared system mem caps with KMD cap
Enabled only by setting EnableSharedSystemUsmSupport=1 flag

Related-To: NEO-12988

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-05-07 06:38:01 +02:00
Dominik Dabek
6afefa1e31 performance: usm device pool, compression handling
Use device compression defaults when allocating pool.
Ignore compression hints when allocating from pool.

Also remove unused late initialization methods.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-06 16:30:58 +02:00
Compute-Runtime-Validation
d477935ab9 Revert "refactor: Gate shared system mem caps with KMD cap"
This reverts commit f38fae3b18.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-06 12:43:16 +02:00
John Falkowski
f38fae3b18 refactor: Gate shared system mem caps with KMD cap
Related-To: NEO-12988

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-05-06 07:28:59 +02:00
Dominik Dabek
75e313ce28 feature: prepare for l0 usm device pooling
Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-04-28 14:55:03 +02:00
Oskar Hubert Weber
1816c8eac4 fix: avoid joining thread in deferred deleter
Join thread in DllMain (which we are not aware of)
can result in hang occurring in DeferredDeleter, if the library is
freed before FreeLibraryAndExitThread call from within the worker thread,
the thread gets stuck, thus the main thread is stuck on worker->join().

Related-To: NEO-14121

Signed-off-by: Oskar Hubert Weber <oskar.hubert.weber@intel.com>
2025-04-17 16:17:42 +02:00
Chandio, Bibrak Qamar
bed2ad1ac0 feature: memAdise for System Allocator
Related-To: NEO-13715

Implements basic functionality for zeCommandListAppendMemoryAdvise for
system allocator

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-04-16 07:11:14 +02:00
Lukasz Jobczyk
f7939735da refactor: Adjust USM cleaner to ULLS light
Related-To: NEO-13922

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-04-15 14:05:02 +02:00
Lukasz Jobczyk
1b5519a880 fix: Adjust standard heaps when on 57 bit address space
Resolves: GSD-10871

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-04-09 12:51:03 +02:00
Dominik Dabek
bd516b3552 fix: usm reuse, clean from largest
When trimming old allocations in usm reuse start from largest
allocations.
This will reduce memory usage more quickly once max hold time is hit.

Related-To: NEO-6893, NEO-14429

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-04-04 14:57:15 +02:00
Dominik Dabek
3703ff550c fix: use real size when putting into usm reuse
Real allocation size should be used to properly apply limits and allow
more usm reuse hits.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-04-04 09:44:32 +02:00
Dominik Dabek
be27367020 performance: usm reuse, avoid looking up svmData
Save svmData on putting into reuse, instead of searching each time.
Change UNRECOVERABLE_IF to DEBUG_BREAK_IF.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-04-03 15:50:49 +02:00
Dominik Dabek
c76edaba4e fix: enable usm reuse limit based on memory usage
Related-To: NEO-14160, NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-27 15:14:08 +01:00
Dominik Dabek
915d657420 fix: flag to limit usm reuse based on memory usage
Host usm and device usm for igfx checks system memory usage.
Device usm for dgfx checks local memory usage.

If used memory is above limit threshold:
- no new allocations will be saved for reuse
- cleaner will use shorter hold time of 2 seconds
- cleaner will free all eligible allocations, regardless of async
deleter thread having work

Motivation: in case of gfx memory being full, making resident new
allocations will require evictions which leads to massive slowdown on
enqueue calls.
This change aims to minimize cases where extra memory usage from usm
reuse mechanism leads to above situation.

Related-To: NEO-6893, NEO-14160

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-27 10:25:19 +01:00
Dominik Dabek
6e998fc3c1 fix: move host usm reuse max size to mem manager
Intialize value on memory manager creation.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-24 08:53:30 +01:00
John Falkowski
a72b22c2ce test: Implement clang tidy fix for prefecth_manager_tests.cpp
Related-To: NEO-12989

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-03-14 19:01:15 +01:00
John Falkowski
4d281cf51d feature: Implement appendMemoryPrefetch for Shared System USM allocations
Related-To: NEO-12989

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-03-13 06:26:38 +01:00
Compute-Runtime-Validation
fa2e3adad3 Revert "feature: Implement appendMemoryPrefetch for Shared System USM Allocat...
This reverts commit 97799b3faf.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-03-12 05:55:32 +01:00
John Falkowski
97799b3faf feature: Implement appendMemoryPrefetch for Shared System USM Allocations
Related-To: NEO-12989

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2025-03-11 09:12:48 +01:00