Commit Graph

830 Commits

Author SHA1 Message Date
Kamil Kopryk
f15c42bd11 performance: optimize checking if ga is imported 1/2
Related-To: NEO-13163
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-09-04 11:08:22 +02:00
Mateusz Hoppe
c96348d1f3 feature: add support for external memmap system extension
Related-To: GSD-11423

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-09-02 13:52:42 +02:00
Michal Mrozek
6ef0af7e22 refactor: remove redundant code
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-08-25 16:14:42 +02:00
Mateusz Jablonski
fd610f7a7b fix: unify initial cpu range reservation size in case of 57bit CPU address range
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-08-20 09:55:11 +02:00
Compute-Runtime-Validation
17c58c1be4 Revert "refactor: reduce the size of graphics allocation"
This reverts commit 167ca6e50c.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-08-19 10:11:15 +02:00
Neil R. Spruit
43ebc17e17 feature: Support for opaque IPC handles on Windows and Linux
- Added support for creating and managing opaque IPC NT handles in the
WDDM layer.
- Introduced a new flag `shareableWithoutNTHandle` to indicate if memory
can be shared without an NT handle.
- Updated the `isShareableMemory` method to accommodate this new flag.
- Added debug variable EnableShareableWithoutNTHandle to control the
behavior of sharing memory without NT handles until requested.
- Updated Linux path to enable sharing DMA Buf FDs between processes
for use in pidfd_getfd
- Updated getfd and get IPC Handle helper functions to support opaque or
previous versions

Related-To: NEO-15345 , NEO-15346 , NEO-15347, NEO-10380

Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
2025-08-18 16:23:44 +02:00
Maciej Bielski
c6398e5d20 fix: acquire lock before checking for totalSize limit
Related-To: NEO-15630

Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-08-18 15:26:06 +02:00
Mateusz Jablonski
3116ebd051 fix: respect num devices when reserving cpu address range for non-svm heaps
in case of reserving in upper 48 bit address space

Related-To: NEO-15843, GSD-11464
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-08-18 15:12:10 +02:00
Mateusz Jablonski
a650e08145 refactor: rename variable related to reserved cpu address range
in case of CPU address range wider than GPU address range, we reserve
CPU address range to avoid CPU side allocations in our non-SVM heaps

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-08-18 12:20:25 +02:00
Dominik Dabek
2cd11d154b fix: disable usm reuse on kmdless
Related-To: NEO-15663

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-08-18 11:40:59 +02:00
Compute-Runtime-Validation
0855219087 Revert "feature: Support for opaque IPC handles on Windows and Linux"
This reverts commit 46b1b2783b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-08-15 02:22:02 +02:00
Neil R. Spruit
46b1b2783b feature: Support for opaque IPC handles on Windows and Linux
- Added support for creating and managing opaque IPC NT handles in the
WDDM layer.
- Introduced a new flag `shareableWithoutNTHandle` to indicate if memory
can be shared without an NT handle.
- Updated the `isShareableMemory` method to accommodate this new flag.
- Added debug variable EnableShareableWithoutNTHandle to control the
behavior of sharing memory without NT handles until requested.
- Updated Linux path to enable sharing DMA Buf FDs between processes
for use in pidfd_getfd

Related-To: NEO-15345 , NEO-15346 , NEO-15347, NEO-10380

Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
2025-08-14 08:33:56 +02:00
Michal Mrozek
167ca6e50c refactor: reduce the size of graphics allocation
- 32 context is way too much, most our devices has 5 context right now

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-08-13 11:43:27 +02:00
Maciej Bielski
478c7ce9a8 fix: possible race-condition in SvmAllocationCache::insert
Related-To: NEO-15630

Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-08-13 11:18:22 +02:00
Chandio, Bibrak Qamar
6acfcb83b9 Revert "Two commits related to pStart feature"
Related-To: NEO-15156, GSD-9939

Revert "test: fix minor errors"

This reverts commit 712dc8221e.

Revert "feature: Support for pStart"

This reverts commit 47caeda487.

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-08-13 07:10:40 +02:00
Chandio, Bibrak Qamar
47caeda487 feature: Support for pStart
Related-To: NEO-15156, GSD-9939

Support for start address hint in zeVirtualMemReserve.

If it fails to find pStart then it defaults to the base line
allocateWithCustomAlignment(...)

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-08-06 12:40:29 +02:00
Compute-Runtime-Validation
c9456cda14 Revert "feature: Support for pStart"
This reverts commit a50c0dbecf.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-08-06 02:26:13 +02:00
Chandio, Bibrak Qamar
a50c0dbecf feature: Support for pStart
Related-To: NEO-15156, GSD-9939

Support for start address hint in zeVirtualMemReserve.

If it fails to find pStart then it defaults to the base line
allocateWithCustomAlignment(...)

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-08-05 10:14:01 +02:00
Lukasz Jobczyk
3e2acefac1 fix: Do not align svm gpu greater than its svm cpu
Resolves: HSD-18042781233

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-08-05 07:52:18 +02:00
Michal Mrozek
3eefd8a4ad refactor: add type for mutex and lock
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-08-04 13:51:35 +02:00
Eric Mortensen
69f2cda2fa fix: add error checks & return to unMapVirtualMem
Related-To: NEO-8566

Signed-off-by: Eric Mortensen <eric.mortensen@intel.com>
2025-07-17 23:53:19 +02:00
Compute-Runtime-Validation
377b99e741 Revert "fix: do not align to 64kb when requested alignment is bigger"
This reverts commit 33749b8b5a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-07-15 14:58:49 +02:00
Maciej Plewka
33749b8b5a fix: do not align to 64kb when requested alignment is bigger
Related-To: NEO-14082
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2025-07-11 19:59:28 +02:00
Dominik Dabek
ff662b3e88 fix: usm reuse aub/tbx indirect access
When allocation is saved for reuse, remove it from allocations map.
On get from reuse, reinsert it with new allocId.

Allocations saved for reuse should not be made resident for indirect
access.

Only applicable on aub/tbx csr types.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-10 08:02:03 +02:00
Dominik Dabek
3dcfef5812 fix: usm reuse aub/tbx update alloc id
On getting from reuse update alloc id and increment allocations counter.
This is to ensure that making indirect allocations resident picks up the
reuse allocation.
Only applicable on aub/tbx csr types.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-07 15:49:16 +02:00
Slawomir Milczarek
76090f041d fix: Fixed out-of-bounds write to usageInfos array with residency task count
Related-To: NEO-12952

Signed-off-by: Slawomir Milczarek <slawomir.milczarek@intel.com>
2025-07-03 18:00:37 +02:00
Dominik Dabek
e52fa32271 fix: disable usm reuse if debugger enabled
Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-07-01 14:22:15 +02:00
Mateusz Hoppe
cb152ba821 fix: copy sip binary to allocation based on blitRequired query
- readOnly ISA allocations must be written through CPU pointer
- command buffer allocation in singleAddressSpaceSbaTracking mode cannot
be readonly - it is written by SBA tracking commands
- this change is fixing ZET_ENABLE_PROGRAM_DEBUGGING mode 2

Related-To: GSD-10359

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-06-30 19:41:19 +02:00
Dominik Dabek
dde8bf1b9b fix: usm reuse blocking free handling
on blocking free wait for engines completion before putting into reuse
mark allocation as completed and do not check is in use status on
getting

Related-To: NEO-6893, HSD-18042686276

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-30 11:46:45 +02:00
Dominik Dabek
bd9b458add fix: correct debug break if, usm reuse
Should break if memory manager is nullptr

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-26 13:25:54 +02:00
Dominik Dabek
4cbd6c79a0 fix: mark aub/tbx writable on get from usm reuse
Related-To: NEO-6893, NEO-15359, HSD-18042681561

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-25 19:36:16 +02:00
Dominik Dabek
5c7fd72d5f performance: enable l0 usm reuse
Do not reuse internal or imported allocations.
Mark as aub writable on getting from reuse.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-25 15:49:15 +02:00
Chandio, Bibrak Qamar
c1867ed981 feature: atomic attributes for shared system alloc
Related-To: NEO-13715, NEO-14862

Signed-off-by: Chandio, Bibrak Qamar <bibrak.qamar.chandio@intel.com>
2025-06-23 20:14:47 +02:00
Dominik Dabek
77e25f90d7 fix: prepare for l0 usm reuse enable
adjust ULTs
disable usm reuse in multi device l0
don't initialize usm caches if max reuse size is 0
check requested alignment when returning from cache
guard against double free

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-06-23 17:17:18 +02:00
Michal Mrozek
f6cb013896 refactor: remove redundant member
- it is true on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-06-18 16:15:30 +02:00
Szymon Morek
6a7f6f1ad6 fix: add noexcept to move operators
Without it, copy operator is selected, even if move is explicitly defined.

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-06-09 10:32:23 +02:00
Fabian Zwoliński
e2228201ce fix: Avoid redundant padding in ISA allocations
- Add isaPaddingIncluded flag to AllocationProperties to control ISA
padding
- Modify MemoryManager to skip automatic padding when flag is set
- Update ISAPoolAllocator to manage padding explicitly and set the
flag
- Use actual buffer size for HeapAllocator instead of requested size
- Add test covering different padding scenarios

This prevents double padding addition which was wasting memory when
ISAPoolAllocator managed its own padding on top of MemoryManager's
automatic padding.

Related-To: NEO-12287, HSD-18042276431
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-05-30 15:40:38 +02:00
Dominik Dabek
ad4d7a2ce8 fix: l0 usm pooling, ipc handling
Move getting usm pool for ptr to its own method.
Move trying to free via pooling to its own method.

Use base ptr of pool for tracking in IPCHandleMap.
Track ipc handle refcount in freeMem.
Return error when trying to use a not allocated ptr within pool.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-29 16:07:19 +02:00
Aleksandra Nizio
1dfc9227c4 fix: Removind address_mapper.h
Related-To: NEO-14718
Signed-off-by: Aleksandra Nizio <aleksandra.nizio@intel.com>
2025-05-27 20:37:56 +02:00
Bartosz Dunajski
9c19e9eb96 fix: enable single temporary allocations list mode 2
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-27 17:35:17 +02:00
Compute-Runtime-Validation
42a8b63725 Revert "fix: enable single temporary allocations list mode"
This reverts commit b496274d1a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-26 13:22:15 +02:00
Bartosz Dunajski
b496274d1a fix: enable single temporary allocations list mode
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-22 08:45:46 +02:00
Bartosz Dunajski
91940d2a79 fix: initial support for single temporary allocations list
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-21 13:16:15 +02:00
Maciej Bielski
0f8ee57f98 feature: add NEO_LOCAL_MEMORY_ALLOCATION_MODE
Allow the application to force storageInfo.localOnly and get the
out-of-memory returned if not possible.

This is a windows-only feature supported on discrete platforms.

Related-To: NEO-13428
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-05-20 14:15:17 +02:00
Dominik Dabek
7a091919a4 fix: usm reuse trim, non locked vector read
Remove checking allocations.empty before lock.

Related-To: NEO-14529

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-19 21:17:17 +02:00
Oskar Hubert Weber
c3d5ee8332 fix: unblock main thread after mutex unlock in DeferredDeleter
DeferredDeleter instance could get deleted before worker thread called
unlock() on queue mutex leading to an error

Related-To: NEO-14896

Signed-off-by: Oskar Hubert Weber <oskar.hubert.weber@intel.com>
2025-05-09 14:17:47 +02:00
Michal Mrozek
6f4a397cfc refactor: remove not required parameter
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-08 23:13:06 +02:00
Szymon Morek
6ae43123f6 fix: correct usages of ULLS-related resources
Related-To: NEO-14360

Current gmm usage type of these resources is causing
them to be cached, which is incorrect.

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-05-08 12:12:45 +02:00
Michal Mrozek
c3eec70a97 refactor: remove not needed hw info member
- it is false on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-07 14:17:49 +02:00
Dominik Dabek
6afefa1e31 performance: usm device pool, compression handling
Use device compression defaults when allocating pool.
Ignore compression hints when allocating from pool.

Also remove unused late initialization methods.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-05-06 16:30:58 +02:00