Commit Graph

282 Commits

Author SHA1 Message Date
Szymon Morek
5d7250ae6f fix: return false for allocInUse when gpu hang detected
Related-To: NEO-16105

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-09-16 14:11:31 +02:00
Michal Mrozek
6ef0af7e22 refactor: remove redundant code
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-08-25 16:14:42 +02:00
Neil R. Spruit
43ebc17e17 feature: Support for opaque IPC handles on Windows and Linux
- Added support for creating and managing opaque IPC NT handles in the
WDDM layer.
- Introduced a new flag `shareableWithoutNTHandle` to indicate if memory
can be shared without an NT handle.
- Updated the `isShareableMemory` method to accommodate this new flag.
- Added debug variable EnableShareableWithoutNTHandle to control the
behavior of sharing memory without NT handles until requested.
- Updated Linux path to enable sharing DMA Buf FDs between processes
for use in pidfd_getfd
- Updated getfd and get IPC Handle helper functions to support opaque or
previous versions

Related-To: NEO-15345 , NEO-15346 , NEO-15347, NEO-10380

Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
2025-08-18 16:23:44 +02:00
Compute-Runtime-Validation
0855219087 Revert "feature: Support for opaque IPC handles on Windows and Linux"
This reverts commit 46b1b2783b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-08-15 02:22:02 +02:00
Neil R. Spruit
46b1b2783b feature: Support for opaque IPC handles on Windows and Linux
- Added support for creating and managing opaque IPC NT handles in the
WDDM layer.
- Introduced a new flag `shareableWithoutNTHandle` to indicate if memory
can be shared without an NT handle.
- Updated the `isShareableMemory` method to accommodate this new flag.
- Added debug variable EnableShareableWithoutNTHandle to control the
behavior of sharing memory without NT handles until requested.
- Updated Linux path to enable sharing DMA Buf FDs between processes
for use in pidfd_getfd

Related-To: NEO-15345 , NEO-15346 , NEO-15347, NEO-10380

Signed-off-by: Neil R. Spruit <neil.r.spruit@intel.com>
2025-08-14 08:33:56 +02:00
Eric Mortensen
69f2cda2fa fix: add error checks & return to unMapVirtualMem
Related-To: NEO-8566

Signed-off-by: Eric Mortensen <eric.mortensen@intel.com>
2025-07-17 23:53:19 +02:00
Compute-Runtime-Validation
377b99e741 Revert "fix: do not align to 64kb when requested alignment is bigger"
This reverts commit 33749b8b5a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-07-15 14:58:49 +02:00
Maciej Plewka
33749b8b5a fix: do not align to 64kb when requested alignment is bigger
Related-To: NEO-14082
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2025-07-11 19:59:28 +02:00
Mateusz Hoppe
cb152ba821 fix: copy sip binary to allocation based on blitRequired query
- readOnly ISA allocations must be written through CPU pointer
- command buffer allocation in singleAddressSpaceSbaTracking mode cannot
be readonly - it is written by SBA tracking commands
- this change is fixing ZET_ENABLE_PROGRAM_DEBUGGING mode 2

Related-To: GSD-10359

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-06-30 19:41:19 +02:00
Michal Mrozek
f6cb013896 refactor: remove redundant member
- it is true on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-06-18 16:15:30 +02:00
Fabian Zwoliński
e2228201ce fix: Avoid redundant padding in ISA allocations
- Add isaPaddingIncluded flag to AllocationProperties to control ISA
padding
- Modify MemoryManager to skip automatic padding when flag is set
- Update ISAPoolAllocator to manage padding explicitly and set the
flag
- Use actual buffer size for HeapAllocator instead of requested size
- Add test covering different padding scenarios

This prevents double padding addition which was wasting memory when
ISAPoolAllocator managed its own padding on top of MemoryManager's
automatic padding.

Related-To: NEO-12287, HSD-18042276431
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-05-30 15:40:38 +02:00
Bartosz Dunajski
9c19e9eb96 fix: enable single temporary allocations list mode 2
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-27 17:35:17 +02:00
Compute-Runtime-Validation
42a8b63725 Revert "fix: enable single temporary allocations list mode"
This reverts commit b496274d1a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-05-26 13:22:15 +02:00
Bartosz Dunajski
b496274d1a fix: enable single temporary allocations list mode
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-22 08:45:46 +02:00
Bartosz Dunajski
91940d2a79 fix: initial support for single temporary allocations list
Related-To: NEO-14641

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-05-21 13:16:15 +02:00
Maciej Bielski
0f8ee57f98 feature: add NEO_LOCAL_MEMORY_ALLOCATION_MODE
Allow the application to force storageInfo.localOnly and get the
out-of-memory returned if not possible.

This is a windows-only feature supported on discrete platforms.

Related-To: NEO-13428
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-05-20 14:15:17 +02:00
Michal Mrozek
c3eec70a97 refactor: remove not needed hw info member
- it is false on all platforms.

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2025-05-07 14:17:49 +02:00
Jaroslaw Warchulski
3e1aa33924 refactor: cleanup headers
Related-To: NEO-5548
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2025-04-14 14:59:40 +02:00
Dominik Dabek
c76edaba4e fix: enable usm reuse limit based on memory usage
Related-To: NEO-14160, NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-27 15:14:08 +01:00
Dominik Dabek
915d657420 fix: flag to limit usm reuse based on memory usage
Host usm and device usm for igfx checks system memory usage.
Device usm for dgfx checks local memory usage.

If used memory is above limit threshold:
- no new allocations will be saved for reuse
- cleaner will use shorter hold time of 2 seconds
- cleaner will free all eligible allocations, regardless of async
deleter thread having work

Motivation: in case of gfx memory being full, making resident new
allocations will require evictions which leads to massive slowdown on
enqueue calls.
This change aims to minimize cases where extra memory usage from usm
reuse mechanism leads to above situation.

Related-To: NEO-6893, NEO-14160

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-27 10:25:19 +01:00
Dominik Dabek
6e998fc3c1 fix: move host usm reuse max size to mem manager
Intialize value on memory manager creation.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-24 08:53:30 +01:00
Maciej Bielski
971b7c27a2 fix: enable usm compression on linux
Related-To: NEO-12056
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2025-02-04 13:09:04 +01:00
Mateusz Hoppe
19a0a27862 refactor: adjust unit tests to work with secondary engines
Related-To: NEO-12952, NEO-13789


Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-01-22 13:31:43 +01:00
Lukasz Jobczyk
2dd9940f60 Revert "fix: count active modules for enabling per-dispatch private memory"
This reverts commit a483b361f9.

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-01-15 15:03:37 +01:00
Wenbin Lu
a483b361f9 fix: count active modules for enabling per-dispatch private memory
Related-To: NEO-13086

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2025-01-10 15:03:34 +01:00
Jack Myers
7f9fadc314 fix: regression caused by tbx fault mngr
Addresses regressions from the reverted merge
of the tbx fault manager for host memory.

Recursive locking of mutex caused deadlock.

To fix, separate tbx fault data from base
cpu fault data, allowing separate mutexes
for each, eliminating recursive locks on
the same mutex.

By separating, we also help ensure that tbx-related
changes don't affect the original cpu fault manager code
paths.

As an added safe guard preventing critical regressions
and avoiding another auto-revert, the tbx fault manager
is hidden behind a new debug flag which is disabled by default.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2025-01-09 07:48:53 +01:00
Compute-Runtime-Validation
124e755b9d Revert "fix: regression caused by tbx fault mngr"
This reverts commit 9a14fe2478.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-12-19 17:35:03 +01:00
Jack Myers
9a14fe2478 fix: regression caused by tbx fault mngr
Addresses regressions from the reverted merge
of the tbx fault manager for host memory.

This fixes attempts by the tbx fault manager
to protect/unprotect host buffer memory, even
if the host ptr was not driver-allocated.

In the case of the smoke test that triggered
the critical regression, clCreateBuffer was
called with the CL_MEM_USE_HOST_PTR flag.
The subsequent `mprotect` calls on the
provided host ptr then failed.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2024-12-18 23:16:36 +01:00
Compute-Runtime-Validation
6c5d9a6ed7 Revert "feature: extend TBX page fault manager from CPU implementation"
This reverts commit 51c0e80299.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-12-12 12:30:22 +01:00
Jack Myers
51c0e80299 feature: extend TBX page fault manager from CPU implementation
In TBX mode, the host could not write to host buffers after access from device
code due to the lack of a migration mechanism post-initial TBX upload.
Migration is unnecessary with real hardware, but required for TBX.

This patch introduces a new page fault manager type that extends the original
CPU fault manager, enabling automatic migration of host buffers in TBX mode.

Refactoring was necessary to avoid diamond inheritance, achieved by using a
template parameter as the base class for OS-specific fault managers.

Related-To: NEO-12268
Signed-off-by: Jack Myers <jack.myers@intel.com>
2024-12-11 09:09:50 +01:00
Fabian Zwoliński
d2ce3badfc fix: bindlessHeapsHelper handle unavailable external heap
This PR handles the situation in which a component
has reserved a front window space for itself in the external heap,
so that the Compute Runtime cannot access this area.

In such a situation, we perform the following steps:
1. reserve 4GB chunk in heapStandard
2. split our chunk into 2 parts: heapFrontWindow, heapRegular
3. from this point on, map all linearStream allocations in reserved 4GB
chunk

Patch applies to Windows and WSL.
Patch only applies when the bindless global allocator is enabled.

Related-To: HSD-16025889919
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-12-05 14:18:01 +01:00
Wenbin Lu
2ba80ce114 feature: support physical host memory
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-11-20 08:19:52 +01:00
Lukasz Jobczyk
5b3d244e97 refactor: Add AIL for hostptrs drain
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-11-19 15:13:30 +01:00
Lukasz Jobczyk
7f3896d05f performance: Ensure hostptrs removed before creating new one
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-11-12 13:55:15 +01:00
Bartosz Dunajski
7f5e6b4124 Revert "fix: Enable 64k pages for TSB allocation"
This reverts commit eed69f45ed.

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-10-28 16:02:34 +01:00
Maciej Bielski
45e78fea76 fix: use productHelper in getPatIndexInfoString() on Windows
Fix the PAT-index reporting in logger as currently on Windows reported
values are simply wrong.

The changed logic dependends on `RootDeviceEnvironment` and in order to
avoid introducing such dependencies into logger.[ch] the
`logAllocation()` is no longer a member of `FileLogger` but
a free-function instead (and a separate .cpp file). This is important
because the source files `logger.[ch]` are also used by ocloc library
and there is no point to contaminate ocloc code structure with
unnecessary dependencies.

Related-To: NEO-9421
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2024-10-22 19:27:13 +02:00
Wenbin Lu
a8a40d2afd feature: support SVM heap in reserveVirtualMem
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-10-22 16:47:14 +02:00
Bartosz Dunajski
eed69f45ed fix: Enable 64k pages for TSB allocation
Related-To: HSD-18040274716

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-10-22 15:12:42 +02:00
Lukasz Jobczyk
8a647f6a39 Revert "performance: Ensure hostptrs removed before creating new one"
This reverts commit 5b2f2f3d83.

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-10-21 10:36:06 +02:00
Mateusz Hoppe
5ae2552b4b fix: track shifted contextIds in bitset in bindlessHeapsHelper
- bitset is 64 bit in size, context ids may go beyond that limit
when multiple devices are available
- this change subtracts contextId of first context for a given root
device - tracked state dirty contexts ids are now zero-based

Resolves: GSD-10025

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-10-09 10:32:29 +02:00
Lukasz Jobczyk
5b2f2f3d83 performance: Ensure hostptrs removed before creating new one
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-10-08 13:04:56 +02:00
Compute-Runtime-Validation
60afb83b3b Revert "performance: Ensure hostptrs removed before creating new one"
This reverts commit a890ed5648.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-10-08 07:45:55 +02:00
Dominik Dabek
3685852ce0 Revert "performance: device usm sets localOnly...
Required"

This reverts commit a479afdbc8.

Related-To: NEO-12879

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-10-07 11:59:37 +02:00
Compute-Runtime-Validation
41df1a6f47 Revert "feature: support SVM heap in reserveVirtualMem"
This reverts commit bfaeeb01d6.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-10-03 14:53:50 +02:00
Lukasz Jobczyk
a890ed5648 performance: Ensure hostptrs removed before creating new one
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-09-19 14:34:19 +02:00
Compute-Runtime-Validation
e4d2f16632 Revert "performance: Ensure hostptrs removed before creating new one"
This reverts commit ac1d203555.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-09-18 19:31:34 +02:00
Lukasz Jobczyk
ac1d203555 performance: Ensure hostptrs removed before creating new one
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-09-18 14:23:16 +02:00
Michal Mrozek
dd631610b3 refactor: move memory tracking to memory manager
- remove wddm specific code
- improve total size reported to be in decimal

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-09-12 17:32:38 +02:00
Michal Mrozek
da59b88122 refactor: improve logging of allocations
add capability to measure total amount of allocated memory
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-09-12 11:00:27 +02:00
Wenbin Lu
bfaeeb01d6 feature: support SVM heap in reserveVirtualMem
Related-To: NEO-11981

Signed-off-by: Wenbin Lu <wenbin.lu@intel.com>
2024-09-09 23:22:04 +02:00