Commit Graph

275 Commits

Author SHA1 Message Date
Mateusz Jablonski
7660b29bbb fix: reduce types for tagSize and tagCount within TagAllocator
Related-To: NEO-16444

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-11-17 10:47:51 +01:00
Compute-Runtime-Validation
ff27bb12d1 Revert "fix: use condition variables instead of busy waits in worker threads"
This reverts commit 4406889b39.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-11-14 15:55:47 +01:00
Kamil Kopryk
129249f022 refactor: correct typo
Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-11-06 15:15:08 +01:00
Kamil Kopryk
8757ecf2f7 refactor: reuse tag allocation for host function data
Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-11-05 14:51:20 +01:00
Kamil Kopryk
f84a5fbee9 feature: add host functions workers
* add common host function worker interface
* add worker as a single thread per csr with 3 modes
* add logic for waiting on internal tag, check gpu hang
* if tag is in pending state, read callback data, run callback
and signal completion
* threads will exit the work loop once stop request
is called in finish
* add multi thread unit tests

Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-11-03 12:11:17 +01:00
Igor Venevtsev
4406889b39 fix: use condition variables instead of busy waits in worker threads
Resolves: NEO-16085, GSD-11678, HSD-14025819208

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2025-10-31 15:28:54 +01:00
Compute-Runtime-Validation
b7d1c32edd Revert "fix: use condition variables instead of busy waits in worker threads"
This reverts commit 1f6039676f.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-10-24 10:55:27 +02:00
Igor Venevtsev
1f6039676f fix: use condition variables instead of busy waits in worker threads
Resolves: NEO-16085, GSD-11678, HSD-14025819208

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2025-10-21 17:37:00 +02:00
Szymon Morek
64b79723cc performance: enable cmd buffers reuse without DC flush
Related-To: NEO-16348

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-10-17 14:26:37 +02:00
Szymon Morek
c78c1515de performance: reuse cmd buffer without dc flush
Related-To: NEO-16348

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-10-16 16:26:54 +02:00
Mateusz Jablonski
35f6dc12b8 refactor: remove not needed code
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-10-15 16:19:04 +02:00
Lukasz Jobczyk
6515e422e9 refactor: move eviction container to residency controller
Related-To: NEO-13315

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-10-13 08:41:34 +02:00
Compute-Runtime-Validation
244dd9b0b4 Revert "fix: use condition variables instead of busy waits in worker threads"
This reverts commit db0b4a616c.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-10-11 03:52:05 +02:00
Compute-Runtime-Validation
2eb8928ec5 Revert "performance: increase heap size to 4MB"
This reverts commit f41bb3517a.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-10-10 22:23:23 +02:00
Igor Venevtsev
db0b4a616c fix: use condition variables instead of busy waits in worker threads
Resolves: NEO-16085, GSD-11678, HSD-14025819208

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2025-10-10 21:42:02 +02:00
Szymon Morek
f41bb3517a performance: increase heap size to 4MB
Related-To: NEO-16348

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-10-09 13:03:53 +02:00
Mateusz Jablonski
f3e4ba9ff6 refactor: remove not needed code
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-10-07 13:44:30 +02:00
Radoslaw Jablonski
54de14a9dc refactor: apply clang-format's InsertBraces rule
Formats code before InsertBraces rule is enabled.

Signed-off-by: Radoslaw Jablonski <radoslaw.jablonski@intel.com>
2025-10-06 15:32:46 +02:00
Szymon Morek
ee032982a6 refactor: print faulted address when waiting for tag
getDeviceState on Windows prints faulted GPU VA
if OOB access happened.

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-10-02 14:59:02 +02:00
Slawomir Milczarek
40ce5eb55e fix: Fix DirectSubmissionController deadlock with try_lock pattern
Related-To: NEO-13325

Replace blocking obtainUniqueOwnership() with tryObtainUniqueOwnership()
in critical paths to prevent deadlock between controller thread holding
directSubmissionsMutex and submission threads holding CSR locks.

Changes:
- Add CommandStreamReceiver::tryObtainUniqueOwnership() method
- Use try_lock in checkNewSubmissions() and context group idle detection
- Skip contended CSRs instead of blocking (conservative approach)
- Add comprehensive unit tests for try_lock functionality

Fixes deadlock scenario where registerDirectSubmission() waits for
directSubmissionsMutex while checkNewSubmissions() holds it and waits
for CSR ownership locks.

Signed-off-by: Slawomir Milczarek <slawomir.milczarek@intel.com>
2025-09-29 11:38:26 +02:00
Kamil Kopryk
8c7fb0dde8 fix: data race in host function data initialization
Make sure hostFunctionInitialized bool is set after actual operations
are done

Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-09-25 11:29:01 +02:00
Compute-Runtime-Validation
6736378c4d Revert "fix: use condition variables instead of busy waits in worker threads"
This reverts commit 452475a0b9.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-09-23 17:06:18 +02:00
Igor Venevtsev
452475a0b9 fix: use condition variables instead of busy waits in worker threads
Resolves: NEO-16085, GSD-11678, HSD-14025819208

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2025-09-23 14:44:39 +02:00
Kamil Kopryk
5f59b4ca08 feature: add host function allocation
Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-09-23 11:10:45 +02:00
Compute-Runtime-Validation
16a6c3ca8a Revert "feature: add host function allocation"
This reverts commit 75b4de70cd.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-09-23 02:33:15 +02:00
Kamil Kopryk
75b4de70cd feature: add host function allocation
Related-To: NEO-14577
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-09-22 17:55:36 +02:00
Slawomir Milczarek
e80b84e699 feature: Don't terminate ULLS contexts if busy context exists
Enhanced direct submission idle detection to ensure that
ULLS contexts are not terminated if any context in the same group
is still busy or has pending work.
Idle detection now accurately considers the state of all CSRs
in a context group before terminating any direct submission.
Controlled with DirectSubmissionControllerContextGroupIdleDetection
(note: the feature is disabled by default in first step).

Related-To: NEO-13325

Signed-off-by: Slawomir Milczarek <slawomir.milczarek@intel.com>
2025-09-12 09:32:59 +02:00
Alicja Lukaszewicz
bca503548a feature: add peer access check on driver init
Related-To: NEO-14885, HSD-14024947073

Signed-off-by: Alicja Lukaszewicz <alicja.lukaszewicz@intel.com>
2025-08-27 15:48:22 +02:00
Dominik Dabek
77470acf7a fix: l0, tag update on mem copy ext host ptr
Require tag update on mem copy with external host ptr.
Without this, temporary allocation might not be cleaned before next copy
operation.
If a second copy operation is passed same ptr that has been reallocated,
there will be a pagefault.

Related-To: NEO-15663

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-08-08 12:42:53 +02:00
Mateusz Jablonski
05cf05e10e refactor: remove redundant tracking of media sampler programming
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2025-07-28 15:17:46 +02:00
Grochowski, Stanislaw
8180e95e0f fix: ensure pollForCompletion in aub on program termination
Related-To: NEO-14867

Signed-off-by: Grochowski, Stanislaw <stanislaw.grochowski@intel.com>
2025-07-10 08:32:00 +02:00
Mateusz Hoppe
cb152ba821 fix: copy sip binary to allocation based on blitRequired query
- readOnly ISA allocations must be written through CPU pointer
- command buffer allocation in singleAddressSpaceSbaTracking mode cannot
be readonly - it is written by SBA tracking commands
- this change is fixing ZET_ENABLE_PROGRAM_DEBUGGING mode 2

Related-To: GSD-10359

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-06-30 19:41:19 +02:00
Kamil Kopryk
bb52c358b9 refactor: flush Task method
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
Related-To: NEO-15011

Also correct ults
2025-06-11 16:36:56 +02:00
Aleksandra Nizio
fa1e3fd6a2 fix: Remove old aub_mem_dump logic and related tests
Related-To: NEO-14718

Signed-off-by: Aleksandra Nizio <aleksandra.nizio@intel.com>
2025-05-13 08:38:59 +02:00
Lukasz Jobczyk
1d1414febc refactor: remove unused dc flush mitigation
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-05-09 08:26:26 +02:00
Lukasz Jobczyk
6f4a56d440 refactor: pass product helper to isFenceAllocationRequired
Related-To: NEO-14642

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-04-28 14:09:02 +02:00
Szymon Morek
3596522637 refactor: remove unused logic in ULLS controller
Related-To: NEO-13843

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-04-17 18:35:20 +02:00
Jaroslaw Warchulski
3e1aa33924 refactor: cleanup headers
Related-To: NEO-5548
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2025-04-14 14:59:40 +02:00
Filip Hazubski
504440fc4d feature: Add ftrHeaplessMode flag
Pass hwInfo to isHeaplessModeEnabled and isForceBindlessRequired functions.

Related-To: NEO-14526

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2025-04-02 21:06:05 +02:00
Lukasz Jobczyk
8a85a96ed2 feature: Add 3-level wait scheme with tpause intrinsic
Related-To: NEO-14336

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-03-21 12:12:57 +01:00
Damian Tomczak
81b0cac65f fix: raytracing heapless missing allocation
Related-to: NEO-12737

Signed-off-by: Damian Tomczak <damian.tomczak@intel.com>
2025-03-06 17:26:09 +01:00
Lukasz Jobczyk
be946ae56c performance: Optimize make resident for ULLS light
Do not check if ULLS light is active during every Csr::makeResident
call. Store that information once during ULLS init.

Related-To: NEO-13922

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-02-24 12:59:56 +01:00
Lukasz Jobczyk
bc2b49b958 feature: Introduce ULLS light
Add core implementation of ULLS without VM_BIND interface aka ULLS
light.

Related-To: NEO-13922

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-02-12 17:52:02 +01:00
Krzysztof Sprzaczkowski
a17745532c performance: Move preemption allocation init to the first submission
Related-To: NEO-12323
Signed-off-by: Krzysztof Sprzaczkowski <krzysztof.sprzaczkowski@intel.com>
2025-01-15 20:22:50 +01:00
Bartosz Dunajski
b1dea19fbd refactor: move tag initialization to allocator [1/n]
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-12-17 17:53:13 +01:00
Mateusz Jablonski
552930a75f fix: don't setup preemption surface when debugger is active
Related-To: NEO-12878
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-10-08 13:58:11 +02:00
Slawomir Milczarek
edeb7bdd4b refactor: Allocate copy source for work partition allocation on heap
No perforamce impact expected since it is initialized once only,
but has the advantage of using custom allocator by overriding malloc.

Related-To: NEO-12846

Signed-off-by: Slawomir Milczarek <slawomir.milczarek@intel.com>
2024-10-01 13:32:55 +02:00
Bartosz Dunajski
5b1bd4b088 refactor: dont mix aub and hw wait prints
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-09-26 08:51:23 +02:00
Kamil Kopryk
ec5beaf616 refactor: reduce csr class size
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-09-18 13:33:55 +02:00
Lukasz Jobczyk
a54a3bf624 performance: Optimize heap handling when mitigate dc flush
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-09-06 04:33:41 +02:00