Commit Graph

339 Commits

Author SHA1 Message Date
Lukasz Jobczyk
8a85a96ed2 feature: Add 3-level wait scheme with tpause intrinsic
Related-To: NEO-14336

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-03-21 12:12:57 +01:00
Chodor, Jaroslaw
0bff9def3b feature: New allocators for opaque arrays
Adds fast allocators for opaque objects of uniform size.

Related-To: NEO-13406

Signed-off-by: Chodor, Jaroslaw <jaroslaw.chodor@intel.com>
2025-03-20 21:59:44 +01:00
Lukasz Jobczyk
75461d190d performance: Adjust umwait counter value
Related-To: NEO-13922, NEO-14336

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-03-20 13:57:25 +01:00
Mateusz Hoppe
a86105814d fix: default initialize allocated memory when memory is overwritten
Related-To: NEO-5093

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2025-03-11 15:32:33 +01:00
Lukasz Jobczyk
be8545f7df refactor: reset all waitpkg variables on test start
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-03-11 08:44:36 +01:00
Szymon Morek
dacbce7f01 performance: introduce staging read for cl_buffer
Related-To: NEO-14026

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-03-11 08:19:28 +01:00
Fabian Zwoliński
f5e37e725c Revert "fix: configure ISA Pool params based on productHelper"
This reverts commit bf20ae7ae8.

Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-03-10 22:41:13 +01:00
Lukasz Jobczyk
53062056ec performance: Enable wait pkg for ULLS light
Related-To: NEO-13922

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2025-03-07 12:18:38 +01:00
Dominik Dabek
9eb8e1812c feature: flag to log usm reuse operations
If flag "LogUsmReuse" is set, usm reuse will log operations to csv file.
Each line will contain: timestamp, host/device, operation type,
allocation size, true/false whether operation succeeded.

This data can then be used to produce graphs and help in analyzing
usm reuse in a particular workload.

Related-To: NEO-6893

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2025-03-06 11:06:27 +01:00
Szymon Morek
01a66a9347 fix: download staging buffer before reading from it
Related-To: NEO-14154

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-02-21 09:30:23 +01:00
Bartosz Dunajski
85b0d1f84d fix: clear all TS packets for combined CB event
Related-To: NEO-13971

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2025-02-20 19:24:15 +01:00
Fabian Zwoliński
bf20ae7ae8 fix: configure ISA Pool params based on productHelper
When is2MBLocalMemAlignmentEnabled returns true,
increase pool size for builtins from 64k to 2MB.

Additionally, set appropriate alignment for kernel ISA heap allocations.
Additionally, configure isaAllocationPageSize based on productHelper

Related-To: NEO-12287
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-02-20 08:42:35 +01:00
Filip Hazubski
b60c02d597 fix: Add asserts to ensure NonCopyable and NonMovable n/n
Related-To: NEO-14068

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2025-02-19 11:36:24 +01:00
Filip Hazubski
6b6202446b fix: Add asserts to ensure NonCopyable and NonMovable 3/n
Related-To: NEO-14068

Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2025-02-18 17:16:03 +01:00
Jack Myers
c26d24e555 fix: tbx page fault manager hang issue
- Updated `isAllocTbxFaultable` to exclude `gpuTimestampDeviceBuffer` from being
faultable.
- Replaced `SpinLock` with `RecursiveSpinLock` in `CpuPageFaultManager` and
`TbxPageFaultManager` to allow recursive locking.
- Added unit tests to verify the correct handling of `gpuTimestampDeviceBuffer`
in `TbxCommandStreamTests`.

Related-To: NEO-13748
Signed-off-by: Jack Myers <jack.myers@intel.com>
2025-02-18 05:05:38 +01:00
Filip Hazubski
4c7900008f refactor: Change wording from NonCopyableOrMovable to NonCopyableAndNonMovable
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2025-02-17 14:19:10 +01:00
Fabian Zwoliński
1d8c8ac843 refactor: use designated initializers for SmallBuffersParams initialization
Related-To: NEO-12287
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-02-13 12:32:52 +01:00
Szymon Morek
8152241693 fix: don't override user data during staging read
Related-To: NEO-13574

User can read image with row pitch larger than region width.
In such cases, don't override memory which is out of region
but within row pitch.

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-02-12 16:22:48 +01:00
Compute-Runtime-Validation
116f7270be Revert "fix: tbx page fault manager hang issue"
This reverts commit 7d4e70a25b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-02-12 10:38:05 +01:00
Jack Myers
7d4e70a25b fix: tbx page fault manager hang issue
- Updated `isAllocTbxFaultable` to exclude `gpuTimestampDeviceBuffer` from being
faultable.
- Replaced `SpinLock` with `RecursiveSpinLock` in `CpuPageFaultManager` and
`TbxPageFaultManager` to allow recursive locking.
- Added unit tests to verify the correct handling of `gpuTimestampDeviceBuffer`
in `TbxCommandStreamTests`.

Related-To: NEO-13748
Signed-off-by: Jack Myers <jack.myers@intel.com>
2025-02-12 02:19:37 +01:00
Kamil Kopryk
03b56dc919 fix: untrusted input
Do not read an unlimited size of environment variables.

Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-02-10 14:54:03 +01:00
Fabian Zwoliński
1eb8e0efd9 fix: configure small buffers params based on productHelper
Refactor buffer pool allocator to support configurable
SmallBuffersParams based on product helper capabilities.

This patch enables setting custom pool
parameters instead of using fixed static values.

For devices with 2MB local memory alignment enabled
(is2MBLocalMemAlignmentEnabled),
use larger pool configuration:
- Pool size: 16MB (up from 2MB)
- Threshold: 2MB (up from 1MB)
- Alignment: 64KB (unchanged)
- Starting offset: 64KB (unchanged)

This improves memory utilization for devices supporting larger memory
alignments
while maintaining original parameters for other devices.

Key changes:
- Moved params from static template to instance member
- Added SmallBuffersParams struct with default/large configs
- Added constructor and setter methods for params configuration

Related-To: NEO-12287
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2025-02-07 12:01:23 +01:00
Szymon Morek
b11322332c performance: enable staging write for cl buffers
Related-To: NEO-13529

Also, add size threshold on iGPU on Linux,
and disable staging if imported host ptr could
be reused

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-02-05 09:41:17 +01:00
Francois Dugast
f91cde2a5b fix: invalid includes and missing rdtsc() on ARM
Signed-off-by: Francois Dugast <francois.dugast@intel.com>
2025-01-31 13:05:06 +01:00
Chodor, Jaroslaw
beedf709bc feature: LookupArray findGreaterEqual
Adding helpers for find first greater equal
value in lookup array. Useful when finding
minimum SLM encoding that satisfies requirements.

Related-To: NEO-12747

Signed-off-by: Chodor, Jaroslaw <jaroslaw.chodor@intel.com>
2025-01-27 13:02:02 +01:00
Kamil Kopryk
962836a07b refactor: use defaults for BaseTag attributes
Related-To: NEO-10767
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2025-01-16 18:24:39 +01:00
Szymon Morek
0b4fe9a0df performance: add staging transfers for cl buffers
Related-To: NEO-13529

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-01-15 12:24:43 +01:00
Compute-Runtime-Validation
58707e0cf7 Revert "refactor: migration to std::make_unique C++17"
This reverts commit 4cf685cb7d.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2025-01-09 01:41:36 +01:00
Semenov Herman (Семенов Герман)
4cf685cb7d refactor: migration to std::make_unique C++17
Signed-off-by: Semenov Herman (Семенов Герман) <GermanAizek@yandex.ru>
2025-01-08 17:20:00 +01:00
Szymon Morek
2815d4167d fix: don't use staging for map allocations
Related-To: NEO-13572

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2025-01-07 13:30:11 +01:00
Szymon Morek
1135c1033e fix: set staging buffer writeable for aub and tbx
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-12-23 13:35:01 +01:00
Mateusz Jablonski
6843632298 refactor: move SWTagsManager::currentCallCount to protected scope
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-12-19 12:31:44 +01:00
Marcel Skierkowski
ee0648c365 refactor: use virtualFileSystem in FileLogger ULTs
restored the possibility of different writing modes in the FileLogger
mocked writeToFile in TestFileLogger

Related-To: NEO-7006
Signed-off-by: Marcel Skierkowski <marcel.skierkowski@intel.com>
2024-12-18 19:01:39 +01:00
Bartosz Dunajski
b1dea19fbd refactor: move tag initialization to allocator [1/n]
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-12-17 17:53:13 +01:00
Dominik Dabek
e61d04a881 fix(ocl): track buffer pool count per device
Track amount of created buffer pools per device. Do not allocate extra
pools if limit is reached. New contexts will have pooling disabled if
limit is reached on device.

Related-To: NEO-13461

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-12-16 11:38:05 +01:00
Marcel Skierkowski
e27a6dc280 refactor: use virtualFileSystem in ULTs
reducing the number of tests that have interactions with filesystem.
writeDataToFile() saves filename and content in std::map.
fileExistsHasSize() checks if file was previously written to virtualFileSystem
loadDataFromVirtualFile() fetches data from std::map based on filename

Related-To: NEO-7006
Signed-off-by: Marcel Skierkowski <marcel.skierkowski@intel.com>
2024-12-13 17:57:07 +01:00
Szymon Morek
6c4eb322b1 performance: introduce staging reads from image
Related-To: NEO-12968

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-12-11 14:43:45 +01:00
Fabian Zwoliński
5f8e761541 fix: HeapAllocator - ensure getBaseAddress returns initial base address
getBaseAddress was incorrectly returning pLeftBound which changes after
memory allocation.
Added baseAddress field to store and return initial address value.

Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-12-10 10:52:36 +01:00
Compute-Runtime-Validation
58e45afd39 Revert "fix: HeapAllocator - ensure getBaseAddress returns initial base address"
This reverts commit ffec97acc5.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-12-08 14:35:22 +01:00
Fabian Zwoliński
ffec97acc5 fix: HeapAllocator - ensure getBaseAddress returns initial base address
getBaseAddress was incorrectly returning pLeftBound which changes after
memory allocation.
Added baseAddress field to store and return initial address value.

Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-12-05 20:42:16 +01:00
Fabian Zwoliński
d2ce3badfc fix: bindlessHeapsHelper handle unavailable external heap
This PR handles the situation in which a component
has reserved a front window space for itself in the external heap,
so that the Compute Runtime cannot access this area.

In such a situation, we perform the following steps:
1. reserve 4GB chunk in heapStandard
2. split our chunk into 2 parts: heapFrontWindow, heapRegular
3. from this point on, map all linearStream allocations in reserved 4GB
chunk

Patch applies to Windows and WSL.
Patch only applies when the bindless global allocator is enabled.

Related-To: HSD-16025889919
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-12-05 14:18:01 +01:00
Szymon Morek
16684d8803 fix: reuse staging buffers from other csr
Related-To: NEO-13247

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-11-18 16:39:57 +01:00
Bartosz Dunajski
7bf22ed33e feature: counter based allocation peer sharing
Related-To: NEO-13079

Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-11-13 15:01:32 +01:00
Jaroslaw Warchulski
72efceb8a3 fix: try to lower alignment in heap allocator when it is too big
Related-To: GSD-8948
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2024-11-13 09:14:12 +01:00
Szymon Morek
1199e19df5 performance: enable image writes through staging chunks
Related-To: NEO-12968

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-11-07 13:43:25 +01:00
Szymon Morek
a25e973205 performance: write image through staging chunks
Related-To: NEO-12968

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-11-06 16:45:22 +01:00
Mrozek, Michal
2febf0597e fix: remove incorrect code
- freed pointer cannot overlap with freed chunks, therefore there is no way
to merge it with freed chunks if they overlap

Signed-off-by: Mrozek, Michal <michal.mrozek@intel.com>
2024-11-05 18:58:41 +01:00
Lukasz Jobczyk
fe82425fa9 performance: Prefer to assign new gpu va rather than reuse
Resolves: NEO-10734

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-11-05 10:16:03 +01:00
Szymon Morek
5d62be2bea performance: enable staging buffer for write image
Related-To: NEO-12968

Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-11-05 06:06:07 +01:00
Compute-Runtime-Validation
aca7875dfd Revert "performance: Prefer to assign new gpu va rather than reuse"
This reverts commit 0f2f3c3764.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2024-10-31 05:06:43 +01:00