05250e7460
fix: Remove patchtokens fallback AIL
...
Related-To: NEO-12358
Signed-off-by: Jaroslaw Warchulski <jaroslaw.warchulski@intel.com>
2024-09-12 18:26:19 +02:00
8bb92ff445
performance: avoid reading from gfx memory when modifying walker command
...
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2024-09-11 18:58:33 +02:00
48dc0a823d
fix: Check dst pitch in copy rect misaligned case
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-09-11 11:45:10 +02:00
7ce4a8adc2
performance: replace virtual calls with native class methods
...
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2024-09-11 11:10:40 +02:00
b2fc7345cf
performance: redesign usm alloc reuse mechanism
...
Dedicated pools for different allocations size ranges.
Additional reused allocations will create their own pools.
Do not reuse allocations >256MB.
Related-To: NEO-6893, NEO-12299, NEO-12349
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-09-10 18:12:14 +02:00
4f1262645b
refactor: pass extra walker params
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-09-10 15:30:03 +02:00
beaddb55e0
fix: Minor code changes
...
Remove needless checks for module and cmdList.
Add explicit check for surfaceState in setArgImageWithMipLevel.
Explicitly initialize WddmResidencyController::csr.
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2024-09-10 13:31:15 +02:00
54bda0e986
fix: In Linux CL/GL sharing
...
- always issue flush request before export
Apparently it's expected to flush the object (which might convert them
from one format to another for export, or remove aux buffer uses or
anything not supported by export).
- use modifier to select tiling mode
Previously we just assumed that whatever tiling mode was picked by mesa
will match the one picked by GMMLIB but that's not always the case
and in particular on Arc and Xe it doesn't work ... Mesa picks Tile4
and GMMLIB picks Tile64 ...
Fixes : #761
Fixes : #736
Signed-off-by: Sylvain Munaut <tnt@246tNt.com>
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-09-06 21:40:18 +02:00
9d5f8eb587
refactor: save hpCopyEngine in Device
...
- remove loop iteration to get hpCopyEngine
Related-To: NEO-11983
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-09-06 17:56:53 +02:00
03690e9b83
fix: Set special queue after its setup
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-09-06 12:05:24 +02:00
95b035a071
fix: global stateless heap creation in heapless path
...
Related-To: NEO-10681
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-09-05 16:56:00 +02:00
b0e7a11e9a
refactor: Improving information transfer about the copy engine
...
Related-To: NEO-11934
Signed-off-by: Andrzej Koska <andrzej.koska@intel.com>
2024-09-05 16:11:52 +02:00
d842f65cf1
Revert "fix: submit dummy exec to pin memory during zeContextMakeMemoryReside...
...
This reverts commit f9b87d53e6
2024-09-05 03:28:03 +02:00
f9b87d53e6
fix: submit dummy exec to pin memory during zeContextMakeMemoryResident call
...
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
Related-To: NEO-11879
2024-09-04 14:07:29 +02:00
37b7caa137
fix: correct program header generation for shared isa allocation
...
- when kernels share single allocation, LOAD address in program headers
should point to correct virtual address including kernel offset
Related-To: NEO-7788, GSD-9836
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-09-03 17:50:29 +02:00
7adfa344c0
performance: get reference from getBindlessInfo
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-09-03 12:50:52 +02:00
8b760f8528
performance: Copy hostptr on cpu for host buffer when mitigate dc flush
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-30 19:55:45 +02:00
847180dc9a
refactor: add debug prints for timestamp waits
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-08-30 19:55:15 +02:00
c934877790
refactor: remove not needed function
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-08-30 12:18:14 +02:00
4552960e1f
performance(ocl): limit flushes on resolve via pc
...
When resolving dependencies via pipecontrol skip hdc and untyped data
port cache flushes. Leave only command streamer stall.
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-08-29 17:19:19 +02:00
7550e55d64
fix: Always copy hostptr on gpu when mitigate dc flush
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-29 14:53:57 +02:00
496012d82f
performance: Use copy buffer rect middle builtin
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-28 15:32:00 +02:00
5aa5d40937
performance: Mitigate dc flush on LNL Windows
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-28 13:35:17 +02:00
0b848a5fdb
fix: Don't use copy buffer rect middle builtin
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-27 12:01:58 +02:00
90bc6c10ea
fix(ocl): use correct surface state index for bindless offset in setArgImage
...
Related-To: NEO-11156
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-08-27 09:42:46 +02:00
ad0d6f5435
Revert "refactor: Add dc flush mitigation infrastructure"
...
This reverts commit e4412e385a
2024-08-27 02:35:06 +02:00
a47ca96a42
fix(ocl): allocate small buffer pool uncompressed
...
Related-To: HSD-15016054429
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-08-26 13:00:47 +02:00
e4412e385a
refactor: Add dc flush mitigation infrastructure
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-26 10:38:56 +02:00
5dbbaa39b9
Revert "fix: ulls controller sleep, windows"
...
This reverts commit 6455d4648c
2024-08-24 06:37:58 +02:00
956dd8e17d
Revert "fix: set properly resource params when setAllocationType"
...
This reverts commit 2e0884a301
2024-08-24 05:01:38 +02:00
2e0884a301
fix: set properly resource params when setAllocationType
...
gmm params: usage, cachable and resource info
should be set properly when override allocation type
Resolves: HSD-22020344331
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
2024-08-23 16:57:23 +02:00
9152b6ac04
performance: Defer special queue init to first use
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-23 14:22:51 +02:00
c9e8931a18
fix: global bindless offsets for implicit args
...
- do not patch bindless offsets for implicit args in with global bindless
allocator, the offsets are patched at kernel initialization time and
do not change
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-08-23 13:21:27 +02:00
6455d4648c
fix: ulls controller sleep, windows
...
Request higher resolution for windows periodic timers for ulls
controller sleep.
Allows for controller thread to sleep with granularity of 1ms.
Related-To: NEO-10800
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-08-23 12:05:26 +02:00
696b02bfd3
fix: improve TBX downloading after L0 Event sync
...
Related-To: HSD-18038498579
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-08-23 10:42:17 +02:00
4b01058706
Revert "performance: Defer special queue init to first use"
...
This reverts commit 25bb3c87ad
2024-08-23 09:09:17 +02:00
c1a5fb089b
performance: Add copy buffer rect middle builtin
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-22 10:30:17 +02:00
25bb3c87ad
performance: Defer special queue init to first use
...
Resolves: NEO-12332
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-22 10:07:44 +02:00
d8ea5516b2
Revert "performance: Add copy buffer rect middle builtin"
...
This reverts commit bbb44c7a4d
2024-08-22 04:41:42 +02:00
7ac41615cd
fix: create thread with function pointer
...
don't create async thread in neo shared tests
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-08-21 18:02:37 +02:00
bbb44c7a4d
performance: Add copy buffer rect middle builtin
...
Resolves: NEO-12132
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-08-21 14:41:44 +02:00
f617093a6a
fix: add missing nullptr check before accessing ail helper
...
Fixes : #755
Fixes : #754
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-08-20 16:58:53 +02:00
579af57161
refactor: don't call OsLibrary::load directly, use function pointer
...
this allows mocking this call in ULT
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-08-20 08:59:26 +02:00
9b652f4a34
Revert "feature: Improving information transfer about the copy engine"
...
This reverts commit 17ffdff4f1
2024-08-15 22:06:31 +02:00
17ffdff4f1
feature: Improving information transfer about the copy engine
...
Related-To: NEO-11934
Signed-off-by: Andrzej Koska <andrzej.koska@intel.com>
2024-08-14 11:28:29 +02:00
27488a8315
fix: Add option to use divergent paths for barriers
...
Related-To: NEO-12159, IGC-9846, HSD-16024235164, HSD-15014782386
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2024-08-12 12:15:54 +02:00
d76ac1d1de
fix: scratch controller residency
...
Related-To: HSD-18039519400
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-08-09 14:47:56 +02:00
775b14a7f6
fix: add ioh alignment in heapless
...
Related-To: NEO-11871
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-08-09 12:20:00 +02:00
ad229377b9
fix: disable indirect detection if any stack calls
...
Don't know if kernels will be initialized in the order needed to check
for indirect accesses in stack calls.
Remove now unused functionPointerWithIndirectAccessExists and reading
this value from zebin.
Related-To: NEO-12235
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-08-07 14:48:58 +02:00
38a194eee6
fix: scratch address from implicit args in ocl
...
Related-To: NEO-12237
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-08-07 09:40:27 +02:00
d52ca080bd
Revert "performance: improve pool handling"
...
This reverts commit a3c3b6533a
2024-08-06 13:04:02 +02:00
2a9bcdeb83
refactor: pass outImplicitArgs to patchImplicitArgs function
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-08-05 17:31:47 +02:00
674c4a15ad
fix: use correct gpu address when bindless heaps helper is enabled
...
Related-To: NEO-7063
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-08-05 15:09:57 +02:00
ec34656e0e
fix: debug flag to defer first device submission
...
Related-To: HSD-18039343751
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-08-02 09:31:25 +02:00
0196a0f72f
refactor(ocl): internal linker version script with OpenCL versions
...
- use the same map as in:
https://github.com/KhronosGroup/OpenCL-ICD-Loader/blob/main/
loader/linux/icd_exports.map
this allows to skip loader and link directly with libigdrcl.so
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-07-31 20:18:05 +02:00
47009cec90
refactor: remove not needed code
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-07-31 17:07:56 +02:00
b33fe6ccf1
feature: adding flag to block dispatch implicit scaling commands
...
- this feature is part of making compute walker command view
- compute walker is programed for implicit scaling but not dispatched
- together with new flag, comes the refactor to reduce number of arguments
Related-To: NEO-11972
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2024-07-31 14:24:27 +02:00
b1a50104a8
fix: include dynamic SLM in clGetKernelWorkGroupInfo and zeKernelGetProperties
...
Current implementation only takes static slmInlineSize into account.
With this change we also include dynamic SLM passed as a kernel arguments.
Related-To: NEO-5761
Signed-off-by: Fabian Zwoliński <fabian.zwolinski@intel.com>
2024-07-30 17:59:45 +02:00
e668b4965c
performance: demote unrecoverable to debug_break
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-07-30 16:51:00 +02:00
65fcbff55c
refactor: Simplify code
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-07-29 14:26:51 +02:00
b1bc4f4cad
Revert "fix: Add missing fp64 extensions in caps initialization"
...
This reverts commit 9a486dd5a1
2024-07-26 14:47:02 +02:00
ace883ca55
performance: don't flush gpgpu if not required
...
Related-To: NEO-12124
If queue is OOQ and there are no cross-engine dependencies,
don't flush CCS before submitting copy on BCS.
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-26 06:49:45 +02:00
1cd00b5b89
fix: use per product cache line size to align heaps
...
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2024-07-24 17:29:20 +02:00
afee8814ef
refactor: get ioh alignment from static function
...
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2024-07-24 14:43:31 +02:00
a7fbc90ebd
fix: re-enable staging buffer copy when ccs is busy
...
Related-To: NEO-11501
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-22 18:14:46 +02:00
39ec7facee
performance: use BCS for transfers if CCS is busy
...
Related-To: NEO-11501
Also, if device is iGPU, don't use staging buffers
in that case.
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-22 15:36:26 +02:00
6a11e8a077
fix: revert changes around zero-copy
...
Related-To: NEO-12018
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-19 12:29:18 +02:00
b6299b8a21
feature: add support for HP copy engine context
...
- add support for contect group with HP copy engine
- choose HP copy engine when available
Related-To: NEO-11983
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-07-19 12:23:03 +02:00
f8867ac3ac
fix: Minor code improvements
...
Add explicit pointer checks to CommandQueue::blitEnqueueAllowed.
Explicitly check result of getDeviceArgValueIdx during ocloc compile.
Explicitly remove unused StagingBufferManager functions.
Move chunkCopyFunc by reference.
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2024-07-19 09:24:37 +02:00
7ad15639fc
Revert "feature: add support for HP copy engine context"
...
This reverts commit 3fbcbcaef2
2024-07-18 21:02:14 +02:00
33ab962121
fix: adjust compression hint usage for ocl buffers
...
Related-To: NEO-11989
Also, use zero-copy on lnl
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-18 18:24:48 +02:00
20d6910b66
performance: move usm pool init to first alloc call
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-07-18 16:07:22 +02:00
3fbcbcaef2
feature: add support for HP copy engine context
...
- add support for contect group with HP copy engine
- choose HP copy engine when available
Related-To: NEO-11983
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2024-07-18 16:07:07 +02:00
c1c9ac634b
performance(ocl): enable host usm alloc recycle
...
Enable at threshold of 2% system memory.
Related-To: NEO-6893
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-07-17 19:33:56 +02:00
8a60742a8d
fix: correct reported num subslices per slice in fused config
...
Related-To: NEO-12073
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-07-17 17:17:28 +02:00
fc9de71feb
fix(ocl): finish in release ogl object if needed
...
Finish cache flushes before exiting api call if releasing displayable
ogl object or dcflush is mitigated.
Related-To: NEO-11694
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-07-16 11:21:32 +02:00
9a6403f3bc
Revert "refactor: Add dc flush mitigation infrastructure"
...
This reverts commit d6076941a8
2024-07-15 11:47:30 +02:00
d6076941a8
refactor: Add dc flush mitigation infrastructure
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-07-12 14:45:51 +02:00
e188de2489
fix: initialize page tables before access for TSP allocation in TBX mode
...
Related-To: NEO-8340
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-07-11 16:03:19 +02:00
b0a5f2cced
fix: Stop direct submission before signal GL event
...
Related-To: NEO-10556
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-07-11 13:45:42 +02:00
bdeccab7aa
fix: bcs enqueue after marker properly waits
...
For an example sequence of:
IOQ_1 -> enqueue copy, enqueue marker with waitlist (out event)
IOQ_2 -> enqueue marker with waitlist (event), enqueue copy
Add missing synchronization between the enqueue copies
Related-To: NEO-11694
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-07-11 10:36:18 +02:00
05eb4e7a0d
performance: add debug flag to disable l1 flush
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-07-11 10:09:46 +02:00
dbd96372be
performance: adjust staging buffer usage
...
Related-To: NEO-11928
Don't copy through staging buffer if dst usm allocation
was not used before and transfer would be splitted.
Also, don't use staging buffers for mapped ocl buffers.
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-10 10:19:18 +02:00
4cabc9e4d2
performance: remove not needed code.
...
events are already created with queued state.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2024-07-10 07:35:05 +02:00
90d1a210d8
refactor: remove not needed check
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2024-07-09 16:11:33 +02:00
778645c11e
feature: add initial support for Xe2 platforms
...
Related-To: NEO-8188, NEO-10774
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-07-09 10:30:27 +02:00
7d25965a78
performance: change buffer type for new coherency model
...
Related-To: NEO-11882
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-09 09:33:53 +02:00
bbeb5224d8
fix: Add IOH alignment to estimation
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-07-08 12:29:54 +02:00
4f4b8fed15
refactor: remove not needed code related to deprecated device ip version
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2024-07-05 09:54:48 +02:00
38872b7e1b
Revert "refactor: Add dc flush mitigation infrastructure"
...
This reverts commit 1cba900ad9
2024-07-04 08:20:18 +02:00
a96f2ea13a
performance: disable blit enqueue on LNL
...
Resolves: NEO-11471
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2024-07-04 05:41:31 +02:00
daeb6e897a
performance: limit barrier usage in non-usm copies
...
Related-To: NEO-11501
Don't emit barrier if non-usm copy through staging
buffers with OOQ and single transfer was issued.
No need to send barrier if there was single transfer -
there's nothing to aggregate
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-03 13:45:52 +02:00
b8246d806f
Revert "performance: change buffer type for new coherency model"
...
This reverts commit c979495265
2024-07-03 02:11:19 +02:00
aed96cede4
fix: flush barrier when profiling enabled
...
Related-To: NEO-10615
Signed-off-by: Morek, Szymon <szymon.morek@intel.com>
2024-07-02 13:25:07 +02:00
c979495265
performance: change buffer type for new coherency model
...
Related-To: NEO-11882
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-02 12:49:41 +02:00
53f9a48fce
fix: set start TS on cpu if TS packet write disabled
...
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-07-01 17:14:10 +02:00
9a486dd5a1
fix: Add missing fp64 extensions in caps initialization
...
In a scenario where fp64 emulation is both supported and enabled,
we want to include 2 additional extensions:
- cl_khr_fp64 to deviceExtensions
- __opencl_c_fp64 to openclCFeatures (OpenCL ver. 3.0)
Related-To: NEO-7611
Signed-off-by: Oskar Hubert Weber <oskar.hubert.weber@intel.com>
2024-07-01 13:09:13 +02:00
76e8be5c39
fix(ocl): handle gl sharing displayable textures
...
Displayable textures always need dc flush.
Related-To: NEO-11694
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2024-06-27 13:43:20 +02:00
70e52ce4d8
refactor: add seperate enablers for compute walker and gpgpu walker
...
Related-To: NEO-10641
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2024-06-27 10:27:48 +02:00
8ee92b840f
fix: don't set start timestamp on cpu for bcs
...
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2024-06-26 13:30:58 +02:00
Jaroslaw Warchulski