eb8cd33dc6
Do not flush tag update if already flushed
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-08-08 14:11:42 +02:00
ada9b5d4a9
Handle if L0 Event Memory is Shareable
...
- Properly check for IPC event handle flag to determine if the event
pool memory is sharable between processes.
- Given Host Visible Event Pool, a check is done to determine if the
Host memory can be shared between the processes.
- Enabled handling if Event Host Memory is shareable for DRM
- If Event Pool Memory is Not shareable, then retrieving the IPC Event
Pool Handle returns unsupported.
Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2022-08-05 00:11:05 +02:00
601ace6a25
Add unit test for append memory prefetch for regular command list
...
Related-To: NEO-6740
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-08-04 10:32:08 +02:00
98d776867f
Add initial support for KernelArgsBuffer allocation
...
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
2022-08-03 20:28:21 +02:00
046f9d95fc
zeCommandListAppendMemoryPrefetch with memory prefetch manager
...
Related-To: NEO-6740
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-08-02 20:07:31 +02:00
17f22990e6
Cleanup includes 4/n
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-28 09:44:48 +02:00
0603819b68
Cleanup includes 3/n
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-27 15:28:13 +02:00
d4d54f5093
Cleanup includes
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-07-25 09:58:38 +02:00
4ac6e09117
Do not reuse mem obj's allocation
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-07-19 11:42:17 +02:00
3e22f30ed5
clEnqueueMigrateMemINTEL with memory prefetch for KMD migrated shared allocation
...
Related-To: NEO-6740
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-07-14 17:35:22 +02:00
bdf0483068
Hide details of AllocationsList from header
...
Signed-off-by: Daniel Chabrowski <daniel.chabrowski@intel.com>
2022-07-14 12:19:19 +02:00
60c819bb72
Fix Memory leak in drm_memory_manager of OsHandleLinux
...
Add virtual deconstructor to OsHandle and deconstructor to OsHandleLinux
Add override keyword to destructor
Add overriding deconstructor to OsHandleWin
Add newline before private members
https://github.com/intel/compute-runtime/pull/550
Signed-off-by: Cameron S Murtagh <cameron.murtagh00@gmail.com>
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-07-13 11:23:48 +02:00
f53ae0a50e
Revert "Add debug flag for EOT WA"
...
This reverts commit cf3817e058
2022-07-09 01:21:56 +02:00
cf3817e058
Add debug flag for EOT WA
...
EOT WA requires allocating last 64KB of kernel heap and putting EOT
signature at the last 16 bytes of kernel heap
Related-To: NEO-7099
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-07-07 16:58:24 +02:00
6a9fcd38b1
Create KMD-migrated unified shared memory with multiple local memory regions
...
Remove the restriction on USM allocation created in a single local memory region
with latest KMD fix for cross tile migration thrashing b/t lmem (dii-3516)
Related-To: NEO-6909
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-07-04 13:33:23 +02:00
31dbc04f23
L0Debug - capture SurfaceState heaps
...
Related-To: NEO-7103
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2022-06-24 15:05:56 +02:00
6ab6e1abff
Fix mutex order for event task and move args to gpu
...
This commit fixes problem with untransfered shared usm memory to gpu
when there is submit to gpu trigerred by user event. Also there is a fix
for dead lock problem caused by mixed orders of locking mutexes in csr
and in direct submission controller.
Related-To: NEO-6762
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-06-21 11:28:25 +02:00
213dc2fe24
Make CPU copy for read buffer when host ptr is write combined on DG2
...
With this commit on DG2 32bit driver will check if passed host ptr for
clEnqueueReadBuffer is write combined memory. If check will be true copy
will be make on CPU.
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-06-13 21:23:21 +02:00
cdfe2ce8ad
Feature: Flag for device usm allocation reusing
...
With flag enabled, when app calls freeSVMAlloc on device usm allocation,
don't free it immediately but save it,
and try to use it on subsequent allocations.
This allocation cache will be trimmed if an allocation fails.
Related-To: NEO-6893
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-06-13 20:02:52 +02:00
39c1c4d530
Remove virtual padding support
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-06-08 12:42:44 +02:00
77dde01503
Pass canonized gpuAddress in setCpuPtrAndGpuAddress
...
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-07 10:05:18 +02:00
c51ce2a35c
Use GmmHelper as an object in freeGraphicsMemoryImpl
...
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-07 09:53:41 +02:00
81899c4477
Add canonized gpuAddress to GraphicsAllocation constructor
...
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-06 18:17:36 +02:00
dc1fe7d59a
Change MemoryPool to enum class
...
Use enum class for MemoryPool in GraphicsAllocation
This change will ensure that GA is constructed in the proper way
- Rename namespace for isSystemMemoryPool method
- Add method getMemoryPoolString for logging actual pool which is in used
- Remove wrong pattern in GraphicsAllocation constructor
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-06-02 12:46:15 +02:00
325db6a99c
Fix P2P support for implicit scaling
...
when using implicit scaling, 2 dma-buf handles, one per tile, are
needed to support dma access from peer.
Related-To: LOCI-3122
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-06-01 20:10:32 +02:00
ae56d50b4f
Pass canonized gpuAddress to GraphicsAllocation
...
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-30 16:54:38 +02:00
7463e1970b
Cleanup headers
...
Make TUs and headers self-contained, remove unused headers
Signed-off-by: Daniel Chabrowski <daniel.chabrowski@intel.com>
2022-05-18 11:42:06 +02:00
3f04769f07
style: configure readability-identifier-naming.FunctionCase
...
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-17 20:55:56 +02:00
4266f861ac
Make implicit flush for cross-device dependency
...
Related-To: NEO-6418
If there's a cross-device dependency, flush batched
submissions to avoid deadlock.
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2022-05-16 16:29:09 +02:00
9d31d36491
Disable cross-tile kmd migration for usm allocations
...
Ensure KMD migrations for USM allocations to occur between smem and lmem only
Related-To: NEO-6969
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-05-16 15:23:34 +02:00
819e0f5515
style: configure readability-identifier-naming.LocalVariableCase
...
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-16 12:39:44 +02:00
e9be9b64c6
clang-tidy configuration cleanup
...
Define single .clang-tidy configuration with all used checks and use
NOLINT to selectively silence tool. That way cleanup should be easier.
third_part/ has its own configuration that disables clang-tidy for this
folder.
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2022-05-11 14:02:04 +02:00
2fcda0a528
Refactor: Change decanonize method accessing point
...
Accessing decanonize method as a member of GmmHelper class object
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-11 12:57:02 +02:00
d935815d74
Move residency data from WddmAllocation to GraphicsAllocation
...
Related-To: NEO-6848
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 18:16:42 +02:00
943ad0e1eb
style: skip redundant unique_ptr::get function
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-05-10 13:22:40 +02:00
4f1e01d279
Create getGmmHelper function in MemoryManager
...
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 16:51:23 +02:00
1c366d1ec0
Refactor: Change canonize method accessing point
...
Accessing canonize method as a member of GmmHelper class object
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-05-09 14:16:31 +02:00
3f26f45c10
Add support for IPC handles with implicit scaling
...
When using implicit scaling, device allocations may have
more than one internal allocation created internally. In that case,
a separate dma-buf handle per internal allocation needs to be
exported.
So introduced two driver experimental extensions to export and
import more than one IPC handle:
- zexMemGetIpcHandles
- zexMemOpenIpcHandles
Related-To: LOCI-2919
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-05-09 00:38:17 +02:00
0a16dc6c47
Fix multi thread usage of external host alloc
...
This is fixed reupload of this commit after auto revert
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.
Resolves: NEO-6807
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-05-05 13:32:15 +02:00
00a1a14652
Revert "Fix multi thread usage of external host alloc"
...
This reverts commit 54eee2a88b
2022-04-28 17:42:07 +02:00
9b778863b4
Store GmmHelper in Gmm class
...
Store GmmHelper in Gmm class instead of GmmClientContext
Related-To: NEO-6523
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2022-04-27 15:45:49 +02:00
54eee2a88b
Fix multi thread usage of external host alloc
...
With this commit OpenCL will track if external host memory is used from
few threads and will secure to update task count in all threads before
destroing allocation.
Resolves: NEO-6807
Signed-off-by: Maciej Plewka <maciej.plewka@intel.com>
2022-04-26 15:31:24 +02:00
c9d61840a2
Allocate SBA buffers per HW context
...
- different physical storage for every HW context
- adds support for debugging with implicit scaling on
- reorganize tests
Relates-To: NEO-6883
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2022-04-15 10:01:28 +02:00
8d1ad5a4f3
Refactor: use stack vector for root device indices
...
Stack vector will not cause dynamic allocations in most circumstances
ie. number of root device indices not more than 16
Related-To: NEO-6837
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-14 14:05:42 +02:00
352583b9d9
Detect GPU hang in evictUnusedAllocations()
...
This change introduces checking of the return value
of wait function in case of blocking version of
evictUnusedAllocations(). Furthermore, it propagates
the error to the callers. It contains also ULTs.
Related-To: NEO-6681
Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
2022-04-13 15:39:02 +02:00
789587bacc
Return nullptr for gfx alloc mmap error
...
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2022-04-12 10:08:19 +02:00
29e4518407
Update formatting in dynamic memory tracking
...
Format as follows:
backtrace start
call_1
...
call_n
backtrace end
Related-To: NEO-6837
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-08 13:17:30 +02:00
fffcf2612e
Flush tag only if needed in deferred deleter
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-04-07 15:36:29 +02:00
7a3976ad64
Revert "Force 64KB page size for cpu alignment in dual storage allocation"
...
This reverts commit 7ff6a5c1fa
2022-04-07 00:45:07 +02:00
7ff6a5c1fa
Force 64KB page size for cpu alignment in dual storage allocation
...
There is no need to force 2MB alignment for CPU allocation in dual
storage usage. Additionaly for WSL this will allow to avoid usage of
malloc in driver path.
Relates-To: NEO-6620
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2022-04-06 15:28:17 +02:00
8d5c674110
Dynamic memory tracking, update function printing
...
Print only demangled name if it succeeded.
Related-To: NEO-6837
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-06 14:34:24 +02:00
f7a767c767
Add demangling to dynamic memory tracking
...
Related-To: NEO-6837
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-04-05 20:38:27 +02:00
88193cc242
Minor fixes
...
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2022-03-31 20:02:11 +02:00
0c0603966b
Add debug functionality to track dynamic allocations.
...
- available only via manual build with ENABLE_DYNAMIC_MEMORY_TRACKING.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-03-31 13:24:15 +02:00
91cfd3cd1a
Revert "Unify command/ring/semaphore buffers placement"
...
This reverts commit e035199de4
2022-03-30 14:05:47 +02:00
e035199de4
Unify command/ring/semaphore buffers placement
...
put them all to the same memory location
Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-29 17:55:48 +02:00
a230f267e1
Poll task count indefinitely on high throttle command queue
...
Resolves: NEO-6781
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-03-25 10:06:16 +01:00
8a8b4866cb
XeHPC: force local memory for command/ring/semaphore buffer
...
require 48bit resource for ring/semaphore buffer
for multi tile allocations select first tile
for single tile allocation select preferred tile
Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-24 10:54:26 +01:00
f03f530327
Extend zeCommandListAppendMemoryPrefetch to migrate to associated device
...
Related-To: NEO-6740
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-23 14:21:17 +01:00
3792481d33
XeHPC Implicit scaling: put command/ring/semaphore buffer to first memory bank
...
In direct submission scenario command/ring/semaphore buffer allocations
are placed in the same memory bank to ensure that their memory is updated in
correct order
Related-To: NEO-6698
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2022-03-21 08:10:50 +01:00
754efbdaff
Cleanup mocks
...
Signed-off-by: Daniel Chabrowski daniel.chabrowski@intel.com
Related-To: NEO-6591
2022-03-18 11:54:03 +01:00
c0e2251ceb
Skip adding allocations to remote devices if not allocated there
...
When making graphics allocations resident in multi-GPU scenarios,
we should make them resident only if there's an allocation for that
device. So return appropriate null pointer and skip it.
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-16 23:58:07 +01:00
a71b88fefb
PageFaultHandler: speedup UM allocations lookup
...
Add a per-instance SVMAllocsManager::nonGpuDomainAllocs container for
all allocations to be removed in
moveAllocationsWithinUMAllocsManagerToGpuDomain. This approach replaces
the current iterative search and performs the task faster.
Add 7 new unit-tests to verify the functionality related to
nonGpuDomainAllocs container, both in expected and unexpected/synthetic
scenarios.
For UTs replace a dummy unifiedMemoryManager pointer with a pointer to
an instace of SVMAllocsManager, otherwise a SegFault error is thrown at
the end of tests.
Perform overall cleanup in related tests implementation, includes but
not limited to removal of:
- givenInitialPlacementGpu\
WhenMovingToGpuDomainThenFirstAccessDoesNotInvokeTransfer
As it is fully covered by:
givenAllocationMovedToGpuDomain\
WhenVerifyingPagefaultThenAllocationIsMovedToCpuDomain
- givenInitialPlacementGpu\
WhenVerifyingPagefaultThenFirstAccessDoesNotInvokeTransfer
As it is fully covered by:
givenTbxAndnitialPlacementGpu\
WhenVerifyingPagefaultThenMemoryIsUnprotectedOnly
Finally, reduce code duplication where possible.
Related-To: NEO-6658
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2022-03-16 11:18:24 +01:00
c0b7f05897
Add memory prefetch for kmd migrated shared allocations
...
This feature is disabled by default, controlled with the knob
AppendMemoryPrefetchForKmdMigratedSharedAllocations
Related-To: NEO-6740
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-03-09 16:02:18 +01:00
a010fb3634
Dont close shared handle on imported allocations
...
Related-To: LOCI-2272
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-03-06 04:35:47 +01:00
090bfb9642
Reuse kernel allocation
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-28 19:26:19 +01:00
e4a93f9870
Set page table cloning on for event device allocs
...
Related-To: NEO-5968
Signed-off-by: Aravind Gopalakrishnan <aravind.gopalakrishnan@intel.com>
2022-02-24 22:35:03 +01:00
0634aa3f1b
Create resource with given address
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-24 17:06:19 +01:00
c405fb5c2c
Use shared mutex for unified memory manager
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-02-23 19:00:38 +01:00
71f48c89c8
CacheSettingsHelper - timestamp allocations
...
Related-To: NEO-6664
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 19:32:39 +01:00
61ca84e94b
Pass memory placement info to StorageInfo
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-09 16:55:32 +01:00
a95198521e
Initial implementation of CacheSettingsHelper
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 16:18:06 +01:00
c88fce0def
Gmm construction cleanup
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-08 10:20:24 +01:00
4b0d986876
Move AllocationType enum out of GraphicsAllocation class
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2022-02-04 17:49:09 +01:00
45d23868de
Remove device enqueue part 15
...
- remove DEVICE_QUEUE_BUFFER
Related-To: NEO-6559
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
2022-02-03 16:19:07 +01:00
2be98a1e62
Create kmd migrated allocation with initial placement
...
Implements ZE_HOST_MEM_ALLOC_FLAG_BIAS_INITIAL_PLACEMENT
for zeMemAllocShared with KMD migrated allocation.
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2022-02-01 15:42:10 +01:00
ae77bd1bd2
Enable Device Memory to be shared in WSL-2 with L0
...
- Add getMemoryManagerType to check which memory manager has been init
to determine if Linux + WDDM memory manager is in use.
- Add isNTHandle to test and verify if a handle is an NT handle during
L0 Open IPC Handle.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-01-27 23:14:29 +01:00
a5067d6b0b
Remove duplicate check in residency container
...
Remove find in SVMAllocsManager
addInternalAllocationsToResidencyContainer,
not needed, CSR resolves duplicates at makeResident calls
Co-authored-by: Michal Mrozek <michal.mrozek@intel.com>
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-01-26 17:09:31 +01:00
592eb4607e
[4/n] Improve indirect allocations handling.
...
Add memory manager interfaces to obtain the status.
Only support when VmBind is supported.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-26 15:24:32 +01:00
dbf0f90186
Return pageSize in getMemAllocProperties
...
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-01-25 18:26:13 +01:00
6df17f5a30
[3/n] Optimize indirect allocations handling.
...
Add new debug variable to trigger new mode.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-25 16:40:56 +01:00
52d636394c
[1/n] Improve indirect allocations handling.
...
Add new functions that would treat all indirect allocations as single pack.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-25 06:27:33 +01:00
151aaf7678
Fix alignment for host allocations.
...
- it is 4k not 64k.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2022-01-24 13:24:03 +01:00
6082865eb4
Revert "Optimize Level Zero indirect allocations handling."
...
This reverts commit 3ecbc55ba9
2022-01-20 11:41:13 +01:00
1b7949432f
Add shareable allocation on windows dGPUs
...
Add default initialization for object members
Related-To: LOCI-2665
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2022-01-19 19:03:18 +01:00
3ecbc55ba9
Optimize Level Zero indirect allocations handling.
...
Make them resident directly instead of populating residency container
Remove finds, not needed, CSR resolves duplicates at makeResident calls
Observed gain is 32x for 10k indirect allocations.
Co-authored-by: Michal Mrozek <michal.mrozek@intel.com>
Co-authored-by: Dominik Dabek <dominik.dabek@intel.com>
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2022-01-19 13:08:35 +01:00
394c0e90e1
Return error when failing on submission
...
Signed-off-by: Raiyan Latif <raiyan.latif@intel.com>
2022-01-12 16:42:30 +01:00
7fc9b2c3dc
Implement UUID using pcie bdf
...
This patch implements the fallback method for Device UUID
using the BDF of the device for all product families.
Related-To: LOCI-2827
Signed-off-by: Joshua Santosh Ranjan <joshua.santosh.ranjan@intel.com>
2022-01-05 23:31:00 +01:00
3b497270c8
Add support for memory free policies
...
Add support for ZE_DRIVER_MEMORY_FREE_POLICY_EXT_FLAG_BLOCKING_FREE
added in v1.3.
Related-To: LOCI-2672
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-12-28 08:27:50 +01:00
e040c8c6bb
Initial compression support for L0 Device allocations
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-16 15:55:13 +01:00
2fd536104d
Extend GfxPartition to support 57b addressing
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-08 16:05:05 +01:00
dfdd3c597a
Remove redundant BUFFER_COMPRESSED allocation type
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-07 13:35:49 +01:00
0346a5679f
Fix returned local memory size
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-06 15:57:55 +01:00
68aea5bf62
Rename compression flags and helpers
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-03 18:09:02 +01:00
9481d8a88b
Compression refactor [2/n]
...
Dont use allocation type for compression preference
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-03 08:47:46 +01:00
3e1023fa1a
Unify memory layout for all multi tile post sync operations
...
Related-To: NEO-6262
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2021-12-02 18:00:40 +01:00
55959d4d1d
Helper method to check if allocation is compressed
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-02 16:13:53 +01:00
7b55d38e34
Refactor [1/n]: Make compression preference allocation property
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-12-02 15:24:35 +01:00
56c05a2f08
Update allocateGraphicsMemoryWithAlignment
...
Allocate Gmm object when the allocation uses compression.
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2021-11-09 13:37:32 +01:00
95610188af
Fallback to cpu copy when filling work partition allocation
...
move some command stream receiver tests to shared
Related-To: NEO-6325
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-11-03 12:17:02 +01:00
782a287b27
Revert "Fallback to cpu copy when filling work partition allocation"
...
This reverts commit 93e3d948f5
2021-10-29 04:39:22 +02:00
515130dbe8
Fail on blitMemoryToAllocation when blitter is not supported
...
Related-To: NEO-6325
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-28 16:13:40 +02:00
93e3d948f5
Fallback to cpu copy when filling work partition allocation
...
move some command stream receiver tests to shared
Related-To: NEO-6325
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-28 14:30:00 +02:00
5fdcb14d57
WSL fallback to staging buffer on blocking reads
...
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2021-10-28 03:34:58 +02:00
ba767dfbaf
Revert "Fail on blitMemoryToAllocation when blitter is not supported"
...
This reverts commit 7bfeeaab86
2021-10-27 09:13:57 +02:00
7bfeeaab86
Fail on blitMemoryToAllocation when blitter is not supported
...
Related-To: NEO-6325
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-26 11:05:51 +02:00
71ed3eba16
Add mapping based colouring
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-10-25 09:51:54 +02:00
d34ef3170a
Remove CPP_ATTRIBUTE_FALLTHROUGH macro
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-10-22 17:27:57 +02:00
d2a29ce458
DrmBO: canonize GPU addresses
...
Related-To: NEO-6364
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2021-10-21 18:25:48 +02:00
28b37aea72
Correct handling unique_ptr in functions
...
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
2021-10-21 18:17:14 +02:00
465bec3d76
Add NTHandle sharing
...
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2021-10-19 14:17:25 +02:00
f8c89fe984
Refactor memory manager so as to support device reset
...
Move out neoDevice dependent pieces of memory manager code into
separate methods. Those methods could be used for recreating a neoDevice
after a device reset is performed.
Related-To: LOCI-2615
Signed-off-by: Jitendra Sharma <jitendra.sharma@intel.com>
2021-10-19 07:16:29 +02:00
24ff191e61
Verify completion on all tiles
...
Related-To: NEO-6244
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2021-10-15 15:43:28 +02:00
0d74ab8ae8
Correct allocating event pool in device memory
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-13 17:36:46 +02:00
b1df167632
Move enable_product.inl to shared
...
extract api agnostic validators to shared
remove not needed opencl includes from neo shared
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-12 22:02:16 +02:00
5aeae0cf99
Add mem advise control flags to drm alocation
...
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2021-10-12 11:39:06 +02:00
06561a06b6
Restore 2MB size & alignment for CPU SVM.
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2021-10-11 23:16:42 +02:00
c8e79b7447
Change shared allocations required alignment to 64KB.
...
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2021-10-08 13:08:11 +02:00
7392b5fb8d
Add new colouring policy
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-10-07 23:33:26 +02:00
48f01f28f5
Move core part of MemoryPropertiesHelpers to shared
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-10-07 23:14:38 +02:00
afa45bd9e7
Add support for mem advise to set cache policy in buffer object
...
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2021-10-07 22:16:58 +02:00
f1c24ddcbb
WDDM - guard against duplicated host ptr allocs
...
Signed-off-by: Jaroslaw Chodor <jaroslaw.chodor@intel.com>
2021-10-06 22:33:40 +02:00
30b3f5cdb0
Move PageTableManager from rootDeviceEnvironment to CommandStreamReceiver
...
Also add init of pageTable registers to blitter path
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
2021-09-30 12:59:11 +02:00
a63b9b1273
move common files to shared/
...
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
2021-09-27 11:10:20 +02:00
d4a4c35759
Zero-initialize debug surface
...
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2021-09-22 14:36:36 +02:00
dc8b92d8a6
Move heapInLocalMem function from HwHelper to HwInfoConfig
...
Signed-off-by: Rafal Maziejuk <rafal.maziejuk@intel.com>
Related-To: NEO-4541
2021-09-22 11:44:44 +02:00
abbe98be07
Fix allocating graphics memory for patterns in appendMemoryFill
...
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
2021-09-16 16:20:10 +02:00
522d2550ee
Add allocation ID to be retrieved by L0 getMemAllocProperties
...
zeMemGetAllocProperties() specifies an ID for the allocation can be
retrieved by the application. This can be used by applications to
identify the allocation throughout the application lifetime.
We were currently returning as id the same allocation address. This
patch fixes it to return a true unique identifier for the allocation.
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-09-15 19:22:50 +02:00
9ccf43e441
Correct branch_dir_suffix in cmake
...
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
Related-To: NEO-6245
2021-09-14 16:00:20 +02:00
bbfbf19a02
Move allowStatelessCompression from HwHelper to HwInfoConfig
...
Signed-off-by: Rafal Maziejuk <rafal.maziejuk@intel.com>
Related-To: NEO-4541
2021-09-09 11:37:47 +02:00
8d3438a5fb
Allocate single Isa instance when debugging enabled
...
- Related-To: NEO-6221
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2021-09-08 11:49:50 +02:00
f2eb7f3aea
Fix DirectSubmission residency handling
...
- allocations should be resident within OsContext
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2021-09-03 16:57:42 +02:00
4390590831
Add shareable allocation for Windows
...
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2021-08-26 13:43:55 +02:00
bf1180753d
Improve private memory allocation.
...
-allocate from single bank when only one sub device passed
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2021-08-26 07:46:23 +02:00
f1264c19dc
Revert "Improve private allocation storage."
...
This reverts commit be3ca800bb
2021-08-20 02:54:09 +02:00
be3ca800bb
Improve private allocation storage.
...
- Do not allocate on multi tile if device limits scope.
Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2021-08-18 15:02:14 +02:00
5c236a62fd
Changes to SW tags
...
Add SW tags to synchronization points
Add ID sequence numbers
Add new allocation type
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-08-18 01:51:37 +02:00
49cf6f79f5
Validate host ptr on VM it will be used
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-08-16 08:15:53 +02:00
fb996a3d5b
Add EngineDescriptor helper
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-08-12 15:19:46 +02:00
2a7757651a
Change percent of global memory returned
...
Related-To: NEO-5796
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2021-08-11 13:33:16 +02:00
2d5d071e66
Extended import device memory functionality
...
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2021-08-10 13:24:56 +02:00
94c97fc14c
Allowing the creation of overlapping buffers
...
Related-To: NEO-5871
Signed-off-by: Andrzej Koska <andrzej.koska@intel.com>
2021-08-10 10:59:29 +02:00
ab73b61b77
Fix multitile ISA placement
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-08-02 15:16:13 +02:00
eb14d8458b
Add helper function to enable stateless compression
...
Related-To: NEO-5107
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2021-08-02 14:34:33 +02:00
d99f20d400
Move semaphore buffer to system memory
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-08-02 14:24:12 +02:00
b057935c7c
Associate shared-allocations with device in multi-device systems
...
When a shared-allocation is created in a multi-device system and
no device is passed during the call, then the allocation needs to be
associated with all the devices. For now, that means we default to
creating the shared-allocation in the host, with no device-backing
memory, so all devices can access it.
However, if a device is passed, then the shared-allocation is
expected to be accessed only by that device, and there is no need
to allocate it solely in the host.
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-07-29 20:28:27 +02:00
575445dbb5
Minor code cleanup
...
prevent nullptr dereference
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-07-22 19:25:23 +02:00
f22251f68a
Add remaining external allocation types for local bank selector
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-07-22 16:57:37 +02:00
890eec6105
Separate local memory usage tracking for internal and external
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-07-22 07:52:43 +02:00
6f3c89decb
Add mechanism to migrate multi root device memory
...
invalidate TLB cache if kernel requires migration
Related-To: NEO-3691
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-07-19 19:49:22 +02:00
e939e97384
Dont replicate GPU_TIMESTAMP_DEVICE_BUFFER
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-07-16 15:23:50 +02:00
3c88492229
Revert "Extended import device memory"
...
This reverts commit ea6555e788c98314160a11898212c2d664999705.
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-07-16 09:56:52 +02:00
fad4ee7e24
Enabling WDDM_LINUX by default
...
Based on libdxg headers, we're able
to support WDDM_LINUX (WSL2) in our
default build configuration
Signed-off-by: Jaroslaw Chodor <jaroslaw.chodor@intel.com>
2021-07-14 12:42:29 +02:00
d5fdb949eb
Extended import device memory
...
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
2021-07-07 16:12:36 +02:00
9166f9223a
Device pool allocation in OsAgnosticMemoryManager
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-07-07 13:09:12 +02:00
449c0dc856
Do not register allocation on fail
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-07-05 19:23:58 +02:00
c7a936d1f4
Add memory banks to Simulated CSR
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-07-05 12:19:58 +02:00
850b38781d
Move RING_BUFFER to system memory
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-06-29 16:24:43 +02:00
73b0df3211
Disable gem close worker in CSR constructor
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-06-29 15:22:51 +02:00
96d14967ac
Partial support for XE_HP_SDV
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-06-24 18:35:54 +02:00
ea214dd26b
Select lowest subdevice when USM memory is allocated in not multicontext
...
Related-To: NEO-5498
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2021-06-21 18:45:11 +02:00
48feca4f44
Check Isa placement for every root device
...
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2021-06-21 11:53:19 +02:00
5e29dccddc
Add IPC events support
...
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-06-08 08:11:15 +02:00
8667f3eafa
Remove incorrect check for Shared USM creation
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-06-07 14:48:33 +02:00
13b2c326fe
Add AlignmentSelector helper
...
Related-To: NEO-5750
Signed-off-by: Maciej Dziuban <maciej.dziuban@intel.com>
2021-06-01 11:44:04 +02:00
9dbffa8d93
Refactoring createMemoryManager
...
Signed-off-by: Jaroslaw Chodor <jaroslaw.chodor@intel.com>
2021-05-25 16:58:21 +02:00
0f32231fc8
Use correct INTERNAL heap base address for ISA in system memory
...
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2021-05-25 10:22:28 +02:00
33e8f73775
Enable custom allocation alignments on Linux
...
Related-To: NEO-5750
Signed-off-by: Maciej Dziuban <maciej.dziuban@intel.com>
2021-05-24 12:56:51 +02:00
21690dcea5
Fix debugSurface initialization
...
Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2021-05-24 12:28:23 +02:00
ed31aaedb8
Make BUFFER allocation lockable on DG1
...
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-21 17:05:51 +02:00
c59f44ecdf
Collision avoidance on 36 bit address space
...
Signed-off-by: Andrzej Koska <andrzej.koska@intel.com>
Related-To: NEO-5356
2021-05-21 11:20:35 +02:00
b50a6bec82
Correct isBlitCopyRequiredForLocalMemory
...
detect not lockable allocation based on gmm flag
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-20 13:17:49 +02:00
abf0649a5b
Improving cpp compliance
...
Signed-off-by: Jaroslaw Chodor <jaroslaw.chodor@intel.com>
2021-05-20 00:10:06 +02:00
a5d38170ad
Move some shared tests to shared
...
Tests that are not specific to openCL or level zero
Related-To: NEO-5161
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2021-05-18 13:51:18 +02:00
3bd4bca911
Copyright header update
...
Dates corrected in copyright headers to reflect original publication date
(2018 for OpenCL, 2020 for Level Zero).
Signed-off-by: lgotszal <lukasz.gotszald@intel.com>
2021-05-17 20:38:19 +02:00
a304e609fe
Respect forceSystemMemory flag at the end of getAllocationData method
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-14 10:58:23 +02:00
b7c7bc0a24
Correct createMultiGraphicsAllocationInSystemMemoryPool method
...
force system memory placement
skip duplicated root device indices
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-13 12:11:49 +02:00
fcbf1dcf32
Mark SVM_GPU allocation as not lockable
...
require blitter usage if allocation is not lockable and in local memory
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-12 17:47:15 +02:00
296634f5b8
Fallback to CPU copy when blit copy fails
...
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-12 15:38:24 +02:00
a552d1fd65
Revert "Mark SVM_GPU allocation as not lockable"
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-12 12:24:02 +02:00
a79f7b678b
Mark SVM_GPU allocation as not lockable
...
require blitter usage if allocation is not lockable and in local memory
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-05-12 09:49:35 +02:00
cffac84a3b
Simplify creating EventPool allocation
...
Signed-off-by: Bartosz Dunajski <bartosz.dunajski@intel.com>
2021-05-11 18:26:54 +02:00
544890c066
Change default placement of preemption allocation
...
Related-To: NEO-5800
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2021-05-11 15:26:24 +02:00
8e1e213bcf
Disable GEM_WAIT when new residency model available
...
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2021-05-10 14:59:21 +02:00
c4502ccf66
Fix OCL MOCS index with UNCACHED flag
...
Related-To: NEO-5742
Signed-off-by: Konstanty Misiak <konstanty.misiak@intel.com>
2021-05-04 10:01:44 +02:00
c453e7fe8b
Revert "Mark SVM_GPU allocation as not lockable"
...
This reverts commit 5341d0663c
2021-04-29 18:06:38 +02:00
5341d0663c
Mark SVM_GPU allocation as not lockable
...
use blitter if available and allocation is not lockable
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-29 15:55:13 +02:00
de116aac5c
Make buffer allocations not lockable
...
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-29 11:27:44 +02:00
bb9d902899
Add padding for ISA allocations
...
Related-To: NEO-5771
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2021-04-27 11:03:58 +02:00
902cce597a
Add new functionality to load SIP from file
...
Related-To: NEO-5718
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2021-04-23 13:48:26 +02:00
495841812f
Free allocations in 2MB heaps
...
Related-To: NEO-5750
Signed-off-by: Maciej Dziuban <maciej.dziuban@intel.com>
2021-04-23 09:07:37 +02:00
9a22d06efe
Store default engine index per root device
...
Resolves: NEO-5768
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-20 14:55:13 +02:00
128cd8a31c
Add support for non-IPC P2P access to L0
...
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-04-20 01:05:40 +02:00
2b956651a7
Mark Shared resource copy allocations as lockable
...
Related-To: NEO-5733
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-19 11:16:34 +02:00
c540af8e5f
Fix multidevice support for events
...
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2021-04-15 22:12:07 +02:00
9026362b76
Add GraphicsAllocation::isLockable method
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-15 12:14:03 +02:00
c9b5de0f1d
Revert "Add GraphicsAllocation::isLockable method"
...
This reverts commit 48d40ad850
2021-04-15 07:58:35 +02:00
48d40ad850
Add GraphicsAllocation::isLockable method
...
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2021-04-14 16:12:51 +02:00
fd9fb3e46b
Remove isInitialized() method from OsContext
...
We can fail early, instead of constructing an invalid object
Signed-off-by: Maciej Dziuban <maciej.dziuban@intel.com>
2021-04-13 14:12:20 +02:00
5f9dbef401
Pass alignment to Gmm
...
Related-To: NEO-5750
Signed-off-by: Maciej Dziuban <maciej.dziuban@intel.com>
2021-04-13 13:40:43 +02:00
Lukasz Jobczyk