Commit Graph

494 Commits

Author SHA1 Message Date
Compute-Runtime-Validation
b2b41e613b Revert "fix: add alignment support to host and shared UnifiedMemoryAllocation"
This reverts commit c3df92ac41.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-05-12 13:04:08 +02:00
Mateusz Jablonski
f3a89bfe07 fix: add missing settings of isLockable flag in WddmMemoryManager class
- allocateGraphicsMemoryUsingKmdAndMapItToCpuVA in case of no compression
- allocate32BitGraphicsMemoryImpl in case of allocate by KMD

remove redundant ctor of StorageInfo class

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-05-11 12:14:18 +02:00
Lu, Wenbin
c3df92ac41 fix: add alignment support to host and shared UnifiedMemoryAllocation
Related-To: LOCI-4334

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-05-11 11:45:12 +02:00
Mateusz Jablonski
1948781b67 fix: set isLockable flag in storage info even if no memory banks are used
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-05-11 07:26:28 +02:00
Lu, Wenbin
5d653c8536 fix: Add alignment support to createUnifiedMemoryAllocation
Allows the user to use alignments > 64KB in `createUnifiedMemoryAllocation`

So that the restriction in `piextUSMDeviceAlloc` of the DPC++ runtime
could be lifted

Related-To: LOCI-4168

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-05-02 09:19:23 +02:00
Fabian Zwolinski
cbce863dc2 refactor: Rename member variables to camelCase 3/n
Additionally enable clang-tidy check for member variables

Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-28 16:01:14 +02:00
Mateusz Jablonski
fd1ad7c1f0 feature: setup heap extended host size based on system memory size
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-28 15:23:01 +02:00
Mateusz Jablonski
5a5c20f99c fix: create separate heap for host and shared usm in 48-56b VA
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-28 10:09:38 +02:00
Fabian Zwolinski
e351a90f81 refactor: Rename member variables to camelCase 2/n
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
2023-04-27 20:39:22 +02:00
Mateusz Jablonski
2f9135a4e6 fix: change type of container with registered engines per root device
use StackVec instead of unordered map
resize container at MemoryManager's creation time

Related-To: NEO-7925
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-27 17:06:42 +02:00
Mateusz Jablonski
32d8a3bc6d fix: store registered engines per root device
in most cases we need to iterate over engines associated to single root device

Related-To: NEO-7925
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-04-27 10:54:07 +02:00
Spruit, Neil R
364c2da9fb feature: Add Support for zeMemPutIpcHandle & zeMemGet IPC Handle converters
Related-To: LOCI-4172, LOCI-4305, LOCI-4306

- Create a new IPC Memory handle upon call to getIpcMemHandle if the
previous handle has been freed.
- Release the Ipc Memory Handle when zeMemPutIpcHandle is called.
- Create a new IPC Handle for tracking thru zeMemGetAllocProperties
when ze_external_memory_export_fd_t is used.
- Convert FD to opaque IPC handle and IPC Handle to FD.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-04-27 03:53:52 +02:00
Krzysztof Gibala
16db7cc890 fix: Add missing checks in multi gpu scenario
- Check allocation root device index during eviction
- Wait for and marked allocation only from the current root device index

Related-To: NEO-7920
Signed-off-by: Krzysztof Gibala <krzysztof.gibala@intel.com>
2023-04-24 23:26:28 +02:00
Maciej Bielski
d4de925711 refactor: add isKernelIsaAllocationType helper
Preparatory step, the helper will be also used by following changes.

Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
2023-04-24 17:16:11 +02:00
rcombs
fc53bbe183 unified_memory_manager: mark comparison operators as const
This fixes the build on some compilers.

Signed-off-by: rcombs <rcombs@rcombs.me>
2023-04-24 10:47:30 +02:00
Compute-Runtime-Validation
858ed05466 Revert "[fix] zeCommandListAppendMemAdvise to clear preferred location correc...
This reverts commit 63470ed698.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-04-20 11:04:34 +02:00
Milczarek, Slawomir
63470ed698 [fix] zeCommandListAppendMemAdvise to clear preferred location correctly
The memadvise with preferred location for kmd-migrated shared allocation
is set to device associated with cmd list by default to migrate data
to lmem on non-atomic gpu page fault too (for performance reasons).

Related-To: NEO-7252

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-19 09:55:32 +02:00
Milczarek, Slawomir
01d03aa5b6 Extended regkey to force prefetch of shared memory in enqueue commands
Extended the regkey ForceMemoryPrefetchForKmdMigratedSharedAllocations
to force meory prefetch of kmd-migrated shared allocation
in clEnqueueNDRangeKernel(), clEnqueueMemFillINTEL, ...

Related-To: NEO-7841

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-04-11 11:23:48 +02:00
John Falkowski
e056082710 refactor graphics allocation structure elements for sub-allocation properties
Resolves:  LOCI-3772

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-07 16:53:23 +02:00
Dominik Dabek
1c52017ceb fix: use correct allocation type in program init
Globals surface allocation via USM manager will have correct allocation
type set (instead of just BUFFER) and will use cpu copy when possible.

Related-To: NEO-7796

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-04-04 11:31:11 +02:00
Compute-Runtime-Validation
2b93126795 Revert "Add alignment support to createUnifiedMemoryAllocation"
This reverts commit ca02bbba4b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 15:43:47 +02:00
Mateusz Jablonski
8119905325 fix usm shared: propagate resource48Bit when allocating with kmd migrations
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-30 13:00:40 +02:00
Mateusz Jablonski
5d482ffe87 refactor: add constexpr for tera byte
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-28 12:55:52 +02:00
Mateusz Jablonski
7e5e27f0b9 feature usm: reserve heap extended in 57 bit address space
use mocked mmap function in unit tests

Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-28 11:06:03 +02:00
Lu, Wenbin
ca02bbba4b Add alignment support to createUnifiedMemoryAllocation
Allows the user to use alignments > 64KB in `createUnifiedMemoryAllocation`

So that the restriction in `piextUSMDeviceAlloc` of the DPC++ runtime
could be lifted

Related-To: LOCI-4168

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-03-28 10:57:04 +02:00
Igor Venevtsev
062abef8ef Revert "Deferred deletion of allocations in main thread"
This reverts commit 2ea734491a.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2023-03-20 18:17:15 +01:00
John Falkowski
a1e2eca9e8 Add zeMemGetAllocProperties extension for sub-allocations
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-03-17 21:21:44 +01:00
Kamil Kopryk
fa8579602f refactor: rename product helper files n/n
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-10 13:24:38 +01:00
Mateusz Hoppe
37dbec305d feature: add AssertHandler
- initial implementation to support assert() on GPU

Related-To: NEO-5753

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-08 17:55:23 +01:00
Spruit, Neil R
86e739e9dc feature: Relaxed Virtual Memory reservation with pStart
Related-To: LOCI-3871

- Relaxed the Virtual Memory Reservation to allow pStart and not fail if
the pStart value is not obtained.
- Moves checks on pStart to the user to check and determine if they want
to re-reserve or use the address allocated.
- Changed reserveGpuAddress to use unit64_t type to allow internal
address range structure assignment without cast.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-08 08:06:50 +01:00
Mateusz Jablonski
8194543ae5 refactor raytracing: unify RT programming logic
reuse EncodeEnableRayTracing in CommandStreamReceiver
add method to determine need for 48b resource flag for RT allocations
Related-To: NEO-7606

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-07 11:51:58 +01:00
Cencelewska, Katarzyna
50da32ffb1 wa: add dummy blits before command MI_ARB_CHECK
to guarantee that all subblt got complete for previous copy
affect xe hpg

Related-To: NEO-7450

Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
2023-03-07 10:21:05 +01:00
Milczarek, Slawomir
674691d170 Fix check for page faults enabled in get memory access properties
Related-To: NEO-6465

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-02-28 10:13:25 +01:00
Lukasz Jobczyk
2f5be7a48d Copy command buffer into ring buffer
Resolves: NEO-7422

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-02-22 16:37:34 +01:00
Wrobel, Patryk
d8a65c6958 Try to use provided pointer when dual storage shared memory is not supported
If user provided not-null hostptr field, then the driver
should try to use it. This change adds omitted functionality,
which handles the described case also in createUnifiedMemoryAllocation().

Related-To: NEO-7600
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-21 09:08:11 +01:00
Zbigniew Zdanowicz
b4544f6f78 Fix dependency resolve in level zero black box tests
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-02-21 08:25:07 +01:00
Warchulski, Jaroslaw
b485c025d0 Cleanup includes 57
Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-17 11:19:59 +01:00
Neil Spruit
2aaebddb37 Revert "Heap allocation allow base address and growable addresses"
This reverts commit 44ec497b1a.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 18:34:40 +01:00
Warchulski, Jaroslaw
8c17313c8b Cleanup includes 53
Cleaned up files:
opencl/source/mem_obj/image.inl
shared/offline_compiler/source/decoder/zebin_manipulator.h
shared/source/aub_mem_dump/aub_alloc_dump.h
shared/source/compiler_interface/intermediate_representations.h
shared/source/helpers/blit_commands_helper_base.inl
shared/source/utilities/debug_file_reader.h
shared/source/utilities/software_tags.h
shared/source/xe_hpc_core/hw_cmds_pvc.h

Related-To: NEO-5548

Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-15 13:21:14 +01:00
Mateusz Jablonski
51887af219 refactor OCL: cleanup after moving multi root device buffers to local mem
remove debug flag AllocateBuffersInLocalMemoryForMultiRootDeviceContexts

Related-To: NEO-5735
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-02-15 09:41:20 +01:00
Spruit, Neil R
44ec497b1a feature: Allow for Allocating a base address in the heap and grow an allocation
Related-To: LOCI-3871

- Enabled allocation of specified base address in the targeted heap.
- Enabled virtual memory reservations to grow by allocating at the start
of the heap vs the end of the heap.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 08:13:38 +01:00
Wrobel, Patryk
4c58eda90d Ensure that BO handle is closed only once
When one process had exported and then opened IPC handle
of memory, then close function was called twice for the
same BO handle. It caused debugBreak() and aborted
an application.

This change allows multiple separate BOs to share one
handle. The last shared handle owner calls close() function.

Related-To: NEO-7200
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-13 14:42:49 +01:00
Warchulski, Jaroslaw
a2e6a8284b Cleanup includes 47
Cleaned up files:
level_zero/tools/source/debug/windows/debug_session.h
level_zero/tools/source/sysman/memory/windows/os_memory_imp.h
level_zero/tools/source/sysman/windows/kmd_sys_manager.h
opencl/test/unit_test/aub_tests/command_stream/copy_engine_aub_tests_xehp_and
shared/source/command_container/command_encoder.inl
shared/source/command_stream/command_stream_receiver_hw_xehp_and_later.inl
shared/source/helpers/blit_commands_helper_base.inl
shared/test/unit_test/image/image_surface_state_fixture.h
shared/test/unit_test/os_interface/windows/os_interface_win_tests.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-10 17:07:30 +01:00
Kacper Nowak
7790e208fd feat(zebin): Add support for ELF section type SHT_NOBITS
This commit adds support for parsing SHT_NOBITS zebin's ELF sections
(containing global/constant zero-initialized data).
- Correction: in CTNI path, do not add related symbol if surface has not
been allocated.

Related-To: NEO-7196
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
2023-02-10 16:17:16 +01:00
Compute-Runtime-Validation
d0c0c60205 Revert "feat(zebin): Add support for ELF section type SHT_NOBITS"
This reverts commit fa03aa9a40.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-02-03 03:44:02 +01:00
Kacper Nowak
fa03aa9a40 feat(zebin): Add support for ELF section type SHT_NOBITS
This commit adds support for parsing SHT_NOBITS zebin's ELF sections
(containing global/constant zero-initialized data).

Related-To: NEO-7196
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
2023-02-02 14:54:51 +01:00
Mateusz Jablonski
b931f31f3d debug: Reduce debug logs when changing allocation type
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-02-02 11:05:46 +01:00
Kamil Kopryk
2484c7ceb2 refactor: rename hw_helper files to gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-01 19:37:51 +01:00
Michal Mrozek
2f847aa7bb Optimize zeMemGetAllocProperties.
- No functional changes only performance optimizations.
- add const modifiers
- add early returns
- remove not needed code
- do fast lookup for aligned pointers
- fix incorrect tests

+21% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=100 --iterations=1000
+32% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=1000 --iterations=1000

Related-To: NEO-7676

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2023-01-30 18:00:57 +01:00
Spruit, Neil R
b5692c04cd Support Mapping Multiple Physical Memory with one VA and Bug Fix
- Added support for mapping any portion of a virtual allocation to a
physical mapping with a lookup function for reserved virtual addresses.
- Added support for multiple mappings linked to the same virtual
reservation.
- Fixed bug with 64 bit addresses on windows with invalid addresses
passed to the user.

Related-To: LOCI-3904, LOCI-3914, LOCI-3931

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-01-30 02:36:10 +01:00