Commit Graph

525 Commits

Author SHA1 Message Date
Dominik Dabek
1c52017ceb fix: use correct allocation type in program init
Globals surface allocation via USM manager will have correct allocation
type set (instead of just BUFFER) and will use cpu copy when possible.

Related-To: NEO-7796

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
2023-04-04 11:31:11 +02:00
Compute-Runtime-Validation
2b93126795 Revert "Add alignment support to createUnifiedMemoryAllocation"
This reverts commit ca02bbba4b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 15:43:47 +02:00
Mateusz Jablonski
8119905325 fix usm shared: propagate resource48Bit when allocating with kmd migrations
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-30 13:00:40 +02:00
Mateusz Jablonski
5d482ffe87 refactor: add constexpr for tera byte
Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-28 12:55:52 +02:00
Mateusz Jablonski
7e5e27f0b9 feature usm: reserve heap extended in 57 bit address space
use mocked mmap function in unit tests

Related-To: NEO-7665
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-28 11:06:03 +02:00
Lu, Wenbin
ca02bbba4b Add alignment support to createUnifiedMemoryAllocation
Allows the user to use alignments > 64KB in `createUnifiedMemoryAllocation`

So that the restriction in `piextUSMDeviceAlloc` of the DPC++ runtime
could be lifted

Related-To: LOCI-4168

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-03-28 10:57:04 +02:00
Igor Venevtsev
062abef8ef Revert "Deferred deletion of allocations in main thread"
This reverts commit 2ea734491a.

Signed-off-by: Igor Venevtsev <igor.venevtsev@intel.com>
2023-03-20 18:17:15 +01:00
John Falkowski
a1e2eca9e8 Add zeMemGetAllocProperties extension for sub-allocations
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-03-17 21:21:44 +01:00
Kamil Kopryk
fa8579602f refactor: rename product helper files n/n
Related-To: NEO-7703
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-03-10 13:24:38 +01:00
Mateusz Hoppe
37dbec305d feature: add AssertHandler
- initial implementation to support assert() on GPU

Related-To: NEO-5753

Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
2023-03-08 17:55:23 +01:00
Spruit, Neil R
86e739e9dc feature: Relaxed Virtual Memory reservation with pStart
Related-To: LOCI-3871

- Relaxed the Virtual Memory Reservation to allow pStart and not fail if
the pStart value is not obtained.
- Moves checks on pStart to the user to check and determine if they want
to re-reserve or use the address allocated.
- Changed reserveGpuAddress to use unit64_t type to allow internal
address range structure assignment without cast.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-08 08:06:50 +01:00
Mateusz Jablonski
8194543ae5 refactor raytracing: unify RT programming logic
reuse EncodeEnableRayTracing in CommandStreamReceiver
add method to determine need for 48b resource flag for RT allocations
Related-To: NEO-7606

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-07 11:51:58 +01:00
Cencelewska, Katarzyna
50da32ffb1 wa: add dummy blits before command MI_ARB_CHECK
to guarantee that all subblt got complete for previous copy
affect xe hpg

Related-To: NEO-7450

Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
2023-03-07 10:21:05 +01:00
Milczarek, Slawomir
674691d170 Fix check for page faults enabled in get memory access properties
Related-To: NEO-6465

Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
2023-02-28 10:13:25 +01:00
Lukasz Jobczyk
2f5be7a48d Copy command buffer into ring buffer
Resolves: NEO-7422

Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2023-02-22 16:37:34 +01:00
Wrobel, Patryk
d8a65c6958 Try to use provided pointer when dual storage shared memory is not supported
If user provided not-null hostptr field, then the driver
should try to use it. This change adds omitted functionality,
which handles the described case also in createUnifiedMemoryAllocation().

Related-To: NEO-7600
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-21 09:08:11 +01:00
Zbigniew Zdanowicz
b4544f6f78 Fix dependency resolve in level zero black box tests
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-02-21 08:25:07 +01:00
Warchulski, Jaroslaw
b485c025d0 Cleanup includes 57
Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-17 11:19:59 +01:00
Neil Spruit
2aaebddb37 Revert "Heap allocation allow base address and growable addresses"
This reverts commit 44ec497b1a.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 18:34:40 +01:00
Warchulski, Jaroslaw
8c17313c8b Cleanup includes 53
Cleaned up files:
opencl/source/mem_obj/image.inl
shared/offline_compiler/source/decoder/zebin_manipulator.h
shared/source/aub_mem_dump/aub_alloc_dump.h
shared/source/compiler_interface/intermediate_representations.h
shared/source/helpers/blit_commands_helper_base.inl
shared/source/utilities/debug_file_reader.h
shared/source/utilities/software_tags.h
shared/source/xe_hpc_core/hw_cmds_pvc.h

Related-To: NEO-5548

Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-15 13:21:14 +01:00
Mateusz Jablonski
51887af219 refactor OCL: cleanup after moving multi root device buffers to local mem
remove debug flag AllocateBuffersInLocalMemoryForMultiRootDeviceContexts

Related-To: NEO-5735
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-02-15 09:41:20 +01:00
Spruit, Neil R
44ec497b1a feature: Allow for Allocating a base address in the heap and grow an allocation
Related-To: LOCI-3871

- Enabled allocation of specified base address in the targeted heap.
- Enabled virtual memory reservations to grow by allocating at the start
of the heap vs the end of the heap.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 08:13:38 +01:00
Wrobel, Patryk
4c58eda90d Ensure that BO handle is closed only once
When one process had exported and then opened IPC handle
of memory, then close function was called twice for the
same BO handle. It caused debugBreak() and aborted
an application.

This change allows multiple separate BOs to share one
handle. The last shared handle owner calls close() function.

Related-To: NEO-7200
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-13 14:42:49 +01:00
Warchulski, Jaroslaw
a2e6a8284b Cleanup includes 47
Cleaned up files:
level_zero/tools/source/debug/windows/debug_session.h
level_zero/tools/source/sysman/memory/windows/os_memory_imp.h
level_zero/tools/source/sysman/windows/kmd_sys_manager.h
opencl/test/unit_test/aub_tests/command_stream/copy_engine_aub_tests_xehp_and
shared/source/command_container/command_encoder.inl
shared/source/command_stream/command_stream_receiver_hw_xehp_and_later.inl
shared/source/helpers/blit_commands_helper_base.inl
shared/test/unit_test/image/image_surface_state_fixture.h
shared/test/unit_test/os_interface/windows/os_interface_win_tests.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-02-10 17:07:30 +01:00
Kacper Nowak
7790e208fd feat(zebin): Add support for ELF section type SHT_NOBITS
This commit adds support for parsing SHT_NOBITS zebin's ELF sections
(containing global/constant zero-initialized data).
- Correction: in CTNI path, do not add related symbol if surface has not
been allocated.

Related-To: NEO-7196
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
2023-02-10 16:17:16 +01:00
Compute-Runtime-Validation
d0c0c60205 Revert "feat(zebin): Add support for ELF section type SHT_NOBITS"
This reverts commit fa03aa9a40.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-02-03 03:44:02 +01:00
Kacper Nowak
fa03aa9a40 feat(zebin): Add support for ELF section type SHT_NOBITS
This commit adds support for parsing SHT_NOBITS zebin's ELF sections
(containing global/constant zero-initialized data).

Related-To: NEO-7196
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
2023-02-02 14:54:51 +01:00
Mateusz Jablonski
b931f31f3d debug: Reduce debug logs when changing allocation type
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-02-02 11:05:46 +01:00
Kamil Kopryk
2484c7ceb2 refactor: rename hw_helper files to gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-01 19:37:51 +01:00
Michal Mrozek
2f847aa7bb Optimize zeMemGetAllocProperties.
- No functional changes only performance optimizations.
- add const modifiers
- add early returns
- remove not needed code
- do fast lookup for aligned pointers
- fix incorrect tests

+21% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=100 --iterations=1000
+32% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=1000 --iterations=1000

Related-To: NEO-7676

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2023-01-30 18:00:57 +01:00
Spruit, Neil R
b5692c04cd Support Mapping Multiple Physical Memory with one VA and Bug Fix
- Added support for mapping any portion of a virtual allocation to a
physical mapping with a lookup function for reserved virtual addresses.
- Added support for multiple mappings linked to the same virtual
reservation.
- Fixed bug with 64 bit addresses on windows with invalid addresses
passed to the user.

Related-To: LOCI-3904, LOCI-3914, LOCI-3931

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-01-30 02:36:10 +01:00
Kamil Kopryk
a72cae15fe refactor: don't use global ProductHelper getter 9/n
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-25 11:04:40 +01:00
Warchulski, Jaroslaw
c43233dabf Cleanup includes 42
Cleaned up files:
level_zero/core/source/kernel/kernel_hw.h
shared/source/helpers/common_types.h
shared/test/common/libult/linux/drm_mock.h
shared/test/common/libult/ult_command_stream_receiver.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-25 09:16:39 +01:00
Warchulski, Jaroslaw
bc13db734d Cleanup includes 41
Cleaned up files:
shared/source/command_stream/aub_command_stream_receiver_hw.h
shared/source/helpers/common_types.h
shared/source/os_interface/linux/drm_neo.h
shared/source/os_interface/windows/hw_device_id.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-24 14:37:36 +01:00
Kamil Kopryk
a3212f8362 refactor: don't use global ProductHelper getter 8/n
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-24 10:43:59 +01:00
Kamil Kopryk
e9990dd124 refactor: don't use global ProductHelper getter 6/n
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-23 14:03:51 +01:00
Warchulski, Jaroslaw
76faad3398 Cleanup includes 36
Cleaned up files:
shared/source/helpers/blit_commands_helper.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-20 10:10:40 +01:00
Spruit, Neil R
f928d695e7 Support for Mapping Physical with Virtual Memory
Related-To: LOCI-3422, LOCI-3421

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-01-20 05:17:10 +01:00
Warchulski, Jaroslaw
77501d86ba Cleanup includes 35
Cleaned up files:
shared/source/command_stream/command_stream_receiver.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-17 18:51:40 +01:00
Warchulski, Jaroslaw
8de3898abc Cleanup includes 31
Cleaned up files:
shared/source/command_stream/experimental_command_buffer.h
shared/source/helpers/get_info.h
shared/source/helpers/local_id_gen.h
shared/source/memory_manager/gfx_partition.h
shared/source/memory_manager/host_ptr_manager.h
shared/source/memory_manager/prefetch_manager.h
shared/test/common/mocks/mock_memory_manager.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-16 11:45:55 +01:00
HeFan2017
2ea734491a Deferred deletion of allocations in main thread
Add a clearQueueTillFirstFailure interface to DeferredDeleter, which
iterates the queue from the front and delete the allocations in the
queue till a failure. It is called by defer deletion of allocations
occupied by mutliple contexts to unlock the execution in main thread

Related-To: NEO-7532

Signed-off-by: HeFan2017 <fan.f.he@intel.com>
2023-01-13 09:20:35 +01:00
Warchulski, Jaroslaw
fecb52ac49 Cleanup includes 29
Cleaned up files:
opencl/source/helpers/cl_memory_properties_helpers.h
shared/source/memory_manager/surface.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-13 07:53:03 +01:00
John Falkowski
01017a5df3 L0: add support for deferred memory free as per ContextImp::freeMemExt
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-01-12 21:05:02 +01:00
Warchulski, Jaroslaw
191d4de033 Cleanup includes 28
Cleaned up files:
opencl/source/command_queue/command_queue.h
opencl/source/command_queue/gpgpu_walker.h
shared/source/helpers/memory_properties_helpers.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-11 17:47:33 +01:00
Warchulski, Jaroslaw
4794648978 Cleanup includes 26
Cleaned up files:
opencl/source/command_queue/csr_selection_args.h
opencl/source/event/event.h
shared/source/helpers/engine_control.h
shared/source/sku_info/definitions/sku_info.h

Related-To: NEO-5548

Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-11 09:10:45 +01:00
Warchulski, Jaroslaw
0dc5fb3e51 Cleanup includes 24
Cleaned up files:
opencl/source/mem_obj/image.h
opencl/source/platform/platform.h
shared/source/command_stream/preemption.h
shared/source/helpers/mt_helpers.h
shared/source/memory_manager/unified_memory_manager.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-10 07:56:16 +01:00
Warchulski, Jaroslaw
5eef40fedd Cleanup includes 22
Cleaned up files:
opencl/source/built_ins/builtins_dispatch_builder.h
opencl/source/context/context.h
opencl/source/gtpin/gtpin_notify.h
opencl/source/kernel/kernel.h
opencl/source/kernel/multi_device_kernel.h
opencl/source/mem_obj/buffer.h
opencl/source/mem_obj/mem_obj.h
shared/source/built_ins/registry/built_ins_registry.h
shared/source/page_fault_manager/cpu_page_fault_manager.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-05 16:59:01 +01:00
Compute-Runtime-Validation
af81c79076 Revert "Deferred deletion of allocations in main thread"
This reverts commit c268e30189.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-01-04 18:33:39 +01:00
Kamil Kopryk
0dbf92d401 Refactor: reduce global productHelper getter usage
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-04 16:35:20 +01:00
Warchulski, Jaroslaw
6fb68dd84b Separation of MemoryAllocation from os_agnostic_memory_manager.h
Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-04 15:09:36 +01:00