Commit Graph

156 Commits

Author SHA1 Message Date
Mateusz Jablonski
1be48965ef fix: parse all extensions passed to zeMemAllocShared
reuse common pNext parser

allow to create raytracing allocation with relaxed size

Related-To: NEO-8075
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-06-20 14:32:16 +02:00
Neil R Spruit
ba6d447b4d feature: Support for using Reserved address with multiple mappings
Related-To: LOCI-4381

- Enabled support for customers to use full Virtual reservation range
with multiple physical mappings with additional allocations implicitly
included in residency.
- Buffer Surface state size extended for first allocation to stretch to
the bufferSize requested.

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-06-07 03:12:29 +02:00
Spruit, Neil R
b589e32c4b fix: Force Close of IPC handles at memFree
Related-To: LOCI-4455

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-05-26 17:09:47 +02:00
Compute-Runtime-Validation
59c8e8f019 Revert "fix: Force Close of IPC handles at memFree"
This reverts commit e1c3cb5ecd.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-05-25 10:15:00 +02:00
Spruit, Neil R
e1c3cb5ecd fix: Force Close of IPC handles at memFree
Related-To: LOCI-4455

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-05-24 21:12:10 +02:00
Neil R Spruit
ded9d7bff2 feature: Get Peer Allocation with specified base Pointer
Related-To: LOCI-4176

- Given a Base Pointer passed into Get Peer Allocation, then the base
pointer is used in the map of the new allocation to the virtual memory.
- Enables users to use the same pointer for all devices in Peer To Peer.
- Currently unsupported on reserved memory due to mapped and exec
resiedency of Virtual addresses.

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-05-24 20:41:20 +02:00
Raiyan Latif
609265a0af fix: Free Peer Allocations in Virtual Memory Path
Related-To: LOCI-4359

Signed-off-by: Raiyan Latif <raiyan.latif@intel.com>
2023-05-03 01:15:18 +02:00
Lu, Wenbin
5d653c8536 fix: Add alignment support to createUnifiedMemoryAllocation
Allows the user to use alignments > 64KB in `createUnifiedMemoryAllocation`

So that the restriction in `piextUSMDeviceAlloc` of the DPC++ runtime
could be lifted

Related-To: LOCI-4168

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-05-02 09:19:23 +02:00
Neil R Spruit
7014ddefc2 fix: Set IPC type in ipcData explicitly
Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-05-01 19:34:17 +02:00
Spruit, Neil R
364c2da9fb feature: Add Support for zeMemPutIpcHandle & zeMemGet IPC Handle converters
Related-To: LOCI-4172, LOCI-4305, LOCI-4306

- Create a new IPC Memory handle upon call to getIpcMemHandle if the
previous handle has been freed.
- Release the Ipc Memory Handle when zeMemPutIpcHandle is called.
- Create a new IPC Handle for tracking thru zeMemGetAllocProperties
when ze_external_memory_export_fd_t is used.
- Convert FD to opaque IPC handle and IPC Handle to FD.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-04-27 03:53:52 +02:00
John Falkowski
e056082710 refactor graphics allocation structure elements for sub-allocation properties
Resolves:  LOCI-3772

Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-04-07 16:53:23 +02:00
Compute-Runtime-Validation
2b93126795 Revert "Add alignment support to createUnifiedMemoryAllocation"
This reverts commit ca02bbba4b.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-30 15:43:47 +02:00
Compute-Runtime-Validation
322c89cd1e Revert "Traverse pNext chain for memory allocations extensions"
This reverts commit e81fb20505.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-03-28 13:02:45 +02:00
Lu, Wenbin
ca02bbba4b Add alignment support to createUnifiedMemoryAllocation
Allows the user to use alignments > 64KB in `createUnifiedMemoryAllocation`

So that the restriction in `piextUSMDeviceAlloc` of the DPC++ runtime
could be lifted

Related-To: LOCI-4168

Signed-off-by: Lu, Wenbin <wenbin.lu@intel.com>
2023-03-28 10:57:04 +02:00
Raiyan Latif
e81fb20505 Traverse pNext chain for memory allocations extensions
Related-To: LOCI-4036

Signed-off-by: Raiyan Latif <raiyan.latif@intel.com>
2023-03-24 07:43:15 +01:00
John Falkowski
a1e2eca9e8 Add zeMemGetAllocProperties extension for sub-allocations
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-03-17 21:21:44 +01:00
Mateusz Jablonski
0da5e6f277 refactor l0: cleanup cmake file level_zero/core/source/CMakeLists.txt
Related-To: NEO-7507
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-16 12:38:15 +01:00
Spruit, Neil R
86e739e9dc feature: Relaxed Virtual Memory reservation with pStart
Related-To: LOCI-3871

- Relaxed the Virtual Memory Reservation to allow pStart and not fail if
the pStart value is not obtained.
- Moves checks on pStart to the user to check and determine if they want
to re-reserve or use the address allocated.
- Changed reserveGpuAddress to use unit64_t type to allow internal
address range structure assignment without cast.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-03-08 08:06:50 +01:00
Mateusz Jablonski
8194543ae5 refactor raytracing: unify RT programming logic
reuse EncodeEnableRayTracing in CommandStreamReceiver
add method to determine need for 48b resource flag for RT allocations
Related-To: NEO-7606

Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
2023-03-07 11:51:58 +01:00
Zbigniew Zdanowicz
49def723b7 Unify layout of command list class
Related-To: NEO-5055

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-03-07 11:38:47 +01:00
Compute-Runtime-Validation
29cbdc2430 Revert "Traverse pNext chain for all memory allocations extensions"
This reverts commit bd8b85c354.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-02-19 10:10:24 +01:00
Raiyan Latif
bd8b85c354 Traverse pNext chain for all memory allocations extensions
Related-To: LOCI-4036
Signed-off-by: Raiyan Latif <raiyan.latif@intel.com>
2023-02-17 18:07:39 +01:00
Neil Spruit
2aaebddb37 Revert "Heap allocation allow base address and growable addresses"
This reverts commit 44ec497b1a.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 18:34:40 +01:00
Spruit, Neil R
44ec497b1a feature: Allow for Allocating a base address in the heap and grow an allocation
Related-To: LOCI-3871

- Enabled allocation of specified base address in the targeted heap.
- Enabled virtual memory reservations to grow by allocating at the start
of the heap vs the end of the heap.

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-02-15 08:13:38 +01:00
Jaime Arteaga
dcea913fbc feature: Add support for host IPC allocations
Add support for importing and exporting host allocations.

Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-02-14 00:50:44 +01:00
Wrobel, Patryk
4c58eda90d Ensure that BO handle is closed only once
When one process had exported and then opened IPC handle
of memory, then close function was called twice for the
same BO handle. It caused debugBreak() and aborted
an application.

This change allows multiple separate BOs to share one
handle. The last shared handle owner calls close() function.

Related-To: NEO-7200
Signed-off-by: Wrobel, Patryk <patryk.wrobel@intel.com>
2023-02-13 14:42:49 +01:00
Jaime Arteaga
b8c1c0f600 Revert "Fix: Remove allocation from sharedMakeResidentAllocations on free"
This reverts commit fd16fee707.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-02-10 00:02:54 +01:00
Jaime Arteaga
51d767daea refactor: Add IPC memory data
Refactor structure and add field to pass USM memory type.

To maintain backwards compatibility with current applications,
pass 0 as type for device allocations, and 1 for host
allocations.

Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-02-02 20:20:31 +01:00
Kamil Kopryk
524d945e8c refactor: rename l0 hw_helpers dir to gfx_core_helpers
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-02 11:15:56 +01:00
Kamil Kopryk
2484c7ceb2 refactor: rename hw_helper files to gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-01 19:37:51 +01:00
Michal Mrozek
2f847aa7bb Optimize zeMemGetAllocProperties.
- No functional changes only performance optimizations.
- add const modifiers
- add early returns
- remove not needed code
- do fast lookup for aligned pointers
- fix incorrect tests

+21% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=100 --iterations=1000
+32% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=1000 --iterations=1000

Related-To: NEO-7676

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2023-01-30 18:00:57 +01:00
Spruit, Neil R
b5692c04cd Support Mapping Multiple Physical Memory with one VA and Bug Fix
- Added support for mapping any portion of a virtual allocation to a
physical mapping with a lookup function for reserved virtual addresses.
- Added support for multiple mappings linked to the same virtual
reservation.
- Fixed bug with 64 bit addresses on windows with invalid addresses
passed to the user.

Related-To: LOCI-3904, LOCI-3914, LOCI-3931

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-01-30 02:36:10 +01:00
Warchulski, Jaroslaw
c43233dabf Cleanup includes 42
Cleaned up files:
level_zero/core/source/kernel/kernel_hw.h
shared/source/helpers/common_types.h
shared/test/common/libult/linux/drm_mock.h
shared/test/common/libult/ult_command_stream_receiver.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-25 09:16:39 +01:00
Kamil Kopryk
eb002acaa1 refactor: rename l0_gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-24 12:59:26 +01:00
Compute-Runtime-Validation
c262ee94c0 Revert "feature: Refactor IPC memory data"
This reverts commit 9c93af7462.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-01-21 12:14:34 +01:00
Zbigniew Zdanowicz
7925e12392 Add improvements and tweaks to event pool runtime and tests
1. Reposition event pool code
- Implementation of event pool methods should be placed in event pool file
2. Change event pool and event classes
- move all class variables to base classes as protected
- add needed setters and getters for runtime
- add extended mocks and whiteboxes for unit tests
- tests using base or mock class when needed
3. Correct event creation in unit tests
- use HW specific timestamp type
- HW unit tests are using template timestamp type for event creation
- other tests are using helper to create event

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 16:47:08 +01:00
Warchulski, Jaroslaw
286c672ef4 Cleanup includes 37
Cleaned up files:
level_zero/core/source/event/event.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-20 12:34:39 +01:00
Zbigniew Zdanowicz
2aacc04ffd Fix two issues in ipc event pool creation
- remove double and triple free in open ipc event pool cleanup section
- correctly initialize device vector

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 12:03:53 +01:00
Spruit, Neil R
f928d695e7 Support for Mapping Physical with Virtual Memory
Related-To: LOCI-3422, LOCI-3421

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-01-20 05:17:10 +01:00
Jaime Arteaga
9c93af7462 feature: Refactor IPC memory data
Refactor structure and add field to pass USM memory type.

Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-01-20 03:50:48 +01:00
Zbigniew Zdanowicz
b9b9155ee0 Fail ipc event pool creation for max event packet misconfiguration
Remote event pool is created with device list that configure max event packet
IPC context can have different device list and configure different
max event packet.
When these two numbers are different driver should not allow creation of
IPC event pool.

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 00:45:29 +01:00
Zbigniew Zdanowicz
f4b7a63a35 validate internal handle for ipc event pools
Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-18 17:10:51 +01:00
Zbigniew Zdanowicz
172ea34fc2 Refactor event pool class hierarchy
Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-17 14:42:20 +01:00
Zbigniew Zdanowicz
ff88930625 Refactor ipc event pool data exchange protocol
- using predefined structure adds security to enforce data limit
- code is simplified and easier to maintain or add new data fields

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-17 10:46:52 +01:00
John Falkowski
01017a5df3 L0: add support for deferred memory free as per ContextImp::freeMemExt
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-01-12 21:05:02 +01:00
Warchulski, Jaroslaw
bd81b5546d Cleanup includes 25
Cleaned up files:
level_zero/core/source/device/device.h
opencl/source/helpers/dispatch_info.h
shared/source/os_interface/os_interface.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-10 12:54:45 +01:00
Kamil Kopryk
0dbf92d401 Refactor: reduce global productHelper getter usage
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-04 16:35:20 +01:00
Jaime Arteaga
fd16fee707 Fix: Remove allocation from sharedMakeResidentAllocations on free
If the shared allocation hasn't been removed up to this point,
then remove it when freeing the pointer.

No need to check if ptr is still in the map, as erase would
automatically return 0 if not found.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-01-03 22:06:08 +01:00
Warchulski, Jaroslaw
f275eea6ec Cleanup includes 14
Cleaned up files:
shared/source/device/device.h

Related-To: NEO-5548

Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2022-12-23 10:46:34 +01:00
Spruit, Neil R
905a0bc0b4 Fix Device Allocated IPC Events for L0
- Given Device Allocated IPC Event Pool, the open IPC event pool process
  must allocate the event pool as a device buffer or usage of the memory
will fail.
- Pass deviceAlloc flag thru the IPC Event Pool Handle to enable the
  child processes to allocate the correct memory type.
- Pass isHostVisible flag thru IPCEvent Pool Handle to indicate if the
  event pool must be allocated as host visible regardless of the
allocation type.

Related-To: LOCI-3764

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-12-23 03:29:16 +01:00