Commit Graph

130 Commits

Author SHA1 Message Date
Jaime Arteaga
b8c1c0f600 Revert "Fix: Remove allocation from sharedMakeResidentAllocations on free"
This reverts commit fd16fee707.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-02-10 00:02:54 +01:00
Jaime Arteaga
51d767daea refactor: Add IPC memory data
Refactor structure and add field to pass USM memory type.

To maintain backwards compatibility with current applications,
pass 0 as type for device allocations, and 1 for host
allocations.

Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-02-02 20:20:31 +01:00
Kamil Kopryk
524d945e8c refactor: rename l0 hw_helpers dir to gfx_core_helpers
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-02 11:15:56 +01:00
Kamil Kopryk
2484c7ceb2 refactor: rename hw_helper files to gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-02-01 19:37:51 +01:00
Michal Mrozek
2f847aa7bb Optimize zeMemGetAllocProperties.
- No functional changes only performance optimizations.
- add const modifiers
- add early returns
- remove not needed code
- do fast lookup for aligned pointers
- fix incorrect tests

+21% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=100 --iterations=1000
+32% improvement in
./api_overhead_benchmark_l0 --test=GetMemoryProperties
--api=l0 --AmountOfUsmAllocations=1000 --iterations=1000

Related-To: NEO-7676

Signed-off-by: Michal Mrozek <michal.mrozek@intel.com>
2023-01-30 18:00:57 +01:00
Spruit, Neil R
b5692c04cd Support Mapping Multiple Physical Memory with one VA and Bug Fix
- Added support for mapping any portion of a virtual allocation to a
physical mapping with a lookup function for reserved virtual addresses.
- Added support for multiple mappings linked to the same virtual
reservation.
- Fixed bug with 64 bit addresses on windows with invalid addresses
passed to the user.

Related-To: LOCI-3904, LOCI-3914, LOCI-3931

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2023-01-30 02:36:10 +01:00
Warchulski, Jaroslaw
c43233dabf Cleanup includes 42
Cleaned up files:
level_zero/core/source/kernel/kernel_hw.h
shared/source/helpers/common_types.h
shared/test/common/libult/linux/drm_mock.h
shared/test/common/libult/ult_command_stream_receiver.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-25 09:16:39 +01:00
Kamil Kopryk
eb002acaa1 refactor: rename l0_gfx_core_helper files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-24 12:59:26 +01:00
Compute-Runtime-Validation
c262ee94c0 Revert "feature: Refactor IPC memory data"
This reverts commit 9c93af7462.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2023-01-21 12:14:34 +01:00
Zbigniew Zdanowicz
7925e12392 Add improvements and tweaks to event pool runtime and tests
1. Reposition event pool code
- Implementation of event pool methods should be placed in event pool file
2. Change event pool and event classes
- move all class variables to base classes as protected
- add needed setters and getters for runtime
- add extended mocks and whiteboxes for unit tests
- tests using base or mock class when needed
3. Correct event creation in unit tests
- use HW specific timestamp type
- HW unit tests are using template timestamp type for event creation
- other tests are using helper to create event

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 16:47:08 +01:00
Warchulski, Jaroslaw
286c672ef4 Cleanup includes 37
Cleaned up files:
level_zero/core/source/event/event.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-20 12:34:39 +01:00
Zbigniew Zdanowicz
2aacc04ffd Fix two issues in ipc event pool creation
- remove double and triple free in open ipc event pool cleanup section
- correctly initialize device vector

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 12:03:53 +01:00
Spruit, Neil R
f928d695e7 Support for Mapping Physical with Virtual Memory
Related-To: LOCI-3422, LOCI-3421

Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
2023-01-20 05:17:10 +01:00
Jaime Arteaga
9c93af7462 feature: Refactor IPC memory data
Refactor structure and add field to pass USM memory type.

Related-To: LOCI-3771

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-01-20 03:50:48 +01:00
Zbigniew Zdanowicz
b9b9155ee0 Fail ipc event pool creation for max event packet misconfiguration
Remote event pool is created with device list that configure max event packet
IPC context can have different device list and configure different
max event packet.
When these two numbers are different driver should not allow creation of
IPC event pool.

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-20 00:45:29 +01:00
Zbigniew Zdanowicz
f4b7a63a35 validate internal handle for ipc event pools
Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-18 17:10:51 +01:00
Zbigniew Zdanowicz
172ea34fc2 Refactor event pool class hierarchy
Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-17 14:42:20 +01:00
Zbigniew Zdanowicz
ff88930625 Refactor ipc event pool data exchange protocol
- using predefined structure adds security to enforce data limit
- code is simplified and easier to maintain or add new data fields

Related-To: NEO-7636

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2023-01-17 10:46:52 +01:00
John Falkowski
01017a5df3 L0: add support for deferred memory free as per ContextImp::freeMemExt
Signed-off-by: John Falkowski <john.falkowski@intel.com>
2023-01-12 21:05:02 +01:00
Warchulski, Jaroslaw
bd81b5546d Cleanup includes 25
Cleaned up files:
level_zero/core/source/device/device.h
opencl/source/helpers/dispatch_info.h
shared/source/os_interface/os_interface.h

Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2023-01-10 12:54:45 +01:00
Kamil Kopryk
0dbf92d401 Refactor: reduce global productHelper getter usage
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2023-01-04 16:35:20 +01:00
Jaime Arteaga
fd16fee707 Fix: Remove allocation from sharedMakeResidentAllocations on free
If the shared allocation hasn't been removed up to this point,
then remove it when freeing the pointer.

No need to check if ptr is still in the map, as erase would
automatically return 0 if not found.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2023-01-03 22:06:08 +01:00
Warchulski, Jaroslaw
f275eea6ec Cleanup includes 14
Cleaned up files:
shared/source/device/device.h

Related-To: NEO-5548

Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2022-12-23 10:46:34 +01:00
Spruit, Neil R
905a0bc0b4 Fix Device Allocated IPC Events for L0
- Given Device Allocated IPC Event Pool, the open IPC event pool process
  must allocate the event pool as a device buffer or usage of the memory
will fail.
- Pass deviceAlloc flag thru the IPC Event Pool Handle to enable the
  child processes to allocate the correct memory type.
- Pass isHostVisible flag thru IPCEvent Pool Handle to indicate if the
  event pool must be allocated as host visible regardless of the
allocation type.

Related-To: LOCI-3764

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-12-23 03:29:16 +01:00
Compute-Runtime-Validation
08f53f4e96 Revert "Fix Device Allocated IPC Events for L0"
This reverts commit 1292ea26f5.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-12-22 21:42:26 +01:00
Compute-Runtime-Validation
429755e0f4 Revert "Check for context in appendMemoryCopy"
This reverts commit 32bab85f7e.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-12-21 05:38:37 +01:00
Spruit, Neil R
1292ea26f5 Fix Device Allocated IPC Events for L0
- Given Device Allocated IPC Event Pool, the open IPC event pool process
  must allocate the event pool as a device buffer or usage of the memory
will fail.
- Pass deviceAlloc flag thru the IPC Event Pool Handle to enable the
  child processes to allocate the correct memory type.
- Pass isHostVisible flag thru IPCEvent Pool Handle to indicate if the
  event pool must be allocated as host visible regardless of the
allocation type.

Related-To: LOCI-3764

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-12-20 23:43:03 +01:00
Kamil Kopryk
4a87266be1 Don't use global getter for L0GfxCoreHelper and GfxCoreHelper in L0 files
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-12-20 16:51:43 +01:00
Jaime Arteaga
32bab85f7e Check for context in appendMemoryCopy
Confirm the allocations used in an appendMemoryCopy operation
belong to the same context as the list.

Related-To: LOCI-1996
Resolves: NEO-6162

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-12-20 00:13:36 +01:00
Compute-Runtime-Validation
ad072b4d5d Revert "Fix Device Allocated IPC Events"
This reverts commit 1360ec9179.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-12-17 05:36:59 +01:00
Jaime Arteaga
050e4fb1da Pass allocation type to import handle functions
This to allow flexibilty on choosing the allocation type from the
callers in layers above on the driver.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-12-16 03:42:40 +01:00
Compute-Runtime-Validation
5a29cf8508 Revert "Check for context in appendMemoryCopy"
This reverts commit afd0784c16.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-12-16 03:28:06 +01:00
Jaime Arteaga
afd0784c16 Check for context in appendMemoryCopy
Confirm the allocations used in an appendMemoryCopy operation
belong to the same context as the list.

Related-To: LOCI-1996
Resolves: NEO-6162

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-12-15 23:42:10 +01:00
Spruit, Neil R
1360ec9179 Fix Device Allocated IPC Events
- Given Device Allocated IPC Event Pool, the open IPC event pool process
  must allocate the event pool as a device buffer or usage of the memory
will fail.
- Pass deviceAlloc flag thru the IPC Event Pool Handle to enable the
  child processes to allocate the correct memory type.

Related-To: LOCI-3764

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-12-15 23:35:39 +01:00
Compute-Runtime-Validation
c54c3d796c Revert "Set isLockable if size small enough for cpu memcpy"
This reverts commit 41a80072b9.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-12-12 09:24:14 +01:00
Szymon Morek
41a80072b9 Set isLockable if size small enough for cpu memcpy
Signed-off-by: Szymon Morek <szymon.morek@intel.com>
2022-12-09 16:33:32 +01:00
Kamil Kopryk
03b687881f Rename HwHelper -> GfxCoreHelper
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-12-09 10:29:06 +01:00
Warchulski, Jaroslaw
be647d42d9 Cleanup includes 12
Related-To: NEO-5548
Signed-off-by: Warchulski, Jaroslaw <jaroslaw.warchulski@intel.com>
2022-12-07 13:14:15 +01:00
Kamil Kopryk
73b2104183 Rename L0HwHelper -> L0GfxCoreHelper
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>
2022-12-05 11:26:05 +01:00
Jaime Arteaga
99655d34f9 Return a unique pointer for multiple calls to openIpcHandle
This to follow specification, which says:

zeMemOpenIpcHandle:

- Multiple calls to this function with the same IPC handle will return
unique pointers.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-12-01 18:42:54 +01:00
Spruit, Neil R
d81b0b14a1 L0 Virtual Memory Reservation support
- Enable support for L0 Virtual Memory reservation on Linux and Windows.
- Excludes support for Linux to allow pStart option

Related-To: LOCI-3397, LOCI-1543

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
2022-12-01 00:55:19 +01:00
Jaime Arteaga
4dfdbd612d Fail when handle cannot be obtain for an allocation
If a handle cannot be obtained, like PRIME_HANDLE_TO_FD, then
properly check for the error and propagate it upwards.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-11-30 20:59:02 +01:00
Compute-Runtime-Validation
565d44e59c Revert "Fail when handle cannot be obtain for an allocation"
This reverts commit 4391ad21bb.

Signed-off-by: Compute-Runtime-Validation <compute-runtime-validation@intel.com>
2022-11-23 05:45:01 +01:00
Jaime Arteaga
4391ad21bb Fail when handle cannot be obtain for an allocation
If a handle cannot be obtained, like PRIME_HANDLE_TO_FD, then
properly check for the error and propagate it upwards.

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
2022-11-22 19:29:19 +01:00
Kamil Kopryk
146e18d9a4 Move L0HwHelper ownership to RootDeviceEnvironment 2/n
Related-To: NEO-6853
Signed-off-by: Kamil Kopryk <kamil.kopryk@intel.com>

Use RootDeviceEnvironment getHelper<L0CoreHelper> for
- imageCompressionSupported
- usmCompressionSupported
- forceDefaultUsmCompressionSupport
2022-11-15 15:49:55 +01:00
Lukasz Jobczyk
ff500e0de6 Add support for external hostptr in shared allocations
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
2022-11-09 14:48:01 +01:00
Zbigniew Zdanowicz
27d042107a Add feature to estimate number of event packets
Related-To: NEO-7469

Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2022-11-04 18:38:17 +01:00
Zbigniew Zdanowicz
b284b727e9 Fix number of available events in IPC event pools
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
2022-11-03 13:06:28 +01:00
Patryk Wrobel
9b26e96b11 Do not copy std::vector of OS handles when it is not needed
This change replaces unneeded copying of std::vectors
with usage of const references. Furthermore, it adds
reserve() call before filling the container via push_back().

Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
2022-10-21 10:28:58 +02:00
Patryk Wrobel
2dd3c11248 Do not copy ze_ipc_mem_handle_t in internal functions of NEO
The size of this type is at least 64 bytes due to
ZE_MAX_IPC_HANDLE_SIZE. This change introduces usage
of const reference instead of copying.

Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
2022-10-17 13:51:03 +02:00