Refactored eviction mechanism works as follows:
- eviction is needed only if
total size of cache binaries + size of the new binary exceed cache limit
- single evition call removes files with a summed size of 1/3 of the cache limit
- if new binary can not fit in the cache size limit
even after eviction, it will not be saved
- cache limit applies only to
files in cache directory with .cl_cache/.l0_cache extension.
Only these files are counted and only these files are removed
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
- set monitor fence dispatch for all cases task count post sync operation
- stand alone flush task count will not happen when already flushed and so
monitor fence
- monitor fence then must be dispatched together with task count post sync
Related-To: NEO-8395
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
What's changed:
- Add early return when open existing config
fails due to different error than `ERROR_FILE_NOT_FOUND`
- Swich `ReadFileEx` to `ReadFile`
- Add `SetFilePointer` to make sure we're reading
from the beginning of the file
- Pass `overlapped` to `WriteFile` to make sure
we're writing from the beginning of the file
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
- program debugSurface's SurfaceState at the beginning of Bindless Surface
State Heap - SPECIAL_SSH
- ensure SPECIAL_SSH is resident
Related-To: NEO-7063
Signed-off-by: Hoppe, Mateusz <mateusz.hoppe@intel.com>
when kmd is not set value of MaxDualSubSlicesSupported
then set it as half of MaxSubSlicesSupported
Related-To: NEO-8397
Signed-off-by: Katarzyna Cencelewska <katarzyna.cencelewska@intel.com>
- For calculating number of threads per workgroup, for SIMD 1, return
local work size (each software thread should be mapped into a whole hardware
thread).
- Correct logic of calculating space for per thread data for SIMD 1.
- Minor: unit tests refactor.
- Corrected naming.
Related-To: NEO-8261
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
Add support for different timestamp packet counts per gfx family.
Change all packet counts to 1 except for xe-hpc.
Related-To: NEO-8154
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
CmdList can be released before Event. In this case, GfxAllocation
destruction must be deferred.
Related-To: NEO-7966
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
config.file should not be created manually by the user.
In a scenaro when the user manually creates an empty config.file,
then reading data from this file ends with failure,
because the file is empty.
Such scenario completely freezes the cache creation
until the user manually deletes the empty config file.
This patch fixes such freeze by automatically deleting config
if read fails with ERROR_HANDLE_EOF error.
Patch applies to windows only.
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
- this change handles level zero immediate command lists on copy engine
- monitor fence will be dispatched for blocking calls
- asynchronous mode will dispatch monitor fence only on host synchronization
Related-To: NEO-8395
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
In L0 its not possible to track objects relations. For example CmdList
may be removed before Event.
In such case, Event needs to safely skip unregister call, without
accessing CmdList/CmdQueue object.
Related-To: NEO-8884
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
So far, there is a separate page allocated for each kernel's ISA within
`KernelImmutableData::initialize()`. Apparently the ISA blocks are often
much smaller than a 64k page, which leads to poor memory utilization and
was even observed to cause the device OOM error if a single module has
several keys.
Improve the situation by reusing the parent allocation (owned by the
module instance) for modules, which kernel ISAs can fit together within
a single 64k page. This improves the memory utilization on a single
module level.
Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
Changes:
- replaced registry keys with environment variables
for cl_cache in OCL
- added compiler cache helpers
- implemented support for new env vars on Windows
- added tests
New env vars mechanism works as follows:
If `PERSISTENT_CACHE` is set,
driver checks if `NEO_CACHE_DIR` is set.
If `NEO_CACHE_DIR` is not set,
driver uses `%LocalAppData%\NEO\neo_compiler_cache`
as `cl_cache` destination folder.
If `NEO_CACHE_DIR` is not set and `%LocalAppData%`
path could not be obtained,
compiler cache is disabled.
In the current Windows implementation,
special characters in the folder path are not supported.
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
value depends on CCS count:
- single CCS mode (default) - 50% available
- two CCS mode - 25% available
- four CCS mode - 12.5% available
Related-To: NEO-8377
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
- unify Linux and Windows default settings
- unify override default code
- correct size estimation when fence is required
- call virtual function once for both estimation and dispatch
Related-To: NEO-8395
Signed-off-by: Zbigniew Zdanowicz <zbigniew.zdanowicz@intel.com>
- when Surface State is reused for new resource, State Cache needs to be
invalidated
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
So far, there is a separate page allocated for each kernel's ISA within
`KernelImmutableData::initialize()`. Apparently the ISA blocks are often
much smaller than a 64k page, which leads to poor memory utilization and
was even observed to cause the device OOM error if a single module has
several keys.
Improve the situation by reusing the parent allocation (owned by the
module instance) for modules, which kernel ISAs can fit together within
a single 64k page. This improves the memory utilization on a single
module level.
Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
Program barrier to task stream, before next enqueue kernel.
This will reduce the number of batch buffer starts for sequences of
enqueue, barrier, enqueue, ... .
Related-To: NEO-8147
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
- this method allocates System Memory
- argument is not needed - ExternalHeap is selected inside this function
- remove unneeded ults
- allocate memory in Device Pool for external heap allocation in
OsAgnosticMemoryManager
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
Use the same file `os_handle.h` on both
Linux and Windows.
Change implementation of `HandleType` -> `UnifiedHandle` to
`std::variant<int, void *>`
use `int` on Linux
use `void *` on Windows
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
value depends on CCS count:
- single CCS mode (default) - no limitations
- two CCS mode - 25% available
- four CCS mode - 12.5% available
Related-To: NEO-8377
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
- For calculating number of threads per workgroup, treat simd 1 as it
was simd 32
- Correct logic of calculating space for per thread data for simd 1
- Minor: unit tests refactor
- Corrected naming
Related-To: NEO-8261
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
it is needed until there is no support in xe kmd for image tiling
Related-To: NEO-8325
Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
Create new file:
`os_handle.h` which contains OS specific Handle type.
Add `shared/source/os_interface/<windows/linux>`
include directory to level_zero.
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
For all of the devices, preferredPlatformName is initialized with
nullptr by default and platform name will be initialized to driver's default
platform name, at the moment this is "Intel(R) OpenCL Graphics".
When Platform is initialized and preferredPlatformName is not nullptr then
Platform name will be set to the value stored in preferredPlatformName.
Add ENABLE_LEGACY_PLATFORM_NAME AIL enum related to added functionality.
Move PlatformInfo to NEO namespace.
Related-To: HSD-22018809561
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
Program barrier immediately to task stream.
This will reduce the number of batch buffer starts.
Related-To: NEO-8147
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
Removed checking for EnableBOChunkingPrefetch in prefetchBOWithChunking
in DrmAllocation so that zeCommandListAppendMemoryPrefetch will work
as expected with chunking enabled
Related-To: NEO-8098
Signed-off-by: Yoon, Young Jin <young.jin.yoon@intel.com>
Products for which zebin has been set as default format in OCLOC:
- ICELAKE_LP
- TIGERLAKE_LP
- ROCKETLAKE
- ALDERLAKE_S
- ALDERLAKE_P
- ALDERLAKE_N
The default format does not override `--format` parameter.
Related-To: NEO-8334
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
This patch avoids returning error for system addresses in setArg
Related-To: GSD-3597
Signed-off-by: Joshua Santosh Ranjan <joshua.santosh.ranjan@intel.com>
Previous implementation skipped resolving the subDeviceId token, which
caused in returning the LinkedPartially status if token was present.
It could result in ZE_RESULT_ERROR_INVALID_MODULE_UNLINKED when kernel
was created and crashing an app.
Related-To: NEO-6875, HSD-14015133115
Signed-off-by: Naklicki, Mateusz <mateusz.naklicki@intel.com>
* When env variables did not have trailing slash, directory creation
could fail.
* If .cache in $HOME did not exist cache would have been disabled.
Signed-off-by: Kacper Kasper <kacper.k.kasper@intel.com>
accept debug flag for all platforms
cleanup ocl unit tests for xe hpg platforms
remove not needed excludes
Related-To: NEO-8187
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
-update last submitted value while flush monitor fence
-dispatch monitor fence with workload when stalling cmds present
Signed-off-by: Lukasz Jobczyk <lukasz.jobczyk@intel.com>
- don't prefer allocation method for TAG_BUFFER
- don't prefer allocation method for TIMESTAMP_PACKET_TAG_BUFFER
these allocations are used from CPU multiple times
Related-To: HSD-18032442707, HSD-18032440987
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
- introduce 2 reuse pools to bindlessHeapHelper
- one pool stores slots for reuse, second pool stores released slots
- stateCacheDirty flags keep track of state cache - when pools are
switched - flags are set indicating flushing caches is needed after
old slots have been reused for new allocations
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
release helper should be always set there
remove invalid setting rev id in xe lpg platform tests
simplify test that rely on release helper
Related-To: NEO-8187
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
DRM_XE_VM_CREATE_FAULT_MODE flag is to be used when
Drm::hasPageFaultSupport returns true.
Related-To: NEO-7996
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
Enabling on pvc after patch in igc.
Enabling only for JIT kernels because AOT could have been compiled with
IGC older than required.
Related-To: NEO-7712
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
- The FsAccess, ProcFsAccess and SysfsAccess classes have been added in
a file in the Sysman Shared directory. The instances of these classes
are maintained in the Sysman Kmd Interface classes.
- Added functions for opendir, readdir and closedir in the shared code.
- Added a ULT in the sysman directory to cover these new functions from
the shared code.
Related-To: LOCI-4689
Signed-off-by: Bari, Pratik <pratik.bari@intel.com>
Related-To: LOCI-4819
- When communicating with the Metrics Libraries, if one is using a
specific sub device, then the correct physical sub device index and
number of sub devices must be used.
- When Affinity Mask is set, this hierarchy information is lost,
therefore the sub device hierarchy data is stored in a map to each
"device" to be returned to the user allowing for the correct sub device
information to be passed to the metrics library.
Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>
- program surface states for redescribed images correctly. Image copy
to/from memory are using redescribed surface states,
- refactor state base address programming - program address and size
together, set max size at the beginning due to lack of Enable flag
- set GpuBase in WddmAllocation when external heap is used
- return max ssh required size from kernelInfo or based on stateful args
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
Add USER_FENCE before PREFETCH call and after the BIND
Related-To: NEO-8098
Signed-off by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
Signed-off-by: John Falkowski <john.falkowski@intel.com>
Taking into account variable ReturnSubDevicesAsApiDevices
during Retain and Release Device
Related-To: NEO-8161
Signed-off-by: Andrzej Koska <andrzej.koska@intel.com>
Created registerFaultHandler() and checkFaultHandlerFromPageFaultManager()
and removed registering sigaction() from the contructor of the
PageFaultManagerLinux class.
Added if statment to check the current pagefault handler is from the
pagefault manager. If not, register the pagefault handler of the current
pagefault manager on linux.
Refactored windows exception vector adding logic to
registerFaultHandler() and call upon the constructor of the
PageFaultManagerWindows, and make
checkFaultHandlerFromPageFaultManager() always return true for windows.
Related-To: NEO-8190
Signed-off-by: Young Jin Yoon <young.jin.yoon@intel.com>
Add support for new GTPin loading logic in OCL path, similar to existing
in L0 - invoking exposed, dedicated API call (OpenGTPinOCL).
- Move logic to shared, including unit tests
- Check whether instrumentation is required on
context creation and if yes,
make a call to OpenGTPinOCL function.
Handle potential errors gracefully without exiting.
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
Add support for new GTPin loading logic in OCL path, similar to existing
in L0 - invoking exposed, dedicated API call (OpenGTPinOCL).
- Move logic to shared, including unit tests
- Check whether instrumentation is required on
clGetPlatformIDs API call, and if yes,
make a call to OpenGTPinOCL function.
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
Perform prefetching of chunks on shared and device allocations
after bind.
Related-To: NEO-8066
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
Signed-off by: John Falkowski <john.falkowski@intel.com>
- assert buffer has header that needs to be accounted for while parsing
the buffer and reading format strings. currentOffset in Printf Formatter
must not exceed size in header. The offset is relative to the beginning
of buffer, not the "begin" field in header
Resolves: NEO-8237
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
- The sysfs filenames have been added in the sysfsNameToFileMap of the
SysmanKmdInterface classes.
- The functions returning the sysfs filenames have been removed from the
shared directory.
- The ULTs have been added to return the sysfs filenames.
Related-To: LOCI-4699
Signed-off-by: Bari, Pratik <pratik.bari@intel.com>
- allocate SSH in cmdContainer when scratch allocation used with
private heaps
- scratch SurfaceStates are addressed relative to
SurfaceStateBaseAddress and have to be placed on SSH
- remove not used SCRATCH_SSH heap type from bindelssHeapHelper
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
Add debug flag ProgramBarrierInCommandStreamTask to program barrier
pipe control in task command stream instead of csr command stream.
This will reduce the number of batch buffer starts.
Related-To: NEO-8147
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
This change disables CPU caching for resources
not accessed by CPU for MTL devices.
Related-To: NEO-7194
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
When upstream ioctl helper is created it will try to create small
allocation, adding I915_GEM_CREATE_EXT_SET_PAT extension. If it
succeeds, for all resources with valid pat index value it will then
explicitly program pat index value with gem create ext call.
PrintBOCreateDestroyResult value can be used to:
- print whether the set pat extension is supported by the kernel, when
ioctl helper is created
- print whether set pat extension was added for a given gem create ext
call and what pat index value was programmed
Note: introduced changes are disabled by defualt.
Toggle DisableGemCreateExtSetPat can be used to enable new functionality.
Related-To: NEO-7896
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
In a scenario where fp64 emulation is both supported and enabled,
we want to include 2 additional extensions:
- cl_khr_fp64 to deviceExtensions
- __opencl_c_fp64 to openclCFeatures (OpenCL ver. 3.0)
Related-To: NEO-7611
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Group allocation types related to cpu side allocations in function to
query gmm usage type. These types will have caching enabled even if
CPU caching is not preferred by GPU.
Add logic to query whether the cpu access is allowed for an allocation
(in cases when it is not preffered by GPU).
Related-To: NEO-7194
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
Temporarily reverting back to previously reported value to avoid corner-case
regression. Intent is reintroduce as soon as regressions are rootcaused.
This only affects Linux.
Related-To: GSD-5474, GSD-5412
This reverts commit e8ac22c265.
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
Enable eviction of CPU side USM allocation for UMD migrations on Windows.
Reverts incorrect auto-revert commit 218de586a4f28b1de3e983b9006e7a99d3a4d10e.
Related-To: NEO-8015
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
- use testTaskCountReady method to check TaskCount value
- download all allocations when TaskCount is ready
Signed-off-by: Dunajski, Bartosz <bartosz.dunajski@intel.com>
This would avoid recalculating reference timestamps
when event is used with different command lists.
Related-To: LOCI-4563
Signed-off-by: Joshua Santosh Ranjan <joshua.santosh.ranjan@intel.com>
- store surface state info for bindless addressing in graphics
allocation
- remove map in BindlessHeapsHelper - bindlessInfo is constant for
the lifetime of an allocation
- program bindless offsets and surface states for images when used in
bindless kernel
- handle ouf of memory on surface state heap - return error
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
Enable eviction of CPU side USM allocation for UMD migrations on Windows.
Related-To: NEO-8015
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
Enable eviction of CPU side USM allocation for UMD migrations on Windows.
Related-To: NEO-8015
Signed-off-by: Milczarek, Slawomir <slawomir.milczarek@intel.com>
Related-To: LOCI-4615
- Added Support for users to set ZE_FLAT_DEVICE_HIERARCHY to either FLAT
or COMPOSITE to change how devices are returned in zeDeviceGet and
clGetDeviceIDs.
- COMPOSITE is default behavior that exists today.
- FLAT returns all sub devices which have no sub devices and all root
devices that have no sub devices in zeDeviceGet ie with all devices
flattened out in order.
- Added zeDeviceGetRootDevice for one to retrieve the Root Device for
any SubDevice.
Signed-off-by: Neil R Spruit <neil.r.spruit@intel.com>
By default prefer allocating memory first by KMD, instead of malloc first.
By default prefer not caching allocations on MTL devices. This results
in allocations being handled with non-coherent pat index.
For integrated devices when caching is not preferred do not allow
direct memory access in CPU domain. For map/unmap operations create
a dedicated memory allocation for CPU access, instead of accessing it
directly, reusing the same logic as when mapping/unmapping local memory.
Signed-off-by: Filip Hazubski <filip.hazubski@intel.com>
If waitForBarrier is not passed outEvent then do
dcFlush on the next synchronize call.
Related-To: NEO-8147
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
Instances returned by `getAllocationsVector()` in some cases cannot be
freed (in the `malloc/new` sense) until the `drain()` function invokes
`allocInUse()` on them. Plus, the `chunksToFree` container operates on
pairs `{offset, size}`, not pointers, so such pair cannot be used to
release allocations either.
Provide an optional callback, which can be implemented by the custom
pool derived from `AbstractBuffersPool`. This callback can be used, for
example, to perform actual release of an allocation related to the
currently processed chunk.
Additionally, provide the `drain()` and `tryFreeFromPoolBuffer()`
functions with pool-independent versions and keep the previous versions
as defaults (for allocators with a single pool). The new versions allow
reusing the code for cases when allocator has multiple pools.
In both cases, there was no such needs so far but it arose when working
on `IsaBuffersAllocator`. The latter is coming with future commits, but
the shared code modifications are extracted as an independent step.
Related-To: NEO-7788
Signed-off-by: Maciej Bielski <maciej.bielski@intel.com>
If workgroup dimension x is 1, use y to ajust for divisible by dispatch
size.
Related-To: NEO-7927, GSD-5417
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
The new classes SysmanKmdInterface, SysmanKmdInterfaceI915 and
SysmanKmdInterfaceXe have been introduced.
A map is maintained in the SysmanKmdInterfaceI915 and
SysmanKmdInterfaceXe class for the sysfs file names.
The access specifier of the function getDrmVersion has been changed from
protected to public so as to use it in the sysman code. This is required
for the SysmanKmdInterface pointer to point to the
SysmanKmdInterfaceI915 and SysmanKmdInterfaceXe accordingly.
The ULTs have been added for the new sysfs file path corresponding to
the i915 and the Xe driver.
Related-To: LOCI-4399
Signed-off-by: Bari, Pratik <pratik.bari@intel.com>
Added if conditions to enable useChunking flag by checking
with ExecutionEnvironment::isDebuggingEnabled.
Related-To: NEO-8164
Signed-off-by: Young Jin Yoon <young.jin.yoon@intel.com>
- Preserve releases on CMake level.
- Instead of generating builtins per platform, generate them per-release
(+ correct naming accordingly).
- Stop using revisions in builtin compilation logic path, as they are
already embedded in release (device ip).
- Remove platform names & revisions from names for generated files
(related to builtins).
- Remove unnecessary code, refactor ULT logic.
Related-To: NEO-7783
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
The previous name "pageSize2Mb" defined in
shared/source/helpers/constant.h is inconsistent to other variable,
i.e. pageSize64k.
Furthermore, it's a bit misleading because the page size is defined in
Megabytes (MB), not in Megabits (Mb).
Related-to: NEO-7695
Signed-off-by: Young Jin Yoon <young.jin.yoon@intel.com>
Related-to: NEO-7695
New debug keys added:
EnableBOChunking is now a mask
0 = no chunking (default).
1 = shared allocations only
2 = device allocations only
3 = shared and device allocations
MinimalAllocationSizeForChunking sets the minimum allocation
size to apply chunking. Default is 2MB.
Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>
- also use helper when checking that is simd1 to have same flow
Related-To: NEO-8087
Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
- Adobe Photoshop
- Adobe Premiere Pro
- Adobe After Effect
use RCS as a default engine
Related-To: NEO-8049
Signed-off-by: Mateusz Jablonski <mateusz.jablonski@intel.com>
- program global bindless ssba when external allocator used (
UseExternalAllocatorForSshAndDsh)
Related-To: NEO-7063
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
- all bos from Module must have requireImmediateBinding
flag set
- this change fixes hang in debugger - where MODULE LOAD event
was not sent
Resolves: NEO-8121
Signed-off-by: Mateusz Hoppe <mateusz.hoppe@intel.com>
Zbigniew Zdanowicz