Refactored eviction mechanism works as follows:
- eviction is needed only if
total size of cache binaries + size of the new binary exceed cache limit
- single evition call removes files with a summed size of 1/3 of the cache limit
- if new binary can not fit in the cache size limit
even after eviction, it will not be saved
- cache limit applies only to
files in cache directory with .cl_cache/.l0_cache extension.
Only these files are counted and only these files are removed
Minor:
- rename variables for better readability
- add `const` where possible
Related-To: NEO-4262
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Refactored eviction mechanism works as follows:
- eviction is needed only if
total size of cache binaries + size of the new binary exceed cache limit
- single evition call removes files with a summed size of 1/3 of the cache limit
- if new binary can not fit in the cache size limit
even after eviction, it will not be saved
- cache limit applies only to
files in cache directory with .cl_cache/.l0_cache extension.
Only these files are counted and only these files are removed
Minor:
- rename variables for better readability
- add `const` where possible
Related-To: NEO-4262
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Refactored eviction mechanism works as follows:
- eviction is needed only if
total size of cache binaries + size of the new binary exceed cache limit
- single evition call removes files with a summed size of 1/3 of the cache limit
- if new binary can not fit in the cache size limit
even after eviction, it will not be saved
- cache limit applies only to
files in cache directory with .cl_cache/.l0_cache extension.
Only these files are counted and only these files are removed
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
What's changed:
- Add early return when open existing config
fails due to different error than `ERROR_FILE_NOT_FOUND`
- Swich `ReadFileEx` to `ReadFile`
- Add `SetFilePointer` to make sure we're reading
from the beginning of the file
- Pass `overlapped` to `WriteFile` to make sure
we're writing from the beginning of the file
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
config.file should not be created manually by the user.
In a scenaro when the user manually creates an empty config.file,
then reading data from this file ends with failure,
because the file is empty.
Such scenario completely freezes the cache creation
until the user manually deletes the empty config file.
This patch fixes such freeze by automatically deleting config
if read fails with ERROR_HANDLE_EOF error.
Patch applies to windows only.
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Changes:
- replaced registry keys with environment variables
for cl_cache in OCL
- added compiler cache helpers
- implemented support for new env vars on Windows
- added tests
New env vars mechanism works as follows:
If `PERSISTENT_CACHE` is set,
driver checks if `NEO_CACHE_DIR` is set.
If `NEO_CACHE_DIR` is not set,
driver uses `%LocalAppData%\NEO\neo_compiler_cache`
as `cl_cache` destination folder.
If `NEO_CACHE_DIR` is not set and `%LocalAppData%`
path could not be obtained,
compiler cache is disabled.
In the current Windows implementation,
special characters in the folder path are not supported.
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Use the same file `os_handle.h` on both
Linux and Windows.
Change implementation of `HandleType` -> `UnifiedHandle` to
`std::variant<int, void *>`
use `int` on Linux
use `void *` on Windows
Related-To: NEO-8092
Signed-off-by: Fabian Zwolinski <fabian.zwolinski@intel.com>
Previous implementation skipped resolving the subDeviceId token, which
caused in returning the LinkedPartially status if token was present.
It could result in ZE_RESULT_ERROR_INVALID_MODULE_UNLINKED when kernel
was created and crashing an app.
Related-To: NEO-6875, HSD-14015133115
Signed-off-by: Naklicki, Mateusz <mateusz.naklicki@intel.com>
* When env variables did not have trailing slash, directory creation
could fail.
* If .cache in $HOME did not exist cache would have been disabled.
Signed-off-by: Kacper Kasper <kacper.k.kasper@intel.com>
This patch add new environment variables to control compiler cache.
Works as follow: If persistent cache is set driver check if NEO_CACHE_DIR
is set. If not then driver checks XDG_CACHE_HOME - If exists
then driver create neo_compiler_cache folder, if
not then driver checks HOME directory. If each NEO_CACHE_DIR,
XDG_CACHE_HOME and HOME are not set then compiler cache is disabled.
Current support is for Linux only.
Signed-off-by: Diedrich, Kamil <kamil.diedrich@intel.com>
Related-To: NEO-4262
- set by default flag ZebinIgnoreIcbeVersion to true
- for zebin icbe version check is only inside flag
- only when use patchtoken then check icbe version is mandatory
Resolves: NEO-7904
Signed-off-by: Cencelewska, Katarzyna <katarzyna.cencelewska@intel.com>
Current flow will be to have one synchronization point
config.file. Read remains unblocking, only write(caching)
operation will be blocking (lock on config.file)
Related-To: NEO-4262
Signed-off-by: Diedrich, Kamil <kamil.diedrich@intel.com>
In rare case of multiple threads trying to set fclBaseTranslationCtx,
there was a potential for use after free (double delete) on
std::unique_ptr::reset.
Resolves: NEO-7767
Signed-off-by: Jaroslaw Chodor <jaroslaw.chodor@intel.com>
* Moved zebin related files to zebin directory.
* Moved zebin related code to Zebin namespace.
* Separated zeInfo from zebin elf.
* Seperated zeInfo decoding from zebin decoder.
* Refactored populateKernelPayloadArgument function.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
This commit adds support for handling local symbols.
* Added 2 fields to SymbolInfo - binding, and associated
instructions segment id.
* Simplified code for decoding elf symbols and relocations.
* Simplified code for patching instruction segments.
* Changed logic of decodeElfSymbolTableAndRelocations:
* Add every global symbol to symbol map.
* Add any local symbol used by relocation to symbol map.
* Changed logic of link:
* After performing relocations remove local symbols from map.
* Replaced UNRECOVERABLE_IF with returning error.
* Removed LocalSymbolInfo structure used before for local kernel jumps.
* Removed old tests.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
This commit adds support for parsing SHT_NOBITS zebin's ELF sections
(containing global/constant zero-initialized data).
- Correction: in CTNI path, do not add related symbol if surface has not
been allocated.
Related-To: NEO-7196
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
This change replaces mechanism of patching global constants and
variables in kernel per relocation to patching them only once. This
would improve linking time performance for kernels with multiple global
symbols.
Signed-off-by: Luzynski, Sebastian Jozef <sebastian.jozef.luzynski@intel.com>
This commits add support for relocating
symbols with local binding and of functional type
(STB_LOCAL, STT_FUNC).
Related-To: NEO-7299
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
In decodeElfSymbolTableAndRelocations, when symbol's section is of
unknown type, then do not add it to linker's symbol table.
Signed-off-by: Kacper Nowak <kacper.nowak@intel.com>
Define single .clang-tidy configuration with all used checks and use
NOLINT to selectively silence tool. That way cleanup should be easier.
third_part/ has its own configuration that disables clang-tidy for this
folder.
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
feat(zebin): Use addend from RELA sections when performing relocations.
Resolves: NEO-6898
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
This commit removes early fail in linking with zebin and external
functions which happens when, there's a relocation to external functions
section, but it's not modifying any external function. And only treats
GLOBAL FUNC symbols pointing to external functions section as external
function symbols.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
This commit adds check in Linker::resolveExternalFunctions checking
if external functions are present before trying to resolve dependencies
and adds default values for ExternalFunctionInfo.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
This change allows for modifying kernel's barrier count
based on called external functions metadata passed
via zeInfo section in zebin.
Added parsing external functions metadata.
Added resolving external functions call graph.
Added updating kernel barriers based on called external functions.
Added support for L0 dynamic link.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
IGC will be queried for max parameter size at the end of initializing
device caps.
Related-To: NEO-4851
Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
Introduced a new warning, which is printed to build log, when the
binary needs to be recompiled. Added a new flag -Wno-recompiled-from-ir
to allow suppression of that message. Removed a bug related to memcpy_s
from ModuleBuildLogImp::getString() and aligned it with specification.
Related-To: NEO-5819
Signed-off-by: Patryk Wrobel <patryk.wrobel@intel.com>
Skip undefined symbols when decoding ELF symbol table
instead of treating them as error in zebin path in
order to support dynamic linking scenarios.
Signed-off-by: Krystian Chmielewski <krystian.chmielewski@intel.com>
- add function for resolving builtins using relocations
- add relocation for __SubDeviceID symbol
Related-To: NEO-6258
Signed-off-by: Kamil Diedrich <kamil.diedrich@intel.com>
- explicitly specify files allowed to dlopen()
- use underscore to prefix fake library name that we want dlopen to
return nullptr
Signed-off-by: Artur Harasimiuk <artur.harasimiuk@intel.com>
Compute-Runtime-Validation