fix: flag to limit usm reuse based on memory usage

Host usm and device usm for igfx checks system memory usage.
Device usm for dgfx checks local memory usage.

If used memory is above limit threshold:
- no new allocations will be saved for reuse
- cleaner will use shorter hold time of 2 seconds
- cleaner will free all eligible allocations, regardless of async
deleter thread having work

Motivation: in case of gfx memory being full, making resident new
allocations will require evictions which leads to massive slowdown on
enqueue calls.
This change aims to minimize cases where extra memory usage from usm
reuse mechanism leads to above situation.

Related-To: NEO-6893, NEO-14160

Signed-off-by: Dominik Dabek <dominik.dabek@intel.com>
This commit is contained in:
Dominik Dabek
2025-03-26 14:09:38 +00:00
committed by Compute-Runtime-Automation
parent 5684eb0dca
commit 915d657420
18 changed files with 373 additions and 170 deletions

View File

@@ -0,0 +1,52 @@
/*
* Copyright (C) 2025 Intel Corporation
*
* SPDX-License-Identifier: MIT
*
*/
#pragma once
#include <cstdint>
#include <mutex>
namespace NEO {
struct UsmReuseInfo {
uint64_t getAllocationsSavedForReuseSize() const {
return allocationsSavedForReuseSize;
}
uint64_t getMaxAllocationsSavedForReuseSize() const {
return maxAllocationsSavedForReuseSize;
}
uint64_t getLimitAllocationsReuseThreshold() const {
return limitAllocationsReuseThreshold;
}
std::unique_lock<std::mutex> obtainAllocationsReuseLock() const {
return std::unique_lock<std::mutex>(allocationsReuseMtx);
}
void recordAllocationSaveForReuse(size_t size) {
allocationsSavedForReuseSize += size;
}
void recordAllocationGetFromReuse(size_t size) {
allocationsSavedForReuseSize -= size;
}
void init(uint64_t maxAllocationsSavedForReuseSize, uint64_t limitAllocationsReuseThreshold) {
this->maxAllocationsSavedForReuseSize = maxAllocationsSavedForReuseSize;
this->limitAllocationsReuseThreshold = limitAllocationsReuseThreshold;
}
static constexpr uint64_t notLimited = std::numeric_limits<uint64_t>::max();
protected:
uint64_t maxAllocationsSavedForReuseSize = 0u;
uint64_t limitAllocationsReuseThreshold = 0u;
uint64_t allocationsSavedForReuseSize = 0u;
mutable std::mutex allocationsReuseMtx;
};
} // namespace NEO