Files
llvm/lldb/source/Target/Statistics.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

287 lines
12 KiB
C++
Raw Normal View History

Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
//===-- Statistics.cpp ----------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "lldb/Target/Statistics.h"
#include "lldb/Core/Debugger.h"
#include "lldb/Core/Module.h"
#include "lldb/Symbol/SymbolFile.h"
#include "lldb/Target/Process.h"
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
#include "lldb/Target/Target.h"
#include "lldb/Target/UnixSignals.h"
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
using namespace lldb;
using namespace lldb_private;
using namespace llvm;
static void EmplaceSafeString(llvm::json::Object &obj, llvm::StringRef key,
const std::string &str) {
if (str.empty())
return;
if (LLVM_LIKELY(llvm::json::isUTF8(str)))
obj.try_emplace(key, str);
else
obj.try_emplace(key, llvm::json::fixUTF8(str));
}
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
json::Value StatsSuccessFail::ToJSON() const {
return json::Object{{"successes", successes}, {"failures", failures}};
}
static double elapsed(const StatsTimepoint &start, const StatsTimepoint &end) {
StatsDuration::Duration elapsed =
end.time_since_epoch() - start.time_since_epoch();
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
return elapsed.count();
}
void TargetStats::CollectStats(Target &target) {
m_module_identifiers.clear();
for (ModuleSP module_sp : target.GetImages().Modules())
m_module_identifiers.emplace_back((intptr_t)module_sp.get());
}
json::Value ModuleStats::ToJSON() const {
json::Object module;
EmplaceSafeString(module, "path", path);
EmplaceSafeString(module, "uuid", uuid);
EmplaceSafeString(module, "triple", triple);
module.try_emplace("identifier", identifier);
module.try_emplace("symbolTableParseTime", symtab_parse_time);
module.try_emplace("symbolTableIndexTime", symtab_index_time);
module.try_emplace("symbolTableLoadedFromCache", symtab_loaded_from_cache);
module.try_emplace("symbolTableSavedToCache", symtab_saved_to_cache);
module.try_emplace("debugInfoParseTime", debug_parse_time);
module.try_emplace("debugInfoIndexTime", debug_index_time);
module.try_emplace("debugInfoByteSize", (int64_t)debug_info_size);
module.try_emplace("debugInfoIndexLoadedFromCache",
debug_info_index_loaded_from_cache);
module.try_emplace("debugInfoIndexSavedToCache",
debug_info_index_saved_to_cache);
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
module.try_emplace("debugInfoEnabled", debug_info_enabled);
module.try_emplace("debugInfoHadVariableErrors",
debug_info_had_variable_errors);
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
module.try_emplace("symbolTableStripped", symtab_stripped);
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-09 18:48:17 -08:00
if (!symfile_path.empty())
module.try_emplace("symbolFilePath", symfile_path);
if (!symfile_modules.empty()) {
json::Array symfile_ids;
for (const auto symfile_id: symfile_modules)
symfile_ids.emplace_back(symfile_id);
module.try_emplace("symbolFileModuleIdentifiers", std::move(symfile_ids));
}
return module;
}
llvm::json::Value ConstStringStats::ToJSON() const {
json::Object obj;
obj.try_emplace<int64_t>("bytesTotal", stats.GetBytesTotal());
obj.try_emplace<int64_t>("bytesUsed", stats.GetBytesUsed());
obj.try_emplace<int64_t>("bytesUnused", stats.GetBytesUnused());
return obj;
}
json::Value TargetStats::ToJSON(Target &target) {
CollectStats(target);
json::Array json_module_uuid_array;
for (auto module_identifier : m_module_identifiers)
json_module_uuid_array.emplace_back(module_identifier);
json::Object target_metrics_json{
{m_expr_eval.name, m_expr_eval.ToJSON()},
{m_frame_var.name, m_frame_var.ToJSON()},
{"moduleIdentifiers", std::move(json_module_uuid_array)}};
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
if (m_launch_or_attach_time && m_first_private_stop_time) {
double elapsed_time =
elapsed(*m_launch_or_attach_time, *m_first_private_stop_time);
target_metrics_json.try_emplace("launchOrAttachTime", elapsed_time);
}
if (m_launch_or_attach_time && m_first_public_stop_time) {
double elapsed_time =
elapsed(*m_launch_or_attach_time, *m_first_public_stop_time);
target_metrics_json.try_emplace("firstStopTime", elapsed_time);
}
target_metrics_json.try_emplace("targetCreateTime",
m_create_time.get().count());
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
json::Array breakpoints_array;
double totalBreakpointResolveTime = 0.0;
// Rport both the normal breakpoint list and the internal breakpoint list.
for (int i = 0; i < 2; ++i) {
BreakpointList &breakpoints = target.GetBreakpointList(i == 1);
std::unique_lock<std::recursive_mutex> lock;
breakpoints.GetListMutex(lock);
size_t num_breakpoints = breakpoints.GetSize();
for (size_t i = 0; i < num_breakpoints; i++) {
Breakpoint *bp = breakpoints.GetBreakpointAtIndex(i).get();
breakpoints_array.push_back(bp->GetStatistics());
totalBreakpointResolveTime += bp->GetResolveTime().count();
}
}
ProcessSP process_sp = target.GetProcessSP();
if (process_sp) {
UnixSignalsSP unix_signals_sp = process_sp->GetUnixSignals();
if (unix_signals_sp)
target_metrics_json.try_emplace("signals",
unix_signals_sp->GetHitCountStatistics());
uint32_t stop_id = process_sp->GetStopID();
target_metrics_json.try_emplace("stopCount", stop_id);
}
target_metrics_json.try_emplace("breakpoints", std::move(breakpoints_array));
target_metrics_json.try_emplace("totalBreakpointResolveTime",
totalBreakpointResolveTime);
target_metrics_json.try_emplace("sourceMapDeduceCount", m_source_map_deduce_count);
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
return target_metrics_json;
}
void TargetStats::SetLaunchOrAttachTime() {
m_launch_or_attach_time = StatsClock::now();
m_first_private_stop_time = llvm::None;
}
void TargetStats::SetFirstPrivateStopTime() {
// Launching and attaching has many paths depending on if synchronous mode
// was used or if we are stopping at the entry point or not. Only set the
// first stop time if it hasn't already been set.
if (!m_first_private_stop_time)
m_first_private_stop_time = StatsClock::now();
}
void TargetStats::SetFirstPublicStopTime() {
// Launching and attaching has many paths depending on if synchronous mode
// was used or if we are stopping at the entry point or not. Only set the
// first stop time if it hasn't already been set.
if (!m_first_public_stop_time)
m_first_public_stop_time = StatsClock::now();
}
void TargetStats::IncreaseSourceMapDeduceCount() {
++m_source_map_deduce_count;
}
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
bool DebuggerStats::g_collecting_stats = false;
llvm::json::Value DebuggerStats::ReportStatistics(Debugger &debugger,
Target *target) {
json::Array json_targets;
json::Array json_modules;
double symtab_parse_time = 0.0;
double symtab_index_time = 0.0;
double debug_parse_time = 0.0;
double debug_index_time = 0.0;
uint32_t symtabs_loaded = 0;
uint32_t symtabs_saved = 0;
uint32_t debug_index_loaded = 0;
uint32_t debug_index_saved = 0;
uint64_t debug_info_size = 0;
if (target) {
json_targets.emplace_back(target->ReportStatistics());
} else {
for (const auto &target : debugger.GetTargetList().Targets())
json_targets.emplace_back(target->ReportStatistics());
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
}
std::vector<ModuleStats> modules;
std::lock_guard<std::recursive_mutex> guard(
Module::GetAllocationModuleCollectionMutex());
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
const uint64_t num_modules = Module::GetNumberAllocatedModules();
uint32_t num_debug_info_enabled_modules = 0;
uint32_t num_modules_has_debug_info = 0;
uint32_t num_stripped_modules = 0;
for (size_t image_idx = 0; image_idx < num_modules; ++image_idx) {
Module *module = Module::GetAllocatedModuleAtIndex(image_idx);
ModuleStats module_stat;
module_stat.identifier = (intptr_t)module;
module_stat.path = module->GetFileSpec().GetPath();
if (ConstString object_name = module->GetObjectName()) {
module_stat.path.append(1, '(');
module_stat.path.append(object_name.GetStringRef().str());
module_stat.path.append(1, ')');
}
module_stat.uuid = module->GetUUID().GetAsString();
module_stat.triple = module->GetArchitecture().GetTriple().str();
module_stat.symtab_parse_time = module->GetSymtabParseTime().get().count();
module_stat.symtab_index_time = module->GetSymtabIndexTime().get().count();
Symtab *symtab = module->GetSymtab();
if (symtab) {
module_stat.symtab_loaded_from_cache = symtab->GetWasLoadedFromCache();
if (module_stat.symtab_loaded_from_cache)
++symtabs_loaded;
module_stat.symtab_saved_to_cache = symtab->GetWasSavedToCache();
if (module_stat.symtab_saved_to_cache)
++symtabs_saved;
}
SymbolFile *sym_file = module->GetSymbolFile();
if (sym_file) {
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-09 18:48:17 -08:00
if (sym_file->GetObjectFile() != module->GetObjectFile())
module_stat.symfile_path =
sym_file->GetObjectFile()->GetFileSpec().GetPath();
module_stat.debug_index_time = sym_file->GetDebugInfoIndexTime().count();
module_stat.debug_parse_time = sym_file->GetDebugInfoParseTime().count();
module_stat.debug_info_size = sym_file->GetDebugInfoSize();
module_stat.debug_info_index_loaded_from_cache =
sym_file->GetDebugInfoIndexWasLoadedFromCache();
if (module_stat.debug_info_index_loaded_from_cache)
++debug_index_loaded;
module_stat.debug_info_index_saved_to_cache =
sym_file->GetDebugInfoIndexWasSavedToCache();
if (module_stat.debug_info_index_saved_to_cache)
++debug_index_saved;
Fix a double debug info size counting in top level stats for "statistics dump". This mainly affects Darwin targets (macOS, iOS, tvOS and watchOS) when these targets don't use dSYM files and the debug info was in the .o files. All modules, including the .o files that are loaded by the debug maps, were in the global module list. This was great because it allows us to see each .o file and how much it contributes. There were virtual functions on the SymbolFile class to fetch the symtab/debug info parse and index times, and also the total debug info size. So the main executable would add all of the .o file's stats together and report them as its own data. Then the "totalDebugInfoSize" and many other "totalXXX" top level totals were all being added together. This stems from the fact that my original patch only emitted the modules for a target at the start of the patch, but as comments from the reviews came in, we switched to emitting all of the modules from the global module list. So this patch fixes it so when we have a SymbolFileDWARFDebugMap that loads .o files, the main executable will have no debug info size or symtab/debug info parse/index times, but each .o file will have its own data as a separate module. Also, to be able to tell when/if we have a dSYM file I have added a "symbolFilePath" if the SymbolFile for the main modules path doesn't match that of the main executable. We also include a "symbolFileModuleIdentifiers" key in each module if the module does have multiple lldb_private::Module objects that contain debug info so that you can track down the information for a module and add up the contributions of all of the .o files. Tests were added that are labeled with @skipUnlessDarwin and @no_debug_info_test that test all of this functionality so it doesn't regress. For a module with a dSYM file, we can see the "symbolFilePath" is included: ``` "modules": [ { "debugInfoByteSize": 1070, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4873280600, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out", "symbolFilePath": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_dsym_binary_has_symfile_in_stats/a.out.dSYM/Contents/Resources/DWARF/a.out", "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 7.8999999999999996e-05, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "E1F7D85B-3A42-321E-BF0D-29B103F5F2E3" }, ``` And for the DWARF in .o file case we can see the "symbolFileModuleIdentifiers" in the executable's module stats: ``` "modules": [ { "debugInfoByteSize": 0, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 0, "identifier": 4603526968, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/a.out", "symbolFileModuleIdentifiers": [ 4604429832 ], "symbolTableIndexTime": 7.9999999999999996e-06, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0.000112, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx12.0.0", "uuid": "57008BF5-A726-3DE9-B1BF-3A9AD3EE8569" }, ``` And the .o file for 4604429832 looks like: ``` { "debugInfoByteSize": 1028, "debugInfoIndexLoadedFromCache": false, "debugInfoIndexSavedToCache": false, "debugInfoIndexTime": 0, "debugInfoParseTime": 6.0999999999999999e-05, "identifier": 4604429832, "path": "/Users/gclayton/Documents/src/lldb/main/Debug/lldb-test-build.noindex/commands/statistics/basic/TestStats.test_no_dsym_binary_has_symfile_identifiers_in_stats/main.o", "symbolTableIndexTime": 0, "symbolTableLoadedFromCache": false, "symbolTableParseTime": 0, "symbolTableSavedToCache": false, "triple": "arm64-apple-macosx" } ``` Differential Revision: https://reviews.llvm.org/D119400
2022-02-09 18:48:17 -08:00
ModuleList symbol_modules = sym_file->GetDebugInfoModules();
for (const auto &symbol_module: symbol_modules.Modules())
module_stat.symfile_modules.push_back((intptr_t)symbol_module.get());
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
module_stat.symtab_stripped = module->GetObjectFile()->IsStripped();
if (module_stat.symtab_stripped)
++num_stripped_modules;
module_stat.debug_info_enabled = sym_file->GetLoadDebugInfoEnabled() &&
module_stat.debug_info_size > 0;
module_stat.debug_info_had_variable_errors =
sym_file->GetDebugInfoHadFrameVariableErrors();
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
if (module_stat.debug_info_enabled)
++num_debug_info_enabled_modules;
if (module_stat.debug_info_size > 0)
++num_modules_has_debug_info;
}
symtab_parse_time += module_stat.symtab_parse_time;
symtab_index_time += module_stat.symtab_index_time;
debug_parse_time += module_stat.debug_parse_time;
debug_index_time += module_stat.debug_index_time;
debug_info_size += module_stat.debug_info_size;
json_modules.emplace_back(module_stat.ToJSON());
}
ConstStringStats const_string_stats;
json::Object json_memory{
{"strings", const_string_stats.ToJSON()},
};
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
json::Object global_stats{
{"targets", std::move(json_targets)},
{"modules", std::move(json_modules)},
{"memory", std::move(json_memory)},
{"totalSymbolTableParseTime", symtab_parse_time},
{"totalSymbolTableIndexTime", symtab_index_time},
{"totalSymbolTablesLoadedFromCache", symtabs_loaded},
{"totalSymbolTablesSavedToCache", symtabs_saved},
{"totalDebugInfoParseTime", debug_parse_time},
{"totalDebugInfoIndexTime", debug_index_time},
{"totalDebugInfoIndexLoadedFromCache", debug_index_loaded},
{"totalDebugInfoIndexSavedToCache", debug_index_saved},
{"totalDebugInfoByteSize", debug_info_size},
Introduce new symbol on-demand for debug info This diff introduces a new symbol on-demand which skips loading a module's debug info unless explicitly asked on demand. This provides significant performance improvement for application with dynamic linking mode which has large number of modules. The feature can be turned on with: "settings set symbols.load-on-demand true" The feature works by creating a new SymbolFileOnDemand class for each module which wraps the actual SymbolFIle subclass as member variable. By default, most virtual methods on SymbolFileOnDemand are skipped so that it looks like there is no debug info for that module. But once the module's debug info is explicitly requested to be enabled (in the conditions mentioned below) SymbolFileOnDemand will allow all methods to pass through and forward to the actual SymbolFile which would hydrate module's debug info on-demand. In an internal benchmark, we are seeing more than 95% improvement for a 3000 modules application. Currently we are providing several ways to on demand hydrate a module's debug info: * Source line breakpoint: matching in supported files * Stack trace: resolving symbol context for an address * Symbolic breakpoint: symbol table match guided promotion * Global variable: symbol table match guided promotion In all above situations the module's debug info will be on-demand parsed and indexed. Some follow-ups for this feature: * Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled * Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs * Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs * Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed) Differential Revision: https://reviews.llvm.org/D121631
2022-04-20 07:30:53 -07:00
{"totalModuleCount", num_modules},
{"totalModuleCountHasDebugInfo", num_modules_has_debug_info},
{"totalDebugInfoEnabled", num_debug_info_enabled_modules},
{"totalSymbolTableStripped", num_stripped_modules},
Modify "statistics dump" to dump JSON. This patch is a smaller version of a previous patch https://reviews.llvm.org/D110804. This patch modifies the output of "statistics dump" to be able to get stats from the current target. It adds 3 new stats as well. The output of "statistics dump" is now emitted as JSON so that it can be used to track performance and statistics and the output could be used to populate a database that tracks performance. Sample output looks like: (lldb) statistics dump { "expressionEvaluation": { "failures": 0, "successes": 0 }, "firstStopTime": 0.34164492800000001, "frameVariable": { "failures": 0, "successes": 0 }, "launchOrAttachTime": 0.31969605400000001, "targetCreateTime": 0.0040863039999999998 } The top level keys are: "expressionEvaluation" which replaces the previous stats that were emitted as plain text. This dictionary contains the success and fail counts. "frameVariable" which replaces the previous stats for "frame variable" that were emitted as plain text. This dictionary contains the success and fail counts. "targetCreateTime" contains the number of seconds it took to create the target and load dependent libraries (if they were enabled) and also will contain symbol preloading times if that setting is enabled. "launchOrAttachTime" is the time it takes from when the launch/attach is initiated to when the first private stop occurs. "firstStopTime" is the time in seconds that it takes to stop at the first stop that is presented to the user via the LLDB interface. This value will only have meaning if you set a known breakpoint or stop location in your code that you want to measure as a performance test. This diff is also meant as a place to discuess what we want out of the "statistics dump" command before adding more funcionality. It is also meant to clean up the previous code that was storting statistics in a vector of numbers within the lldb_private::Target class. Differential Revision: https://reviews.llvm.org/D111686
2021-10-20 14:49:09 -07:00
};
return std::move(global_stats);
}