These functions from C++ ABI are defined in
compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp and are supposed to
replace implementations from libstdc++/libc++abi.
We need to export them similar to why we need to export other
interceptors and TSan runtime functions - e.g. if a dlopen-ed shared
library depends on `__cxa_guard_acquire`, it needs to pick up the
exported definition from the TSan runtime that was linked into the main
executable calling the dlopen()
However, because the `__cxa_guard_` functions don't use traditional
interceptor machinery, they are omitted from the auto-generated
`libclang_rt.tsan.a.syms` files. Fix this by adding them to
tsan.syms.extra file explicitly.
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
Commit c6f501d479 fixed an issue where some tests were incorrectly
marked as unsupported for a bootstrapping build. This exposed in our
'slow' full-bootstrap qemu-system CI that the fdr-mode.cpp fails on
RISC-V. We mark it as unsupported. I believe _xray_ArgLoggerEntry needs
to be implemented in xray_trampoline_risc*.S for this to work.
Trace.local_head is currently uninitialized when Trace is created. It is
first initialized when the first event is added to the trace, via the
first call to TraceSwitchPartImpl.
However, ThreadContext::OnFinished uses local_head, assuming that it is
initialized. If it has not been initialized, we have undefined behavior,
likely crashing if the contents are garbage. The allocator (Alloc)
reuses previously allocations, so the contents of the uninitialized
memory are arbitrary.
In a C/C++ TSAN binary it is likely very difficult for a thread to start
and exit without a single event inbetween. For Go programs, code running
in the Go runtime itself is not TSan-instrumented, so goroutines that
exclusively run runtime code (such as GC workers) can quite reasonably
have no TSan events.
The addition of such a goroutine to the Go test.c is sufficient to
trigger this case, though for reliable failure (segfault) I've found it
necessary to poison the ThreadContext allocation like so:
```
diff --git a/compiler-rt/lib/tsan/rtl/tsan_rtl.cpp b/compiler-rt/lib/tsan/rtl/tsan_rtl.cpp index feee566f44..352db9aa7c 100644
--- a/compiler-rt/lib/tsan/rtl/tsan_rtl.cpp
+++ b/compiler-rt/lib/tsan/rtl/tsan_rtl.cpp
@@ -392,7 +392,9 @@
report_mtx(MutexTypeReport),
nreported(),
thread_registry([](Tid tid) -> ThreadContextBase* {
- return new (Alloc(sizeof(ThreadContext))) ThreadContext(tid);
+ void* ptr = Alloc(sizeof(ThreadContext));
+ internal_memset(ptr, 0xde, sizeof(ThreadContext));
+ return new (ptr) ThreadContext(tid);
}),
racy_mtx(MutexTypeRacy),
racy_stacks(),
```
The fix is trivial: local_head should be zero-initialized.
Move the test utility to get the resident pages into linux.cpp.
Add some specific tests for this function.
Add displaying of resident pages from in the secondary cache when
calling getStats.
Adds a flag COMPILER_RT_PROFILE_BAREMETAL, which disables the parts of
the profile runtime which require a filesystem or malloc. This minimal
library only requires string.h from the C library.
This is useful for profiling or code coverage of baremetal images, which
don't have filesystem APIs, and might not have malloc configured (or
have limited heap space).
Expected usage:
- Add code to your project to call
`__llvm_profile_get_size_for_buffer()` and
`__llvm_profile_write_buffer()` to write the profile data to a buffer in
memory, and then copy that data off the device using target-specific
tools.
- If you're using a linker script, set up your linker script to map the
profiling and coverage input sections to corresponding output sections
with the same name, and mark them KEEP. `__llvm_covfun` and
`__llvm_covmap` are non-allocatable, `__llvm_prf_names` is read-only
allocatable, and `__llvm_prf_cnts` and `__llvm_prf_data` are read-write
allocatable.
- The resulting data is in same format as the non-baremetal profiles.
There's some room for improvement here in the future for doing profiling
and code coverage for baremetal. If we revised the profiling format, and
introduced some additional host tooling, we could move some of the
metadata into non-allocated sections, and construct the profraw file on
the host. But this patch is sufficient for some use-cases.
This test has begun failing on iossim with 'sh: sort: command not found'
in the stderr. I believe this may be due to the change to the lit
internal shell not having 'sort' in it's path.
This patch adds the full path /usr/bin/sort to work around this.
This commit addresses a longstanding TODO comment, by doing the
following:
* Modifies the CMakeLists to add the new test configs
* Modifies the relevant lit file to add the required envs
* Fixes the FileCheck match in
`Darwin/symbolizer-function-offset-atos.cpp`
* XFAILs any appropriate tests
rdar://107758331
Instead of getting a lock and then checking/modifying the Initialization
variable, make it an atomic. Doing this, we can remove one of the
mutexes in shared TSDs and avoid any potential lock contention in both
shared TSDs and exclusive TSDs if multiple threads do allocation
operations at the same time.
Add two new tests that make sure no crashes occur if multiple threads
try and do allocations at the same time.
#170809 added the child_stdin_fd_ field on SymbolizerProcess to allow
the parent process to hold on to the read in of the child's stdin pipe.
This was to avoid SIGPIPE.
However, the `StartSubprocess` path still closes the stdin fd in the
parent here:
7f5ed91684/compiler-rt/lib/sanitizer_common/sanitizer_posix_libcdep.cpp (L525-L535)
This could cause a double-close of this fd (problematic in the case of
fd reuse).
This moves the `child_stdin_fd_` field to only be initialized on the
posix_spawn path. This should ensure #170809 only truly affects Darwin.
Due to a legacy incompatibility with `atos`, we were allocating a pty
whenever we spawned the symbolizer. This is no longer necessary and we
can use a regular ol' pipe.
This PR is split into two commits:
- The first removes the pty allocation and replaces it with a pipe. This
relocates the `CreateTwoHighNumberedPipes` call to be common to the
`posix_spawn` and `StartSubprocess` path.
- The second commit adds the `child_stdin_fd_` field to
`SymbolizerProcess`, storing the read end of the stdin pipe. By holding
on to this fd for the lifetime of the symbolizer, we are able to avoid
getting SIGPIPE (which would occur when we write to a pipe whose
read-end had been closed due to the death of the symbolizer). This will
be very close to solving #120915, but this PR is intentionally not
touching the non-posix_spawn path.
rdar://165894284
Use mmap.PAGESIZE and fallback to os.sysconf if it is not available.
This allows to query the page size on Windows too, which has mmap but
not sysconf.
This is a pedantic change because Windows has a fixed page size of 4KiB.
I found this solution independently of #168857, and thought it might be
worth committing, since this is technically more correct.
[mmap.PAGESIZE implementation in
CPython](88cd5d9850/Modules/mmapmodule.c (L47))
This is leveraging work which has already been done for Android, which
ships 39-bit VA kernels, and extending it to other embedded Linux
targets.
(SANITIZER_AARCH64_39BIT_VA was added in 58c8f57681.)
This is leveraging work which has already been done for Android, which
ships 39-bit VA kernels, and extending it to other embedded Linux
targets.
(SANITIZER_AARCH64_39BIT_VA was added in 58c8f57681.)
When doing a LLVM_ENABLE_RUNTIMES-based build of clang+compiler_rt, the
`check-builtins` target is missing and builtins tests are not run
(#112105, #144090).
This provides one possible path forward for enabling these tests.
The approach taken here is to test the builtins with the `compiler-rt`
runtime build (i.e. only if you pass
`LLVM_ENABLE_RUNTIMES='compiler-rt'`). The builtins tests currently rely
on shared test infrastructure in `compiler-rt/`, so this is probably the
easiest solution without relocating the builtins and their tests outside
of `compiler-rt/`.
The main challenge is that the built-ins test configuration expects to
be able to inspect the builtin target and see the sources used to build
it:
```
get_target_property(BUILTIN_LIB_SOURCES "${BUILTIN_LIB_TARGET_NAME}" SOURCES)
```
Since the builtins build and runtimes build are separate under
LLVM_ENABLE_RUNTIMES, this target inspection is not possible. To get
around this, we write a temporary file alongside each builtins library
containing the list of sources (e.g.
`"${CMAKE_BINARY_DIR}/clang_rt.builtins-${arch}.sources.txt"`). Then, we
introduce an undocumented compiler-rt option
`COMPILER_RT_FORCE_TEST_BUILTINS_DIR` (which is only intended to be used
by the LLVM_ENABLE_RUNTIMES build, and could be removed after builtins
re relocated) that passes the path to the directory containing this
file, and configures builtins tests (even though the runtimes build has
`COMPILER_RT_BUILD_BUILTINS=OFF`)
rdar://163518748
This test is failing on some buildbots now that the internal shell has
been turned on and was failing previously on some ppc bots when turning
it on a while back (before it got reverted).
At least one X86 bot is barely hitting the limit
(https://lab.llvm.org/buildbot/#/builders/174/builds/28487 224MB-235MB).
This likely needs to be bumped due to changes in the process tree (now
that we invoke things through python rather than a bash shell) with the
enablement of the internal shell.
I have seen a failure whereby the fuzzer failed to reach the expected
input and thus failed the test.
This patch bumps the max executions to 10,000,000 in order to give the
fuzzer a better chance of reaching the expected input. Most runs
complete successfully, so I do not see this adding test time in the
general case; I believe it's a fair tradeoff for the unlucky seed to run
for longer if it reduces the noise from false positives. Note, this
updates a different `RUN:` to
https://github.com/llvm/llvm-project/pull/165402.
rdar://162122184
This patch reverts the change that made clang prefer the toolchain
libc++.dylib when there is one (#170303), and the subsequent test
workaround we landed to fix bots (#170912).
We are seeing some failure on macOS LLDB bots that need to be
investigated, and that will require more time than I can spare
before the end of today.
This reverts commits bad1a88963 and 190b8d0b.
The recent relanding of the internal shell broke one of the debuginfod
tests as it is not tested by any upstream buildbot due to the use of
curl. Rewriting the test to not use subshells is pretty simple.
When libc++ is bootstrapped with clang, the resulting clang uses the just-built libc++
headers from <install>/bin/../include/c++/v1. However, before this patch, clang would
still use the system-provided libc++.dylib (usually in the Apple SDK) because it would
fail to add the corresponding linker flag to find the just-built libc++. After this
patch, Clang will instead link against the toolchain provided `libc++.dylib` in
`<install>/lib` if it exists, which will result in programs being linked against
corresponding libc++ headers and dylib.
Fixes#77653
rdar://107060541
Intended use-case is for threads that use (or switch to) stack with
special properties e.g. backed by MADV_DONTDUMP memory.
---------
Co-authored-by: Vitaly Buka <vitalybuka@google.com>
This test fails with the internal shell as we implement ulimit with a
python wrapper. This python wrapper fails on recent versions of MacOS to
set the stack size limit lower, or even to the current values. More
discussion is in https://github.com/python/cpython/issues/78783 towards
the bottom (it seems like a new issue should be opened).
It does not seem like we lose significant test coverage by disabling
this on MacOS as the test running on Linux should catch any major
regressions. We can also simply reenable once the issue is fixed
(although writing a simple c program with calls setrlimit directly also
fails in the smae manner).
Tbis addresses a buildbot failure now that these tests actually run more
broadly.
error: ALWAYSINSTR: expected string not found in input
// ALWAYSINSTR: {{.*function-name:.*main.*}}
^
<stdin>:1:1: note: scanning from here
COMPILER_RT_STANDALONE_BUILD is set when doing a bootstrapping build
through LLVM_ENABLE_RUNTIMES with the CMake source directory being in
llvm/. This patch changes the XRay tests to also detect that we have
LLVM sources and the llvm-xray tool if we are in a bootstrapping build
through the use of the LLVM_TREE_AVAILABLE variable which is set in
runtimes/CMakeLists.txt.
