Commit Graph

3442 Commits

Author SHA1 Message Date
Julian Lettner
65e9f98861 [LSan] Fix __sanitizer_print_stack_trace via fast unwinder
Summary: Quick follow-up to: https://reviews.llvm.org/D58156

Reviewers: vitalybuka

Differential Revision: https://reviews.llvm.org/D58358

llvm-svn: 354522
2019-02-20 22:28:11 +00:00
Vitaly Buka
e4a7a46151 Fix license headers
llvm-svn: 354500
2019-02-20 18:55:52 +00:00
Evgeniy Stepanov
bdbbfdc8c2 [msan] Fix name_to_handle_at test on overlayfs.
Udev supports name_to_handle_at. Use /dev/null instead of /bin/cat.

llvm-svn: 354402
2019-02-19 23:41:42 +00:00
Vitaly Buka
8ca98c2bc2 [msan] Remove cxa_atexit_race.cc
Summary:
The goal of the test to check that msan does not crash when code is racy on __cxa_atexit. Original crash was caused by race condition in the glibc. With
the msan patch the msan does not crashes however the race is still there and the test triggers it.

Because the test relies on triggering of undefined behavior results are not
very predictable and it may occasionally crashes or hangs.

I don't see how to reasonably improve the test, so I remove it.

Reviewers: eugenis, peter.smith

Subscribers: jfb, jdoerfert, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D58396

llvm-svn: 354377
2019-02-19 20:36:52 +00:00
Julian Lettner
9f14d169cc [LSan][Darwin][NFC] Add comment explaining test failure
llvm-svn: 354290
2019-02-18 22:29:17 +00:00
Julian Lettner
eb3bcc1c95 [Sanitizer] On Darwin __sanitizer_print_stack_trace only prints topmost frame
In compiler-rt we have the notion of a `fast` and a `slow` stack
unwinder. Darwin currently only supports the fast unwinder.

From reading the code, my understanding is that
`BufferedStackTrace::Unwind` can be called with `bp=0, stack_top=0,
stack_bottom=0, request_fast_unwind=false`. If
`request_fast_unwind=true`, then we alos need to supply bp, stack_top,
and stack_bottom.

However, `BufferedStackTrace::Unwind` uses
`StackTrace::WillUseFastUnwind` which will adapt `request_fast_unwind`
if the requested unwinder is not supported. On Darwin, the result is
that we don't pass actual values for bp, stack_top, and stack_bottom,
but end up using the fast unwinder. The tests then fail because we only
print the topmost stack frame.

This patch adds a check to `WillUseFastUnwind` at the point of usage to
avoid the mismatch between `request_fast_unwind` and what `Unwind`
actually does. I am also interested in cleaning up the
`request_fast_unwind` machinery so this patch just the simplest thing
possible so I can enable the tests.

Reviewers: vitalybuka, vsk

Differential Revision: https://reviews.llvm.org/D58156

llvm-svn: 354282
2019-02-18 18:47:49 +00:00
Jonas Hahnfeld
0a9cb239a6 [compiler-rt] Fix broken sanitizer bots (hopefully)
According to the logs and local debugging there were two issues:
1) tsan tests listed libc++.a before the source file. That's usually
   ok for shared libraries, but the linker will not add symbols from
   a static library unless needed at that time. As a result the tests
   that rely upon symbols from the library (and not only include the
   headers) had undefined references.
   To solve this I'm adding a new substitution %link_libcxx_tsan which
   expands to libc++.a if available.
2) The target Fuzzer-x86_64-Test linked in SANITIZER_TEST_CXX_LIBRARIES
   which defaults to -lstdc++. This resulted in error messages like
     hidden symbol '_ZdlPv' is not defined locally
     hidden symbol '_Znwm' is not defined locally
   when using GNU gold (ld.bfd and lld are fine). Removing the linkage
   is fine because we build a custom libc++ for that purpose.

llvm-svn: 354231
2019-02-17 18:47:33 +00:00
Jonas Hahnfeld
66c60d9d71 [compiler-rt] Build custom libcxx with libcxxabi
This changes add_custom_libcxx to also build libcxxabi and merges
the two into a static and hermetic library.
There are multiple advantages:
1) The resulting libFuzzer doesn't expose C++ internals and looks
   like a plain C library.
2) We don't have to manually link in libstdc++ to provide cxxabi.
3) The sanitizer tests cannot interfere with an installed version
   of libc++.so in LD_LIBRARY_PATH.

Differential Revision: https://reviews.llvm.org/D58013

llvm-svn: 354212
2019-02-17 12:16:20 +00:00
Kostya Serebryany
b6ca1e7259 [libFuzzer] make len_control less agressive: set the initial max len to the length of the largest seed. This was the original intent, but... Now, with a test, to ensure it stays this way
llvm-svn: 354191
2019-02-16 01:23:41 +00:00
Kostya Serebryany
312af158b0 [libFuzzer] fork mode: try harder to cleanup after itself
llvm-svn: 354186
2019-02-16 00:14:16 +00:00
Julian Lettner
68c6764c20 [Sanitizer] iOS: Pull up parallelism_group handling into common.lit.config
Serial execution on iOS devices is not specific to sanitizers. We want
to throttle all on-device tests. Pull the setting of the
parallelism_group up into the common lit configuration file.

Rename `darwin-ios-device-sanitizer` to `ios-device`. This group is not
specific to sanitizers and (theoretically) independent from the host OS.

Note that we don't support running unit tests on-device (there are no
configurations generated for that). If that ever changes, we also need
this configuration in `unittests/lit.common.unit.cfg`.

Reviewers: delcypher

Differential Revision: https://reviews.llvm.org/D58209

llvm-svn: 354179
2019-02-15 22:12:36 +00:00
Kostya Serebryany
9982ee5472 [libFuzzer] form mode: add -ignore_crashes flag, honor the max_total_time flag, print the number of ooms/timeouts/crashes, fix a typo
llvm-svn: 354175
2019-02-15 21:51:15 +00:00
Evgeniy Stepanov
360163f671 Fix unsymbolized stack history printing.
Summary:
When symbols are unavailable, the current code prints
  sp: ... pc: ... (null) (null)
instead of module name + offset.

Change the output to include module name and offset, and also to match
the regular sanitizer stack trace format so that it is recognized by
symbolize.py out of the box.

Reviewers: kcc, pcc

Subscribers: kubamracek, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58267

llvm-svn: 354157
2019-02-15 18:38:23 +00:00
Evgeniy Stepanov
a70d88c7a3 Runtime flags for malloc bisection.
Reviewers: kcc, pcc

Subscribers: kubamracek, mgorny, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58162

llvm-svn: 354156
2019-02-15 18:38:14 +00:00
Evgeniy Stepanov
2c0483f5a6 Fix false positive when tag_in_malloc=0,tag_in_free=1.
Summary:
With tag_in_free=1, malloc() can not assume that the memory is untagged,
and needs to retag is to 0.

Reviewers: pcc, kcc

Subscribers: kubamracek, jfb, jdoerfert, llvm-commits

Tags: #llvm

Differential Revision: https://reviews.llvm.org/D58158

llvm-svn: 354155
2019-02-15 18:38:03 +00:00
Kostya Serebryany
96f81bc679 [libFuzzer] when doing the merge, keep track of the coveraged edges, not just features
llvm-svn: 354076
2019-02-14 23:12:33 +00:00
Ana Pazos
bbb8129b2c Set hidden attribute on lprofMergeValueProfData
Summary:
The changes in https://reviews.llvm.org/D44847 cause load time failure
due to lprofMergeValueProfData in Android libs enabled with profile generation:

"dlopen failed: cannot locate symbol "lprofMergeValueProfData" referenced by..."

Marking lprofMergeValueProfData as hidden so the correct in-module definition
 is picked by the linker.

Reviewers: davidxl

Reviewed By: davidxl

Subscribers: efriedma, xur, davidxl, llvm-commits

Differential Revision: https://reviews.llvm.org/D55893

llvm-svn: 354064
2019-02-14 21:38:40 +00:00
Douglas Yung
7f95f96391 Revert "Temporarily disable calls to getgrnam/getgrnam_r in test due to it hitting unrelated issues in EGLIBC 2.19."
This reverts commit r353594.

We have updated our internal build bot to a newer version of LIBC which does not have this problem.

llvm-svn: 354014
2019-02-14 10:39:11 +00:00
Vitaly Buka
9a370ccca2 [msan] Don't delete MSanAtExitRecord
Summary:
Pre 2.27 libc can run same atexit handler twice
We will keep MSanAtExitRecord and reset fun to mark it as executed.

Fix PR40162

Reviewers: eugenis

Subscribers: jfb, jdoerfert, #sanitizers, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D58221

llvm-svn: 354005
2019-02-14 02:51:55 +00:00
Dmitry Vyukov
76e961207b tsan: add fiber support
This patch adds functions for managing fibers:

__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()

See the added tests for use examples.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889

[The previous commit of this change was reverted,
this is a resubmit with a squashed fix for check_analyze.sh
and COMMON_INTERCEPTOR_NOTHING_IS_INITIALIZED]

llvm-svn: 353947
2019-02-13 13:21:24 +00:00
Diana Picus
4152a09e55 Revert "tsan: add fiber support"
This reverts commit r353817 because we think it broke AARch64 and
PowerPC buildbots.

llvm-svn: 353939
2019-02-13 11:32:56 +00:00
Kostya Serebryany
d085748484 [libFuzzer] a bit of refactoring of the fork mode
llvm-svn: 353910
2019-02-13 04:04:45 +00:00
Julian Lettner
55fb5314b3 [Sanitizer][NFC] Darwin: limit parallism for sanitizer_common tests
Many sanitizer_common tests (ASan, TSan) run with a "sanitized process
space" so we need to limit their parallism.

llvm-svn: 353909
2019-02-13 01:56:53 +00:00
Dmitry Vyukov
6e7089ad40 tsan: add fiber support
This patch adds functions for managing fibers:

__tsan_get_current_fiber()
__tsan_create_fiber()
__tsan_destroy_fiber()
__tsan_switch_to_fiber()
__tsan_set_fiber_name()

See the added tests for use examples.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D54889

llvm-svn: 353817
2019-02-12 10:11:58 +00:00
Petr Hosek
61b513351c [CMake] Avoid internal dependencies in the runtimes build
When performing runtimes build, dependencies like clang and
clang-headers aren't available. This was accidentally omitted in D57992.

llvm-svn: 353796
2019-02-12 04:13:58 +00:00
Kostya Serebryany
cdbb9dc962 [libFuzzer] teach the fork mode to ignore OOMs and timeouts
llvm-svn: 353792
2019-02-12 02:18:53 +00:00
Julian Lettner
ee82db57ec [libFuzzer] Make coverage.test work on ARM64
Summary:
This test instruments the following code with coverage, runs the fuzzer
once, and asserts that there are uncovered PCs. The ARM64 backend
optimizes this code using the `csel` (Conditional select) instruction,
which removes all branching from the resulting machine code. The test
then fails because we do not have any uncovered PCs. The easiest
solution for now is to turn off optimization for the DSOs used in this
test.

```
int DSO1(int a) {
  if (a < 123456)
    return 0;
  return 1;
}
```

rdar://47646400

Reviewers: kcc

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D58087

llvm-svn: 353780
2019-02-12 00:37:40 +00:00
Kostya Serebryany
8cc38effe2 [libFuzzer] run the -fork=1 tests only on linux
llvm-svn: 353777
2019-02-12 00:29:51 +00:00
Kostya Serebryany
63f48717b5 [libFuzzer] extend the -fork=1 functionality. Still not fully usable, but good enough for the first unit test
llvm-svn: 353775
2019-02-12 00:12:33 +00:00
Julian Lettner
6142bf2c51 [NFC] Re-enable XFAILed fuzzer test on iOS
llvm-svn: 353737
2019-02-11 19:33:01 +00:00
Vitaly Buka
4fefb72019 [tsan] Remove debug logging and disable test on ppc64be
llvm-svn: 353624
2019-02-09 19:28:52 +00:00
Vitaly Buka
3eca7f5139 [tsan] Remove debug logging and disable test on ppc64be
llvm-svn: 353623
2019-02-09 18:43:24 +00:00
Vitaly Buka
9419b5bd04 [tsan] Debug failing test on PPC bot
llvm-svn: 353617
2019-02-09 16:28:41 +00:00
Douglas Yung
b041a18bcf Temporarily disable calls to getgrnam/getgrnam_r in test due to it hitting unrelated issues in EGLIBC 2.19.
llvm-svn: 353594
2019-02-09 01:28:47 +00:00
Kostya Serebryany
b1e8b8149b [libFuzzer] remove two unused experimental flags
llvm-svn: 353573
2019-02-08 22:02:37 +00:00
Reid Kleckner
987d331fab [InstrProf] Implement static profdata registration
Summary:
The motivating use case is eliminating duplicate profile data registered
for the same inline function in two object files. Before this change,
users would observe multiple symbol definition errors with VC link, but
links with LLD would succeed.

Users (Mozilla) have reported that PGO works well with clang-cl and LLD,
but when using LLD without this static registration, we would get into a
"relocation against a discarded section" situation. I'm not sure what
happens in that situation, but I suspect that duplicate, unused profile
information was retained. If so, this change will reduce the size of
such binaries with LLD.

Now, Windows uses static registration and is in line with all the other
platforms.

Reviewers: davidxl, wmi, inglorion, void, calixte

Subscribers: mgorny, krytarowski, eraman, fedor.sergeev, hiraditya, #sanitizers, dmajor, llvm-commits

Tags: #sanitizers, #llvm

Differential Revision: https://reviews.llvm.org/D57929

llvm-svn: 353547
2019-02-08 19:03:50 +00:00
Kostya Serebryany
0719b3527f [libFuzzer] refactor the way we choose the element to cross-over with, NFC (expected1); add a flag -seed_inputs= to pass extra seed inputs as file paths, not dirs
llvm-svn: 353494
2019-02-08 01:20:54 +00:00
Kostya Serebryany
28e967aaa1 [libFuzzer] add a test for built-in CrossOver (there are unit tests for this, but it's worth having a full integration test like this)
llvm-svn: 353488
2019-02-08 00:41:29 +00:00
Reid Kleckner
17e2521724 [InstrProf] Port test suite to Windows
Summary:
Before this change, check-profile would run, but all tests would be
marked unsupported on Windows. This is the new status of 'check-profile'
after this change:

Testing Time: 6.66s
  Expected Passes    : 29
  Expected Failures  : 5
  Unsupported Tests  : 39

I moved many tests that exercise posix-y features like dlopen and DSOs
into the Posix subdirectory, and ran the tests on Linux to validate my
changes.

These are the remaining tests that I handled on a case by case basis:
- instrprof-path.c
  Passes, Fixed some path portability issues
- instrprof-gcov-exceptions.test
  Passes, the FileCheck actually succeeds on Windows, so I RUNX'd it
- instrprof-icall-promo.test
  XFAILed, probably due to C++ ABI differences in vtables
- instrprof-merge-match.test
- instrprof-merge.c
- instrprof-merging.cpp
  XFAILed, These seem like real bugs that need fixing
- instrprof-version-mismatch.c
  XFAILed, Overriding the weak version symbol doesn't work
- instrprof-without-libc.c
  UNSUPPORTED, test needs an executable symbol table, Windows has none

Reviewers: davidxl, wmi, void

Subscribers: fedor.sergeev, #sanitizers, llvm-commits

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57853

llvm-svn: 353435
2019-02-07 17:52:05 +00:00
Dmitry Vyukov
17132b62e0 tsan: Implement pthread_exit() interceptor for Thread sanitizer
This change is preparation for fiber support.

Author: yuri (Yuri Per)
Reviewed in: https://reviews.llvm.org/D57876
Context: https://reviews.llvm.org/D54889

llvm-svn: 353385
2019-02-07 10:46:02 +00:00
Vitaly Buka
c41835230a [sanitizer] Fix Android tests
On Android some fields can be null

llvm-svn: 353377
2019-02-07 07:40:33 +00:00
Vitaly Buka
2b93ef7ef6 [sanitizer] Re-enabled getpw_getgr.cc on Android
Reviewers: eugenis

Subscribers: srhines, kubamracek, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57785

llvm-svn: 353366
2019-02-07 01:55:07 +00:00
Vitaly Buka
eb7031ecf6 [sanitizer] Don't unpoison buffer in getpw/getgr functions
Summary:
Buffer should be referenced by results so used parts will be unpoisoned with unpoison_group and unpoison_passwd.

This fixes TSAN performance issue made us to disable this interceptors.

Reviewers: eugenis, dvyukov

Subscribers: srhines, kubamracek, krytarowski, #sanitizers

Tags: #sanitizers

Differential Revision: https://reviews.llvm.org/D57731

llvm-svn: 353351
2019-02-07 00:08:14 +00:00
Evgeniy Stepanov
443c034391 [sanitizer] Decorate /proc/self/maps better.
Summary:
Refactor the way /proc/self/maps entries are annotated to support most
(all?) posix platforms, with a special implementation for Android.
Extend the set of decorated Mmap* calls.

Replace shm_open with internal_open("/dev/shm/%s"). Shm_open is
problematic because it calls libc open() which may be intercepted.

Generic implementation has limits (max number of files under /dev/shm is
64K on my machine), which can be conceivably reached when sanitizing
multiple programs at once. Android implemenation is essentially free, and
enabled by default.

The test in sanitizer_common is copied to hwasan and not reused directly
because hwasan fails way too many common tests at the moment.

Reviewers: pcc, vitalybuka

Subscribers: srhines, kubamracek, jfb, llvm-commits, kcc

Differential Revision: https://reviews.llvm.org/D57720

llvm-svn: 353255
2019-02-06 01:14:50 +00:00
Peter Collingbourne
6141b037a9 gn build: Upgrade to NDK r19.
NDK r19 includes a sysroot that can be used directly by the compiler
without creating a standalone toolchain, so we just need a handful
of flags to point Clang there.

Differential Revision: https://reviews.llvm.org/D57733

llvm-svn: 353139
2019-02-05 05:10:19 +00:00
Kostya Kortchinsky
47f0d136f1 [scudo] Initial standalone skeleton check-in
Summary:
This is the initial check-in for the Standalone version of Scudo.

The project is initially going to live in scudo/standalone then will
replace scudo. See http://lists.llvm.org/pipermail/llvm-dev/2019-January/129113.html
for details.

This initial CL is meant to lay out the project structure, of both
code & tests, providing a minimal amount of functionalities, namely
various definitions, some atomic helpers and an intrusive list.
(empty.cc is just here to have a compilation unit, but will go away
in the upcoming CLs).

Initial support is restricted to Linux i386 & x86_64 in make files
and will be extended once things land & work.

We will grow organically from here, adding functionalities in limited
amounts.

Reviewers: morehouse, eugenis, vitalybuka, kcc, mcgrathr, flowerhack

Reviewed By: morehouse, vitalybuka

Subscribers: srhines, mgorny, krytarowski, delcypher, jfb, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D57412

llvm-svn: 353055
2019-02-04 16:25:40 +00:00
Jonathan Metzman
e2469b11a5 [libFuzzer][Windows] Reenable passing tests
Summary:
Enable tests that were previously disabled because they didn't work on
Windows.

Reviewers: morehouse

Reviewed By: morehouse

Subscribers: morehouse

Differential Revision: https://reviews.llvm.org/D57563

llvm-svn: 353000
2019-02-03 16:53:32 +00:00
Reid Kleckner
33706e3cbf [ubsan] Make suppressions.cpp test pass for me on Windows
The test seems to be failing because the module suppression file
contains a colon. I found that it was sufficient to just use the
basename of the suppression file.

While I was here, I noticed that we don't implement IsAbsolutePath for
Windows, so I added it.

llvm-svn: 352921
2019-02-01 21:35:17 +00:00
Kostya Serebryany
ae667c49d2 [libFuzzer] make a test for exploding dfsan labels more agressive
llvm-svn: 352819
2019-01-31 23:43:54 +00:00
Jonathan Metzman
6f94a033a7 [libFuzzer][Windows] Temporarily disable value-profile-cmp2.test on Win
Summary:
Temporarily disable value-profile-cmp2.test on Win.
https://reviews.llvm.org/D57465 causes the test to fail on Win.
However, it seems that the behavior of libFuzzer on Win was broken
before that patch. It crashes in the exit handler when not used with
ASAN. Prior to the patch, the crash handler would run, tricking the
test into thinking libFuzzer on Win had exited properly.

Reviewers: morehouse, vitalybuka

Reviewed By: morehouse

Subscribers: yln

Differential Revision: https://reviews.llvm.org/D57551

llvm-svn: 352815
2019-01-31 23:34:56 +00:00