Commit Graph

178 Commits

Author SHA1 Message Date
Nirbheek Chauhan 88aafd363e Normalize the path of a configured file to avoid dupes (#640) 2016-07-19 20:10:57 +03:00
Nirbheek Chauhan cc4826b85f run_tests.py: Make 'no-installed-files' stricter
When the file 'no-installed-files' is installed, require that the test
not install any other files. A test for this is pending.
2016-07-13 01:17:46 +05:30
Nirbheek Chauhan 45c8557dc6 Fix tests for the new library/executable naming scheme
Also add new tests for the platform-specific and compiler-specific
versioning scheme.

A rough summary is:

1. A bug in how run_tests.py:validate_install checked for files has been
   fixed. Earlier it wasn't checking the install directory properly.
2. Shared libraries are no longer installed in common tests, and the
   library name/path testing is now done in platform-specific tests.
3. Executables are now always called something?exe in the
   installed_files.txt file, and the suffix automatically corrected
   depending on the platform.
4. If a test installs a file called 'no-installed-files', the installed
   files for that test are not validated. This is required to implement
   compiler-specific tests for library names/paths such as MSVC vs MinGW
5. The platform-specific file renaming in run_tests.py has been mostly
   removed since it is broken for shared libraries and isn't needed for
   static libraries.
6. run_tests.py now reports all missing and extra files. The logic for
   finding these has been reworked.
2016-07-01 20:50:47 +05:30
Nirbheek Chauhan d61656d43c Fix typo argument order to TestRunner (#615) 2016-06-24 13:44:07 +03:00
Jussi Pakkanen 436eab9b85 Print full mesonlog on failed tests when run under CI. 2016-06-23 00:03:59 +03:00
Jussi Pakkanen 15ab984d8e Finish appveyor integration by moving static library in a subdir to avoid clashes. 2016-06-13 22:50:47 +03:00
Jussi Pakkanen 1e9e91f25a Print full log when failures happen under Appveyor. 2016-06-13 22:27:12 +03:00
Jussi Pakkanen 0bc0056064 Merge pull request #577 from nioncode/vs2015
Implement vs2015 backend
2016-06-05 13:00:57 +03:00
Nicolas Schneider 3bedca0257 add vs2015 backend 2016-05-30 21:49:35 +02:00
Jussi Pakkanen 37c5d45d07 Print all of the things when CI fails. 2016-05-30 22:29:27 +03:00
Nicolas Schneider 4a102fd445 cancel all pending test cases when interrupted by signals 2016-05-25 23:38:15 +02:00
Nicolas Schneider 1d2b4ed8e9 simplify unit test output (#506)
Print status on single line

only print a single line for succeeded tests and two lines for failed

tests. This makes it easier to scan the output for failed tests.
2016-04-07 20:26:53 +03:00
Nicolas Schneider 435700aeb0 add exponential backoff for deleting temp directories 2016-04-06 21:39:51 +02:00
Nicolas Schneider 5decddf09f fix off by one, since range() does not include the end of the range 2016-04-06 21:35:51 +02:00
Nicolas Schneider 32e0973ef1 fix randomly failing test execution on Windows
shutil.rmtree, which is used by tempfile.TemporaryDirectory, randomly fails
on Windows, because the directory is not empty although it should be,
because all files were deleted by shutil.rmtree internals before trying to
remove the directory.
A simple retry approach fixes the issue.
2016-04-06 12:38:39 +02:00
Nicolas Schneider 42e6b78351 do not print anything during tests
Otherwise, output from parallel tests might interleave. Let the main loop
handle printing of additional info.
2016-04-05 23:05:09 +02:00
Nicolas Schneider ff9d879eea properly fix Windows parallel tests by not using global variables
The _run_test method uses several global variables (unity_flags,
backend_flags, compile_commands, install_commands) which are
not set when the method is run by the executor (at least on Windows).
To resolve this, pass the variables as method parameters.
2016-04-05 23:02:15 +02:00
Nicolas Schneider eb69b268d4 Revert "Fix Windows. Again."
This reverts commit e522a9f268.
2016-04-05 22:03:57 +02:00
Jussi Pakkanen e522a9f268 Fix Windows. Again. 2016-04-01 23:27:50 +03:00
Jussi Pakkanen a7e9301215 Output is all pretty again. 2016-04-01 23:06:58 +03:00
Jussi Pakkanen 81e37e1220 Run tests in parallel. 2016-04-01 23:06:19 +03:00
Jussi Pakkanen 9587715585 Fix Windows. 2016-04-01 22:51:36 +03:00
Jussi Pakkanen ea092fefc2 Use individual tempdirs for building and installing in unit tests. 2016-04-01 20:20:48 +03:00
Jussi Pakkanen 5b3c9e262b Check option files too. 2016-03-20 22:52:46 +02:00
Nicolas Schneider 5482a23936 enhance test framework to read meson arguments from a file per test
A 'test_args.txt' file in the same directory as the test case will be
parsed by the test framework and the content will be passed as arguments
to meson during configuration. The arguments are put before any
'extra_args' to make them overwritable from the command line.
2016-03-12 17:41:42 +01:00
Jussi Pakkanen 455e55f8e5 Back to .py suffix in files. Closes #394. 2016-03-02 21:49:29 +02:00
Jussi Pakkanen ea60a22cd5 Merge Python 3 module support. 2016-02-26 21:21:53 +02:00
Nicolas Schneider 681a14a695 use sys.stdout.encoding when parsing subprocess output 2016-02-22 22:58:17 +01:00
Jussi Pakkanen a8120eb519 Added plain Python sample project. 2016-02-21 14:52:36 +02:00
Jussi Pakkanen 23b98cd6e6 Renamed meson package to mesonbuild so that we can have a script named meson in the same toplevel dir. 2016-01-16 17:35:29 +02:00
Jussi Pakkanen 4c31e7774d Finalize moduleification so that full test suite runs. 2016-01-16 00:04:57 +02:00
Jussi Pakkanen a5508d3fd3 Can run most of test suite (with hacks). 2016-01-15 21:43:15 +02:00
Jussi Pakkanen 8b1039fa30 Organise files into a module structure. 2016-01-15 21:22:09 +02:00
Jussi Pakkanen c86ee8158a Make vs dependency detector work on non-english locales. 2015-12-18 17:30:30 +02:00
Jussi Pakkanen d92504797f Can build simple Swift executables. 2015-12-07 21:12:23 +02:00
Jussi Pakkanen 26873801bc Can define benchmarks and run them. 2015-11-25 23:29:06 +02:00
Jussi Pakkanen 067935ce76 Added conf option for build tree layout. 2015-09-17 19:21:26 +03:00
Jussi Pakkanen 9d559b2b39 More robust windows detection and more logs. 2015-08-25 23:09:54 +03:00
jpakkane f15eb466fb Merge pull request #162 from mesonbuild/lint_errors
Fix some bugs when linting code
2015-06-19 13:10:07 +03:00
Igor Gnatenko 4dc8dbb7c1 run_tests: Bad first argument 'Exception' given to super()
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-06-19 10:37:54 +03:00
Igor Gnatenko 816013ef0d run_tests: fix bad-continuation
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-06-19 10:17:25 +03:00
Jussi Pakkanen cf0bd77ddd Do not invoke linker via cmd /c on Windows. 2015-05-17 15:23:33 +03:00
Jussi Pakkanen ff7c8b1647 Run unit tests in-process too for extra speed. 2015-04-23 17:34:31 +03:00
Jussi Pakkanen 144b2314ce Run Meson test invocations in-process because spawning a new Python process for each is too slow. 2015-04-23 17:08:08 +03:00
Jussi Pakkanen 324cca808d Print test time breakdown. 2015-04-23 12:51:41 +03:00
Jussi Pakkanen edda2f6888 Store time taken to configure, build and test individual cases. 2015-04-23 12:40:51 +03:00
Jussi Pakkanen a92fcb711b Fix windows tests when using newer MinGW. 2015-04-20 22:04:58 +03:00
Jussi Pakkanen f19fd50702 Encoding of result file is always utf-8. 2015-04-13 23:45:10 +03:00
Igor Gnatenko 2c8c6fbf20 tests: add all platform tests and skip needed tests
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-03-25 23:15:23 +03:00
Igor Gnatenko e0ba9ba2b9 tests: add skipped tests
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-03-25 22:44:01 +03:00
Igor Gnatenko dccb6a170e tests: add time property
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-03-25 20:39:11 +03:00
Jussi Pakkanen 9287d7e2c5 Write out padding zeroes to numbers because Jenkins is too stupid to do logical order sorting. 2015-03-25 18:38:24 +02:00
Igor Gnatenko 7249ff9b00 tests: use name as class name
Signed-off-by: Igor Gnatenko <i.gnatenko.brain@gmail.com>
2015-03-25 19:22:13 +03:00
Jussi Pakkanen 99a955491d Minor fixes. 2015-03-25 18:11:34 +02:00
Jussi Pakkanen 3a1a3404ba Fix empty java case. 2015-03-25 17:52:12 +02:00
Jussi Pakkanen fd30a81fa9 Made run_tests.py output test results in junit xml to allow better integration with CI tools. 2015-03-25 17:32:58 +02:00
Jussi Pakkanen bb28b6b51c Refactor unit test running. 2015-03-25 17:04:57 +02:00
Jussi Pakkanen c3645e2b45 Use argparse instead of optparse in run_tests. 2015-03-02 16:19:49 +02:00
Jussi Pakkanen 436a81f920 Can specify backend to run_tests.py. 2015-03-02 03:31:41 +02:00
Jussi Pakkanen 1ab5b850ef Use lib as library path in unit tests. 2015-02-16 01:24:58 +02:00
Jussi Pakkanen 73b8ff5341 Rework installation logic so we can go outside the install prefix. 2015-02-04 22:00:06 +02:00
Jussi Pakkanen 08ef881c86 Extracted compiler definitions to their own file. 2014-12-03 22:37:06 +02:00
Jussi Pakkanen 525f0ae6d2 Harmonised copyright declarations. 2014-11-16 20:19:12 +02:00
Robin McCorkell 85586ce1ba Signal handling for run_tests.py
SIGINT and SIGTERM are handled, causing further test execution to stop and
the results to be printed immediately. Also cleans up prebuilt objects
2014-11-16 17:11:00 +00:00
Jussi Pakkanen c4860ff60f Updated copyright years. 2014-08-11 00:20:17 +03:00
Jussi Pakkanen 100c20e568 Scan Fortran dependencies at the beginning for all sources. 2014-08-03 21:29:13 +03:00
Jussi Pakkanen 9abc5e0570 Vestigial removal. 2014-07-20 14:24:06 +03:00
Jussi Pakkanen b3ca519fea Message clarification. 2014-07-20 13:50:25 +03:00
Jussi Pakkanen 09c24654a6 Run C# tests during suite. 2014-07-18 23:45:23 +03:00
Jussi Pakkanen 5df4e62dfd More verbosity. 2014-07-14 18:15:54 +03:00
Jussi Pakkanen 4a23c01992 Make it possible to print all test output so it shows up in e.g. buildbot logs. 2014-07-08 17:39:54 +03:00
Jussi Pakkanen b7bdf5b7bd Prebuild object file with MSVC. 2014-06-24 19:20:44 +03:00
Jussi Pakkanen 5df1818e2b MinGW fix. 2014-06-24 19:03:15 +03:00
Jussi Pakkanen 85972c848d Generate prebuilt object on demand so we don't need to ship object files in the source tarball. 2014-06-22 20:36:19 +03:00
Jussi Pakkanen e4b17ff3e8 Add Rust to main test suite. 2014-06-18 23:50:33 +03:00
Jussi Pakkanen 576b0f26dd Print test skip. 2014-05-10 19:49:00 +03:00
Jussi Pakkanen 752fa1e4af Run Vala tests as part of the test suite. 2014-05-10 02:33:44 +03:00
Jussi Pakkanen c7e7d1a4c3 Can execute found programs with run_command. 2014-04-20 13:58:20 +03:00
Jussi Pakkanen c21637a01d Fixed unity builds for Qt5. 2014-04-16 22:38:29 +03:00
Jussi Pakkanen 4c02a35fe0 Updated msvc compile and link flag functions. 2014-04-16 21:38:49 +03:00
Jussi Pakkanen 668d679c01 Some windows cleanups. 2014-04-16 21:30:14 +03:00
Jussi Pakkanen 6024f277dc Do not run Java tests on OSX because starting from Mavericks it has a fake javac program that just annoys you with a popup. 2014-04-02 21:23:31 +03:00
Jussi Pakkanen 6b56bfd10e Run test suite with xcodebuild on OSX. 2014-03-31 02:17:57 +03:00
Jussi Pakkanen f692555a27 Now can use run_tests.py with MSBuild. 2014-03-19 20:53:35 +02:00
Jussi Pakkanen 889e4b03c0 Fix test runner on Windows. 2014-03-16 11:57:07 +02:00
Jussi Pakkanen a2cb207762 Run Java tests as part of test suite. 2014-03-11 22:41:27 +02:00
Jussi Pakkanen 0a4818164d Run object tests on i686, too. 2014-03-08 23:19:32 +02:00
Jussi Pakkanen 71112e1cdb Cleaner test run output. 2014-03-08 00:31:45 +02:00
Jussi Pakkanen 337b14e602 Some test tuning. 2013-11-05 00:29:39 +02:00
Jussi Pakkanen 18c92dc1c5 Indent fix. 2013-09-29 18:50:02 +03:00
Jussi Pakkanen 72577a3ee3 Do not test install on Windows. 2013-09-29 18:48:33 +03:00
Jussi Pakkanen 260d6eaa6e Mangle install file names so test suite passes on OSX. 2013-09-15 22:20:41 +03:00
Jussi Pakkanen 26837fda48 Added installed_files.txt files to all places that needed them. 2013-09-13 21:06:09 +03:00
Jussi Pakkanen 2aed361390 Check that files that should be installed are. 2013-09-13 20:57:05 +03:00
Jussi Pakkanen 6536354cc3 Updated symbolextractor to work with cross builds. 2013-08-24 00:30:13 +03:00
Jussi Pakkanen d02e4ce1c9 Created new suite of tests that should fail when executed. 2013-08-10 23:02:51 +03:00
Jussi Pakkanen 65be7a9ab7 A few Fedora fixes. 2013-07-04 18:02:44 +03:00
Jussi Pakkanen 30dbccd346 Line numbers start from 1. 2013-04-19 19:03:29 +03:00
Jussi Pakkanen a1d367ba8b Guard against Windows line endings on Windows too. 2013-04-19 19:01:48 +03:00
Jussi Pakkanen a7b7e4b235 Added a Windows test. 2013-04-19 18:24:48 +03:00