summaryrefslogtreecommitdiff
path: root/tools/testing/ktest
AgeCommit message (Collapse)Author
2018-04-06Ktest: Add email supportTim Tianyang Chen
Users can define optional variables to get email notifications. Ktest can send emails when the script: * was started * failed with fatal errors and called dodie() * completed all testing Users have to setup the mailer provided in config prior to using this script. Supported mailers: mailx, mail, sendmail mailer specific routines are _sendmail_send(), _mailx_send() Link: http://lkml.kernel.org/r/1522094884-22718-2-git-send-email-tianyang.chen@oracle.com Suggested-by: Dhaval Giani <dhaval.giani@oracle.com> Signed-off-by: Tim Tianyang Chen <tianyang.chen@oracle.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest.pl: Detect if a config-bisect was interruptedSteven Rostedt (VMware)
If a config-bisect was interrupted, then allow the user to continue, or restart a new config-bisect. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest.pl: Make finding config-bisect.pl dynamicSteven Rostedt (VMware)
Just looking for config-bisect.pl in the source tree can be risky, especially, if the source tree being tested doesn't have config-bisect.pl in place. Instead, allow the user to set where to find config-bisect.pl with a new option CONFIG_BISECT_EXEC. If this option is not set, by default, ktest.pl will look for config-bisect.pl in the following locations: `pwd`/config-bisect.pl # where ktest.pl was called from `dirname /path/to/ktest.pl`/config-bisect.pl # where ktest.pl exists ${BUILD_DIR}/tools/testing/ktest/config-bisect.pl # where config-bisect.pl exists in the source tree. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest.pl: Have ktest.pl pass -r to config-bisect.pl to reset bisectSteven Rostedt (VMware)
If config-bisect.pl sees that a config_bisect has already been started, it will ask on the command line if it should bisect or not. This will mess up running config_bisect from ktest.pl. Have ktest.pl pass in '-r' to config-bisect.pl and have config-bisect.pl recognize that to reset without asking. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest.pl: Use diffconfig if available for failed config bisectsSteven Rostedt (VMware)
Check to see if diffconfig is available and use that to diff the configs instead of using 'diff -u', as diffconfig produces much better output of kernel config files. It checks the source directory for the executable. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest.pl: Allow for the config-bisect.pl output to display to consoleSteven Rostedt (VMware)
When commands are run in ktest, they are only displayed in the ktest log file, but that is not sufficient for outputting the display for config bisects. The result of a config bisect is not shown. Add a way to display the output of "run_command" which is the subroutine used by ktest to execute commands. Use this feature to display the output of config-bisect.pl executions to see the progress as well as the result. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest: Use config-bisect.pl in ktest.plScott Wood
Reduce code duplication and take advantage of bisection logic improvements by calling config-bisect.pl. The output of make oldconfig is now copied directly to the desired file, rather than doing assign_configs+save_config, in order to preserve the ordering so that diffing the configs at the end will provide useful output. Link: http://lkml.kernel.org/r/20170717001630.10518-8-swood@redhat.com Signed-off-by: Scott Wood <swood@redhat.com> [ Modified to use with new version of config-bisect.pl ] Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-04-06ktest: Add standalone config-bisect.pl programSteven Rostedt (Red Hat)
Started working on a stand alone program that can do a config bisect. It is based on the config bisect code of ktest.pl. Instead of needing all the infrastructure of ktest.pl, all that is required for config-bisect.pl is two config files. One that works, and one that does not. The goal is to pass in the two files, and it will create a new "good" and a new "bad" config file based on input from the user. After several iterations (calls to this program), it will eventually end with a minimum config value that allows one config to work and the other config to break. The program uses a technique that takes the good config and then makes half of the configs that differ from the bad config just like the bad config. The code will use make oldconfig to make sure the configs that are set are not all converted back due to incorrect dependencies on other configs set in the bad config but not in the new test config. This is still a work in progress, but as it was written while I was working at Red Hat, I want this code to be submitted as such. Signed-off-by: Steven Rostedt (Red Hat) <rostedt@goodmis.org>
2018-03-26ktest: remove obsolete architecturesArnd Bergmann
A number of architectures are being removed from the kernel, so we no longer need to test them. Acked-by: Steven Rostedt (VMware) <rostedt@goodmis.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2018-03-21ktest: Set do_not_reboot=y for CONFIG_BISECT_TYPE=buildScott Wood
Currently setting do_not_reboot is triggered by simple builds and bisect builds, but not config bisect builds. Link: http://lkml.kernel.org/r/20170717001630.10518-3-swood@redhat.com Signed-off-by: Scott Wood <swood@redhat.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-21ktest: Set buildonly=1 for CONFIG_BISECT_TYPE=buildScott Wood
Rather than adding a third copy of the same logic, rework it to cover all three buildonly cases at once. In the future, please consider using the same variable to perform the same function regardless of context... Link: http://lkml.kernel.org/r/20170717001630.10518-2-swood@redhat.com Signed-off-by: Scott Wood <swood@redhat.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-21ktest: Comment about other names than just ktest.confSteven Rostedt (VMware)
ktest.pl will read any file as long as its name is specified as the first argument on the command line. Comment this fact in sample.conf. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-21ktest: Clarify config file usageScott Wood
Simply telling a new user to edit "the config file" without giving any hints on where that file should go, what it should be named, or where a template can be found, is not particularly helpful. Link: http://lkml.kernel.org/r/20170717001630.10518-1-swood@redhat.com Signed-off-by: Scott Wood <swood@redhat.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-21ktest: Add CONNECT_TIMEOUT to change the connection timeout timeSteven Rostedt (VMware)
Before ktest issues a reboot, it will try to connect to the target machine to make sure that it is still alive. If the target does not respond within 5 seconds, it will power cycle the box instead of issuing a reboot. Five seconds may be too short, and ktest may unnecessarially power cycle the box. I have found 25 seconds seems to be a better timeout for this purpose. But even 25 may be too arbitrary. Add a CONNECT_TIMEOUT option to let the user determine the timeout time before rebooting. By default, it has been raised to 25 seconds. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-21ktest: Wait for console process to exitSteven Rostedt (VMware)
To clean up the console processes that are forked to monitor the console, there needs to be a waitpid(). Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2018-03-16arch: remove tile portArnd Bergmann
The Tile architecture port was added by Chris Metcalf in 2010, and maintained until early 2018 when he orphaned it due to his departure from Mellanox, and nobody else stepped up to maintain it. The product line is still around in the form of the BlueField SoC, but no longer uses the Tile architecture. There are also still products for sale with Tile-GX SoCs, notably the Mikrotik CCR router family. The products all use old (linux-3.3) kernels with lots of patches and won't be upgraded by their manufacturers. There have been efforts to port both OpenWRT and Debian to these, but both projects have stalled and are very unlikely to be continued in the future. Given that we are reasonably sure that nobody is still using the port with an upstream kernel any more, it seems better to remove it now while the port is in a good shape than to let it bitrot for a few years first. Cc: Chris Metcalf <chris.d.metcalf@gmail.com> Cc: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de> Link: http://www.mellanox.com/page/npu_multicore_overview Link: https://jenkins.debian.net/view/rebootstrap/job/rebootstrap_tilegx_gcc7/ Signed-off-by: Arnd Bergmann <arnd@arndb.de>
2017-11-02License cleanup: add SPDX GPL-2.0 license identifier to files with no licenseGreg Kroah-Hartman
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-03-08Merge tag 'ktest-v4.11-rc1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest Pull ktest fixes from Steven Rostedt: "Greg Kroah-Hartman reported to me that the ktest of v4.11-rc1 locked up in an infinite loop while doing the make mrproper. Looking into the cause I noticed that a recent update to the function run_command (used for running all shell commands, including "make mrproper") changed the internal loop to use the function wait_for_input. The wait_for_input function uses select to look at two file descriptors. One is the file descriptor of the command it is running, the other is STDIN. The STDIN check was not checking the return status of the sysread call, and was also just writing a lot of data into syswrite without regard to the size of the data read. Changing the code to check the return status of sysread, and also to still process the passed in descriptor data without looping back to the select fixed Greg's problem. While looking at this code I also realized that the loop did not honor the timeout if STDIN always had input (or for some reason return error). this could prevent wait_for_input to timeout on the file descriptor it is suppose to be waiting for. That is fixed too" * tag 'ktest-v4.11-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Make sure wait_for_input does honor the timeout ktest: Fix while loop in wait_for_input
2017-03-08ktest: Make sure wait_for_input does honor the timeoutSteven Rostedt (VMware)
The function wait_for_input takes in a timeout, and even has a default timeout. But if for some reason the STDIN descriptor keeps sending in data, the function will never time out. The timout is to wait for the data from the passed in file descriptor, not for STDIN. Adding a test in the case where there's no data from the passed in file descriptor that checks to see if the timeout passed, will ensure that it will timeout properly even if there's input in STDIN. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-03-08ktest: Fix while loop in wait_for_inputSteven Rostedt (VMware)
The run_command function was changed to use the wait_for_input function to allow having a timeout if the command to run takes too much time. There was a bug in the wait_for_input where it could end up going into an infinite loop. There's two issues here. One is that the return value of the sysread wasn't used for the write (to write a proper size), and that it should continue processing the passed in file descriptor too even if there was input. There was no check for error, if for some reason STDIN returned an error, the function would go into an infinite loop and never exit. Reported-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Tested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Fixes: 6e98d1b4415f ("ktest: Add timeout to ssh command") Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-02-27Merge tag 'ktest-v4.11' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest Pull ktest updates from Steven Rostedt: "These are various fixes that I have made and never got around to pushing. I've been asked to get the upstream repo back up-to-date" * tag 'ktest-v4.11' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Add variable run_command_status to save status of commands executed ktest.pl: Powercycle the box on reboot if no connection can be made ktest: Add timeout to ssh command ktest: Fix child exit code processing ktest: Have POST_TEST run after the test has totally completed
2017-02-07ktest: Add variable run_command_status to save status of commands executedSteven Rostedt (VMware)
Create a variable called run_command_status that saves the status of the executed commands and can be used by other functions later to test for status. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-02-07ktest.pl: Powercycle the box on reboot if no connection can be madeSteven Rostedt (VMware)
When performing a reboot of the test box, try to ssh to it. If it can't connect for 5 seconds, then powercycle the box. This is useful because the reboot is done via ssh, and if you can't ssh to the box because it is hung, the reboot fails to reboot. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-02-07ktest: Add timeout to ssh commandSteven Rostedt (VMware)
Add a timeout to performing an ssh command. This will let testing if a machine is alive or not, or if something else may be amiss. A timeout can be passed to ssh, where ssh will fail if it does not complete within the given timeout. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-02-07ktest: Fix child exit code processingSteven Rostedt (VMware)
The child_exit errno needs to be shifted by 8 bits to compare against the return values for the bisect variables. Fixes: c5dacb88f0a64 ("ktest: Allow overriding bisect test results") Cc: stable@vger.kernel.org Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-02-07ktest: Have POST_TEST run after the test has totally completedSteven Rostedt (VMware)
The POST_TEST config is to be executed after a test has fully compeleted, whether the test passed or failed. It currently is executed at the moment that the test has been decided if it failed or not. As the test does other clean ups, it isn't truly finished. Move the POST_TEST execution to after all the test cleanups have been done. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2016-12-14ktest.pl: fix englishPavel Machek
Ajdust spelling to more common "mandatory". Variant "mandidory" is certainly wrong. Link: http://lkml.kernel.org/r/20161011073003.GA19476@amd Signed-off-by: Pavel Machek <pavel@ucw.cz> Acked-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-02-03ktest: Place quotes around item variableSteven Rostedt (Red Hat)
Seems that some of the new console logic causes doprint to possibly get evaluated. When printing a commit message that contains parenthesis, it fails with a shell parsing error. This gets fixed when we add quotes around the $item variable, and prevent it from being evaluated by any shell commands. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-02-02ktest: Cleanup terminal on dodie() failureJosh Poimboeuf
If dodie() is called with the console open, restore the terminal's original settings before dying. Link: http://lkml.kernel.org/r/20150130025453.GB20952@treble.redhat.com Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-02-02ktest: Print build,install,boot,test times at success and failureSteven Rostedt (Red Hat)
Since both success and failure may shortcut and exit ktest, it is better to print the status times there too. Once times are printed, the values for the times are reset, so they will not print more than once. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-02-02ktest: Enable user input to the consoleJosh Poimboeuf
Allow the user to send input to the console by putting the terminal in cbreak mode (to allow reading stdin one character at a time) and copying all stdin data to the console's pty. Link: http://lkml.kernel.org/r/bb1bbe7d202c95a3ce7894cfffdd8c725875978e.1422473610.git.jpoimboe@redhat.com Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-02-02ktest: Give console process a dedicated ttyJosh Poimboeuf
Create a pseudoterminal (pty pair) to give the console a dedicated tty so it doesn't mess with ktest's terminal settings. Link: http://lkml.kernel.org/r/37b0127f9efad09ff4fc994334db998141e4f6ca.1422473610.git.jpoimboe@redhat.com Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-01-29ktest: Rename start_monitor_and_boot to start_monitor_and_installSteven Rostedt (Red Hat)
The function start_monitor_and_boot is a misnomer. It use to, but now it starts the monitor and installs. It does not boot. Rename it before I get confused by it again. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-01-29ktest: Show times for build, install, boot and testSteven Rostedt (Red Hat)
Seeing the times for how long a build, install, reboot and the test takes is helpful for analyzing the test process. Seeing how different changes affect the timings. Show the build, install, boot and test times when at the end of the test, or between each interval for tests that do those mulitple times (like bisect and patchcheck). Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-01-27ktest: Restore tty settings after closing consoleJosh Poimboeuf
When ktest runs the console program as a child process, the parent and child share the same tty for stdin and stderr. This is problematic when using a libvirt target. The "virsh console" program makes a lot of changes to the tty settings, making ktest's output hard to read (carriage returns don't work). After ktest exits, the terminal is unusable (CRs broken, stdin isn't echoed). I think the best way to fix this issue would be to create a pseudoterminal (pty pair) so the child process would have a dedicated tty, and then use pipes to connect the two ttys. I'm not sure if that's overkill, but it's far beyond my current Perl abilities. This patch is a much easier way to (partially) fix this issue. It saves the tty settings before opening the console and restores them after closing it. There are still a few places where ktest prints mangled output while the console is open, but the output is much more legible overall, and the terminal works just fine after ktest exits. Link: http://lkml.kernel.org/r/1bb89abc0025cf1d6da657c7ba58bbeb4381a515.1422382008.git.jpoimboe@redhat.com Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2015-01-27ktest: Add timings for commandsSteven Rostedt (Red Hat)
I find that I usually like to see how long a make or other command takes, and adding a start and end time and reporting how long each command runs (in seconds) is helpful. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-11-23ktest: Add back "tail -1" to kernelrelease makeSteven Rostedt (Red Hat)
Commit 52d21580b362 "ktest: Use make -s kernelrelease" fixed commit 7ff525712acf "kbuild: fake the "Entering directory ..." message more simply" as that commit added output after the make kernelrelease. But there's still some build scripts that are used by ktest that has output before the make is executed, and requires that only the last line is printed. Cc: Michal Marek <mmarek@suse.cz> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-11-21ktest: Add name to running titleSteven Rostedt (Red Hat)
Instead of just showing the test type of test in the start of the test, like this: RUNNING TEST 1 of 26 with option build defconfig Add the name (if it is defined) as well, like this: RUNNING TEST 1 of 26 (arm64 aarch64-linux) with option build defconfig Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-11-21ktest: Allow tests to undefine default optionsSteven Rostedt (Red Hat)
Tests can set options that override the default ones. But if a test tries to undefine a default option, it is simply ignored and the default option stays as is. For example, if you want to have a test that defines no MIN_CONFIG then the test should be able to do that with: TEST_START MIN_CONFIG = Which should make MIN_CONFIG not defined for that test. But the way the code currently works, undefined options in tests are dropped. This is because the NULL options are evaluated during the reading of the config file and since one can disable default options in the default section with this method, it is evaluated there (the option turns to a undef). But undef options in the test section mean to use the default option. To fix this, keep the empty string in the option during the reading of the config file, and then evaluate it when running the test. This will allow tests to null out default options. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-11-21ktest: Fix make_min_config to handle new assign_configs callSteven Rostedt (Red Hat)
Commit 6071c22e1755 "ktest: Rewrite the config-bisect to actually work" fixed the config-bisect to work nicely but in doing so it broke make_min_config by changing the way assign_configs works. The assign_configs function now adds the config to the hash even if it is disabled, but changes the hash value to be that of the line "# CONFIG_FOO is not set". Unfortunately, the make_min_config test only checks to see if the config is removed. It now needs to check if the config is in the hash and not set to be disabled. Cc: stable@vger.kernel.org # 3.17+ Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-11-21ktest: Use make -s kernelreleaseMichal Marek
The previous tail -1 broke with commit 7ff525712acf ("kbuild: fake the "Entering directory ..." message more simply") Link: http://lkml.kernel.org/r/20141022194408.GA20989@pobox.suse.cz Reported-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: Michal Marek <mmarek@suse.cz> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-10-07ktest: Don't bother with bisect good or bad on replaySteven Rostedt (Red Hat)
If git bisect reply is being used in the bisect tests, don't bother doing the git bisect good or git bisect bad calls. The git bisect reply will override them anyway, and that's called immediately after the other two. Going the git bisect (good|bad) is just a waste of time. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-10-07ktest: Fix check for new kernel success on rebooting to good kernelSteven Rostedt (Red Hat)
The reboot function when rebooting back to a good kernel has a check to make sure that a new kernel was indeed booted. But that check uses a timeout value, which when calling the monitor will still return success if the timeout is hit (no bug was found). It should return an error to let the reboot code know that a new kernel was not reached. Only the reboot code checks the return value of the monitor. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-09-19ktest: add ability to skip during BISECT_MANUALChris J Arges
When doing a manual bisect, a build can fail or a test can be inconclusive. In these cases it would be helpful to be able to skip the test entirely. Link: http://lkml.kernel.org/r/1409164021-2136-1-git-send-email-chris.j.arges@canonical.com Reviewed-by: Satoru Takeuchi <satoru.takeuchi@gmail.com> Signed-off-by: Chris J Arges <chris.j.arges@canonical.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-09-19ktest: Add PATCHCHECK_CHERRYSteven Rostedt (Red Hat)
Add a way to run a patchcheck test on the commits that are in one branch but not in another. This uses git cherry to find a list of commits to test each one with. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-04-23ktest: Update documentation on config_bisectSteven Rostedt (Red Hat)
With the more robust config_bisect, the documentation is out of date and needs to be updated. The new rewrite allows for finding missing configs and such, and is much more robust to use. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-04-23ktest: Add the config bisect manual backSteven Rostedt (Red Hat)
After the rewrite of the config bisect, the bisect manual was removed. Add it back. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-04-23ktest: Remove unused functionsSteven Rostedt (Red Hat)
After the rewrite of the config bisect, there were several unused functions that can be removed. One of the unused functions printed out the failed config nicer than what the rewrite did, so I kept that and used it to output the bad config. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-04-23ktest: Put back in the CONFIG_BISECT_CHECKSteven Rostedt (Red Hat)
The new rewrite left out the CONFIG_BISECT_CHECK, which allows the user to test that their "bad" config still is bad and their "good" config still is good. This is especially important as the configs are passed through a "make oldconfig" to update them with the lastest kernel. Things could change that causes a bad config to work, or a good config to break. The check is done after the configs have run through the oldconfig processing. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2014-04-23ktest: Rewrite the config-bisect to actually workSteven Rostedt (Red Hat)
I never liked the way config-bisect worked. I would assume the bad config had some config that broke the system. But it would not work if the bad config just happened to be missing something that the good config had. I rewrote the config-bisect to do this properly. It does a diff of the two configs, and sets half of the configs that are in one and not the other. The way it works is that when it "sets", it really just makes one copy what the other has. That is, a "set" can be setting a: # CONFIG_FOO is not set Basically, it looks at the differences between the two files and makes them similar until it comes down to one config that makes it work or not work depending on if it is set or not. Note, if more than one config change makes the bad config not work, it will only find one of them. But this is true with all bisect logic. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>