summaryrefslogtreecommitdiff
path: root/tools/testing
AgeCommit message (Collapse)Author
2012-07-19ktest: Add CONFIG_BISECT_CHECK optionSteven Rostedt
The config-bisect can take a bad config and bisect it down to find out what config actually breaks the config. But as all tests will apply a minconfig (defined by a user) to apply before booting, it is possible that the minconfig could actually make the bad config work (minconfigs can disable configs). The end result is that the config bisect test will not find a config that breaks. This can be rather frustrating to the user. The CONFIG_BISECT_CHECK option, when set to 1, will make sure that the bad config (with the minconfig applied) still fails before trying to bisect. And yes, I did get burned by this. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-07-19ktest: Add PRE_INSTALL optionSteven Rostedt
Add the PRE_INSTALL option that will allow a user to specify a shell command to be executed before the install operation executes. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-07-19ktest: Add PRE/POST_KTEST and TEST optionsSteven Rostedt
In order to let the user add commands before and after ktest runs, the PRE_KTEST and POST_KTEST options are defined. They hold shell commands that will execute befor ktest runs its first test, as well as when it completed its last test. The PRE_TEST and POST_TEST will be run befor and after (respectively) for a given test. They can either be global (done for all tests) or defined by a single test. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-07-19ktest: Remove commented exitSteven Rostedt
A debug 'exit' was left in ktest.pl. Remove it. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-31syscalls, x86: add __NR_kcmp syscallCyrill Gorcunov
While doing the checkpoint-restore in the user space one need to determine whether various kernel objects (like mm_struct-s of file_struct-s) are shared between tasks and restore this state. The 2nd step can be solved by using appropriate CLONE_ flags and the unshare syscall, while there's currently no ways for solving the 1st one. One of the ways for checking whether two tasks share e.g. mm_struct is to provide some mm_struct ID of a task to its proc file, but showing such info considered to be not that good for security reasons. Thus after some debates we end up in conclusion that using that named 'comparison' syscall might be the best candidate. So here is it -- __NR_kcmp. It takes up to 5 arguments - the pids of the two tasks (which characteristics should be compared), the comparison type and (in case of comparison of files) two file descriptors. Lookups for pids are done in the caller's PID namespace only. At moment only x86 is supported and tested. [akpm@linux-foundation.org: fix up selftests, warnings] [akpm@linux-foundation.org: include errno.h] [akpm@linux-foundation.org: tweak comment text] Signed-off-by: Cyrill Gorcunov <gorcunov@openvz.org> Acked-by: "Eric W. Biederman" <ebiederm@xmission.com> Cc: Pavel Emelyanov <xemul@parallels.com> Cc: Andrey Vagin <avagin@openvz.org> Cc: KOSAKI Motohiro <kosaki.motohiro@gmail.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Glauber Costa <glommer@parallels.com> Cc: Andi Kleen <andi@firstfloor.org> Cc: Tejun Heo <tj@kernel.org> Cc: Matt Helsley <matthltc@us.ibm.com> Cc: Pekka Enberg <penberg@kernel.org> Cc: Eric Dumazet <eric.dumazet@gmail.com> Cc: Vasiliy Kulikov <segoon@openwall.com> Cc: Alexey Dobriyan <adobriyan@gmail.com> Cc: Valdis.Kletnieks@vt.edu Cc: Michal Marek <mmarek@suse.cz> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-05-31tools/selftests: add mq_perf_testsDoug Ledford
Add the mq_perf_tests tool I used when creating my mq performance patch. Also add a local .gitignore to keep the binaries from showing up in git status output. [akpm@linux-foundation.org: checkpatch fixes] Signed-off-by: Doug Ledford <dledford@redhat.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Manfred Spraul <manfred@colorfullife.com> Cc: Frederic Weisbecker <fweisbec@gmail.com> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-05-31selftests: add mq_open_testsDoug Ledford
Add a directory to house POSIX message queue subsystem specific tests. Add first test which checks the operation of mq_open() under various corner conditions. Signed-off-by: Doug Ledford <dledford@redhat.com> Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: Doug Ledford <dledford@redhat.com> Cc: Joe Korty <joe.korty@ccur.com> Cc: Amerigo Wang <amwang@redhat.com> Cc: Serge E. Hallyn <serue@us.ibm.com> Cc: Jiri Slaby <jslaby@suse.cz> Cc: Manfred Spraul <manfred@colorfullife.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-05-23ktest: Change singular "paranthesis" to plural "parentheses"Jesper Juhl
Acked-by: Randy Dunlap <rdunlap@xenotime.net> Signed-off-by: Jesper Juhl <jj@chaosbits.net> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add README to explain what is in the examples directorySteven Rostedt
Add a README that explains what the different example configs in the ktest example directory are about. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add the snowball.conf example configSteven Rostedt
I used the snowball.conf in a live demo that demonstrated how to use ktest.pl with a snowball ARM board. I've been asked to included that config in the ktest repository. Here it is. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add an example config that does cross compiling of several archsSteven Rostedt
Add the config that I use to test several archs. I downloaded several cross compilers from: http://kernel.org/pub/tools/crosstool/files/bin/x86_64/ and this config is an example to crosscompile several archs to make sure that your changes do not break archs that you are not working on. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add kvm.conf example configSteven Rostedt
Add an example config that explains how to use ktest with a virtual guest as the target. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add useful example configsSteven Rostedt
I've been asked several times to provide more useful example configs for ktest.pl, as the sample.conf is too complex (because it explains all configs). This adds configs broken up by use case, and these configs are based on actual configs that I use on a daily basis. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-22ktest: Add USE_OUTPUT_MIN_CONFIG to avoid prompt on make_min_configSteven Rostedt
If the file that OUTPUT_MIN_CONFIG exists then ktest.pl will prompt the user and ask them if the OUTPUT_MIN_CONFIG should be used as the starting point for make_min_config instead of MIN_CONFIG. This is usually the case, and to allow the user to do so, which is helpful if the user is creating different min configs based on tests, and they know one is a superset of another test, they can set USE_OUTPUT_MIN_CONFIG to one, which will prevent kest.pl from prompting to use the OUTPUT_MIN_CONFIG and it will just use it. If USE_OUTPUT_MIN_CONIFG is set to zero, then ktest.pl will continue to use MIN_CONFIG instead. The default is that USE_OUTPUT_MIN_CONFIG is undefined. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-21ktest: Add MIN_CONFIG_TYPE to allow making a minum .config that has networkSteven Rostedt
Add a MIN_CONFIG_TYPE that can be set to 'test' or 'boot'. The default is 'boot' which is what make_min_config has done previously: makes a config file that is the minimum needed to boot the target. But when MIN_CONFIG_TYPE is set to 'test', not only must the target boot, but it must also successfully run the TEST. This allows the creation of a config file that is the minimum to boot and also perform ssh to the target, or anything else a developer wants. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-18ktest: Fix kernelrevision with POST_BUILDSteven Rostedt
The PRE_BUILD and POST_BUILD options of ktest are added to allow the user to add temporary patch to the system and remove it on builds. This is sometimes use to take a change from another git branch and add it to a series without the fix so that this series can be tested, when an unrelated bug exists in the series. The problem comes when a tagged commit is being used. For example, if v3.2 is being tested, and we add a patch to it, the kernelrelease for that commit will be 3.2.0+, but without the patch the version will be 3.2.0. This can cause problems when the kernelrelease is determined for creating the /lib/modules directory. The kernel booting has the '+' but the module directory will not, and the modules will be missing for that boot, and may not allow the kernel to succeed. The fix is to put the creation of the kernelrelease in the POST_BUILD logic, before it applies the POST_BUILD operation. The POST_BUILD is where the patch may be removed, removing the '+' from the kernelrelease. The calculation of the kernelrelease will also stay in its current location but will be ignored if it was already calculated previously. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-01ktest: Fix reboot on success stopping all rebootsSteven Rostedt
The change to let individual tests decide to reboot the machine on success of the entire test also prevented errors from rebooting when an error was detected. The "no_reboot" variable was only cleared if the test had reboot_on_success set. But the no_reboot variable also prevents the test rebooting when an error was detected even when REBOOT_ON_ERROR was set. Add a new "reboot_success" variable that is used to determine if the test should reboot on success and not touch the no_reboot variable. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-05-01ktest.pl: Fix combined usage of BISECT_REVERSE and BISECT_SKIPRuss Dill
When BISECT_REVERSE and BISECT_SKIP are used together with boot or test testing, build failures are treated as boot or test failures and 'git bisect bad' is executed instead of 'git bisect skip'. This is because the $ret value of -1 is treated as a build failure, but the $reverse_bisect logic does not properly handle this. Simple fix, only invert it if it is positive. Link: http://lkml.kernel.org/r/1335235380-8509-1-git-send-email-Russ.Dill@ti.com Signed-off-by: Russ Dill <Russ.Dill@ti.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-28mm: move hugepage test examples to tools/testing/selftests/vmDave Young
hugepage-mmap.c, hugepage-shm.c and map_hugetlb.c in Documentation/vm are simple pass/fail tests, It's better to promote them to tools/testing/selftests. Thanks suggestion of Andrew Morton about this. They all need firstly setting up proper nr_hugepages and hugepage-mmap need to mount hugetlbfs. So I add a shell script run_vmtests to do such work which will call the three test programs and check the return value of them. Changes to original code including below: a. add run_vmtests script b. return error when read_bytes mismatch with writed bytes. c. coding style fixes: do not use assignment in if condition [akpm@linux-foundation.org: build the targets before trying to execute them] [akpm@linux-foundation.org: Documentation/vm/ no longer has a Makefile. Fixes "make clean"] Signed-off-by: Dave Young <dyoung@redhat.com> Cc: Wu Fengguang <fengguang.wu@intel.com> Cc: Christoph Lameter <cl@linux.com> Cc: Pekka Enberg <penberg@cs.helsinki.fi> Cc: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-03-28selftests/Makefile: make `run_tests' depend on `all'Andrew Morton
So a "make run_tests" will build the tests before trying to run them. Acked-by: Frederic Weisbecker <fweisbec@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-03-28selftests: launch individual selftests from the main MakefileFrederic Weisbecker
Remove the run_tests script and launch the selftests by calling "make run_tests" from the selftests top directory instead. This delegates to the Makefile in each selftest directory, where it is decided how to launch the local test. This removes the need to add each selftest directory to the now removed "run_tests" top script. Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Dave Young <dyoung@redhat.com> Cc: Christoph Lameter <cl@linux.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-03-23Merge tag 'ktest-v3.4' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest Pull ktest changes from Steven Rostedt. * tag 'ktest-v3.4' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Allow a test to override REBOOT_ON_SUCCESS ktest: Fix SWITCH_TO_GOOD to also reboot the machine ktest: Add SCP_TO_TARGET_INSTALL option ktest: Add warning when bugs are ignored ktest: Add INSTALL_MOD_STRIP=1 when installing modules
2012-03-21ktest: Allow a test to override REBOOT_ON_SUCCESSSteven Rostedt
The option REBOOT_ON_SUCCESS is global, and will have the machine reboot the the box if all tests are successful. But a test may not want the machine to reboot, and perhaps have the kernel it loaded be used to install the next kernel. Or the last test may set up a kernel that the user may want to look at. In this case, the user could have the global option REBOOT_ON_SUCCESS be true, but if a test is defined to run at the end, that test can override the global option and keep the kernel it installed for the user to log in with. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-21ktest: Fix SWITCH_TO_GOOD to also reboot the machineSteven Rostedt
When the option SWITCH_TO_GOOD is set, it will be called when the system needs to reboot to the good server. But currently, this keeps the reboot from happening. The SWITCH_TO_GOOD is just a way to get to a new kernel, it may not mean to not reboot. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-21ktest: Add SCP_TO_TARGET_INSTALL optionSteven Rostedt
Currently the option used to scp both the modules to the target as well as the kernel image are the same (SCP_TO_TARGET). But some embedded boards may require them to be different. The modules may need to be put directly on the board, but the kernel image may need to go to a tftpserver. Add the option SCP_TO_TARGET_INSTALL that will allow the user to change the config so that they may have the modules and image got to different machines. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-21ktest: Add warning when bugs are ignoredSteven Rostedt
When IGNORE_ERRORS is set, ktest will not fail a test if a backtrace is detected. But this can be an issue if the user added it in the config but forgot to remove it. They may be left wondering why their test did not fail, or even worse, why their bisect gave the wrong commit. Add a warning in the output if IGNORE_WARNINGS is set, and ktest detects a kernel error. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-21ktest: Add INSTALL_MOD_STRIP=1 when installing modulesSteven Rostedt
To keep the modules from bloating the target's filesystem strip them during the install. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-03-20Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial Pull trivial tree from Jiri Kosina: "It's indeed trivial -- mostly documentation updates and a bunch of typo fixes from Masanari. There are also several linux/version.h include removals from Jesper." * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (101 commits) kcore: fix spelling in read_kcore() comment constify struct pci_dev * in obvious cases Revert "char: Fix typo in viotape.c" init: fix wording error in mm_init comment usb: gadget: Kconfig: fix typo for 'different' Revert "power, max8998: Include linux/module.h just once in drivers/power/max8998_charger.c" writeback: fix fn name in writeback_inodes_sb_nr_if_idle() comment header writeback: fix typo in the writeback_control comment Documentation: Fix multiple typo in Documentation tpm_tis: fix tis_lock with respect to RCU Revert "media: Fix typo in mixer_drv.c and hdmi_drv.c" Doc: Update numastat.txt qla4xxx: Add missing spaces to error messages compiler.h: Fix typo security: struct security_operations kerneldoc fix Documentation: broken URL in libata.tmpl Documentation: broken URL in filesystems.tmpl mtd: simplify return logic in do_map_probe() mm: fix comment typo of truncate_inode_pages_range power: bq27x00: Fix typos in comment ...
2012-02-27Merge tag 'ktest-fix-make-min-failed-build-for-real' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest While demoing ktest at ELC in 2012, it was embarrassing that the make_min_config test failed to work because the snowball board I was testing it against had a config that would not build. But the make_min_config only tested the testing part and ignored build failures. The end result was a config file that would not boot. This time, for real. * tag 'ktest-fix-make-min-failed-build-for-real' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Fix make_min_config test when build fails
2012-02-27ktest: Fix make_min_config test when build failsSteven Rostedt
The make_min_config does not take into account when the build fails, resulting in a invalid MIN_CONFIG .config file. When the build fails, it is ignored and the boot test is executed, using the previous built kernel. The configs that should be tested are not tested and they may be added or removed depending on the result of the last kernel that succeeded to be built. If the build fails, mark the current config as a failure and the configs that were disabled may still be needed. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-02-21tools: Fix typo in ktest.plMasanari Iida
Correct spelling "faied" to "failed" in tools/testing/ktest/ktest.pl Signed-off-by: Masanari Iida <standby24x7@gmail.com> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2012-01-16Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest: ktest: Add INGORE_ERRORS to ignore warnings in boot up ktest: Still do reboot even for REBOOT_TYPE = script ktest: Fix compare script to test if options are not documented ktest: Detect typos in option names ktest: Have all values be set by defaults ktest: Change initialization of defaults hash to perl format ktest: Add options SWITCH_TO_GOOD and SWITCH_TO_TEST ktest: Allow overriding bisect test results ktest: Evaluate options before processing them ktest: Evaluate $KERNEL_VERSION in both install and post install ktest: Only ask options needed for install ktest: When creating a new config, ask for BUILD_OPTIONS ktest: Do not ask for some options if the only test is build ktest: Ask for type of test when creating a new config ktest: Allow bisect test to restart where it left off ktest: When creating new config, allow the use of ${THIS_DIR} ktest: Add default for ssh-user, build-target and target-image ktest: Allow success logs to be stored ktest: Save test output
2012-01-12selftests: new x86 breakpoints selftestFrederic Weisbecker
Bring a first selftest in the relevant directory. This tests several combinations of breakpoints and watchpoints in x86, as well as icebp traps and int3 traps. Given the amount of breakpoint regressions we raised after we merged the generic breakpoint infrastructure, such selftest became necessary and can still serve today as a basis for new patches that touch the do_debug() path. Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@elte.hu> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Jason Wessel <jason.wessel@windriver.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Michal Marek <mmarek@suse.cz> Cc: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-12selftests: new very basic kernel selftests directoryFrederic Weisbecker
Bring a new kernel selftests directory in tools/testing/selftests. To add a new selftest, create a subdirectory with the sources and a makefile that creates a target named "run_test" then add the subdirectory name to the TARGET var in tools/testing/selftests/Makefile and tools/testing/selftests/run_tests script. This can help centralizing and maintaining any useful selftest that developers usually tend to let rust in peace on some random server. Suggested-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Ingo Molnar <mingo@elte.hu> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Jason Wessel <jason.wessel@windriver.com> Cc: Will Deacon <will.deacon@arm.com> Cc: Steven Rostedt <rostedt@goodmis.org> Cc: Michal Marek <mmarek@suse.cz> Cc: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2012-01-08Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (53 commits) Kconfig: acpi: Fix typo in comment. misc latin1 to utf8 conversions devres: Fix a typo in devm_kfree comment btrfs: free-space-cache.c: remove extra semicolon. fat: Spelling s/obsolate/obsolete/g SCSI, pmcraid: Fix spelling error in a pmcraid_err() call tools/power turbostat: update fields in manpage mac80211: drop spelling fix types.h: fix comment spelling for 'architectures' typo fixes: aera -> area, exntension -> extension devices.txt: Fix typo of 'VMware'. sis900: Fix enum typo 'sis900_rx_bufer_status' decompress_bunzip2: remove invalid vi modeline treewide: Fix comment and string typo 'bufer' hyper-v: Update MAINTAINERS treewide: Fix typos in various parts of the kernel, and fix some comments. clockevents: drop unknown Kconfig symbol GENERIC_CLOCKEVENTS_MIGR gpio: Kconfig: drop unknown symbol 'CS5535_GPIO' leds: Kconfig: Fix typo 'D2NET_V2' sound: Kconfig: drop unknown symbol ARCH_CLPS7500 ... Fix up trivial conflicts in arch/powerpc/platforms/40x/Kconfig (some new kconfig additions, close to removed commented-out old ones)
2012-01-04ktest: Add INGORE_ERRORS to ignore warnings in boot upSteven Rostedt
When testing a kernel that has warnings, ktest.pl will fail the test when it sees the warning. If you need to test the the kernel and want to ignore the errors that are produced, the option IGNORE_ERRORS has been added. When IGNORE_ERRORS is set to something other than 0, it will ignore call traces due to WARN_ON(). Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-01-04ktest: Still do reboot even for REBOOT_TYPE = scriptSteven Rostedt
The REBOOT_TYPE may be either grub or script, if it is script it is expected that a REBOOT_SCRIPT is defined. With the SWITCH_TO_TEST which is the complement of SWITCH_TO_GOOD, which does basically the same thing as REBOOT_SCRIPT and but for both grub and script, the REBOOT_SCRIPT does not need to be mandatory anymore. Do not require the REBOOT_SCRIPT and always run the reboot code for both grub and script. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2012-01-02misc latin1 to utf8 conversionsAl Viro
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2011-12-22ktest: Fix compare script to test if options are not documentedSteven Rostedt
The compare script compare-ktest-sample.pl checks for options that are defined in ktest.pl and not documented in samples.conf, as well as samples in samples.conf that are not used in ktest.pl. With the switch to the hash format to initialize the ktest variables the compare script needs to be updated to handle the change. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Detect typos in option namesSteven Rostedt
It becomes quite annoying when you go to run a test and then realize that you typed an option name wrong, and the test starts doing the default action and not what you expected it to do. It is even more annoying when you wake up the next day after running the test over night when you discover this. By testing if all options specified in a config file are used by either ktest or were used in one of the option's values we can see if there are any dangling options that were not used. In such a case, show the user the options that were not used and ask them if they want to continue or not. The option IGNORE_UNUSED was also added to allow the user to override this feature. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Have all values be set by defaultsSteven Rostedt
Currently the patchcheck, bisect, and config_bisect variables are only able to be set per test. You can not set a default value for them. By letting default values be set, it makes some config files a bit easier, and also makes it easier to find typos in the option names. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Change initialization of defaults hash to perl formatSteven Rostedt
Initializing each default value by specifying the hash name is ugly. This is one of the rare cases that the "perl way" is actually much cleaner and easier to read. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Add options SWITCH_TO_GOOD and SWITCH_TO_TESTSteven Rostedt
For machines that do no use grub, it may be needed to update an external image (tftp) before doing a reboot into either the test image or the known good image. The option SWITCH_TO_GOOD is added, where if it is defined, the command that is specified as its value will be executed before doing a reboot into a known good image. The option SWITCH_TO_TEST is added, where if it is defined, the command that is specified as its value will be executed before doing a reboot into the test image. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Allow overriding bisect test resultsSteven Rostedt
When running the ktest git bisect test, if the BISECT_TYPE is "test", the bisect is determined to be good or bad based off of the error code of the test that is run. Currently, if the test returns 0, it is considered a pass (good), a non-zero is considered a fail (bad). But it has been requested to add more options, and also change the meanings of the error codes of the test. For example, one may want the test to detect if the commit is not good or bad, (maybe the bisect came to a point where the code in question does not exist). The test could report an error code that should tell ktest to skip the commit. Also, a test could detect that something is horribly wrong and the biscet should just be aborted. The new options: BISECT_RET_GOOD BISECT_RET_BAD BISECT_RET_SKIP BISECT_RET_ABORT BISECT_RET_DEFAULT have been added. The first 4 take an integer value that will represent if the test should be considered a pass, fail, neither good nor bad, or abort respectively. The BISECT_RET_DEFAULT will bo whatever is not defined by the above codes. If only BISECT_RET_DEFAULT is defined, then all tests will do the default. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Evaluate options before processing themSteven Rostedt
All options can take variables "${var}". Before doing any processing or decision making on the content of an option, evaluate it incase there are variables that may change the outcome. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Evaluate $KERNEL_VERSION in both install and post installSteven Rostedt
The install process may also need to know what the kernel version is, to add it to the name. Evaluate it for both install and post install. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Only ask options needed for installSteven Rostedt
If all the tests are only for build or install, do not ask for options not needed to do the install, if the options do not exist. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: When creating a new config, ask for BUILD_OPTIONSSteven Rostedt
When creating a new config, ask for the BUILD_OPTIONS variable that lets users add things like -j20 to the make. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Do not ask for some options if the only test is buildSteven Rostedt
When creating a ktest config or if te config only has build only tests, some of the manditory config options are not needed. Do not ask for them if all tests in the config file are just build tests. Suggested-by: Darren Hart <dvhart@linux.intel.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2011-12-22ktest: Ask for type of test when creating a new configSteven Rostedt
When no argument is supplied to ktest, or the config applied does not exist and a new config is being created, instead of just using the default test type, give the user an option to pick the test type of either 'build, install, or boot'. Other options may be added later but then those would require more questions as they require more fields. But that's for another release of ktest to add that feature. Signed-off-by: Steven Rostedt <rostedt@goodmis.org>