summaryrefslogtreecommitdiff
path: root/scripts/link-vmlinux.sh
AgeCommit message (Collapse)Author
2019-05-08Merge tag 'kbuild-v5.2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: - allow users to invoke 'make' out of the source tree - refactor scripts/mkmakefile - deprecate KBUILD_SRC, which was used to track the source tree location for O= build. - fix recordmcount.pl in case objdump output is localized - turn unresolved symbols in external modules to errors from warnings by default; pass KBUILD_MODPOST_WARN=1 to get them back to warnings - generate modules.builtin.modinfo to collect .modinfo data from built-in modules - misc Makefile cleanups * tag 'kbuild-v5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (21 commits) .gitignore: add more all*.config patterns moduleparam: Save information about built-in modules in separate file Remove MODULE_ALIAS() calls that take undefined macro .gitignore: add leading and trailing slashes to generated directories scripts/tags.sh: fix direct execution of scripts/tags.sh scripts: override locale from environment when running recordmcount.pl samples: kobject: allow CONFIG_SAMPLE_KOBJECT to become y samples: seccomp: turn CONFIG_SAMPLE_SECCOMP into a bool option kbuild: move Documentation to vmlinux-alldirs kbuild: move samples/ to KBUILD_VMLINUX_OBJS modpost: make KBUILD_MODPOST_WARN also configurable for external modules kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build kbuild: remove unneeded dependency for include/config/kernel.release memory: squash drivers/memory/Makefile.asm-offsets kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build kbuild: mkmakefile: generate a simple wrapper of top Makefile kbuild: mkmakefile: do not check the generated Makefile marker kbuild: allow Kbuild to start from any directory kbuild: pass $(MAKECMDGOALS) to sub-make as is kbuild: fix warning "overriding recipe for target 'Makefile'" ...
2019-05-07moduleparam: Save information about built-in modules in separate fileAlexey Gladkov
Problem: When a kernel module is compiled as a separate module, some important information about the kernel module is available via .modinfo section of the module. In contrast, when the kernel module is compiled into the kernel, that information is not available. Information about built-in modules is necessary in the following cases: 1. When it is necessary to find out what additional parameters can be passed to the kernel at boot time. 2. When you need to know which module names and their aliases are in the kernel. This is very useful for creating an initrd image. Proposal: The proposed patch does not remove .modinfo section with module information from the vmlinux at the build time and saves it into a separate file after kernel linking. So, the kernel does not increase in size and no additional information remains in it. Information is stored in the same format as in the separate modules (null-terminated string array). Because the .modinfo section is already exported with a separate modules, we are not creating a new API. It can be easily read in the userspace: $ tr '\0' '\n' < modules.builtin.modinfo ext4.softdep=pre: crc32c ext4.license=GPL ext4.description=Fourth Extended Filesystem ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others ext4.alias=fs-ext4 ext4.alias=ext3 ext4.alias=fs-ext3 ext4.alias=ext2 ext4.alias=fs-ext2 md_mod.alias=block-major-9-* md_mod.alias=md md_mod.description=MD RAID framework md_mod.license=GPL md_mod.parmtype=create_on_open:bool md_mod.parmtype=start_dirty_degraded:int ... Co-Developed-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org> Signed-off-by: Gleb Fotengauer-Malinovskiy <glebfm@altlinux.org> Signed-off-by: Alexey Gladkov <gladkov.alexey@gmail.com> Acked-by: Jessica Yu <jeyu@kernel.org> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-05-06kbuild: tolerate missing pahole when generating BTFAndrii Nakryiko
When BTF generation is enabled through CONFIG_DEBUG_INFO_BTF, scripts/link-vmlinux.sh detects if pahole version is too old and gracefully continues build process, skipping BTF generation build step. But if pahole is not available, build will still fail. This patch adds check for whether pahole exists at all and bails out gracefully, if not. Cc: Alexei Starovoitov <ast@fb.com> Reported-by: Yonghong Song <yhs@fb.com> Fixes: e83b9f55448a ("kbuild: add ability to generate BTF type info for vmlinux") Signed-off-by: Andrii Nakryiko <andriin@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-16kbuild: handle old pahole more gracefully when generating BTFAndrii Nakryiko
When CONFIG_DEBUG_INFO_BTF is enabled but available version of pahole is too old to support BTF generation, build script is supposed to emit warning and proceed with the build. Due to using exit instead of return from BASH function, existing handling code prematurely exits exit code 0, not completing some of the build steps. This patch fixes issue by correctly returning just from gen_btf() function only. Fixes: e83b9f55448a ("kbuild: add ability to generate BTF type info for vmlinux") Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Alexei Starovoitov <ast@fb.com> Cc: Yonghong Song <yhs@fb.com> Cc: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Andrii Nakryiko <andriin@fb.com> Acked-by: Song Liu <songliubraving@fb.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-04-03kbuild: add ability to generate BTF type info for vmlinuxAndrii Nakryiko
This patch adds new config option to trigger generation of BTF type information from DWARF debuginfo for vmlinux and kernel modules through pahole, which in turn relies on libbpf for btf_dedup() algorithm. The intent is to record compact type information of all types used inside kernel, including all the structs/unions/typedefs/etc. This enables BPF's compile-once-run-everywhere ([0]) approach, in which tracing programs that are inspecting kernel's internal data (e.g., struct task_struct) can be compiled on a system running some kernel version, but would be possible to run on other kernel versions (and configurations) without recompilation, even if the layout of structs changed and/or some of the fields were added, removed, or renamed. This is only possible if BPF loader can get kernel type info to adjust all the offsets correctly. This patch is a first time in this direction, making sure that BTF type info is part of Linux kernel image in non-loadable ELF section. BTF deduplication ([1]) algorithm typically provides 100x savings compared to DWARF data, so resulting .BTF section is not big as is typically about 2MB in size. [0] http://vger.kernel.org/lpc-bpf2018.html#session-2 [1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.html Cc: Masahiro Yamada <yamada.masahiro@socionext.com> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Daniel Borkmann <daniel@iogearbox.net> Cc: Alexei Starovoitov <ast@fb.com> Cc: Yonghong Song <yhs@fb.com> Cc: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Andrii Nakryiko <andriin@fb.com> Acked-by: David S. Miller <davem@davemloft.net> Acked-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2019-03-14kbuild: source include/config/auto.conf instead of ${KCONFIG_CONFIG}Masahiro Yamada
As commit 423a8155facf ("kbuild: Fix reading of .config in link-vmlinux.sh") addressed, some shells fail to perform '.' if ${KCONFIG_CONFIG} does not contain a slash at all. Instead, we can source include/config/auto.conf, which obviously contain slashes, and we do not expect its file path overridden by a user. Perhaps, the performance might be slightly better since unset CONFIG options are stripped from include/config/auto.conf. scripts/setlocalversion already works this way. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-01-28kbuild: merge KBUILD_VMLINUX_{INIT,MAIN} into KBUILD_VMLINUX_OBJSMasahiro Yamada
The top Makefile does not need to export KBUILD_VMLINUX_INIT and KBUILD_VMLINUX_MAIN separately. Put every built-in.a into KBUILD_VMLINUX_OBJS. The order of $(head-y), $(init-y), $(core-y), ... is still retained. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2019-01-28kbuild: remove top-level built-in.aMasahiro Yamada
The symbol table in the final archive is unneeded; the linker does not require the symbol table after the --whole-archive option. Every object file in the archive is included in the link anyway. Pass thin archives from subdirectories directly to the linker, and remove the final archiving step. Fix up the document and comments as well. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Nicholas Piggin <npiggin@gmail.com>
2018-08-24kbuild: rename LDFLAGS to KBUILD_LDFLAGSMasahiro Yamada
Commit a0f97e06a43c ("kbuild: enable 'make CFLAGS=...' to add additional options to CC") renamed CFLAGS to KBUILD_CFLAGS. Commit 222d394d30e7 ("kbuild: enable 'make AFLAGS=...' to add additional options to AS") renamed AFLAGS to KBUILD_AFLAGS. Commit 06c5040cdb13 ("kbuild: enable 'make CPPFLAGS=...' to add additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS. For some reason, LDFLAGS was not renamed. Using a well-known variable like LDFLAGS may result in accidental override of the variable. Kbuild generally uses KBUILD_ prefixed variables for the internally appended options, so here is one more conversion to sanitize the naming convention. I did not touch Makefiles under tools/ since the tools build system is a different world. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com>
2018-05-17kallsyms: remove symbol prefix supportMasahiro Yamada
CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG. They were removed by commit 4ba66a976072 ("arch: remove blackfin port"), commit bb6fb6dfcc17 ("metag: Remove arch/metag/"), respectively. No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX, hence the --symbol-prefix option is unnecessary. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com> Reviewed-by: Sam Ravnborg <sam@ravnborg.org>
2018-03-26kbuild: Use ls(1) instead of stat(1) to obtain file sizeMichael Forney
stat(1) is not standardized and different implementations have their own (conflicting) flags for querying the size of a file. ls(1) provides the same information (value of st.st_size) in the 5th column, except when the file is a character or block device. This output is standardized[0]. The -n option turns on -l, which writes lines formatted like "%s %u %s %s %u %s %s\n", <file mode>, <number of links>, <owner name>, <group name>, <size>, <date and time>, <pathname> but instead of writing the <owner name> and <group name>, it writes the numeric owner and group IDs (this avoids /etc/passwd and /etc/group lookups as well as potential field splitting issues). The <size> field is specified as "the value that would be returned for the file in the st_size field of struct stat". To avoid duplicating logic in several locations in the tree, create scripts/file-size.sh and update callers to use that instead of stat(1). [0] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73_10 Signed-off-by: Michael Forney <forney@google.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26kbuild: rename built-in.o to built-in.aNicholas Piggin
Incremental linking is gone, so rename built-in.o to built-in.a, which is the usual extension for archive files. This patch does two things, first is a simple search/replace: git grep -l 'built-in\.o' | xargs sed -i 's/built-in\.o/built-in\.a/g' The second is to invert nesting of nested text manipulations to avoid filtering built-in.a out from libs-y2: -libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.a, $(libs-y))) +libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y))) Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-26kbuild: remove incremental linking optionNicholas Piggin
This removes the old `ld -r` incremental link option, which has not been selected by any architecture since June 2017. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-03-02kbuild: drop superfluous GCC_PLUGINS_CFLAGS assignmentCao jin
GCC_PLUGINS_CFLAGS is already in the environment, so it is superfluous to add it in commandline of final build of init/. Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-11-17Merge tag 'kbuild-v4.15' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild Pull Kbuild updates from Masahiro Yamada: "One of the most remarkable improvements in this cycle is, Kbuild is now able to cache the result of shell commands. Some variables are expensive to compute, for example, $(call cc-option,...) invokes the compiler. It is not efficient to redo this computation every time, even when we are not actually building anything. Kbuild creates a hidden file ".cache.mk" that contains invoked shell commands and their results. The speed-up should be noticeable. Summary: - Fix arch build issues (hexagon, sh) - Clean up various Makefiles and scripts - Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles - Cache variables that are expensive to compute - Improve cc-ldopton and ld-option for Clang - Optimize output directory creation" * tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits) kbuild: move coccicheck help from scripts/Makefile.help to top Makefile sh: decompressor: add shipped files to .gitignore frv: .gitignore: ignore vmlinux.lds selinux: remove unnecessary assignment to subdir- kbuild: specify FORCE in Makefile.headersinst as .PHONY target kbuild: remove redundant mkdir from ./Kbuild kbuild: optimize object directory creation for incremental build kbuild: create object directories simpler and faster kbuild: filter-out PHONY targets from "targets" kbuild: remove redundant $(wildcard ...) for cmd_files calculation kbuild: create directory for make cache only when necessary sh: select KBUILD_DEFCONFIG depending on ARCH kbuild: fix linker feature test macros when cross compiling with Clang kbuild: shrink .cache.mk when it exceeds 1000 lines kbuild: do not call cc-option before KBUILD_CFLAGS initialization kbuild: Cache a few more calls to the compiler kbuild: Add a cache for generated variables kbuild: add forward declaration of default target to Makefile.asm-generic kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE ...
2017-11-02License cleanup: add SPDX GPL-2.0 license identifier to files with no licenseGreg Kroah-Hartman
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2017-10-09kbuild: link-vmlinux.sh: simplify .version incrementMasahiro Yamada
Since commit 1f2bfbd00e46 ("kbuild: link of vmlinux moved to a script"), it is easy to increment .version without using a temporary file .old_version. I do not see anybody who creates the .tmp_version. Probably it is a left-over of commit 4e25d8bb9550fb ("[PATCH] kbuild: adjust .version updating"). Just remove it. Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-06-30kbuild: handle libs-y archives separately from built-in.o archivesNicholas Piggin
The thin archives build currently puts all lib.a and built-in.o files together and links them with --whole-archive. This works because thin archives can recursively refer to thin archives. However some architectures include libgcc.a, which may not be a thin archive, or it may not be constructed with the "P" option, in which case its contents do not get linked correctly. So don't pull .a libs into the root built-in.o archive. These libs should already have symbol tables and indexes built, so they can be direct linker inputs. Move them out of the --whole-archive option, which restore the conditional linking behaviour of lib.a to thin archives builds. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-06-30kbuild: thin archives use P option to arNicholas Piggin
The P option makes ar do full path name matching and can prevent ar from discarding files with duplicate names in some cases of creating thin archives from thin archives. The sh architecture in particular loses some object files from its kernel/cpu/sh*/ directories without this option. This could be a bug in binutils ar, but the P option should not cause any negative effects so it is safe to use to work around this with. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2017-06-30kbuild: thin archives final link close --whole-archives optionNicholas Piggin
Close the --whole-archives option with --no-whole-archive. Some architectures end up including additional .o and files multiple times after this, and they get duplicate symbols when they are brought under the --whole-archives option. This matches more closely with the incremental final link. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-11-29kbuild: minor improvement for thin archives buildNicholas Piggin
The root built-in.o archive is currently generated before all object files are built for the final link, due to final build of init/ after version update. In practice it seems like it doesn't matter because the archive symbol table does not change, but it is more logical to create the final archive as the last step. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-11-29kbuild: kallsyms allow 3-pass generation if symbols size has changedNicholas Piggin
kallsyms generation is not foolproof, due to some linkers adding symbols (e.g., branch trampolines) when a binary size changes. Have it attempt a 3rd pass automatically if the kallsyms size changes in the 2nd pass. This allows powerpc64 allyesconfig to build without adding another pass when it's not required. This can be solved other ways by directing the linker not to add labels on branch stubs, or to move kallsyms near the end of the image. The former is undesirable for debugging/tracing, and the latter is a more significant change that requires more testing and review. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-09-09kbuild: allow architectures to use thin archives instead of ld -rStephen Rothwell
ld -r is an incremental link used to create built-in.o files in build subdirectories. It produces relocatable object files containing all its input files, and these are are then pulled together and relocated in the final link. Aside from the bloat, this constrains the final link relocations, which has bitten large powerpc builds with unresolvable relocations in the final link. Alan Modra has recommended the kernel use thin archives for linking. This is an alternative and means that the linker has more information available to it when it links the kernel. This patch enables a config option architectures can select, which causes all built-in.o files to be built as thin archives. built-in.o files in subdirectories do not get symbol table or index attached, which improves speed and size. The final link pass creates a built-in.o archive in the root output directory which includes the symbol table and index. The linker then uses takes this file to link. The --whole-archive linker option is required, because the linker now has visibility to every individual object file, and it will otherwise just completely avoid including those without external references (consider a file with EXPORT_SYMBOL or initcall or hardware exceptions as its only entry points). The traditional built works "by luck" as built-in.o files are large enough that they're going to get external references. However this optimisation is unpredictable for the kernel (due to above external references), ineffective at culling unused, and costly because the .o files have to be searched for references. Superior alternatives for link-time culling should be used instead. Build characteristics for inclink vs thinarc, on a small powerpc64le pseries VM with a modest .config: inclink thinarc sizes vmlinux 15 618 680 15 625 028 sum of all built-in.o 56 091 808 1 054 334 sum excluding root built-in.o 151 430 find -name built-in.o | xargs rm ; time make vmlinux real 22.772s 21.143s user 13.280s 13.430s sys 4.310s 2.750s - Final kernel pulled in only about 6K more, which shows how ineffective the object file culling is. - Build performance looks improved due to less pagecache activity. On IO constrained systems it could be a bigger win. - Build size saving is significant. Side note, the toochain understands archives, so there's some tricks, $ ar t built-in.o # list all files you linked with $ size built-in.o # and their sizes $ objdump -d built-in.o # disassembly (unrelocated) with filenames Implementation by sfr, minor tweaks by npiggin. Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-06-07GCC plugin infrastructureEmese Revfy
This patch allows to build the whole kernel with GCC plugins. It was ported from grsecurity/PaX. The infrastructure supports building out-of-tree modules and building in a separate directory. Cross-compilation is supported too. Currently the x86, arm, arm64 and uml architectures enable plugins. The directory of the gcc plugins is scripts/gcc-plugins. You can use a file or a directory there. The plugins compile with these options: * -fno-rtti: gcc is compiled with this option so the plugins must use it too * -fno-exceptions: this is inherited from gcc too * -fasynchronous-unwind-tables: this is inherited from gcc too * -ggdb: it is useful for debugging a plugin (better backtrace on internal errors) * -Wno-narrowing: to suppress warnings from gcc headers (ipa-utils.h) * -Wno-unused-variable: to suppress warnings from gcc headers (gcc_version variable, plugin-version.h) The infrastructure introduces a new Makefile target called gcc-plugins. It supports all gcc versions from 4.5 to 6.0. The scripts/gcc-plugin.sh script chooses the proper host compiler (gcc-4.7 can be built by either gcc or g++). This script also checks the availability of the included headers in scripts/gcc-plugins/gcc-common.h. The gcc-common.h header contains frequently included headers for GCC plugins and it has a compatibility layer for the supported gcc versions. The gcc-generate-*-pass.h headers automatically generate the registration structures for GIMPLE, SIMPLE_IPA, IPA and RTL passes. Note that 'make clean' keeps the *.so files (only the distclean or mrproper targets clean all) because they are needed for out-of-tree modules. Based on work created by the PaX Team. Signed-off-by: Emese Revfy <re.emese@gmail.com> Acked-by: Kees Cook <keescook@chromium.org> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-04-07ARM: 8552/1: kallsyms: remove special lower address limit for CONFIG_ARMArd Biesheuvel
Now that we no longer emit .stubs symbols into a section VMA loaded at absolute address 0x1000, we can drop the ARM-specific override that sets a lower limit based on CONFIG_PAGE_OFFSET, below which symbols are filtered from the kallsyms output. Acked-by: Nicolas Pitre <nico@linaro.org> Acked-by: Chris Brandt <chris.brandt@renesas.com> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
2016-03-24Merge branch 'kbuild' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild Pull kbuild updates from Michal Marek: - make dtbs_install fix - Error handling fix fixdep and link-vmlinux.sh - __UNIQUE_ID fix for clang - Fix for if_changed_* to suppress the "is up to date." message - The kernel is built with -Werror=incompatible-pointer-types * 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild: kbuild: Add option to turn incompatible pointer check into error kbuild: suppress annoying "... is up to date." message kbuild: fixdep: Check fstat(2) return value scripts/link-vmlinux.sh: force error on kallsyms failure Kbuild: provide a __UNIQUE_ID for clang dtbsinstall: don't move target directory out of the way
2016-03-15kallsyms: add support for relative offsets in kallsyms address tableArd Biesheuvel
Similar to how relative extables are implemented, it is possible to emit the kallsyms table in such a way that it contains offsets relative to some anchor point in the kernel image rather than absolute addresses. On 64-bit architectures, it cuts the size of the kallsyms address table in half, since offsets between kernel symbols can typically be expressed in 32 bits. This saves several hundreds of kilobytes of permanent .rodata on average. In addition, the kallsyms address table is no longer subject to dynamic relocation when CONFIG_RELOCATABLE is in effect, so the relocation work done after decompression now doesn't have to do relocation updates for all these values. This saves up to 24 bytes (i.e., the size of a ELF64 RELA relocation table entry) per value, which easily adds up to a couple of megabytes of uncompressed __init data on ppc64 or arm64. Even if these relocation entries typically compress well, the combined size reduction of 2.8 MB uncompressed for a ppc64_defconfig build (of which 2.4 MB is __init data) results in a ~500 KB space saving in the compressed image. Since it is useful for some architectures (like x86) to retain the ability to emit absolute values as well, this patch also adds support for capturing both absolute and relative values when KALLSYMS_ABSOLUTE_PERCPU is in effect, by emitting absolute per-cpu addresses as positive 32-bit values, and addresses relative to the lowest encountered relative symbol as negative values, which are subtracted from the runtime address of this base symbol to produce the actual address. Support for the above is enabled by default for all architectures except IA-64 and Tile-GX, whose symbols are too far apart to capture in this manner. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Reviewed-by: Kees Cook <keescook@chromium.org> Tested-by: Kees Cook <keescook@chromium.org> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Ingo Molnar <mingo@kernel.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Michal Marek <mmarek@suse.cz> Cc: Rusty Russell <rusty@rustcorp.com.au> Cc: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-03-15x86: kallsyms: disable absolute percpu symbols on !SMPArd Biesheuvel
scripts/kallsyms.c has a special --absolute-percpu command line option which deals with the zero based per cpu offsets that are used when building for SMP on x86_64. This means that the option should only be passed in that case, so add a Kconfig symbol with the correct predicate, and use that instead. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Tested-by: Guenter Roeck <linux@roeck-us.net> Reviewed-by: Kees Cook <keescook@chromium.org> Tested-by: Kees Cook <keescook@chromium.org> Acked-by: Rusty Russell <rusty@rustcorp.com.au> Cc: Heiko Carstens <heiko.carstens@de.ibm.com> Cc: Michael Ellerman <mpe@ellerman.id.au> Cc: Ingo Molnar <mingo@kernel.org> Cc: H. Peter Anvin <hpa@zytor.com> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org> Cc: Michal Marek <mmarek@suse.cz> Cc: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2016-02-08scripts/link-vmlinux.sh: force error on kallsyms failureArd Biesheuvel
Since the output of the invocation of scripts/kallsyms is piped directly into the assembler, error messages it emits are visible on stderr, but a non-zero return code is ignored, and the build simply proceeds in that case. However, the resulting kernel is most likely broken, and will crash at boot. So instead, capture the output of kallsyms in a separate .S file, and pass that to the assembler in a separate step. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> Signed-off-by: Michal Marek <mmarek@suse.com>
2016-01-10um: link with -lpthreadVegard Nossum
Similarly to commit fb1770aa78a43530940d0c2dd161e77bc705bdac, with gcc 5 on Ubuntu and CONFIG_STATIC_LINK=y I was seeing these linker errors: /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/librt.a(timer_create.o): In function `__timer_create_new': (.text+0xcd): undefined reference to `pthread_once' /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/librt.a(timer_create.o): In function `__timer_create_new': (.text+0x126): undefined reference to `pthread_attr_init' /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/librt.a(timer_create.o): In function `__timer_create_new': (.text+0x168): undefined reference to `pthread_attr_setdetachstate' [...] Obviously we also need -lpthread for librt.a. Cc: stable@vger.kernel.org # 4.4 Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com> Signed-off-by: Richard Weinberger <richard@nod.at>
2015-12-08arch: um: fix error when linking vmlinux.Lorenzo Colitti
On gcc Ubuntu 4.8.4-2ubuntu1~14.04, linking vmlinux fails with: arch/um/os-Linux/built-in.o: In function `os_timer_create': /android/kernel/android/arch/um/os-Linux/time.c:51: undefined reference to `timer_create' arch/um/os-Linux/built-in.o: In function `os_timer_set_interval': /android/kernel/android/arch/um/os-Linux/time.c:84: undefined reference to `timer_settime' arch/um/os-Linux/built-in.o: In function `os_timer_remain': /android/kernel/android/arch/um/os-Linux/time.c:109: undefined reference to `timer_gettime' arch/um/os-Linux/built-in.o: In function `os_timer_one_shot': /android/kernel/android/arch/um/os-Linux/time.c:132: undefined reference to `timer_settime' arch/um/os-Linux/built-in.o: In function `os_timer_disable': /android/kernel/android/arch/um/os-Linux/time.c:145: undefined reference to `timer_settime' This is because -lrt appears in the generated link commandline after arch/um/os-Linux/built-in.o. Fix this by removing -lrt from arch/um/Makefile and adding it to the UM-specific section of scripts/link-vmlinux.sh. Signed-off-by: Lorenzo Colitti <lorenzo@google.com> Signed-off-by: Richard Weinberger <richard@nod.at>
2015-05-21scripts: link-vmlinux: Don't pass page offset to kallsyms if XIP KernelMaxime Coquelin
When Kernel is executed in place from ROM, the symbol addresses can be lower than the page offset. Tested-by: Chanwoo Choi <cw00.choi@samsung.com> Signed-off-by: Maxime Coquelin <mcoquelin.stm32@gmail.com> Tested-by: Andreas Färber <afaerber@suse.de> Signed-off-by: Michal Marek <mmarek@suse.cz>
2015-05-07scripts: fix link-vmlinux.sh bash-ismSylvain BERTRAND
While building linux with dash shell: LINK vmlinux trap: SIGHUP: bad trap /src/linux-4.0/Makefile:933: recipe for target 'vmlinux' failed make[1]: *** [vmlinux] Error 1 See the following document for behavior of posix shell trap instruction: http://pubs.opengroup.org/onlinepubs/000095399/utilities/trap.html Cc: Ingo Molnar <mingo@kernel.org> Signed-off-by: Sylvain BERTRAND <sylvain.bertrand@gmail.com> Signed-off-by: Michal Marek <mmarek@suse.cz>
2014-08-20kbuild: Make scripts executableMichal Marek
The Makefiles call the respective interpreter explicitly, but this makes it easier to use the scripts manually. Signed-off-by: Michal Marek <mmarek@suse.cz>
2014-03-17kallsyms: fix percpu vars on x86-64 with relocation.Rusty Russell
x86-64 has a problem: per-cpu variables are actually represented by their absolute offsets within the per-cpu area, but the symbols are not emitted as absolute. Thus kallsyms naively creates them as offsets from _text, meaning their values change if the kernel is relocated (especially noticeable with CONFIG_RANDOMIZE_BASE): $ egrep ' (gdt_|_(stext|_per_cpu_))' /root/kallsyms.nokaslr 0000000000000000 D __per_cpu_start 0000000000004000 D gdt_page 0000000000014280 D __per_cpu_end ffffffff810001c8 T _stext ffffffff81ee53c0 D __per_cpu_offset $ egrep ' (gdt_|_(stext|_per_cpu_))' /root/kallsyms.kaslr1 000000001f200000 D __per_cpu_start 000000001f204000 D gdt_page 000000001f214280 D __per_cpu_end ffffffffa02001c8 T _stext ffffffffa10e53c0 D __per_cpu_offset Making them absolute symbols is the Right Thing, but requires fixes to the relocs tool. So for the moment, we add a --absolute-percpu option which makes them absolute from a kallsyms perspective: $ egrep ' (gdt_|_(stext|_per_cpu_))' /proc/kallsyms # no KASLR 0000000000000000 A __per_cpu_start 000000000000a000 A gdt_page 0000000000013040 A __per_cpu_end ffffffff802001c8 T _stext ffffffff8099b180 D __per_cpu_offset ffffffff809a3000 D __per_cpu_load $ egrep ' (gdt_|_(stext|_per_cpu_))' /proc/kallsyms # With KASLR 0000000000000000 A __per_cpu_start 000000000000a000 A gdt_page 0000000000013040 A __per_cpu_end ffffffff89c001c8 T _stext ffffffff8a39d180 D __per_cpu_offset ffffffff8a3a5000 D __per_cpu_load Based-on-the-original-screenplay-by: Andy Honig <ahonig@google.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Acked-by: Kees Cook <keescook@chromium.org>
2013-12-10scripts/link-vmlinux.sh: only filter kernel symbols for armMing Lei
Actually CONFIG_PAGE_OFFSET isn't same with PAGE_OFFSET, so it isn't easy to figue out PAGE_OFFSET defined in header file from scripts. Because CONFIG_PAGE_OFFSET may not be defined in some ARCHs( 64bit ARCH), or defined as bogus value in !MMU case, so this patch only applys the filter on ARM when CONFIG_PAGE_OFFSET is defined as the original problem is only on ARM. Cc: <stable@vger.kernel.org> Cc: Rusty Russell <rusty@rustcorp.com.au> Fixes: f6537f2f0eba4eba3354e48dbe3047db6d8b6254 Singed-off-by: Ming Lei <tom.leiming@gmail.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
2013-11-02scripts/kallsyms: filter symbols not in kernel address spaceMing Lei
This patch uses CONFIG_PAGE_OFFSET to filter symbols which are not in kernel address space because these symbols are generally for generating code purpose and can't be run at kernel mode, so we needn't keep them in /proc/kallsyms. For example, on ARM there are some symbols which may be linked in relocatable code section, then perf can't parse symbols any more from /proc/kallsyms, this patch fixes the problem (introduced b9b32bf70f2fb710b07c94e13afbc729afe221da) Cc: Russell King <linux@arm.linux.org.uk> Cc: linux-arm-kernel@lists.infradead.org Cc: Michal Marek <mmarek@suse.cz> Signed-off-by: Ming Lei <tom.leiming@gmail.com> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Cc: stable@vger.kernel.org
2013-03-15CONFIG_SYMBOL_PREFIX: cleanup.Rusty Russell
We have CONFIG_SYMBOL_PREFIX, which three archs define to the string "_". But Al Viro broke this in "consolidate cond_syscall and SYSCALL_ALIAS declarations" (in linux-next), and he's not the first to do so. Using CONFIG_SYMBOL_PREFIX is awkward, since we usually just want to prefix it so something. So various places define helpers which are defined to nothing if CONFIG_SYMBOL_PREFIX isn't set: 1) include/asm-generic/unistd.h defines __SYMBOL_PREFIX. 2) include/asm-generic/vmlinux.lds.h defines VMLINUX_SYMBOL(sym) 3) include/linux/export.h defines MODULE_SYMBOL_PREFIX. 4) include/linux/kernel.h defines SYMBOL_PREFIX (which differs from #7) 5) kernel/modsign_certificate.S defines ASM_SYMBOL(sym) 6) scripts/modpost.c defines MODULE_SYMBOL_PREFIX 7) scripts/Makefile.lib defines SYMBOL_PREFIX on the commandline if CONFIG_SYMBOL_PREFIX is set, so that we have a non-string version for pasting. (arch/h8300/include/asm/linkage.h defines SYMBOL_NAME(), too). Let's solve this properly: 1) No more generic prefix, just CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX. 2) Make linux/export.h usable from asm. 3) Define VMLINUX_SYMBOL() and VMLINUX_SYMBOL_STR(). 4) Make everyone use them. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reviewed-by: James Hogan <james.hogan@imgtec.com> Tested-by: James Hogan <james.hogan@imgtec.com> (metag)
2013-02-25kbuild: Fix reading of .config in link-vmlinux.shMichal Marek
The shell '.' command is not required to search the current directory as a fallback and in fact newer versions of bash in sh-mode do not do this. Force reading the file from the current directory if $KCONFIG_CONFIG contains no '/'. Reported-by: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Michal Marek <mmarek@suse.cz>
2013-02-22scripts/link-vmlinux.sh: source variables from KCONFIG_CONFIGMichael Grzeschik
Its possible to superseed the config file with KCONFIG_CONFIG and have completely no .config in the tree. The current script is sourcing .config in every case, so the kernel will never build succesfully. This patch fixes that issue by sourcing KCONFIG_CONFIG instead. Signed-off-by: Michael Grzeschik <m.grzeschik@pengutronix.de> Signed-off-by: Michal Marek <mmarek@suse.cz>
2012-09-12Merge branch 'for-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/lliubbo/blackfin Pull blackfin updates from Bob Liu: "One kbuild and a smp build fix." * 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/lliubbo/blackfin: kbuild: add symbol prefix arg to kallsyms blackfin: smp: adapt to generic smp helpers
2012-09-11kbuild: add symbol prefix arg to kallsymsJames Hogan
Commit 1f2bfbd00e466ff3489b2ca5cc75b1cccd14c123 ("kbuild: link of vmlinux moved to a script") introduced in v3.5-rc1 broke kallsyms on architectures which have symbol prefixes. The --symbol-prefix argument used to be added to the KALLSYMS command line from the architecture Makefile, however this isn't picked up by the new scripts/link-vmlinux.sh. This resulted in symbols like kallsyms_addresses being added which weren't correctly overriding the weak symbols such as _kallsyms_addresses. These could then trigger BUG_ONs in kallsyms code. This is fixed by removing the KALLSYMS addition from the architecture Makefile, and using CONFIG_SYMBOL_PREFIX in the link-vmlinux.sh script to determine whether to add the --symbol-prefix argument. Signed-off-by: James Hogan <james.hogan@imgtec.com> Signed-off-by: Bob Liu <lliubbo@gmail.com>
2012-08-10link-vmlinux.sh: Fix stray "echo" in error messageMichal Marek
Reported-by: Jan Engelhardt <jengelh@inai.de> Signed-off-by: Michal Marek <mmarek@suse.cz>
2012-07-07kbuild: Print errors to stderrMichal Marek
... at least in the top-level Makefile and scripts/link-vmlinux.sh. There are some more instances of the 'echo <error>; exit 1' pattern in some arch Makefiles and kconfig. Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Michal Marek <mmarek@suse.cz>
2012-05-28Merge branch 'kbuild' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild Pull kbuild updates from Michal Marek. Fixed up nontrivial merge conflict in Makefile as per Stephen Rothwell and linux-next (and trivial arch/sparc/Makefile changes due to removed sparc32 logic). * 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild: mips: Fix KBUILD_CPPFLAGS definition kbuild: fix ia64 link kbuild: document KBUILD_LDS, KBUILD_VMLINUX_{INIT,MAIN} and LDFLAGS_vmlinux kbuild: link of vmlinux moved to a script kbuild: refactor final link of sparc32 kbuild: drop unused KBUILD_VMLINUX_OBJS from top-level Makefile kbuild: Makefile: remove unnecessary check for m68knommu ARCH
2012-05-10kbuild: fix ia64 linkSam Ravnborg
ia64 build failed like this: CC init/version.o LD init/built-in.o KSYM .tmp_kallsyms1.o ld: .tmp_kallsyms1.o: linking constant-gp files with non-constant-gp files ld: failed to merge target specific data of file .tmp_kallsyms1.o make: *** [vmlinux] Error 1 This was introduced when link of vmlinux was migrated to a script. Add missing option to as to fix this. Reported-by: Tony Luck <tony.luck@gmail.com> Tested-by: Tony Luck <tony.luck@gmail.com> Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Michal Marek <mmarek@suse.cz>
2012-05-05kbuild: link of vmlinux moved to a scriptSam Ravnborg
Move the final link of vmlinux to a script to improve readability and maintainability of the code. The Makefile fragments used to link vmlinux has over the years seen far too many changes and the logic had become hard to follow. As the process by nature is serialized there was nothing gained including this in the Makefile. "um" has special link requirments - and the only way to handle this was to hard-code the linking of "um" in the script. This was better than trying to modularize it only for the benefit of "um" anyway. The shell script has been improved after input from: Arnaud Lacombe <lacombar@gmail.com> Nick Bowler <nbowler@elliptictech.com> Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Cc: Arnaud Lacombe <lacombar@gmail.com> Cc: Nick Bowler <nbowler@elliptictech.com> Cc: Richard Weinberger <richard@nod.at> Signed-off-by: Michal Marek <mmarek@suse.cz>