Re: [PATCH v3 0/3] admin-guide: extend perf-security with resource control, data categories and privileged users
On 18.02.2019 2:14, Jonathan Corbet wrote: > On Mon, 11 Feb 2019 16:32:33 +0300 > Alexey Budankov wrote: > >> The patch set extends the first version of perf-security.rst documentation >> file [1], [2], [3] with the following topics: >> >> 1) perf_events/Perf resource limits and control management that describes >>RLIMIT_NOFILE and perf_event_mlock_kb settings for processes conducting >>performance monitoring; >> >> 2) categories of system and performance data that can be captured by >>perf_events/Perf with explicit designation of process sensitive data; >> >> 3) possible steps to create perf_event/Perf privileged users groups for >>the current implementations of perf_events syscall API [4] and Perf tool; > > I've applied the set, thanks. Thanks a lot Jon! ~Alexey > > jon >
Re: [RFC v7 2/5] dt-bindings: pstore-block: new support for blkoops
On Thu, Feb 14, 2019 at 7:06 PM liaoweixiong wrote: > > On 2019-02-14 04:30, Rob Herring wrote: > > On Wed, Feb 13, 2019 at 7:51 AM liaoweixiong > > wrote: > >> > >> > >> On 2019-01-31 00:07, Rob Herring wrote:> On Wed, Jan 23, 2019 at > >> 08:05:13PM +0800, liaoweixiong wrote: > Create DT binding document for blkoops. > > Signed-off-by: liaoweixiong > --- > .../devicetree/bindings/pstore-block/blkoops.txt | 32 > ++ > >>> > >>> /bindings/pstore/... > >>> > >>> I wouldn't call it blkoops either. I believe ramoops is called that to > >>> maintain compatibility keeping the same kernel module name that > >>> preceeded pstore. > >>> > >> > >> Fixed. > >> > >> In addition, I don't known whether should we move > >> ramreserved-memory/ooos.txt to /bindings/pstore. This is for maintainer > >> to decide, and do it on other patch. > >> > MAINTAINERS| 1 + > 2 files changed, 33 insertions(+) > create mode 100644 > Documentation/devicetree/bindings/pstore-block/blkoops.txt > > diff --git a/Documentation/devicetree/bindings/pstore-block/blkoops.txt > b/Documentation/devicetree/bindings/pstore-block/blkoops.txt > new file mode 100644 > index 000..a25835b > --- /dev/null > +++ b/Documentation/devicetree/bindings/pstore-block/blkoops.txt > @@ -0,0 +1,32 @@ > +Blkoops oops logger > +=== > + > +Blkoops provides a block partition for oops, excluding panics now, so > they can > +be recovered after a reboot. > + > +Any space of block partition will be used for a circular buffer of oops > records. > +These records have a configurable size, with a size of 0 indicating > that they > +should be disabled. > + > +"partition-size" and at least one of "dmesg-size" or "pmsg-size" must > be set > +non-zero, but are otherwise optional as listed below. > + > +Blkoops will take value from Kconfig if device tree do not set, but > settings > +from module parameters can also overwrite them. > >>> > >>> That's all kernel details not relevant to the binidng. > >>> > >> > >> Deleted. > >> > + > +Required properties: > + > +- compatible: must be "blkoops". > + > +- partition-size: size in kbytes, must be a multiple of 4. > >>> > >>> This seems unnecessary given a partition has a known size. > >>> > >> > >> partition-size is necessary for psotre/blk. User should tell pstore/blk > >> how large space can it use. > > > > The partition table says how big a partition is. If you only want to > > use part of it, then make the partition smaller or use a file system. > > This is a solved problem, so we don't need a new way in DT to handle > > this. > > > > You are right, partition size is known and pstore/blk can get it > automatically from specified block device. I will try to do it on next > version patch. > But i prefer to rename partition-size to total-size and move it to > optional properties. Total-size means how big space pstore/blk can use. > It is not only about partition size as pstore/blk can use ram if no > partition specified. > So, I will process as follow logic: > 1. If specify total size, use total size. > 2. If no total size but specify partition, get size from partition. You haven't given any reason why we need to support this. > +Optional properties: > + > +- partition-path: strings must begin with "/dev", tell blkoops which > partition > + it can used. If it is not set, blkoops will drop all data when reboot. > >>> > >>> No. '/dev/...' is a Linux thing and doesn't belong in DT. > >>> > >>> You should define a partition UUID and/or label and the kernel can find > >>> the right partition to use. > >>> > >> > >> pstore/blk do general read/write by filp_open/kernel_read/kernel_write, > >> which need device path. > >> In addition, i have no idea how to use UUID and/or label to do general > >> read/write on kernel layer, can you give me a tip? > > > > The kernel can mount a filesystem by label or UUID though I think > > those are filesystem UUID and label, not partition UUID and label. But > > certainly bootloaders find the EFI system partition by UUID. > > > > As your words, those are file system UUID and label, not partition > UUID/label. Actually, looking at do_mounts.c, it is the partition UUID and label. See PARTUUID and PARTLABEL. > Pstore is a file system, there is no other file system any > more for specified partition. Pstore/blk can't get specified partition > by UUID or label. > > As far as i know, block device manager on user space scans each block > device and matches file system table to get UUID and label. Not even all > file systems have UUID/label. Yes, userspace has more capabilities for mounting. > MTD device may has label, but it is not suitable for all block device. MTD is speci
[PATCH v11] dm: add support to directly boot to a mapped device
Add a dm-mod.create= kernel module parameter. It allows device-mapper targets to be configured at boot time for use early in the boot process (as the root device or otherwise). Signed-off-by: Will Drewry Signed-off-by: Kees Cook [rework to use dm_ioctl calls] Signed-off-by: Enric Balletbo i Serra [refactored for upstream] Signed-off-by: Helen Koike --- Hello, As mentioned in previous discussions, the need to create an initramfs adds another layer of complexity when performing tests or preparing a BSP for a board. Taking care of initramfs is always a bit painful, it also occupies space and is another level of complexity in the stack. A practical example as mentioned by Kees is that Chrome OS has a limited amount of storage available for the boot image as it is covered by the static root of trust signature. This feature is already used by Android and Chrome OS in devices already shipped to the market/end-users. So, as mentioned in the previous version, this patchset allows dm devices to be configured in the kernel command line parameter for use early in the boot process without an initramfs. One of the main difference in this version is that instead of using "dm=", I'm using a parameter in the module "dm-mod.create=" (thanks Ezequiel for the idea). The syntax used in the boot param is based on the concise format from the dmsetup tool as described in its man page http://man7.org/linux/man-pages/man8/dmsetup.8.html#CONCISE_FORMAT Which is: dm-mod.create=[,+][;[,+]+] Where, ::= The device name. ::= ---- | "" ::= The device minor number | "" ::= "ro" | "rw" ::= ::= "verity" | "linear" | ... Example, the following could be added in the boot parameters. dm-mod.create="lroot,,,rw, 0 4096 linear 98:16 0, 4096 4096 linear 98:32 0" root=/dev/dm-0 Please check the patch with the documentation on the format. The idea to make it compatible with the dmsetup concise format is to make it easier for users, allowing just copy & paste from the output of the command: sudo dmsetup table --concise /dev/mapper/lroot I refactored the code for this version, instead of pretending to be userspace and performing ioctls, the code just parse the cmd line argument and create the device directly, which simplified and reduced a lot the code (and it is much easier to read). Also, not all dm targets are allowed. Only the ones that I tested were allowed and the ones that doesn't change any block device when the dm is create as read-only, i.e. mirror and cache are not allowed, because if the user makes a mistake and chose the wrong device to be the mirror or choose the wrong device to be the cache, it can corrupt data. So the only targets allowed are: * crypt * delay * linear * snapshot-origin * striped * verity I wrote a script to perform several tests using qemu that can be found at https://gitlab.collabora.com/koike/dm-cmdline-test And you can see the result of the tests at: https://people.collabora.com/~koike/dm-test-module-param-logs.txt At the end of this file: "Total successes: 52. Total failures: 0" And the output of qemu for each run: https://people.collabora.com/~koike/dm-test-module-param-logs.tar.gz I think it would nice to integrate these tests in some CI (maybe kernel CI?) Please let me know your comments. Changes in v11: - Configure the device directly instead of performing in IOCTL (this removed a lot of parsing code) - Just enable the targets that were tested. - Simplify/refactor parsing, as a consequence, escaping characters is not allowed (but it wans't properly used in the previous version anyway) - don't use sscanf, the size wans't being limited and we could have a buffer overflow. - change constrained targets list - remove code from init/ - use a module parameter instead of a kernel comand line parameter in init/ - rename dm-boot to dm-init Changes in v10: - https://lore.kernel.org/patchwork/project/lkml/list/?series=371523 - new file: drivers/md/dm-boot.c - most of the parsing code was moved from init/do_mounts_dm.c to drivers/md/dm-boot.c - parsing code was in essence replaced by the concise parser from dmsetup _create_concise function: https://sourceware.org/git/?p=lvm2.git;a=blob;f=libdm/dm-tools/dmsetup.c;h=835fdcdc75e8f0f0f7c4ed46cc9788a6616f58b8;hb=7498f8383397a93db95655ca227257836cbcac82#l1265 the main reason is that this code is already being used/tested by dmsetup, so we can have some level of confidence that it works as expected. Besides this, it also looks more efficient. - Not all targets are allowed to be used by dm=, as pointed previously, there are some risks in creating a mapped device without some validation from userspace (see documentation from the patch listing which targets are allowed). - Instead of using a simple singly linked list (for devices and tables), use the struct list_head. This
[PATCH] linux/kernel.h: fix reST markup warning for roundup
From: Randy Dunlap
Fix reST inline markup warning:
../include/linux/kernel.h:137: WARNING: Inline interpreted text or phrase
reference start-string without end-string.
Signed-off-by: Randy Dunlap
---
include/linux/kernel.h |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- linux-next-20190218.orig/include/linux/kernel.h
+++ linux-next-20190218/include/linux/kernel.h
@@ -108,7 +108,7 @@
* Rounds @x up to next multiple of @y. If @y will always be a power
* of 2, consider using the faster round_up().
*
- * The `const' here prevents gcc-3.3 from calling __divdi3
+ * The ``const`` here prevents gcc-3.3 from calling __divdi3
*/
#define roundup(x, y) (\
{ \
[PATCH 04/11] cpu topology: export die_id
Export die_id in cpu topology, for the benefit of hardware that
has multiple die per package. die_id is quite similar to core_id,
in that it holds multiple CPUs, but is inside a package.
This needed by topology-aware user-space. In particular,
an application, such as turbostat(8), which needs to know
the actual scope of "package-scope" MSRs.
Signed-off-by: Len Brown
Cc: linux-doc@vger.kernel.org
Signed-off-by: Len Brown
---
Documentation/cputopology.txt | 10 --
arch/x86/include/asm/topology.h | 1 +
drivers/base/topology.c | 4
include/linux/topology.h| 3 +++
4 files changed, 16 insertions(+), 2 deletions(-)
diff --git a/Documentation/cputopology.txt b/Documentation/cputopology.txt
index 2698da7e4f49..287213b4517b 100644
--- a/Documentation/cputopology.txt
+++ b/Documentation/cputopology.txt
@@ -12,6 +12,12 @@ physical_package_id:
socket number, but the actual value is architecture and platform
dependent.
+die_id:
+
+ the CPU die ID of cpuX. Typically it is the hardware platform's
+ identifier (rather than the kernel's). The actual value is
+ architecture and platform dependent.
+
core_id:
the CPU core ID of cpuX. Typically it is the hardware platform's
@@ -43,12 +49,12 @@ thread_siblings_list:
core_siblings:
internal kernel map of cpuX's hardware threads within the same
- physical_package_id.
+ die_id.
core_siblings_list:
human-readable list of cpuX's hardware threads within the same
- physical_package_id.
+ die_id.
book_siblings:
diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h
index 453cf38a1c33..281be6bbc80d 100644
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -106,6 +106,7 @@ extern const struct cpumask *cpu_coregroup_mask(int cpu);
#define topology_logical_package_id(cpu) (cpu_data(cpu).logical_proc_id)
#define topology_physical_package_id(cpu) (cpu_data(cpu).phys_proc_id)
+#define topology_die_id(cpu) (cpu_data(cpu).cpu_die_id)
#define topology_core_id(cpu) (cpu_data(cpu).cpu_core_id)
#ifdef CONFIG_SMP
diff --git a/drivers/base/topology.c b/drivers/base/topology.c
index 5fd9f167ecc1..50352cf96f85 100644
--- a/drivers/base/topology.c
+++ b/drivers/base/topology.c
@@ -43,6 +43,9 @@ static ssize_t name##_list_show(struct device *dev,
\
define_id_show_func(physical_package_id);
static DEVICE_ATTR_RO(physical_package_id);
+define_id_show_func(die_id);
+static DEVICE_ATTR_RO(die_id);
+
define_id_show_func(core_id);
static DEVICE_ATTR_RO(core_id);
@@ -72,6 +75,7 @@ static DEVICE_ATTR_RO(drawer_siblings_list);
static struct attribute *default_attrs[] = {
&dev_attr_physical_package_id.attr,
+ &dev_attr_die_id.attr,
&dev_attr_core_id.attr,
&dev_attr_thread_siblings.attr,
&dev_attr_thread_siblings_list.attr,
diff --git a/include/linux/topology.h b/include/linux/topology.h
index cb0775e1ee4b..5cc8595dd0e4 100644
--- a/include/linux/topology.h
+++ b/include/linux/topology.h
@@ -184,6 +184,9 @@ static inline int cpu_to_mem(int cpu)
#ifndef topology_physical_package_id
#define topology_physical_package_id(cpu) ((void)(cpu), -1)
#endif
+#ifndef topology_die_id
+#define topology_die_id(cpu) ((void)(cpu), -1)
+#endif
#ifndef topology_core_id
#define topology_core_id(cpu) ((void)(cpu), 0)
#endif
--
2.18.0-rc0
[PATCH 01/11] x86 topology: fix doc typo
reflect actual cpuinfo_x86 field name: s/logical_id/logical_proc_id/ Signed-off-by: Len Brown Cc: linux-doc@vger.kernel.org Signed-off-by: Len Brown --- Documentation/x86/topology.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/x86/topology.txt b/Documentation/x86/topology.txt index 2953e3ec9a02..06b3cdbc4048 100644 --- a/Documentation/x86/topology.txt +++ b/Documentation/x86/topology.txt @@ -51,7 +51,7 @@ The topology of a system is described in the units of: The physical ID of the package. This information is retrieved via CPUID and deduced from the APIC IDs of the cores in the package. - - cpuinfo_x86.logical_id: + - cpuinfo_x86.logical_proc_id: The logical ID of the package. As we do not trust BIOSes to enumerate the packages in a consistent way, we introduced the concept of logical package -- 2.18.0-rc0
[PATCH 05/11] x86 topology: export die_siblings
like core_siblings, except it shows which die are in the same package.
This is needed for lscpu(1) to correctly display die topology.
Signed-off-by: Len Brown
Cc: linux-doc@vger.kernel.org
Signed-off-by: Len Brown
---
Documentation/cputopology.txt | 10 ++
arch/x86/include/asm/smp.h | 1 +
arch/x86/include/asm/topology.h | 1 +
arch/x86/kernel/smpboot.c | 20
arch/x86/xen/smp_pv.c | 1 +
drivers/base/topology.c | 6 ++
include/linux/topology.h| 3 +++
7 files changed, 42 insertions(+)
diff --git a/Documentation/cputopology.txt b/Documentation/cputopology.txt
index 287213b4517b..7dd2ae3df233 100644
--- a/Documentation/cputopology.txt
+++ b/Documentation/cputopology.txt
@@ -56,6 +56,16 @@ core_siblings_list:
human-readable list of cpuX's hardware threads within the same
die_id.
+die_siblings:
+
+ internal kernel map of cpuX's hardware threads within the same
+ physical_package_id.
+
+die_siblings_list:
+
+ human-readable list of cpuX's hardware threads within the same
+ physical_package_id.
+
book_siblings:
internal kernel map of cpuX's hardware threads within the same
diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h
index 2e95b6c1bca3..39266d193597 100644
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -23,6 +23,7 @@ extern unsigned int num_processors;
DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_map);
+DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_die_map);
/* cpus sharing the last level cache: */
DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map);
DECLARE_PER_CPU_READ_MOSTLY(u16, cpu_llc_id);
diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h
index 281be6bbc80d..a52a572147ba 100644
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -110,6 +110,7 @@ extern const struct cpumask *cpu_coregroup_mask(int cpu);
#define topology_core_id(cpu) (cpu_data(cpu).cpu_core_id)
#ifdef CONFIG_SMP
+#define topology_die_cpumask(cpu) (per_cpu(cpu_die_map, cpu))
#define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu))
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 4250a87f57db..42d37e4a1918 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -90,6 +90,10 @@ EXPORT_PER_CPU_SYMBOL(cpu_sibling_map);
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_core_map);
EXPORT_PER_CPU_SYMBOL(cpu_core_map);
+/* representing HT and core and die siblings of each logical CPU */
+DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_die_map);
+EXPORT_PER_CPU_SYMBOL(cpu_die_map);
+
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map);
/* Per CPU bogomips and other parameters */
@@ -461,6 +465,12 @@ static bool match_llc(struct cpuinfo_x86 *c, struct
cpuinfo_x86 *o)
* multicore group inside a NUMA node. If this happens, we will
* discard the MC level of the topology later.
*/
+static bool match_pkg(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
+{
+ if (c->phys_proc_id == o->phys_proc_id)
+ return true;
+ return false;
+}
static bool match_die(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o)
{
if (c->cpu_die_id == o->cpu_die_id)
@@ -530,6 +540,7 @@ void set_cpu_sibling_map(int cpu)
cpumask_set_cpu(cpu, topology_sibling_cpumask(cpu));
cpumask_set_cpu(cpu, cpu_llc_shared_mask(cpu));
cpumask_set_cpu(cpu, topology_core_cpumask(cpu));
+ cpumask_set_cpu(cpu, topology_die_cpumask(cpu));
c->booted_cores = 1;
return;
}
@@ -576,8 +587,12 @@ void set_cpu_sibling_map(int cpu)
} else if (i != cpu && !c->booted_cores)
c->booted_cores = cpu_data(i).booted_cores;
}
+
if (match_die(c, o) && !topology_same_node(c, o))
x86_has_numa_in_package = true;
+
+ if ((i == cpu) || (has_mp && match_pkg(c, o)))
+ link_mask(topology_die_cpumask, cpu, i);
}
threads = cpumask_weight(topology_sibling_cpumask(cpu));
@@ -1173,6 +1188,7 @@ static __init void disable_smp(void)
physid_set_mask_of_physid(0, &phys_cpu_present_map);
cpumask_set_cpu(0, topology_sibling_cpumask(0));
cpumask_set_cpu(0, topology_core_cpumask(0));
+ cpumask_set_cpu(0, topology_die_cpumask(0));
}
/*
@@ -1268,6 +1284,7 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus)
for_each_possible_cpu(i) {
zalloc_cpumask_var(&per_cpu(cpu_sibling_map, i), GFP_KERNEL);
zalloc_cpumask_var(&per_cpu(cpu_core_map, i), GFP_KERNEL);
+
[PATCH 03/11] x86 topology: Add CPUID.1F multi-die/package support
Some new systems have multiple software-visible die within each package.
The new CPUID.1F leaf can enumerate this multi-die/package topology.
CPUID.1F a super-set of the CPUID.B "Extended Toplogy Leaf",
and a common updated routine can parse either leaf.
Legacy systems without CPUID.1F, and systems without multi-die/package
hardware, will see no functional change from this patch series.
Multi-die/package systems will use CPUID.B before this patch,
and CPUID.1F after this patch. In the CPUID.B case, all die appear
(incorrectly) to software as individual packages. In the CPUID.1F case,
the package id's reflect reality, and die_id's become meaningful.
Subsequent patches in this series update the kernel to be multi-die aware.
In particular, some software needs to know the difference between
a die-scope MSR and a package-scope MSR.
Signed-off-by: Len Brown
Cc: linux-doc@vger.kernel.org
Signed-off-by: Len Brown
---
Documentation/x86/topology.txt | 4 ++
arch/x86/include/asm/processor.h | 4 +-
arch/x86/kernel/cpu/topology.c | 82
arch/x86/kernel/smpboot.c| 4 +-
4 files changed, 73 insertions(+), 21 deletions(-)
diff --git a/Documentation/x86/topology.txt b/Documentation/x86/topology.txt
index 06b3cdbc4048..8107b6cfc9ea 100644
--- a/Documentation/x86/topology.txt
+++ b/Documentation/x86/topology.txt
@@ -46,6 +46,10 @@ The topology of a system is described in the units of:
The number of cores in a package. This information is retrieved via CPUID.
+ - cpuinfo_x86.x86_max_dies:
+
+The number of dies in a package. This information is retrieved via CPUID.
+
- cpuinfo_x86.phys_proc_id:
The physical ID of the package. This information is retrieved via CPUID
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index 33051436c864..f2856fe03715 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -105,7 +105,8 @@ struct cpuinfo_x86 {
int x86_power;
unsigned long loops_per_jiffy;
/* cpuid returned max cores value: */
- u16 x86_max_cores;
+ u16 x86_max_cores;
+ u16 x86_max_dies;
u16 apicid;
u16 initial_apicid;
u16 x86_clflush_size;
@@ -117,6 +118,7 @@ struct cpuinfo_x86 {
u16 logical_proc_id;
/* Core id: */
u16 cpu_core_id;
+ u16 cpu_die_id;
/* Index into per_cpu list: */
u16 cpu_index;
u32 microcode;
diff --git a/arch/x86/kernel/cpu/topology.c b/arch/x86/kernel/cpu/topology.c
index 8f6c784141d1..6dce6ee77849 100644
--- a/arch/x86/kernel/cpu/topology.c
+++ b/arch/x86/kernel/cpu/topology.c
@@ -15,33 +15,61 @@
/* leaf 0xb SMT level */
#define SMT_LEVEL 0
-/* leaf 0xb sub-leaf types */
+/* extended topology sub-leaf types */
#define INVALID_TYPE 0
#define SMT_TYPE 1
#define CORE_TYPE 2
+#define DIE_TYPE 5
#define LEAFB_SUBTYPE(ecx) (((ecx) >> 8) & 0xff)
#define BITS_SHIFT_NEXT_LEVEL(eax) ((eax) & 0x1f)
#define LEVEL_MAX_SIBLINGS(ebx)((ebx) & 0x)
-int detect_extended_topology_early(struct cpuinfo_x86 *c)
-{
#ifdef CONFIG_SMP
+/*
+ * Check if given CPUID extended toplogy "leaf" is implemented
+ */
+static int check_extended_topology_leaf(int leaf)
+{
unsigned int eax, ebx, ecx, edx;
- if (c->cpuid_level < 0xb)
+ cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+
+ if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
return -1;
- cpuid_count(0xb, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
+ return 0;
+}
+/*
+ * Return best CPUID Extended Toplogy Leaf supported
+ */
+static int detect_extended_topology_leaf(struct cpuinfo_x86 *c)
+{
+ if (c->cpuid_level >= 0x1f)
+ if (check_extended_topology_leaf(0x1f) == 0)
+ return 0x1f;
- /*
-* check if the cpuid leaf 0xb is actually implemented.
-*/
- if (ebx == 0 || (LEAFB_SUBTYPE(ecx) != SMT_TYPE))
+ if (c->cpuid_level >= 0xb)
+ if (check_extended_topology_leaf(0xb) == 0)
+ return 0xb;
+
+ return -1;
+}
+#endif
+
+int detect_extended_topology_early(struct cpuinfo_x86 *c)
+{
+#ifdef CONFIG_SMP
+ unsigned int eax, ebx, ecx, edx;
+ int leaf;
+
+ leaf = detect_extended_topology_leaf(c);
+ if (leaf < 0)
return -1;
set_cpu_cap(c, X86_FEATURE_XTOPOLOGY);
+ cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
/*
* initial apic id, which also represents 32-bit extended x2apic id.
*/
@@ -52,7 +80,7 @@ int detect_extended_topology_early(struct cpuinfo_x86 *c)
}
/*
[PATCH 02/11] topolgy: simplify cputopology.txt formatting and wording
No semantic changes. Signed-off-by: Len Brown Cc: linux-doc@vger.kernel.org Signed-off-by: Len Brown --- Documentation/cputopology.txt | 34 +- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/Documentation/cputopology.txt b/Documentation/cputopology.txt index c6e7e9196a8b..2698da7e4f49 100644 --- a/Documentation/cputopology.txt +++ b/Documentation/cputopology.txt @@ -3,76 +3,76 @@ How CPU topology info is exported via sysfs === Export CPU topology info via sysfs. Items (attributes) are similar -to /proc/cpuinfo output of some architectures: +to /proc/cpuinfo output of some architectures. They reside in +/sys/devices/system/cpu/cpuX/topology/: -1) /sys/devices/system/cpu/cpuX/topology/physical_package_id: +physical_package_id: physical package id of cpuX. Typically corresponds to a physical socket number, but the actual value is architecture and platform dependent. -2) /sys/devices/system/cpu/cpuX/topology/core_id: +core_id: the CPU core ID of cpuX. Typically it is the hardware platform's identifier (rather than the kernel's). The actual value is architecture and platform dependent. -3) /sys/devices/system/cpu/cpuX/topology/book_id: +book_id: the book ID of cpuX. Typically it is the hardware platform's identifier (rather than the kernel's). The actual value is architecture and platform dependent. -4) /sys/devices/system/cpu/cpuX/topology/drawer_id: +drawer_id: the drawer ID of cpuX. Typically it is the hardware platform's identifier (rather than the kernel's). The actual value is architecture and platform dependent. -5) /sys/devices/system/cpu/cpuX/topology/thread_siblings: +thread_siblings: internal kernel map of cpuX's hardware threads within the same core as cpuX. -6) /sys/devices/system/cpu/cpuX/topology/thread_siblings_list: +thread_siblings_list: human-readable list of cpuX's hardware threads within the same core as cpuX. -7) /sys/devices/system/cpu/cpuX/topology/core_siblings: +core_siblings: internal kernel map of cpuX's hardware threads within the same physical_package_id. -8) /sys/devices/system/cpu/cpuX/topology/core_siblings_list: +core_siblings_list: human-readable list of cpuX's hardware threads within the same physical_package_id. -9) /sys/devices/system/cpu/cpuX/topology/book_siblings: +book_siblings: internal kernel map of cpuX's hardware threads within the same book_id. -10) /sys/devices/system/cpu/cpuX/topology/book_siblings_list: +book_siblings_list: human-readable list of cpuX's hardware threads within the same book_id. -11) /sys/devices/system/cpu/cpuX/topology/drawer_siblings: +drawer_siblings: internal kernel map of cpuX's hardware threads within the same drawer_id. -12) /sys/devices/system/cpu/cpuX/topology/drawer_siblings_list: +drawer_siblings_list: human-readable list of cpuX's hardware threads within the same drawer_id. -To implement it in an architecture-neutral way, a new source file, -drivers/base/topology.c, is to export the 6 to 12 attributes. The book -and drawer related sysfs files will only be created if CONFIG_SCHED_BOOK -and CONFIG_SCHED_DRAWER are selected. +Architecture-neutral, drivers/base/topology.c, exports these attributes. +However, the book and drawer related sysfs files will only be created if +CONFIG_SCHED_BOOK and CONFIG_SCHED_DRAWER are selected, respectively. CONFIG_SCHED_BOOK and CONFIG_DRAWER are currently only used on s390, where they reflect the cpu and cache hierarchy. -- 2.18.0-rc0
Re: [PATCH v2 1/2] Provide in-kernel headers for making it easy to extend the kernel
On Fri, Feb 15, 2019 at 11:48 PM Alexei Starovoitov wrote: > > On Mon, Feb 11, 2019 at 09:35:59AM -0500, Joel Fernandes (Google) wrote: > > Introduce in-kernel headers and other artifacts which are made available > > as an archive through proc (/proc/kheaders.txz file). The extension '.txz' is not used in kernel code. '.tar.xz' is used for 'tarxz-pkg', 'perf-tarxz-src-pkg' etc. $ git grep '\.txz' $ git grep '\.tar\.xz' Documentation/admin-guide/README.rst: xz -cd linux-4.X.tar.xz | tar xvf - arch/x86/crypto/camellia-aesni-avx-asm_64.S: * http://koti.mbnet.fi/axh/crypto/camellia-BSD-1.2.0-aesni1.tar.xz crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz scripts/package/Makefile: @echo ' perf-tarxz-src-pkg - Build $(perf-tar).tar.xz source tarball' tools/testing/selftests/gen_kselftest_tar.sh: ext=".tar.xz" I prefer '.tar.xz' for consistency. BTW, have you ever looked at scripts/extract-ikconfig? You added IKHD_ST and IKHD_ED just to mimic kernel/configs.c It is currently pointless without the extracting tool, but you might think it is useful to extract headers from vmlinux or the module without mounting procfs. > > This archive makes > > it possible to build kernel modules, run eBPF programs, and other > > tracing programs that need to extend the kernel for tracing purposes > > without any dependency on the file system having headers and build > > artifacts. > > > > On Android and embedded systems, it is common to switch kernels but not > > have kernel headers available on the file system. Raw kernel headers > > also cannot be copied into the filesystem like they can be on other > > distros, due to licensing and other issues. There's no linux-headers > > package on Android. Further once a different kernel is booted, any > > headers stored on the file system will no longer be useful. By storing > > the headers as a compressed archive within the kernel, we can avoid these > > issues that have been a hindrance for a long time. > > The set looks good to me and since the main use case is building bpf progs > I can route it via bpf-next tree if there are no objections. > Masahiro, could you please ack it? Honestly, I was not tracking this thread since I did not know I was responsible for this. I just started to take a closer look, then immediately got scared. This version is not mature enough for the merge. First of all, this patch cannot be compiled out-of-tree (O= option). I do not know why 0-day bot did not catch this apparent breakage. $ make -j8 O=hoge make[1]: Entering directory '/home/masahiro/workspace/bsp/linux/hoge' GEN Makefile Using .. as source for kernel DESCEND objtool CALL../scripts/checksyscalls.sh CHK include/generated/compile.h make[2]: *** No rule to make target 'Module.symvers', needed by 'kernel/kheaders_data.txz'. Stop. make[2]: *** Waiting for unfinished jobs /home/masahiro/workspace/bsp/linux/Makefile:1043: recipe for target 'kernel' failed make[1]: *** [kernel] Error 2 make[1]: *** Waiting for unfinished jobs make[1]: Leaving directory '/home/masahiro/workspace/bsp/linux/hoge' Makefile:152: recipe for target 'sub-make' failed make: *** [sub-make] Error 2 I was able to compile it in-tree but it makes the incremental build extremely slow. (Here, the incremental build means "make" without changing any code after the full build.) Before this patch, "make -j8" took 11 sec on my machine. real 0m11.777s user 0m16.608s sys 0m5.164s After this patch, x86_64_defconfig + CONFIG_IKHEADERS_PROC=y takes 53 sec for me since kernel/kheaders_data.txz is regenerated every time even when you did not touch any source file. $ time make -j8 DESCEND objtool CALLscripts/checksyscalls.sh CHK include/generated/compile.h GEN kernel/kheaders_data.txz UPD kernel/kheaders_data.h CC kernel/kheaders.o AR kernel/built-in.a GEN .version CHK include/generated/compile.h UPD include/generated/compile.h CC init/version.o AR init/built-in.a AR built-in.a LD vmlinux.o MODPOST vmlinux.o KSYM.tmp_kallsyms1.o KSYM.tmp_kallsyms2.o LD vmlinux SORTEX vmlinux SYSMAP System.map Building modules, stage 2. CC arch/x86/boot/version.o MODPOST 17 modules VOFFSET arch/x86/boot/compressed/../voffset.h OBJCOPY arch/x86/boot/compressed/vmlinux.bin RELOCS arch/x86/boot/compressed/vmlinux.relocs CC arch/x86/boot/compressed/kaslr.o GZIParch/x86/boot/compressed/vmlinux.bin.gz CC arch/x86/boot/compressed/misc.o MKPIGGY arch/x86/boot/compressed/piggy.S AS arch
Re: [PATCH] linux/kernel.h: fix reST markup warning for roundup
On Mon, Feb 18, 2019 at 04:34:26PM -0800, Randy Dunlap wrote: > - * The `const' here prevents gcc-3.3 from calling __divdi3 > + * The ``const`` here prevents gcc-3.3 from calling __divdi3 Probably time to remove this hack ... willy@bobo:~/kernel/idrext$ grep gcc Documentation/Changes GNU C 4.6 gcc --version
Re: [PATCH v2 1/2] Provide in-kernel headers for making it easy to extend the kernel
On Tue, Feb 19, 2019 at 01:14:11PM +0900, Masahiro Yamada wrote: > > I was able to compile it in-tree > but it makes the incremental build extremely slow. > > (Here, the incremental build means > "make" without changing any code after the full build.) > > Before this patch, "make -j8" took 11 sec on my machine. > > After this patch, x86_64_defconfig + CONFIG_IKHEADERS_PROC=y > takes 53 sec for me since kernel/kheaders_data.txz is regenerated > every time even when you did not touch any source file. > > We should take time for careful review and test. > > Please give me more time for thorough review. Thanks for taking a look. All are very valid points. Incremental build is must have.
Re: [PATCH v2 1/2] Provide in-kernel headers for making it easy to extend the kernel
On Tue, Feb 19, 2019 at 01:14:11PM +0900, Masahiro Yamada wrote: > On Fri, Feb 15, 2019 at 11:48 PM Alexei Starovoitov > wrote: > > > > On Mon, Feb 11, 2019 at 09:35:59AM -0500, Joel Fernandes (Google) wrote: > > > Introduce in-kernel headers and other artifacts which are made available > > > as an archive through proc (/proc/kheaders.txz file). > > > The extension '.txz' is not used in kernel code. > > > '.tar.xz' is used for 'tarxz-pkg', 'perf-tarxz-src-pkg' etc. > > > $ git grep '\.txz' > $ git grep '\.tar\.xz' > Documentation/admin-guide/README.rst: xz -cd linux-4.X.tar.xz | tar xvf - > arch/x86/crypto/camellia-aesni-avx-asm_64.S: * > http://koti.mbnet.fi/axh/crypto/camellia-BSD-1.2.0-aesni1.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > scripts/package/Makefile: @echo ' perf-tarxz-src-pkg - Build > $(perf-tar).tar.xz source tarball' > tools/testing/selftests/gen_kselftest_tar.sh: > ext=".tar.xz" > > > > I prefer '.tar.xz' for consistency. Ok, I will change it to that. > BTW, have you ever looked at scripts/extract-ikconfig? > > You added IKHD_ST and IKHD_ED > just to mimic kernel/configs.c > > It is currently pointless without the extracting tool, > but you might think it is useful to extract headers > from vmlinux or the module without mounting procfs. I am planing add to extraction support in the future, so I prefer to leave the markers as it is for now. Hope that's Ok with you. > > > This archive makes > > > it possible to build kernel modules, run eBPF programs, and other > > > tracing programs that need to extend the kernel for tracing purposes > > > without any dependency on the file system having headers and build > > > artifacts. > > > > > > On Android and embedded systems, it is common to switch kernels but not > > > have kernel headers available on the file system. Raw kernel headers > > > also cannot be copied into the filesystem like they can be on other > > > distros, due to licensing and other issues. There's no linux-headers > > > package on Android. Further once a different kernel is booted, any > > > headers stored on the file system will no longer be useful. By storing > > > the headers as a compressed archive within the kernel, we can avoid these > > > issues that have been a hindrance for a long time. > > > > The set looks good to me and since the main use case is building bpf progs > > I can route it via bpf-next tree if there are no objections. > > Masahiro, could you please ack it? > > > Honestly, I was not tracking this thread > since I did not know I was responsible for this. > > > I just started to take a closer look, then immediately got scared. > > This version is not mature enough for the merge. > First of all, this patch cannot be compiled out-of-tree (O= option). Oh, ok. This is not how I build the kernel. So I am sorry for missing that. I will try this out-of-tree build option and fix it. > I do not know why 0-day bot did not catch this apparent breakage. > > > $ make -j8 O=hoge > make[1]: Entering directory '/home/masahiro/workspace/bsp/linux/hoge' > GEN Makefile > Using .. as source for kernel > DESCEND objtool > CALL../scripts/checksyscalls.sh > CHK include/generated/compile.h > make[2]: *** No rule to make target 'Module.symvers', needed by > 'kernel/kheaders_data.txz'. Stop. > make[2]: *** Waiting for unfinished jobs > /home/masahiro/workspace/bsp/linux/Makefile:1043: recipe for target > 'kernel' failed > make[1]: *** [kernel] Error 2 > make[1]: *** Waiting for unfinished jobs > make[1]: Leaving directory '/home/masahiro/workspace/bsp/linux/hoge' > Makefile:152: recipe for target 'sub-make' failed > make: *** [sub-make] Error 2 > > > > > > I was able to compile it in-tree > but it makes the incremental build extremely slow. > > (Here, the incremental build means > "make" without changing any code after the full build.) > > > > > Before this patch, "make -j8" took 11 sec on my machine. > > real 0m11.777s > user 0m16.608s > sys 0m5.164s > > > > After this patch, x86_64_defconfig + CONFIG_IKHEADERS_PROC=y > takes 53 sec for me since kernel/kheaders_data.txz is regenerated > every time even when you did not touch any source file. Hmm, true. Let me know look into that as well.. > $ time make -j8 > DESCEND objtool > CALLscripts/checksyscalls.sh > CHK include/generated/compile.h > GEN kernel/kheaders_data.txz > UPD kernel/kheaders_data.h > CC kernel/kheaders.o > AR kernel/built-in.a > GEN .version > CHK include/generated/compile.h > UPD include/generated/compile.h
Re: [PATCH v2 1/2] Provide in-kernel headers for making it easy to extend the kernel
On Tue, Feb 19, 2019 at 1:14 PM Masahiro Yamada wrote: > > On Fri, Feb 15, 2019 at 11:48 PM Alexei Starovoitov > wrote: > > > > On Mon, Feb 11, 2019 at 09:35:59AM -0500, Joel Fernandes (Google) wrote: > > > Introduce in-kernel headers and other artifacts which are made available > > > as an archive through proc (/proc/kheaders.txz file). > > > The extension '.txz' is not used in kernel code. > > > '.tar.xz' is used for 'tarxz-pkg', 'perf-tarxz-src-pkg' etc. > > > $ git grep '\.txz' > $ git grep '\.tar\.xz' > Documentation/admin-guide/README.rst: xz -cd linux-4.X.tar.xz | tar xvf - > arch/x86/crypto/camellia-aesni-avx-asm_64.S: * > http://koti.mbnet.fi/axh/crypto/camellia-BSD-1.2.0-aesni1.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > crypto/testmgr.h: * https://bench.cr.yp.to/supercop/supercop-20170228.tar.xz > scripts/package/Makefile: @echo ' perf-tarxz-src-pkg - Build > $(perf-tar).tar.xz source tarball' > tools/testing/selftests/gen_kselftest_tar.sh: > ext=".tar.xz" > > > > I prefer '.tar.xz' for consistency. > > > > > > > BTW, have you ever looked at scripts/extract-ikconfig? > > You added IKHD_ST and IKHD_ED > just to mimic kernel/configs.c > > It is currently pointless without the extracting tool, > but you might think it is useful to extract headers > from vmlinux or the module without mounting procfs. > > > > > > > This archive makes > > > it possible to build kernel modules, run eBPF programs, and other > > > tracing programs that need to extend the kernel for tracing purposes > > > without any dependency on the file system having headers and build > > > artifacts. > > > > > > On Android and embedded systems, it is common to switch kernels but not > > > have kernel headers available on the file system. Raw kernel headers > > > also cannot be copied into the filesystem like they can be on other > > > distros, due to licensing and other issues. There's no linux-headers > > > package on Android. Further once a different kernel is booted, any > > > headers stored on the file system will no longer be useful. By storing > > > the headers as a compressed archive within the kernel, we can avoid these > > > issues that have been a hindrance for a long time. > > > > The set looks good to me and since the main use case is building bpf progs > > I can route it via bpf-next tree if there are no objections. > > Masahiro, could you please ack it? > > > Honestly, I was not tracking this thread > since I did not know I was responsible for this. > > > I just started to take a closer look, then immediately got scared. > > This version is not mature enough for the merge. > > > > First of all, this patch cannot be compiled out-of-tree (O= option). > > > I do not know why 0-day bot did not catch this apparent breakage. > > > $ make -j8 O=hoge > make[1]: Entering directory '/home/masahiro/workspace/bsp/linux/hoge' > GEN Makefile > Using .. as source for kernel > DESCEND objtool > CALL../scripts/checksyscalls.sh > CHK include/generated/compile.h > make[2]: *** No rule to make target 'Module.symvers', needed by > 'kernel/kheaders_data.txz'. Stop. > make[2]: *** Waiting for unfinished jobs > /home/masahiro/workspace/bsp/linux/Makefile:1043: recipe for target > 'kernel' failed > make[1]: *** [kernel] Error 2 > make[1]: *** Waiting for unfinished jobs > make[1]: Leaving directory '/home/masahiro/workspace/bsp/linux/hoge' > Makefile:152: recipe for target 'sub-make' failed > make: *** [sub-make] Error 2 I saw this build error for in-tree building as well. We cannot build this from a pristine source tree. For example, I observed the build error in the following procedure. $ make mrproper $ make defconfig Set CONFIG_IKHEADERS_PROC=y $ make Module.symvers is generated in the modpost stage (the very last stage of build). But, vmlinux depends on kernel/kheaders_data.txz, which includes Module.symvers. So, this is not so simple since it is a circular dependency... -- Best Regards Masahiro Yamada
Re: [PATCH v2 1/2] Provide in-kernel headers for making it easy to extend the kernel
On Tue, Feb 19, 2019 at 01:42:13PM +0900, Masahiro Yamada wrote: [..] > > > > This archive makes > > > > it possible to build kernel modules, run eBPF programs, and other > > > > tracing programs that need to extend the kernel for tracing purposes > > > > without any dependency on the file system having headers and build > > > > artifacts. > > > > > > > > On Android and embedded systems, it is common to switch kernels but not > > > > have kernel headers available on the file system. Raw kernel headers > > > > also cannot be copied into the filesystem like they can be on other > > > > distros, due to licensing and other issues. There's no linux-headers > > > > package on Android. Further once a different kernel is booted, any > > > > headers stored on the file system will no longer be useful. By storing > > > > the headers as a compressed archive within the kernel, we can avoid > > > > these > > > > issues that have been a hindrance for a long time. > > > > > > The set looks good to me and since the main use case is building bpf progs > > > I can route it via bpf-next tree if there are no objections. > > > Masahiro, could you please ack it? > > > > > > Honestly, I was not tracking this thread > > since I did not know I was responsible for this. > > > > > > I just started to take a closer look, then immediately got scared. > > > > This version is not mature enough for the merge. > > > > > > > > First of all, this patch cannot be compiled out-of-tree (O= option). > > > > > > I do not know why 0-day bot did not catch this apparent breakage. > > > > > > $ make -j8 O=hoge > > make[1]: Entering directory '/home/masahiro/workspace/bsp/linux/hoge' > > GEN Makefile > > Using .. as source for kernel > > DESCEND objtool > > CALL../scripts/checksyscalls.sh > > CHK include/generated/compile.h > > make[2]: *** No rule to make target 'Module.symvers', needed by > > 'kernel/kheaders_data.txz'. Stop. > > make[2]: *** Waiting for unfinished jobs > > /home/masahiro/workspace/bsp/linux/Makefile:1043: recipe for target > > 'kernel' failed > > make[1]: *** [kernel] Error 2 > > make[1]: *** Waiting for unfinished jobs > > make[1]: Leaving directory '/home/masahiro/workspace/bsp/linux/hoge' > > Makefile:152: recipe for target 'sub-make' failed > > make: *** [sub-make] Error 2 > > > > > I saw this build error for in-tree building as well. > > We cannot build this from a pristine source tree. > > For example, I observed the build error in the following procedure. > > $ make mrproper > $ make defconfig > Set CONFIG_IKHEADERS_PROC=y > $ make > > > > > Module.symvers is generated in the modpost stage > (the very last stage of build). > > But, vmlinux depends on kernel/kheaders_data.txz, > which includes Module.symvers. > > So, this is not so simple since it is a circular dependency... I guess I was not building a pristine tree either and missed this circular dependency :-/ . Any ideas on how we can fix the Module.symvers issue? One idea is to reserve the space in the binaries, but only populate the space reserved in the binary *after* the modpost stage, once the archive is ready.. thanks, - Joel
