On 3/10/20 1:07 PM, Martin Liška wrote:
On 3/10/20 12:24 PM, Richard Biener wrote:
Not sure how symtab is encoded right now but we also could have
Ok, right now I don't see symtab entry much extensible.
But what am I suggesting is to parse LTO bytecode version and then
process conditional parsing of lto_symtab section.
Thoughts?
Martin
So as H.J. correctly pointed I can't add new symbol_type into ld_plugin_symbol
struct.
It would make ABI change as correctly identified by abidiff:
abidiff /tmp/before.o /tmp/after.o
Functions changes summary: 0 Removed, 1 Changed, 0 Added function
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
Function symbols changes summary: 0 Removed, 0 Added function symbol not
referenced by debug info
Variable symbols changes summary: 0 Removed, 1 Added variable symbol not
referenced by debug info
1 function with some indirect sub-type change:
[C]'function ld_plugin_status onload(ld_plugin_tv*)' at lto-plugin.c:1275:1
has some indirect sub-type changes:
parameter 1 of type 'ld_plugin_tv*' has sub-type changes:
in pointed to type 'struct ld_plugin_tv' at plugin-api.h:451:1:
type size hasn't changed
1 data member changes (1 filtered):
type of 'union {int tv_val; const char* tv_string;
ld_plugin_register_claim_file tv_register_claim_file;
ld_plugin_register_all_symbols_read tv_register_all_symbols_read;
ld_plugin_register_cleanup tv_register_cleanup; ld_plugin_add_symbols
tv_add_symbols; ld_plugin_get_symbols tv_get_symbols; ld_plugin_add_input_file
tv_add_input_file; ld_plugin_message tv_message; ld_plugin_get_input_file
tv_get_input_file; ld_plugin_get_view tv_get_view; ld_plugin_release_input_file
tv_release_input_file; ld_plugin_add_input_library tv_add_input_library;
ld_plugin_set_extra_library_path tv_set_extra_library_path;
ld_plugin_get_input_section_count tv_get_input_section_count;
ld_plugin_get_input_section_type tv_get_input_section_type;
ld_plugin_get_input_section_name tv_get_input_section_name;
ld_plugin_get_input_section_contents tv_get_input_section_contents;
ld_plugin_update_section_order tv_update_section_order;
ld_plugin_allow_section_ordering tv_allow_section_ordering;
ld_plugin_allow_unique_segment_for_sections
tv_allow_unique_segment_for_sections; ld_plugin_unique_segment_for_sections
tv_unique_segment_for_sections; ld_plugin_get_input_section_alignment
tv_get_input_section_alignment; ld_plugin_get_input_section_size
tv_get_input_section_size; ld_plugin_register_new_input tv_register_new_input;
ld_plugin_get_wrap_symbols tv_get_wrap_symbols;} ld_plugin_tv::tv_u' changed:
type size hasn't changed
1 data member changes (1 filtered):
type of 'ld_plugin_add_symbols tv_add_symbols' changed:
underlying type 'enum ld_plugin_status (void*, int, const
ld_plugin_symbol*)*' changed:
in pointed to type 'function type enum ld_plugin_status (void*,
int, const ld_plugin_symbol*)':
parameter 3 of type 'const ld_plugin_symbol*' has sub-type
changes:
in pointed to type 'const ld_plugin_symbol':
in unqualified underlying type 'struct ld_plugin_symbol'
at plugin-api.h:86:1:
type size changed from 256 to 288 (in bits)
1 data member insertion:
'int ld_plugin_symbol::symbol_type', at offset 256
(in bits) at plugin-api.h:95:1
So that I need to come up with ld_plugin_symbol_v2. It brings more challenges:
one has 2 parallel symbol
tables:
struct plugin_symtab
{
...
struct ld_plugin_symbol_v2 *syms;
struct ld_plugin_symbol *syms_v1;
...
};
and the information of these should by aligned.
The patch can survive lto.exp and I would like to ask H.J. to write bintuils
counterpart that will
utilize the new LDPT_GET_SYMBOLS_V4, LDPT_ADD_SYMBOLS_V2.
Thoughts?
Martin
>From 1271e9cc487847c1aa07e5bc0470e8c5b71900e9 Mon Sep 17 00:00:00 2001
From: Martin Liska <mli...@suse.cz>
Date: Fri, 6 Mar 2020 18:09:35 +0100
Subject: [PATCH] API extension for binutils (type of symbols).
gcc/ChangeLog:
2020-03-11 Martin Liska <mli...@suse.cz>
* lto-streamer-out.c (write_symbol): Stream type
of symbol (function, variable - bss/data).
include/ChangeLog:
2020-03-11 Martin Liska <mli...@suse.cz>
* lto-symtab.h (enum gcc_plugin_symbol_type):
New.
* plugin-api.h (struct ld_plugin_symbol_v2): New
ld_plugin_symbol structure with added symbol_type.
(enum ld_plugin_symbol_type): New.
(ld_plugin_add_symbols_v2): New.
(ld_plugin_get_symbols_v2): New.
(ld_plugin_tag): Add LDPT_GET_SYMBOLS_V4
and LDPT_ADD_SYMBOLS_V2.
(struct ld_plugin_tv): Add tv_add_symbols_v2
and tv_get_symbols_v4.
lto-plugin/ChangeLog:
2020-03-11 Martin Liska <mli...@suse.cz>
* lto-plugin.c (LTO_LTO_PREFIX): New.
(LTO_LTO_PREFIX_LEN): Likewise.
(struct plugin_symtab): Make syms of new
type ld_plugin_symbol_v2 and add syms_v1
for older API hooks.
(struct lto_section): New.
(parse_table_entry): Change type to ld_plugin_symbol_v2
and conditionally (based on LTO bytecode) parse symbol
type.
(translate): Change type to ld_plugin_symbol_v2.
(fill_up_older_syms): New function.
(free_1): Release also syms_v1.
(write_resolution): Support new version of get_symbols.
(eq_sym): Change type to ld_plugin_symbol_v2.
(hash_sym): Likewise.
(symbol_strength): Likewise.
(resolve_conflicts): Likewise.
(process_symtab): Parse LTO_LTO_PREFIX section.
(claim_file_handler): Support new version of add_symbols.
(onload): Fill up new hooks.
---
gcc/lto-streamer-out.c | 14 ++++
include/lto-symtab.h | 8 +++
include/plugin-api.h | 42 +++++++++++-
lto-plugin/lto-plugin.c | 148 ++++++++++++++++++++++++++++++++++------
4 files changed, 190 insertions(+), 22 deletions(-)
diff --git a/gcc/lto-streamer-out.c b/gcc/lto-streamer-out.c
index cea5e71cffb..ead606eb665 100644
--- a/gcc/lto-streamer-out.c
+++ b/gcc/lto-streamer-out.c
@@ -45,6 +45,7 @@ along with GCC; see the file COPYING3. If not see
#include "print-tree.h"
#include "tree-dfa.h"
#include "file-prefix-map.h" /* remap_debug_filename() */
+#include "output.h"
static void lto_write_tree (struct output_block*, tree, bool);
@@ -2773,6 +2774,19 @@ write_symbol (struct streamer_tree_cache_d *cache,
lto_write_data (&c, 1);
c = (unsigned char) visibility;
lto_write_data (&c, 1);
+
+ gcc_plugin_symbol_type st;
+ if (TREE_CODE (t) == VAR_DECL)
+ {
+ section *s = get_variable_section (t, false);
+ st = (s->common.flags & SECTION_BSS
+ ? GCCST_VARIABLE_BSS : GCCST_VARIABLE_DATA);
+ }
+ else
+ st = GCCST_FUNCTION;
+
+ c = (unsigned char) st;
+ lto_write_data (&c, 1);
lto_write_data (&size, 8);
lto_write_data (&slot_num, 4);
}
diff --git a/include/lto-symtab.h b/include/lto-symtab.h
index 0ce0de10121..901bc3585c2 100644
--- a/include/lto-symtab.h
+++ b/include/lto-symtab.h
@@ -38,4 +38,12 @@ enum gcc_plugin_symbol_visibility
GCCPV_HIDDEN
};
+enum gcc_plugin_symbol_type
+{
+ GCCST_UNKNOWN,
+ GCCST_FUNCTION,
+ GCCST_VARIABLE_DATA,
+ GCCST_VARIABLE_BSS
+};
+
#endif /* GCC_LTO_SYMTAB_H */
diff --git a/include/plugin-api.h b/include/plugin-api.h
index 09e1202df07..123c4a58a6d 100644
--- a/include/plugin-api.h
+++ b/include/plugin-api.h
@@ -94,6 +94,21 @@ struct ld_plugin_symbol
int resolution;
};
+/* A symbol belonging to an input file managed by the plugin library
+ (version 2). */
+
+struct ld_plugin_symbol_v2
+{
+ char *name;
+ char *version;
+ int def;
+ int visibility;
+ uint64_t size;
+ char *comdat_key;
+ int resolution;
+ int symbol_type;
+};
+
/* An object's section. */
struct ld_plugin_section
@@ -123,6 +138,16 @@ enum ld_plugin_symbol_visibility
LDPV_HIDDEN
};
+/* The type of the symbol. */
+
+enum ld_plugin_symbol_type
+{
+ LDST_UNKNOWN,
+ LDST_FUNCTION,
+ LDST_VARIABLE_DATA,
+ LDST_VARIABLE_BSS
+};
+
/* How a symbol is resolved. */
enum ld_plugin_symbol_resolution
@@ -210,6 +235,12 @@ enum ld_plugin_status
(*ld_plugin_add_symbols) (void *handle, int nsyms,
const struct ld_plugin_symbol *syms);
+typedef
+enum ld_plugin_status
+(*ld_plugin_add_symbols_v2) (void *handle, int nsyms,
+ const struct ld_plugin_symbol_v2 *syms);
+
+
/* The linker's interface for getting the input file information with
an open (possibly re-opened) file descriptor. */
@@ -235,6 +266,11 @@ enum ld_plugin_status
(*ld_plugin_get_symbols) (const void *handle, int nsyms,
struct ld_plugin_symbol *syms);
+typedef
+enum ld_plugin_status
+(*ld_plugin_get_symbols_v2) (const void *handle, int nsyms,
+ struct ld_plugin_symbol_v2 *syms);
+
/* The linker's interface for adding a compiled input file. */
typedef
@@ -431,7 +467,9 @@ enum ld_plugin_tag
LDPT_GET_INPUT_SECTION_ALIGNMENT = 29,
LDPT_GET_INPUT_SECTION_SIZE = 30,
LDPT_REGISTER_NEW_INPUT_HOOK = 31,
- LDPT_GET_WRAP_SYMBOLS = 32
+ LDPT_GET_WRAP_SYMBOLS = 32,
+ LDPT_GET_SYMBOLS_V4 = 33,
+ LDPT_ADD_SYMBOLS_V2 = 34,
};
/* The plugin transfer vector. */
@@ -447,7 +485,9 @@ struct ld_plugin_tv
ld_plugin_register_all_symbols_read tv_register_all_symbols_read;
ld_plugin_register_cleanup tv_register_cleanup;
ld_plugin_add_symbols tv_add_symbols;
+ ld_plugin_add_symbols_v2 tv_add_symbols_v2;
ld_plugin_get_symbols tv_get_symbols;
+ ld_plugin_get_symbols_v2 tv_get_symbols_v4;
ld_plugin_add_input_file tv_add_input_file;
ld_plugin_message tv_message;
ld_plugin_get_input_file tv_get_input_file;
diff --git a/lto-plugin/lto-plugin.c b/lto-plugin/lto-plugin.c
index c307fc871bf..b1b4516ec1b 100644
--- a/lto-plugin/lto-plugin.c
+++ b/lto-plugin/lto-plugin.c
@@ -90,6 +90,8 @@ along with this program; see the file COPYING3. If not see
#define LTO_SECTION_PREFIX ".gnu.lto_.symtab"
#define LTO_SECTION_PREFIX_LEN (sizeof (LTO_SECTION_PREFIX) - 1)
+#define LTO_LTO_PREFIX ".gnu.lto_.lto"
+#define LTO_LTO_PREFIX_LEN (sizeof (LTO_LTO_PREFIX) - 1)
#define OFFLOAD_SECTION ".gnu.offload_lto_.opts"
#define OFFLOAD_SECTION_LEN (sizeof (OFFLOAD_SECTION) - 1)
@@ -109,7 +111,8 @@ struct plugin_symtab
{
int nsyms;
struct sym_aux *aux;
- struct ld_plugin_symbol *syms;
+ struct ld_plugin_symbol_v2 *syms;
+ struct ld_plugin_symbol *syms_v1;
unsigned long long id;
};
@@ -155,11 +158,13 @@ static char *arguments_file_name;
static ld_plugin_register_claim_file register_claim_file;
static ld_plugin_register_all_symbols_read register_all_symbols_read;
static ld_plugin_get_symbols get_symbols, get_symbols_v2;
+static ld_plugin_get_symbols_v2 get_symbols_v4;
static ld_plugin_register_cleanup register_cleanup;
static ld_plugin_add_input_file add_input_file;
static ld_plugin_add_input_library add_input_library;
static ld_plugin_message message;
static ld_plugin_add_symbols add_symbols;
+static ld_plugin_add_symbols_v2 add_symbols_v2;
static struct plugin_file_info *claimed_files = NULL;
static unsigned int num_claimed_files = 0;
@@ -221,6 +226,20 @@ check_1 (int gate, enum ld_plugin_level level, const char *text)
}
}
+/* Structure that represents LTO ELF section with information
+ about the format. */
+
+struct lto_section
+ {
+ int16_t major_version;
+ int16_t minor_version;
+ unsigned char slim_object: 1;
+ unsigned char compression: 4;
+ int32_t reserved0: 27;
+};
+
+struct lto_section lto_header;
+
/* This little wrapper allows check to be called with a non-integer
first argument, such as a pointer that must be non-NULL. We can't
use c99 bool type to coerce it into range, so we explicitly test. */
@@ -231,7 +250,7 @@ check_1 (int gate, enum ld_plugin_level level, const char *text)
Returns the address of the next entry. */
static char *
-parse_table_entry (char *p, struct ld_plugin_symbol *entry,
+parse_table_entry (char *p, struct ld_plugin_symbol_v2 *entry,
struct sym_aux *aux)
{
unsigned char t;
@@ -252,6 +271,14 @@ parse_table_entry (char *p, struct ld_plugin_symbol *entry,
LDPV_HIDDEN
};
+ enum ld_plugin_symbol_type symbol_types[] =
+ {
+ LDST_UNKNOWN,
+ LDST_FUNCTION,
+ LDST_VARIABLE_DATA,
+ LDST_VARIABLE_BSS
+ };
+
switch (sym_style)
{
case ss_win32:
@@ -296,6 +323,15 @@ parse_table_entry (char *p, struct ld_plugin_symbol *entry,
entry->visibility = translate_visibility[t];
p++;
+ /* Symbol type was added in GCC 10.1 (LTO version 9.0). */
+ if (lto_header.major_version >= 9)
+ {
+ t = *p;
+ check (t <= 3, LDPL_FATAL, "invalid symbol type found");
+ entry->symbol_type = symbol_types[t];
+ p++;
+ }
+
memcpy (&entry->size, p, sizeof (uint64_t));
p += 8;
@@ -316,14 +352,14 @@ static void
translate (char *data, char *end, struct plugin_symtab *out)
{
struct sym_aux *aux;
- struct ld_plugin_symbol *syms = NULL;
+ struct ld_plugin_symbol_v2 *syms = NULL;
int n, len;
/* This overestimates the output buffer sizes, but at least
the algorithm is O(1) now. */
len = (end - data)/8 + out->nsyms + 1;
- syms = xrealloc (out->syms, len * sizeof (struct ld_plugin_symbol));
+ syms = xrealloc (out->syms, len * sizeof (struct ld_plugin_symbol_v2));
aux = xrealloc (out->aux, len * sizeof (struct sym_aux));
for (n = out->nsyms; data < end; n++)
@@ -339,6 +375,33 @@ translate (char *data, char *end, struct plugin_symtab *out)
out->aux = aux;
}
+/* Copy information from syms to syms_v1 for older hooks. */
+
+static void
+fill_up_older_syms (struct plugin_symtab *out)
+{
+ if (out->syms_v1)
+ return;
+
+ struct ld_plugin_symbol *syms
+ = xrealloc (out->syms_v1, out->nsyms * sizeof (struct ld_plugin_symbol));
+ for (unsigned i = 0; i < out->nsyms; i++)
+ {
+ struct ld_plugin_symbol *old = &syms[i];
+ struct ld_plugin_symbol_v2 *new = &out->syms[i];
+
+ old->name = new->name;
+ old->version = new->version;
+ old->def = new->def;
+ old->visibility = new->visibility;
+ old->size = new->size;
+ old->comdat_key = new->comdat_key;
+ old->resolution = new->resolution;
+ }
+
+ out->syms_v1 = syms;
+}
+
/* Free all memory that is no longer needed after writing the symbol
resolution. */
@@ -353,12 +416,18 @@ free_1 (struct plugin_file_info *files, unsigned num_files)
unsigned int j;
for (j = 0; j < symtab->nsyms; j++)
{
- struct ld_plugin_symbol *s = &symtab->syms[j];
+ struct ld_plugin_symbol_v2 *s = &symtab->syms[j];
free (s->name);
free (s->comdat_key);
}
free (symtab->syms);
symtab->syms = NULL;
+
+ if (symtab->syms_v1 != NULL)
+ {
+ free (symtab->syms_v1);
+ symtab->syms_v1 = NULL;
+ }
}
}
@@ -478,7 +547,7 @@ free_symtab (struct plugin_symtab *symtab)
static void
write_resolution (void)
{
- unsigned int i;
+ unsigned int i, j;
FILE *f;
check (resolution_file, LDPL_FATAL, "resolution file not specified");
@@ -491,14 +560,23 @@ write_resolution (void)
{
struct plugin_file_info *info = &claimed_files[i];
struct plugin_symtab *symtab = &info->symtab;
- struct ld_plugin_symbol *syms = symtab->syms;
/* Version 2 of API supports IRONLY_EXP resolution that is
accepted by GCC-4.7 and newer. */
- if (get_symbols_v2)
- get_symbols_v2 (info->handle, symtab->nsyms, syms);
+ if (get_symbols_v4)
+ get_symbols_v4 (info->handle, symtab->nsyms, symtab->syms);
else
- get_symbols (info->handle, symtab->nsyms, syms);
+ {
+ fill_up_older_syms (symtab);
+ if (get_symbols_v2)
+ get_symbols_v2 (info->handle, symtab->nsyms, symtab->syms_v1);
+ else
+ get_symbols (info->handle, symtab->nsyms, symtab->syms_v1);
+
+ /* Update resolution from syms_v1 to syms. */
+ for (j = 0; j < symtab->nsyms; j++)
+ symtab->syms[j].resolution = symtab->syms_v1[j].resolution;
+ }
finish_conflict_resolution (symtab, &info->conflicts);
@@ -827,8 +905,8 @@ cleanup_handler (void)
static int eq_sym (const void *a, const void *b)
{
- const struct ld_plugin_symbol *as = (const struct ld_plugin_symbol *)a;
- const struct ld_plugin_symbol *bs = (const struct ld_plugin_symbol *)b;
+ const struct ld_plugin_symbol_v2 *as = (const struct ld_plugin_symbol_v2 *)a;
+ const struct ld_plugin_symbol_v2 *bs = (const struct ld_plugin_symbol_v2 *)b;
return !strcmp (as->name, bs->name);
}
@@ -837,14 +915,14 @@ static int eq_sym (const void *a, const void *b)
static hashval_t hash_sym (const void *a)
{
- const struct ld_plugin_symbol *as = (const struct ld_plugin_symbol *)a;
+ const struct ld_plugin_symbol_v2 *as = (const struct ld_plugin_symbol_v2 *)a;
return htab_hash_string (as->name);
}
/* Determine how strong a symbol is */
-static int symbol_strength (struct ld_plugin_symbol *s)
+static int symbol_strength (struct ld_plugin_symbol_v2 *s)
{
switch (s->def)
{
@@ -885,14 +963,14 @@ resolve_conflicts (struct plugin_symtab *t, struct plugin_symtab *conflicts)
int outlen;
outlen = t->nsyms;
- conflicts->syms = xmalloc (sizeof (struct ld_plugin_symbol) * outlen);
+ conflicts->syms = xmalloc (sizeof (struct ld_plugin_symbol_v2) * outlen);
conflicts->aux = xmalloc (sizeof (struct sym_aux) * outlen);
/* Move all duplicate symbols into the auxiliary conflicts table. */
out = 0;
for (i = 0; i < t->nsyms; i++)
{
- struct ld_plugin_symbol *s = &t->syms[i];
+ struct ld_plugin_symbol_v2 *s = &t->syms[i];
struct sym_aux *aux = &t->aux[i];
void **slot;
@@ -900,13 +978,14 @@ resolve_conflicts (struct plugin_symtab *t, struct plugin_symtab *conflicts)
if (*slot != NULL)
{
int cnf;
- struct ld_plugin_symbol *orig = (struct ld_plugin_symbol *)*slot;
+ struct ld_plugin_symbol_v2
+ *orig = (struct ld_plugin_symbol_v2 *)*slot;
struct sym_aux *orig_aux = &t->aux[orig - t->syms];
/* Always let the linker resolve the strongest symbol */
if (symbol_strength (orig) < symbol_strength (s))
{
- SWAP (struct ld_plugin_symbol, *orig, *s);
+ SWAP (struct ld_plugin_symbol_v2, *orig, *s);
SWAP (uint32_t, orig_aux->slot, aux->slot);
SWAP (unsigned long long, orig_aux->id, aux->id);
/* Don't swap conflict chain pointer */
@@ -951,7 +1030,20 @@ process_symtab (void *data, const char *name, off_t offset, off_t length)
{
struct plugin_objfile *obj = (struct plugin_objfile *)data;
char *s;
- char *secdatastart, *secdata;
+ char *secdatastart = NULL, *secdata;
+
+ if (strncmp (name, LTO_LTO_PREFIX, LTO_LTO_PREFIX_LEN) == 0)
+ {
+ if (offset != lseek (obj->file->fd, offset, SEEK_SET))
+ goto err;
+
+ ssize_t got = read (obj->file->fd, <o_header,
+ sizeof (struct lto_section));
+ if (got != sizeof (struct lto_section))
+ goto err;
+
+ return 1;
+ }
if (strncmp (name, LTO_SECTION_PREFIX, LTO_SECTION_PREFIX_LEN) != 0)
return 1;
@@ -1080,8 +1172,16 @@ claim_file_handler (const struct ld_plugin_input_file *file, int *claimed)
if (obj.found > 0)
{
- status = add_symbols (file->handle, lto_file.symtab.nsyms,
- lto_file.symtab.syms);
+ if (add_symbols_v2)
+ status = add_symbols_v2 (file->handle, lto_file.symtab.nsyms,
+ lto_file.symtab.syms);
+ else
+ {
+ fill_up_older_syms (<o_file.symtab);
+ status = add_symbols (file->handle, lto_file.symtab.nsyms,
+ lto_file.symtab.syms_v1);
+ }
+
check (status == LDPS_OK, LDPL_FATAL, "could not add symbols");
num_claimed_files++;
@@ -1242,12 +1342,18 @@ onload (struct ld_plugin_tv *tv)
case LDPT_REGISTER_CLAIM_FILE_HOOK:
register_claim_file = p->tv_u.tv_register_claim_file;
break;
+ case LDPT_ADD_SYMBOLS_V2:
+ add_symbols_v2 = p->tv_u.tv_add_symbols_v2;
+ break;
case LDPT_ADD_SYMBOLS:
add_symbols = p->tv_u.tv_add_symbols;
break;
case LDPT_REGISTER_ALL_SYMBOLS_READ_HOOK:
register_all_symbols_read = p->tv_u.tv_register_all_symbols_read;
break;
+ case LDPT_GET_SYMBOLS_V4:
+ get_symbols_v4 = p->tv_u.tv_get_symbols_v4;
+ break;
case LDPT_GET_SYMBOLS_V2:
get_symbols_v2 = p->tv_u.tv_get_symbols;
break;
--
2.25.1
