On 20.12.24 16:23, Tom Lane wrote:
Ok, we can fix that, but maybe this is also a good moment to think about
whether that is useful. I could not reproduce the issue with flex
2.5.39. I could find no download of flex 2.5.35. The github site only
offers back to 2.5.39, the sourceforce site back to 2.5.36. lapwing
says it's Debian 7.0, which went out of support in 2016 and out of
super-duper-extended support in 2020. It also doesn't have a supported
OpenSSL version anymore, and IIRC, it has a weird old compiler that
occasionally gives bogus warnings. I think it's time to stop supporting
this.
OK, that's fair. I do see lapwing called out a lot in the commit log,
though it's not clear how much of that is about 32-bitness and how
much about old tools. It's surely still valuable to have i386
machines in the buildfarm, but I agree that supporting unobtainable
tool versions is a bit much. Could we get that animal updated to
some newer OS version?
Presumably, we should also rip out the existing yyget_column and
yyset_column kluges in
src/backend/parser/scan.l: extern int core_yyget_column(yyscan_t
yyscanner);
src/bin/psql/psqlscanslash.l: extern int slash_yyget_column(yyscan_t
yyscanner);
src/bin/pgbench/exprscan.l: extern int expr_yyget_column(yyscan_t
yyscanner);
src/fe_utils/psqlscan.l: extern int psql_yyget_column(yyscan_t
yyscanner);
All my flex-related patches are in now.
Here is a patch that removes the workarounds for compiler warnings with
flex 2.5.35. This ended up being a whole lot, including the whole
fix-old-flex-code.pl script.
The second patch contemplates raising the minimum required flex version,
but what to?
The most recent incrementing was exactly because 2.5.35 was the oldest
in the buildfarm. The previous incrementings were apparently because
certain features were required or some bugs had to be avoided.
Options:
- Leave at 2.5.35 as long as it's present in the buildfarm.
- Set to 2.5.36 because it's the oldest that compiles without warnings.
Also, the oldest you can still download from the flex sourceforge site.
- Set to 2.5.37 because that's the next oldest in the buildfarm (for
CentOS/RHEL 7, so it will stay around for a while).
- Set to 2.5.34 because that's the oldest we actually require as of
commit b1ef48980dd.
- Remove version check, because these are all so old that no one cares
anymore.
From fc1e8aedfe7371ae563ec51dd04346c0efd69fff Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Thu, 9 Jan 2025 08:36:16 +0100
Subject: [PATCH 1/2] Drop warning-free support for flex 2.5.35
This removes all the various workarounds for avoiding compiler
warnings with flex 2.5.35. Several recent patches have added
additional warnings that would either need to be fixed along the lines
of the existing workarounds, or we decide to no longer care about
this, which we do here.
Flex 2.5.35 is extremely outdated, and you can't even download it
anymore from any of the Flex project sites, so it's nearly impossible
to support.
After this, using flex 2.5.35 will still work, but the generated code
will produce numerous compiler warnings.
Discussion:
https://www.postgresql.org/message-id/1a204ccd-7ae6-478c-a431-407b5c48c...@eisentraut.org
---
src/Makefile.global.in | 1 -
src/backend/parser/Makefile | 1 -
src/backend/parser/meson.build | 2 +-
src/backend/parser/scan.l | 9 -----
src/bin/pgbench/exprscan.l | 9 -----
src/bin/psql/Makefile | 1 -
src/bin/psql/meson.build | 2 +-
src/bin/psql/psqlscanslash.l | 9 -----
src/fe_utils/Makefile | 1 -
src/fe_utils/meson.build | 2 +-
src/fe_utils/psqlscan.l | 9 -----
src/tools/fix-old-flex-code.pl | 66 ----------------------------------
src/tools/pgflex | 13 -------
13 files changed, 3 insertions(+), 122 deletions(-)
delete mode 100644 src/tools/fix-old-flex-code.pl
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index eac3d001211..1278b7744f4 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -788,7 +788,6 @@ TAS = @TAS@
%.c: %.l
$(FLEX) $(if $(FLEX_NO_BACKUP),-b) $(FLEXFLAGS) -o'$@' $<
@$(if $(FLEX_NO_BACKUP),if [ `wc -l <lex.backup` -eq 1 ]; then rm
lex.backup; else echo "Scanner requires backup; see lex.backup." 1>&2; exit 1;
fi)
- $(if $(FLEX_FIX_WARNING),$(PERL)
$(top_srcdir)/src/tools/fix-old-flex-code.pl '$@')
%.c: %.y
$(if $(BISON_CHECK_CMD),$(BISON_CHECK_CMD))
diff --git a/src/backend/parser/Makefile b/src/backend/parser/Makefile
index 3162a01f302..8c0fe28d63f 100644
--- a/src/backend/parser/Makefile
+++ b/src/backend/parser/Makefile
@@ -59,7 +59,6 @@ gram.c: BISON_CHECK_CMD = $(PERL) $(srcdir)/check_keywords.pl
$< $(top_srcdir)/s
scan.c: FLEXFLAGS = -CF -p -p
scan.c: FLEX_NO_BACKUP=yes
-scan.c: FLEX_FIX_WARNING=yes
# Force these dependencies to be known even without dependency info built:
diff --git a/src/backend/parser/meson.build b/src/backend/parser/meson.build
index 4c3ca25dd49..874aa749aa6 100644
--- a/src/backend/parser/meson.build
+++ b/src/backend/parser/meson.build
@@ -30,7 +30,7 @@ parser_sources = files('parser.c')
backend_scanner = custom_target('scan',
input: 'scan.l',
output: 'scan.c',
- command: [flex_cmd, '--no-backup', '--fix-warnings', '--', '-CF', '-p',
'-p'],
+ command: [flex_cmd, '--no-backup', '--', '-CF', '-p', '-p'],
)
generated_sources += backend_scanner
parser_sources += backend_scanner
diff --git a/src/backend/parser/scan.l b/src/backend/parser/scan.l
index 8031a78b908..08990831fe8 100644
--- a/src/backend/parser/scan.l
+++ b/src/backend/parser/scan.l
@@ -130,15 +130,6 @@ static void addunicode(pg_wchar c, yyscan_t yyscanner);
static void check_string_escape_warning(unsigned char ychar, core_yyscan_t
yyscanner);
static void check_escape_warning(core_yyscan_t yyscanner);
-/*
- * Work around a bug in flex 2.5.35: it emits a couple of functions that
- * it forgets to emit declarations for. Since we use -Wmissing-prototypes,
- * this would cause warnings. Providing our own declarations should be
- * harmless even when the bug gets fixed.
- */
-extern int core_yyget_column(yyscan_t yyscanner);
-extern void core_yyset_column(int column_no, yyscan_t yyscanner);
-
%}
%option reentrant
diff --git a/src/bin/pgbench/exprscan.l b/src/bin/pgbench/exprscan.l
index a1cd232248f..46f6ea05121 100644
--- a/src/bin/pgbench/exprscan.l
+++ b/src/bin/pgbench/exprscan.l
@@ -44,15 +44,6 @@ static const char *expr_command = NULL;
/* indicates whether last yylex() call read a newline */
static bool last_was_newline = false;
-/*
- * Work around a bug in flex 2.5.35: it emits a couple of functions that
- * it forgets to emit declarations for. Since we use -Wmissing-prototypes,
- * this would cause warnings. Providing our own declarations should be
- * harmless even when the bug gets fixed.
- */
-extern int expr_yyget_column(yyscan_t yyscanner);
-extern void expr_yyset_column(int column_no, yyscan_t yyscanner);
-
/* LCOV_EXCL_START */
%}
diff --git a/src/bin/psql/Makefile b/src/bin/psql/Makefile
index e9e4ade1d2e..5b1545d9948 100644
--- a/src/bin/psql/Makefile
+++ b/src/bin/psql/Makefile
@@ -60,7 +60,6 @@ sql_help.h: create_help.pl $(wildcard $(REFDOCDIR)/*.sgml)
psqlscanslash.c: FLEXFLAGS = -Cfe -p -p
psqlscanslash.c: FLEX_NO_BACKUP=yes
-psqlscanslash.c: FLEX_FIX_WARNING=yes
tab-complete.c: gen_tabcomplete.pl tab-complete.in.c
$(PERL) $^ --outfile $@
diff --git a/src/bin/psql/meson.build b/src/bin/psql/meson.build
index c2ef14ccead..f795ff28271 100644
--- a/src/bin/psql/meson.build
+++ b/src/bin/psql/meson.build
@@ -19,7 +19,7 @@ psql_sources = files(
psqlscanslash = custom_target('psqlscanslash',
input: 'psqlscanslash.l',
output: 'psqlscanslash.c',
- command: [flex_cmd, '--no-backup', '--fix-warnings', '--', '-Cfe', '-p',
'-p'])
+ command: [flex_cmd, '--no-backup', '--', '-Cfe', '-p', '-p'])
generated_sources += psqlscanslash
psql_sources += psqlscanslash
diff --git a/src/bin/psql/psqlscanslash.l b/src/bin/psql/psqlscanslash.l
index f76b7722ac7..ae7602a61df 100644
--- a/src/bin/psql/psqlscanslash.l
+++ b/src/bin/psql/psqlscanslash.l
@@ -57,15 +57,6 @@ static void evaluate_backtick(PsqlScanState state);
#define ECHO psqlscan_emit(cur_state, yytext, yyleng)
-/*
- * Work around a bug in flex 2.5.35: it emits a couple of functions that
- * it forgets to emit declarations for. Since we use -Wmissing-prototypes,
- * this would cause warnings. Providing our own declarations should be
- * harmless even when the bug gets fixed.
- */
-extern int slash_yyget_column(yyscan_t yyscanner);
-extern void slash_yyset_column(int column_no, yyscan_t yyscanner);
-
/* LCOV_EXCL_START */
%}
diff --git a/src/fe_utils/Makefile b/src/fe_utils/Makefile
index f9ca5de2805..28196ce0f6a 100644
--- a/src/fe_utils/Makefile
+++ b/src/fe_utils/Makefile
@@ -51,7 +51,6 @@ libpgfeutils.a: $(OBJS)
psqlscan.c: FLEXFLAGS = -Cfe -p -p
psqlscan.c: FLEX_NO_BACKUP=yes
-psqlscan.c: FLEX_FIX_WARNING=yes
# libpgfeutils could be useful to contrib, so install it
install: all installdirs
diff --git a/src/fe_utils/meson.build b/src/fe_utils/meson.build
index 171268d65e1..a18cbc939e4 100644
--- a/src/fe_utils/meson.build
+++ b/src/fe_utils/meson.build
@@ -23,7 +23,7 @@ fe_utils_sources = files(
psqlscan = custom_target('psqlscan',
input: 'psqlscan.l',
output: 'psqlscan.c',
- command: [flex_cmd, '--no-backup', '--fix-warnings', '--', '-Cfe', '-p',
'-p'],
+ command: [flex_cmd, '--no-backup', '--', '-Cfe', '-p', '-p'],
)
generated_sources += psqlscan
fe_utils_sources += psqlscan
diff --git a/src/fe_utils/psqlscan.l b/src/fe_utils/psqlscan.l
index e1cb726f326..b3c6b88e9ca 100644
--- a/src/fe_utils/psqlscan.l
+++ b/src/fe_utils/psqlscan.l
@@ -61,15 +61,6 @@ typedef int YYSTYPE;
#define ECHO psqlscan_emit(cur_state, yytext, yyleng)
-/*
- * Work around a bug in flex 2.5.35: it emits a couple of functions that
- * it forgets to emit declarations for. Since we use -Wmissing-prototypes,
- * this would cause warnings. Providing our own declarations should be
- * harmless even when the bug gets fixed.
- */
-extern int psql_yyget_column(yyscan_t yyscanner);
-extern void psql_yyset_column(int column_no, yyscan_t yyscanner);
-
%}
%option reentrant
diff --git a/src/tools/fix-old-flex-code.pl b/src/tools/fix-old-flex-code.pl
deleted file mode 100644
index c1c91db114c..00000000000
--- a/src/tools/fix-old-flex-code.pl
+++ /dev/null
@@ -1,66 +0,0 @@
-#!/usr/bin/perl
-#----------------------------------------------------------------------
-#
-# fix-old-flex-code.pl
-#
-# flex versions before 2.5.36, with certain option combinations, produce
-# code that causes an "unused variable" warning. That's annoying, so
-# let's suppress it by inserting a dummy reference to the variable.
-# (That's exactly what 2.5.36 and later do ...)
-#
-# Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group
-# Portions Copyright (c) 1994, Regents of the University of California
-#
-# src/tools/fix-old-flex-code.pl
-#
-#----------------------------------------------------------------------
-
-use strict;
-use warnings FATAL => 'all';
-
-# Get command line argument.
-usage() if $#ARGV != 0;
-my $filename = shift;
-
-# Suck in the whole file.
-local $/ = undef;
-my $cfile;
-open($cfile, '<', $filename) || die "opening $filename for reading: $!";
-my $ccode = <$cfile>;
-close($cfile);
-
-# No need to do anything if it's not flex 2.5.x for x < 36.
-exit 0 if $ccode !~ m/^#define YY_FLEX_MAJOR_VERSION 2$/m;
-exit 0 if $ccode !~ m/^#define YY_FLEX_MINOR_VERSION 5$/m;
-exit 0 if $ccode !~ m/^#define YY_FLEX_SUBMINOR_VERSION (\d+)$/m;
-exit 0 if $1 >= 36;
-
-# Apply the desired patch.
-$ccode =~
- s|(struct yyguts_t \* yyg = \(struct yyguts_t\*\)yyscanner; /\* This var may
be unused depending upon options. \*/
-.*?)
- return yy_is_jam \? 0 : yy_current_state;
-|$1
- (void) yyg;
- return yy_is_jam ? 0 : yy_current_state;
-|s;
-
-# Write the modified file back out.
-open($cfile, '>', $filename) || die "opening $filename for writing: $!";
-print $cfile $ccode;
-close($cfile);
-
-exit 0;
-
-
-sub usage
-{
- die <<EOM;
-Usage: fix-old-flex-code.pl c-file-name
-
-fix-old-flex-code.pl modifies a flex output file to suppress
-an unused-variable warning that occurs with older flex versions.
-
-Report bugs to <pgsql-bugs\@lists.postgresql.org>.
-EOM
-}
diff --git a/src/tools/pgflex b/src/tools/pgflex
index baabe2df1c8..3986b06874e 100755
--- a/src/tools/pgflex
+++ b/src/tools/pgflex
@@ -4,7 +4,6 @@
# Wrapper around flex that:
# - ensures lex.backup is created in a private directory
# - can error out if lex.backup is created (--no-backup)
-# - can fix warnings (--fix-warnings)
# - works around concurrency issues with win_flex.exe:
# https://github.com/lexxmark/winflexbison/issues/86
@@ -28,8 +27,6 @@ parser.add_argument('-o', dest='output_file', type=abspath,
required=True,
parser.add_argument('-i', dest='input_file', type=abspath, help='input file')
-parser.add_argument('--fix-warnings', action='store_true',
- help='whether to fix warnings in generated file')
parser.add_argument('--no-backup', action='store_true',
help='whether no_backup is enabled or not')
@@ -72,14 +69,4 @@ if args.no_backup:
sys.exit('Scanner requires backup; see lex.backup.')
os.remove('lex.backup')
-# fix warnings
-if args.fix_warnings:
- fix_warning_script = os.path.join(args.srcdir,
- 'src/tools/fix-old-flex-code.pl')
-
- command = [args.perl, fix_warning_script, args.output_file]
- sp = subprocess.run(command)
- if sp.returncode != 0:
- sys.exit(sp.returncode)
-
sys.exit(0)
base-commit: 69ab44651422c49a6256d1b6cca6c20b5060ad92
--
2.47.1
From b33f36166fdb5b9dec746ba721881d696241e291 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Thu, 9 Jan 2025 08:41:28 +0100
Subject: [PATCH 2/2] Raise minimum flex version to 2.5.37
---
config/programs.m4 | 8 ++++----
configure | 4 ++--
doc/src/sgml/installation.sgml | 4 ++--
meson.build | 2 +-
src/backend/utils/misc/guc-file.l | 2 +-
5 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/config/programs.m4 b/config/programs.m4
index 490ec9fe9f5..417a4f6d06c 100644
--- a/config/programs.m4
+++ b/config/programs.m4
@@ -59,8 +59,8 @@ AC_SUBST(BISONFLAGS)
# PGAC_PATH_FLEX
# --------------
# Look for Flex, set the output variable FLEX to its path if found.
-# Reject versions before 2.5.35 (the earliest version in the buildfarm
-# as of 2022). Also find Flex if its installed under `lex', but do not
+# Reject versions before 2.5.37 (the earliest version in the buildfarm
+# as of 2025). Also find Flex if its installed under `lex', but do not
# accept other Lex programs.
AC_DEFUN([PGAC_PATH_FLEX],
@@ -84,14 +84,14 @@ else
echo '%%' > conftest.l
if $pgac_candidate -t conftest.l 2>/dev/null | grep FLEX_SCANNER
>/dev/null 2>&1; then
pgac_flex_version=`$pgac_candidate --version 2>/dev/null`
- if echo "$pgac_flex_version" | sed ['s/[.a-z]/ /g'] | $AWK '{ if
([$]1 == 2 && ([$]2 > 5 || ([$]2 == 5 && [$]3 >= 35))) exit 0; else exit 1;}'
+ if echo "$pgac_flex_version" | sed ['s/[.a-z]/ /g'] | $AWK '{ if
([$]1 == 2 && ([$]2 > 5 || ([$]2 == 5 && [$]3 >= 37))) exit 0; else exit 1;}'
then
pgac_cv_path_flex=$pgac_candidate
break 2
else
AC_MSG_ERROR([
*** The installed version of Flex, $pgac_candidate, is too old to use with
PostgreSQL.
-*** Flex version 2.5.35 or later is required, but this is $pgac_flex_version.])
+*** Flex version 2.5.37 or later is required, but this is $pgac_flex_version.])
fi
fi
fi
diff --git a/configure b/configure
index a0b5e10ca39..6f50ab4d7eb 100755
--- a/configure
+++ b/configure
@@ -9955,14 +9955,14 @@ else
echo '%%' > conftest.l
if $pgac_candidate -t conftest.l 2>/dev/null | grep FLEX_SCANNER
>/dev/null 2>&1; then
pgac_flex_version=`$pgac_candidate --version 2>/dev/null`
- if echo "$pgac_flex_version" | sed 's/[.a-z]/ /g' | $AWK '{ if ($1
== 2 && ($2 > 5 || ($2 == 5 && $3 >= 35))) exit 0; else exit 1;}'
+ if echo "$pgac_flex_version" | sed 's/[.a-z]/ /g' | $AWK '{ if ($1
== 2 && ($2 > 5 || ($2 == 5 && $3 >= 37))) exit 0; else exit 1;}'
then
pgac_cv_path_flex=$pgac_candidate
break 2
else
as_fn_error $? "
*** The installed version of Flex, $pgac_candidate, is too old to use with
PostgreSQL.
-*** Flex version 2.5.35 or later is required, but this is $pgac_flex_version."
"$LINENO" 5
+*** Flex version 2.5.37 or later is required, but this is $pgac_flex_version."
"$LINENO" 5
fi
fi
fi
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
index ebdb5b3bc2d..c6dfbf0bfd3 100644
--- a/doc/src/sgml/installation.sgml
+++ b/doc/src/sgml/installation.sgml
@@ -100,7 +100,7 @@ <title>Requirements</title>
<primary>yacc</primary>
</indexterm>
- <application>Flex</application> 2.5.35 or later and
+ <application>Flex</application> 2.5.37 or later and
<application>Bison</application> 2.3 or later are required. Other
<application>lex</application> and <application>yacc</application>
programs cannot be used.
@@ -3813,7 +3813,7 @@ <title>Requirements</title>
<para>
<productname>Bison</productname> and <productname>Flex</productname>
are
required. Only <productname>Bison</productname> versions 2.3 and later
- will work. <productname>Flex</productname> must be version 2.5.35 or
later.
+ will work. <productname>Flex</productname> must be version 2.5.37 or
later.
Binaries can be downloaded from <ulink
url="https://github.com/lexxmark/winflexbison";></ulink>.
</para>
diff --git a/meson.build b/meson.build
index cfd654d2916..ae0dfa852af 100644
--- a/meson.build
+++ b/meson.build
@@ -334,7 +334,7 @@ endif
# External programs
perl = find_program(get_option('PERL'), required: true, native: true)
python = find_program(get_option('PYTHON'), required: true, native: true)
-flex = find_program(get_option('FLEX'), native: true, version: '>= 2.5.35')
+flex = find_program(get_option('FLEX'), native: true, version: '>= 2.5.37')
bison = find_program(get_option('BISON'), native: true, version: '>= 2.3')
sed = find_program(get_option('SED'), 'sed', native: true, required: false)
prove = find_program(get_option('PROVE'), native: true, required: false)
diff --git a/src/backend/utils/misc/guc-file.l
b/src/backend/utils/misc/guc-file.l
index c0ecb8b2ce2..280c3320954 100644
--- a/src/backend/utils/misc/guc-file.l
+++ b/src/backend/utils/misc/guc-file.l
@@ -302,7 +302,7 @@ record_config_file_error(const char *errmsg,
/*
* Flex fatal errors bring us here. Stash the error message and jump back to
* ParseConfigFp(). Assume all msg arguments point to string constants; this
- * holds for flex 2.5.35 (earliest we support). Otherwise, we would need to
+ * holds for flex 2.5.37 (earliest we support). Otherwise, we would need to
* copy the message.
*
* We return "int" since this takes the place of calls to fprintf().
--
2.47.1
