Hi,
I noticed that configure can be a bit slow - few minutes on my macOS
system, 1:30m on linux/bash, 30s on linux/dash, and even if
pathological, still a too-long 10 minutes on Windows (MSYS2) (same
duration as a full build with make -j4 after configure).
I added some timing information printouts for various parts in
configure, and eventually identified few culprits.
The attachment "config-timing.patch" adds these printouts and sorted
summary. It's attached for reference and not intended to be merged.
It applies cleanly before or after the main patch.
The attached "main.patch" addresses three areas I identified as slow:
1. About 50-70% of configure runtime was being spent inside one
function: flatten_extralibs() and callees resolve() and unique().
It manipulates strings and invoked nearly 20K (20000) subshells.
It was rewritten to avoid subshells, and ended up x50-x250 faster.
2. print_enabled_components() was invoking sed about 350 times on one
file. This is never instant but takes many seconds where fork is
slow (Windows). Invoke sed only once instead = x4-x10 speedup.
3. After the previous speedups, configure spent 20-60% of its runtime
at check_deps(). It's particularly slow with bash. After some local
optimizations - mainly avoid pushvar/popvar and abort early in one
notable case (empty deps), it's now x4-x25 faster.
Some resulting speedups (more details at "before-after.txt"):
- macOS/bash: total: 98s -> 22s flatten_extralibs: 53s -> 0.7s
- Linux/bash: total: 87s -> 11s flatten_extralibs: 59s -> 0.9s
- Linux/dash: total: 27s -> 8s flatten_extralibs: 17s -> 0.2s
- FreeBSD/sh: total: 34s -> 9s flatten_extralibs: 23s -> 0.2s
- MSYS2/bash: total: 10m -> 2:30m flatten_extralibs: 400s -> 1.5s (!)
Notes:
- unique() is modified with flatten_extralibs. It now outputs different
order: it was keeping the last instance of recurring items, now it
keeps the first. It affects libs order at ffbuild/config.{mak,sh} -
but I don't think it matters. If it does, "opt1-reorder-unique.patch"
restores the original order. Let me know if/why it matters and I'll
squash it and update the commit message accordingly if required.
- After the check_deps() patch, pushvar() and popvar() are not used but
I was hesitant to remove them (nice to have). If you think it should
be removed, the patch "opt2-remove-pushvar.patch" removes them.
- The patches assume POSIX shell and don't use anything "tricky".
It was tested with dash, bash, busybox-ash, freebsd-sh, ksh93u, mksh.
- Thanks to tmm1 and atomnuker for their help with testing.
This is my first mail to this list, and I wasn't sure how to split the
patches. After asking a veteran, it was suggested to me that I should
put all of them in one email. Just let me know if you prefer something
else (dev mailing lists are not my forte...).
For convinience, the patches are also available at:
https://gist.github.com/avih/f51008225d4a20a0981daed1faca4bc2
Regards,
AviFrom 4d0654798f50bcf1c68a07927511bde17ab723be Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Tue, 31 Jul 2018 03:22:52 +0300
Subject: [PATCH] configure: print timing information
Add the file tools/time.sh, and use it from configure to print timing
information and sorted summary, while isolating prominent sections.
The chosen sections/names are (in runtime order):
- Init
- Options review
- Compilation tests
- check_deps()
- Prepare deps
- flatten_extralibs_wrapper()
- Writing config
- print_enabled_components()
- Finalize
Where the foo_bar() names are execution solely of the function foo_bar, and
the rest are descriptive-ish.
---
configure | 21 ++++++++++++
tools/time.sh | 94 +++++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 115 insertions(+)
create mode 100644 tools/time.sh
diff --git a/configure b/configure
index b9c9d0b3..07398852 100755
--- a/configure
+++ b/configure
@@ -11,6 +11,9 @@
LC_ALL=C
export LC_ALL
+. "$(dirname "$0")"/tools/time.sh
+time_switch_to Init
+
# make sure we are running under a compatible shell
# try to make this part work with most shells
@@ -3633,6 +3636,8 @@ host_ldflags_filter=echo
target_path='$(CURDIR)'
+time_switch_to "Options review"
+
# since the object filename is not given with the -MM flag, the compiler
# is only able to print the basename, and we must add the path ourselves
DEPCMD='$(DEP$(1)) $(DEP$(1)FLAGS) $($(1)DEP_FLAGS) $< 2>/dev/null | sed -e "/^\#.*/d" -e "s,^[[:space:]]*$(@F),$(@D)/$(@F)," > $(@:.o=.d)'
@@ -4116,6 +4121,8 @@ exesuf() {
EXESUF=$(exesuf $target_os)
HOSTEXESUF=$(exesuf $host_os)
+time_switch_to "Compilation tests"
+
# set temporary file name
: ${TMPDIR:=$TEMPDIR}
: ${TMPDIR:=$TMP}
@@ -6720,11 +6727,15 @@ enable frame_thread_encoder
enabled asm || { arch=c; disable $ARCH_LIST $ARCH_EXT_LIST; }
+time_switch_to "check_deps()"
+
check_deps $CONFIG_LIST \
$CONFIG_EXTRA \
$HAVE_LIST \
$ALL_COMPONENTS \
+time_switch_to "Prepare deps"
+
enabled threads && ! enabled pthreads && ! enabled atomics_native && die "non pthread threading without atomics not supported, try adding --enable-pthreads or --cpu=i486 or higher if you are on x86"
enabled avresample && warn "Building with deprecated library libavresample"
@@ -6772,10 +6783,14 @@ for linkunit in $LIBRARY_LIST; do
eval prepend ${linkunit}_extralibs $current_extralibs
done
+time_switch_to "flatten_extralibs_wrapper()"
+
for linkunit in $LIBRARY_LIST $PROGRAM_LIST $EXTRALIBS_LIST; do
eval ${linkunit}_extralibs=\$\(flatten_extralibs_wrapper ${linkunit}_extralibs\)
done
+time_switch_to "Writing config"
+
map 'enabled $v && intrinsics=${v#intrinsics_}' $INTRINSICS_LIST
for thread in $THREADS_LIST; do
@@ -7211,6 +7226,8 @@ cat > $TMPH <<EOF
#define AVUTIL_AVCONFIG_H
EOF
+time_switch_to "print_enabled_components()"
+
print_config AV_HAVE_ $TMPH $HAVE_LIST_PUB
echo "#endif /* AVUTIL_AVCONFIG_H */" >> $TMPH
@@ -7263,6 +7280,8 @@ print_enabled_components libavdevice/indev_list.c AVInputFormat indev_list $INDE
print_enabled_components libavdevice/outdev_list.c AVOutputFormat outdev_list $OUTDEV_LIST
print_enabled_components libavformat/protocol_list.c URLProtocol url_protocols $PROTOCOL_LIST
+time_switch_to Finalize
+
# Settings for pkg-config files
cat > $TMPH <<EOF
@@ -7293,3 +7312,5 @@ for lib in $LIBRARY_LIST; do
done
cp_if_changed $TMPH ffbuild/config.sh
+
+time_finish
diff --git a/tools/time.sh b/tools/time.sh
new file mode 100644
index 00000000..375fc28c
--- /dev/null
+++ b/tools/time.sh
@@ -0,0 +1,94 @@
+# time.sh - simple flat/top-level timing measurement helper for shell scripts
+# Copyright 2018 Avi Halachmi @License MIT https://github.com/avih/time.sh
+#
+# Example (see README.md for more info):
+# . /path/to/time.sh
+# time_switch_to foo
+# <do task foo>
+# time_switch_to "bar baz"
+# <do bar maybe with an argument baz>
+# time_finish
+
+
+[ "${TIME_PRINT-}" ] || TIME_PRINT=3
+
+# find a command to print a timestamp in ms, or fail to 'echo 0'.
+# perl/python will have bigger overhead, but we auto-compensate reasonably-ish.
+if [ -z "${TIME_CMD-}" ]; then
+ for TIME_CMD in \
+ 'date +%s%3N' \
+ 'python -c "from time import time; print (int(time()*1000))"' \
+ 'perl -MPOSIX=strftime -MTime::HiRes=time -le "print int(time*1000)"' \
+ 'date +%s000' \
+ 'echo 0' ;
+ do
+ _tsh_tmp=$(eval "$TIME_CMD") 2>/dev/null &&
+ case $_tsh_tmp in *[!0-9]*|'') ;; *) break; esac
+ done
+fi
+
+_tsh_started=
+_tsh_NL="
+"
+
+# stops the measurement if one was started, and start a new one named $1
+# only one measurement at a time is supported, and one (automatic) overall.
+time_switch_to() {
+ _tsh_now=$(eval "$TIME_CMD")
+
+ if [ "$_tsh_started" ]; then
+ _tsh_dur=$(( _tsh_now - _tsh_t0 - TIME_COMP ))
+ [ $_tsh_dur -ge 0 ] || _tsh_dur=0
+ time_total=$(( time_total + _tsh_dur ))
+ time_details="${time_details}${_tsh_dur} $_tsh_name${_tsh_NL}"
+ case $TIME_PRINT in 2|3) >&2 echo "[ $_tsh_name: $_tsh_dur ms ]"; esac
+ else
+ _tsh_started=yes; time_total=0; time_details=
+ fi
+
+ case ${1-} in
+ --|'') _tsh_started= ;; # "finish". caller can use total and details
+ *) _tsh_name=$1
+ _tsh_t0=$_tsh_now
+ case $TIME_PRINT in 1|3) >&2 echo "[ $1 ... ]"; esac
+ esac
+}
+
+# assess compensation in ms/measure, floored to err on the side of caution.
+# not perfect since cpu frequency scaling can change etc, but reasonable.
+_tsh_assess_overhead() {
+ TIME_PRINT=0; TIME_COMP=0
+ time_switch_to x; time_switch_to -- # warmup and reset
+ for x in 1 2 3 4; do time_switch_to $x; done; time_switch_to --
+ echo $(( time_total / 4 )) # 5 calls = 4 deltas, floored average
+}
+
+case ${TIME_COMP-} in *[!0-9]*|'') TIME_COMP=$(_tsh_assess_overhead); esac
+
+# $1: number, $2: number to be printed as rounded percentage (number) of $1
+_tsh_percent() {
+ [ "$1" -gt 0 ] || set -- 1 0
+ [ "$2" -ge 0 ] || set -- $1 0
+ echo $(( (200 * $2 / $1 + 1 ) / 2 ))
+}
+
+# print a summary and reset the measurements. slow-ish but outside the measure.
+# percentage values are rounded and might not add up to 100.
+time_finish() {
+ time_switch_to --
+ _tsh_shell="$(basename "$( (readlink /proc/$$/exe || ps -o comm= -p $$) 2>/dev/null)")"
+
+ >&2 printf "\n%s\n%s ms%s, %s ms/measure, %s\n%s\n" \
+ ----- \
+ $time_total "${TIME_TITLE:+ - $TIME_TITLE}" \
+ $TIME_COMP "$(uname)${_tsh_shell:+/$_tsh_shell}" \
+ -----
+
+ printf %s "$time_details" | sort -rn | \
+ (while read line; do
+ >&2 printf "%${#time_total}s ms %3s %% %s\n" \
+ "${line%% *}" "$(_tsh_percent $time_total $line)" "${line#* }"
+ done)
+
+ >&2 printf "%s\n" -----
+}
--
2.17.1
From 7593d9f6580afd8a1066df1e1910a817d7c2fff1 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Mon, 30 Jul 2018 22:39:15 +0300
Subject: [PATCH 1/3] configure: flatten_extralibs_wrapper(): speedup x50 -
x200
Currently configure spends 50-70% of its runtime inside a single
function: flatten_extralibs[_wrapper] - which does string processing.
During its run, nearly 20K command substitutions (subshells) are used,
including its callees unique() and resolve(), which is the reason
for its lengthy run.
This commit avoids all subshells during its execution, speeding it up
by about two orders of magnitude, and reducing the overall configure
runtime by 50-70% .
resolve() is rewritten to avoid subshells, and in unique() and
flatten_extralibs() we "inline" the filter[_out] functionality.
Note that unique() now produces different output order than before.
Previously it kept the last occurence of re-occuring items, and now
it keeps the first. However, this shouldn't make a difference as it's
used for unique library dependencies where the order doesn't matter.
---
configure | 33 ++++++++++++++++++++-------------
1 file changed, 20 insertions(+), 13 deletions(-)
diff --git a/configure b/configure
index b9c9d0b3..465d808d 100755
--- a/configure
+++ b/configure
@@ -845,21 +845,23 @@ prepend(){
}
unique(){
- var=$1
- uniq_list=""
- for tok in $(eval echo \$$var); do
- uniq_list="$(filter_out $tok $uniq_list) $tok"
+ unique_out=
+ eval unique_in=\$$1
+ for v in $unique_in; do
+ # " $unique_out" +space such that every item is surrounded with spaces
+ case " $unique_out" in *" $v "*) continue; esac # already in list
+ unique_out="$unique_out$v "
done
- eval "$var=\"${uniq_list}\""
+ eval $1=\$unique_out
}
resolve(){
- var=$1
- tmpvar=
- for entry in $(eval echo \$$var); do
- tmpvar="$tmpvar $(eval echo \$${entry})"
+ resolve_out=
+ eval resolve_in=\$$1
+ for v in $resolve_in; do
+ eval 'resolve_out="$resolve_out$'$v' "'
done
- eval "$var=\"${tmpvar}\""
+ eval $1=\$resolve_out
}
add_cppflags(){
@@ -6734,14 +6736,19 @@ if test $target_os = "haiku"; then
fi
flatten_extralibs(){
- unset nested_entries
+ nested_entries=
list_name=$1
eval list=\$${1}
for entry in $list; do
entry_copy=$entry
resolve entry_copy
- append nested_entries $(filter '*_extralibs' $entry_copy)
- flat_entries=$(filter_out '*_extralibs' $entry_copy)
+ flat_entries=
+ for e in $entry_copy; do
+ case $e in
+ *_extralibs) nested_entries="$nested_entries$e ";;
+ *) flat_entries="$flat_entries$e ";;
+ esac
+ done
eval $entry="\$flat_entries"
done
append $list_name "$nested_entries"
--
2.17.1
From d1cf17fdb518b3a456bbac61e306599d0a4a7400 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Mon, 30 Jul 2018 22:48:04 +0300
Subject: [PATCH 2/3] configure: print_enabled_components(): speedup x4 - x10
Inside print_enabled components, the filter_list case invokes sed
about 350 times to parse the same source file and extract different
info for each arg. This is never instant, and on systems where fork is
slow (notably MSYS2/Cygwin on windows) it takes many seconds.
Change it to use sed once on the source file and set env vars with the
parse results, then use these results inside the loop.
Additionally, the cases of indev_list and outdev_list are very
infrequent, but nevertheless they're faster, and arguably cleaner, with
shell parameter substitutions than with command substitutions.
---
configure | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/configure b/configure
index 465d808d..b74c44e7 100755
--- a/configure
+++ b/configure
@@ -7224,9 +7224,10 @@ echo "#endif /* AVUTIL_AVCONFIG_H */" >> $TMPH
cp_if_changed $TMPH libavutil/avconfig.h
-full_filter_name(){
- sed -n "s/^extern AVFilter ff_\([avfsinkrc]\{2,5\}\)_$1;/\1_$1/p" $source_path/libavfilter/allfilters.c
-}
+# full_filter_name_foo=vf_foo
+# full_filter_name_bar=asrc_bar
+# ...
+eval "$(sed -n "s/^extern AVFilter ff_\([avfsinkrc]\{2,5\}\)_\(.*\);/full_filter_name_\2=\1_\2/p" $source_path/libavfilter/allfilters.c)"
# generate the lists of enabled components
print_enabled_components(){
@@ -7239,13 +7240,13 @@ print_enabled_components(){
if enabled $c; then
case $name in
filter_list)
- c=$(full_filter_name $(remove_suffix _filter $c))
+ eval c=\$full_filter_name_${c%_filter}
;;
indev_list)
- c=$(add_suffix _demuxer $(remove_suffix _indev $c))
+ c=${c%_indev}_demuxer
;;
outdev_list)
- c=$(add_suffix _muxer $(remove_suffix _outdev $c))
+ c=${c%_outdev}_muxer
;;
esac
printf " &ff_%s,\n" $c >> $TMPH
--
2.17.1
From a068913d2c4c460ef0a256cb3676bd1227f59667 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Sun, 5 Aug 2018 06:16:44 +0300
Subject: [PATCH 3/3] configure: check_deps(): x4 - x25 speedup
check_deps() recursively enables/disables components, and its loop is
iterated nearly 6000 times. It's particularly slow in bash - currently
consuming more than 50% of configure runtime, and about 20% with other
shells.
This commit applies few local optimizations, most effective first:
- Use $1 $2 ... instead of pushvar/popvar, and same at enable_deep*
- Abort early in one notable case - empty deps, to avoid costly no-op.
- Smaller changes which do add up:
- Handle ${cfg}_checking locally instead of via enable[d]/disable
- ${cfg}_checking: test done before inprogress - x2 faster in 50%+
- one eval instead of several at the empty-deps early abort path.
- The "actual work" part is unmodified - just its surroundings.
Biggest speedups (relative and absolute) are observed with bash.
---
configure | 86 +++++++++++++++++++++++++++++--------------------------
1 file changed, 46 insertions(+), 40 deletions(-)
diff --git a/configure b/configure
index b74c44e7..e9cb7703 100755
--- a/configure
+++ b/configure
@@ -676,13 +676,10 @@ disable_sanitized(){
do_enable_deep(){
for var; do
enabled $var && continue
- eval sel="\$${var}_select"
- eval sgs="\$${var}_suggest"
- pushvar var sgs
- enable_deep $sel
- popvar sgs
- enable_deep_weak $sgs
- popvar var
+ set -- $var
+ eval enable_deep \$${var}_select
+ var=$1
+ eval enable_deep_weak \$${var}_suggest
done
}
@@ -694,9 +691,9 @@ enable_deep(){
enable_deep_weak(){
for var; do
disabled $var && continue
- pushvar var
+ set -- $var
do_enable_deep $var
- popvar var
+ var=$1
enable_weak $var
done
}
@@ -756,40 +753,49 @@ is_in(){
return 1
}
+# The cfg loop is very hot (several thousands iterations), and in bash also
+# potentialy quite slow. Try to abort the iterations early, preferably without
+# calling functions. 70%+ of the time cfg is already done or without deps.
check_deps(){
for cfg; do
- enabled ${cfg}_checking && die "Circular dependency for $cfg."
- disabled ${cfg}_checking && continue
- enable ${cfg}_checking
-
- eval dep_all="\$${cfg}_deps"
- eval dep_any="\$${cfg}_deps_any"
- eval dep_con="\$${cfg}_conflict"
- eval dep_sel="\$${cfg}_select"
- eval dep_sgs="\$${cfg}_suggest"
- eval dep_ifa="\$${cfg}_if"
- eval dep_ifn="\$${cfg}_if_any"
-
- pushvar cfg dep_all dep_any dep_con dep_sel dep_sgs dep_ifa dep_ifn
- check_deps $dep_all $dep_any $dep_con $dep_sel $dep_sgs $dep_ifa $dep_ifn
- popvar cfg dep_all dep_any dep_con dep_sel dep_sgs dep_ifa dep_ifn
-
- [ -n "$dep_ifa" ] && { enabled_all $dep_ifa && enable_weak $cfg; }
- [ -n "$dep_ifn" ] && { enabled_any $dep_ifn && enable_weak $cfg; }
- enabled_all $dep_all || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not all dependencies are satisfied: $dep_all"; }
- enabled_any $dep_any || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not any dependency is satisfied: $dep_any"; }
- disabled_all $dep_con || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some conflicting dependencies are unsatisfied: $dep_con"; }
- disabled_any $dep_sel && { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some selected dependency is unsatisfied: $dep_sel"; }
-
- enabled $cfg && enable_deep_weak $dep_sel $dep_sgs
-
- for dep in $dep_all $dep_any $dep_sel $dep_sgs; do
- # filter out library deps, these do not belong in extralibs
- is_in $dep $LIBRARY_LIST && continue
- enabled $dep && eval append ${cfg}_extralibs ${dep}_extralibs
- done
+ eval [ x\$${cfg}_checking = xdone ] && continue
+ eval [ x\$${cfg}_checking = xinprogress ] && die "Circular dependency for $cfg."
+
+ eval "
+ dep_all=\$${cfg}_deps
+ dep_any=\$${cfg}_deps_any
+ dep_con=\$${cfg}_conflict
+ dep_sel=\$${cfg}_select
+ dep_sgs=\$${cfg}_suggest
+ dep_ifa=\$${cfg}_if
+ dep_ifn=\$${cfg}_if_any
+ "
+
+ # most of the time here $cfg has no deps - avoid costly no-op work
+ if [ "$dep_all$dep_any$dep_con$dep_sel$dep_sgs$dep_ifa$dep_ifn" ]; then
+ eval ${cfg}_checking=inprogress
+
+ set -- $cfg "$dep_all" "$dep_any" "$dep_con" "$dep_sel" "$dep_sgs" "$dep_ifa" "$dep_ifn"
+ check_deps $dep_all $dep_any $dep_con $dep_sel $dep_sgs $dep_ifa $dep_ifn
+ cfg=$1; dep_all=$2; dep_any=$3; dep_con=$4; dep_sel=$5 dep_sgs=$6; dep_ifa=$7; dep_ifn=$8
+
+ [ -n "$dep_ifa" ] && { enabled_all $dep_ifa && enable_weak $cfg; }
+ [ -n "$dep_ifn" ] && { enabled_any $dep_ifn && enable_weak $cfg; }
+ enabled_all $dep_all || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not all dependencies are satisfied: $dep_all"; }
+ enabled_any $dep_any || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but not any dependency is satisfied: $dep_any"; }
+ disabled_all $dep_con || { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some conflicting dependencies are unsatisfied: $dep_con"; }
+ disabled_any $dep_sel && { disable $cfg && requested $cfg && die "ERROR: $cfg requested, but some selected dependency is unsatisfied: $dep_sel"; }
+
+ enabled $cfg && enable_deep_weak $dep_sel $dep_sgs
+
+ for dep in $dep_all $dep_any $dep_sel $dep_sgs; do
+ # filter out library deps, these do not belong in extralibs
+ is_in $dep $LIBRARY_LIST && continue
+ enabled $dep && eval append ${cfg}_extralibs ${dep}_extralibs
+ done
+ fi
- disable ${cfg}_checking
+ eval ${cfg}_checking=done
done
}
--
2.17.1
From 94b80da14fbca1296ac744a53581b817ae56d370 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Wed, 1 Aug 2018 09:06:36 +0300
Subject: [PATCH] configure: unique(): fixup to restore original output order
Originally unique() was keeping the last occurence of each non-unique
item, but commit XXX changed it to keep the first.
This commit restores the original order due to *TBD*.
---
configure | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/configure b/configure
index e9cb7703..ba123ead 100755
--- a/configure
+++ b/configure
@@ -850,14 +850,27 @@ prepend(){
eval "$var=\"$* \$$var\""
}
+reverse () {
+ eval '
+ reverse_out=
+ for v in $'$1'; do
+ reverse_out="$v $reverse_out"
+ done
+ '$1'=$reverse_out
+ '
+}
+
+# keeps the last occurence of each non-unique item
unique(){
unique_out=
eval unique_in=\$$1
+ reverse unique_in
for v in $unique_in; do
# " $unique_out" +space such that every item is surrounded with spaces
case " $unique_out" in *" $v "*) continue; esac # already in list
unique_out="$unique_out$v "
done
+ reverse unique_out
eval $1=\$unique_out
}
--
2.17.1
From 77f897c8ed4eec9119d758037b0311629f549a5b Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Wed, 1 Aug 2018 09:10:12 +0300
Subject: [PATCH] configure: remove unused pushvar()/popvar()
---
configure | 19 -------------------
1 file changed, 19 deletions(-)
diff --git a/configure b/configure
index e9cb7703..d6c1d032 100755
--- a/configure
+++ b/configure
@@ -619,25 +619,6 @@ get_sanitized(){
eval echo \$$(sanitize_var_name "$1")
}
-pushvar(){
- for pvar in $*; do
- eval level=\${${pvar}_level:=0}
- eval ${pvar}_${level}="\$$pvar"
- eval ${pvar}_level=$(($level+1))
- done
-}
-
-popvar(){
- for pvar in $*; do
- eval level=\${${pvar}_level:-0}
- test $level = 0 && continue
- eval level=$(($level-1))
- eval $pvar="\${${pvar}_${level}}"
- eval ${pvar}_level=$level
- eval unset ${pvar}_${level}
- done
-}
-
request(){
for var in $*; do
eval ${var}_requested=yes
--
2.17.1
configure - speedup - before/after details
Systems:
- Linux: Ubuntu 16.04 VM: bash 4.3/4.4, dash 4.4, busybox-ash 1.29, ksh93u
- macOS: iMac 5k 2017 i7: bash 3.2
- macOS: MBA late 2010: bash 3.2, dash 0.5.10
- FreeBSD: GhostBSD 11.1: sh
- MSYS2: latest (gcc 8): bash 4.4, dash 0.5.10
- Linux: debian unstable: dash, bash
---------------------------------------------
Linux - Ubuntu 16.04 - VM (i7 3630qm)
/bin/sh (dash): 28s -> 8.3s = x3.3
bash 4.3 : 103s -> 12.5s = x8
bash 4.4 : 87s -> 11.3s = x8
busybox ash : 37s -> 9s = x4
ksh93 : 23s -> 9s = x2.5
---------------------------------------------
bash 4.3 BEFORE:
102787 ms, 1 ms/measure, Linux/bash (4.3.48)
-----
72526 ms 71 % flatten_extralibs_wrapper()
18329 ms 18 % check_deps()
8295 ms 8 % Compilation tests
2062 ms 2 % print_enabled_components()
992 ms 1 % Writing config
343 ms 0 % Prepare deps
190 ms 0 % Options review
33 ms 0 % Init
17 ms 0 % Finalize
bash 4.3 AFTER:
12529 ms, 1 ms/measure, Linux/bash (4.3.48)
-----
8358 ms 67 % Compilation tests
1408 ms 11 % check_deps()
920 ms 7 % flatten_extralibs_wrapper()
735 ms 6 % Writing config
520 ms 4 % print_enabled_components()
346 ms 3 % Prepare deps
190 ms 2 % Options review
35 ms 0 % Init
17 ms 0 % Finalize
bash 4.4 BEFORE:
86601 ms, 1 ms/measure, Linux/bash (4.4.19)
-----
58889 ms 68 % flatten_extralibs_wrapper()
16288 ms 19 % check_deps()
8127 ms 9 % Compilation tests
2013 ms 2 % print_enabled_components()
793 ms 1 % Writing config
253 ms 0 % Prepare deps
188 ms 0 % Options review
34 ms 0 % Init
16 ms 0 % Finalize
bash 4.4 AFTER:
11373 ms, 1 ms/measure, Linux/bash (4.4.19)
-----
8170 ms 72 % Compilation tests
985 ms 9 % check_deps()
758 ms 7 % flatten_extralibs_wrapper()
555 ms 5 % Writing config
425 ms 4 % print_enabled_components()
250 ms 2 % Prepare deps
182 ms 2 % Options review
32 ms 0 % Init
16 ms 0 % Finalize
dash 0.5.8 BEFORE:
28308 ms, 1 ms/measure, Linux/dash
-----
18306 ms 65 % flatten_extralibs_wrapper()
7450 ms 26 % Compilation tests
1260 ms 4 % check_deps()
895 ms 3 % print_enabled_components()
227 ms 1 % Writing config
88 ms 0 % Options review
59 ms 0 % Prepare deps
14 ms 0 % Init
9 ms 0 % Finalize
dash 0.5.8 AFTER:
8312 ms, 1 ms/measure, Linux/dash
-----
7390 ms 89 % Compilation tests
298 ms 4 % check_deps()
218 ms 3 % flatten_extralibs_wrapper()
136 ms 2 % Writing config
100 ms 1 % print_enabled_components()
88 ms 1 % Options review
59 ms 1 % Prepare deps
14 ms 0 % Init
9 ms 0 % Finalize
busybox-ash 1.29 BEFORE:
36580 ms, 1 ms/measure, Linux/busybox (ash)
-----
25559 ms 70 % flatten_extralibs_wrapper()
7580 ms 21 % Compilation tests
1974 ms 5 % check_deps()
962 ms 3 % print_enabled_components()
277 ms 1 % Writing config
107 ms 0 % Options review
92 ms 0 % Prepare deps
20 ms 0 % Init
9 ms 0 % Finalize
busybox-ash 1.29 AFTER:
8888 ms, 1 ms/measure, Linux/busybox (ash)
-----
7603 ms 86 % Compilation tests
452 ms 5 % check_deps()
311 ms 3 % flatten_extralibs_wrapper()
189 ms 2 % Writing config
113 ms 1 % print_enabled_components()
101 ms 1 % Options review
91 ms 1 % Prepare deps
18 ms 0 % Init
10 ms 0 % Finalize
ksh93u BEFORE:
22483 ms, 0 ms/measure, Linux/ksh93
-----
11985 ms 53 % flatten_extralibs_wrapper()
7353 ms 33 % Compilation tests
1690 ms 8 % check_deps()
914 ms 4 % print_enabled_components()
330 ms 1 % Writing config
108 ms 0 % Options review
79 ms 0 % Prepare deps
14 ms 0 % Init
10 ms 0 % Finalize
ksh93u AFTER:
8898 ms, 1 ms/measure, Linux/ksh93
-----
7511 ms 84 % Compilation tests
353 ms 4 % check_deps()
333 ms 4 % flatten_extralibs_wrapper()
328 ms 4 % Writing config
162 ms 2 % print_enabled_components()
107 ms 1 % Options review
80 ms 1 % Prepare deps
15 ms 0 % Init
9 ms 0 % Finalize
---------------------------------------------
tmm1:
macOS (iMac 5k 2017 i7)
/bin/sh (bash 3.2): 98s -> 22s = x4.5
---------------------------------------------
bash BEFORE:
98134 ms, 30 ms/measure, Darwin/sh
-----
52947 ms 54 % flatten_extralibs_wrapper()
23608 ms 24 % check_deps()
17392 ms 18 % Compilation tests
2603 ms 3 % print_enabled_components()
994 ms 1 % Writing config
281 ms 0 % Prepare deps
251 ms 0 % Options review
39 ms 0 % Init
19 ms 0 % Finalize
bash AFTER:
21981 ms, 29 ms/measure, Darwin/sh
-----
17783 ms 81 % Compilation tests
1396 ms 6 % check_deps()
791 ms 4 % Writing config
733 ms 3 % print_enabled_components()
682 ms 3 % flatten_extralibs_wrapper()
285 ms 1 % Prepare deps
248 ms 1 % Options review
40 ms 0 % Init
23 ms 0 % Finalize
---------------------------------------------
macOS (MBA late 2010)
/bin/sh (bash 3.2): 272s -> 61s = x4.5
dash : 114s -> 51s = x2.3
---------------------------------------------
bash 3.2 BEFORE:
272643 ms, 67 ms/measure, Darwin/sh (bash 3.2.57)
-----
140802 ms 52 % flatten_extralibs_wrapper()
71122 ms 26 % check_deps()
48587 ms 18 % Compilation tests
7445 ms 3 % print_enabled_components()
2898 ms 1 % Writing config
797 ms 0 % Prepare deps
791 ms 0 % Options review
133 ms 0 % Init
68 ms 0 % Finalize
bash 3.2 AFTER:
60802 ms, 68 ms/measure, Darwin/sh (bash 3.2.57)
-----
48835 ms 80 % Compilation tests
3663 ms 6 % check_deps()
2212 ms 4 % Writing config
2209 ms 4 % print_enabled_components()
2084 ms 3 % flatten_extralibs_wrapper()
807 ms 1 % Options review
790 ms 1 % Prepare deps
147 ms 0 % Init
55 ms 0 % Finalize
dash 0.5.10 before and after:
Note, I got some reports that configure sometimes fails in dash, possibly under
specific configurtions. This wasn't investigated further.
dash 0.5.10 BEFORE:
114689 ms, 66 ms/measure, Darwin/dash
-----
56977 ms 50 % flatten_extralibs_wrapper()
46503 ms 41 % Compilation tests
5466 ms 5 % check_deps()
3732 ms 3 % print_enabled_components()
1216 ms 1 % Writing config
393 ms 0 % Options review
256 ms 0 % Prepare deps
102 ms 0 % Init
44 ms 0 % Finalize
dash 0.5.10 AFTER:
51010 ms, 67 ms/measure, Darwin/dash
-----
46599 ms 91 % Compilation tests
1201 ms 2 % check_deps()
999 ms 2 % flatten_extralibs_wrapper()
835 ms 2 % Writing config
577 ms 1 % print_enabled_components()
405 ms 1 % Options review
254 ms 0 % Prepare deps
93 ms 0 % Init
47 ms 0 % Finalize
---------------------------------------------
FreeBSD - GhostBSD 11.1 live DVD - VM (i7 4500u)
/bin/sh: 34.5s -> 9s = x4
---------------------------------------------
sh BEFORE:
34410 ms, 15 ms/measure, FreeBSD/sh
-----
23270 ms 68 % flatten_extralibs_wrapper()
7606 ms 22 % Compilation tests
1875 ms 5 % print_enabled_components()
789 ms 2 % check_deps()
506 ms 1 % Writing config
271 ms 1 % Options review
53 ms 0 % Prepare deps
23 ms 0 % Init
17 ms 0 % Finalize
sh AFTER:
9045 ms, 15 ms/measure, FreeBSD/sh
-----
7691 ms 85 % Compilation tests
381 ms 4 % Writing config
270 ms 3 % check_deps()
266 ms 3 % Options review
186 ms 2 % flatten_extralibs_wrapper()
149 ms 2 % print_enabled_components()
59 ms 1 % Prepare deps
23 ms 0 % Init
20 ms 0 % Finalize
---------------------------------------------
MSYS2 - Windows 8.1 (i7 4500u)
/bin/sh (bash 4.4.19): 600s -> 148s = x4
dash : 438s -> 139s = x3
---------------------------------------------
bash BEFORE:
599395 ms, 37 ms/measure, MINGW64_NT-6.3/sh
-----
396352 ms 66 % flatten_extralibs_wrapper()
133812 ms 22 % Compilation tests
36044 ms 6 % check_deps()
21561 ms 4 % print_enabled_components()
6902 ms 1 % Writing config
3364 ms 1 % Options review
742 ms 0 % Prepare deps
351 ms 0 % Init
267 ms 0 % Finalize
bash AFTER:
147484 ms, 36 ms/measure, MINGW64_NT-6.3/sh (bash 4.4.19)
-----
133968 ms 91 % Compilation tests
3502 ms 2 % Options review
3247 ms 2 % Writing config
2307 ms 2 % print_enabled_components()
1553 ms 1 % flatten_extralibs_wrapper()
1504 ms 1 % check_deps()
728 ms 0 % Prepare deps
379 ms 0 % Init
296 ms 0 % Finalize
FFmpeg's configure doesn't work in MSYS2's dash out of the box.
mingw pkg-config outputs \r\n line endings, and while MSYS2's bash is patched
to handle them, MSYS2's dash is not, and requires to strip them, for instance
using this pkg-config-wrapper file:
```
#!/bin/sh
out=$("$MINGW_PREFIX"/bin/pkg-config "$@"; printf %s "-$?")
e=${out##*-}
out=${out%-*}
printf %s "$out" | tr -d '\r'
exit $e
```
and then invoke configure --pkg-config=path/to/pkg-config-wrapper ...
dash BEFORE:
497919 ms, 30 ms/measure, MINGW64_NT-6.3/dash
-----
337320 ms 68 % flatten_extralibs_wrapper()
129702 ms 26 % Compilation tests
18802 ms 4 % print_enabled_components()
6150 ms 1 % Writing config
3269 ms 1 % Options review
1707 ms 0 % check_deps()
400 ms 0 % Prepare deps
331 ms 0 % Init
238 ms 0 % Finalize
dash AFTER:
138592 ms, 29 ms/measure, MINGW64_NT-6.3/dash
-----
128895 ms 93 % Compilation tests
3067 ms 2 % Options review
2570 ms 2 % Writing config
1508 ms 1 % print_enabled_components()
1013 ms 1 % flatten_extralibs_wrapper()
425 ms 0 % Prepare deps
422 ms 0 % check_deps()
374 ms 0 % Finalize
318 ms 0 % Init
------------------------------------------
atomnuker:
Debian Unstable, Thinkpad X1 Carbon 4th Gen, Intel(R) Core(TM) i7-6600U CPU @
2.60GHz
dash: 16s -> 6.3s = x2.6
bash: 50s -> 8.3s = x6
------------------------------------------
dash BEFORE:
Command line: dash ./configure --disable-autodetect
-----
16132 ms, 2 ms/measure, Linux/dash
-----
8402 ms 52 % flatten_extralibs_wrapper()
5796 ms 36 % Compilation tests
1024 ms 6 % check_deps()
626 ms 4 % print_enabled_components()
129 ms 1 % Writing config
72 ms 0 % Options review
48 ms 0 % Prepare deps
30 ms 0 % Init
5 ms 0 % Finalize
dash AFTER:
Command line: dash ./configure --disable-autodetect
-----
6366 ms, 0 ms/measure, Linux/dash
-----
5661 ms 89 % Compilation tests
244 ms 4 % check_deps()
205 ms 3 % flatten_extralibs_wrapper()
90 ms 1 % Writing config
52 ms 1 % print_enabled_components()
49 ms 1 % Prepare deps
48 ms 1 % Options review
9 ms 0 % Init
8 ms 0 % Finalize
bash BEFORE:
Command line: bash ./configure --disable-autodetect
-----
50069 ms, 3 ms/measure, Linux/bash
-----
31212 ms 62 % flatten_extralibs_wrapper()
9665 ms 19 % check_deps()
6309 ms 13 % Compilation tests
1955 ms 4 % print_enabled_components()
591 ms 1 % Writing config
168 ms 0 % Prepare deps
99 ms 0 % Options review
61 ms 0 % Init
9 ms 0 % Finalize
bash AFTER:
-----
8279 ms, 3 ms/measure, Linux/bash
-----
6255 ms 76 % Compilation tests
623 ms 8 % check_deps()
432 ms 5 % flatten_extralibs_wrapper()
397 ms 5 % Writing config
208 ms 3 % print_enabled_components()
193 ms 2 % Prepare deps
102 ms 1 % Options review
61 ms 1 % Init
8 ms 0 % Finalize
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
