For overview see:
http://ffmpeg.org/pipermail/ffmpeg-devel/2018-August/233665.html
Attached is patch 1/3 which was at "main.patch" of that message.
About 50-70% of configure runtime was being spent inside one
function: flatten_extralibs() and callees resolve() and unique().
It manipulates strings and invoked nearly 20K (20000) subshells.
It was rewritten to avoid subshells, and ended up x50-x250 faster.
unique() now outputs a different order:
it was keeping the last instance of recurring items, now it
keeps the first. It affects libs order at ffbuild/config.{mak,sh} -
but I don't think it matters. If it does, "opt1-reorder-unique.patch"
restores the original order. Let me know if/why it matters and I'll
squash it and update the commit message accordingly if required.
On Saturday, August 25, 2018 7:55 PM, Timo Rothenpieler
<t...@rothenpieler.org> wrote:
Please use git send-email to send your patches, or at least send each
patch, created by git format-patch, as individual attachment. Your files
seem to contain multiple patches one after another, which makes them
very hard to follow.
But nice work! Let's hope this does not cause any regressions.
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
From 7593d9f6580afd8a1066df1e1910a817d7c2fff1 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Mon, 30 Jul 2018 22:39:15 +0300
Subject: [PATCH] configure: flatten_extralibs_wrapper(): speedup x50 - x200
Currently configure spends 50-70% of its runtime inside a single
function: flatten_extralibs[_wrapper] - which does string processing.
During its run, nearly 20K command substitutions (subshells) are used,
including its callees unique() and resolve(), which is the reason
for its lengthy run.
This commit avoids all subshells during its execution, speeding it up
by about two orders of magnitude, and reducing the overall configure
runtime by 50-70% .
resolve() is rewritten to avoid subshells, and in unique() and
flatten_extralibs() we "inline" the filter[_out] functionality.
Note that unique() now produces different output order than before.
Previously it kept the last occurence of re-occuring items, and now
it keeps the first. However, this shouldn't make a difference as it's
used for unique library dependencies where the order doesn't matter.
---
configure | 33 ++++++++++++++++++++-------------
1 file changed, 20 insertions(+), 13 deletions(-)
diff --git a/configure b/configure
index b9c9d0b3..465d808d 100755
--- a/configure
+++ b/configure
@@ -845,21 +845,23 @@ prepend(){
}
unique(){
- var=$1
- uniq_list=""
- for tok in $(eval echo \$$var); do
- uniq_list="$(filter_out $tok $uniq_list) $tok"
+ unique_out=
+ eval unique_in=\$$1
+ for v in $unique_in; do
+ # " $unique_out" +space such that every item is surrounded with spaces
+ case " $unique_out" in *" $v "*) continue; esac # already in list
+ unique_out="$unique_out$v "
done
- eval "$var=\"${uniq_list}\""
+ eval $1=\$unique_out
}
resolve(){
- var=$1
- tmpvar=
- for entry in $(eval echo \$$var); do
- tmpvar="$tmpvar $(eval echo \$${entry})"
+ resolve_out=
+ eval resolve_in=\$$1
+ for v in $resolve_in; do
+ eval 'resolve_out="$resolve_out$'$v' "'
done
- eval "$var=\"${tmpvar}\""
+ eval $1=\$resolve_out
}
add_cppflags(){
@@ -6734,14 +6736,19 @@ if test $target_os = "haiku"; then
fi
flatten_extralibs(){
- unset nested_entries
+ nested_entries=
list_name=$1
eval list=\$${1}
for entry in $list; do
entry_copy=$entry
resolve entry_copy
- append nested_entries $(filter '*_extralibs' $entry_copy)
- flat_entries=$(filter_out '*_extralibs' $entry_copy)
+ flat_entries=
+ for e in $entry_copy; do
+ case $e in
+ *_extralibs) nested_entries="$nested_entries$e ";;
+ *) flat_entries="$flat_entries$e ";;
+ esac
+ done
eval $entry="\$flat_entries"
done
append $list_name "$nested_entries"
--
2.17.1
From 94b80da14fbca1296ac744a53581b817ae56d370 Mon Sep 17 00:00:00 2001
From: "Avi Halachmi (:avih)" <avih...@yahoo.com>
Date: Wed, 1 Aug 2018 09:06:36 +0300
Subject: [PATCH] configure: unique(): fixup to restore original output order
Originally unique() was keeping the last occurence of each non-unique
item, but commit XXX changed it to keep the first.
This commit restores the original order due to *TBD*.
---
configure | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/configure b/configure
index e9cb7703..ba123ead 100755
--- a/configure
+++ b/configure
@@ -850,14 +850,27 @@ prepend(){
eval "$var=\"$* \$$var\""
}
+reverse () {
+ eval '
+ reverse_out=
+ for v in $'$1'; do
+ reverse_out="$v $reverse_out"
+ done
+ '$1'=$reverse_out
+ '
+}
+
+# keeps the last occurence of each non-unique item
unique(){
unique_out=
eval unique_in=\$$1
+ reverse unique_in
for v in $unique_in; do
# " $unique_out" +space such that every item is surrounded with spaces
case " $unique_out" in *" $v "*) continue; esac # already in list
unique_out="$unique_out$v "
done
+ reverse unique_out
eval $1=\$unique_out
}
--
2.17.1
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel@ffmpeg.org
http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
