Hi Gábor,
On Tue, 17 Apr 2018, SZEDER Gábor wrote:
> Completion functions see all words on the command line verbatim,
> including any backslash-escapes, single and double quotes that might
> be there. Furthermore, git commands quote pathnames if they contain
> certain special characters. All these create various issues when
> doing git-aware path completion.
>
> Add a couple of failing tests to demonstrate these issues.
>
> Later patches in this series will discuss these issues in detail as
> they fix them.
>
> Signed-off-by: SZEDER Gábor <szeder....@gmail.com>
> ---
>
> Notes:
> Do any more new tests need FUNNYNAMES* prereq?
Yes.
> t/t9902-completion.sh | 91 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 91 insertions(+)
>
> diff --git a/t/t9902-completion.sh b/t/t9902-completion.sh
> index b7f5b1e632..ff2e4a8f5f 100755
> --- a/t/t9902-completion.sh
> +++ b/t/t9902-completion.sh
> @@ -1427,6 +1427,97 @@ test_expect_success 'complete files' '
> test_completion "git add mom" "momified"
> '
>
> +# The next tests only care about how the completion script deals with
> +# unusual characters in path names. By defining a custom completion
> +# function to list untracked files they won't be influenced by future
> +# changes of the completion functions of real git commands, and we
> +# don't have to bother with adding files to the index in these tests.
> +_git_test_path_comp ()
> +{
> + __git_complete_index_file --others
> +}
> +
> +test_expect_failure 'complete files - escaped characters on cmdline' '
> + test_when_finished "rm -rf \"New|Dir\"" &&
> + mkdir "New|Dir" &&
> + >"New|Dir/New&File.c" &&
> +
> + test_completion "git test-path-comp N" \
> + "New|Dir" && # Bash will turn this into "New\|Dir/"
> + test_completion "git test-path-comp New\\|D" \
> + "New|Dir" &&
> + test_completion "git test-path-comp New\\|Dir/N" \
> + "New|Dir/New&File.c" && # Bash will turn this into
> + # "New\|Dir/New\&File.c "
> + test_completion "git test-path-comp New\\|Dir/New\\&F" \
> + "New|Dir/New&File.c"
> +'
This fails with:
2018-04-18T11:12:55.0436371Z expecting success:
2018-04-18T11:12:55.0436665Z test_when_finished "rm -rf \"New|Dir\"" &&
2018-04-18T11:12:55.0436799Z mkdir "New|Dir" &&
2018-04-18T11:12:55.0436904Z >"New|Dir/New&File.c" &&
2018-04-18T11:12:55.0436972Z
2018-04-18T11:12:55.0437158Z test_completion "git test-path-comp N" \
2018-04-18T11:12:55.0437296Z "New|Dir" && # Bash will
turn this into "New\|Dir/"
2018-04-18T11:12:55.0437413Z test_completion "git test-path-comp New\\|D" \
2018-04-18T11:12:55.0437522Z "New|Dir" &&
2018-04-18T11:12:55.0437629Z test_completion "git test-path-comp
New\\|Dir/N" \
2018-04-18T11:12:55.0437767Z "New|Dir/New&File.c" && # Bash
will turn this into
2018-04-18T11:12:55.0438040Z #
"New\|Dir/New\&File.c "
2018-04-18T11:12:55.0438152Z test_completion "git test-path-comp
New\\|Dir/New\\&F" \
2018-04-18T11:12:55.0438504Z "New|Dir/New&File.c"
2018-04-18T11:12:55.0438742Z
2018-04-18T11:12:55.0590984Z ++ test_when_finished 'rm -rf "New|Dir"'
2018-04-18T11:12:55.0591722Z ++ test 0 = 0
2018-04-18T11:12:55.0592001Z ++ test_cleanup='{ rm -rf "New|Dir"
2018-04-18T11:12:55.0592290Z } && (exit "$eval_ret"); eval_ret=$?; :'
2018-04-18T11:12:55.0592472Z ++ mkdir 'New|Dir'
2018-04-18T11:12:55.0717255Z ++ test_completion 'git test-path-comp N' 'New|Dir'
2018-04-18T11:12:55.0717680Z ++ test 2 -gt 1
2018-04-18T11:12:55.0718062Z ++ printf '%s\n' 'New|Dir'
2018-04-18T11:12:55.0718275Z ++ run_completion 'git test-path-comp N'
2018-04-18T11:12:55.0718447Z ++ local -a COMPREPLY _words
2018-04-18T11:12:55.0718631Z ++ local _cword
2018-04-18T11:12:55.0718806Z ++ _words=($1)
2018-04-18T11:12:55.0718965Z ++ test N = ' '
2018-04-18T11:12:55.0719124Z ++ (( _cword = 3 - 1 ))
2018-04-18T11:12:55.0719286Z ++ __git_wrap__git_main
2018-04-18T11:12:55.0719467Z ++ __git_func_wrap __git_main
2018-04-18T11:12:55.0719633Z ++ local cur words cword prev
2018-04-18T11:12:55.0719801Z ++ _get_comp_words_by_ref -n =: cur words cword
prev
2018-04-18T11:12:55.0720074Z ++ '[' 6 -gt 0 ']'
2018-04-18T11:12:55.0720239Z ++ case "$1" in
2018-04-18T11:12:55.0720406Z ++ shift
2018-04-18T11:12:55.0720584Z ++ '[' 5 -gt 0 ']'
2018-04-18T11:12:55.0720742Z ++ case "$1" in
2018-04-18T11:12:55.0720899Z ++ shift
2018-04-18T11:12:55.0721054Z ++ '[' 4 -gt 0 ']'
2018-04-18T11:12:55.0721240Z ++ case "$1" in
2018-04-18T11:12:55.0721392Z ++ cur=N
2018-04-18T11:12:55.0721547Z ++ shift
2018-04-18T11:12:55.0721717Z ++ '[' 3 -gt 0 ']'
2018-04-18T11:12:55.0721879Z ++ case "$1" in
2018-04-18T11:12:55.0722040Z ++ words=("${_words[@]}")
2018-04-18T11:12:55.0722201Z ++ shift
2018-04-18T11:12:55.0722396Z ++ '[' 2 -gt 0 ']'
2018-04-18T11:12:55.0722931Z ++ case "$1" in
2018-04-18T11:12:55.0723070Z ++ cword=2
2018-04-18T11:12:55.0723221Z ++ shift
2018-04-18T11:12:55.0723357Z ++ '[' 1 -gt 0 ']'
2018-04-18T11:12:55.0723575Z ++ case "$1" in
2018-04-18T11:12:55.0723735Z ++ prev=test-path-comp
2018-04-18T11:12:55.0723874Z ++ shift
2018-04-18T11:12:55.0724009Z ++ '[' 0 -gt 0 ']'
2018-04-18T11:12:55.0724397Z ++ __git_main
2018-04-18T11:12:55.0724984Z ++ local i c=1 command __git_dir __git_repo_path
2018-04-18T11:12:55.0725183Z ++ local __git_C_args C_args_count=0
2018-04-18T11:12:55.0725353Z ++ '[' 1 -lt 2 ']'
2018-04-18T11:12:55.0725537Z ++ i=test-path-comp
2018-04-18T11:12:55.0725712Z ++ case "$i" in
2018-04-18T11:12:55.0725882Z ++ command=test-path-comp
2018-04-18T11:12:55.0726057Z ++ break
2018-04-18T11:12:55.0726270Z ++ '[' -z test-path-comp ']'
2018-04-18T11:12:55.0726446Z ++ __git_complete_command test-path-comp
2018-04-18T11:12:55.0726621Z ++ local command=test-path-comp
2018-04-18T11:12:55.0726816Z ++ local completion_func=_git_test_path_comp
2018-04-18T11:12:55.0726992Z ++ declare -f _git_test_path_comp
2018-04-18T11:12:55.0727353Z ++ declare -f _git_test_path_comp
2018-04-18T11:12:55.0727547Z ++ _git_test_path_comp
2018-04-18T11:12:55.0727716Z ++ __git_complete_index_file --others
2018-04-18T11:12:55.0727890Z ++ local dequoted_word pfx= cur_
2018-04-18T11:12:55.0728234Z ++ __git_dequote N
2018-04-18T11:12:55.0728418Z ++ local rest=N len ch
2018-04-18T11:12:55.0728869Z ++ dequoted_word=
2018-04-18T11:12:55.0729020Z ++ test -n N
2018-04-18T11:12:55.0729152Z ++ len=0
2018-04-18T11:12:55.0729309Z ++ dequoted_word=N
2018-04-18T11:12:55.0729440Z ++ rest=
2018-04-18T11:12:55.0729666Z ++ case "${rest:0:1}" in
2018-04-18T11:12:55.0729822Z ++ test -n ''
2018-04-18T11:12:55.0729993Z ++ case "$dequoted_word" in
2018-04-18T11:12:55.0730133Z ++ cur_=N
2018-04-18T11:12:55.0782504Z +++ __git_index_files --others '' N
2018-04-18T11:12:55.0782805Z +++ local root= match=N
2018-04-18T11:12:55.0845235Z +++ __git_ls_files_helper '' --others N
2018-04-18T11:12:55.0845440Z +++ '[' --others == --committable ']'
2018-04-18T11:12:55.0845567Z +++ __git -C '' -c core.quotePath=false ls-files
--exclude-standard --others -- 'N*'
2018-04-18T11:12:55.0845706Z +++ git -C '' -c core.quotePath=false ls-files
--exclude-standard --others -- 'N*'
2018-04-18T11:12:55.0907632Z +++ awk -F / -v pfx= '{
2018-04-18T11:12:55.0907806Z paths[$1] = 1
2018-04-18T11:12:55.0908985Z }
2018-04-18T11:12:55.0942839Z END {
2018-04-18T11:12:55.0943072Z for (p in paths) {
2018-04-18T11:12:55.0949175Z if (substr(p, 1, 1) != "\"") {
2018-04-18T11:12:55.0949458Z # No special
characters, easy!
2018-04-18T11:12:55.0949659Z print pfx p
2018-04-18T11:12:55.0949823Z continue
2018-04-18T11:12:55.0949999Z }
2018-04-18T11:12:55.0950121Z
2018-04-18T11:12:55.0950335Z # The path is quoted.
2018-04-18T11:12:55.0950829Z p = dequote(p)
2018-04-18T11:12:55.0951171Z if (p == "")
2018-04-18T11:12:55.0951555Z continue
2018-04-18T11:12:55.0951672Z
2018-04-18T11:12:55.0951856Z # Even when a directory name
itself does not contain
2018-04-18T11:12:55.0952038Z # any special characters, it
will still be quoted if
2018-04-18T11:12:55.0952213Z # any of its (stripped)
trailing path components do.
2018-04-18T11:12:55.0952407Z # Because of this we may have
seen the same direcory
2018-04-18T11:12:55.0952583Z # both quoted and unquoted.
2018-04-18T11:12:55.0952762Z if (p in paths)
2018-04-18T11:12:55.0952948Z # We have seen the same
directory unquoted,
2018-04-18T11:12:55.0953117Z # skip it.
2018-04-18T11:12:55.0953276Z continue
2018-04-18T11:12:55.0953441Z else
2018-04-18T11:12:55.0953613Z print pfx p
2018-04-18T11:12:55.0953766Z }
2018-04-18T11:12:55.0953914Z }
2018-04-18T11:12:55.0954461Z function dequote(p, bs_idx, out, esc,
esc_idx, dec) {
2018-04-18T11:12:55.0954650Z # Skip opening double quote.
2018-04-18T11:12:55.0954813Z p = substr(p, 2)
2018-04-18T11:12:55.0954935Z
2018-04-18T11:12:55.0955237Z # Interpret backslash escape sequences.
2018-04-18T11:12:55.0955415Z while ((bs_idx = index(p, "\\")) != 0) {
2018-04-18T11:12:55.0955533Z out = out substr(p, 1, bs_idx -
1)
2018-04-18T11:12:55.0955638Z esc = substr(p, bs_idx + 1, 1)
2018-04-18T11:12:55.0955743Z p = substr(p, bs_idx + 2)
2018-04-18T11:12:55.0955830Z
2018-04-18T11:12:55.0955939Z if ((esc_idx =
index("abtvfr\"\\", esc)) != 0) {
2018-04-18T11:12:55.0956079Z # C-style one-character
escape sequence.
2018-04-18T11:12:55.0956513Z out = out
substr("\a\b\t\v\f\r\"\\",
2018-04-18T11:12:55.0956631Z esc_idx, 1)
2018-04-18T11:12:55.0956745Z } else if (esc == "n") {
2018-04-18T11:12:55.0956853Z # Uh-oh, a newline
character.
2018-04-18T11:12:55.0956973Z # We cant reliably put
a pathname
2018-04-18T11:12:55.0957086Z # containing a newline
into COMPREPLY,
2018-04-18T11:12:55.0957193Z # and the newline would
create a mess.
2018-04-18T11:12:55.0957300Z # Skip this path.
2018-04-18T11:12:55.0957413Z return ""
2018-04-18T11:12:55.0957510Z } else {
2018-04-18T11:12:55.0957808Z # Must be a \nnn octal
value, then.
2018-04-18T11:12:55.0958070Z dec = esc * 64 + \
2018-04-18T11:12:55.0958184Z substr(p, 1, 1) *
8 + \
2018-04-18T11:12:55.0958274Z substr(p, 2, 1)
2018-04-18T11:12:55.0958369Z out = out sprintf("%c",
dec)
2018-04-18T11:12:55.0958587Z p = substr(p, 3)
2018-04-18T11:12:55.0958692Z }
2018-04-18T11:12:55.0958769Z }
2018-04-18T11:12:55.0958862Z # Drop closing double quote, if there
is one.
2018-04-18T11:12:55.0958969Z # (There isnt any if this is a
directory, as it was
2018-04-18T11:12:55.0959153Z # already stripped with the trailing
path components.)
2018-04-18T11:12:55.0959256Z if (substr(p, length(p), 1) == "\"")
2018-04-18T11:12:55.0959356Z out = out substr(p, 1,
length(p) - 1)
2018-04-18T11:12:55.0959441Z else
2018-04-18T11:12:55.0959541Z out = out p
2018-04-18T11:12:55.0959598Z
2018-04-18T11:12:55.0959682Z return out
2018-04-18T11:12:55.0959763Z }'
2018-04-18T11:12:55.1182135Z ++ __gitcomp_file_direct $'New∩\201╝Dir'
2018-04-18T11:12:55.1182355Z ++ local 'IFS=
2018-04-18T11:12:55.1182439Z '
2018-04-18T11:12:55.1182518Z ++ COMPREPLY=($1)
2018-04-18T11:12:55.1182622Z ++ compopt -o filenames +o nospace
2018-04-18T11:12:55.1182877Z ++ compgen -f /non-existing-dir/
2018-04-18T11:12:55.1182979Z ++ return 0
2018-04-18T11:12:55.1183055Z ++ return
2018-04-18T11:12:55.1183147Z ++ print_comp
2018-04-18T11:12:55.1183224Z ++ local 'IFS=
2018-04-18T11:12:55.1183300Z '
2018-04-18T11:12:55.1183398Z ++ echo $'New∩\201╝Dir'
2018-04-18T11:12:55.1183508Z ++ sort out
2018-04-18T11:12:55.1183605Z ++ /usr/bin/sort out
2018-04-18T11:12:55.1306331Z ++ test_cmp expected out_sorted
2018-04-18T11:12:55.1306825Z ++ mingw_test_cmp expected out_sorted
2018-04-18T11:12:55.1307024Z ++ local test_cmp_a= test_cmp_b=
2018-04-18T11:12:55.1307233Z ++ local stdin_for_diff=
2018-04-18T11:12:55.1307401Z ++ test -s expected
2018-04-18T11:12:55.1307568Z ++ test -s out_sorted
2018-04-18T11:12:55.1307742Z ++ mingw_read_file_strip_cr_ test_cmp_a
2018-04-18T11:12:55.1308083Z ++ local line
2018-04-18T11:12:55.1308424Z ++ :
2018-04-18T11:12:55.1308566Z ++ IFS=$'\r'
2018-04-18T11:12:55.1308717Z ++ read -r -d '
2018-04-18T11:12:55.1308852Z ' line
2018-04-18T11:12:55.1317521Z ++ line='New|Dir
2018-04-18T11:12:55.1317784Z '
2018-04-18T11:12:55.1318257Z ++ eval 'test_cmp_a=$test_cmp_a$line'
2018-04-18T11:12:55.1318424Z +++ test_cmp_a='New|Dir
2018-04-18T11:12:55.1318569Z '
2018-04-18T11:12:55.1318724Z ++ :
2018-04-18T11:12:55.1318871Z ++ IFS=$'\r'
2018-04-18T11:12:55.1319027Z ++ read -r -d '
2018-04-18T11:12:55.1319170Z ' line
2018-04-18T11:12:55.1319334Z ++ test -z ''
2018-04-18T11:12:55.1319476Z ++ break
2018-04-18T11:12:55.1319628Z ++ mingw_read_file_strip_cr_ test_cmp_b
2018-04-18T11:12:55.1319797Z ++ local line
2018-04-18T11:12:55.1319939Z ++ :
2018-04-18T11:12:55.1320081Z ++ IFS=$'\r'
2018-04-18T11:12:55.1320240Z ++ read -r -d '
2018-04-18T11:12:55.1320384Z ' line
2018-04-18T11:12:55.1320555Z ++ line='NewDir
2018-04-18T11:12:55.1320915Z '
2018-04-18T11:12:55.1321099Z ++ eval 'test_cmp_b=$test_cmp_b$line'
2018-04-18T11:12:55.1321266Z +++ test_cmp_b='NewDir
2018-04-18T11:12:55.1321422Z '
2018-04-18T11:12:55.1321570Z ++ :
2018-04-18T11:12:55.1321705Z ++ IFS=$'\r'
2018-04-18T11:12:55.1321859Z ++ read -r -d '
2018-04-18T11:12:55.1321994Z ' line
2018-04-18T11:12:55.1322219Z ++ test -z ''
2018-04-18T11:12:55.1322361Z ++ break
2018-04-18T11:12:55.1322497Z ++ test -n 'New|Dir
2018-04-18T11:12:55.1322649Z '
2018-04-18T11:12:55.1322828Z ++ test -n 'NewDir
2018-04-18T11:12:55.1322977Z '
2018-04-18T11:12:55.1323109Z ++ test 'New|Dir
2018-04-18T11:12:55.1323397Z ' = 'NewDir
2018-04-18T11:12:55.1323540Z '
2018-04-18T11:12:55.1323680Z ++ eval 'diff -u "$@" '
2018-04-18T11:12:55.1323840Z +++ diff -u expected out_sorted
2018-04-18T11:12:55.1454977Z --- expected 2018-04-18 11:12:55.065444100
+0000
2018-04-18T11:12:55.1455785Z error: last command exited with $?=1
2018-04-18T11:12:55.1456722Z +++ out_sorted 2018-04-18 11:12:55.127568400
+0000
2018-04-18T11:12:55.1457211Z @@ -1 +1 @@
2018-04-18T11:12:55.1457408Z -New|Dir
2018-04-18T11:12:55.1457752Z +NewDir
2018-04-18T11:12:55.1457975Z not ok 111 - complete files - escaped characters
on cmdline
2018-04-18T11:12:55.1645995Z #
2018-04-18T11:12:55.1646221Z # test_when_finished "rm -rf \"New|Dir\""
&&
2018-04-18T11:12:55.1646380Z # mkdir "New|Dir" &&
2018-04-18T11:12:55.1646487Z # >"New|Dir/New&File.c" &&
2018-04-18T11:12:55.1646583Z #
2018-04-18T11:12:55.1646865Z # test_completion "git test-path-comp N" \
2018-04-18T11:12:55.1646986Z # "New|Dir" && # Bash
will turn this into "New\|Dir/"
2018-04-18T11:12:55.1647108Z # test_completion "git test-path-comp
New\\|D" \
2018-04-18T11:12:55.1647212Z # "New|Dir" &&
2018-04-18T11:12:55.1647346Z # test_completion "git test-path-comp
New\\|Dir/N" \
2018-04-18T11:12:55.1647510Z # "New|Dir/New&File.c" && # Bash will turn this
into
2018-04-18T11:12:55.1647636Z # # "New\|Dir/New\&File.c "
2018-04-18T11:12:55.1647775Z # test_completion "git test-path-comp
New\\|Dir/New\\&F" \
2018-04-18T11:12:55.1647886Z # "New|Dir/New&File.c"
I suspect that the culprit is once again Cygwin's trick where illegal
characters are mapped into a private Unicode page. Cygwin (and therefore
MSYS2 runtime, and therefore the Bash used to run the test script) can use
those filenames all right, but Git cannot.
So even testing whether you could write an illegal file name via shell
script is *not* enough to determine whether the file system supports funny
characters.
As far as I can tell from a *really* cursory glance, this is the only
affected test case. Apparently your prereq catches, somehow, on Windows:
2018-04-18T11:12:43.0459702Z Your filesystem does not allow \ and " in
filenames.
2018-04-18T11:12:43.0459823Z skipped: complete files - C-style escapes in
ls-files output (missing FUNNYNAMES_BS_DQ)
Ciao,
Dscho
