5c327502db
The following sequence leads to a "BUG" assertion running under MacOS:
DIR=git-test-restore-p
Adiarnfd=$(printf 'A\314\210')
DIRNAME=xx${Adiarnfd}yy
mkdir $DIR &&
cd $DIR &&
git init &&
mkdir $DIRNAME &&
cd $DIRNAME &&
echo "Initial" >file &&
git add file &&
echo "One more line" >>file &&
echo y | git restore -p .
Initialized empty Git repository in /tmp/git-test-restore-p/.git/
BUG: pathspec.c:495: error initializing pathspec_item
Cannot close git diff-index --cached --numstat
[snip]
The command `git restore` is run from a directory inside a Git repo.
Git needs to split the $CWD into 2 parts:
The path to the repo and "the rest", if any.
"The rest" becomes a "prefix" later used inside the pathspec code.
As an example, "/path/to/repo/dir-inside-repå" would determine
"/path/to/repo" as the root of the repo, the place where the
configuration file .git/config is found.
The rest becomes the prefix ("dir-inside-repå"), from where the
pathspec machinery expands the ".", more about this later.
If there is a decomposed form, (making the decomposing visible like this),
"dir-inside-rep°a" doesn't match "dir-inside-repå".
Git commands need to:
(a) read the configuration variable "core.precomposeunicode"
(b) precocompose argv[]
(c) precompose the prefix, if there was any
The first commit,
76759c7dff "git on Mac OS and precomposed unicode"
addressed (a) and (b).
The call to precompose_argv() was added into parse-options.c,
because that seemed to be a good place when the patch was written.
Commands that don't use parse-options need to do (a) and (b) themselfs.
The commands `diff-files`, `diff-index`, `diff-tree` and `diff`
learned (a) and (b) in
commit 90a78b83e0 "diff: run arguments through precompose_argv"
Branch names (or refs in general) using decomposed code points
resulting in decomposed file names had been fixed in
commit 8e712ef6fc "Honor core.precomposeUnicode in more places"
The bug report from above shows 2 things:
- more commands need to handle precomposed unicode
- (c) should be implemented for all commands using pathspecs
Solution:
precompose_argv() now handles the prefix (if needed), and is renamed into
precompose_argv_prefix().
Inside this function the config variable core.precomposeunicode is read
into the global variable precomposed_unicode, as before.
This reading is skipped if precomposed_unicode had been read before.
The original patch for preocomposed unicode, 76759c7dff, placed
precompose_argv() into parse-options.c
Now add it into git.c::run_builtin() as well. Existing precompose
calls in diff-files.c and others may become redundant, and if we
audit the callflows that reach these places to make sure that they
can never be reached without going through the new call added to
run_builtin(), we might be able to remove these existing ones.
But in this commit, we do not bother to do so and leave these
precompose callsites as they are. Because precompose() is
idempotent and can be called on an already precomposed string
safely, this is safer than removing existing calls without fully
vetting the callflows.
There is certainly room for cleanups - this change intends to be a bug fix.
Cleanups needs more tests in e.g. t/t3910-mac-os-precompose.sh, and should
be done in future commits.
[1] git-bugreport-2021-01-06-1209.txt (git can't deal with special characters)
[2] https://lore.kernel.org/git/A102844A-9501-4A86-854D-E3B387D378AA@icloud.com/
Reported-by: Daniel Troger <random_n0body@icloud.com>
Helped-By: Philippe Blain <levraiphilippeblain@gmail.com>
Signed-off-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
221 lines
6.3 KiB
Bash
Executable File
221 lines
6.3 KiB
Bash
Executable File
#!/bin/sh
|
|
#
|
|
# Copyright (c) 2012 Torsten Bögershausen
|
|
#
|
|
|
|
test_description='utf-8 decomposed (nfd) converted to precomposed (nfc)'
|
|
|
|
. ./test-lib.sh
|
|
|
|
if ! test_have_prereq UTF8_NFD_TO_NFC
|
|
then
|
|
skip_all="filesystem does not corrupt utf-8"
|
|
test_done
|
|
fi
|
|
|
|
# create utf-8 variables
|
|
Adiarnfc=$(printf '\303\204')
|
|
Adiarnfd=$(printf 'A\314\210')
|
|
|
|
Odiarnfc=$(printf '\303\226')
|
|
Odiarnfd=$(printf 'O\314\210')
|
|
AEligatu=$(printf '\303\206')
|
|
Invalidu=$(printf '\303\377')
|
|
|
|
|
|
#Create a string with 255 bytes (decomposed)
|
|
Alongd=$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd$Adiarnfd #21 Byte
|
|
Alongd=$Alongd$Alongd$Alongd #63 Byte
|
|
Alongd=$Alongd$Alongd$Alongd$Alongd$Adiarnfd #255 Byte
|
|
|
|
#Create a string with 254 bytes (precomposed)
|
|
Alongc=$AEligatu$AEligatu$AEligatu$AEligatu$AEligatu #10 Byte
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #50 Byte
|
|
Alongc=$Alongc$Alongc$Alongc$Alongc$Alongc #250 Byte
|
|
Alongc=$Alongc$AEligatu$AEligatu #254 Byte
|
|
|
|
test_expect_success "detect if nfd needed" '
|
|
precomposeunicode=$(git config core.precomposeunicode) &&
|
|
test "$precomposeunicode" = true &&
|
|
git config core.precomposeunicode true
|
|
'
|
|
test_expect_success "setup" '
|
|
>x &&
|
|
git add x &&
|
|
git commit -m "1st commit" &&
|
|
git rm x &&
|
|
git commit -m "rm x"
|
|
'
|
|
test_expect_success "setup case mac" '
|
|
git checkout -b mac_os
|
|
'
|
|
# This will test nfd2nfc in git diff
|
|
test_expect_success "git diff f.Adiar" '
|
|
touch f.$Adiarnfc &&
|
|
git add f.$Adiarnfc &&
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
git diff f.$Adiarnfd >expect &&
|
|
git diff f.$Adiarnfc >actual &&
|
|
test_cmp expect actual &&
|
|
git reset HEAD f.Adiarnfc &&
|
|
rm f.$Adiarnfc expect actual
|
|
'
|
|
# This will test nfd2nfc in git diff-files
|
|
test_expect_success "git diff-files f.Adiar" '
|
|
touch f.$Adiarnfc &&
|
|
git add f.$Adiarnfc &&
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
git diff-files f.$Adiarnfd >expect &&
|
|
git diff-files f.$Adiarnfc >actual &&
|
|
test_cmp expect actual &&
|
|
git reset HEAD f.Adiarnfc &&
|
|
rm f.$Adiarnfc expect actual
|
|
'
|
|
# This will test nfd2nfc in git diff-index
|
|
test_expect_success "git diff-index f.Adiar" '
|
|
touch f.$Adiarnfc &&
|
|
git add f.$Adiarnfc &&
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
git diff-index HEAD f.$Adiarnfd >expect &&
|
|
git diff-index HEAD f.$Adiarnfc >actual &&
|
|
test_cmp expect actual &&
|
|
git reset HEAD f.Adiarnfc &&
|
|
rm f.$Adiarnfc expect actual
|
|
'
|
|
# This will test nfd2nfc in readdir()
|
|
test_expect_success "add file Adiarnfc" '
|
|
echo f.Adiarnfc >f.$Adiarnfc &&
|
|
git add f.$Adiarnfc &&
|
|
git commit -m "add f.$Adiarnfc"
|
|
'
|
|
# This will test nfd2nfc in git diff-tree
|
|
test_expect_success "git diff-tree f.Adiar" '
|
|
echo f.Adiarnfc >>f.$Adiarnfc &&
|
|
git diff-tree HEAD f.$Adiarnfd >expect &&
|
|
git diff-tree HEAD f.$Adiarnfc >actual &&
|
|
test_cmp expect actual &&
|
|
git checkout f.$Adiarnfc &&
|
|
rm expect actual
|
|
'
|
|
# This will test nfd2nfc in git stage()
|
|
test_expect_success "stage file d.Adiarnfd/f.Adiarnfd" '
|
|
mkdir d.$Adiarnfd &&
|
|
echo d.$Adiarnfd/f.$Adiarnfd >d.$Adiarnfd/f.$Adiarnfd &&
|
|
git stage d.$Adiarnfd/f.$Adiarnfd &&
|
|
git commit -m "add d.$Adiarnfd/f.$Adiarnfd"
|
|
'
|
|
test_expect_success "add link Adiarnfc" '
|
|
ln -s d.$Adiarnfd/f.$Adiarnfd l.$Adiarnfc &&
|
|
git add l.$Adiarnfc &&
|
|
git commit -m "add l.Adiarnfc"
|
|
'
|
|
# This will test git log
|
|
test_expect_success "git log f.Adiar" '
|
|
git log f.$Adiarnfc > f.Adiarnfc.log &&
|
|
git log f.$Adiarnfd > f.Adiarnfd.log &&
|
|
test -s f.Adiarnfc.log &&
|
|
test -s f.Adiarnfd.log &&
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
'
|
|
# This will test git ls-files
|
|
test_expect_success "git lsfiles f.Adiar" '
|
|
git ls-files f.$Adiarnfc > f.Adiarnfc.log &&
|
|
git ls-files f.$Adiarnfd > f.Adiarnfd.log &&
|
|
test -s f.Adiarnfc.log &&
|
|
test -s f.Adiarnfd.log &&
|
|
test_cmp f.Adiarnfc.log f.Adiarnfd.log &&
|
|
rm f.Adiarnfc.log f.Adiarnfd.log
|
|
'
|
|
# This will test git mv
|
|
test_expect_success "git mv" '
|
|
git mv f.$Adiarnfd f.$Odiarnfc &&
|
|
git mv d.$Adiarnfd d.$Odiarnfc &&
|
|
git mv l.$Adiarnfd l.$Odiarnfc &&
|
|
git commit -m "mv Adiarnfd Odiarnfc"
|
|
'
|
|
# Files can be checked out as nfc
|
|
# And the link has been corrected from nfd to nfc
|
|
test_expect_success "git checkout nfc" '
|
|
rm f.$Odiarnfc &&
|
|
git checkout f.$Odiarnfc
|
|
'
|
|
# Make it possible to checkout files with their NFD names
|
|
test_expect_success "git checkout file nfd" '
|
|
rm -f f.* &&
|
|
git checkout f.$Odiarnfd
|
|
'
|
|
# Make it possible to checkout links with their NFD names
|
|
test_expect_success "git checkout link nfd" '
|
|
rm l.* &&
|
|
git checkout l.$Odiarnfd
|
|
'
|
|
test_expect_success "setup case mac2" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_2
|
|
'
|
|
# This will test nfd2nfc in git commit
|
|
test_expect_success "commit file d2.Adiarnfd/f.Adiarnfd" '
|
|
mkdir d2.$Adiarnfd &&
|
|
echo d2.$Adiarnfd/f.$Adiarnfd >d2.$Adiarnfd/f.$Adiarnfd &&
|
|
git add d2.$Adiarnfd/f.$Adiarnfd &&
|
|
git commit -m "add d2.$Adiarnfd/f.$Adiarnfd" -- d2.$Adiarnfd/f.$Adiarnfd
|
|
'
|
|
test_expect_success "setup for long decomposed filename" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_long_nfd_fn
|
|
'
|
|
test_expect_success "Add long decomposed filename" '
|
|
echo longd >$Alongd &&
|
|
git add * &&
|
|
git commit -m "Long filename"
|
|
'
|
|
test_expect_success "setup for long precomposed filename" '
|
|
git checkout master &&
|
|
git reset --hard &&
|
|
git checkout -b mac_os_long_nfc_fn
|
|
'
|
|
test_expect_success "Add long precomposed filename" '
|
|
echo longc >$Alongc &&
|
|
git add * &&
|
|
git commit -m "Long filename"
|
|
'
|
|
|
|
test_expect_failure 'handle existing decomposed filenames' '
|
|
echo content >"verbatim.$Adiarnfd" &&
|
|
git -c core.precomposeunicode=false add "verbatim.$Adiarnfd" &&
|
|
git commit -m "existing decomposed file" &&
|
|
git ls-files --exclude-standard -o "verbatim*" >untracked &&
|
|
test_must_be_empty untracked
|
|
'
|
|
|
|
test_expect_success "unicode decomposed: git restore -p . " '
|
|
DIRNAMEPWD=dir.Odiarnfc &&
|
|
DIRNAMEINREPO=dir.$Adiarnfc &&
|
|
export DIRNAMEPWD DIRNAMEINREPO &&
|
|
git init "$DIRNAMEPWD" &&
|
|
(
|
|
cd "$DIRNAMEPWD" &&
|
|
mkdir "$DIRNAMEINREPO" &&
|
|
cd "$DIRNAMEINREPO" &&
|
|
echo "Initial" >file &&
|
|
git add file &&
|
|
echo "More stuff" >>file &&
|
|
echo y | git restore -p .
|
|
)
|
|
'
|
|
|
|
# Test if the global core.precomposeunicode stops autosensing
|
|
# Must be the last test case
|
|
test_expect_success "respect git config --global core.precomposeunicode" '
|
|
git config --global core.precomposeunicode true &&
|
|
rm -rf .git &&
|
|
git init &&
|
|
precomposeunicode=$(git config core.precomposeunicode) &&
|
|
test "$precomposeunicode" = "true"
|
|
'
|
|
|
|
test_done
|