Repository navigation
Description
Describe the bug
Disclosure: LLM assisted. As a quick summary I think this is very closely related to #9480 while the export ignore is supported in the git attribute file, some other entries changing line feed will still break the hash. below text are generated by LLM and I've reviewed them
When a .gitattributes rule rewrites a file on checkout, for example *.csv text eol=crlf, a clean git working tree and the commit it is at give different narHashes, and both reads are stored under the same fetcher cache key.
An input without rev or ref goes to getAccessorFromWorkdir (L1126-L1128), which reads the files from disk (L1031) and, for a clean tree, records rev = HEAD (L1072-L1081). Once the input has a rev, getFingerprint returns the bare rev (L1140-L1141). Fetching that rev goes through getAccessorFromCommit, which reads the blobs as stored, without checkout conversions. git reports the tree as clean because it compares after conversion, while the bytes differ: CRLF on disk, LF in the blob.
Whichever read runs first fills the cache entry for that rev, and the other read then returns it. On a machine with a warm cache the two agree. In a fresh process they do not, and a lock taken from the working tree fails once the locked rev is fetched. We hit this in CI: the job locks . from a fresh actions/checkout, then nix-eval-jobs workers fetch the locked rev and fail with mismatch in field 'narHash'.
Steps To Reproduce
#!/usr/bin/env bash
set -eu
NIX=${NIX:-nix}
repo=$(mktemp -d); cd "$repo"
git init -q
echo '*.csv text eol=crlf' > .gitattributes
printf 'a,b\n' > data.csv
git add .gitattributes data.csv
git -c user.name=x -c user.email=x@x commit -qm init
# check out again so git writes data.csv through the eol=crlf conversion
rm data.csv && git checkout -- data.csv
REV=$(git rev-parse HEAD)
echo "# git status --porcelain: '$(git status --porcelain)'"
echo "# CR bytes in the working tree: $(tr -cd '\r' < data.csv | wc -c), in the blob: $(git cat-file blob HEAD:data.csv | tr -cd '\r' | wc -c)"
workdir="{ type = \"git\"; url = \"file://$repo\"; }"
commit="{ type = \"git\"; url = \"file://$repo\"; rev = \"$REV\"; }"
f() { XDG_CACHE_HOME=$1 $NIX eval --impure --raw --expr "(builtins.fetchTree $2).narHash" 2>&1 | tail -1; echo; }
echo "# fresh cache, working tree:"; f "$(mktemp -d)" "$workdir"
echo "# fresh cache, rev:"; f "$(mktemp -d)" "$commit"
shared=$(mktemp -d)
echo "# shared cache, working tree then rev:"; f "$shared" "$workdir"; f "$shared" "$commit"
lock=$(XDG_CACHE_HOME=$(mktemp -d) $NIX eval --impure --raw --expr "(builtins.fetchTree $workdir).narHash")
echo "# fresh cache, rev locked to the working tree's narHash:"
f "$(mktemp -d)" "{ type = \"git\"; url = \"file://$repo\"; rev = \"$REV\"; narHash = \"$lock\"; }"Output with Nix 2.35.6 on aarch64-darwin (the temporary path is shortened, the rev differs per run):
warning: in the working copy of 'data.csv', LF will be replaced by CRLF the next time Git touches it
# git status --porcelain: ''
# CR bytes in the working tree: 1, in the blob: 0
# fresh cache, working tree:
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
# fresh cache, rev:
sha256-JWNvQHdj0Qbacc9bPp/ubGq4H6i0np3U8qh3Bzsd9X4=
# shared cache, working tree then rev:
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
# fresh cache, rev locked to the working tree's narHash:
error: NAR hash mismatch in input 'git+file:///tmp/tmp.XXXXXXXXXX?rev=ea8305db1d829386c0e148652ea522809842e297&shallow=1', expected 'sha256-JWNvQHdj0Qbacc9bPp/ubGq4H6i0np3U8qh3Bzsd9X4=' but got 'sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA='
The working tree read and the rev read differ with fresh caches. With a shared cache the rev read returns the working tree read's hash, and the reverse order gives the rev read's hash to both. A rev locked to the working tree's narHash fails.
Expected behavior
A clean working tree and its HEAD commit give the same tree, so git+file:///repo and git+file:///repo?rev=<HEAD> agree. Commit reads deliberately ignore checkout conversions (#11428), so reading a clean working tree through the commit path would keep the result independent of git attributes and configuration. At the least, a working tree read should not share a fetcher cache entry with the commit read.
Metadata
nix (Nix) 2.35.6 on aarch64-darwin. The same output on 2.36pre20261110 on aarch64-darwin, and the CI failure was on 2.35.3 on x86_64-linux.
Additional context
- NAR hash mismatch cloning git repository with crlfs introduced in 2.20 #11428: repositories with CRLF changed
narHashin 2.20, with the explanation that commit reads ignore local git configuration for purity - Local git rev fetch uses checked out .gitattributes #16556: a local rev fetch reads
export-ignorefrom the checked-out.gitattributesand leaves a wrong fetcher cache entry - nix flakes: git CRLF line endings are not handled #10640: CRLF line endings breaking flake builds on WSL
- Optionally allow git filters to be applied when reading blobs #13993: optionally applying git filters when reading blobs
Workaround: avoid .gitattributes rules that change content on checkout, such as eol=crlf, working-tree-encoding or smudge filters.
Checklist
- checked latest Nix manual (source)
- checked open bug issues and pull requests for possible duplicates
- reviewed the contributing guide
Activity