8000 Clean git working tree and its HEAD rev give different narHashes when .gitattributes converts files on checkout · Issue #16802 · NixOS/nix · GitHub
[go: up one dir, main page]

Skip to content

Clean git working tree and its HEAD rev give different narHashes when .gitattributes converts files on checkout #16802

Open

Description

@stepbrobd

Describe the bug

Disclosure: LLM assisted. As a quick summary I think this is very closely related to #9480 while the export ignore is supported in the git attribute file, some other entries changing line feed will still break the hash. below text are generated by LLM and I've reviewed them

When a .gitattributes rule rewrites a file on checkout, for example *.csv text eol=crlf, a clean git working tree and the commit it is at give different narHashes, and both reads are stored under the same fetcher cache key.

An input without rev or ref goes to getAccessorFromWorkdir (L1126-L1128), which reads the files from disk (L1031) and, for a clean tree, records rev = HEAD (L1072-L1081). Once the input has a rev, getFingerprint returns the bare rev (L1140-L1141). Fetching that rev goes through getAccessorFromCommit, which reads the blobs as stored, without checkout conversions. git reports the tree as clean because it compares after conversion, while the bytes differ: CRLF on disk, LF in the blob.

Whichever read runs first fills the cache entry for that rev, and the other read then returns it. On a machine with a warm cache the two agree. In a fresh process they do not, and a lock taken from the working tree fails once the locked rev is fetched. We hit this in CI: the job locks . from a fresh actions/checkout, then nix-eval-jobs workers fetch the locked rev and fail with mismatch in field 'narHash'.

Steps To Reproduce

#!/usr/bin/env bash
set -eu
NIX=${NIX:-nix}
repo=$(mktemp -d); cd "$repo"
git init -q
echo '*.csv text eol=crlf' > .gitattributes
printf 'a,b\n' > data.csv
git add .gitattributes data.csv
git -c user.name=x -c user.email=x@x commit -qm init
# check out again so git writes data.csv through the eol=crlf conversion
rm data.csv && git checkout -- data.csv
REV=$(git rev-parse HEAD)
echo "# git status --porcelain: '$(git status --porcelain)'"
echo "# CR bytes in the working tree: $(tr -cd '\r' < data.csv | wc -c), in the blob: $(git cat-file blob HEAD:data.csv | tr -cd '\r' | wc -c)"

workdir="{ type = \"git\"; url = \"file://$repo\"; }"
commit="{ type = \"git\"; url = \"file://$repo\"; rev = \"$REV\"; }"
f() { XDG_CACHE_HOME=$1 $NIX eval --impure --raw --expr "(builtins.fetchTree $2).narHash" 2>&1 | tail -1; echo; }

echo "# fresh cache, working tree:"; f "$(mktemp -d)" "$workdir"
echo "# fresh cache, rev:";          f "$(mktemp -d)" "$commit"
shared=$(mktemp -d)
echo "# shared cache, working tree then rev:"; f "$shared" "$workdir"; f "$shared" "$commit"

lock=$(XDG_CACHE_HOME=$(mktemp -d) $NIX eval --impure --raw --expr "(builtins.fetchTree $workdir).narHash")
echo "# fresh cache, rev locked to the working tree's narHash:"
f "$(mktemp -d)" "{ type = \"git\"; url = \"file://$repo\"; rev = \"$REV\"; narHash = \"$lock\"; }"

Output with Nix 2.35.6 on aarch64-darwin (the temporary path is shortened, the rev differs per run):

warning: in the working copy of 'data.csv', LF will be replaced by CRLF the next time Git touches it
# git status --porcelain: ''
# CR bytes in the working tree: 1, in the blob: 0
# fresh cache, working tree:
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
# fresh cache, rev:
sha256-JWNvQHdj0Qbacc9bPp/ubGq4H6i0np3U8qh3Bzsd9X4=
# shared cache, working tree then rev:
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA=
# fresh cache, rev locked to the working tree's narHash:
error: NAR hash mismatch in input 'git+file:///tmp/tmp.XXXXXXXXXX?rev=ea8305db1d829386c0e148652ea522809842e297&shallow=1', expected 'sha256-JWNvQHdj0Qbacc9bPp/ubGq4H6i0np3U8qh3Bzsd9X4=' but got 'sha256-Z0zekprFyU1nC4uk8oqPwigavXM/gSlGPEssDEQj4FA='

The working tree read and the rev read differ with fresh caches. With a shared cache the rev read returns the working tree read's hash, and the reverse order gives the rev read's hash to both. A rev locked to the working tree's narHash fails.

Expected behavior

A clean working tree and its HEAD commit give the same tree, so git+file:///repo and git+file:///repo?rev=<HEAD> agree. Commit reads deliberately ignore checkout conversions (#11428), so reading a clean working tree through the commit path would keep the result independent of git attributes and configuration. At the least, a working tree read should not share a fetcher cache entry with the commit read.

Metadata

nix (Nix) 2.35.6 on aarch64-darwin. The same output on 2.36pre20261110 on aarch64-darwin, and the CI failure was on 2.35.3 on x86_64-linux.

Additional context

Workaround: avoid .gitattributes rules that change content on checkout, such as eol=crlf, working-tree-encoding or smudge filters.

Checklist

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0