Hello,
I’m Ahmad Elmoursi, a student at KAIST doing security research at the
NetS&P Lab (https://netsp.kaist.ac.kr/ https://netsp.kaist.ac.kr/).
We found a bug in GNU grep’s --exclude-from handling that can cause
inconsistent file exclusions and an indefinite hang when using a FIFO as the
pattern source. We’ve attached the report and reproduction script with the
details.
Please let us know if this is a known issue or if you need any additional
information.
(Disclaimer: LLM was used to assist with writing the report and PoC)
Thank you,
Ahmad
# GNU grep `--exclude-from` reads FILE once per pattern table; the two tables filter different name populations from different reads, and a FIFO/producer-backed FILE causes a permanent hang
## Summary
`grep --exclude-from=FILE` opens and reads FILE **once per pattern table** — two reads per invocation (one `add_exclude_file()` call per table, `src/grep.c`). The two tables filter two different name populations: recursively discovered names and command-line operands. Because each table is built from its own independent read:
1. **Correctness**: the two populations can be governed by **different pattern sets** if the file content changes between the reads. The exclusion result then depends on file-state timing: an output impossible under any single consistent read is reachable. The implementation-supported `--exclude-from=-` (stdin) form is also broken: the second read gets an exhausted stdin, so command-line operands are never excluded while recursive ones are.
2. **Conditional permanent hang**: if FILE is a FIFO (or any producer-backed special file) whose writer exits before every open has been satisfied, grep blocks forever inside `open()` (kernel wait channel `wait_for_partner`), producing no output and never exiting.
By contrast, grep's own analogous option `-f PATTERNFILE` opens the file exactly once — the repeated reads are an implementation artifact, not a design decision.
## Affected versions
| Version | Result |
|---|---|
| GNU grep **3.12** (LLVM-instrumented build, Ubuntu 24.04 container) | **Affected** — FIFO serving test hangs (`wchan = wait_for_partner`); divergent-read outcomes confirmed |
| GNU grep **3.7** (Ubuntu 22.04) | **Affected** — strace shows two `openat()` of the exclude-from file; stdin and FIFO divergences reproduced |
The `for (int cmd = 0; cmd < 2; cmd++)` loop is still present in current master (`src/grep.c`, `case EXCLUDE_FROM_OPTION:`), so current releases are affected.
## Root cause
`src/grep.c`, option handling:
```c
case EXCLUDE_FROM_OPTION:
for (int cmd = 0; cmd < 2; cmd++)
{
if (!excluded_patterns[cmd])
excluded_patterns[cmd] = new_exclude ();
if (add_exclude_file (add_exclude, excluded_patterns[cmd],
optarg, exclude_options (cmd), '\n')
!= 0)
die (EXIT_TROUBLE, errno, "%s", optarg); /* blocks inside fopen() here */
}
break;
```
The two tables select **two different name populations** (`src/grep.c`):
```c
static int
exclude_options (bool command_line)
{
return EXCLUDE_WILDCARDS | (command_line ? 0 : EXCLUDE_ANCHORED);
}
static bool
skipped_file (char const *name, bool command_line, bool is_dir)
{
struct exclude **pats;
if (! is_dir)
pats = excluded_patterns; /* files */
...
return pats[command_line] && excluded_file_name (pats[command_line], name);
}
```
`skipped_file(name, command_line, …)` is called with `command_line = false` for entries discovered by the recursive traversal and with `command_line = true` for command-line operands. The intent of the loop is to populate both tables **from the same file snapshot**; the implementation re-reads the FILE for the second table instead of reusing the first read's content. Deviations:
- the recursion filter and the command-line filter can be built from **two different file states**, so identical names can be treated differently depending on how they were discovered (TOCTOU on the exclusion set);
- each `add_exclude_file` performs its own `fopen()` — for a FIFO or producer-backed FILE, an unsatisfiable open blocks indefinitely.
## Evidence
### E1 — open counts (strace, regular file, grep 3.7)
```
$ strace -f -e trace=openat grep -G -e needle --exclude-from=/wd/ex.file -r /wd/trees/tr
openat(AT_FDCWD, "/tmp/.../ex.file", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/tmp/.../ex.file", O_RDONLY|O_CLOEXEC) = 3 ← second open
```
`-f PATTERNFILE` under identical conditions: 1 open.
### E2 — `--exclude-from=-` : supported stream form (`-`) is broken for command-line operands (grep 3.7; `a.txt` contains `needle`)
```
$ printf '*.txt\n' | grep -e needle --exclude-from=- a.txt
needle one ← command-line operand NOT excluded (second read got EOF)
$ printf '*.txt\n' > f.reg
$ grep -e needle --exclude-from=f.reg a.txt
(no output, rc=1) ← the same patterns from a regular file DO exclude it
```
The first read populates the recursion table; the second read re-reads the exhausted stream and yields an empty command-line table.
### E3 — divergent snapshots: both operands suppressed, an impossible single-snapshot outcome (grep 3.7; recursive `trees/a.txt` and command-line `b.log`, both containing `needle`)
| read #1 → recursion table | read #2 → command-line table | output |
|---|---|---|
| `*.txt` | `*.txt` (consistent) | `b.log:needle two` |
| `*.log` | `*.log` (consistent) | `a.txt:needle one` |
| `*.txt` | `*.log` (divergent) | **`(no output) — both operands suppressed`** |
The divergent row suppresses both operands simultaneously — an outcome neither consistent read can produce, proving the two tables were built from two different snapshots.
### E4 — the conditional permanent hang (grep 3.7 and 3.12)
```
$ mkfifo patterns
$ (printf '*.log\n' > patterns &) # one-shot producer, exits
$ grep -r -e needle --exclude-from=patterns ./
<no output, no exit — hangs forever; wchan = wait_for_partner>
```
Blocked inside `open()`, not in `read()`: each pass requires a live writer rendezvous, and a producer that exits leaves the next open unsatisfied permanently. The hang is conditional on the producer's lifetime: with a serving peer that answers every open, grep completes normally.
## Impact
- **Correctness**: exclusion behavior depends on operand origin (recursive vs command-line) whenever the two reads differ — including the implementation-supported `--exclude-from=-` form, where command-line operands are never excluded at all. A file rewritten during startup is applied inconsistently across the two populations (TOCTOU on the exclusion set).
- **Availability**: permanent, silent hang for any user whose `--exclude-from` FILE is a FIFO or producer-backed special file with a one-shot producer.
- Not memory-unsafe; no data corruption beyond the exclusion semantics.
## Remediation
1. **Read once, populate both tables**: read FILE into memory (bounded), then run the two `add_exclude` passes over the snapshot — this matches the loop's evident intent and removes the divergence and the extra read.
2. Documentation: state in `grep(1)` how many times FILE is read and that producers must remain open; note the `--exclude-from=-` single-read consequence.
## Known reports
No prior report of this behavior is known: the double-read loop is still present in current master, the git history of `src/grep.c` contains no relevant change, and the issue tracker has no entry about exclude-from file handling, FIFOs, or repeated opens (all outstanding `grep` bugs reviewed as of 2026-10-09). `--exclude-from=-` (stdin) is supported by the implementation and verified working (the manual does not explicitly document `-` for this option), so stream inputs are within the option's intended scope.
## Reproduction
`./reproduce.sh [grep-binary]` — or manually:
```bash
mkfifo patterns
(printf '*.log\n' > patterns &) # one-shot producer exits after writing
grep -r -e needle --exclude-from=patterns . # hangs forever
```
Compare: `strace -e trace=openat grep -G -e needle --exclude-from=<regular file> -r dir` shows two `openat()` calls for the same path; `grep -G -f PATTERNFILE …` shows one.
#!/usr/bin/env bash
# Reproduction: GNU grep --exclude-from reads FILE once per pattern table
# [1] two opens (strace), vs one open for -f
# [2] --exclude-from=- : command-line operands never excluded (stream re-read)
# [3] divergent snapshots: both operands suppressed (impossible single-snapshot outcome)
# [4] FIFO permanent hang with one-shot producer
#
# Usage: ./reproduce.sh [grep-binary] (default: system grep)
set -u
GREP_BIN="${1:-/usr/bin/grep}"
WD=$(mktemp -d /tmp/grep-excl-repro.XXXXXX)
mkdir -p "$WD/trees"
printf 'needle one\n' > "$WD/trees/a.txt"
printf 'needle two\n' > "$WD/b.log"
echo "grep binary: $GREP_BIN"
"$GREP_BIN" --version | head -1
echo
echo "== [1] double open (strace; needs strace) =="
printf '*.log\n' > "$WD/ex.file"
if command -v strace >/dev/null; then
if strace -f -e trace=openat -o "$WD/excl.strace" \
"$GREP_BIN" -G -e needle --exclude-from="$WD/ex.file" -r "$WD/trees" >/dev/null 2>&1 \
&& [ -s "$WD/excl.strace" ]; then
echo " exclude-from openat count: $(grep -c "ex.file" "$WD/excl.strace") (-f patternfile opens once)"
else
echo " strace unavailable/blocked in this environment — open count not measured"
fi
else
echo " strace not installed — open count not measured"
fi
echo
echo "== [2] --exclude-from=- (stdin): command-line operand exclusion =="
printf '*.txt\n' > "$WD/ak.reg"
out=$(printf '*.txt\n' | "$GREP_BIN" -e needle --exclude-from=- "$WD/trees/a.txt")
echo " stream (-) : rc=$? out=${out:-<suppressed>}"
out=$("$GREP_BIN" -e needle --exclude-from="$WD/ak.reg" "$WD/trees/a.txt"); rc=$?
echo " regular file : rc=$rc out=${out:-<suppressed>}"
echo " -> same patterns: stream form leaves the command-line operand searched,"
echo " regular-file form excludes it. Inconsistent."
echo
echo "== [3] divergent snapshots: recursion vs command-line operands =="
python3 - "$GREP_BIN" "$WD" << 'PYEOF'
import os, subprocess, threading, sys, time
GREP, WD = sys.argv[1], sys.argv[2]
fifo = WD + "/divergent.fifo"
def run(name, first, after):
"""Serving peer: serves `first` on open #1 (recursion table), then `after`
on every subsequent open (command-line table). Persistent = deterministic.
Returns the set of files with output, or None if the case hung."""
if os.path.exists(fifo): os.unlink(fifo)
os.mkfifo(fifo)
p = subprocess.Popen([GREP, "-e", "needle", "--exclude-from=" + fifo,
"-r", "--", WD + "/trees", WD + "/b.log"],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stop = threading.Event()
def peer():
n = 0
while not stop.is_set() and p.poll() is None:
try:
fd = os.open(fifo, os.O_WRONLY)
if p.poll() is not None:
os.close(fd); return
os.write(fd, first if n == 0 else after)
os.close(fd)
except OSError:
return
n += 1
t = threading.Thread(target=peer, daemon=True); t.start()
try:
out, _ = p.communicate(timeout=20)
except subprocess.TimeoutExpired:
stop.set(); p.kill(); p.wait()
print(" [%s] -> HUNG (killed after 20s)" % name)
t.join(timeout=2)
return None
stop.set()
files = sorted(set(l.split(":")[0].split("/")[-1] for l in out.decode().strip().splitlines() if l))
print(" [%s] -> output from: %s" % (name, files or "NOTHING"))
t.join(timeout=2)
return files
a = run("open1=*.txt / rest=*.log", b"*.txt\n", b"*.log\n")
b = run("open1=*.log / rest=*.log", b"*.log\n", b"*.log\n")
if a is None or b is None:
print(" DEMONSTRATION INCONCLUSIVE: a case hung — rerun this section")
sys.exit(0)
print(" consistent read -> output from: %s" % (b or "NOTHING"))
print(" divergent reads -> output from: %s" % (a or "NOTHING"))
if a != b:
print(" same command; only the first read differed -> outputs differ =")
print(" the recursion and command-line tables were built from different snapshots")
else:
print(" no divergence observed in this run (racy environment) — rerun")
PYEOF
echo
echo "== [4] FIFO permanent hang with one-shot producer =="
mkfifo "$WD/patterns"
( printf '*.log\n' > "$WD/patterns" ) &
PROD=$!
timeout 8 "$GREP_BIN" -r -e needle --exclude-from="$WD/patterns" "$WD/trees" >/dev/null 2>&1
rc=$?
wait $PROD 2>/dev/null
if [ $rc -eq 124 ]; then
echo " HANG reproduced (timeout 8s): grep blocked in open() of the FIFO"
else
echo " no hang (rc=$rc)"
fi
echo
echo "workdir: $WD (left in place for inspection; safe to delete)"
