Home › GATE CSE › computerscience › Compiler Design › The scanner usually emits an EOF (end-of-file) t…
The scanner usually emits an EOF (end-of-file) token after the last real token. Why?
Ato mark file corruption
BEOF is never emitted; the parser checks input length
Cto track file size
Dto inform the parser to stop requesting more tokens, simplifying parser logic
Answer & Solution
Correct answer: D. to inform the parser to stop requesting more tokens, simplifying parser logic
1. Many parsers use a CURRENT TOKEN and check it against expected token types.
2. Without an EOF token, the parser would need a separate 'end of input' check — making code less uniform.
3. By emitting an EOF token, the parser can treat 'end of input' just like any other token. The parser knows it's done when it sees EOF after the top-level rule.
4. EOF also helps error recovery: the parser knows when there's nothing more to consume.
5. Other options are wrong.
_Source: Bob Nystrom, "Crafting Interpreters", Ch 4.4 (Scanner Class — EOF emission)._
Related questions
The FOLLOW(A) set in LL parsing isThe FIRST(α) set of a string α in LL parsing isAfter the parser produces an AST, the next compiler phase is typicallyA SHIFT/REDUCE CONFLICT in an LR parser occurs whenWhich of these is a BOTTOM-UP (LR) parser feature?An LL(1) grammar requires that, given the leftmost non-terminal and ONE token of lookaheadLL parsers (recursive descent) parse from thePANIC MODE error recovery in a parser works by