A scanner usually IGNORES which characters?
Aalphabetic letters of identifiers
Bdigits within numeric literals
Cwhitespace and comment characters
Doperator characters like + or *
Answer & Solution
Correct answer: C. whitespace and comment characters
1. WHITESPACE (spaces, tabs, newlines) generally has no syntactic meaning in C-family languages. Scanner consumes and discards.
2. COMMENTS: same — consumed without producing a token.
3. EXCEPTIONS:
- Python uses INDENTATION (whitespace) syntactically; the scanner must track it carefully.
- String literals contain whitespace meaningfully; the scanner preserves the contents.
4. Newlines may be relevant for STATEMENT TERMINATION (e.g. JS with auto-semicolon insertion, Lua).
5. Other options name characters the scanner DOES tokenise.
_Source: Bob Nystrom, "Crafting Interpreters", Ch 4.5 (Recognizing Lexemes — whitespace)._
Related questions
The FOLLOW(A) set in LL parsing isThe FIRST(α) set of a string α in LL parsing isAfter the parser produces an AST, the next compiler phase is typicallyA SHIFT/REDUCE CONFLICT in an LR parser occurs whenWhich of these is a BOTTOM-UP (LR) parser feature?An LL(1) grammar requires that, given the leftmost non-terminal and ONE token of lookaheadLL parsers (recursive descent) parse from thePANIC MODE error recovery in a parser works by