RTUComputer ScienceYr 2023 · Sem 52023

Q2Compiler Design

Question

2 marks

What is a Token?

Answer

A token is the smallest logical sequence of characters meaningful to a compiler.

In compiler design, a token represents the absolute smallest atomic unit of source code that possesses a distinct logical meaning. Tokens are generated during the lexical analysis phase. Common examples of tokens include language keywords (if, while), mathematical identifiers (variable names), logical operators (+, =), and punctuation symbols like semicolons.

Back to Paper