Complete Lexer Class

Building a Lexer Class

So far we've used a standalone function. Real interpreters use a Lexer class that encapsulates the scanning state:

class Lexer {
    constructor(input) {
        this.input = input;
        this.pos = 0;
        this.tokens = [];
    }

    peek() { return this.input[this.pos]; }
    advance() { return this.input[this.pos++]; }
    isAtEnd() { return this.pos >= this.input.length; }
}

Benefits

• Encapsulation: All state (pos, input) is contained
• Methods: peek(), advance(), isAtEnd() make scanning clear
• Extensibility: Easy to add new token types

Your Task

Build a Lexer class with a tokenize() method that returns all tokens. Support:

• NUMBER, STRING, IDENT
• PLUS, MINUS, STAR, SLASH, LPAREN, RPAREN
• EQUALS for =
• SEMICOLON for ;
• Whitespace (skip) and EOF

The class should be reusable: new Lexer(input).tokenize().