Poison

more

发表于

The ‘more’ command forces the lexer to get another token but without throwing out the current text. The token type will be that of the “final” rule matched (i.e., the one without a more or skip command).

TestParser.g4:

1
2
3
4
5
6
7
parser grammar TestParser;

options { tokenVocab=TestLexer; }

root
    : STRING EOF
    ;

TestLexer.g4:

1
2
3
4
5
6
7
8
lexer grammar TestLexer;

LQUOTE:        '"' -> more, mode(STR);
WS:            [ \r\t\n]+ -> skip;

mode STR;
STRING:        '"' -> mode(DEFAULT_MODE); // token we want parser to see
TEXT:          . -> more; // collect more text for string

Java Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import org.antlr.v4.runtime.CharStream;
import org.antlr.v4.runtime.CharStreams;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.Lexer;
import org.antlr.v4.runtime.tree.ParseTree;

import java.io.IOException;
import java.net.URISyntaxException;

public class TestParseTest {

    public static void main(String[] args) throws URISyntaxException, IOException {
        CharStream charStream = CharStreams.fromString("  \"abc\"  ");
        Lexer lexer = new TestLexer(charStream);

        CommonTokenStream tokens = new CommonTokenStream(lexer);
        TestParser parser = new TestParser(tokens);
        ParseTree parseTree = parser.root();
        System.out.println(parseTree.toStringTree(parser));
    }

}

Output:

1
(root "abc" <EOF>)
Reference

Lexer Rules - mode(), pushMode(), popMode, and more
antlr4/Lexer.java at 4.9.2 · antlr/antlr4 · GitHub