Skip to content

Specify case insensitive matching of token upon definition #12

@ktsivkov

Description

@ktsivkov

First I want to say that I really like the api of this library, it is very easy to use.

What I want to propose:

Would be great if we could specify specific tokens to be matched case insensitive.
Lets say I have a token called WHERE, would be great if it also matched where.
i.e.
Tokenizer.DefineTokens(TOKEN_TYPE, []string{"WHERE"}, true)
or
Tokenizer.DefineInsensitiveTokens(TOKEN_TYPE, []string{"WHERE"})

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions