Step 2: Implementation

Implement the custom tokens

Use the defineToken function to implement the tokens. This function takes the name of the token and its implementation and returns the implemented token.

Constant tokens

• Implement the constant function

• this in the function is a RegExpToken that contains the expression preceding the custom token

• The token can append to, wrap around, or modify this in any way

const severity = defineToken('severity', {
  constant(this: RegExpToken) {
    // Append a constant expression
    return this.oneOf`error``warning``info``debug`;
  },
});

const matchAll = defineToken('matchAll', {
  constant(this: RegExpToken) {
    // Wrap around the existing expression
    return lineStart.match(this).lineEnd;
  },
});

Dynamic tokens

• Implement the dynamic function

• this in the function is a RegExpToken that contains the expression preceding the custom token

• Token arguments are passed to the dynamic function as arguments

• Template string arguments are converted to ordinary strings automatically

const notExactly = defineToken('notExactly', {
  // Tagged template literals are converted to ordinary strings in the "value" argument
  dynamic(this: RegExpToken, value: string) {
    return this.notAhead(exactly(value)).repeat(value.length).notCharIn``;
  },
});

const exactValue = defineToken('exactValue', {
  // Implementation of function overloads
  dynamic(this: RegExpToken, num: number | boolean) {
    return this.exactly(String(num));
  },
});

Mixed tokens

• Implement both the constant and dynamic functions

• Same rules apply for both functions

• If the custom token is called, the dynamic function will handle the call. Otherwise, the constant function will be used.

const alpha = defineToken('alpha', {
  constant(this: RegExpToken) {
    return this.charIn`a-zA-Z`;
  },
  dynamic(this: RegExpToken, upper: boolean) {
    return upper ? this.charIn`A-Z` : this.charIn`a-z`;
  },
});