wdte: github.com/DeedleFake/wdte/scanner Index | Examples | Files

package scanner

import "github.com/DeedleFake/wdte/scanner"

Package scanner provides a scanner for WDTE tokens.



Package Files

doc.go scanner.go token.go util.go

type MacroMap Uses

type MacroMap map[string]func(string) ([]Token, error)

MacroMap specifies mappings of macro names to their definitions. For example, an entry with the key "example" would be available in scanned code as

@example[some input or another]

In this example, the string "some input or another" would be passed to the definition. The tokens returned by the macro are inserted raw into the token stream that is yielded by the scanner. Returned tokens of type Macro are reprocessed via the same map.

type Scanner Uses

type Scanner struct {
    // contains filtered or unexported fields

A Scanner tokenizes runes from an io.Reader.


s := scanner.New(r, nil)
for s.Scan() {
    /* Do something with s.Tok(). */
if err := s.Err(); err != nil {

func New Uses

func New(r io.Reader, macros MacroMap) *Scanner

New returns a new Scanner that reads from r. macros, which may be nil, specifies mappings of macro names to their definitions.

func (*Scanner) Err Uses

func (s *Scanner) Err() error

Err returns the error that stopped the scanner, if any.

func (*Scanner) Pos Uses

func (s *Scanner) Pos() (line, col int)

Pos returns the line and column of the input that the scanner is currently on.

func (*Scanner) Scan Uses

func (s *Scanner) Scan() bool

Scan reads the next token from the underlying io.Reader. If a token was successfully read, it returns true. It is designed to be used in a loop, similarly to bufio.Scanner's API.

func (*Scanner) Tok Uses

func (s *Scanner) Tok() Token

Tok returns the latest token scanned. If there was an error or a token hasn't been scanned yet, its return value is undefined.

type Token Uses

type Token struct {
    Line, Col int
    Type      TokenType
    Val       interface{}

A Token is a usable element parsed from a string.

type TokenType Uses

type TokenType uint

TokenType is the type of a token.

const (
    Invalid TokenType = iota // nolint

func (TokenType) String Uses

func (t TokenType) String() string

Package scanner imports 7 packages (graph) and is imported by 4 packages. Updated 2019-03-31. Refresh now. Tools for package owners.