wdte: github.com/DeedleFake/wdte Index | Files | Directories

package wdte

import "github.com/DeedleFake/wdte"

Package wdte implements the WDTE scripting language.

WDTE is an embeddable, functionalish scripting language with a primary goal of simplicity of use from the embedding side, which is what this package provides.

Quick Language Overview

In order to understand how this package works, an overview of the language itself is first necessary. WDTE is functional-ish, with some emphasis on the "-ish". Although it generally follows a functional design, it is not purely functional. In WDTE, everything is a function in that everything can be "called", optionally with arguments. Value types return themselves, however, allowing them to be passed around.

WDTE contains a construct called a "compound" which is similar to a function body in most languages. It is surrounded by parentheses and contains a semicolon separated list of expressions, with the last semicolon being optional. The top-level of a WDTE script is a compound without the parentheses. When a compound is executed, each expression is evaluated in turn. If any yield an error, that error is immediately returned from the entire compound. If not, the result of the last expression is returned.

Example 1

# Declare a function called print3 that takes no arguments.
let print3 => (
  let x => 3;
  io.writeln io.stdout x;

There are very few functions built-in in WDTE, but the standard library, found in the std directory and its subdirectories, contains a number of useful functions and definitions. For example, the stream module contains iterator functionality, which provides a means of looping over expressions, something which is otherwise not possible.

Example 2

# Import 'stream' and 'array' and assign them to s and a,
# respectively. Note that an import is a compile-time operation,
# unlike normal functions. As such, it must be passed a string
# literal, not a variable.
let s => import 'stream';
let a => import 'arrays';

# Create a function called flatten that takes one argument,
# array.
let flatten array =>
  # Create a new stream that iterates over array.
  a.stream array

  # Create a stream from the previous one that performs a flat
  # map operation. The (@ name arg => ...) syntax is a lambda
  # declaration.
  -> s.flatMap (@ f v => v {
      # If the current element of the stream, v, is an array,
      # recursively flatten it into the stream.
      reflect 'Array' => a.stream v -> s.flatMap f;

  # Collect the previous stream into an array.
  -> s.collect

This example also demonstrates "chains" and "switches", some features that seem complicated at first but quickly become second nature so with some practice.

A chain is a series of expressions separated by either the chain operator, "->", or the ignored chain operator, "--". Each piece of the chain is executed in turn, and the output of the previous section is passed as an argument to the output of the current section. In other words, in the previous example, the chain's execution matches the following pseudocode

r1 = a.stream(array)
r2 = s.flatMap(<lambda>)
r1 = r2(r1)
r2 = s.collect
return r2(r1)

A chain with a use of "--" operates in much the same way, but the output of the piece of the chain immediately following the operator is ignored, meaning that it doesn't affect the remainder of the chain.

Chains can also have "slots" assigned to each piece. This is an identifier immediately following the expression of a piece of chain. This identifier is inserted into the scope for the remainder of the chain, allowing manual access to earlier sections of the chain. For example

let io => import 'io';
let file => import 'io/file';

let readFile path =>
  file.open path : f
  -> io.string
  -- io.close f

The aforementioned switch expression is the only conditional provided by WDTE. It looks like an expression followed by a semicolon separated series of cases in squiggly braces. A case is two expressions separated by the assignment operator, "=>". The original expression is first evaluated, following which each case's left-hand side is evaluated and the result of the original expression's evaluation is passed to it. If and only if this call results in the boolean value true, the right-hand side of that case is returned. If no cases match, the original expression is returned. For example,

func arg1 {
  lhs1 arg2 => rhs1 arg3;
  lhs2 arg4 => rhs2 arg6;

This is analogous to the following pseudocode

check = func(arg1)
if lhs := lhs1(arg2); lhs(check) {
  return rhs1(arg3)
if lhs := lhs2(arg4); lhs(check) {
  return rhs2(arg6)
return check

A few more minor points exist as well:

Array literals are a semicolon list of expression surrounded by
square brackets. Like in compounds and switches, the last
semicolon is optional.

Identifier parsing rules are very loose; essentially, anything
that isn't ambiguous with an existing keyword, operator, or
other syntactic construct is allowed.

All strings are essentially heredocs, allowing newlines like
they're any other character. There's no difference between
single-quoted and double-quoted strings.

There are no boolean literals, but the standard library provides
true and false functions that are essentially the same thing.


As previously mentioned, everything in WDTE is a function. In Go terms, everything in WDTE implements the Func type defined in this package. This includes syntactic constructs as well, such as compounds, switches, and chains.

When a script is parsed by one of the parsing functions in this package, it is translated into a recursive series of Func implementations. The specific types that it is translated to are all defined in and exported by this package. For example, the top-level of a script, being itself a compound, results in the instantiation of a Compound.

What this means in terms of embedding is that the only thing required for interaction between Go and WDTE is an interoperative layer of Func implementations. As a functional language, WDTE is stateless; there is no global interpreter state to keep track of at all. Systems for tracking interpreter state, should they be required, are provided by the repl package.

When a Func is called, it is passed a Frame. A Frame keeps track of anything the function needs that isn't directly an argument to the function. This includes the scope in which the Func call should be evaluated. For example, the expression

func arg1 arg2

translates to an instance of the FuncCall implementation of Func. When the FuncCall is "called", it must be given a scope which contains, at a minimum, "func", "arg1", and "arg2", or the call will fail with an error. It is through this mechanism that new functions can be provided to WDTE. A custom scope can be created with new implementations of Func inserted into it. If this scope is inserted into a Frame which is then passed to a call of, for example, the top-level compound created by parsing a script, they will be available during the evaluation.


const src = `
  let io => import 'io';
  io.writeln io.stdout example;

c, _ := wdte.Parse(strings.NewReader(src), std.Import)

scope := std.Scope.Add("example", wdte.String("This is an example."))
r := c.Call(std.F().WithScope(scope))
if err, ok := r.(error); ok {

This will print "This is an example." to stdout.

For convenience, a simple function wrapper around the single method required by Func is provided in the form of GoFunc. GoFunc provides a number of extra features, such as automatically converting panics into errors, but for the most part is just a simple wrapper around manual implementations of Func. If more automatic behavior is required, possibly at the cost of some runtime performance, functions for automatically wrapping Go functions are provided in the auto package.

One final note: WDTE is lazily-evaluated. Very, very lazily-evaluated. Until Go code manually calls a Func implementation, most WDTE code is never evaluated at all past the initial parsing. In some cases some code may get called more times than expected as well. Because of this, it is highly recommended that any Go code that is expected to be directly called by WDTE provide a purely functional interface, deterministically returning the same thing for the same arguments. If code does not follow this guideline, expect occasional odd behavior for seemingly no reason.


Package Files

doc.go translate.go types.go wdte.go


const (
    NormalChain  = 0
    IgnoredChain = 1 << (iota - 1)

func AssignPattern Uses

func AssignPattern(frame Frame, scope *Scope, ids []ID, val Func) (*Scope, Func)

AssignPattern performs a pattern matching assignment, placing values retrieved from an Atter into the corresponding provided IDs.

func AssignSimple Uses

func AssignSimple(frame Frame, scope *Scope, ids []ID, val Func) (*Scope, Func)

AssignSimple is an AssignFunc which places a single value into the scope with a single ID.

func Reflect Uses

func Reflect(f Func, name string) bool

Reflect checks if a Func can be considered to be of a given type. If v implements Reflector, v.Reflect(name) is used to check for compatability. If not, a simple string comparison is done against whatever Go's reflect package claims the short name of the underlying type to be.

type Array Uses

type Array []Func

An Array represents a WDTE array type. It's similar to a Compound, but when evaluated, it returns itself with its own members replaced with their own evaluations. This allows it to be passed around as a value in the same way as strings and numbers.

func (Array) At Uses

func (a Array) At(i Func) (Func, bool)

func (Array) Call Uses

func (a Array) Call(frame Frame, args ...Func) Func

func (Array) Len Uses

func (a Array) Len() int

func (Array) Reflect Uses

func (a Array) Reflect(name string) bool

func (Array) String Uses

func (a Array) String() string

type AssignFunc Uses

type AssignFunc func(Frame, *Scope, []ID, Func) (*Scope, Func)

AssignFunc places items into a scope. How exactly it does this differs, but the general idea is that it should return a scope which contains the IDs given with data somehow gotten from the provided Func, possibly involving calls using the given Frame. It returns the new scope and a Func. Ideally, this should be the Func that was originally provided, possibly wrapped in something, but it may not be.

In the event of an error, an AssignFunc should return a nil scope alongside the returned Func to indicate that it didn't simply store an error value in the scope, which would be completely valid.

type Assigner Uses

type Assigner struct {
    AssignFunc AssignFunc

    IDs  []ID
    Expr Func

An Assigner bundles a known list of IDs and an expression with an AssignFunc.

func (Assigner) Assign Uses

func (a Assigner) Assign(frame Frame, scope *Scope) (*Scope, Func)

func (Assigner) Call Uses

func (a Assigner) Call(frame Frame, args ...Func) Func

type Atter Uses

type Atter interface {
    // At returns the value at index i. If the index is out of range, it
    // should return false as its second return value.
    At(i Func) (Func, bool)

An Atter is a Func that can be indexed, like an array or a string.

type Bool Uses

type Bool bool

Bool is a boolean. Like other primitive types, it simply returns itself when called.

func (Bool) Call Uses

func (b Bool) Call(frame Frame, args ...Func) Func

func (Bool) Compare Uses

func (b Bool) Compare(other Func) (int, bool)

func (Bool) Reflect Uses

func (b Bool) Reflect(name string) bool

type Chain Uses

type Chain []*ChainPiece

Chain is an unevaluated chain expression.

func (Chain) Call Uses

func (f Chain) Call(frame Frame, args ...Func) Func

func (Chain) String Uses

func (f Chain) String() string

type ChainPiece Uses

type ChainPiece struct {
    Expr Func

    Flags      uint
    Slots      []ID
    AssignFunc AssignFunc

A ChainPiece is, as you can probably guess from the name, a piece of a Chain. It stores the underlying expression as well as some extra information necessary for properly evaluating the Chain.

func (ChainPiece) Call Uses

func (c ChainPiece) Call(frame Frame, args ...Func) Func

func (ChainPiece) String Uses

func (p ChainPiece) String() string

type Comparer Uses

type Comparer interface {
    // Compare returns two values. The meaning of the first is dependent
    // upon the second. If the second is true, then the first indicates
    // ordering via the standard negative, positive, and zero results to
    // indicate less than, greater than, and equal, respectively. If the
    // second is false, then the first indicates only equality, with
    // zero still meaning equal, but other values simply meaning unequal.
    Compare(other Func) (int, bool)

A Comparer is a Func that is able to be compared to other functions.

type Compound Uses

type Compound []Func

A Compound represents a compound expression. Calling it calls each of the expressions in the compound, returning the value of the last one. If the compound is empty, nil is returned.

If an element of a compound is an Assigner, it is used to build a new subscope under which the remainder of the elements of the compound will be evaluated. If the element is the last element of the compound, the Func returned by its assignment is returned from the whole compound.

func FromAST Uses

func FromAST(root ast.Node, im Importer) (Compound, error)

FromAST translates an AST into a top-level compound. im is used to handle import statements. If im is nil, a no-op importer is used.

func Parse Uses

func Parse(r io.Reader, im Importer, macros scanner.MacroMap) (Compound, error)

Parse parses an AST from r and then translates it into a top-level compound. im is used to handle import statements. If im is nil, a no-op importer is used. In most cases, std.Import is a good default.

func (Compound) Call Uses

func (c Compound) Call(frame Frame, args ...Func) Func

func (Compound) Collect Uses

func (c Compound) Collect(frame Frame) (letScope *Scope, last Func)

Collect executes the compound the same as Call, but also returns the collected scope that has been modified by let expressions alongside the usual return value. This is useful when dealing with scopes as modules, as it allows you to evaluate specific functions in a script.

type Error Uses

type Error struct {
    // Err is the error that generated the Error. In a lot of cases,
    // this is just a simple error message.
    Err error

    // Frame is the frame of the function that the error was first
    // generated in.
    Frame Frame

An Error is returned by any of the built-in functions when they run into an error.

func (Error) Call Uses

func (e Error) Call(frame Frame, args ...Func) Func

func (Error) Error Uses

func (e Error) Error() string

func (Error) Reflect Uses

func (e Error) Reflect(name string) bool

type Frame Uses

type Frame struct {
    // contains filtered or unexported fields

A Frame tracks information about the current function call, such as the scope that the function is being executed in and debugging info.

func F Uses

func F() Frame

F returns a top-level frame. This can be used by Go code calling WDTE functions directly if another frame is not available.

In many cases, it may be preferable to use std.F() instead.

func (Frame) Backtrace Uses

func (f Frame) Backtrace(w io.Writer) error

Backtrace prints a backtrace to w.

func (Frame) Context Uses

func (f Frame) Context() context.Context

func (Frame) ID Uses

func (f Frame) ID() ID

ID returns the ID of the frame. This is generally the function that created the frame.

func (Frame) Parent Uses

func (f Frame) Parent() Frame

Parent returns the frame that this frame was created from, or a blank frame if there was none.

func (Frame) Scope Uses

func (f Frame) Scope() *Scope

Scope returns the scope associated with the frame.

func (Frame) Sub Uses

func (f Frame) Sub(id ID) Frame

Sub returns a new child frame of f with the given ID and the same scope as f.

Under most circumstances, a GoFunc should call this before calling any WDTE functions, as it is useful for debugging. For example:

func Example(frame wdte.Frame, args ...wdte.Func) wdte.Func {
    frame = frame.Sub("example")

func (Frame) WithContext Uses

func (f Frame) WithContext(ctx context.Context) Frame

WithContext returns a copy of f with the given context.

func (Frame) WithScope Uses

func (f Frame) WithScope(scope *Scope) Frame

WithScope returns a copy of f with the given scope.

type Func Uses

type Func interface {
    // Call calls the function with the given arguments, returning its
    // return value. frame represents the current call frame, which
    // tracks scope as well as debugging info.
    Call(frame Frame, args ...Func) Func

Func is the base type through which all data is handled by WDTE. It represents everything that can be passed around in the language. This includes functions, of course, expressions, strings, numbers, Go functions, and anything else the client wants to pass into WDTE.

type FuncCall Uses

type FuncCall struct {
    Func Func
    Args []Func

A FuncCall is an unevaluated function call. This is usually the right-hand side of a function declaration, but could also be any of various pieces of switches, compounds, or arrays.

func (FuncCall) Call Uses

func (f FuncCall) Call(frame Frame, args ...Func) Func

func (FuncCall) String Uses

func (f FuncCall) String() string

type GoFunc Uses

type GoFunc func(frame Frame, args ...Func) Func

A GoFunc is an implementation of Func that calls a Go function. This is the easiest way to implement lower-level systems for WDTE scripts to make use of.

For example, to implement a simple, non-type-safe addition function:

GoFunc(func(frame wdte.Frame, args ...wdte.Func) wdte.Func {
  frame = frame.Sub("+")
  var sum wdte.Number
  for _, arg := range(args) {
    sum += arg.Call(frame).(wdte.Number)
  return sum

If placed into a scope with the ID "+", this function can then be called from WDTE as follows:

+ 3 6 9

As shown, it is recommended that arguments be passed the given frame when evaluating them. Failing to do so without knowing what you're doing can cause unexpected behavior, including sending the evaluation system into infinite loops or causing panics.

In the event that a GoFunc panics with an error value, it will be automatically caught and converted into an Error, which will then be returned.

func (GoFunc) Call Uses

func (f GoFunc) Call(frame Frame, args ...Func) (r Func)

func (GoFunc) String Uses

func (f GoFunc) String() string

type ID Uses

type ID string

ID represents a WDTE ID, such as a local variable.

type ImportFunc Uses

type ImportFunc func(from string) (*Scope, error)

ImportFunc is a wrapper around simple functions to allow them to be used as Importers.

func (ImportFunc) Import Uses

func (f ImportFunc) Import(from string) (*Scope, error)

type Importer Uses

type Importer interface {
    Import(from string) (*Scope, error)

An Importer creates scopes from strings. When parsing a WDTE script, an importer is used to import scopes into namespaces.

When the WDTE import expression

import 'example'

is parsed, the associated Importer will be invoked as follows:


type Lambda Uses

type Lambda struct {
    ID   ID
    Expr Func
    Args []ID

    Stored   []Func
    Scope    *Scope
    Original *Lambda

A Lambda is a closure. When called, it calls its inner expression with itself and its own arguments placed into the scope. In other words, given the lambda

(@ ex x y => + x y)

it will create a new subscope containing itself under the ID "ex", and its first and second arguments under the IDs "x" and "y", respectively. It will then evaluate `+ x y` in that new scope.

The arguments in the subscope, not including the self-reference, are contained in the boundary "args". The self-reference is contained in the boundary "self".

func (*Lambda) Call Uses

func (lambda *Lambda) Call(frame Frame, args ...Func) Func

func (*Lambda) String Uses

func (lambda *Lambda) String() string

type Lenner Uses

type Lenner interface {
    Len() int

A Lenner is a Func that has a length, such as arrays and strings.

type Memo Uses

type Memo struct {
    Func Func
    Args []ID
    // contains filtered or unexported fields

A Memo wraps another function, caching the results of calls with the same arguments.

func (*Memo) Call Uses

func (m *Memo) Call(frame Frame, args ...Func) Func

type Number Uses

type Number float64

A Number is a number, as parsed from a number literal. That's about it. Like everything else, it's a function. It simply returns itself when called.

func (Number) Call Uses

func (n Number) Call(frame Frame, args ...Func) Func

func (Number) Compare Uses

func (n Number) Compare(other Func) (int, bool)

func (Number) Reflect Uses

func (n Number) Reflect(name string) bool

func (Number) String Uses

func (n Number) String() string

type Reflector Uses

type Reflector interface {
    Reflect(name string) bool

A Reflector is a Func that can determine if it can be treated as the named type or not. For example,

s := wdte.String("example")
return s.Reflect("string")

returns true.

type Scope Uses

type Scope struct {
    // contains filtered or unexported fields

Scope is a tiered storage space for local variables. This includes function parameters and chain slots. A nil *Scope is equivalent to a blank, top-level scope.

func S Uses

func S() *Scope

S is a convenience function that returns a blank, top-level scope.

func (*Scope) Add Uses

func (s *Scope) Add(id ID, val Func) *Scope

Add returns a new subscope with the given variable stored in it.

func (*Scope) At Uses

func (s *Scope) At(i Func) (Func, bool)

func (*Scope) Call Uses

func (s *Scope) Call(frame Frame, args ...Func) Func

func (*Scope) Custom Uses

func (s *Scope) Custom(getFunc func(ID) Func, known func(map[ID]struct{})) *Scope

Custom returns a new subscope that uses the given lookup function to retrieve values. If getFunc returns nil, the parent of s will be searched. known is an optional function which adds all variables known to this layer of the scope into the map that it is passed as keys.

func (*Scope) Freeze Uses

func (s *Scope) Freeze(f Func) Func

Freeze returns a new function which executes in the scope s regardless of whatever Frame it is called with.

func (*Scope) Get Uses

func (s *Scope) Get(id ID) Func

Get returns the value of the variable with the given ID. If the variable doesn't exist in either the current scope or any of its parent scopes, nil is returned.

func (*Scope) Known Uses

func (s *Scope) Known() []ID

Known returns a sorted list of variables that are in scope.

func (*Scope) Map Uses

func (s *Scope) Map(vars map[ID]Func) *Scope

Map returns a subscope that includes the given mapping of variable names to functions. Note that no copy is made of vars, so changing the map after passing it to this method may result in undefined behavior.

func (*Scope) Parent Uses

func (s *Scope) Parent() *Scope

Parent returns the parent of the current scope.

func (*Scope) Reflect Uses

func (s *Scope) Reflect(name string) bool

func (*Scope) String Uses

func (s *Scope) String() string

func (*Scope) Sub Uses

func (s *Scope) Sub(sub *Scope) *Scope

Sub subscopes sub to s such that variables in sub will shadow variables in s.

type ScopedFunc Uses

type ScopedFunc struct {
    Func  Func
    Scope *Scope

A ScopedFunc is an expression that uses a predefined scope instead of the one that comes with its frame. This is to make sure that a lazily evaluated expression has access to the correct scope. It caches the result of its evaluation the first time it is evaluated, preventing expressions that are being lazily evaluated from being evaluated twice.

func (*ScopedFunc) Call Uses

func (f *ScopedFunc) Call(frame Frame, args ...Func) Func

func (ScopedFunc) String Uses

func (f ScopedFunc) String() string

type String Uses

type String string

A String is a string, as parsed from a string literal. That's about it. Like everything else, it's a function. It simply returns itself when called.

func (String) At Uses

func (s String) At(i Func) (Func, bool)

func (String) Call Uses

func (s String) Call(frame Frame, args ...Func) Func

func (String) Compare Uses

func (s String) Compare(other Func) (int, bool)

func (String) Len Uses

func (s String) Len() int

func (String) Reflect Uses

func (s String) Reflect(name string) bool

type Sub Uses

type Sub []Func

A Sub is a function that is in a subscope. This is most commonly an imported function.

func (Sub) Call Uses

func (sub Sub) Call(frame Frame, args ...Func) Func

type Switch Uses

type Switch struct {
    // Check is the condition at the front of the switch.
    Check Func

    // Cases is the switch's cases. Each contains two functions. The
    // first index is the left-hand side, while the second is the
    // right-hand side. When the switch is evaluated, the cases are run
    // in order. If any matches, the right-hand side is evaluated and
    // its return value is returned.
    Cases [][2]Func

Switch represents a switch expression.

func (Switch) Call Uses

func (s Switch) Call(frame Frame, args ...Func) Func

type Var Uses

type Var ID

A Var represents a local variable. When called, it looks itself up in the frame that it's given and calls whatever it finds.

func (Var) Call Uses

func (v Var) Call(frame Frame, args ...Func) Func


astPackage ast provides the parser for WDTE.
autoPackage auto provides higher-level automatic wrappers and convenience for Go/WDTE interoperability.
replPackage repl provides a layer intended to help with the development of a read-eval-print loop.
scannerPackage scanner provides a scanner for WDTE tokens.
stdPackage std provides a number of basic WDTE functions.
std/allPackage all is a convenience package that imports the entire standard library, thus registering it with std.Import.
std/arraysPackage arrays contains functions for manipulating arrays.
std/ioPackage io contains WDTE functions for dealing with files and other types of data streams.
std/io/filePackage file provides functions for dealing with files.
std/mathPackage math contains wdte.Funcs for performing mathematical operations.
std/streamPackage stream provides WDTE functions for manipulating streams of data.
std/stringsPackage strings contains functions for dealing with strings.

Package wdte imports 11 packages (graph) and is imported by 9 packages. Updated 2019-02-11. Refresh now. Tools for package owners.