arsd.script

A small script interpreter that builds on arsd.jsvar to be easily embedded inside and to have has easy two-way interop with the host D program. The script language it implements is based on a hybrid of D and Javascript. The type the language uses is based directly on var from arsd.jsvar.

The interpreter is slightly buggy and poorly documented, but the basic functionality works well and much of your existing knowledge from Javascript will carry over, making it hopefully easy to use right out of the box. See the examples to quickly get the feel of the script language as well as the interop.

I haven't benchmarked it, but I expect it is pretty slow. My goal is to see what is possible for easy interoperability with dynamic functionality and D rather than speed.

Public Imports

arsd.jsvar
public import arsd.jsvar;
Undocumented in source.

Members

Aliases

helper
alias helper(alias T) = T
Undocumented in source.

Classes

ArrayLiteralExpression
class ArrayLiteralExpression
Undocumented in source.
AssertKeyword
class AssertKeyword
Undocumented in source.
AssignExpression
class AssignExpression
Undocumented in source.
BinaryExpression
class BinaryExpression
Undocumented in source.
BitFlipExpression
class BitFlipExpression
Undocumented in source.
BoolLiteralExpression
class BoolLiteralExpression
Undocumented in source.
CallExpression
class CallExpression
Undocumented in source.
CaseExpression
class CaseExpression
Undocumented in source.
CastExpression
class CastExpression
Undocumented in source.
DotVarExpression
class DotVarExpression
Undocumented in source.
ExceptionBlockExpression
class ExceptionBlockExpression
Undocumented in source.
Expression
class Expression
Undocumented in source.
FloatLiteralExpression
class FloatLiteralExpression
Undocumented in source.
ForExpression
class ForExpression
Undocumented in source.
ForeachExpression
class ForeachExpression
Undocumented in source.
FunctionDeclaration
class FunctionDeclaration
Undocumented in source.
FunctionLiteralExpression
class FunctionLiteralExpression
Undocumented in source.
IfExpression
class IfExpression
Undocumented in source.
IndexExpression
class IndexExpression
Undocumented in source.
IntLiteralExpression
class IntLiteralExpression
Undocumented in source.
LoopControlExpression
class LoopControlExpression
Undocumented in source.
MacroPrototype
class MacroPrototype
Undocumented in source.
MixinExpression
class MixinExpression
Undocumented in source.
NegationExpression
class NegationExpression
Undocumented in source.
NewExpression
class NewExpression
Undocumented in source.
NonScriptCatchableException
class NonScriptCatchableException

A base class for exceptions that can never be caught by scripts; throwing it from a function called from a script is guaranteed to bubble all the way up to your interpret call.. (scripts can also never catch Error btw)

NotExpression
class NotExpression
Undocumented in source.
NullLiteralExpression
class NullLiteralExpression
Undocumented in source.
ObjectLiteralExpression
class ObjectLiteralExpression
Undocumented in source.
OpAssignExpression
class OpAssignExpression
Undocumented in source.
ParentheticalExpression
class ParentheticalExpression
Undocumented in source.
PipelineExpression
class PipelineExpression
Undocumented in source.
ReturnExpression
class ReturnExpression
Undocumented in source.
ScopeExpression
class ScopeExpression
Undocumented in source.
ScriptCompileException
class ScriptCompileException

Thrown on script syntax errors and the sort.

ScriptException
class ScriptException

This represents an exception thrown by throw x; inside the script as it is interpreted.

ScriptFunctionMetadata
class ScriptFunctionMetadata
Undocumented in source.
ScriptRuntimeException
class ScriptRuntimeException

Thrown on things like interpretation failures.

ShallowCopyExpression
class ShallowCopyExpression
Undocumented in source.
SliceExpression
class SliceExpression
Undocumented in source.
StringLiteralExpression
class StringLiteralExpression
Undocumented in source.
SuperExpression
class SuperExpression
Undocumented in source.
SwitchExpression
class SwitchExpression
Undocumented in source.
TernaryExpression
class TernaryExpression
Undocumented in source.
ThrowExpression
class ThrowExpression
Undocumented in source.
TokenStream
class TokenStream(TextStream)
Undocumented in source.
VariableDeclaration
class VariableDeclaration
Undocumented in source.
VariableExpression
class VariableExpression
Undocumented in source.

Functions

addChildElementsOfExpressionToScriptExpressionObject
void addChildElementsOfExpressionToScriptExpressionObject(ClassInfo c, Expression _thisin, PrototypeObject sc, var obj)
Undocumented in source. Be warned that the author may not have intended to support it.
interpret
var interpret(string code, PrototypeObject variables, string scriptFilename)
Undocumented in source. Be warned that the author may not have intended to support it.
interpret
var interpret(string code, var variables, string scriptFilename, string file, size_t line)

This is likely your main entry point to the interpreter. It will interpret the script code given, with the given global variable object (which will be modified by the script, meaning you can pass it to subsequent calls to interpret to store context), and return the result of the last expression given.

interpretExpressions
var interpretExpressions(ExpressionStream expressions, PrototypeObject variables)
Undocumented in source. Be warned that the author may not have intended to support it.
interpretFile
var interpretFile(File file, var globals)
interpretStream
var interpretStream(MyTokenStreamHere tokens, PrototypeObject variables)
Undocumented in source. Be warned that the author may not have intended to support it.
interpretStream
var interpretStream(MyTokenStreamHere tokens, var variables)
Undocumented in source. Be warned that the author may not have intended to support it.
isCompatibleType
bool isCompatibleType(var v, string specifier, PrototypeObject sc)
Undocumented in source. Be warned that the author may not have intended to support it.
lexScript
TokenStream!TextStream lexScript(TextStream textStream, string scriptFilename)
Undocumented in source. Be warned that the author may not have intended to support it.
parseAddend
Expression parseAddend(MyTokenStreamHere tokens)
Undocumented in source. Be warned that the author may not have intended to support it.
parseArguments
Expression parseArguments(MyTokenStreamHere tokens, Expression exp, Expression[] where)
Undocumented in source. Be warned that the author may not have intended to support it.
parseCompoundStatement
CompoundStatementRange!MyTokenStreamHere parseCompoundStatement(MyTokenStreamHere tokens, int startingLine, string terminatingSymbol)
Undocumented in source. Be warned that the author may not have intended to support it.
parseExpression
Expression parseExpression(MyTokenStreamHere tokens, bool consumeEnd)
Undocumented in source. Be warned that the author may not have intended to support it.
parseFactor
Expression parseFactor(MyTokenStreamHere tokens)
Undocumented in source. Be warned that the author may not have intended to support it.
parseFunctionCall
Expression parseFunctionCall(MyTokenStreamHere tokens, Expression e)
Undocumented in source. Be warned that the author may not have intended to support it.
parsePart
Expression parsePart(MyTokenStreamHere tokens)
Undocumented in source. Be warned that the author may not have intended to support it.
parseScript
auto parseScript(MyTokenStreamHere tokens)
Undocumented in source. Be warned that the author may not have intended to support it.
parseStatement
Expression parseStatement(MyTokenStreamHere tokens, string terminatingSymbol)
Undocumented in source. Be warned that the author may not have intended to support it.
parseVariableDeclaration
VariableDeclaration parseVariableDeclaration(MyTokenStreamHere tokens, string termination)
Undocumented in source. Be warned that the author may not have intended to support it.
parseVariableName
VariableExpression parseVariableName(MyTokenStreamHere tokens)
Undocumented in source. Be warned that the author may not have intended to support it.
peekNextToken
bool peekNextToken(MyTokenStreamHere tokens, ScriptToken.Type type, string str, string file, size_t line)
Undocumented in source. Be warned that the author may not have intended to support it.
repl
void repl(var globals)

Enhanced repl uses arsd.terminal for better ux. Added April 26, 2020. Default just uses std.stdio.

requireNextToken
ScriptToken requireNextToken(MyTokenStreamHere tokens, ScriptToken.Type type, string str, string file, size_t line)
Undocumented in source. Be warned that the author may not have intended to support it.
toType
var toType(string specifier, PrototypeObject sc)
Undocumented in source. Be warned that the author may not have intended to support it.
toTypes
var[] toTypes(string[] specifiers, PrototypeObject sc)
Undocumented in source. Be warned that the author may not have intended to support it.

Structs

CompoundStatementRange
struct CompoundStatementRange(MyTokenStreamHere)
Undocumented in source.
InterpretResult
struct InterpretResult
Undocumented in source.
ScriptToken
struct ScriptToken
Undocumented in source.

Templates

CtList
template CtList(T...)
Undocumented in source.

Variables

AssertKeywordObject
PrototypeObject AssertKeywordObject;
Undocumented in source.
DefaultArgumentDummyObject
PrototypeObject DefaultArgumentDummyObject;
Undocumented in source.

Detailed Description

A goal of this language is to blur the line between D and script, but in the examples below, which are generated from D unit tests, the non-italics code is D, and the italics is the script. Notice how it is a string passed to the interpret function.

In some smaller, stand-alone code samples, there will be a tag "adrscript" in the upper right of the box to indicate it is script. Otherwise, it is D.

Installation instructions

This script interpreter is contained entirely in two files: jsvar.d and script.d. Download both of them and add them to your project. Then, import arsd.script;, declare and populate a var globals = var.emptyObject;, and interpret("some code", globals); in D.

There's nothing else to it, no complicated build, no external dependencies.

$ wget https://raw.githubusercontent.com/adamdruppe/arsd/master/script.d
$ wget https://raw.githubusercontent.com/adamdruppe/arsd/master/jsvar.d

$ dmd yourfile.d script.d jsvar.d

Script features

OVERVIEW

  • Can subclass D objects in script. See [http://dpldocs.info/this-week-in-d/Blog.Posted_2020_04_27.html#subclasses-in-script
  • easy interop with D thanks to arsd.jsvar. When interpreting, pass a var object to use as globals. This object also contains the global state when interpretation is done.
  • mostly familiar syntax, hybrid of D and Javascript
  • simple implementation is moderately small and fairly easy to hack on (though it gets messier by the day), but it isn't made for speed.

SPECIFICS

  • Allows identifiers starting with a dollar sign.
  • string literals come in "foo" or 'foo', like Javascript, or raw string like D. Also come as “nested “double quotes” are an option!”
  • double quoted string literals can do Ruby-style interpolation: "Hello, #{name}".
  • mixin aka eval (does it at runtime, so more like eval than mixin, but I want it to look like D)
  • scope guards, like in D
  • Built-in assert() which prints its source and its arguments
  • try/catch/finally/throw You can use try as an expression without any following catch to return the exception:
    var a = try throw "exception";; // the double ; is because one closes the try, the second closes the var
    // a is now the thrown exception
  • for/while/foreach
  • D style operators: +-/* on all numeric types, ~ on strings and arrays, |&^ on integers. Operators can coerce types as needed: 10 ~ "hey" == "10hey". 10 + "3" == 13. Any math, except bitwise math, with a floating point component returns a floating point component, but pure int math is done as ints (unlike Javascript btw). Any bitwise math coerces to int.

    So you can do some type coercion like this:

    a = a|0; // forces to int
    a = "" ~ a; // forces to string
    a = a+0.0; // coerces to float

    Though casting is probably better.

  • var a = "12";
    a.typeof == "String";
    a = cast(int) a;
    a.typeof == "Integral";
    a == 12;
    Type coercion via cast, similarly to D. Supported types for casting to: int/long (both actually an alias for long, because of how var works), float/double/real, string, char/dchar (these return *integral* types), and arrays, int[], string[], and float[].

    This forwards directly to the D function var.opCast.

  • some operator overloading on objects, passing opBinary(op, rhs), length, and perhaps others through like they would be in D. opIndex(name) opIndexAssign(value, name) // same order as D, might some day support [n1, n2] => (value, n1, n2)

    obj.__prop("name", value); // bypasses operator overloading, useful for use inside the opIndexAssign especially

    Note: if opIndex is not overloaded, getting a non-existent member will actually add it to the member. This might be a bug but is needed right now in the D impl for nice chaining. Or is it? FIXME

    FIXME: it doesn't do opIndex with multiple args.

  • if/else
  • array slicing, but note that slices are rvalues currently
  • variables must start with A-Z, a-z, _, or $, then must be [A-Za-z0-9_]*. (The $ can also stand alone, and this is a special thing when slicing, so you probably shouldn't use it at all.). Variable names that start with __ are reserved and you shouldn't use them.
  • int, float, string, array, bool, and #{} (previously known as json!q{} aka object) literals
  • var.prototype, var.typeof. prototype works more like Mozilla's __proto__ than standard javascript prototype.
  • the |> pipeline operator
  • // inheritance works
    class Foo : bar {
    	// constructors, D style
    	this(var a) { ctor.... }
    
    	// static vars go on the auto created prototype
    	static var b = 10;
    
    	// instance vars go on this instance itself
    	var instancevar = 20;
    
    	// "virtual" functions can be overridden kinda like you expect in D, though there is no override keyword
    	function virt() {
    		b = 30; // lexical scoping is supported for static variables and functions
    
    		// but be sure to use this. as a prefix for any class defined instance variables in here
    		this.instancevar = 10;
    	}
    }
    
    var foo = new Foo(12);
    
    foo.newFunc = function() { this.derived = 0; }; // this is ok too, and scoping, including 'this', works like in Javascript
    classes: You can also use 'new' on another object to get a copy of it.
  • return, break, continue, but currently cannot do labeled breaks and continues
  • __FILE__, __LINE__, but currently not as default arguments for D behavior (they always evaluate at the definition point)
  • most everything are expressions, though note this is pretty buggy! But as a consequence: for(var a = 0, b = 0; a < 10; a+=1, b+=1) {} won't work but this will: for(var a = 0, b = 0; a < 10; {a+=1; b+=1}) {}

    You can encase things in {} anywhere instead of a comma operator, and it works kinda similarly.

    {} creates a new scope inside it and returns the last value evaluated.

  • functions: var fn = function(args...) expr; or function fn(args....) expr;

    Special function local variables: _arguments = var[] of the arguments passed _thisfunc = reference to the function itself this = reference to the object on which it is being called - note this is like Javascript, not D.

    args can say var if you want, but don't have to default arguments supported in any position when calling, you can use the default keyword to use the default value in any position

  • macros: A macro is defined just like a function, except with the macro keyword instead of the function keyword. The difference is a macro must interpret its own arguments - it is passed AST objects instead of values. Still a WIP.

Todo list

I also have a wishlist here that I may do in the future, but don't expect them any time soon.

FIXME: maybe some kind of splat operator too. choose([1,2,3]...) expands to choose(1,2,3)

make sure superclass ctors are called

FIXME: prettier stack trace when sent to D

FIXME: support more escape things in strings like \n, \t etc.

FIXME: add easy to use premade packages for the global object.

FIXME: the debugger statement from javascript might be cool to throw in too.

FIXME: add continuations or something too - actually doing it with fibers works pretty well

FIXME: Also ability to get source code for function something so you can mixin.

FIXME: add COM support on Windows ????

Might be nice: varargs lambdas - maybe without function keyword and the x => foo syntax from D.

Examples

This example shows the basics of how to interact with the script. The string enclosed in q{ .. } is the script language source.

The var type comes from arsd.jsvar and provides a dynamic type to D. It is the same type used in the script language and is weakly typed, providing operator overloads to work with many D types seamlessly.

However, if you do need to convert it to a static type, such as if passing to a function, you can use get!T to get a static type out of it.

var globals = var.emptyObject;
globals.x = 25; // we can set variables on the global object
globals.name = "script.d"; // of various types
// and we can make native functions available to the script
globals.sum = (int a, int b) {
	return a + b;
};

// This is the source code of the script. It is similar
// to javascript with pieces borrowed from D, so should
// be pretty familiar.
string scriptSource = q{
	function foo() {
		return 13;
	}

	var a = foo() + 12;
	assert(a == 25);

	// you can also access the D globals from the script
	assert(x == 25);
	assert(name == "script.d");

	// as well as call D functions set via globals:
	assert(sum(5, 6) == 11);

	// I will also set a function to call from D
	function bar(str) {
		// unlike Javascript though, we use the D style
		// concatenation operator.
		return str ~ " concatenation";
	}
};

// once you have the globals set up, you call the interpreter
// with one simple function.
interpret(scriptSource, globals);

// finally, globals defined from the script are accessible here too:
// however, notice the two sets of parenthesis: the first is because
// @property is broken in D. The second set calls the function and you
// can pass values to it.
assert(globals.foo()() == 13);

assert(globals.bar()("test") == "test concatenation");

// this shows how to convert the var back to a D static type.
int x = globals.x.get!int;

Macros

Macros are like functions, but instead of evaluating their arguments at the call site and passing value, the AST nodes are passed right in. Calling the node evaluates the argument and yields the result (this is similar to to lazy parameters in D), and they also have methods like toSourceCode, type, and interpolate, which forwards to the given string.

The language also supports macros and custom interpolation functions. This example shows an interpolation string being passed to a macro and used with a custom interpolation string.

You might use this to encode interpolated things or something like that.

var globals = var.emptyObject;
interpret(q{
	macro test(x) {
		return x.interpolate(function(str) {
			return str ~ "test";
		});
	}

	var a = "cool";
	assert(test("hey #{a}") == "hey cooltest");
}, globals);

Classes demo

See also: arsd.jsvar.subclassable for more interop with D classes.

var globals = var.emptyObject;
interpret(q{
	class Base {
		function foo() { return "Base"; }
		function set() { this.a = 10; }
		function get() { return this.a; } // this MUST be used for instance variables though as they do not exist in static lookup
		function test() { return foo(); } // I did NOT use `this` here which means it does STATIC lookup!
						// kinda like mixin templates in D lol.
		var a = 5;
		static var b = 10; // static vars are attached to the class specifically
	}
	class Child : Base {
		function foo() {
			assert(super.foo() == "Base");
			return "Child";
		};
		function set() { this.a = 7; }
		function get2() { return this.a; }
		var a = 9;
	}

	var c = new Child();
	assert(c.foo() == "Child");

	assert(c.test() == "Base"); // static lookup of methods if you don't use `this`

	/*
	// these would pass in D, but do NOT pass here because of dynamic variable lookup in script.
	assert(c.get() == 5);
	assert(c.get2() == 9);
	c.set();
	assert(c.get() == 5); // parent instance is separate
	assert(c.get2() == 7);
	*/

	// showing the shared vars now.... I personally prefer the D way but meh, this lang
	// is an unholy cross of D and Javascript so that means it sucks sometimes.
	assert(c.get() == c.get2());
	c.set();
	assert(c.get2() == 7);
	assert(c.get() == c.get2());

	// super, on the other hand, must always be looked up statically, or else this
	// next example with infinite recurse and smash the stack.
	class Third : Child { }
	var t = new Third();
	assert(t.foo() == "Child");
}, globals);

Properties from D

Note that it is not possible yet to define a property function from the script language.

static class Test {
	// the @scriptable is required to make it accessible
	@scriptable int a;

	@scriptable @property int ro() { return 30; }

	int _b = 20;
	@scriptable @property int b() { return _b; }
	@scriptable @property int b(int val) { return _b = val; }
}

Test test = new Test;

test.a = 15;

var globals = var.emptyObject;
globals.test = test;
// but once it is @scriptable, both read and write works from here:
interpret(q{
	assert(test.a == 15);
	test.a = 10;
	assert(test.a == 10);

	assert(test.ro == 30); // @property functions from D wrapped too
	test.ro = 40;
	assert(test.ro == 30); // setting it does nothing though

	assert(test.b == 20); // reader still works if read/write available too
	test.b = 25;
	assert(test.b == 25); // writer action reflected

	// however other opAssign operators are not implemented correctly on properties at this time so this fails!
	//test.b *= 2;
	//assert(test.b == 50);
}, globals);

// and update seen back in D
assert(test.a == 10); // on the original native object
assert(test.b == 25);

assert(globals.test.a == 10); // and via the var accessor for member var
assert(globals.test.b == 25); // as well as @property func

Meta

History

September 1, 2020: added overloading for functions and type matching in catch blocks among other bug fixes

April 28, 2020: added #{} as an alternative to the json!q{} syntax for object literals. Also fixed unary ! operator.

April 26, 2020: added switch, fixed precedence bug, fixed doc issues and added some unittests

Started writing it in July 2013. Yes, a basic precedence issue was there for almost SEVEN YEARS. You can use this as a toy but please don't use it for anything too serious, it really is very poorly written and not intelligently designed at all.