2008-10-17 Dynamic vs Lexical Scope, Implementing Lexical Scope, Functional Mappings

========================================================================
>>> Dynamic vs Lexical Scope

And back to the discussion of whether we should use dynamic or lexical
scope:

* The most important fact is that we want to view programs as executed
  by the normal substituting evaluator.  Our original motivation was to
  optimize evaluation only -- not to *change* the semantics!  It follows
  that we want the result of this optimization to behave in the same
  way.  All we need is to evaluate:

    (run "{with {x 3}
            {with {f {fun {y} {+ x y}}}
              {with {x 5}
                {call f 4}}}}")

  in the original evaluator to get convinced that 7 should be the
  correct result (note also that the same code, when translated into
  Scheme, evaluates to 7).

  (Yet, this is a very important optimization, which without it lots of
  programs become too slow to be feasible, so you might claim that
  you're fine with the modified semantics...)

* It does not allow using functions as objects, for example, we have
  seen that we have a functional representation for pairs:

    (define (kons x y)
      (lambda (n)
        (match n
          [1 x]
          [2 y]
          [else (error ...)])))

    (define my-pair (kons 1 2))

  If this is evaluated in a dynamically-scoped language, we do get a
  function as a result, but the values bound to x and y are now gone!
  Using the substitution model we substituted these values in, but now
  they were only held in a cache which no has no entries for them...

  In the same way, currying would not work, our nice `deriv' function
  would not work etc etc etc.

  (Try this in Emacs Lisp -- the last Lisp dialect that is in real use
  and uses dynamic scoping.)

* Makes reasoning impossible, because any piece of code behaves in a way
  that *cannot* be predicted until run-time.  For example, if dynamic
  scoping was used in Scheme, then you wouldn't be able to know what
  this function is doing:

    (define (foo)
      x)

  As it is, it will cause a run-time error, but if you call it like
  this:

    (let ([x 1])
      (foo))

  then it will return 1, and if you later do this:

    (define (bar x)
      (foo))

    (let ([x 1])
      (bar 2))

  then you would get 2!

  These problems can be demonstrated in Emacs Lisp too, but Scheme goes
  one step further -- it uses the same rule for evaluating a function as
  well as its values (Lisp uses a different name-space for functions).
  Because of this, you cannot even rely on the following function:

    (define (add x y)
      (+ x y))

  to always add x and y!  -- A similar example to the above:

    (let ([+ -])
      (add 1 2))

  would return -1!

* Many so-called "scripting" languages begin their lives with dynamic
  scoping.  The main reason, as we've seen, is that implementing it is
  extremely simple (no, *nobody* does substitution in the real world!
  (Well, *almost* nobody...)).

  Another reason is that these problems make life impossible if you want
  to use functions as object like you do in Scheme, so you notice them
  very fast -- but in a `normal' language without first-class functions,
  problems are not as obvious.

* For example, bash has `local' variables, but they have dynamic scope:

    x="the global x"
    print_x() { echo "The current value of x is \"$x\""; }
    foo() { local x="x from foo"; print_x; }
    print_x; foo; print_x

  Perl began its life with dynamic scope for variables that are declared
  `local':

    $x="the global x";
    sub print_x { print "The current value of x is \"$x\"\n"; }
    sub foo { local($x); $x="x from foo"; print_x; }
    print_x; foo; print_x;

  When faced with this problem, "the Perl way" was, obviously, not to
  remove or fix features, but to pile them up -- so local *still*
  behaves in this way, and now there is a `my' declaration which
  achieves proper lexical scope...

  There are other examples of languages that changed, and languages that
  want to change (e.g, nobody likes dynamic scope in Emacs Lisp, but
  there's just too much code now).

* This is still a tricky issue, like any other issue with bindings.  For
  example, googling got me quickly to this site:

    http://www.hetland.org/python/instant-python.php

  which is confused about what "dynamic scoping" is...  It claims that
  Python uses dynamic scope (Search for "Python uses dynamic as opposed
  to lexical scoping"), yet python always used lexical scope rules, as
  can be seen by translating their code to Scheme (ignore side-effects
  in this computation):

    (define (orange-juice)
      (* x 2))
    (define x 3)
    (define y (orange-juice)) ; y is now 6
    (define x 1)
    (define y (orange-juice)) ; y is now 2

  or by trying this in Python:

    def orange_juice():
      return x*2
    def foo(x):
      return orange_juice()
    foo(2)

  The real problem of python (pre 2.1, and pre 2.2 without the funny
    from __future__ import nested_scope
  line) is that it didn't create closures, which we will talk about
  shortly.

* Another example, which is an indicator of how easy it is to mess up
  your scope is the following Ruby bug (which I think is going to (or
  has been) fixed in a future version) -- running in `irb':

    winooski:~/csu660 eli> irb
    irb(main):001:0> x = 0
    => 0
    irb(main):002:0> lambda{|x| x}.call(5)
    => 5
    irb(main):003:0> x
    => 5

========================================================================
>>> Implementing Lexical Scope: Closures and Environments

So how do we fix this?

Lets go back to the root of the problem: the new evaluator does not
behave in the same way as the substituting evaluator.  In the old
evaluator, it was easy to see how functions can behave as objects that
remember values.  For example, when we do this:

  {with {x 1}
    {fun {y}
      {+ x y}}}

the result was a function value, which actually was the syntax object
for this:

  {fun {y} {+ 1 y}}

Now if we call this function from someplace else like:

  {with {f {with {x 1} {fun {y} {+ x y}}}}
    {with {x 2}
      {call f 3}}}

it is clear what the result will be: f is bound to a function that adds
1 to its input, so in the above the later binding for `x' has no effect
at all.

But with the caching evaluator, the value of

  {with {x 1}
    {fun {y}
      {+ x y}}}

is simply:

  {fun {y} {+ x y}}

and there is no place where we save the 1 -- *that's* the root of our
problem.  (That's also what makes people suspect that using `lambda' in
Scheme involves some inefficient code-recompiling magic.)  In fact, we
can verify that by inspecting the returned value, and see that it does
contain a free identifier.

Clearly, we need to create an object that contains the body and the
argument list, like the function syntax object -- but we don't do any
substitution, so in addition to the body an argument name(s) we need to
remember that we still need to substitute x by 1.  This means that the
pieces of information we need to know are:

  - formal argument(s):   {y}
  - body:                 {+ x y}
  - substitutions we owe: [1/x]

and that last bit has the missing 1.  The resulting object is called a
`closure' because it closes the function body over the substitutions
that are still pending (its environment).

So, the first change is in the value of functions which now need all
these pieces, unlike the `Fun' case for the syntax object.

A second place that needs changing is the when functions are called.
When we're done evaluating the `call' arguments (the function value and
the argument value) but before we apply the function we have two
*values* -- there is no more use for the current substitution cache at
this point: we have finished dealing with all substitutions that were
necessary over the current expression -- we now continue with evaluating
the body of the function, with the new substitutions for the formal
arguments and actual values given.  But the body itself is the same one
we had before -- which is the previous body with its suspended
substitutions that we *still* did not do.

Rewrite the evaluation rules -- all are the same except for evaluating a
`fun' form and a `call' form:

  eval(N,sc)                = N
  eval({+ E1 E2},sc)        = eval(E1,sc) + eval(E2,sc)
  eval({- E1 E2},sc)        = eval(E1,sc) - eval(E2,sc)
  eval({* E1 E2},sc)        = eval(E1,sc) * eval(E2,sc)
  eval({/ E1 E2},sc)        = eval(E1,sc) / eval(E2,sc)
  eval(x,sc)                = lookup(x,sc)
  eval({with {x E1} E2},sc) = eval(E2,extend(x,eval(E1,sc),sc))
  eval({fun {x} E},sc)      = <{fun {x} E},sc>
  eval({call E1 E2},sc1)
           = eval(Ef,extend(x,eval(E2,sc1),sc2))
                             if eval(E1,sc1)=<{fun {x} Ef},sc2>
           = error!          otherwise

(The "flat algorithm" for evaluating a `call' is therefore:
  1. f := evaluate E1 in sc1
  2. if f is not a <{fun ...},...> closure then error!
  3. a := evaluate E2 in sc1
  4. new_sc := extend sc_of(f) by mapping arg_of(f) to a
  5. evaluate (and return) body_of(f) in new_sc
)

Note how the scoping rules that are implied by this definition match the
scoping rules that were implied by the substitution-based rules.  (It
should be possible to prove that they are the same.)

The changes to the code are almost trivial, except that we need a way to
represent <{fun {x} Ef},sc> pairs.

========================================================================

The implication of this change is that we now cannot use the same type
for function syntax and function values since function values have more
than just syntax.  There is a simple solution to this -- we never do any
substitutions now, so we don't need to translate values into expressions
-- we can come up with a new type for values, separate from the type of
abstract syntax trees.

When we do this, we will also fix our hack of using FLANG as the type of
values: this was merely a convenience since the AST type had cases for
all kinds of values that we needed.  (In fact, you should have noticed
that Scheme does this too: numbers, strings, booleans, etc are all used
by both programs, and in syntax representation (s-expressions) -- but
note that procedure values are *not* used in syntax.)  We will now
implement a separate `VAL' type for runtime values.

As a side note, these substitution caches are a little more than just a
cache now -- they actually hold an "environment" of substitutions in
which the expression should be evaluated.  So the usual name used for
them is an environment, we will use this name.

First, we need now a type for such environments -- we can use `Listof'
for this:

  ;; a type for environments:
  (define-type ENV = (Listof (List Symbol VAL)))

but we can just as well define a new type for environment values:

  (define-type ENV
    [EmptyEnv]
    [Extend (id Symbol) (v VAL) (rest-env ENV)])

Reimplementing `lookup' is now simple:

  (: lookup : Symbol ENV -> VAL)
  (define (lookup name env)
    (cases env
      [(EmptyEnv) (error 'lookup "no binding for ~s" name)]
      [(Extend id val rest-env)
       (if (eq? id name) val (lookup name rest-env))]))

... we don't need `extend' because we get `Extend' from the type
definition, and we also get `(EmptyEnv)' instead of `empty-subst'.

We now use this with the new type for values -- two variants of these:

  (define-type VAL
    [NumV (n Number)]
    [FunV (name Symbol) (body FLANG) (env ENV)])

And now the new implementation of `eval' which uses the new type and
implements lexical scope:

  (: eval : FLANG ENV -> VAL)
  ;; evaluates FLANG expressions by reducing them to values
  (define (eval expr env)
    (cases expr
      [(Num n) (NumV n)]
      [(Add l r) (arith-op + (eval l env) (eval r env))]
      [(Sub l r) (arith-op - (eval l env) (eval r env))]
      [(Mul l r) (arith-op * (eval l env) (eval r env))]
      [(Div l r) (arith-op / (eval l env) (eval r env))]
      [(With bound-id named-expr bound-body)
       (eval bound-body
             (Extend bound-id (eval named-expr env) env))]
      [(Id name) (lookup name env)]
      [(Fun bound-id bound-body)
       (FunV bound-id bound-body env)]
      [(Call fun-expr arg-expr)
       (let ([fval (eval fun-expr env)])
         (cases fval
           [(FunV bound-id bound-body f-env)
            (eval bound-body
                  (Extend bound-id (eval arg-expr env) f-env))]
           [else (error 'eval "`call' expects a function, got: ~s"
                              fval)]))]))

We also need to update `arith-op' to use VAL objects.  The full code
follows -- it now passes all tests, including the example that we used
to find the problem.

  ---<<<FLANG-ENV>>>----------------------------------------------------
  ;; The Flang interpreter, using environments

  #lang CSU660

  #|
  The grammar:
    <FLANG> ::= <num>
              | { + <FLANG> <FLANG> }
              | { - <FLANG> <FLANG> }
              | { * <FLANG> <FLANG> }
              | { / <FLANG> <FLANG> }
              | { with { <id> <FLANG> } <FLANG> }
              | <id>
              | { fun { <id> } <FLANG> }
              | { call <FLANG> <FLANG> }

  Evaluation rules:
    eval(N,env)                = N
    eval({+ E1 E2},env)        = eval(E1,env) + eval(E2,env)
    eval({- E1 E2},env)        = eval(E1,env) - eval(E2,env)
    eval({* E1 E2},env)        = eval(E1,env) * eval(E2,env)
    eval({/ E1 E2},env)        = eval(E1,env) / eval(E2,env)
    eval(x,env)                = lookup(x,env)
    eval({with {x E1} E2},env) = eval(E2,extend(x,eval(E1,env),env))
    eval({fun {x} E},env)      = <{fun {x} E},env>
    eval({call E1 E2},env1)
             = eval(Ef,extend(x,eval(E2,env1),env2))
                               if eval(E1,env1)=<{fun {x} Ef},env2>
             = error!          otherwise
  |#

  (define-type FLANG
    [Num  (n Number)]
    [Add  (lhs FLANG) (rhs FLANG)]
    [Sub  (lhs FLANG) (rhs FLANG)]
    [Mul  (lhs FLANG) (rhs FLANG)]
    [Div  (lhs FLANG) (rhs FLANG)]
    [Id   (name Symbol)]
    [With (name Symbol) (named FLANG) (body FLANG)]
    [Fun  (name Symbol) (body FLANG)]
    [Call (fun-expr FLANG) (arg-expr FLANG)])

  (: parse-sexpr : Sexpr -> FLANG)
  ;; to convert s-expressions into FLANGs
  (define (parse-sexpr sexpr)
    (match sexpr
      [(number: n)    (Num n)]
      [(symbol: name) (Id name)]
      [(cons 'with more)
       (match sexpr
         [(list 'with (list (symbol: name) named) body)
          (With name (parse-sexpr named) (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
      [(cons 'fun more)
       (match sexpr
         [(list 'fun (list (symbol: name)) body)
          (Fun name (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
      [(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list 'call fun arg) (Call (parse-sexpr fun) (parse-sexpr arg))]
      [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

  (: parse : String -> FLANG)
  ;; parses a string containing an FLANG expression to a FLANG AST
  (define (parse str)
    (parse-sexpr (string->sexpr str)))

  ;; Types for environments, values, and a lookup function

  (define-type ENV
    [EmptyEnv]
    [Extend (id Symbol) (v VAL) (rest-env ENV)])

  (define-type VAL
    [NumV (n Number)]
    [FunV (name Symbol) (body FLANG) (env ENV)])

  (: lookup : Symbol ENV -> VAL)
  (define (lookup name env)
    (cases env
      [(EmptyEnv) (error 'lookup "no binding for ~s" name)]
      [(Extend id val rest-env)
       (if (eq? id name) val (lookup name rest-env))]))

  (: arith-op : (Number Number -> Number) VAL VAL -> VAL)
  ;; gets a Scheme numeric binary operator, and uses it within a NumV
  ;; wrapper
  (define (arith-op op val1 val2)
    (: NumV->number : VAL -> Number)
    (define (NumV->number v)
      (cases v
        [(NumV n) n]
        [else (error 'arith-op "expects a number, got: ~s" v)]))
    (NumV (op (NumV->number val1) (NumV->number val2))))

  (: eval : FLANG ENV -> VAL)
  ;; evaluates FLANG expressions by reducing them to values
  (define (eval expr env)
    (cases expr
      [(Num n) (NumV n)]
      [(Add l r) (arith-op + (eval l env) (eval r env))]
      [(Sub l r) (arith-op - (eval l env) (eval r env))]
      [(Mul l r) (arith-op * (eval l env) (eval r env))]
      [(Div l r) (arith-op / (eval l env) (eval r env))]
      [(With bound-id named-expr bound-body)
       (eval bound-body
             (Extend bound-id (eval named-expr env) env))]
      [(Id name) (lookup name env)]
      [(Fun bound-id bound-body)
       (FunV bound-id bound-body env)]
      [(Call fun-expr arg-expr)
       (let ([fval (eval fun-expr env)])
         (cases fval
           [(FunV bound-id bound-body f-env)
            (eval bound-body
                  (Extend bound-id (eval arg-expr env) f-env))]
           [else (error 'eval "`call' expects a function, got: ~s"
                              fval)]))]))

  (: run : String -> Number)
  ;; evaluate a FLANG program contained in a string
  (define (run str)
    (let ([result (eval (parse str) (EmptyEnv))])
      (cases result
        [(NumV n) n]
        [else (error 'run
                     "evaluation returned a non-number: ~s" result)])))

  ;; tests

  (test (run "{call {fun {x} {+ x 1}} 4}")
        => 5)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {call add3 1}}")
        => 4)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {with {add1 {fun {x} {+ x 1}}}
                  {with {x 3}
                    {call add1 {call add3 x}}}}}")
        => 7)
  (test (run "{with {identity {fun {x} x}}
                {with {foo {fun {x} {+ x 1}}}
                  {call {call identity foo} 123}}}")
        => 124)
  (test (run "{with {x 3}
                {with {f {fun {y} {+ x y}}}
                  {with {x 5}
                    {call f 4}}}}")
        => 7)
  (test (run "{call {call {fun {x} {call x 1}}
                          {fun {x} {fun {y} {+ x y}}}}
                    123}")
        => 124)
  ----------------------------------------------------------------------

========================================================================
>>> Implementing Lexical Scope using Scheme Closures and Environments

An alternative representation for an environment

We've already seen how first-class functions can be used to implement
"objects" that contain some information.  We can use the same idea to
represent an environment.  The basic intuition is -- an environment is a
*mapping* (a function) between an identifier and some value.  For
example, we can represent the environment that maps 'a to 1 and 'b to 2
(using just Scheme numbers for simplicity) using this function:

  (: my-map : Symbol -> Number)
  (define (my-map id)
    (cond [(eq? 'a id) 1]
          [(eq? 'b id) 2]
          [else (error ...)]))

An empty mapping that is implemented in this way has the same type:

  (: empty-mapping : Symbol -> Number)
  (define (empty-mapping id)
    (error ...))

We can use this idea to implement our environments: we only need to
define three things -- `EmptyEnv', `Extend', and `lookup'.  If we manage
to keep the contract to these functions intact, we will be able to
simply plug it into the same evaluator code with no other changes.  It
will also be more convenient to define `ENV' as the appropriate function
type for use in the VAL type definition instead of using the actual type:

  ;; Define a type for functional environments
  (define-type ENV = (Symbol -> VAL))

Now we get to `EmptyEnv' -- this is expected to be a procedure that
expects no arguments and creates an empty environment, one that behaves
like the `empty-mapping' procedure defined above.  We could define it
like this (changing the `empty-mapping' type to return a VAL):

  (define (EmptyEnv) empty-mapping)

but we can skip the need for an extra definition and simply return an
empty mapping procedure:

  (: EmptyEnv : -> ENV)
  (define (EmptyEnv)
    (lambda (id) (error ...)))

(The un-Schemely name is to avoid replacing previous code that used the
`EmptyEnv' name for the constructor that was created by the type
definition.)

The next thing we tackle is `lookup'.  The previous definition that was
used is:

  (: lookup : Symbol ENV -> VAL)
  (define (lookup name env)
    (cases env
      [(EmptyEnv) (error 'lookup "no binding for ~s" name)]
      [(Extend id val rest-env)
       (if (eq? id name) val (lookup name rest-env))]))

How should it be modified now?  Easy -- an environment is a mapping: a
Scheme procedure that will do the searching job itself.  We don't need
to modify the contract since we're still using `ENV', except a different
implementation for it.  The new definition is:

  (: lookup : Symbol ENV -> VAL)
  (define (lookup name env)
    (env name))

Note that `lookup' does almost nothing -- it simply delegates the real
work to the `env' argument.  This is a good hint for the error message
that empty mappings should throw --

  (: EmptyEnv : -> ENV)
  (define (EmptyEnv)
    (lambda (id) (error 'lookup "no binding for ~s" id)))

Finally, `Extend' -- this was previously created by the variant case of
the ENV type definition:

  [Extend (id Symbol) (v VAL) (rest-env ENV)]

keeping the same type that is implied by this variant means that the new
`Extend' should look like this:

  (: Extend : Symbol VAL ENV -> ENV)
  (define (Extend id v rest-env)
    ...)

The question is -- how do we extend a given environment?  Well, first,
we know that the result should be mapping -- a `symbol -> VAL' function
that expects an identifier to look for:

  (: Extend : Symbol VAL ENV -> ENV)
  (define (Extend id v rest-env)
    (lambda (name)
      ...))

Next, we know that in the generated mapping, if we look for `id' then
the result should be `v':

  (: Extend : Symbol VAL ENV -> ENV)
  (define (Extend id v rest-env)
    (lambda (name)
      (if (eq? name id)
        v
        ...)))

If the `name' that we're looking for is not the same as `id', then we
need to search through the previous environment, eg: (lookup name rest).
But we know what `lookup' does -- it simply delegates back to the
mapping function (which is our `rest' argument), so we can take a direct
route:

  (: Extend : Symbol VAL ENV -> ENV)
  (define (Extend id v rest-env)
    (lambda (name)
      (if (eq? name id)
        v
        (rest-env name))))

(Note that the last line is simply `(lookup name rest-env)', but we know
that we have a functional implementation.)

To see how all this works, try out extending an empty environment a few
times and examine the result.  For example, the environment that we
began with:

  (define (my-map id)
    (cond [(eq? 'a id) 1]
          [(eq? 'b id) 2]
          [else (error ...)]))

behaves in the same way (if the type of values is numbers) as

  (Extend 'a 1 (Extend 'b 2 (EmptyEnv)))

The new code is now the same, except for the environment code:

  ----------------------------------------------------------------------
  #lang CSU660

  #|
  The grammar:
    <FLANG> ::= <num>
              | { + <FLANG> <FLANG> }
              | { - <FLANG> <FLANG> }
              | { * <FLANG> <FLANG> }
              | { / <FLANG> <FLANG> }
              | { with { <id> <FLANG> } <FLANG> }
              | <id>
              | { fun { <id> } <FLANG> }
              | { call <FLANG> <FLANG> }

  Evaluation rules:
    eval(N,env)                = N
    eval({+ E1 E2},env)        = eval(E1,env) + eval(E2,env)
    eval({- E1 E2},env)        = eval(E1,env) - eval(E2,env)
    eval({* E1 E2},env)        = eval(E1,env) * eval(E2,env)
    eval({/ E1 E2},env)        = eval(E1,env) / eval(E2,env)
    eval(x,env)                = lookup(x,env)
    eval({with {x E1} E2},env) = eval(E2,extend(x,eval(E1,env),env))
    eval({fun {x} E},env)      = <{fun {x} E},env>
    eval({call E1 E2},env1)
             = eval(Ef,extend(x,eval(E2,env1),env2))
                               if eval(E1,env1)=<{fun {x} Ef},env2>
             = error!          otherwise
  |#

  (define-type FLANG
    [Num  (n Number)]
    [Add  (lhs FLANG) (rhs FLANG)]
    [Sub  (lhs FLANG) (rhs FLANG)]
    [Mul  (lhs FLANG) (rhs FLANG)]
    [Div  (lhs FLANG) (rhs FLANG)]
    [Id   (name Symbol)]
    [With (name Symbol) (named FLANG) (body FLANG)]
    [Fun  (name Symbol) (body FLANG)]
    [Call (fun-expr FLANG) (arg-expr FLANG)])

  (: parse-sexpr : Sexpr -> FLANG)
  ;; to convert s-expressions into FLANGs
  (define (parse-sexpr sexpr)
    (match sexpr
      [(number: n)    (Num n)]
      [(symbol: name) (Id name)]
      [(cons 'with more)
       (match sexpr
         [(list 'with (list (symbol: name) named) body)
          (With name (parse-sexpr named) (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
      [(cons 'fun more)
       (match sexpr
         [(list 'fun (list (symbol: name)) body)
          (Fun name (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
      [(list '+ lhs rhs) (Add (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '- lhs rhs) (Sub (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '* lhs rhs) (Mul (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list '/ lhs rhs) (Div (parse-sexpr lhs) (parse-sexpr rhs))]
      [(list 'call fun arg) (Call (parse-sexpr fun) (parse-sexpr arg))]
      [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

  (: parse : String -> FLANG)
  ;; parses a string containing an FLANG expression to a FLANG AST
  (define (parse str)
    (parse-sexpr (string->sexpr str)))

  ;; Types for environments, values, and a lookup function

  (define-type VAL
    [NumV (n Number)]
    [FunV (name Symbol) (body FLANG) (env ENV)])

  ;; Define a type for functional environments
  (define-type ENV = (Symbol -> VAL))

  (: EmptyEnv : -> ENV)
  (define (EmptyEnv)
    (lambda (id) (error 'lookup "no binding for ~s" id)))

  (: Extend : Symbol VAL ENV -> ENV)
  (define (Extend id v rest-env)
    (lambda (name)
      (if (eq? name id)
        v
        (rest-env name))))

  (: lookup : Symbol ENV -> VAL)
  (define (lookup name env)
    (env name))

  (: arith-op : (Number Number -> Number) VAL VAL -> VAL)
  ;; gets a Scheme numeric binary operator, and uses it within a NumV
  ;; wrapper
  (define (arith-op op val1 val2)
    (: NumV->number : VAL -> Number)
    (define (NumV->number v)
      (cases v
        [(NumV n) n]
        [else (error 'arith-op "expects a number, got: ~s" v)]))
    (NumV (op (NumV->number val1) (NumV->number val2))))

  (: eval : FLANG ENV -> VAL)
  ;; evaluates FLANG expressions by reducing them to values
  (define (eval expr env)
    (cases expr
      [(Num n) (NumV n)]
      [(Add l r) (arith-op + (eval l env) (eval r env))]
      [(Sub l r) (arith-op - (eval l env) (eval r env))]
      [(Mul l r) (arith-op * (eval l env) (eval r env))]
      [(Div l r) (arith-op / (eval l env) (eval r env))]
      [(With bound-id named-expr bound-body)
       (eval bound-body
             (Extend bound-id (eval named-expr env) env))]
      [(Id name) (lookup name env)]
      [(Fun bound-id bound-body)
       (FunV bound-id bound-body env)]
      [(Call fun-expr arg-expr)
       (let ([fval (eval fun-expr env)])
         (cases fval
           [(FunV bound-id bound-body f-env)
            (eval bound-body
                  (Extend bound-id (eval arg-expr env) f-env))]
           [else (error 'eval "`call' expects a function, got: ~s"
                              fval)]))]))

  (: run : String -> Number)
  ;; evaluate a FLANG program contained in a string
  (define (run str)
    (let ([result (eval (parse str) (EmptyEnv))])
      (cases result
        [(NumV n) n]
        [else (error 'run
                     "evaluation returned a non-number: ~s" result)])))

  ;; tests

  (test (run "{call {fun {x} {+ x 1}} 4}")
        => 5)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {call add3 1}}")
        => 4)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {with {add1 {fun {x} {+ x 1}}}
                  {with {x 3}
                    {call add1 {call add3 x}}}}}")
        => 7)
  (test (run "{with {identity {fun {x} x}}
                {with {foo {fun {x} {+ x 1}}}
                  {call {call identity foo} 123}}}")
        => 124)
  (test (run "{with {x 3}
                {with {f {fun {y} {+ x y}}}
                  {with {x 5}
                    {call f 4}}}}")
        => 7)
  (test (run "{call {call {fun {x} {call x 1}}
                          {fun {x} {fun {y} {+ x y}}}}
                    123}")
        => 124)
  ----------------------------------------------------------------------

========================================================================