2008-02-01 First Class Functions, Implementing Function Values

========================================================================
>>> Functions & First Class Function Values

Now that we have a form for local bindings, which forced us to deal with
proper substitutions and everything that is related, we can get to
functions.  The concept of a function is itself very close to
substitution, and to our `with' form.  For example, when we write:

  {with {x 5}
    {+ x x}}

then the "{+ x x}" body is itself parametrized over some value for `x'.
If we take this expression and take out the "5", we're left with
something that looks like a function:

  {with {x}
    {+ x x}}

We only need to replace `with' to indicate the difference from a real
`with' -- this form is missing a value:

  {fun {x}
    {+ x x}}

Now we have a new form in our language, one that should have a function
as its meaning.  As done with the `with' expression, we also need a form
to use these functions.  We can use `call' for this, so we want:

  {call {fun {x} {+ x x}} 5}

to be the same as the original thing we began with -- the `fun'
expression is like the `with' expression with no value, and applying it
on `5' is the same getting back to:

  {with {x 5} {+ x x}}

Of course, this did not help much.  So far, we just came up with a way
to use local bindings that is more verbose from what we started with.

What we're really missing is a way to name these functions.  If we get
the right evaluation rules, we can evaluate a `fun' expression to some
value -- which will allow us to bind it to a variable using `with'.
Something like this:

  {with {double {fun {x} {+ x x}}}
    {* {call double 5}
       {call double 6}}}

In this expression, we say that `x' is the formal parameter (or
argument), and the `5' and `6' are actual parameters (sometimes
abbreviated as formals and actuals).

[A different approach was used in the homework: a function definition is
much like a `with' form but without any value, and dealing with a
function application is much like dealing with a `with' form, except
that the value is from one place (the call site), and the body
expression and named variable are from a different place (the function
definition).]

========================================================================
>>> Implementing First Class Function Values

This is a simple plan, but it is directly related to how functions are
used in our language -- there are three basic approaches that classify
programming languages:

1. First order: functions are not real values.  They cannot be used or
   returned as values by other functions.  This means that they cannot
   be stored in data values.  This is what you are/will be implementing
   in homework 3, and what most `conventional' languages have.

2. Higher order: functions can receive and return other functions as
   values.  This is what you get in C.

3. First class: functions are values with all the rights of other
   values.  In particular, they can be supplied to other functions,
   returned from functions, stored in data structures, and new functions
   can be created at run-time.

The last category is the most interesting one.  Back in the old days,
complex expressions were not first-class in that they could not be
freely composed.  This is still the case in machine-code: as we've seen
earlier, to compute an expression such as

  (-b + sqrt(b^2 - 4*a*c)) / 2a

you have to do something like this:

  x = b * b
  y = 4 * a
  y = y * c
  x = x - y
  x = sqrt(x)
  y = -b
  x = y + x
  y = 2 * a
  s = x / y

In other words, every intermediate value needs to have its own name.
But with proper ("high-level") programming languages (at least most of
them...) you can just write the original expression, with no names for
these values.

With first-class functions something similar happens -- it is possible
to have complex expressions that consume and return functions, and they
do not need to be named.

What we get with our `fun' expression (if we can make it work) is
exactly this: it generates a function, and you can choose to either bind
it to a name, or not.

This has a major effect on the "personality" of a programming language
as we will see.  In fact, just adding this feature will make our
language much more advanced than some popular languages you know so far.

========================================================================

Now for the implementation -- we call this new language FLANG.

First, the BNF:

  <FLANG> ::= <num>
            | { + <FLANG> <FLANG> }
            | { - <FLANG> <FLANG> }
            | { * <FLANG> <FLANG> }
            | { / <FLANG> <FLANG> }
            | { with { <id> <FLANG> } <FLANG> }
            | <id>
            | { fun { <id> } <FLANG> }
            | { call <FLANG> <FLANG> }

And the matching type definition:

  (define-type FLANG
    [Num  (n Number)]
    [Add  (lhs FLANG) (rhs FLANG)]
    [Sub  (lhs FLANG) (rhs FLANG)]
    [Mul  (lhs FLANG) (rhs FLANG)]
    [Div  (lhs FLANG) (rhs FLANG)]
    [Id   (name Symbol)]
    [With (name Symbol) (named FLANG) (body FLANG)]
    [Fun  (name Symbol) (body FLANG)]
    [Call (fun-expr FLANG) (arg-expr FLANG)])

The parser for this grammar is, as usual, straightforward:

  (: parse-sexpr : (Sexpr -> FLANG))
  ;; to convert s-expressions into FLANGs
  (define (parse-sexpr sexpr)
    (match sexpr
      [(number: n)    (Num n)]
      [(symbol: name) (Id name)]
      [(cons 'with more)
       (match sexpr
         [(list 'with (list (symbol: name) named) body)
          (With name (parse-sexpr named) (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
      [(cons 'fun more)
       (match sexpr
         [(list 'fun (list (symbol: name)) body)
          (Fun name (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
      [(list op left right)
       (let ([make-node
              (match op
                ['+ Add]
                ['- Sub]
                ['* Mul]
                ['/ Div]
                ['call Call]
                [else (error 'parse-sexpr "don't know about ~s" op)])])
         (make-node (parse-sexpr left)
                    (parse-sexpr right)))]
      [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

We also need to patch up the substitution function to deal with these
things.  The scoping rule for the new function form is, unsurprisingly,
similar to the rule of `with', except that there is no extra expression
now, and the scoping rule for `call' is the same as for the arithmetic
operators:

    N[v/x]                = N

    {+ E1 E2}[v/x]        = {+ E1[v/x] E2[v/x]}

    {- E1 E2}[v/x]        = {- E1[v/x] E2[v/x]}

    {* E1 E2}[v/x]        = {* E1[v/x] E2[v/x]}

    {/ E1 E2}[v/x]        = {/ E1[v/x] E2[v/x]}

    y[v/x]                = y
    x[v/x]                = v

    {with {y E1} E2}[v/x] = {with {y E1[v/x]} E2[v/x]}
    {with {x E1} E2}[v/x] = {with {x E1[v/x]} E2}

    {call E1 E2}[v/x]     = {call E1[v/x] E2[v/x]}

    {fun {y} E}[v/x]      = {fun {y} E[v/x]}
    {fun {x} E}[v/x]      = {fun {x} E}

And the matching code:

  (: subst : (FLANG Symbol FLANG -> FLANG))
  ;; substitutes the second argument with the third argument in the
  ;; first argument, as per the rules of substitution; the resulting
  ;; expression contains no free instances of the second argument
  (define (subst expr from to)
    (cases expr
      [(Num n) expr]
      [(Add l r) (Add (subst l from to) (subst r from to))]
      [(Sub l r) (Sub (subst l from to) (subst r from to))]
      [(Mul l r) (Mul (subst l from to) (subst r from to))]
      [(Div l r) (Div (subst l from to) (subst r from to))]
      [(Id name) (if (eq? name from) to expr)]
      [(With bound-id named-expr bound-body)
       (With bound-id
             (subst named-expr from to)
             (if (eq? bound-id from)
               bound-body
               (subst bound-body from to)))]
      [(Call l r) (Call (subst l from to)
                        (subst r from to))]
      [(Fun bound-id bound-body)
       (if (eq? bound-id from)
         expr
         (Fun bound-id (subst bound-body from to)))]))

========================================================================

Now, before we start working on an evaluator, we need to decide on what
exactly do we use to represent values of this language.  Before we had
functions, we had only numbers and we used (Scheme) numbers to represent
them.  Now we have two kinds of values -- numbers and functions.  It
seems easy enough to continue using Scheme numbers to represent numbers,
but what about functions?  What should be the result of evaluating

  {fun {x} {+ x 1}}

?  Well, this is the new toy we have: it is a function *value*, which is
something that can be used just like numbers, but instead of arithmetic
operations, we can `call' these things.

To accommodate this, we will change our implementation strategy a
little: we will use our syntax objects for numbers (`(Num n)' instead of
just `n'), which will be a little inconvenient when we do the arithmetic
operations, but it will simplify life by making it possible to evaluate
functions in a similar way: simply return their own syntax object as
their values.  This means that evaluating:

  (Add (Num 1) (Num 2))

now yields

  (Num 3)

and a number `(Num 5)' evaluates to `(Num 5)'.

In a similar way, `(Fun 'x (Num 2))' evaluates to `(Fun 'x (Num 2))'.

Why would this work?  Well, because `call' will be very similar to
`with' -- the only difference is that its arguments are ordered a little
differently, being retrieved from the function that is applied and the
argument.

The formal evaluation rules are therefore treating functions like
numbers, and use the syntax object to represent both values:

  eval(N)         = N

  eval({+ E1 E2}) = eval(E1) + eval(E2)

  eval({- E1 E2}) = eval(E1) - eval(E2)

  eval({* E1 E2}) = eval(E1) * eval(E2)

  eval({/ E1 E2}) = eval(E1) / eval(E2)

  eval(id)        = error!

  eval({with {x E1} E2}) = eval(E2[eval(E1)/x])

  eval(FUN)       = FUN ; assuming FUN is a function expression

  eval({call E1 E2})
                  = eval(Ef[eval(E2)/x])   if eval(E1)={fun {x} Ef}
                  = error!                 otherwise

but note that we have two kinds of values, so we need to check the
arithmetic operation's arguments too:

  eval({+ E1 E2}) = eval(E1) + eval(E2)
                      if eval(E1) & eval(E2) are numbers
                      otherwise error!
  ...

And the corresponding code is:

  (: eval : (FLANG -> FLANG))                    ; <- note return type
  ;; evaluates FLANG expressions by reducing them to *expressions*
  (define (eval expr)
    (cases expr
      [(Num n) expr]                             ; <- change here
      [(Add l r) (arith-op + (eval l) (eval r))]
      [(Sub l r) (arith-op - (eval l) (eval r))]
      [(Mul l r) (arith-op * (eval l) (eval r))]
      [(Div l r) (arith-op / (eval l) (eval r))]
      [(With bound-id named-expr bound-body)
       (eval (subst bound-body
                    bound-id
                    (eval named-expr)))]         ; <- no `(Num ...)'
      [(Id name) (error 'eval "free identifier: ~s" name)]
      [(Fun bound-id bound-body) expr]           ; <- similar to `Num'
      [(Call (Fun bound-id bound-body) arg-expr) ; <- nested pattern
       (eval (subst bound-body                   ; <- just like `with'
                    bound-id
                    (eval arg-expr)))]
      [(Call something arg-expr)
       (error 'eval "`call' expects a function, got: ~s" something)]))

and we can make it easier to use if we make `run' convert the result to
a number:

  (: run : (String -> Number))
  ;; evaluate a FLANG program contained in a string
  (define (run str)
    (let ([result (eval (parse str))])
      (cases result
        [(Num n) n]
        [else (error 'run
                     "evaluation returned a non-number: ~s" result)])))

We only need this simple utility to make things work:

  (: arith-op : ((Number Number -> Number) FLANG FLANG -> FLANG))
  ;; gets a Scheme numeric binary operator, and uses it within an FLANG
  ;; `Num' wrapper (note H.O type)
  (define (arith-op op expr1 expr2)
    (define: (Num->number [e : FLANG]) : Number
      (cases e
        [(Num n) n]
        [else (error 'arith-op "expects a number, got: ~s" e)]))
    (Num (op (Num->number expr1) (Num->number expr2))))

A few simple tests:

  (test (run "{call {fun {x} {+ x 1}} 4}")
        => 5)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {call add3 1}}")
        => 4)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {with {add1 {fun {x} {+ x 1}}}
                  {with {x 3}
                    {call add1 {call add3 x}}}}}")
        => 7)

There is still a problem with this version.  First a question -- if
`call' is similar to arithmetic operations (and to `with' in what it
actually does), then how come the code is different enough that it
doesn't even need an auxiliary function?

Second question: what *should* happen if we evaluate this:

  (run "{with {identity {fun {x} x}}
          {with {foo {fun {x} {+ x 1}}}
            {call {call identity foo} 123}}}")
  (run "{call {call {fun {x} {call x 1}}
                    {fun {x} {fun {y} {+ x y}}}}
              123}")

Third question, what *will* happen if we do the above?

The following simple fix takes care of this:

  ;; eval : FLANG -> FLANG
  ;; evaluates FLANG expressions by reducing them to *expressions*
  (define (eval expr)
    (cases expr
      [(Num n) expr]
      [(Add l r) (arith-op + (eval l) (eval r))]
      [(Sub l r) (arith-op - (eval l) (eval r))]
      [(Mul l r) (arith-op * (eval l) (eval r))]
      [(Div l r) (arith-op / (eval l) (eval r))]
      [(With bound-id named-expr bound-body)
       (eval (subst bound-body
                    bound-id
                    (eval named-expr)))]
      [(Id name) (error 'eval "free identifier: ~s" name)]
      [(Fun bound-id bound-body) expr]
      [(Call fun-expr arg-expr)
       (let ([fval (eval fun-expr)])        ; <- need to evaluate this!
         (cases fval
           [(Fun bound-id bound-body)
            (eval (subst bound-body
                         bound-id
                         (eval arg-expr)))]
           [else (error 'eval "`call' expects a function, got: ~s"
                              fval)]))]))

The complete code is:

  ----------------------------------------------------------------------
  (define-type FLANG
    [Num  (n Number)]
    [Add  (lhs FLANG) (rhs FLANG)]
    [Sub  (lhs FLANG) (rhs FLANG)]
    [Mul  (lhs FLANG) (rhs FLANG)]
    [Div  (lhs FLANG) (rhs FLANG)]
    [Id   (name Symbol)]
    [With (name Symbol) (named FLANG) (body FLANG)]
    [Fun  (name Symbol) (body FLANG)]
    [Call (fun-expr FLANG) (arg-expr FLANG)])

  (: parse-sexpr : (Sexpr -> FLANG))
  ;; to convert s-expressions into FLANGs
  (define (parse-sexpr sexpr)
    (match sexpr
      [(number: n)    (Num n)]
      [(symbol: name) (Id name)]
      [(cons 'with more)
       (match sexpr
         [(list 'with (list (symbol: name) named) body)
          (With name (parse-sexpr named) (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `with' syntax in ~s" sexpr)])]
      [(cons 'fun more)
       (match sexpr
         [(list 'fun (list (symbol: name)) body)
          (Fun name (parse-sexpr body))]
         [else (error 'parse-sexpr "bad `fun' syntax in ~s" sexpr)])]
      [(list op left right)
       (let ([make-node
              (match op
                ['+ Add]
                ['- Sub]
                ['* Mul]
                ['/ Div]
                ['call Call]
                [else (error 'parse-sexpr "don't know about ~s" op)])])
         (make-node (parse-sexpr left)
                    (parse-sexpr right)))]
      [else (error 'parse-sexpr "bad syntax in ~s" sexpr)]))

  (: parse : (String -> FLANG))
  ;; parses a string containing an FLANG expression to a FLANG AST
  (define (parse str)
    (parse-sexpr (string->sexpr str)))

  (: subst : (FLANG Symbol FLANG -> FLANG))
  ;; substitutes the second argument with the third argument in the
  ;; first argument, as per the rules of substitution; the resulting
  ;; expression contains no free instances of the second argument
  (define (subst expr from to)
    (cases expr
      [(Num n) expr]
      [(Add l r) (Add (subst l from to) (subst r from to))]
      [(Sub l r) (Sub (subst l from to) (subst r from to))]
      [(Mul l r) (Mul (subst l from to) (subst r from to))]
      [(Div l r) (Div (subst l from to) (subst r from to))]
      [(Id name) (if (eq? name from) to expr)]
      [(With bound-id named-expr bound-body)
       (With bound-id
             (subst named-expr from to)
             (if (eq? bound-id from)
               bound-body
               (subst bound-body from to)))]
      [(Call l r) (Call (subst l from to) (subst r from to))]
      [(Fun bound-id bound-body)
       (if (eq? bound-id from)
         expr
         (Fun bound-id (subst bound-body from to)))]))

  (: arith-op : ((Number Number -> Number) FLANG FLANG -> FLANG))
  ;; gets a Scheme numeric binary operator, and uses it within an FLANG
  ;; `Num' wrapper
  (define (arith-op op expr1 expr2)
    (define: (Num->number [e : FLANG]) : Number
      (cases e
        [(Num n) n]
        [else (error 'arith-op "expects a number, got: ~s" e)]))
    (Num (op (Num->number expr1) (Num->number expr2))))

  (: eval : (FLANG -> FLANG))
  ;; evaluates FLANG expressions by reducing them to *expressions*
  (define (eval expr)
    (cases expr
      [(Num n) expr]
      [(Add l r) (arith-op + (eval l) (eval r))]
      [(Sub l r) (arith-op - (eval l) (eval r))]
      [(Mul l r) (arith-op * (eval l) (eval r))]
      [(Div l r) (arith-op / (eval l) (eval r))]
      [(With bound-id named-expr bound-body)
       (eval (subst bound-body
                    bound-id
                    (eval named-expr)))]
      [(Id name) (error 'eval "free identifier: ~s" name)]
      [(Fun bound-id bound-body) expr]
      [(Call fun-expr arg-expr)
       (let ([fval (eval fun-expr)])
         (cases fval
           [(Fun bound-id bound-body)
            (eval (subst bound-body
                         bound-id
                         (eval arg-expr)))]
           [else (error 'eval "`call' expects a function, got: ~s"
                              fval)]))]))

  (: run : (String -> Number))
  ;; evaluate a FLANG program contained in a string
  (define (run str)
    (let ([result (eval (parse str))])
      (cases result
        [(Num n) n]
        [else (error 'run
                     "evaluation returned a non-number: ~s" result)])))

  ;; tests
  (test (run "{call {fun {x} {+ x 1}} 4}")
        => 5)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {call add3 1}}")
        => 4)
  (test (run "{with {add3 {fun {x} {+ x 3}}}
                {with {add1 {fun {x} {+ x 1}}}
                  {with {x 3}
                    {call add1 {call add3 x}}}}}")
        => 7)
  (test (run "{with {identity {fun {x} x}}
                {with {foo {fun {x} {+ x 1}}}
                  {call {call identity foo} 123}}}")
        => 124)
  (test (run "{call {call {fun {x} {call x 1}}
                          {fun {x} {fun {y} {+ x y}}}}
                    123}")
        => 124)
  ----------------------------------------------------------------------

========================================================================