COS 441 - Semantics - Feb 22, 1996

A Meta-circular Interpreter

To begin our study of semantics, we will build an interpreter for Micro-Scheme, which is a subset of Scheme. The implementation will be written in Scheme. The first definition of eval follows. The purpose of this procedure is to evaluate a Micro-Scheme expression. 
(define eval
  (lambda (e env)
    (variant-case e
      (Const (value) value)
      (Var (name) (or (lookup env name)
                      (lambda (x) (cond ((eq? name 'not) (not x)) ...))))
      (Lam (formal body) 
           (lambda (arg) (eval body (extend env formal arg))))
      (Ap (fun arg)
          (let ((vfun (eval fun env))
                (varg (eval arg env)))
            (vfun varg))))))
Here e is the expression to be evaluated in environment env. We need the environment parameter to record bindings for procedure arguments. (In Micro-Scheme functions can only take one parameter.)

This is a meta-circular interpreter. This term refers to using the various constructs of Scheme to implement substantially the same constructs in Micro-Scheme. For instance, we use lambda to represent Micro-Scheme procedures as procedures, and application to apply Micro-Scheme procedures. Provided you understand Scheme, this interpreter provides a precise explanation for how (Micro-) Scheme works.

A First-Order Interpreter

Suppose, though, that you don't really understand Scheme, or that you want to build an interpreter for Scheme in a language like C that doesn't have higher-order procedures. We can use a first-order transformation to derive a new interpreter that represents procedures as records: 
(define-record Prim (name))
(define-record Closure (formal body env))

(define eval
  (lambda (e env)
    (variant-case e
      (Const (value) value)
      (Var (name) (or (lookup env name)
                      (make-Prim name)))
      (Lam (formal body)
           (make-Closure formal body env))
      (Ap (fun arg)
          (let ((vfun (eval fun env))
                (varg (eval arg env)))
            (variant-case vfun
              (Prim (name)
                (cond ((eq? name 'not) (not varg)) ...))
              (Closure (formal body env)
                (eval body (extend env formal varg)))))))))

The Semantics of Letrec

Let's return to the meta-circular interpreter and try to add letrec, to be sure we understand it. The additional clause we need is: 
      (Letrec (name exp body)
        (letrec ((env2 (extend env name (eval exp env2))))
          (eval body env2)))
We must run the body in an environment env2 that includes a binding of name to the value of exp. To get the value of exp, we must evaluate it in this same extended environment env2.

This would be a great explanation if it worked, but it doesn't. The problem is that we have to run the application (eval exp env2) before we know the value of env2.

Recall that we are representing Micro-Scheme procedures as Scheme procedures. For now, let's restrict Micro-Scheme to require that the expression bound by a letrec be a lambda. Ie., the syntax for Micro-Scheme's letrec expression is: 

  (letrec ((x (lambda ...)) body))
Then (eval exp env2) must return a procedure (of one argument), since exp is a lambda-expression. Therefore, we can wrap (eval exp env2) with a lambda without changing the value it will compute: 
      (Letrec (name exp body)
        (letrec ((env2 (extend env name (lambda (x) ((eval exp env2) x)))))
          (eval body env2)))
Now everything works, because we can compute (extend env name (lambda (x) ((eval exp env2) x))) without knowing the value of env2.

... and back to First-Order

So now let's do a first-order transform again. We've added another lambda that represents a Micro-Scheme procedure, so we need a new kind of record. 
      (define-record Rec (exp env))
      ...
      (Letrec (name exp body)
        (letrec ((env2 (extend env name (make-Rec exp env2))))

          (eval body env2)))
      (Ap (fun arg)
          (let ((vfun (eval fun env))
                (varg (eval arg env)))
            (variant-case vfun
              (Prim (name)
                (cond ((eq? name 'not) (not varg)) ...))
              (Closure (formal body env)
                (eval body (extend env formal varg)))
              (Rec (exp env)
                (eval exp env))))))))
Right?? Wrong! Once again, we need to know the value of env2 to evaluate (make-Rec exp env2). You have to be careful with the first-order transform when recursion is involved.

A Correct First-Order Interpreter with Letrec

Let's try again. We'll use a first-order implementation of finite functions, and proceed by merging the extension of env and the evaluation of exp into a single operation rec-extend. 
(define-record rec-extend-ff (f dom exp))
(define rec-extend make-rec-extend-ff)
...
      (Letrec (name exp body)
	(let ((env2 (rec-extend env name exp)))
	  (eval body env2)))
We must also change lookup. 
(define lookup
  (lambda (env x)
    (variant-case env
       (empty-ff () #f)
       (extend-ff (fun dom ran) (if (eq? x dom) ran (lookup fun x)))
       (extend-record-ff (fun dom exp)
	  (if (eq? x dom) (eval exp env) (lookup fun x))))))
All is now correct.

... using Boxes

The previous interpreter delays the evaluation of exp until it is used and repeats the evaluation each time.

What if we were to do the following?

This sure sounds like achieving our goal through assignment. We'll use the following procedures for assignment: (You can build these by representing boxes as pairs, say, and ignoring one of the fields of the pair.) Now env must map names to boxes. Let's first transform our first-order interpreter without letrec to one using boxes. 
(define eval
  (lambda (e env)
    (variant-case e
      (Const (value) value)
      (Var (name) 
           (let ((b (lookup env name)))
             (if b (unbox b) (make-Prim name))))
      (Lam (formal body) (make-Closure formal body env))
      (Ap (fun arg)
          (let ((vfun (eval fun env))
                (varg (eval arg env)))
            (variant-case vfun
	      (Prim (name)
                (cond ((eq? 'not name)) (not varg) ...))
	      (Closure (formal body env)
                (eval body (extend env formal (make-box varg))))))))))
This new interpreter behaves the same as the old one. Now let's add letrec, using our placeholder idea: 
       (Letrec (name exp body)
	 (let* ((b (make-box #f))
		(env2 (extend env name b))
		(v (eval exp env2)))
	   (set-box! b v)
	   (eval body env2)))
Hey, it works!

By the way, this new interpreter does not have the limitation that the right-hand side of a letrec must be a lambda expression. We can use any expression as the right-hand side of a letrec so long as it doesn't use the variable before we've computed its value. Unfortunately, this condition is impossible to check. Hence the set of legal Scheme programs is undecidable.

Assignment

Since we've used assignment in Scheme to explain Micro-Scheme's letrec form, perhaps we had better add assignment to Micro-Scheme. We add the following expression form: 
       (set! x e)
Here x must be a variable. Let's try to support set! in our interpreter. 
       (Set! (name body)
	 (begin
           (set-box! (lookup env name) (eval body env))
           #f))
Here we've chosen to return #f as the value of a set! expression. In Scheme the result of set! is unspecified. Its a bad idea to use the result of set!.

We could add the box-related functions to our interpreter, but let's instead build them from set!. 

(define-record Boxrep (unbox set-box!))

(define make-Box
  (lambda (v)
    (make-Boxrep
      (lambda () v)
      (lambda (new set! v new)))))

(define unbox
  (lambda (b)
    ((Boxrep->unbox b))))

Exercise